Jakub Wilk | 2 Sep 08:16 2015
Picon

[Cython] array->memoryview typecast causes compiler crash in AnalyseExpressionsTransform

In https://bugs.debian.org/793129 Dan Greene reported that trying to 
cast a pointer to a memoryview object causes a compiler crash:

$ python -c 'import cython; cython.inline("<int[:12]>NULL")'
Compiling /home/jwilk/.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx
because it changed.
[1/1] Cythonizing /home/jwilk/.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx

Error compiling Cython file:
------------------------------------------------------------
...

def __invoke():
    <int[:12]>NULL
   ^
------------------------------------------------------------

.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4: Compiler crash in AnalyseExpressionsTransform

ModuleNode.body = StatListNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:4:0)
StatListNode.stats[0] = DefNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:4:0,
    modifiers = [...]/0,
    name = u'__invoke',
    py_wrapper_required = True,
    reqd_kw_flags_cname = '0',
    used = True)
File 'Nodes.py', line 430, in analyse_expressions: StatListNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4,
    is_terminator = True)
File 'Nodes.py', line 4741, in analyse_expressions: ExprStatNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4)
File 'ExprNodes.py', line 460, in analyse_expressions: CythonArrayNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4,
(Continue reading)

Carlos Pita | 2 Sep 01:40 2015
Picon

[Cython] [Bug] Cannot cython.cast to 'object'

Something like:

cython.cast('object', x)

won't work because base_type will be None in ExprNodes.py:9323.

Here's is a simple fix, albeit I'm not sure the best one:

base_type = self.base_type.analyse(env) if self.base_type else py_object_type

Cheers
--
Carlos
Jakub Wilk | 30 Aug 19:44 2015
Picon

[Cython] DEF converts byte strings to unicode

DEF silently converts byte strings to Unicode strings:

$ cat testcase.pyx
DEF ABYTE = b'a'
print type(ABYTE)

$ python --version
Python 2.7.10

$ python -c 'import pyximport as p; p.install(); import testcase'
<type 'unicode'>

Tested with Cython 0.23.1.

--

-- 
Jakub Wilk
Jakub Wilk | 30 Aug 21:29 2015
Picon

[Cython] Compiler crash in MarkOverflowingArithmetic

Tested with Cython 0.23.1 on i386. You will probably need a bigger 
number to trigger it on a 64-bit machine.

$ cython test.pyx

Error compiling Cython file:
------------------------------------------------------------
...
DEF X = 0x811C9DC5
cdef void f():
    cdef unsigned int x = X
        ^
------------------------------------------------------------

test.pyx:3:9: Compiler crash in MarkOverflowingArithmetic

ModuleNode.body = StatListNode(test.pyx:2:5)
StatListNode.stats[0] = CFuncDefNode(test.pyx:2:5,
    args = [...]/0,
    modifiers = [...]/0,
    visibility = 'private')
CFuncDefNode.body = StatListNode(test.pyx:3:9)
StatListNode.stats[0] = SingleAssignmentNode(test.pyx:3:9,
    first = True)

Compiler crash traceback from this point on:
  File "/usr/local/lib/python2.7/dist-packages/Cython/Compiler/Visitor.py", line 183, in _visit
    return handler_method(obj)
  File "/usr/local/lib/python2.7/dist-packages/Cython/Compiler/TypeInference.py", line 327, in visit_SingleAssignmentNode
    self.visit_assignment(node.lhs, node.rhs)
(Continue reading)

Dominic Sacré | 19 Aug 00:02 2015
Picon
Picon

[Cython] Cython fails to build if pgen is available

Hi,

I'm getting the following error trying to build Cython 0.23:

Cython/Parser/Grammar: No such file or directory
Traceback (most recent call last):
  File "setup.py", line 289, in <module>
    compile_cython_modules(cython_profile, cython_compile_more,
cython_with_refnanny)
  File "setup.py", line 135, in compile_cython_modules
    os.path.join(parser_dir, 'graminit.c'),
  File
"/data/master/build/tmp/sysroots/x86_64-linux/usr/lib/python2.7/subprocess.py",
line 540, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command
'['/data/master/build/tmp/sysroots/x86_64-linux/usr/bin/python-native/pgen',
'Cython/Parser/Grammar', 'Cython/Parser/graminit.h',
'Cython/Parser/graminit.c']' returned non-zero exit status 1

The toolchain is based on OpenEmbedded, and unlike most distribution
packages, its Python installation actually contains pgen.

Where would that Cython/Parser/Grammar file come from?
Alternatively, is there a clean way (without patching setup.py) to skip
compiling the grammar?

Cheers,

Dominic
(Continue reading)

Thomas Holenstein | 17 Aug 11:29 2015
Picon

[Cython] gcc error when compiling cython

Hey all,

recently, I had a gcc compile time error with cython. The work around
is obvious, as the problem happens when a cdef-ed numpy array in a
function is unused. Nevertheless, I feel it's useful to report the
problem I had.

The following is a test case
---------------------------------------- test.pyx
import numpy as np
cimport numpy as np

def f(np.ndarray[dtype=np.int_t, ndim=2] arg1):
    cdef np.ndarray[dtype=np.int_t, ndim=2] arr
    return 3
----------------------------------------- setup.py
from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize("test.pyx"),
)
----------------------------------------------------------
This gives me the error

error: ‘__pyx_pybuffernd_arr’ undeclared

Full shell output:
 ~/tmp/ctest > python setup.py build_ext --inplace
running build_ext
(Continue reading)

Jakub Wilk | 14 Aug 10:41 2015
Picon

[Cython] Cython 0.23: long + int == int

$ cython --version
Cython version 0.23
$ python2.6 --version
Python 2.6.8
$ cat testlong.pyx
def f(x):
        return 0L + x
$ python2.6 -c 'import pyximport as p; p.install(); import testlong; print type(testlong.f(42))'
<type 'int'>

I added a long to an int, so I'd expect the result to be a long.
This works correctly in Python 2.7:

$ python2.7 -c 'import pyximport as p; p.install(); import testlong; print type(testlong.f(42))'
<type 'long'>

It also worked correctly in Cython 0.22.1 + Python 2.6.

--

-- 
Jakub Wilk
Phil Thompson | 12 Aug 19:34 2015

[Cython] Cython Modules as builtins

Is there any reason why cython generated modules don’t use the fully qualified name (ie.
__Pyx_MODULE_NAME) in the call to Py_InitModule4() and the PyModuleDef structure? I want to statically
link a module and add it to Python’s inittab but things don’t work without a fully qualified name.

Thanks,
Phil
_______________________________________________
cython-devel mailing list
cython-devel <at> python.org
https://mail.python.org/mailman/listinfo/cython-devel
Ian Henriksen | 20 Jul 23:06 2015
Picon

[Cython] Non-type template parameters

Hi all,
I've spent a little time working on adding support for non-type
template parameters. In doing this, it has been very easy to add
support for doing something like the following:

cdef extern from "add_const.hpp" nogil:
    int myfunc1[i](int)
def test():
    print myfunc1[2](a)

The downside of this is that it does not let the Cython compiler
distinguish between different kinds of template parameters.

Stricter checking could be added using a syntax like this:

cdef extern from "add_const.hpp" nogil:
    int myfunc1[int i](int)
def test():
    print myfunc1[2](a)

The downsides here are that the syntax doesn't really match the
existing template syntax. It will also complicate the Cython codebase
since we'll have to go to greater lengths to allow or disallow all the
different special cases for templates.

Which version would be preferable?

On a similar note, for variadic templates, would we prefer something
like

cdef extern from "my_variadic.hpp" nogil:
    T myfunc2[T,...](T, ...)

or something like:

cdef extern from "my_variadic.hpp" nogil:
    T myfunc2[T, Types...](T, Types... args)

Again, the latter syntax is more explicit, but it will require much more
complicated code in Cython. It also doesn't match the existing syntax
very well. The former syntax matches the existing syntax for templates
better, but will make it hard for Cython to raise errors early on in
compilation.

I'd greatly appreciate any input on the best syntax for either use-case.

Regards,

-Ian Henriksen
<div><div dir="ltr">Hi all,<div>I've spent a little time working on adding support for non-type</div>
<div>template parameters. In doing this, it has been very easy to add</div>
<div>support for doing something like the following:</div>
<div><br></div>
<div>
<div>cdef extern from "add_const.hpp" nogil:</div>
<div>&nbsp; &nbsp; int myfunc1[i](int)</div>
</div>
<div>def test():</div>
<div>&nbsp; &nbsp;&nbsp;print myfunc1[2](a)</div>
<div><br></div>
<div>The downside of this is that it does not let the Cython compiler</div>
<div>distinguish between different kinds of template parameters.</div>
<div><br></div>
<div>Stricter checking could be added using a syntax like this:</div>
<div><br></div>
<div>
<div>
<div>cdef extern from "add_const.hpp" nogil:</div>
<div>&nbsp; &nbsp; int myfunc1[int i](int)</div>
</div>
<div>def test():</div>
<div>&nbsp; &nbsp;&nbsp;print myfunc1[2](a)</div>
</div>
<div><br></div>
<div>The downsides here are that the syntax doesn't really match the</div>
<div>existing template syntax. It will also complicate the Cython codebase</div>
<div>since we'll have to go to greater lengths to allow or disallow all the</div>
<div>different special cases for templates.</div>
<div><br></div>
<div>Which version would be preferable?</div>
<div><br></div>
<div>On a similar note, for variadic templates, would we prefer something</div>
<div>like</div>
<div><br></div>
<div>cdef extern from "my_variadic.hpp" nogil:</div>
<div>&nbsp; &nbsp; T myfunc2[T,...](T, ...)</div>
<div><br></div>
<div>or something like:</div>
<div><br></div>
<div>cdef extern from "my_variadic.hpp" nogil:</div>
<div>&nbsp; &nbsp; T myfunc2[T, Types...](T, Types... args)</div>
<div><br></div>
<div>Again, the latter syntax is more explicit, but it will require much more</div>
<div>complicated code in Cython. It also doesn't match the existing syntax</div>
<div>very well. The former syntax matches the existing syntax for templates</div>
<div>better, but will make it hard for Cython to raise errors early on in</div>
<div>compilation.</div>
<div><br></div>
<div>I'd greatly appreciate any input on the best syntax for either use-case.</div>
<div><br></div>
<div>Regards,</div>
<div><br></div>
<div>-Ian Henriksen</div>
</div></div>
Brian McAndrews | 16 Jul 23:13 2015

[Cython] Problem with declaring a map with an enum type.

I have a C enum type that I’m attempting to put in a C++ map:

Here’s what I got:

 

colors.h:

enum Color

  {

    GREEN,BLUE,RED

  };

 

val.pyx:

from libcpp.string cimport string

from libcpp.map cimport map

 

cdef extern from "colors.h":

    cpdef enum _Color "Color":

       GREEN,BLUE,RED

 

cdef class TestMe:

    cdef public map[string, _Color] vals

 

 

I get a compile error (VSC++ 2012)  

VAL.cpp(1239) : error C2440: '=' : cannot convert from 'long' to 'Color'

        Conversion to enumeration type requires an explicit cast (static_cast, C-style cast or function-style cast)

 

The offending statement is this:

    __pyx_t_9 = PyInt_AsLong(__pyx_v_value); if (unlikely(PyErr_Occurred())) {__pyx_filename = __pyx_f[1]; __pyx_lineno = 202; __pyx_clineno = __LINE__; goto __pyx_L1_error;}

 

In the __pyx_convert_map_from_py_std_3a__3a_string__and_enum__Color function.

 

The cast from PyInt_AsLong(…) to an enum Color is not specified when using maps.

 

If I don’t put it in a map (just as a member variable), the cast is performed.

 

Has anyone seen this before?

 

Thanks,

Brian







This electronic mail message and any attached files contain information intended for the exclusive use of the individual or entity to whom it is addressed and may contain information that is proprietary, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or distribution of this information may be subject to legal restriction or sanction. Please notify the sender, by electronic mail or telephone, of any unintended recipients and delete the original message without making any copies.
<div>
<div class="WordSection1">
<p class="MsoNormal">I have a C enum type that I&rsquo;m attempting to put in a C++ map:<p></p></p>
<p class="MsoNormal">Here&rsquo;s what I got:<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">colors.h:<p></p></p>
<p class="MsoNormal">enum Color<p></p></p>
<p class="MsoNormal">&nbsp; {<p></p></p>
<p class="MsoNormal">&nbsp;&nbsp;&nbsp; GREEN,BLUE,RED<p></p></p>
<p class="MsoNormal">&nbsp; };<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">val.pyx:<p></p></p>
<p class="MsoNormal">from libcpp.string cimport string<p></p></p>
<p class="MsoNormal">from libcpp.map cimport map<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">cdef extern from "colors.h":<p></p></p>
<p class="MsoNormal">&nbsp;&nbsp;&nbsp; cpdef enum _Color "Color":<p></p></p>
<p class="MsoNormal">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; GREEN,BLUE,RED<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">cdef class TestMe:<p></p></p>
<p class="MsoNormal">&nbsp;&nbsp;&nbsp; cdef public map[string, _Color] vals<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">I get a compile error (VSC++ 2012) &nbsp;<p></p></p>
<p class="MsoNormal">VAL.cpp(1239) : error C2440: '=' : cannot convert from 'long' to 'Color'<p></p></p>
<p class="MsoNormal">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Conversion to enumeration type requires an explicit cast (static_cast, C-style cast or function-style cast)<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">The offending statement is this:<p></p></p>
<p class="MsoNormal">&nbsp;&nbsp;&nbsp; __pyx_t_9 = PyInt_AsLong(__pyx_v_value); if (unlikely(PyErr_Occurred())) {__pyx_filename = __pyx_f[1]; __pyx_lineno = 202; __pyx_clineno = __LINE__; goto __pyx_L1_error;}<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">In the __pyx_convert_map_from_py_std_3a__3a_string__and_enum__Color function.<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">The cast from PyInt_AsLong(&hellip;) to an enum Color is not specified when using maps.<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">If I don&rsquo;t put it in a map (just as a member variable), the cast is performed.<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">Has anyone seen this before?<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">Thanks,<p></p></p>
<p class="MsoNormal">Brian<p></p></p>
</div>
<br><br><br><br><br><br><h5>This electronic mail message and any attached files contain information intended for the exclusive use of the individual or entity to whom it is addressed and may contain information that is proprietary, confidential and/or exempt from
 disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any viewing, copying, disclosure or distribution of this information may be subject to legal restriction or sanction. Please notify the sender, by electronic
 mail or telephone, of any unintended recipients and delete the original message without making any copies.
</h5>

</div>
Alex S. | 15 Jul 13:51 2015
Picon

[Cython] Fwd: [BUG] Cython failing performance when Numpy is not installed


Hello,
being unable to acquire the Trac account (Robert Bradshaw's email is not listed on his site), I write here.
The issue was described here: http://trac.cython.org/ticket/847 but was, however, misidentified. The wrapper for a function which receives "fused slice" (D[:]) arguments automatically includes the following code:
>                 cdef type ndarray
>                 try:
>                     import numpy
>                     ndarray = numpy.ndarray
>                 except (ImportError, AttributeError, TypeError):
>                     ndarray = None
(Compiler/FusedNode.py:468).
When numpy is not installed, each import tries to search for it in the system path. (When it is, it's just cached). This severely degrades the performance.


<div><div dir="ltr">
<br><div class="gmail_quote">
<div dir="ltr">
<div>
<div>Hello,<br>being unable to acquire the Trac account (Robert Bradshaw's email is not listed on his site), I write here.<br>The issue was described here: <a href="http://trac.cython.org/ticket/847" target="_blank">http://trac.cython.org/ticket/847</a> but was, however, misidentified. The wrapper for a function which receives "fused slice" (D[:]) arguments automatically includes the following code:<br>&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; cdef type ndarray<br>&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; try:<br>&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; import numpy<br>&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ndarray = numpy.ndarray<br>&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; except (ImportError, AttributeError, TypeError):<br>&gt; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; ndarray = None<br>
</div>(Compiler/FusedNode.py:468).<br>
</div>When numpy is not installed, each import tries to search for it in the system path. (When it is, it's just cached). This severely degrades the performance.<br>
</div>
<br>
</div>
<br>
</div></div>

Gmane