Yury Selivanov | 14 May 23:31 2016
Picon

[Cython] segfault in 'async def' coroutines

Hi,

Under some circumstances, in asyncio code that runs in uvloop [1],
cython code segfaults in cython/Cython/Utility/Coroutine.c:

   static PyObject *
   __Pyx_Coroutine_get_qualname(__pyx_CoroutineObject *self)
   {
     Py_INCREF(self->gi_qualname);
     return self->gi_qualname;
   }

"self->gi_qualname" can be NULL.  The correct code is probably:

   __Pyx_Coroutine_get_qualname(__pyx_CoroutineObject *self)
   {
     if (self->gi_qualname == NULL) { return __pyx_empty_unicode; }
     Py_INCREF(self->gi_qualname);
     return self->gi_qualname;
   }

Thanks,
Yury
Yury Selivanov | 15 May 00:28 2016
Picon

[Cython] async def coroutines miss __module__

In CPython:

 >>> async def foo(): pass
 >>> foo.__module__
'__main__'

In Cython, async def coroutines lack __module__ attribute

Yury
Jeroen Demeyer | 12 May 10:29 2016
Picon

[Cython] Illegal C code generated with numpy complex type

Current Cython git master breaks SageMath because Cython generates calls 
to non-existing functions like fabs_npy_float64(). Since this is a 
regression since Cython 0.24, I made a blocker ticket
http://trac.cython.org/ticket/879

Jeroen.
Jeroen Demeyer | 11 May 22:28 2016
Picon

[Cython] <at> staticmethod decorator applied twice

When investigating some issue with decorators, I noticed the following 
stange thing: it seems that the  <at> staticmethod decorator is actually 
applied twice in some cases. For example, the testsuite file 
tests/run/static_methods.pyx contains the following code:

cdef class A:
      <at> staticmethod
     def static_def(int x):
         ...

The relevant part of the Cython-generated code:

   __pyx_t_1 = 
PyCFunction_NewEx(&__pyx_mdef_14static_methods_1A_1static_def, NULL, 
__pyx_n_s_static_methods);
...
PyTuple_SET_ITEM(__pyx_t_2, 0, __pyx_t_1);
   __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_staticmethod, 
__pyx_t_2, NULL);
...
   if (PyDict_SetItem((PyObject 
*)__pyx_ptype_14static_methods_A->tp_dict, __pyx_n_s_static_def, 
__pyx_t_1) < 0)
...
   __pyx_t_1 = __Pyx_GetNameInClass((PyObject 
*)__pyx_ptype_14static_methods_A, __pyx_n_s_static_def);
...
PyTuple_SET_ITEM(__pyx_t_2, 0, __pyx_t_1);
   __pyx_t_1 = __Pyx_PyObject_Call(__pyx_builtin_staticmethod, 
__pyx_t_2, NULL);
(Continue reading)

Christoph Groth | 3 May 16:19 2016

[Cython] missing cimport in module '.' - bug?

Hello,

The setup.py script of our package "kwant" uses Cython.Build.cythonize.
Everything works well, except that for pyx files that cimport a module
from "." an error message is generated.  E.g. for the file
"kwant/linalg/lapack.pyx" that contains the line

from . cimport f_lapack

The following message appears whenever setup.py is run:

missing cimport in module '.': kwant/linalg/lapack.pyx

No error message appears for files that cimport only from somewhere else
than ".", e.g.:

from .defs cimport gint

The said messages do not seem to cause any problems, but they are of
course somewhat of a nuiscance.  They seem to be created by
Cython.Build.Dependencies.cimported_files().  The culprit seems to be
the function find_pxd in that module that does not seem to handle
properly the case when module is ".".

To check out the problem, download "kwant" from PyPI and execute, for
example, "./setup.py --help".

Cheers,
Christoph

(Continue reading)

Erik Bray | 26 Apr 16:58 2016
Picon
Gravatar

[Cython] Cygwin: Handling missing C99 long double functions

Hi again,

Sorry if I'm spamming the list too much, but I've encountered another
pretty serious and unfortunate issue with Cython on Cygwin.

The problem now is that many of libm long double functions like sqrtl,
tanl, etc. are missing on Cygwin (missing from newlib to be specific).
I think this is a previously known issue, but nothing's ever really
been done about it.  Well, to be clear, sometimes they're present, but
only when sizeof(double) == sizeof(long double).  However on 64-bit
Cygwin sizeof(long double) == 16, so the functions are simply not
defined.

This seems to be due to lack of interest / effort:
https://www.cygwin.com/ml/cygwin/2011-04/msg00231.html  That post is 5
years old, but I can't find any evidence that this has changed.

There are quite a few tests in Cygwin's test suite that test long
double support.  I guess what I'm asking is what would be the best
thing to do about it.

I could just skip those tests on Cygwin, though I'm not sure the best
way to go about skipping an entire test for a given platform.

More generally though, outside the context of testing, this means
Cygwin will sometimes generate code that cannot be compiled on this
platform, and a question arises as to what to do about that.  I have
some thoughts, but am not sure if it's worth discussing any further or
not.

(Continue reading)

Erik Bray | 25 Apr 17:18 2016
Picon
Gravatar

[Cython] Minor issue with running tests and -Wlong-long

Hello,

As some of you already know I've been doing some work on Cython on
Cygwin [in the process I I'm constantly mixing the two up in speech,
but maybe in writing I'll do better :)].

There are several issues with the tests on Cygwin, and that's one
thing I'll work on.  But a major annoyance I've encountered when
running any tests is a huge number of warnings from gcc such as:

embray <at> PC-pret-47 ~/src/cython
$ CFLAGS="-O0" ./runtests.py -vv --no-cpp addloop
Python 2.7.10 (default, Jun  1 2015, 18:05:38)
[GCC 4.9.2]

Running tests against Cython 0.24 f68b5bd0fa620d0dc26166bffe5fe42d94068720
Backends: c

runTest (__main__.CythonRunTestCase)
compiling (c) and running addloop ...
=== C/C++ compiler error output: ===
In file included from /usr/include/python2.7/Python.h:58:0,
                 from addloop.c:4:
/usr/include/python2.7/pyport.h:69:27: warning: ISO C90 does not
support ‘long long’ [-Wlong-long]
 #define PY_LONG_LONG long long
                           ^
/usr/include/python2.7/pyport.h:793:34: note: in definition of macro
‘PyAPI_FUNC’
 #       define PyAPI_FUNC(RTYPE) RTYPE
(Continue reading)

Nathaniel Smith | 24 Apr 10:15 2016

[Cython] bug report on cython-mode.el: freezes when using which-function-mode

Hi all,

Bug report here -- trying to edit some cython code in emacs just now,
emacs was repeatedly freezing until I'd hit C-g repeatedly. Made
things literally unusable -- I couldn't type characters into the
buffer. M-x toggle-debug-on-quit gives the backtrace:

Debugger entered--Lisp error: (quit)
  syntax-ppss()
  python-nav-beginning-of-statement()
  cython-beginning-of-block()
  cython-current-defun()
  run-hook-with-args-until-success(cython-current-defun)
  which-function()
  which-func-update-1(#<window 3 on _http_parser.pyx>)
  which-func-update()
  apply(which-func-update nil)
  timer-event-handler([t 0 0 500000 t which-func-update nil idle 0])

Which strongly suggests that the problem has something to do with my
having which-function-mode enabled, and likely that something is wrong
with cython-current-defun. (which-function-mode is a minor mode built
into emacs.) Toggling which-function-mode off seems tentatively to
have fixed the problem. So there's a workaround, but really
cython-mode + which-function-mode shouldn't cause freezes :-).

Possible contributing factor: this emacs is built from a git snapshot
of master ("GNU Emacs 25.1.50.1 (x86_64-pc-linux-gnu, GTK+ Version
3.18.9) of 2016-04-22"), so it has the git versions of python-mode and
which-function-mode. (I'm just using the python.el that ships with
(Continue reading)

Isuru Fernando | 22 Apr 11:14 2016
Picon
Gravatar

[Cython] Cython compiler crash in 0.24

Hi,

When cythonizing a .pyx I get an error in Cython 0.24 which was not there in Cython 0.23.3


where a None object is passed and the following line calls the None object.


Let me know if you need more information.

Thanks,

Isuru Fernando



[ 33%] Cythonizing symengine_wrapper.pyx

Error compiling Cython file:
------------------------------------------------------------
...
    cdef double complex[::1] cmplx_view
    if real:
        try:
            real_view = iterable
        except (ValueError, TypeError):
            real_view = cython.view.array(shape=(_size(iterable),),
                                                     ^
------------------------------------------------------------

symengine_wrapper.pyx:2464:54: Compiler crash in TransformBuiltinMethods

ModuleNode.body = StatListNode(symengine_wrapper.pyx:1:0)
StatListNode.stats[163] = StatListNode(symengine_wrapper.pyx:2455:0)
StatListNode.stats[0] = DefNode(symengine_wrapper.pyx:2455:0,
    doc = ' if iterable supports the buffer interface: return iterable,\n        if not, return a cython.view.array object (which does) ',
    modifiers = [...]/0,
    name = 'with_buffer',
    num_required_args = 1,
    py_wrapper_required = True,
    reqd_kw_flags_cname = '0')
DefNode.body = StatListNode(symengine_wrapper.pyx:2456:4)
StatListNode.stats[0] = IfStatNode(symengine_wrapper.pyx:2460:4)
IfStatNode.if_clauses[0] = IfClauseNode(symengine_wrapper.pyx:2460:7)
IfClauseNode.body = StatListNode(symengine_wrapper.pyx:2461:8)
StatListNode.stats[0] = TryExceptStatNode(symengine_wrapper.pyx:2461:8)
TryExceptStatNode.except_clauses[0] = ExceptClauseNode(symengine_wrapper.pyx:2463:8)
ExceptClauseNode.body = StatListNode(symengine_wrapper.pyx:2464:12,
    is_terminator = True)
StatListNode.stats[0] = SingleAssignmentNode(symengine_wrapper.pyx:2464:41)
SingleAssignmentNode.rhs = GeneralCallNode(symengine_wrapper.pyx:2464:41,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 8035, in compile_time_value: DictNode(symengine_wrapper.pyx:2464:47,
    is_dict_literal = True,
    is_temp = 1,
    obj_conversion_errors = [...]/0,
    reject_duplicates = True,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 7334, in compile_time_value: TupleNode(symengine_wrapper.pyx:2464:49,
    is_sequence_constructor = 1,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 6730, in compile_time_value_list: TupleNode(symengine_wrapper.pyx:2464:49,
    is_sequence_constructor = 1,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 4981, in compile_time_value: SimpleCallNode(symengine_wrapper.pyx:2464:54,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 1783, in compile_time_value: NameNode(symengine_wrapper.pyx:2464:54,
    cf_maybe_null = True,
    is_name = True,
    name = '_size',
    result_is_used = True,
    use_managed_ref = True)

Compiler crash traceback from this point on:
  File "/home/isuru/miniconda3/envs/test-cython/lib/python3.5/site-packages/Cython/Compiler/ExprNodes.py", line 1781, in compile_time_value
    return denv.lookup(self.name)
AttributeError: 'NoneType' object has no attribute 'lookup'


<div><div dir="ltr">Hi,<br><br>When cythonizing a .pyx I get an error in Cython 0.24 which was not there in Cython 0.23.3<div><br></div>
<div>After printing the stacktrace, this seems to be because of this line&nbsp;<a href="https://github.com/cython/cython/commit/6d55fd189f6ee9d4374d00b8c9c320bd04332bab#diff-28c66ef9e2ff564619ef82aa9d72ee7dR2762" target="_blank">https://github.com/cython/cython/commit/6d55fd189f6ee9d4374d00b8c9c320bd04332bab#diff-28c66ef9e2ff564619ef82aa9d72ee7dR2762</a>
</div>
<div><br></div>
<div>where a None object is passed and the following line calls the None object.<br>
</div>
<div><br></div>
<div>
<a href="https://github.com/cython/cython/blob/6d55fd189f6ee9d4374d00b8c9c320bd04332bab/Cython/Compiler/ExprNodes.py#L1780" target="_blank">https://github.com/cython/cython/blob/6d55fd189f6ee9d4374d00b8c9c320bd04332bab/Cython/Compiler/ExprNodes.py#L1780</a><br>
</div>
<div><br></div>
<div>Let me know if you need more information.</div>
<div>
<br>Thanks,</div>
<div><br></div>
<div>Isuru Fernando</div>
<div><br></div>
<div><br></div>
<div><br></div>
<div>
<div>[ 33%] Cythonizing symengine_wrapper.pyx</div>
<div><br></div>
<div>Error compiling Cython file:</div>
<div>------------------------------------------------------------</div>
<div>...</div>
<div>&nbsp; &nbsp; cdef double complex[::1] cmplx_view</div>
<div>&nbsp; &nbsp; if real:</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; try:</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; real_view = iterable</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; except (ValueError, TypeError):</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; real_view = cython.view.array(shape=(_size(iterable),),</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;^</div>
<div>------------------------------------------------------------</div>
<div><br></div>
<div>symengine_wrapper.pyx:2464:54: Compiler crash in TransformBuiltinMethods</div>
<div><br></div>
<div>ModuleNode.body = StatListNode(symengine_wrapper.pyx:1:0)</div>
<div>StatListNode.stats[163] = StatListNode(symengine_wrapper.pyx:2455:0)</div>
<div>StatListNode.stats[0] = DefNode(symengine_wrapper.pyx:2455:0,</div>
<div>&nbsp; &nbsp; doc = ' if iterable supports the buffer interface: return iterable,\n &nbsp; &nbsp; &nbsp; &nbsp;if not, return a cython.view.array object (which does) ',</div>
<div>&nbsp; &nbsp; modifiers = [...]/0,</div>
<div>&nbsp; &nbsp; name = 'with_buffer',</div>
<div>&nbsp; &nbsp; num_required_args = 1,</div>
<div>&nbsp; &nbsp; py_wrapper_required = True,</div>
<div>&nbsp; &nbsp; reqd_kw_flags_cname = '0')</div>
<div>DefNode.body = StatListNode(symengine_wrapper.pyx:2456:4)</div>
<div>StatListNode.stats[0] = IfStatNode(symengine_wrapper.pyx:2460:4)</div>
<div>IfStatNode.if_clauses[0] = IfClauseNode(symengine_wrapper.pyx:2460:7)</div>
<div>IfClauseNode.body = StatListNode(symengine_wrapper.pyx:2461:8)</div>
<div>StatListNode.stats[0] = TryExceptStatNode(symengine_wrapper.pyx:2461:8)</div>
<div>TryExceptStatNode.except_clauses[0] = ExceptClauseNode(symengine_wrapper.pyx:2463:8)</div>
<div>ExceptClauseNode.body = StatListNode(symengine_wrapper.pyx:2464:12,</div>
<div>&nbsp; &nbsp; is_terminator = True)</div>
<div>StatListNode.stats[0] = SingleAssignmentNode(symengine_wrapper.pyx:2464:41)</div>
<div>SingleAssignmentNode.rhs = GeneralCallNode(symengine_wrapper.pyx:2464:41,</div>
<div>&nbsp; &nbsp; result_is_used = True,</div>
<div>&nbsp; &nbsp; use_managed_ref = True)</div>
<div>File 'ExprNodes.py', line 8035, in compile_time_value: DictNode(symengine_wrapper.pyx:2464:47,</div>
<div>&nbsp; &nbsp; is_dict_literal = True,</div>
<div>&nbsp; &nbsp; is_temp = 1,</div>
<div>&nbsp; &nbsp; obj_conversion_errors = [...]/0,</div>
<div>&nbsp; &nbsp; reject_duplicates = True,</div>
<div>&nbsp; &nbsp; result_is_used = True,</div>
<div>&nbsp; &nbsp; use_managed_ref = True)</div>
<div>File 'ExprNodes.py', line 7334, in compile_time_value: TupleNode(symengine_wrapper.pyx:2464:49,</div>
<div>&nbsp; &nbsp; is_sequence_constructor = 1,</div>
<div>&nbsp; &nbsp; result_is_used = True,</div>
<div>&nbsp; &nbsp; use_managed_ref = True)</div>
<div>File 'ExprNodes.py', line 6730, in compile_time_value_list: TupleNode(symengine_wrapper.pyx:2464:49,</div>
<div>&nbsp; &nbsp; is_sequence_constructor = 1,</div>
<div>&nbsp; &nbsp; result_is_used = True,</div>
<div>&nbsp; &nbsp; use_managed_ref = True)</div>
<div>File 'ExprNodes.py', line 4981, in compile_time_value: SimpleCallNode(symengine_wrapper.pyx:2464:54,</div>
<div>&nbsp; &nbsp; result_is_used = True,</div>
<div>&nbsp; &nbsp; use_managed_ref = True)</div>
<div>File 'ExprNodes.py', line 1783, in compile_time_value: NameNode(symengine_wrapper.pyx:2464:54,</div>
<div>&nbsp; &nbsp; cf_maybe_null = True,</div>
<div>&nbsp; &nbsp; is_name = True,</div>
<div>&nbsp; &nbsp; name = '_size',</div>
<div>&nbsp; &nbsp; result_is_used = True,</div>
<div>&nbsp; &nbsp; use_managed_ref = True)</div>
<div><br></div>
<div>Compiler crash traceback from this point on:</div>
<div>&nbsp; File "/home/isuru/miniconda3/envs/test-cython/lib/python3.5/site-packages/Cython/Compiler/ExprNodes.py", line 1781, in compile_time_value</div>
<div>&nbsp; &nbsp; return denv.lookup(<a href="http://self.name" target="_blank">self.name</a>)</div>
<div>AttributeError: 'NoneType' object has no attribute 'lookup'<br><br><br>
</div>
</div>
</div></div>
Xuancong Wang | 19 Apr 11:13 2016
Picon

[Cython] unsupported meta-programming-related features

Dear Cython developers,

Python supports meta-programming, in which a variable with name
specified in a string can be created at run-time. One built-in library
which make use of this is argparse.

For example:

parser.add_argument('-N', '--max_threads', help='maximum number of
concurrent decoding threads', type=int, default=16)

in this case, the variable max_threads is created from the string
argument. And then Cython will generate an incorrect C program with
the following error:

smt.py:78:88: undeclared name not builtin: headtail
smt.c:1:2: error: #error Do not use this file, it is the result of a
failed Cython compilation.

In comparison, I found that nuitka can convert this kind of Python
programs sucessfully. I hope Cython can be improved. Thanks!

Cheers,
Xuancong
Stefan Behnel | 14 Apr 18:15 2016
Picon

[Cython] PEP 509: detect dict modification by version tag

Hi!

This new PEP seems interesting for Cython optimisations, too:

https://www.python.org/dev/peps/pep-0509/

Essentially, it adds a 64 bit modification counter to dicts that allows
detecting unmodified dicts, e.g. during lookups of methods or globals.

It's currently proposed for Py3.6.

Stefan

Gmane