Stefan Behnel | 2 Feb 18:13 2014

Re: [Cython] Bug: _PyType_Lookup shortcut in Cython 0.20 breaks lookup of __exit__ in Python 2.6

[forwarding to cython-devel]

Wouter Bolsterlee, 02.02.2014 16:40:
> Cython 0.20 introduced __Pyx_PyObject_LookupSpecial as a shortcut to
> lookup special methods. The commit that introduced this change is
> 8f6412275c4c2d1dcf43ad40306f858b114104ed and can be seen here:
> It seems this change does not work with Python 2.6. The newly introduced
> function calls _PyType_Lookup(), which fails to lookup (at least)
> __exit__ (on a threading.Lock) instance when run under Python 2.6. The
> exact call is here:
> For my Plyvel project ( and
> this issue manifests as follows:
> $ cython --version
>         Cython version 0.20
>         $ python 
>         Python 2.6.8 (unknown, Jan 26 2013, 14:35:25) 
>         [GCC 4.7.2] on linux2
>         Type "help", "copyright", "credits" or "license" for more
> information.
>         >>> import plyvel
>         >>> db = plyvel.DB('/tmp/testdb', create_if_missing=True)
>         >>> db.close()
(Continue reading)

Stefan Behnel | 31 Jan 12:55 2014

[Cython] Sage build problem


There's some kind of C++ enum issue in the Sage build.

Generated code in sage/libs/ppl.cpp:

static PyObject *__pyx_pf_4sage_4libs_3ppl_11MIP_Problem_30solve(
             struct __pyx_obj_4sage_4libs_3ppl_MIP_Problem *__pyx_v_self) {
  enum Parma_Polyhedra_Library::PPL_MIP_Problem_Status __pyx_v_tmp;

gcc error:

sage/libs/ppl.cpp: In function 'PyObject*

sage/libs/ppl.cpp:5514:33: error: 'PPL_MIP_Problem_Status' in namespace
'Parma_Polyhedra_Library' does not name a type
sage/libs/ppl.cpp:5514:67: error: invalid type in declaration before ';' token

The type of "tmp" seems to be inferred. That might be a hint that it could
be a problem in Cython.

The Cython code in question is also a bit funny:

(Continue reading)

R. Andrew Ohana | 28 Jan 23:34 2014

[Cython] Broken list multiplication

It seems like eaca39b00b1451fe6c846a0a670e4c8b39935525 broke some (very simple) instances of list multiplication when there are at least 3 terms in the product. e.g. on my system I'm getting these results:

L = [None] * 2 * 3

L = 2 * [None] * 3

L = [None] * 2 * 3 * 5

<div><div dir="ltr">
<div>It seems like eaca39b00b1451fe6c846a0a670e4c8b39935525 broke some (very simple) instances of list multiplication when there are at least 3 terms in the product. e.g. on my system I'm getting these results:</div>

<div><br></div>L = [None] * 2 * 3<br>
</div>L = 2 * [None] * 3<br>
</div>L = [None] * 2 * 3 * 5<br>
</div>print(len(L))<br>5<br clear="all"><div><div><div><div>

<br>-- <br><div dir="ltr">Andrew</div>
Syam Gadde | 28 Jan 19:41 2014

[Cython] memoryview refcount error

Hi again,

I'm exploring the typed memoryview feature with numpy and encountered a 
refcount error.  It seems to happen when I create a slice of a 
memoryview, and transpose it.  If I don't do both, I don't get the 
refcount error.  It happens in the error cleanup code.  I suspect that 
any error that causes the function to jump to the error cleanup code 
will do the same thing as the Exception below -- but that seemed the 
easiest way to demonstrate the problem. However, if there truly is an 
error in my code, please let me know! Thanks,


import numpy
cimport numpy

class MyException(Exception):
     def __init__(self):

def testfunc():
    a = numpy.arange(12).reshape([3,4])

    cdef numpy.int_t[:,:] a_view = a

    ## here is a slice followed by a transpose
    cdef numpy.int_t[:,:] b = a_view[:,:].T
    ## same thing happens with a more realistic slicing:
    #cdef numpy.int_t[:,:] b = a_view[1:2,:].T
    ## however, there is no error if I do this instead:
    #cdef numpy.int_t[:,:] b = a_view.T
    ## also no error if I do this instead:
    #cdef numpy.int_t[:,:] b = a_view[:,:]

    # The exception below is just to force the function to abort
    # and run the extraneous __PYX_XDEC_MEMVIEW in the error
    # cleanup code:
    #   Fatal Python error: Acquisition count is 0 (line XXXX)
    # Comment out this exception and we don't get the error
    raise MyException()

except MyException:

Julian Taylor | 24 Jan 22:26 2014

[Cython] 0.20 tests fail with python3.4b2


with python3.4b2 numpy_memoryview.acquire_release_cycle fails
looking at the source the test should be disabled, but its still run.

This might be a py3.4 change to doctest, but I know nothing about
doctest, so I wanted to check first if you know something about this.
Interestingly py3.4 runs 64 tests, while python3.3 only runs 32.
the __test__ variable contains the same functions in both python
versions (acquire_release is not in it)

FAIL: acquire_release_cycle (numpy_memoryview)
Doctest: numpy_memoryview.acquire_release_cycle
Traceback (most recent call last):
  File "/usr/lib/python3.4/", line 2187, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for
line unknown line number, in acquire_release_cycle

line ?, in numpy_memoryview.acquire_release_cycle
Failed example:
Got nothing
Stefan Behnel | 23 Jan 21:10 2014

Re: [Cython] segfault due to using DECREF instead of XDECREF

Victor Makarov, 23.01.2014 21:02:
> I'd like to fix this if you don't mind.

Oh, I certainly don't mind. :)


Stefan Behnel | 23 Jan 20:49 2014

[Cython] C code churn and benchmark diffs


to get a better idea of what new optimisations (or C code changes in
general) actually bring, I've added a diffing step to the benchmark runner
jobs. The bigger ones now generate an additional output file
"cfiles.diff.gz" that contains a diff of the C files generated in the
current run against those of the last (successful) run. E.g. here:

(That one should start looking a lot smaller tomorrow :)

While doing that, I noticed that there are a couple of unnecessary
differences in the C files that are due to dict iteration, which is
randomised in Py3.3+ (for security reasons). I fixed (most of?) them, so
repeated Cython runs should now generate pretty much identical output
files, if you strip the file header comment (diff can do that). The rest is
generated deterministically in source code order already.

It should be possible to do the same thing for the Sage build, I guess.
That would also be interesting to look at from time to time, especially
when things start failing for some reason.

Syam Gadde | 21 Jan 23:00 2014

[Cython] segfault due to using DECREF instead of XDECREF

(apologies to the moderator for multiple postings -- I'm trying to figure out which of my email addresses is
actually being exported to the public!)


It seems that cython's static code analysis is failing on the following
code.  It thinks that a particular variable (myobject2) must not be
NULL, but in fact it is, so the __Pyx_DECREF_SET segfaults.  This came
from third-party code, so I could fix it with an explicit initialization
to None, but I'd love to have Cython deal with it directly.  Thanks for
your help!

# When compiled with Cython 0.19.2 or 0.20dev, the following code
# segfaults on import

import random

def myfunc():
      ## uncommenting the following fixes things
      #myobject2 = None

      myfalse = random.random() > 2

      if myfalse:
          myobject = None
          myobject2 = None

      if not myfalse:
          # here Cython uses __Pyx_XDECREF_SET
          myobject = None

      print "Made it past myobject assignment"

      if not myfalse or myobject2 is None:
          # here Cython uses Pyx_DECREF_SET because it has determined
          # that __pyx_v_myobject2 can't be null, but it really, really can!
          # (if no one assigned myobject2 yet)
          myobject2 = None

      print "Made it past myobject2 assignment"


#<from distutils.core import setup
from Cython.Build import cythonize

      ext_modules = cythonize("")

[prompt]$ python build_ext --inplace
[prompt]$ ls tmpnone.*
[prompt]$ python
Python 2.6.6 (r266:84292, Nov 21 2013, 12:39:37)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
  >>> import tmpnone
Made it past myobject assignment
Segmentation fault

Julian Taylor | 22 Jan 00:03 2014

[Cython] bytearray tests fail with default unsigned char

the bytearray tests are broken when chars are unsigned.

tests/run/bytearraymethods.pyx defines following function:
def bytearray_append(b, char c, int i, object o):

this gets converted to an effective __Pyx_PyInt_AsUnsignedChar which
then errors out when -1 is passed in.
chars are unsigned like they are by default on arm, s390x and powerpc.
This causes a couple build failures in debian:

the tests can be fixed by adding signed char to the interface.

to reproduce on x86 with gcc (note the -funsigned-char to change the

cython tests/run/bytearraymethods.pyx
gcc -funsigned-char bytearraymethods.c -fPIC $(python-config --includes)
$(python-config --libs) -shared -O2 -o

python -c "import doctest; import bytearraymethods;

Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'bytearraymethods' is not defined
root <at> ubuntu:/# python -c "import doctest; import bytearraymethods;
File "", line ?, in
bytearraymethods.__test__.bytearray_append (line 202)
Failed example:
    b = bytearray_append(b, -1, ord('y'), ord('z'))  # doctest: +ELLIPSIS
    Traceback (most recent call last):
    ValueError: ...
    Traceback (most recent call last):
      File "/usr/lib/python2.7/", line 1315, in __run
        compileflags, 1) in test.globs
      File "<doctest bytearraymethods.__test__.bytearray_append (line
202)[14]>", line 1, in <module>
        b = bytearray_append(b, -1, ord('y'), ord('z'))  # doctest:
      File "bytearraymethods.pyx", line 202, in
bytearraymethods.bytearray_append (bytearraymethods.c:1339)
        def bytearray_append(bytearray b, char c, int i, object o):
    OverflowError: can't convert negative value to char

Andriy Kornatskyy | 19 Jan 21:00 2014

[Cython] Compiler crash in RemoveUnreachableCode

The cython compiler crash report below. Steps to reproduce:

1. virtualenv env
2. env/bin/easy_install cython
3. env/bin/easy_install wheezy.http

The wheezy.http has dependency on wheezy.core. If I install those two packages separately there is no
error, only if through dependencies.


Andriy Kornatskyy

  File "", line 178, in Cython.Compiler.Visitor.TreeVisitor._visit (/var/folders/g8/2kym1h8n7qbgrwg4qkfqw1gw0000gn/T/easy_install-JVXbGE/Cython-0.20/Cython/Compiler/Visitor.c:4437)
  File "", line 137, in Cython.Compiler.Visitor.TreeVisitor._raise_compiler_error (/var/folders/g8/2kym1h8n7qbgrwg4qkfqw1gw0000gn/T/easy_install-JVXbGE/Cython-0.20/Cython/Compiler/Visitor.c:3655)
Error compiling Cython file:


__version__ = '0.1.129'

src/wheezy/core/ Compiler crash in RemoveUnreachableCode

ModuleNode.body = StatListNode(

Compiler crash traceback from this point on:
  File "", line 170, in Cython.Compiler.Visitor.TreeVisitor._visit (/var/folders/g8/2kym1h8n7qbgrwg4qkfqw1gw0000gn/T/easy_install-JVXbGE/Cython-0.20/Cython/Compiler/Visitor.c:4275)
line 2135, in visit_StatListNode
    if not self.current_directives['remove_unreachable']:
TypeError: 'NoneType' object has no attribute '__getitem__'
Robert Byrnes | 16 Jan 18:43 2014

[Cython] assignment to reference argument bug, revisited

Are there any plans to fix the "Assignment to reference" bug mentioned here?!topic/cython-users/j58Sp3QMrD4

> Oh, yes, of course. Good old C++, where local T& x is unassignable but
> assigning to an argument T& x is a common idiom. This is a bug in Cython.

This is still broken in 0.19.2, and a casual inspection of 0.20rc1
suggests that it is broken there as well.

Is the fix as simple as this?

---       2014-01-16 12:31:08.377573000 -0500
+++        2014-01-16 12:30:56.945501000 -0500
 <at>  <at>  -1605,7 +1605,7  <at>  <at>  class NameNode(AtomicExprNode):

         if self.type.is_const:
             error(self.pos, "Assignment to const '%s'" %
-        if self.type.is_reference:
+        if self.type.is_reference and not self.entry.is_arg:
             error(self.pos, "Assignment to reference '%s'" %
         if not self.is_lvalue():
             error(self.pos, "Assignment to non-lvalue '%s'"

This seems to work for me, but ...

I wonder if it is also correct to set self.entry.cf_unused = True
for the case of assignment to reference arguments, to avoid CYTHON_UNUSED
annotations in the function definitions and unused variable warnings.