Emmanuel Gil Peyrot | 28 May 13:25 2014
Picon

[Cython] Wrong order of __Pyx_DECREF when calling a function with an implicit str → char* conversion.

Hi,

I was testing my cython codebase on top of pypy/cpyext, and I found a
memory corruption.  After investigating it with the pypy guys (thanks
arigato!), we identified it as a cython bug:

 cdef void test(char *string):
     print(string)

 def run(array):
     test(array[0])

This code works fine under cpython, but looking at the generated code
for the last line, the call to test is done with a pointer (__pyx_t_2)
to some potentially deallocated string (__pyx_t_1), which isn’t freed
just yet on cpython but is on pypy:

 __pyx_t_1 = __Pyx_GetItemInt(__pyx_v_array, 0, …);
 __Pyx_GOTREF(__pyx_t_1);
 __pyx_t_2 = __Pyx_PyObject_AsString(__pyx_t_1);
 __Pyx_DECREF(__pyx_t_1); __pyx_t_1 = 0;
 __pyx_f_1a_test(__pyx_t_2);

The obvious solution is to swap the last two lines, it then works fine
on pypy (although not necessarily if the test function stores the
pointer somewhere, but it’s not cython’s fault then).

This issue can also happen with an explicit cast:

 pointer = <char *>array[0]
(Continue reading)

Ulf Worsoe | 30 Apr 12:45 2014

[Cython] Bug in C code generation

Hi,

I have encounterered a bug in the following situation with Cython 0.20.1:

cdef funname():
  cdef numpy.ndarray[numpy.int32_t,ndim=1] myvar
  if True:
    ... block that never initializes myvar
  else
    ... block that initializes myvar

The resulting C code does not declare the variable for the buffer object of myvar (my guess is that cython detects that it is never used and drops it), but the cleanup part of the C function contains code that frees the buffer, and thus refers to a variable that was never declared. 

I have not posted the complete code since the .pyx is around 50k lines long, but the offending method looks like this, and the problematic variable is _3_perm in line 4:

---------------
  cdef _toStringArray__3JJ_3S(self,numpy.ndarray[numpy.int64_t,ndim=1] _0_subi,numpy.int64_t _1_dstidx,numpy.ndarray[object,ndim=1] _2_result):
    assert _0_subi is None or isinstance(_0_subi,numpy.ndarray)
    assert _2_result is None or isinstance(_2_result,numpy.ndarray)
    cdef numpy.ndarray[numpy.int64_t,ndim=1] _3_perm
    cdef numpy.float64_t _4_c 
    cdef numpy.int32_t _5_b 
    cdef numpy.int32_t _6_ni
    cdef numpy.int64_t _7_i  
    cdef numpy.int32_t _8_p 
    cdef mosek_fusion_Utils_StringBuffer _9_sb
    for _10_i in xrange(0,(_0_subi).shape[0]): 
      if ((_0_subi[_10_i] < 0) or (_0_subi[_10_i] >= (self.shape.size))):
        raise mosek_fusion_IndexError_ctor_S("Constraint index out of bounds")
    if (((_0_subi).shape[0] + _1_dstidx) > (_2_result).shape[0]):
      raise mosek_fusion_LengthError_ctor_S("Result array is too small to hold the result")
    _9_sb=mosek.fusion.Utils.StringBuffer()
    _8_p=0 
    if True:
      for _11_k in xrange(0,(_0_subi).shape[0]):
        _7_i=_0_subi[_11_k] 
        _6_ni=self.nativeindexes[_7_i]
        _5_b=_8_p 
        while ((_8_p < ((self.cache.subi)).shape[0]) and ((self.cache.subi)[_8_p] == _7_i)):
          _8_p += 1 
        _9_sb._clear_()._a_S(self.name)._a_S("[")._a_S(self.shape.indexToString(_7_i))._a_S("] : ")
        for _12_j in xrange(_5_b,_8_p):
          _4_c=(self.cache.cof)[_12_j]
          if (_4_c > 0):
            _9_sb._a_S(" + ")._a_D(_4_c)._a_S(" ")
          elif (_4_c < 0):
            _9_sb._a_S(" - ")._a_D((- _4_c))._a_S(" ")
          self.model._nativeVarToStr_ILmosek_4fusion_4Utils_4StringBuffer_2((self.cache.subj)[_12_j],_9_sb)
        if (self.cache_bfix != None):
          if (self.cache_bfix[_7_i] > 0):
            _9_sb._a_S(" + ")._a_D(self.cache_bfix[_7_i])
          elif (self.cache_bfix[_7_i] < 0):
            _9_sb._a_S(" + ")._a_D(self.cache_bfix[_7_i]) 
        _9_sb._a_S(" ") 
        self._domainToString_JLmosek_4fusion_4Utils_4StringBuffer_2(_7_i,_9_sb)
        _2_result[(numpy.int64(_11_k) + _1_dstidx)] = _9_sb._toString_()
    else:      
      _3_perm=numpy.zeros(((_0_subi).shape[0],), dtype=numpy.dtype(numpy.int64))
      mosek_fusion_CommonTools_argQsort__3J_3J_3JJJ(_3_perm,_0_subi,None,0l,(_0_subi).shape[0])
---------------


--
Ulf Worsøe
Mosek ApS

ulf.worsoe-rMKUnKubvFEAvxtiuMwx3w@public.gmane.org
www.mosek.com

<div><div dir="ltr">
<div>Hi,</div>
<div><br></div>
<div>I have encounterered a bug in the following situation with Cython 0.20.1:</div>
<div><br></div>
<div>cdef funname():</div>
<div>&nbsp; cdef numpy.ndarray[numpy.int32_t,ndim=1] myvar</div>

<div>&nbsp; if True:</div>
<div>&nbsp; &nbsp; ... block that never initializes myvar</div>
<div>&nbsp; else</div>
<div>&nbsp; &nbsp; ... block that initializes myvar</div>
<div><br></div>
<div>The resulting C code does not declare the variable for the buffer object of myvar (my guess is that cython detects that it is never used and drops it), but the cleanup part of the C function contains code that frees the buffer, and thus refers to a variable that was never declared.&nbsp;</div>

<div><br></div>
<div>I have not posted the complete code since the .pyx is around 50k lines long, but the offending method looks like this, and the problematic variable is _3_perm in line 4:</div>
<div><br></div>
<div>---------------</div>

<div>
<div>&nbsp; cdef _toStringArray__3JJ_3S(self,numpy.ndarray[numpy.int64_t,ndim=1] _0_subi,numpy.int64_t _1_dstidx,numpy.ndarray[object,ndim=1] _2_result):</div>
<div>&nbsp; &nbsp; assert _0_subi is None or isinstance(_0_subi,numpy.ndarray)</div>

<div>&nbsp; &nbsp; assert _2_result is None or isinstance(_2_result,numpy.ndarray)</div>
<div>&nbsp; &nbsp; cdef numpy.ndarray[numpy.int64_t,ndim=1] _3_perm</div>
<div>&nbsp; &nbsp; cdef numpy.float64_t _4_c&nbsp;</div>
<div>&nbsp; &nbsp; cdef numpy.int32_t _5_b&nbsp;</div>

<div>&nbsp; &nbsp; cdef numpy.int32_t _6_ni</div>
<div>&nbsp; &nbsp; cdef numpy.int64_t _7_i &nbsp;</div>
<div>&nbsp; &nbsp; cdef numpy.int32_t _8_p&nbsp;</div>
<div>&nbsp; &nbsp; cdef mosek_fusion_Utils_StringBuffer _9_sb</div>
<div>&nbsp; &nbsp; for _10_i in xrange(0,(_0_subi).shape[0]):&nbsp;</div>

<div>&nbsp; &nbsp; &nbsp; if ((_0_subi[_10_i] &lt; 0) or (_0_subi[_10_i] &gt;= (self.shape.size))):</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; raise mosek_fusion_IndexError_ctor_S("Constraint index out of bounds")</div>
<div>&nbsp; &nbsp; if (((_0_subi).shape[0] + _1_dstidx) &gt; (_2_result).shape[0]):</div>

<div>&nbsp; &nbsp; &nbsp; raise mosek_fusion_LengthError_ctor_S("Result array is too small to hold the result")</div>
<div>&nbsp; &nbsp; _9_sb=mosek.fusion.Utils.StringBuffer()</div>
<div>&nbsp; &nbsp; _8_p=0&nbsp;</div>
<div>&nbsp; &nbsp; if True:</div>
<div>&nbsp; &nbsp; &nbsp; for _11_k in xrange(0,(_0_subi).shape[0]):</div>

<div>&nbsp; &nbsp; &nbsp; &nbsp; _7_i=_0_subi[_11_k]&nbsp;</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; _6_ni=self.nativeindexes[_7_i]</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; _5_b=_8_p&nbsp;</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; while ((_8_p &lt; ((self.cache.subi)).shape[0]) and ((self.cache.subi)[_8_p] == _7_i)):</div>

<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; _8_p += 1&nbsp;</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; _9_sb._clear_()._a_S(<a href="http://self.name">self.name</a>)._a_S("[")._a_S(self.shape.indexToString(_7_i))._a_S("] : ")</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; for _12_j in xrange(_5_b,_8_p):</div>

<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; _4_c=(self.cache.cof)[_12_j]</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if (_4_c &gt; 0):</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; _9_sb._a_S(" + ")._a_D(_4_c)._a_S(" ")</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; elif (_4_c &lt; 0):</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; _9_sb._a_S(" - ")._a_D((- _4_c))._a_S(" ")</div>

<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; self.model._nativeVarToStr_ILmosek_4fusion_4Utils_4StringBuffer_2((self.cache.subj)[_12_j],_9_sb)</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; if (self.cache_bfix != None):</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if (self.cache_bfix[_7_i] &gt; 0):</div>

<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; _9_sb._a_S(" + ")._a_D(self.cache_bfix[_7_i])</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; elif (self.cache_bfix[_7_i] &lt; 0):</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; _9_sb._a_S(" + ")._a_D(self.cache_bfix[_7_i])&nbsp;</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; _9_sb._a_S(" ")&nbsp;</div>

<div>&nbsp; &nbsp; &nbsp; &nbsp; self._domainToString_JLmosek_4fusion_4Utils_4StringBuffer_2(_7_i,_9_sb)</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; _2_result[(numpy.int64(_11_k) + _1_dstidx)] = _9_sb._toString_()</div>
<div>&nbsp; &nbsp; else: &nbsp; &nbsp; &nbsp;</div>
<div>&nbsp; &nbsp; &nbsp; _3_perm=numpy.zeros(((_0_subi).shape[0],), dtype=numpy.dtype(numpy.int64))</div>

<div>&nbsp; &nbsp; &nbsp; mosek_fusion_CommonTools_argQsort__3J_3J_3JJJ(_3_perm,_0_subi,None,0l,(_0_subi).shape[0])</div>
</div>
<div>---------------</div>
<div><br></div>
<div><br></div>-- <br>Ulf Wors&oslash;e<br>Mosek ApS<br><br><a href="mailto:ulf.worsoe@..." target="_blank">ulf.worsoe@...</a><br><a href="http://www.mosek.com" target="_blank">www.mosek.com</a><br><br>
</div></div>
Philip Herron | 30 Apr 17:54 2014
Picon

[Cython] GCC Pxd

Hey all,

Some of you may remember a project i worked on for GSoC 2012, i wasn't
really happy with the project over all for a long time it basicaly
required at the time people to be running GCC from latest SVN/GIT for
it to work correctly.

Now gcc >= 4.7 is becoming more normal, so i went back into it and
rewrote it from scratch now and its shaping up to look fairly decent
and usable now.

https://github.com/redbrain/cython/tree/master/Tools/gccpxd

So the idea for this project is that you write a small configuration
file and it will integrate with your C/C++ build system for example
you could do:

./configure CC=cygcc ...
make
...

And you will get a project.pxd file after the build of your software.
the cygcc is simply a wrapper over GCC that shoves in the fplugin
arguments so you can run:

cygcc -g -O2 -Wall -c test.c -o test.o

So build systems wont really notice anything different it should just
think its gcc. The configuration file example in the folder .cygcc:

$ cat .cygcc
header = 'test.h'             # the header to get declaratiosn for
inputs = ['test.h', 'test.c'] # any locations of declarations that
would be involved
output = 'test.pxd'          # the output

The cygcc compiler looks for this in the current director or looks up
at the parent directory untill it finds it or it will fail with an
error message.

It will only ouput once if the test.pxd file exists in the current
working directory it wont do anything.

$ cat test.pxd
cdef extern from "test.h":
  int test (int, int)
  int global_variable
  int static_variable # this is a bug but hey...
  struct bla:
    int a
    float b

So far for the example code in there its working pretty well the gcc
python plugin has stabalized alot and so has some of gcc's bits a
bobs. I will try and keep you all updated i think the plugin is doing
alot more work now so this doesnt have to do so much.

Thanks to Robert Bradshaw for baring with me in the whole thing
because i probably wasn't the easiest to work with ;) lol.

Thanks everyone cython is awesome.

--Phil
Lisandro Dalcin | 21 Apr 13:53 2014
Picon

[Cython] MPI_Type_create_hindexed_block() segfaults

I believe the problem is in the following source code line
(file:ompi_datatype_args.c, line:221):

https://bitbucket.org/ompiteam/ompi-svn-mirror/src/v1.8/ompi/datatype/ompi_datatype_args.c?at=v1.8#cl-221

I think you should just remove that bogus line, and that's all.

[dalcinl <at> kw2060 openmpi]$ cat type_hindexed_block.c
#include <mpi.h>
int main(int argc, char *argv[])
{
  MPI_Aint disps[] = {0};
  MPI_Datatype datatype;
  MPI_Init(&argc, &argv);
  MPI_Type_create_hindexed_block(1, 1, disps, MPI_BYTE, &datatype);
  MPI_Finalize();
  return 0;
}
[dalcinl <at> kw2060 openmpi]$ mpicc type_hindexed_block.c
[dalcinl <at> kw2060 openmpi]$ ./a.out
[kw2060:20304] *** Process received signal ***
[kw2060:20304] Signal: Segmentation fault (11)
[kw2060:20304] Signal code: Address not mapped (1)
[kw2060:20304] Failing at address: 0x6a
[kw2060:20304] [ 0] /lib64/libpthread.so.0[0x327c40f750]
[kw2060:20304] [ 1] /lib64/libc.so.6[0x327bc94126]
[kw2060:20304] [ 2]
/home/devel/mpi/openmpi/1.8.0/lib/libmpi.so.1(ompi_datatype_set_args+0x7f1)[0x7f8f0158b62a]
[kw2060:20304] [ 3]
/home/devel/mpi/openmpi/1.8.0/lib/libmpi.so.1(MPI_Type_create_hindexed_block+0x24d)[0x7f8f015cedc8]
[kw2060:20304] [ 4] ./a.out[0x40080c]
[kw2060:20304] [ 5] /lib64/libc.so.6(__libc_start_main+0xf5)[0x327bc21d65]
[kw2060:20304] [ 6] ./a.out[0x4006f9]
[kw2060:20304] *** End of error message ***
Segmentation fault (core dumped)

--

-- 
Lisandro Dalcin
---------------
CIMEC (UNL/CONICET)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1016)
Tel/Fax: +54-342-4511169
Martín Gaitán | 4 Apr 19:55 2014
Picon

[Cython] Move IPython's cython extension to cython

As you probably know, IPython has a "cython magic" [1], an extension to interface ipython with cython, allowing to use cython code in the interactive session.

Originally, few of these "language specific magics" (R, cython, octave, etc) were included builtin in IPython, but then the IPython core developers decided break out them (with fair arguments IMHO [2]), either to be included in the "language package" they depend on or as a third party extensions.

At the moment,  cython-magic is the only one language specific extension left to move.
The rest were moved to the packages they depend on [3]

So, this message is to ask if the Cython project would like to import this code, task that  Brian Grangrer (the author of the cython-magic extension and IPython core dev)  entrusted me [3].

Of course, I could help you preparing a PR that migrate the module, its tests and documentation, preserving the git history as far as possible.

let me know your opinion/decision




--
<div><div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">
<div>As you probably know, IPython has a "cython magic" [1], an extension to interface ipython with cython, allowing to use cython code in the interactive session. <br><br>
</div>
<div>Originally, few of these "language specific magics" (R, cython, octave, etc) were included builtin in IPython, but then the IPython core developers decided break out them (with fair arguments IMHO [2]), <span lang="en"><span>either to be included </span></span>in the "language package" they depend on or as a third party extensions. <br><br>At the moment,&nbsp; cython-magic is the only one language specific extension left to move. <br>The rest were moved to the packages they depend on [3]<br>
</div>
<div><br></div>
<div>So, this message is to ask if the Cython project would like to import this code, task that&nbsp; Brian Grangrer (the author of the cython-magic extension and IPython core dev)&nbsp; entrusted me [3]. <br><br>Of course, I could help you preparing a PR that migrate the module, its tests and documentation, preserving the git history as far as possible. <br><br>
</div>
<div>let me know your opinion/decision <br>
</div>
<div><br></div>
<div><div><div>
<div></div>
<div>
<div>cheers<br>
</div>
<div>Martin. <br>
</div>
<div>
<br><br>[1] <a href="http://ipython.org/ipython-doc/dev/config/extensions/cythonmagic.html" target="_blank">http://ipython.org/ipython-doc/dev/config/extensions/cythonmagic.html</a><br>

[2] <a href="https://github.com/ipython/ipython/issues/3150" target="_blank">https://github.com/ipython/ipython/issues/3150</a> <br>[3] <a href="https://github.com/ipython/ipython/issues/3803" target="_blank">https://github.com/ipython/ipython/issues/3803</a><br>

[4] <a href="https://github.com/ipython/ipython/issues/3150#issuecomment-37203217" target="_blank">https://github.com/ipython/ipython/issues/3150#issuecomment-37203217</a><span class="HOEnZb"><br><br clear="all"><div>
<br>-- <br><div dir="ltr">
<a href="http://mgaitan.github.io" target="_blank">mgaitan.github.io</a><br><a href="http://textosyprextextos.com.ar" target="_blank">textosyprextextos.com.ar</a>
</div>
</div></span>
</div>
</div>
</div></div></div>
</div>
</div>
<br><br clear="all"><br>-- <br><div dir="ltr">
<a href="http://mgaitan.github.io" target="_blank">mgaitan.github.io</a><br><a href="http://textosyprextextos.com.ar" target="_blank">textosyprextextos.com.ar</a>
</div>

</div></div>
Syam Gadde | 27 Mar 21:29 2014

[Cython] memoryview slice transpose

I wanted to re-offer for discussion a fix for a bug I reported a while 
back.  The code that breaks is this:

import numpy
cimport numpy

def testfunc():
    a = numpy.arange(12).reshape([3,4])
    cdef numpy.int_t[:,:] a_view = a
    cdef numpy.int_t[:,:] b = a_view[:,:].T

One problem is that when a memoryview slice is decremented, the .memview 
and .data fields are not set to NULL, and any error that causes a jump 
to cleanup code will attempt to decref it again.

The second problem (I think) is that assignment of a transpose of 
memoryview slices to a variable is not resulting in an incref on the 
underlying slice, even though the variable is not a temporary and 
continues to persist.

Here is a patch that solves the problem for me.

https://github.com/SyamGadde/cython/commit/5739d8b908f18c4fc9103ef04e39964d61af3495

The part I am least sure about is removing the "if" qualifications on 
the incref and decref:

   if self.obj.is_name  or self.obj.is_attribute and self.obj.is_memslice_transpose:

that was obviously there for a reason, but it causes problems in the above case.  If there is a less extreme
solution I'd be happy to take it.

Thanks,

-syam
Pauli Virtanen | 18 Mar 23:56 2014
Picon
Picon

[Cython] Fused type signature resolution failures

Hi,

Here are the fused types + memoryview signature resolution failures I
promised earlier (Cython 0.20.1):

(A)

TypeError: No matching signature

----------------------------- asd.pyx
cimport numpy as cnp

ctypedef fused value_t:
    cnp.float32_t
    cnp.float64_t

cpdef foo(value_t x):
    pass
----------------------------- quux.py
import numpy as np
import asd
asd.foo(np.float32(1.0))
-----------------------------

(B)

ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'double'

----------------------------- asd.pyx
cimport numpy as cnp

ctypedef fused idx_t:
    cnp.int32_t
    cnp.int64_t

ctypedef fused value_t:
    cnp.int64_t
    cnp.float64_t

cpdef foo(idx_t[:,:] i, idx_t[:,:] j, value_t[:,:] x):
    pass
----------------------------- quux.py
import numpy as np
import asd
i = np.zeros((3, 3), np.int64)
j = np.zeros((3, 3), np.int64)
x = np.zeros((3, 3), np.float64)
asd.foo(i, j, x)
-----------------------------

(C)

Then some nasty platform-dependent failures:

https://github.com/scipy/scipy/issues/3461

The relevant code is:

https://github.com/scipy/scipy/blob/master/scipy/sparse/_csparsetools.pyx#L202

The code looks nothing special. However, call to `lil_fancy_get` fails
with "TypeError: No matching signature found" when the inputs have types

<class 'int'> <class 'int'> <class 'numpy.ndarray'>
<class 'numpy.ndarray'> <class 'numpy.ndarray'> <class 'numpy.ndarray'>
<class 'numpy.ndarray'> <class 'numpy.ndarray'>

with ndarray dtypes: object object object object int32 int32

The failure occurs only on Christoph Gohlke's Win64 build, but not on
Linux/OSX/MINGW32. This sounds like some integer size combination
issue, but I'm far from sure.

Unfortunately, I'm not easily able to isolate/understand what's going
wrong here.

--

-- 
Pauli Virtane

Pauli Virtanen | 8 Mar 21:11 2014
Picon
Picon

[Cython] Too many instantiations with fused type memoryviews

Hi,

FYI: Cython instantiates fused type routines with memoryview arguments
unnecessarily many times.

Example:
```
ctypedef fused idx_t:
    short
    long

# Cython 0.20.1 instantiates this function 128 times,
# even though only 2 would be needed
def fubar_128(idx_t[:] a,
              idx_t[:] b,
              idx_t[:] c,
              idx_t[:] d,
              idx_t[:] e,
              idx_t[:] f,
              idx_t[:] g):
    print("HALLO")

def fubar_2(idx_t a,
            idx_t b,
            idx_t c,
            idx_t d,
            idx_t e,
            idx_t f,
            idx_t g):
    print("HULLO")
```
$ cython cymod.pyx
$ cython --version
Cython version 0.20.1
$ grep -c 'print("HALLO")' cymod.c
128
$ grep -c 'print("HULLO")' _cymod.c
2

The n**m explosion starts to hurt quite quickly when there are several
array arguments and more than one fused type. I think this issue is
also accompanied by some signature resolution bugs (I'll try to come
up with an example case).

Cheers,

--

-- 
Pauli Virtanen

Stefan Behnel | 3 Mar 17:38 2014
Picon

[Cython] Crash with freelist() and __slots__

Hi,

I was made aware of crashes in the last lxml release, which turned out to
be due to the use of freelists for types that could be subtyped from Python
code. I was able to work around them in lxml, however, the real problem is
in Cython. There was supposed to be a safe guard for that case in the
freelist code based on the object struct size, which increases for subtypes
that have a __dict__. However, if the Python subtype uses an empty
__slots__ declaration, the object struct size will not increase, thus
passing the guard.

The correct fix is to also test if the type being instantiated lives on the
heap and exclude it from the freelist if so.

https://github.com/cython/cython/commit/d2aff824dd0900982a420ebaaa1dac6120a9d72e

This, together with the buffer/memory view related bugs I fixed since the
last release, suggest (at least to me) that we shouldn't wait all too long
with the next bug fix release.

From my POV, all changes in current master are safe enough to go out.

Stefan
Syam Gadde | 27 Feb 16:22 2014

[Cython] line tracing/profiling code objects

Hi all,

I tried using line tracing in conjunction with line_profiler/kernprof to 
attempt to see if line-based profiling works in Cython.  I think it's 
pretty close.  But I think the problem is that line_profiler is getting 
a different PyCodeObject when it wraps the function and when it actually 
gets the trace call.  It adds the initial code object to a map, and 
later when it gets the trace call, decides that the trace call is not 
something it needs to pay attention to because the new code object that 
it gets is not the same as the original one.

The first code object is created at function declaration by 
__Pyx_PyCode_New (called in__Pyx_InitCachedConstants) and is assigned to 
a variable __pyx_k_codeobj_NNN.  The second code object is created, 
essentially, during the first entry into the function (in 
__Pyx_TraceCall, via __Pyx_TraceSetupAndCall).  It seems that setting 
__pyx_frame_code to the initial code object before calling TraceCall() 
would fix this.

Is this easy to do?  I'd do it myself, but I'd need help figuring out 
how to get the name of the appropriate __pyx_k_codeobj_NNN variable from 
within FuncDefNode.generate_function_definitions(), which calls 
put_trace_call().

Thanks for your help...

-syam
Stefan Behnel | 22 Feb 10:02 2014
Picon

[Cython] 0.20.2

Hi,

there are a couple of nice additions and fixes in the current master that
could go out as a 0.20.2 some time soon. I suggest waiting another couple
of days for more bug reports to come in and then just send it out.

I'd like to also fix the buffer formatting bug by then, looks like I got a
hold of it.

Stefan

Gmane