maldun | 2 May 11:37 2011
Picon
Picon

Problems with structs in numpy arrays

Hi,

I'm Using ubuntu 10.04 and cython in sage.

I have the followoing Code snippet:

# -*- coding: utf-8 -*-

from numpy import zeros
cimport numpy as cnumpy
import numpy as np

cdef struct MyStruct:
     cnumpy.int32_t a
     cnumpy.float64_t b

def test(int n):

    DataLine_desc = {'names': ('a', 'b'), 'formats': (np.int32 ,
np.float64)}

    cdef cnumpy.ndarray[MyStruct] tester = zeros(n,dtype =
np.dtype(DataLine_desc,align = True))

    print tester[0]

If I call now
test(3)

I get the following error:
(Continue reading)

maldun | 2 May 16:37 2011
Picon
Picon

Re: How can I create 10 Cython objects in a for loop?

Interesting. I had a similar problem but with containers:

I have a container class:

cdef class my_container:
  cdef int a

  def __init__(self,int a):
    self.a = a

now i pack them in a c++ vector:

from libcpp.vector cimport vector

cdef vector[my_container] vect
cdef int i
for i in range(10):
    vect.push_back(my_container(i))

cdef int result = 0
for i in range(10):
    result += vect[i].a

now the result is 90 and not 45, because If you look in the vector
every entry holds the same object.
But I don't think this behavior is wanted...

greez maldun

On Apr 28, 12:08 am, Robert Bradshaw <rober... <at> math.washington.edu>
(Continue reading)

Robert Bradshaw | 2 May 18:02 2011

Re: Re: How can I create 10 Cython objects in a for loop?

On Mon, May 2, 2011 at 7:37 AM, maldun <domors <at> gmx.net> wrote:
> Interesting. I had a similar problem but with containers:
>
> I have a container class:
>
> cdef class my_container:
>  cdef int a
>
>  def __init__(self,int a):
>    self.a = a
>
> now i pack them in a c++ vector:
>
> from libcpp.vector cimport vector
>
> cdef vector[my_container] vect
> cdef int i
> for i in range(10):
>    vect.push_back(my_container(i))
>
> cdef int result = 0
> for i in range(10):
>    result += vect[i].a
>
> now the result is 90 and not 45, because If you look in the vector
> every entry holds the same object.
> But I don't think this behavior is wanted...

I didn't even know we supported vector[my_cpp_class], we probably
shouldn't, as the use needs to manually incref/decref when the object
(Continue reading)

Stefan Behnel | 2 May 18:41 2011
Picon

Re: Re: How can I create 10 Cython objects in a for loop?

Robert Bradshaw, 02.05.2011 18:02:
> On Mon, May 2, 2011 at 7:37 AM, maldun wrote:
>> Interesting. I had a similar problem but with containers:
>>
>> I have a container class:
>>
>> cdef class my_container:
>>   cdef int a
>>
>>   def __init__(self,int a):
>>     self.a = a
>>
>> now i pack them in a c++ vector:
>>
>> from libcpp.vector cimport vector
>>
>> cdef vector[my_container] vect
>> cdef int i
>> for i in range(10):
>>     vect.push_back(my_container(i))
>>
>> cdef int result = 0
>> for i in range(10):
>>     result += vect[i].a
>>
>> now the result is 90 and not 45, because If you look in the vector
>> every entry holds the same object.
>> But I don't think this behavior is wanted...
>
> I didn't even know we supported vector[my_cpp_class], we probably
(Continue reading)

Dag Sverre Seljebotn | 2 May 20:15 2011
Picon
Picon

Re: Re: How can I create 10 Cython objects in a for loop?

On 05/02/2011 06:41 PM, Stefan Behnel wrote:
> Robert Bradshaw, 02.05.2011 18:02:
>> On Mon, May 2, 2011 at 7:37 AM, maldun wrote:
>>> Interesting. I had a similar problem but with containers:
>>>
>>> I have a container class:
>>>
>>> cdef class my_container:
>>> cdef int a
>>>
>>> def __init__(self,int a):
>>> self.a = a
>>>
>>> now i pack them in a c++ vector:
>>>
>>> from libcpp.vector cimport vector
>>>
>>> cdef vector[my_container] vect
>>> cdef int i
>>> for i in range(10):
>>> vect.push_back(my_container(i))
>>>
>>> cdef int result = 0
>>> for i in range(10):
>>> result += vect[i].a
>>>
>>> now the result is 90 and not 45, because If you look in the vector
>>> every entry holds the same object.
>>> But I don't think this behavior is wanted...
>>
(Continue reading)

Robert Bradshaw | 2 May 20:23 2011

Re: Re: How can I create 10 Cython objects in a for loop?

On Mon, May 2, 2011 at 11:15 AM, Dag Sverre Seljebotn
<d.s.seljebotn <at> astro.uio.no> wrote:
> On 05/02/2011 06:41 PM, Stefan Behnel wrote:
>>
>> Robert Bradshaw, 02.05.2011 18:02:
>>>
>>> On Mon, May 2, 2011 at 7:37 AM, maldun wrote:
>>>>
>>>> Interesting. I had a similar problem but with containers:
>>>>
>>>> I have a container class:
>>>>
>>>> cdef class my_container:
>>>> cdef int a
>>>>
>>>> def __init__(self,int a):
>>>> self.a = a
>>>>
>>>> now i pack them in a c++ vector:
>>>>
>>>> from libcpp.vector cimport vector
>>>>
>>>> cdef vector[my_container] vect
>>>> cdef int i
>>>> for i in range(10):
>>>> vect.push_back(my_container(i))
>>>>
>>>> cdef int result = 0
>>>> for i in range(10):
>>>> result += vect[i].a
(Continue reading)

Pauli Virtanen | 2 May 21:55 2011
Picon
Picon

Re: Problems with structs in numpy arrays

On Mon, 02 May 2011 02:37:46 -0700, maldun wrote:
> cdef struct MyStruct:
>      cnumpy.int32_t a
>      cnumpy.float64_t b
[clip]
>     cdef cnumpy.ndarray[MyStruct] tester
[clip]
> I get the following error:
> 
> ValueError: Does not understand character buffer dtype format string
> (':')

Using structs in Numpy arrays has proper support at the moment only in 
the development version of Cython, AFAIK. It works properly for me with 
the current dev version.

    ***

Nevertheless, using cast=True probably shouldn't segfault. Looks like a 
refcount failure somewhere, either in Cython or in Numpy on the error 
handling path. Don't know yet which. Valgrind has this to say:

==5619== Invalid free() / delete / delete[]
==5619==    at 0x4C282ED: free (vg_replace_malloc.c:366)
==5619==    by 0x754D2C0: ??? (in /usr/lib/pyshared/python2.7/numpy/core/
multiarray.so)
==5619==    by 0x756D098: ??? (in /usr/lib/pyshared/python2.7/numpy/core/
multiarray.so)
==5619==    by 0x731F95B: __pyx_pf_3foo_test (foo.c:995)
==5619==    by 0x496B3E: PyEval_EvalFrameEx (in /usr/bin/python2.7)
(Continue reading)

n1ywb | 2 May 23:52 2011
Picon

Shared declarations, shared libraries

I'm not sure how to describe this but here goes. I've got two Cython
modules. The first one has no actual Cython code, but it links with
a .c file and exports a declaration in it's .pxd. The second module
actually cimports the first module and calls something implemented in
the c file.

It builds fine but when I try to import the second module it complains
about undefined symbols; symbols that are in fact defined in the first
module.

SHOULD this work? It seems like it intuitively should. But what do I
know, maybe this isn't even a Cython problem.

As a hack I realized I can just add the c file to the test2 sources
and everything seems fine. But that sort of defeats the purpose of a
shared library.

What's the RIGHT approach to this sort of thing? I'm guessing it would
be to compile the C file into a non-python shared library and link the
two python modules against that. Is there a nice way to do that with
distutils?

Here is a contrived example that exhibits this behaviour.

::::::::::::::
setup.py
::::::::::::::
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
(Continue reading)

Pauli Virtanen | 3 May 01:13 2011
Picon
Picon

Re: Shared declarations, shared libraries

On Mon, 02 May 2011 14:52:48 -0700, n1ywb wrote:
> I'm not sure how to describe this but here goes. I've got two Cython
> modules. The first one has no actual Cython code, but it links with a .c
> file and exports a declaration in it's .pxd. The second module actually
> cimports the first module and calls something implemented in the c file.
[clip]
> ::::::::::::::
> ctest.pxd
> ::::::::::::::
> cdef extern from "ctest.h":
>     int foo()

The file should only contain the line

	cdef int foo()

If you use the "cdef extern", you need to link them from an external 
library (and be aware that Cython does some name mangling).

--

-- 
Pauli Virtanen

Pauli Virtanen | 3 May 01:17 2011
Picon
Picon

Re: Shared declarations, shared libraries

On Mon, 02 May 2011 23:13:46 +0000, Pauli Virtanen wrote:
> On Mon, 02 May 2011 14:52:48 -0700, n1ywb wrote:
>> I'm not sure how to describe this but here goes. I've got two Cython
>> modules. The first one has no actual Cython code, but it links with a
>> .c file and exports a declaration in it's .pxd. The second module
>> actually cimports the first module and calls something implemented in
>> the c file.
> [clip]
>> ::::::::::::::
>> ctest.pxd
>> ::::::::::::::
>> cdef extern from "ctest.h":
>>     int foo()
> 
> The file should only contain the line
> 
> 	cdef int foo()
> 
> If you use the "cdef extern", you need to link them from an external
> library (and be aware that Cython does some name mangling).

Sorry, I read your mail carelessly. Cython-defined functions can be 
cimported by using the above construction, but for functions defined in 
the C file, I'm not sure if that is supported --- maybe not. You could 
perhaps write cdef wrappers for those, or just link the C file also to 
the second library.

--

-- 
Pauli Virtanen

(Continue reading)


Gmane