headers
রামি চৌধুরি | 14 May 2013 16:48
Picon
Gravatar

build error using cython on OS X and Python 3.3 -- __time_t appearing from nowhere

Hi everyone,


I'm having an error building a Cython module on OS X, using Python 3.3.

The code I'm trying to build is https://github.com/esplorio/couchbase-python-client/tree/pylibcouchbase -- it's a fairly simple wrapper for a C library, using an autogenerated .pxd

The output from `python setup.py build_ext --inplace` is: http://pastebin.com/aPv93K5N

I used to get the same error on Python 2.7 but hacked around it using a few lines like: https://github.com/esplorio/couchbase-python-client/blob/pylibcouchbase/couchbase/libcouchbase.pxd#L68 -- so it compiles with Python 2.7 now.

The erroring C typedef is `__time_t`, which is `__darwin_time_t` on OS X, but I have no idea where the `__time_t` is appearing from, or why it's being emitted in the first place (since `time_t` is available on both platforms and doesn't need to be typedef'd).

I'd be grateful for any comments or suggestions.

Thanks,
Rami


--
 
---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Permalink | Reply | 4 comments |
headers
Alexander Eisenhuth | 13 May 2013 11:50
Picon

Function pointer as part of a C struct

Hello everybody,

I want to use a callback as part of a C struct with read/write access. 
Is this possible? The following describes the problem. And if you can 
help, how do I connect a python function to the callback?

Any help welcome!

Regards
Alexander

---- stderr of compiling Device.pyx ---
Error compiling Cython file:
------------------------------------------------------------
...
     def __setattr__(self, name, value):
         print "DeviceDescriptor_.__setattr__", name, value
         if name == "deviceNumber":
             self._device_descriptor.deviceNumber = <Uint8> value
         elif name == "deviceMeasurement":
             self._device_descriptor.deviceMeasurement = 
<DeviceMeasureFunc> value
                                                        ^
------------------------------------------------------------
Device.pyx:55:56: Python objects cannot be cast to pointers of primitive 
types

----- Device.h ----
typedef ErrorStatus (*DeviceMeasureFunc)(Uint32*);

typedef struct DeviceDescriptorTag
{
	Uint8 deviceNumber;
	DeviceMeasureFunc deviceMeasurement;

} DeviceDescriptor;
------- Device.pyx ---
cdef extern from "Device.h":

     ctypedef ErrorStatus (*DeviceMeasureFunc)(Uint32*)

     ctypedef struct DeviceDescriptor_c "DeviceDescriptor":
         Uint8 deviceNumber
         DeviceMeasureFunc deviceMeasurement

cdef class DeviceDescriptor:

     cdef DeviceDescriptor_c _device_descriptor

     def __getattr__(self, name):
         ret_val = None
         if name == "deviceNumber":
             ret_val = self._device_descriptor.deviceNumber
         else:
             self.__raise_attrib_not_found(name)
         return ret_val

     def __setattr__(self, name, value):
         if name == "deviceNumber":
             self._device_descriptor.deviceNumber = <Uint8> value
         elif name == "deviceMeasurement":
             self._device_descriptor.deviceMeasurement = 
<DeviceMeasureFunc> value
         else:
             self.__raise_attrib_not_found(name)

     def __raise_attrib_not_found(self, name):
         raise AttributeError("Attribute %s not in %s" % (name, type(self)))

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Permalink | Reply | 7 comments |
headers
Shriramana Sharma | 12 May 2013 09:24
Picon

TypeError: can't apply this __setattr__

For practice, I am implementing a subclass of builtins.list in Cython
which will store only str-s and has one data member "approved" which
should take only a boolean. It's entirely Python syntax except that it
has a cdef at the head.

However I am getting a TypeError saying "can't apply this __setattr__"
when I try to initialize an object of this type:

>>> from strlist import strlist
>>> a=strlist()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "strlist.pyx", line 24, in strlist.strlist.__init__ (strlist.c:1215)
    self.approved = False
  File "strlist.pyx", line 33, in strlist.strlist.__setattr__ (strlist.c:1417)
    super().__setattr__ ( name, value )
TypeError: can't apply this __setattr__ to strlist.strlist object

OTOH when I try the same using the pure Python implementation of the
class (i.e. just removed cdef) I have no problem:

>>> from strlist_py import strlist
>>> a=strlist()
>>> a
strlist(<super: <class 'strlist'>, <strlist object>>,approved=False)

The code is attached. Can anyone please help me figure out what I'm
doing wrong? Thanks.

-- 
Shriramana Sharma ஶ்ரீரமணஶர்மா श्रीरमणशर्मा

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Attachment (subclass-pyclass.tar.gz): application/x-gzip, 1201 bytes
Permalink | Reply | 7 comments |
headers
Alok Singhal | 11 May 2013 22:52
Picon
Gravatar

if condition transformation to switch-case

Consider the following code:


cdef extern from "errno.h":
    enum:
        EWOULDBLOCK
        EAGAIN

cdef int is_eagain(int i):
    return i != EWOULDBLOCK and i != EAGAIN

The code checks for EWOULDBLOCK and EAGAIN both, although they could be the same (on linux for example).  This is mostly done for portability (for example, see the "Portability note" at http://www.gnu.org/savannah-checkouts/gnu/libc/manual/html_node/Error-Codes.html#index-EAGAIN-97).

Cython transforms the above code to:

  switch (__pyx_v_i) {
    case EWOULDBLOCK:
    case EAGAIN:
    __pyx_t_1 = 0;
    break;
    default:
    __pyx_t_1 = 1;
    break;
  }

If EWOULDBLOCK == EAGAIN, this results in compilation error since the two case labels are the same.

For my immediate use, I changed my code to only test for EAGAIN, but is there a better way to write the code so that I can still use both EWOULDBLOCK and EAGAIN?

Thanks,
Alok

--
 
---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Permalink | Reply | 6 comments |
headers
Stefan Behnel | 11 May 2013 11:01
Picon
Favicon

Cython 0.19.1 released

Hi all,

Cython 0.19.1 is following the course of the recent 0.19 release. This is
mainly a bug fix release that adds only minor new features and fixes some
bugs and regressions in the latest release. Upgrading is generally recommended.

You can get it from here:

http://cython.org/release/Cython-0.19.1.tar.gz

http://cython.org/release/Cython-0.19.1.zip

Release notes:
https://github.com/cython/cython/blob/8f07f8d8265799cada5046fe0b7c0807d7b41fb3/CHANGES.rst

Documentation: http://docs.cython.org/

Major changes include:

* A warning is emitted when negative literal indices are found inside of
  a code section that disables ``wraparound`` handling.  This helps with
  fixing invalid code that might fail in the face of future compiler
  optimisations.

* A regression in 0.19 was fixed that rejected valid C expressions from
  being used in C array size declarations.

* Testing large (> int) C integer values for their truth value could fail
  due to integer wrap-around.

* The automatic signature documentation tries to preserve more semantics
  of default arguments and argument types.  Specifically, ``bint``
  arguments now appear as type ``bool``.

Have fun,

Stefan

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Permalink | Reply | 0 comments |
headers
Martin Fiers | 11 May 2013 10:17
Picon

Numpy access reduces reference count. Numpy array not recognized?

Dear Cython community!

Yesterday I bumped onto a fairly involved problem regarding numpy buffers. The basic idea is that I wanted to send a numpy-generated array from a self-made C++ class (TestObject.h) to a cython function (pass_np_array.pyx). Apparently, there's something wrong with the reference counting and the code crashes after a few cycles.

I made a minimal working example, which can be found in attachment, because it's fairly involved. To run locally, you can extract the tar and call run_test.sh.

Comments are inside the test file test_pass_array.py and in the pyx file which I inline here because most information is in there.

I also tried compiling with the refnanny support to see if there's something wrong there. But it seems it doesn't detect counting errors. For more information, see the test_pass_array.py.


pass_np_array.pyx
------------------------------------

import numpy as np
cimport numpy as np
import cython

cdef extern from "TestObject.h":
    cdef cppclass TestObject:
        TestObject()
       
        # Create a numpy array, and deliberately increase refcount to demonstrate crash.
        void create_object(int additional_refcount)
       
        # Does this actually help defining the buffer correctly?
        np.ndarray[complex, ndim=1] get_numpy_array()
       



cdef print_object(TestObject obj):

    # First problem: I need an explicit cast arr = obj.get_numpy_array()
    # So I cannot access the matrix elements using obj.get_numpy_array()[0]
    cdef np.ndarray[complex, ndim=1] arr
   
    # Second problem:
    # In cython, the code below generates __pyx_t_1 = ((PyObject *)__pyx_v_obj.get_numpy_array());
    # I guess this is wrong behavior, as this decreases the reference count later (Py_GOTREF/...)
    # Something like PyArray_FromArray would maybe be more appropriate, but hacking the generated cpp file
    # and adding PyArray_FromArray does not resolve the problem.
    arr = obj.get_numpy_array()
   
    # Printing the reference count: After each call to print_object, the reference count decreases.
    import sys
    print 'Ref count arr: ', sys.getrefcount(arr)
   
    print arr[0]
   
    # The code below crashes the cython compiler  "Compiler crash in AnalyseExpressionsTransform"
    #print obj.get_numpy_array()[0]
   
    # (all the problems above are probably related to the fact that the get_numpy_array
    # does not contain the correct information)
   
def run():
    cdef TestObject O
    O = TestObject()
   
    # Run Py_INCREF twice on the created object
    O.create_object(2)

    # First time it works
    print_object(O)
   
    # Second time: reference counting went down with one!?
    # We deliberately increased the ref-count in TestObject in create_object.
    print_object(O)

    print_object(O)
   
    # On the line below it will SEGFAULT/GLIBC/double corrupted list, or other
    # nice crash
    print_object(O)

    print_object(O)


Thank you for looking at this problem!

Regards,
Martin

--
 
---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Permalink | Reply | 1 comment |
headers
Martin Fiers | 11 May 2013 10:04
Picon

Numpy array reference count decreases with each function call

Hello,

I think I found some bug in the reference counting of numpy buffers. I isolated the part to make a minimal working example, which can be found in the attachment. After extracting, you can call run_test.sh which compiles the example and crashes. I tested cython-0.17,0.18 and 0.19.

The idea is that TestObject.h creates a numpy array and passes it to a cython function. It appears, after a full day of debugging and searching on the internet, that there's something wrong with reference counting and I cannot directly access the buffer. My guess is that it's caused by the get_numpy_array not being recognized correctly by the cython compiler. I inlined some of the possible causes/problems in comments.

Eventually, the program crashes because the reference counting decreases with one each time the function is called.
Also, I compile with the refnanny support to see if there's something wrong there. But it seems it doesn't detect counting errors. For more information, see the test_pass_array.py.

If there is no easy fix, is there some kind of workaround in order to keep the object 'alive'?

Here's the pyx file to illustrate the problems (for full working example, see attachment):


import numpy as np
cimport numpy as np
import cython

cdef extern from "TestObject.h":
    cdef cppclass TestObject:
        TestObject()
       
        # Create a numpy array, and deliberately increase refcount to demonstrate crash.
        void create_object(int additional_refcount)
       
        # Does this actually help defining the buffer correctly?
        np.ndarray[complex, ndim=1] get_numpy_array()
       

cdef print_object(TestObject obj):

    # First problem: I need an explicit cast arr = obj.get_numpy_array()
    # So I cannot access the matrix elements using obj.get_numpy_array()[0]
    cdef np.ndarray[complex, ndim=1] arr
   
    # Second problem:
    # In cython, the code below generates __pyx_t_1 = ((PyObject *)__pyx_v_obj.get_numpy_array());
    # I guess this is wrong behavior, as this decreases the reference count later (Py_GOTREF/...)
    # Something like PyArray_FromArray would maybe be more appropriate, but hacking the generated cpp file
    # and adding PyArray_FromArray does not resolve the problem.
    arr = obj.get_numpy_array()
   
    # Printing the reference count: After each call to print_object, the reference count decreases.
    import sys
    print 'Ref count arr: ', sys.getrefcount(arr)
   
    print arr[0]
   
    # The code below crashes the cython compiler  "Compiler crash in AnalyseExpressionsTransform"
    #print obj.get_numpy_array()[0]
   
    # (all the problems above are probably related to the fact that the get_numpy_array
    # does not contain the correct information)
   
def run():
    cdef TestObject O
    O = TestObject()
   
    # Run Py_INCREF twice on the created object
    O.create_object(2)

    # First time it works
    print_object(O)
   
    # Second time: reference counting went down with one.
    # We deliberately increased the ref-count in TestObject in create_object.
    print_object(O)

    print_object(O)
   
    # On the line below it will SEGFAULT/GLIBC/double corrupted list, or other
    # nice crash
    print_object(O)

    print_object(O)

Thank you for helping me solve this quite complex problem!
Martin

--
 
---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Attachment (cython_bug.tar): application/x-tar, 230 KiB
Permalink | Reply | 0 comments |
headers
Shriramana Sharma | 10 May 2013 13:17
Picon

Using Clang

This is really probably a distutils thing and not Cython-specific but
I don't want to subscribe to yet another mailing list for this one
thing so:

What do I have to change in the setup.py file to use Clang i.o. GCC
for compiling my Cython modules?

Thanks.

-- 
Shriramana Sharma ஶ்ரீரமணஶர்மா श्रीरमणशर्मा

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Permalink | Reply | 2 comments |
headers
Gabriel Jacobo | 9 May 2013 17:49
Picon
Gravatar

Two potential bugs in Cython 0.19

I've just got two user reports of issues compiling my engine with Cython 0.19 (which are not present in Cython 0.18).

The statement "import time", produces a "cythonization" error of 

import time, sys
      ^
------------------------------------------------------------

backends/sdl/GameLoop.pyx:16:7: Assignment to non-lvalue 'time'


The other error happens when compiling. On certain modules (not all, I haven't figured out the reason why yet) I get ‘PyDateTime_IMPORT’ and PyDateTimeAPI undeclared symbols errors.

Are these bugs or is there some new requirement on Cython 0.19 I'm not complying with? (additional imports required, etc)

Thanks for any help!

--
Gabriel.

--
 
---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 
Permalink | Reply | 7 comments |
headers
Shriramana Sharma | 9 May 2013 14:08
Picon

Python headers not a dependency for Cython?

I'm using Kubuntu Raring and surprisingly the cython and cython3
packages don't seem to depend on the python-dev and python3-dev
packages -- they only "suggest" them (green bullets):

http://packages.ubuntu.com/raring/cython
http://packages.ubuntu.com/raring/cython3

Likewise they don't depend on gcc, but then again there are other C compilers.

IIUC python-dev and python3-dev should be dependencies since it is not
possible to compile Cython files without the Python headers. if this
is correct, then I'll file a Debian packaging bug.

-- 
Shriramana Sharma ஶ்ரீரமணஶர்மா श्रीरमणशर्मा

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Permalink | Reply | 6 comments |
headers
Stefan Behnel | 8 May 2013 20:43
Picon
Favicon

project proposal: line tracing/profiling, coverage, etc.

Hi,

in the past, Robert and I have invested some work into supporting profiling
and tracing in Cython, and I would like to see this finalised. While Python
level tracing and profiling might not seem like the ideal tools for
performance critical Cython code, they are actually pretty handy when much
of your code is Python code and you want to do, say, coverage analysis over
your whole code base, without having to exclude the Cython parts.

So I've opened a ticket that highlights the missing bits.

http://trac.cython.org/cython_trac/ticket/815

If anyone's interested in pursuing this, I'd be glad to help getting you on
the right track. I don't even think there's all that much left to do, but
it's still a bit of cleanup and maybe some naughty little details along the
way. Would make a good entry project, IMHO.

Anyone interested?

Stefan

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Permalink | Reply | 1 comment |

Gmane