headers
Magnus Lie Hetland | 2 Jan 22:02
Gravatar

[Cython] Array parameter...

Hi!

I'm writing a method of an extension class that will take an array as  
an argument (or, rather, a pointer and a size, for example). Now, I  
don't, strictly speaking, need to access this method from Python  
(i.e., it can be internal to the Cython code), but I'm writing my unit  
tests in Python, so it'd be nice... On the other hand, I'm thinking I  
wouldn't want to use list or the like as the type (in a cpdef) just to  
accomplish this, as that would, most likely, slow things down in  
actual Cython-to-Cython (or C-to-C) calls.

Is there a standard way of doing this sort of thing? My thinking is  
that I can write a simple wrapper for the Python use, which does the  
proper conversion, but I don't know if that's ideal... (It doesn't  
*have* to be an array. I just need to pass in a variable number of  
doubles in an efficient manner.)

--

-- 
Magnus Lie Hetland
http://hetland.org
Permalink | Reply | 2 comments |
headers
Robert Bradshaw | 2 Jan 22:11
Favicon

Re: [Cython] Array parameter...

On Jan 2, 2009, at 1:02 PM, Magnus Lie Hetland wrote:

> Hi!
>
> I'm writing a method of an extension class that will take an array as
> an argument (or, rather, a pointer and a size, for example). Now, I
> don't, strictly speaking, need to access this method from Python
> (i.e., it can be internal to the Cython code), but I'm writing my unit
> tests in Python, so it'd be nice... On the other hand, I'm thinking I
> wouldn't want to use list or the like as the type (in a cpdef) just to
> accomplish this, as that would, most likely, slow things down in
> actual Cython-to-Cython (or C-to-C) calls.

Using a list to wrap a list of doubles will *vastly* slow things  
down, probably by a factor of 100 or more.

> Is there a standard way of doing this sort of thing? My thinking is
> that I can write a simple wrapper for the Python use, which does the
> proper conversion, but I don't know if that's ideal... (It doesn't
> *have* to be an array. I just need to pass in a variable number of
> doubles in an efficient manner.)

The standard way to do things now is to write a cdef function that  
takes a double* and length argument, then write a python wrapper for  
testing. You could also look at using NumPy arrays with the buffer  
interface, depending on what you're doing.

- Robert
(Continue reading)

Permalink | Reply | 1 comment |
headers
Magnus Lie Hetland | 2 Jan 22:31
Gravatar

Re: [Cython] Array parameter...

On Jan 2, 2009, at 22:11 , Robert Bradshaw wrote:

> The standard way to do things now is to write a cdef function that
> takes a double* and length argument, then write a python wrapper for
> testing.

Yeah, that's what I had planned. Sounds like a good solution, I guess.  
(Now I just need a naming convention to separate the two ;-)

> You could also look at using NumPy arrays with the buffer
> interface, depending on what you're doing.

Indeed. I played with that a bit; it might be the best bet, perhaps  
(although sticking with only Cython has the advantage that I can ship  
the generated C code, and the user needn't install anything beyond  
Python).

--

-- 
Magnus Lie Hetland
http://hetland.org
Permalink | Reply | 0 comments |
headers
Kirill Simonov | 2 Jan 22:41
Picon

[Cython] long long and VC6

Hi,

I'm using Cython 0.10.3 to build an extension module and the generated C 
file does not compile under VC6, which I use to create a Windows binary 
package for Python 2.3.  The culprit is the following expression:

PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) : 
PyLong_FromUnsignedLongLong((sizeof(unsigned long long) > 
sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i);

in the function __Pyx_GetItemInt.  It looks like VC6 does not know about 
"long long".  I suppose Cython should respect HAVE_LONG_LONG and use 
PY_LONG_LONG macro instead of "long long".

Thanks,
Kirill
Permalink | Reply | 5 comments |
headers
Robert Bradshaw | 2 Jan 23:20
Favicon

Re: [Cython] long long and VC6

On Jan 2, 2009, at 1:41 PM, Kirill Simonov wrote:

> Hi,
>
> I'm using Cython 0.10.3 to build an extension module and the  
> generated C
> file does not compile under VC6, which I use to create a Windows  
> binary
> package for Python 2.3.  The culprit is the following expression:
>
> PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) :
> PyLong_FromUnsignedLongLong((sizeof(unsigned long long) >
> sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i);
>
> in the function __Pyx_GetItemInt.  It looks like VC6 does not know  
> about
> "long long".  I suppose Cython should respect HAVE_LONG_LONG and use
> PY_LONG_LONG macro instead of "long long".

Hmm... I don't have any experience with VC6, but it looks like a bug  
to me. Could you file a bug report on trac? Also, do you know if  
PY_LONG_LONG guaranteed to be 64+ bits?

- Robert
Permalink | Reply | 4 comments |
headers
Kirill Simonov | 3 Jan 00:14
Picon

Re: [Cython] long long and VC6

Hi Robert,
>>
>> I'm using Cython 0.10.3 to build an extension module and the  
>> generated C
>> file does not compile under VC6, which I use to create a Windows  
>> binary
>> package for Python 2.3.  The culprit is the following expression:
>>
>> PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) :
>> PyLong_FromUnsignedLongLong((sizeof(unsigned long long) >
>> sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i);
>>
>> in the function __Pyx_GetItemInt.  It looks like VC6 does not know  
>> about
>> "long long".  I suppose Cython should respect HAVE_LONG_LONG and use
>> PY_LONG_LONG macro instead of "long long".
> 
> Hmm... I don't have any experience with VC6, but it looks like a bug  
> to me. Could you file a bug report on trac? Also, do you know if  
> PY_LONG_LONG guaranteed to be 64+ bits?

I can't file a bug report, got Forbidden here:

http://trac.cython.org/cython_trac/newticket

I don't know about PY_LONG_LONG on 64+, but long long is C99, so I 
suppose it's a reasonably safe assumption.

Thanks,
Kirill.
(Continue reading)

Permalink | Reply | 3 comments |
headers
Michael Abshoff | 3 Jan 00:22

Re: [Cython] long long and VC6

Kirill Simonov wrote:
> Hi Robert,

Hi,

>>> I'm using Cython 0.10.3 to build an extension module and the  
>>> generated C
>>> file does not compile under VC6, which I use to create a Windows  
>>> binary
>>> package for Python 2.3.  The culprit is the following expression:
>>>
>>> PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) :
>>> PyLong_FromUnsignedLongLong((sizeof(unsigned long long) >
>>> sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i);
>>>
>>> in the function __Pyx_GetItemInt.  It looks like VC6 does not know  
>>> about
>>> "long long".  I suppose Cython should respect HAVE_LONG_LONG and use
>>> PY_LONG_LONG macro instead of "long long".
>> Hmm... I don't have any experience with VC6, but it looks like a bug  
>> to me. Could you file a bug report on trac? Also, do you know if  
>> PY_LONG_LONG guaranteed to be 64+ bits?
> 
> I can't file a bug report, got Forbidden here:
> 
> http://trac.cython.org/cython_trac/newticket
> 
> I don't know about PY_LONG_LONG on 64+, but long long is C99, so I 
> suppose it's a reasonably safe assumption.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Kirill Simonov | 3 Jan 13:06
Picon

Re: [Cython] long long and VC6

>>>> I'm using Cython 0.10.3 to build an extension module and the  
>>>> generated C
>>>> file does not compile under VC6, which I use to create a Windows  
>>>> binary
>>>> package for Python 2.3.  The culprit is the following expression:
>>>>
>>>> PyObject *j = (likely(i >= 0) || !is_unsigned) ? PyInt_FromLong(i) :
>>>> PyLong_FromUnsignedLongLong((sizeof(unsigned long long) >
>>>> sizeof(Py_ssize_t) ? (1ULL << (sizeof(Py_ssize_t)*8)) : 0) + i);
>>>>
>>>> in the function __Pyx_GetItemInt.  It looks like VC6 does not know  
>>>> about
>>>> "long long".  I suppose Cython should respect HAVE_LONG_LONG and use
>>>> PY_LONG_LONG macro instead of "long long".
>>> Hmm... I don't have any experience with VC6, but it looks like a bug  
>>> to me. Could you file a bug report on trac? Also, do you know if  
>>> PY_LONG_LONG guaranteed to be 64+ bits?
>> I can't file a bug report, got Forbidden here:
>>
>> http://trac.cython.org/cython_trac/newticket
>>
>> I don't know about PY_LONG_LONG on 64+, but long long is C99, so I 
>> suppose it's a reasonably safe assumption.
> 
> I am not sure what Python 2.3 does, but the standard workaround for VC 6 
> is to define long long as __int64. But if PY_LONG_LONG is supported I 
> would obviously recommend to use that.

Indeed you are right.  After replacing "unsigned long long" with 
"unsigned PY_LONG_LONG" and "1ULL" with "((unsigned PY_LONG_LONG) 1)",
(Continue reading)

Permalink | Reply | 0 comments |
headers
Dag Sverre Seljebotn | 3 Jan 16:57
Picon
Picon

Re: [Cython] Array parameter...

Another thing you could do is create your own mini-array Cython class. E.g.: A class "DoubleArray" has C
fields "ptr" and "len", and a subclass "PyDoubleArray" allows initializing those using a Python list
(and managing the memory needed to hold the converted result etc.). That's about 20 lines, nothing fancy...

So from the unit tests you'd pass inn "PyDoubleArray" and from Cython code raw "DoubleArray".

Your methods could then be "cpdef" and use the same interface.

Doesn't quite work out if you want to create C-callable code though.

Dag Sverre Seljebotn
-----Original Message-----
From: Magnus Lie Hetland <magnus@...>
Date: Friday, Jan 2, 2009 10:31 pm
Subject: Re: [Cython] Array parameter...
To: cython-dev@...: cython-dev@...

On Jan 2, 2009, at 22:11 , Robert Bradshaw wrote:
>
>> The standard way to do things now is to write a cdef function that
> takes a double* and length argument, then write a python wrapper for
> testing.
>
>Yeah, that's what I had planned. Sounds like a good solution, I guess.  
>(Now I just need a naming convention to separate the two ;-)
>
>> You could also look at using NumPy arrays with the buffer
> interface, depending on what you're doing.
>
>Indeed. I played with that a bit; it might be the best bet, perhaps
(Continue reading)

Permalink | Reply | 501 comments |
headers
Magnus Lie Hetland | 3 Jan 17:40
Gravatar

Re: [Cython] Array parameter...

On Jan 3, 2009, at 16:57 , Dag Sverre Seljebotn wrote:

> Another thing you could do is create your own mini-array Cython  
> class. E.g.: A class "DoubleArray" has C fields "ptr" and "len", and  
> a subclass "PyDoubleArray" allows initializing those using a Python  
> list (and managing the memory needed to hold the converted result  
> etc.). That's about 20 lines, nothing fancy...
>
> So from the unit tests you'd pass inn "PyDoubleArray" and from  
> Cython code raw "DoubleArray".

Cool idea. I thought about using some form of struct, but I didn't  
think quite hard enough, I guess (i.e., the subclassing bit :)

> Your methods could then be "cpdef" and use the same interface.

Indeed. If it turns out that this double array is the main cause of  
"wrapping", that would save me quite a bit of cookie-cutter conversion  
code (as it might apply to several methods).

> Doesn't quite work out if you want to create C-callable code though.

Right. Not an issue for now; I guess I'll just deal with that if it  
crops up.

Thanks,

- M

--

--
(Continue reading)

Permalink | Reply | 501 comments |

Gmane