[Cython] [patch] Optional setuptools-based cython build

Hi,

I needed to be able to build an egg for cython on windows, and thought
it could be interesting to integrate it to cython sources. Besides an
egg, it provides cython as a console_script, which is easier to handle
on windows.

The patch is minimal and non invasive, as by default, setuptools is
*not* run if you use setup.py. I attached the patch here (I can't
connect to trac ATM),

cheers,

David

Hi,

I needed to be able to build an egg for cython on windows, and thought
it could be interesting to integrate it to cython sources. Besides an
egg, it provides cython as a console_script, which is easier to handle
on windows.

The patch is minimal and non invasive, as by default, setuptools is
*not* run if you use setup.py. I attached the patch here (I can't
connect to trac ATM),

cheers,

Re: [Cython] getting rid of the PY_NEW() hack

Lisandro Dalcin, 31.10.2009 22:12:
> On Sat, Oct 31, 2009 at 10:18 AM, Stefan Behnel wrote:
>> To fix this, I pushed a change that optimises this
>>
>>        cdef type some_type = ...
>>        obj = some_type.__new__(some_type)
>>
> 
> So it seems that the optimization only works if __new__ is called with
> just a single arg some_type... but your __new__ optimization will
> definitely call __cinit__. Do you think that optimizing a many-args
> __cinit__ is a nonsense?

Not nonsense, but certainly less important. Since the arguments are passed
as Python arguments in that case, it doesn't really fit the use case of
fast instantiation. Without having benchmarked it any recently, I assume
it's still quite a bit faster to do

	cdef MyType obj = MyType.__new__(MyType)
	obj.some_attr = something

than to do

	cdef MyType obj = MyType.__new__(MyType, something)

and set the attribute in __cinit__(). It may not be that a big difference
if you are only passing Python arguments, but it certainly is a bigger
difference if you additionally need to convert arguments from C types to
Python types and back, just to get them passed into __cinit__().

Re: [Cython] getting rid of the PY_NEW() hack

Hi Lisandro,

thanks for the feed-back.

I should note that I actually consider the current implementation also a
bit of a hack. The right place to implement this would be as part of the
builtin 'type' object, so that we could more easily support efficient
references to __new__, __init__, __call__, __add__, etc. But I think what's
there now will make life easier for users, as they no longer have to keep
header files around to do this. I'll be very happy if this can be ripped
back out in favour of a more general solution, but until then, it'll do its
job IMO.

Lisandro Dalcin, 31.10.2009 21:43:
> On Sat, Oct 31, 2009 at 10:18 AM, Stefan Behnel wrote:
>> I think this is safe, but I wanted to mention it here so that others can
>> give it another bit of thought.
> 
> Stefan, could you cythonize the line below:
> 
> cdef str s = str.__new__(str)
> 
> In the generated C code you will see a superfluous check "if(str is
> None): raise TypeError()" ... Can you figure out why this is
> happening?

Well, the above is also part of the tests, so I actually know that this is
happening - and I even know why! :)

The problem is that I wasn't sure how to figure out that a variable refers

Re: [Cython] error: ‘__Pyx_SafeReleaseBuffer’ was not declared in this scope

Neal Becker wrote:

> This code:
> -------------------
> import numpy as np
> cimport numpy as np
> 
> np.import_array()
> 
> cdef inline int x_ (int k, int n):
>     cdef int center = (n-1)/2
>     return k-center
> 
> def lagrange (float x, int n):
>     
>     cdef np.ndarray[np.float64_t, ndim=1] ret # = np.empty (n,
>     dtype=float) cdef int j, k
>     cdef float prod, top, bot
>     
>     for j in range (n):
>         prod = 1
>         for k in xrange (0, n):
>             if (k != j):
>                 top = x - x_(k, n)
>                 bot = x_(j, n) - x_(k, n)
>                 prod *= top/bot
>         ret[j] = prod
>     return ret
> --------------
> 

Re: [Cython] error: ‘__Pyx_SafeReleaseBuffer’ was not declared in this scope

Neal Becker wrote:
> Neal Becker wrote:
> 
>> This code:
>> -------------------
>> import numpy as np
>> cimport numpy as np
>>
>> np.import_array()
>>
>> cdef inline int x_ (int k, int n):
>>     cdef int center = (n-1)/2
>>     return k-center
>>
>> def lagrange (float x, int n):
>>     
>>     cdef np.ndarray[np.float64_t, ndim=1] ret # = np.empty (n,
>>     dtype=float) cdef int j, k
>>     cdef float prod, top, bot
>>     
>>     for j in range (n):
>>         prod = 1
>>         for k in xrange (0, n):
>>             if (k != j):
>>                 top = x - x_(k, n)
>>                 bot = x_(j, n) - x_(k, n)
>>                 prod *= top/bot
>>         ret[j] = prod
>>     return ret
>> --------------

Re: [Cython] More complex bugs

Robert Bradshaw wrote:
> On Oct 28, 2009, at 8:39 AM, Dag Sverre Seljebotn wrote:
> 
>> I just added http://trac.cython.org/cython_trac/ticket/441
>>
>> Basically,
>>
>> ctypedef float myfloat;
>> ...
>> print some_myfloat * some_complex
>>
>> miscompiles as "some_myfloat" is coerced to "myfloat complex" (which,
>> interestingly, is a type which can't be created or used in any other  
>> way!)
>>
>> Fixing this requires some thought.
>>
>> This probably broke because (in order to fix another bug) I stepped  
>> away
>> from "the order something is needed in Cython decides output order in
>> C". I believe that was a correct decision and don't want to step back.
>> The proper solution is a DAG of all types and their dependencies. I
>> believe that is a bit heavy/destabilizing for 0.12 though. Perhaps
>> output all very simply typedefs (typedefs of ints and floats) first,
>> then complex, then the rest?
> 
> Yep, I don't see the full DAG analysis happening in 0.12. I think any  
> arbitrary order we come up with is potentially prone to errors. I'm OK  
> with just disallowing that for 0.12, it's not a regression.

Re: [Cython] More complex bugs

On Nov 2, 2009, at 11:53 AM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Oct 28, 2009, at 8:39 AM, Dag Sverre Seljebotn wrote:
>>
>>> I just added http://trac.cython.org/cython_trac/ticket/441
>>>
>>> Basically,
>>>
>>> ctypedef float myfloat;
>>> ...
>>> print some_myfloat * some_complex
>>>
>>> miscompiles as "some_myfloat" is coerced to "myfloat  
>>> complex" (which,
>>> interestingly, is a type which can't be created or used in any other
>>> way!)
>>>
>>> Fixing this requires some thought.
>>>
>>> This probably broke because (in order to fix another bug) I stepped
>>> away
>>> from "the order something is needed in Cython decides output order  
>>> in
>>> C". I believe that was a correct decision and don't want to step  
>>> back.
>>> The proper solution is a DAG of all types and their dependencies. I
>>> believe that is a bit heavy/destabilizing for 0.12 though. Perhaps
>>> output all very simply typedefs (typedefs of ints and floats) first,
>>> then complex, then the rest?

Re: [Cython] [patch] Optional setuptools-based cython build

On Nov 1, 2009, at 1:32 AM, David Cournapeau wrote:

> Hi,
>
> I needed to be able to build an egg for cython on windows, and thought
> it could be interesting to integrate it to cython sources. Besides an
> egg, it provides cython as a console_script, which is easier to handle
> on windows.
>
> The patch is minimal and non invasive, as by default, setuptools is
> *not* run if you use setup.py. I attached the patch here (I can't
> connect to trac ATM),

This sounds like a good idea, though distutils and packaging are  
outside my area of expertise, especially on Windows. I am a bit wary  
of the line

if 'setuptools' in sys.modules

is this always safe?

- Robert

Re: [Cython] "real" module support

On Oct 29, 2009, at 12:31 PM, Sebastien Binet wrote:

> hi there,
>
> I have some freecycles these days and I was wondering if I couldn't  
> dive a
> little bit more into cython's codebase.
>
> my pet peeve would be to add some more user friendly module support  
> in cython,
> so one could write:
>
> # mymodule.pyx
> __all__ = [ 'fct1', 'fct2', 'cls1', 'obj', ]
> # definitions follow
> # EOF
>
> and be able to cimport those in various other cython modules,  
> instead of
> having to create .pxd and .pyx files.
>
> so, where should I start ?

That is a highly requested feature, even without the __all__. To start  
I would look at find_module in Main.py, and change it to look for .pyx  
files if the search for .pxd files failed. There may or may not be  
other changes that are needed, but that's where to start.

- Robert

Re: [Cython] More complex bugs

Robert Bradshaw wrote:
> On Nov 2, 2009, at 11:53 AM, Dag Sverre Seljebotn wrote:
> 
>> Robert Bradshaw wrote:
>>> On Oct 28, 2009, at 8:39 AM, Dag Sverre Seljebotn wrote:
>>>
>>>> I just added http://trac.cython.org/cython_trac/ticket/441
>>>>
>>>> Basically,
>>>>
>>>> ctypedef float myfloat;
>>>> ...
>>>> print some_myfloat * some_complex
>>>>
>>>> miscompiles as "some_myfloat" is coerced to "myfloat  
>>>> complex" (which,
>>>> interestingly, is a type which can't be created or used in any other
>>>> way!)
>>>>
>>>> Fixing this requires some thought.
>>>>
>>>> This probably broke because (in order to fix another bug) I stepped
>>>> away
>>>> from "the order something is needed in Cython decides output order  
>>>> in
>>>> C". I believe that was a correct decision and don't want to step  
>>>> back.
>>>> The proper solution is a DAG of all types and their dependencies. I
>>>> believe that is a bit heavy/destabilizing for 0.12 though. Perhaps
>>>> output all very simply typedefs (typedefs of ints and floats) first,