John Salvatier | 1 Dec 06:40 2010
Picon

How can I use PyArray_IterAllButAxis to iterate along an axis

I am trying to make a ufunc like function which takes an axis
argument, applying along an axis, using PyArray_IterAllButAxis (http://
docs.scipy.org/doc/numpy/reference/c-
api.array.html#PyArray_IterAllButAxis). However, I haven't been able
to figure out how I should step along that axis on each iteration. I
think I should be able to do something like the following, but I am
not sure if it's correct to index into the ITER_DATA like that and/or
what stride I should be using. This compiles and runs without error
but gives me junk for most of the output array values.

def ewma(a, d,  int axis = -1):

	out = np.empty(a.shape, dtype)

	cdef np.flatiter ita, ito

	ita = np.PyArray_IterAllButAxis(a,   &axis)
	ito = np.PyArray_IterAllButAxis(out, &axis)

	cdef int i
	cdef int axis_length = a.shape[axis]
	cdef int L = a.strides[axis]

	cdef double avg = 0.0
	cdef double weight = 1.0 - np.exp(-d)

	while np.PyArray_ITER_NOTDONE(ita):

		avg = 0.0
		for i in range(axis_length):
(Continue reading)

Dag Sverre Seljebotn | 1 Dec 08:01 2010
Picon
Picon

Re: How can I use PyArray_IterAllButAxis to iterate along an axis

John Salvatier wrote:
> I am trying to make a ufunc like function which takes an axis
> argument, applying along an axis, using PyArray_IterAllButAxis (http://
> docs.scipy.org/doc/numpy/reference/c-
> api.array.html#PyArray_IterAllButAxis). However, I haven't been able
> to figure out how I should step along that axis on each iteration. I
> think I should be able to do something like the following, but I am
> not sure if it's correct to index into the ITER_DATA like that and/or
> what stride I should be using. This compiles and runs without error
> but gives me junk for most of the output array values.
>   
As a note to future posters, I think questions like this one is better 
asked on the numpy-discussion mailing list. Most people there know 
Cython anyway, and the difference to C is very small in this case.

Having said that,  you really should get hold of a *different* stride 
for the out-array. If you sliced the input array along the way, this is 
likely your problem.

Dag Sverre

>
> def ewma(a, d,  int axis = -1):
>
> 	out = np.empty(a.shape, dtype)
>
> 	cdef np.flatiter ita, ito
>
> 	ita = np.PyArray_IterAllButAxis(a,   &axis)
> 	ito = np.PyArray_IterAllButAxis(out, &axis)
(Continue reading)

Dag Sverre Seljebotn | 1 Dec 08:03 2010
Picon
Picon

Re: How can I use PyArray_IterAllButAxis to iterate along an axis

Dag Sverre Seljebotn wrote:
> John Salvatier wrote:
>> I am trying to make a ufunc like function which takes an axis
>> argument, applying along an axis, using PyArray_IterAllButAxis (http://
>> docs.scipy.org/doc/numpy/reference/c-
>> api.array.html#PyArray_IterAllButAxis). However, I haven't been able
>> to figure out how I should step along that axis on each iteration. I
>> think I should be able to do something like the following, but I am
>> not sure if it's correct to index into the ITER_DATA like that and/or
>> what stride I should be using. This compiles and runs without error
>> but gives me junk for most of the output array values.
>>   
> As a note to future posters, I think questions like this one is better 
> asked on the numpy-discussion mailing list. Most people there know 
> Cython anyway, and the difference to C is very small in this case.

BTW, feel free to raise your opinion against this. There's a lot of 
people with NumPy knowledge reading this list too, and a case could be 
made either way.

Dag Sverre

>
> Having said that,  you really should get hold of a *different* stride 
> for the out-array. If you sliced the input array along the way, this 
> is likely your problem.
>
> Dag Sverre
>
>
(Continue reading)

Stefan Behnel | 1 Dec 09:04 2010
Picon

Re: Re: how to test for cython version

Ondrej Certik, 30.11.2010 20:22:
>>> On 11/30/2010 07:59 PM, Ondrej Certik wrote:
>>>> I understand, that trying to "compile some simple codes" to determine,
>>>> if the user's Cython is new enough, is not optimal, but all other
>>>> ways, that I can see, will not catch this at the "configure time", and
>>>> simply fail during the build, like for the above user.
>>>
>
> I chose to use the latest cython git, because it had some things fixed
> (in particular some "bool" issue in this case I believe, maybe it was
> something else).
>
> Well, in principle, you are right, that I could work around that, but
> I chose to simply use the latest cython, where things are fixed. But
> then we run into these problems, that users can't compile our code.
>
> I chose to go the "try to compile" route. That way, in all the cases
> that we encountered so far, it would tell the user right away,
> "install latest cython". Yes, maybe even some newer cython would do a
> better job, but my only concern is that people can compile our code.
>
> Otherwise, yes, we also bundle the right Cython in femhub (as well as
> all the other dependencies). For some people it's not an option
> though.

Why would your users need Cython at all? Aren't the generated C sources 
enough? It's common (and I recommend) to ship a tested version of those and 
to let users build without using Cython at all.

Stefan
(Continue reading)

Ondrej Certik | 1 Dec 09:36 2010
Picon

Re: Re: how to test for cython version

On Wed, Dec 1, 2010 at 12:04 AM, Stefan Behnel <stefan_ml <at> behnel.de> wrote:
> Ondrej Certik, 30.11.2010 20:22:
>>>>
>>>> On 11/30/2010 07:59 PM, Ondrej Certik wrote:
>>>>>
>>>>> I understand, that trying to "compile some simple codes" to determine,
>>>>> if the user's Cython is new enough, is not optimal, but all other
>>>>> ways, that I can see, will not catch this at the "configure time", and
>>>>> simply fail during the build, like for the above user.
>>>>
>>
>> I chose to use the latest cython git, because it had some things fixed
>> (in particular some "bool" issue in this case I believe, maybe it was
>> something else).
>>
>> Well, in principle, you are right, that I could work around that, but
>> I chose to simply use the latest cython, where things are fixed. But
>> then we run into these problems, that users can't compile our code.
>>
>> I chose to go the "try to compile" route. That way, in all the cases
>> that we encountered so far, it would tell the user right away,
>> "install latest cython". Yes, maybe even some newer cython would do a
>> better job, but my only concern is that people can compile our code.
>>
>> Otherwise, yes, we also bundle the right Cython in femhub (as well as
>> all the other dependencies). For some people it's not an option
>> though.
>
> Why would your users need Cython at all? Aren't the generated C sources
> enough? It's common (and I recommend) to ship a tested version of those and
(Continue reading)

Ondrej Certik | 1 Dec 09:38 2010
Picon

Re: Re: how to test for cython version

On Wed, Dec 1, 2010 at 12:36 AM, Ondrej Certik <ondrej <at> certik.cz> wrote:
> On Wed, Dec 1, 2010 at 12:04 AM, Stefan Behnel <stefan_ml <at> behnel.de> wrote:
>> Ondrej Certik, 30.11.2010 20:22:
>>>>>
>>>>> On 11/30/2010 07:59 PM, Ondrej Certik wrote:
>>>>>>
>>>>>> I understand, that trying to "compile some simple codes" to determine,
>>>>>> if the user's Cython is new enough, is not optimal, but all other
>>>>>> ways, that I can see, will not catch this at the "configure time", and
>>>>>> simply fail during the build, like for the above user.
>>>>>
>>>
>>> I chose to use the latest cython git, because it had some things fixed
>>> (in particular some "bool" issue in this case I believe, maybe it was
>>> something else).
>>>
>>> Well, in principle, you are right, that I could work around that, but
>>> I chose to simply use the latest cython, where things are fixed. But
>>> then we run into these problems, that users can't compile our code.
>>>
>>> I chose to go the "try to compile" route. That way, in all the cases
>>> that we encountered so far, it would tell the user right away,
>>> "install latest cython". Yes, maybe even some newer cython would do a
>>> better job, but my only concern is that people can compile our code.
>>>
>>> Otherwise, yes, we also bundle the right Cython in femhub (as well as
>>> all the other dependencies). For some people it's not an option
>>> though.
>>
>> Why would your users need Cython at all? Aren't the generated C sources
(Continue reading)

Dag Sverre Seljebotn | 1 Dec 09:40 2010
Picon
Picon

Re: Re: how to test for cython version

On 12/01/2010 09:36 AM, Ondrej Certik wrote:
> On Wed, Dec 1, 2010 at 12:04 AM, Stefan Behnel<stefan_ml <at> behnel.de>  wrote:
>    
>> Ondrej Certik, 30.11.2010 20:22:
>>      
>>>>> On 11/30/2010 07:59 PM, Ondrej Certik wrote:
>>>>>            
>>>>>> I understand, that trying to "compile some simple codes" to determine,
>>>>>> if the user's Cython is new enough, is not optimal, but all other
>>>>>> ways, that I can see, will not catch this at the "configure time", and
>>>>>> simply fail during the build, like for the above user.
>>>>>>              
>>>>>            
>>> I chose to use the latest cython git, because it had some things fixed
>>> (in particular some "bool" issue in this case I believe, maybe it was
>>> something else).
>>>
>>> Well, in principle, you are right, that I could work around that, but
>>> I chose to simply use the latest cython, where things are fixed. But
>>> then we run into these problems, that users can't compile our code.
>>>
>>> I chose to go the "try to compile" route. That way, in all the cases
>>> that we encountered so far, it would tell the user right away,
>>> "install latest cython". Yes, maybe even some newer cython would do a
>>> better job, but my only concern is that people can compile our code.
>>>
>>> Otherwise, yes, we also bundle the right Cython in femhub (as well as
>>> all the other dependencies). For some people it's not an option
>>> though.
>>>        
(Continue reading)

Dag Sverre Seljebotn | 1 Dec 09:45 2010
Picon
Picon

Re: Re: how to test for cython version

On 12/01/2010 09:38 AM, Ondrej Certik wrote:
> On Wed, Dec 1, 2010 at 12:36 AM, Ondrej Certik<ondrej <at> certik.cz>  wrote:
>    
>> On Wed, Dec 1, 2010 at 12:04 AM, Stefan Behnel<stefan_ml <at> behnel.de>  wrote:
>>      
>>> Ondrej Certik, 30.11.2010 20:22:
>>>        
>>>>>> On 11/30/2010 07:59 PM, Ondrej Certik wrote:
>>>>>>              
>>>>>>> I understand, that trying to "compile some simple codes" to determine,
>>>>>>> if the user's Cython is new enough, is not optimal, but all other
>>>>>>> ways, that I can see, will not catch this at the "configure time", and
>>>>>>> simply fail during the build, like for the above user.
>>>>>>>                
>>>>>>              
>>>> I chose to use the latest cython git, because it had some things fixed
>>>> (in particular some "bool" issue in this case I believe, maybe it was
>>>> something else).
>>>>
>>>> Well, in principle, you are right, that I could work around that, but
>>>> I chose to simply use the latest cython, where things are fixed. But
>>>> then we run into these problems, that users can't compile our code.
>>>>
>>>> I chose to go the "try to compile" route. That way, in all the cases
>>>> that we encountered so far, it would tell the user right away,
>>>> "install latest cython". Yes, maybe even some newer cython would do a
>>>> better job, but my only concern is that people can compile our code.
>>>>
>>>> Otherwise, yes, we also bundle the right Cython in femhub (as well as
>>>> all the other dependencies). For some people it's not an option
(Continue reading)

Ondrej Certik | 1 Dec 10:02 2010
Picon

Re: Re: how to test for cython version

On Wed, Dec 1, 2010 at 12:45 AM, Dag Sverre Seljebotn
<dagss <at> student.matnat.uio.no> wrote:
> On 12/01/2010 09:38 AM, Ondrej Certik wrote:
>>
>> On Wed, Dec 1, 2010 at 12:36 AM, Ondrej Certik<ondrej <at> certik.cz>  wrote:
>>
>>>
>>> On Wed, Dec 1, 2010 at 12:04 AM, Stefan Behnel<stefan_ml <at> behnel.de>
>>>  wrote:
>>>
>>>>
>>>> Ondrej Certik, 30.11.2010 20:22:
>>>>
>>>>>>>
>>>>>>> On 11/30/2010 07:59 PM, Ondrej Certik wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> I understand, that trying to "compile some simple codes" to
>>>>>>>> determine,
>>>>>>>> if the user's Cython is new enough, is not optimal, but all other
>>>>>>>> ways, that I can see, will not catch this at the "configure time",
>>>>>>>> and
>>>>>>>> simply fail during the build, like for the above user.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>> I chose to use the latest cython git, because it had some things fixed
>>>>> (in particular some "bool" issue in this case I believe, maybe it was
>>>>> something else).
(Continue reading)

Sebastien Binet | 1 Dec 13:06 2010
Picon

Re: Re: how to test for cython version

On Wed, 1 Dec 2010 00:38:02 -0800, Ondrej Certik <ondrej <at> certik.cz> wrote:
> Plus there is another problem, that I was not able to setup cmake to
> automatically pickup the changes in .pyx, but *not* to try to compile
> the .pyx files when you switch branches in git (which would fail,
> because users were having old cython installed). I believe that git
> touches some files when switching branches, just to be sure. I don't
> know a solution to this, and it's a show stopper.

fyi,

waf computes a md5 checksum of the file (rather than relying on the
timestamp)

waf-trunk has support for cython :)

cheers,
sebastien.

Gmane