headers
Vitja Makarov | 1 Dec 2010 06:10
Picon
Gravatar

Re: [Cython] splitting up Python functions

2010/12/1 Robert Kern <robert.kern@...>:
> On 11/30/10 4:01 PM, Robert Bradshaw wrote:
>> On Tue, Nov 30, 2010 at 1:57 PM, Robert Kern<robert.kern@...>  wrote:
>>> On 11/30/10 3:31 PM, Robert Bradshaw wrote:
>>>> On Tue, Nov 30, 2010 at 1:04 PM, Vitja
Makarov<vitja.makarov@...>    wrote:
>>>>
>>>>> I don't like the idea to add one more external dependency. But if that
>>>>> really helps its okay.
>>>>
>>>> It's not an external dependency if it ships with Python. Actually,
>>>> ctypes didn't ship with Python until 2.5, so that may be an issue. I'm
>>>> thinking we'd only use it where we couldn't do it otherwise.
>>>
>>> ctypes/libffi is deliberately considered an optional component of Python for
>>> several reasons, one of which is the lack of support on certain platforms. I
>>> would say that it exists in a somewhat unique middle ground between "ships with
>>> Python" and "external dependency".
>>
>> Thanks for the perspective. Are there any good alternatives
>> (especially that would be less of an "external" dependency)?
>
> I'm afraid not.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
>  that is made terrible by our own mad attempt to interpret it as though it had
>  an underlying truth."
(Continue reading)

Permalink | Reply | 1 comment |
headers
Robert Bradshaw | 1 Dec 2010 18:38
Favicon

Re: [Cython] splitting up Python functions

On Tue, Nov 30, 2010 at 9:10 PM, Vitja Makarov
<vitja.makarov@...> wrote:
> 2010/12/1 Robert Kern <robert.kern@...>:
>> On 11/30/10 4:01 PM, Robert Bradshaw wrote:
>>> On Tue, Nov 30, 2010 at 1:57 PM, Robert
Kern<robert.kern@...>  wrote:
>>>> On 11/30/10 3:31 PM, Robert Bradshaw wrote:
>>>>> On Tue, Nov 30, 2010 at 1:04 PM, Vitja Makarov<vitja.makarov@...m>    wrote:
>>>>>
>>>>>> I don't like the idea to add one more external dependency. But if that
>>>>>> really helps its okay.
>>>>>
>>>>> It's not an external dependency if it ships with Python. Actually,
>>>>> ctypes didn't ship with Python until 2.5, so that may be an issue. I'm
>>>>> thinking we'd only use it where we couldn't do it otherwise.
>>>>
>>>> ctypes/libffi is deliberately considered an optional component of Python for
>>>> several reasons, one of which is the lack of support on certain platforms. I
>>>> would say that it exists in a somewhat unique middle ground between "ships with
>>>> Python" and "external dependency".
>>>
>>> Thanks for the perspective. Are there any good alternatives
>>> (especially that would be less of an "external" dependency)?
>>
>> I'm afraid not.
>>
>> --
>> Robert Kern
>>
>> "I have come to believe that the whole world is an enigma, a harmless enigma
(Continue reading)

Permalink | Reply | 0 comments |
headers
Dag Sverre Seljebotn | 2 Dec 2010 10:08
Picon
Picon

Re: [Cython] [Numpy-discussion] A Cython apply_along_axis function

On 12/02/2010 08:17 AM, Robert Bradshaw wrote:
> On Wed, Dec 1, 2010 at 6:09 PM, John Salvatier
> <jsalvati <at> u.washington.edu>  wrote:
>    
>> On Wed, Dec 1, 2010 at 6:07 PM, Keith Goodman<kwgoodman@...>  wrote:
>>      
>>> On Wed, Dec 1, 2010 at 5:53 PM, David<david@...>  wrote:
>>>
>>>        
>>>> On 12/02/2010 04:47 AM, Keith Goodman wrote:
>>>>          
>>>>> It's hard to write Cython code that can handle all dtypes and
>>>>> arbitrary number of dimensions. The former is typically dealt with
>>>>> using templates, but what do people do about the latter?
>>>>>            
>>>> The only way that I know to do that systematically is iterator. There is
>>>> a relatively simple example in scipy/signal (lfilter.c.src).
>>>>
>>>> I wonder if it would be possible to add better support for numpy
>>>> iterators in cython...
>>>>          
>>> Thanks for the tip. I'm starting to think that for now I should just
>>> template both dtype and ndim.
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion@...
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>        
>> I enthusiastically support better iterator support for cython
>>
(Continue reading)

Permalink | Reply | 5 comments |
headers
Robert Kern | 2 Dec 2010 17:00
Picon
Gravatar

Re: [Cython] [Numpy-discussion] A Cython apply_along_axis function

On 12/2/10 3:08 AM, Dag Sverre Seljebotn wrote:

> Just in case anybody is wondering what something like this could look
> like, here's a rough scetch complete with bugs. The idea would be to a)
> add some rudimentary support for using the yield keyword in Cython to
> make a generator function, b) inline the generator function if the
> generator is used directly in a for-loop. This should result in very
> efficient code, and would also be much easier to implement than a
> general purpose generator.

I expect that being able to inline a generator would be quite a trick. I would 
go so far as to say that it is a completely different feature from adding 
generators to Cython in general. It's tantamount to adding (hygienic?) macros to 
Cython.

I like it.

>  <at> cython.inline
> cdef array_iter_double(np.ndarray a, int axis=-1):
>       cdef np.flatiter it
>       ita = np.PyArray_IterAllButAxis(a,&axis)
>       cdef Py_ssize_t stride = a.strides[axis], length = a.shape[axis], i
>       while np.PyArray_ITER_NOTDONE(ita):
>           for i in range(length):
>               yield<double*>(np.PyArray_ITER_DATA(it) + )[i * stride])[0]
>               # TODO: Probably yield indices as well
>           np.PyArray_ITER_NEXT(it)
>       # TODO: add faster special-cases for stride == sizeof(double)
>
>
(Continue reading)

Permalink | Reply | 4 comments |
headers
Stefan Behnel | 2 Dec 2010 17:44
Picon
Favicon

Re: [Cython] [Numpy-discussion] A Cython apply_along_axis function

Robert Kern, 02.12.2010 17:00:
> On 12/2/10 3:08 AM, Dag Sverre Seljebotn wrote:
>
>> Just in case anybody is wondering what something like this could look
>> like, here's a rough scetch complete with bugs. The idea would be to a)
>> add some rudimentary support for using the yield keyword in Cython to
>> make a generator function, b) inline the generator function if the
>> generator is used directly in a for-loop.

I know that we discussed these things ages ago, but I can't find the link ATM.

>> This should result in very
>> efficient code, and would also be much easier to implement than a
>> general purpose generator.
>
> I expect that being able to inline a generator would be quite a trick. I would
> go so far as to say that it is a completely different feature from adding
> generators to Cython in general. It's tantamount to adding (hygienic?) macros to
> Cython.
>
> I like it.

Note that we already have inlined generator expressions, i.e. things like

    sum(2*x for x in seq)

get unfolded into a straight sum loop. Same for any, all and I forget what 
others. And they already use the yield keyword for that purpose, so much of 
what is needed is already there.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Stefan Behnel | 3 Dec 2010 08:41
Picon
Favicon

Re: [Cython] Redefinition of Python functions (ticket #489)

Vitja Makarov, 29.11.2010 21:18:
> Attached patch tries to solve the issue.

I added comments on the bug tracker.

Stefan
Permalink | Reply | 10 comments |
headers
Vitja Makarov | 3 Dec 2010 08:56
Picon
Gravatar

Re: [Cython] Redefinition of Python functions (ticket #489)

2010/12/3 Stefan Behnel <stefan_ml@...>:
> Vitja Makarov, 29.11.2010 21:18:
>> Attached patch tries to solve the issue.
>
> I added comments on the bug tracker.
>
> Stefan

Okay, I'll take a look. What's about #605?

--

-- 
vitja.
Permalink | Reply | 9 comments |
headers
Stefan Behnel | 3 Dec 2010 09:21
Picon
Favicon

Re: [Cython] Redefinition of Python functions (ticket #489)

Vitja Makarov, 03.12.2010 08:56:
> What's about #605?

Comment posted there.

Stefan
Permalink | Reply | 8 comments |
headers
Sturla Molden | 3 Dec 2010 13:56
Picon
Gravatar

Re: [Cython] [Numpy-discussion] A Cython apply_along_axis function

> On 12/2/10 3:08 AM, Dag Sverre Seljebotn wrote:

> I expect that being able to inline a generator would be quite a trick.

Except that applying a function along an axis is the kind of thing onw
might want to workshare/multithread in a "nogil" block. Using a Python
object like an iterator here kind of takes that advantage away, i.e. one
has to hold the GIL to access the iterator. In additon to make C level
threads compete for the GIL, it also prevents OpenMP's scheduler from
balancing the work load. That is why I rather collect a temporary array of
pointers instead.

Sturla
Permalink | Reply | 2 comments |
headers
Vitja Makarov | 3 Dec 2010 14:27
Picon
Gravatar

[Cython] Lamba support in class scope and cclass scope, #605

Stefan Behnel (from trac):
> Why not have a method generate_lambda_definitions() somewhere up the AST hierarchy? Maybe in
BlockNode? You could just pass in the relevant scope explicitly.

I've updated patch. Now all lambdas are defined in
BlockNode.generate_lambda_definitions()
This patch (I tried with function-redefine applied) gives about ~+300
pyregr tests ;)

--

-- 
vitja.
Permalink | Reply | 5 comments |

Gmane