headers
Ask F. Jakobsen | 1 May 2012 09:53
Picon

[Cython] Code generated for the expression int(x)+1

Hi all,

I am having a simple performance problem that can be resolved by splitting up an expression in two lines. I
don't know if it is a bug or I am missing something.

The piece of code below is translated to slow code

1)
    cdef int i
    i=int(x)+1

whereas the code below is translated to fast code

2)
    cdef int i
    i=int(x)
    i=i+1

Snippet of generated code by cython

1)

  /* "test.pyx":4
 *     cdef double x=3.2
 *     cdef int i
 *     i=int(x)+1             # <<<<<<<<<<<<<<
 *     return i
 * 
 */
  __pyx_t_1 = PyFloat_FromDouble(__pyx_v_x); if (unlikely(!__pyx_t_1)) {__pyx_filename =
(Continue reading)

Permalink | Reply | 1 comment |
headers
Stefan Behnel | 1 May 2012 10:28
Picon
Favicon

Re: [Cython] Code generated for the expression int(x)+1

Ask F. Jakobsen, 01.05.2012 09:53:
> I am having a simple performance problem that can be resolved by splitting up an expression in two lines. I
don't know if it is a bug or I am missing something.
> 
> The piece of code below is translated to slow code
> 
> 1)
>     cdef int i
>     i=int(x)+1

What you are saying here is:

Convert x (known to be a C double) to an arbitrary size Python integer
value, add 1, convert it to a C int and assign it to i.

> whereas the code below is translated to fast code
> 
> 2)
>     cdef int i
>     i=int(x)
>     i=i+1

This means:

Convert x (known to be a C double) to an arbitrary size Python integer
value, convert that to a C int and assign it to i, then add 1 and assign
the result to i.

In the first case, Cython cannot safely assume that the result of the int()
conversion will fit into a C int and will therefore evaluate the expression
(Continue reading)

Permalink | Reply | 0 comments |
headers
Dag Sverre Seljebotn | 1 May 2012 10:29
Picon
Picon
Gravatar

Re: [Cython] Wacky idea: proper macros

On 04/30/2012 11:36 PM, William Stein wrote:
> On Mon, Apr 30, 2012 at 2:32 PM, Dag Sverre Seljebotn
> <d.s.seljebotn@...>  wrote:
>>
>>
>> Wes McKinney<wesmckinn@...>  wrote:
>>
>>> On Mon, Apr 30, 2012 at 4:55 PM, Nathaniel Smith<njs@...>  wrote:
>>>> On Mon, Apr 30, 2012 at 9:49 PM, Dag Sverre Seljebotn
>>>> <d.s.seljebotn@...>  wrote:
>>>>> JIT is really the way to go. It is one thing that a JIT could
>>> optimize the
>>>>> case where you pass a callback to a function and inline it run-time.
>>> But
>>>>> even if it doesn't get that fancy, it'd be great to just be able to
>>> write
>>>>> something like "cython.eval(s)" and have that be compiled (I guess
>>> you could
>>>>> do that now, but the sheer overhead of the C compiler and all the
>>> .so files
>>>>> involved means nobody would sanely use that as the main way of
>>> stringing
>>>>> together something like pandas).
>>>>
>>>> The overhead of running a fully optimizing compiler over pandas on
>>>> every import is pretty high, though. You can come up with various
>>>> caching mechanisms, but they all mean introducing some kind of
>>> compile
>>>> time/run time distinction. So I'm skeptical we'll just be able to get
>>>> rid of that concept, even in a brave new LLVM/PyPy/Julia world.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Dag Sverre Seljebotn | 1 May 2012 10:32
Picon
Picon
Gravatar

Re: [Cython] Wacky idea: proper macros

On 05/01/2012 10:29 AM, Dag Sverre Seljebotn wrote:
> On 04/30/2012 11:36 PM, William Stein wrote:
>> On Mon, Apr 30, 2012 at 2:32 PM, Dag Sverre Seljebotn
>> <d.s.seljebotn@...> wrote:
>>>
>>>
>>> Wes McKinney<wesmckinn@...> wrote:
>>>
>>>> On Mon, Apr 30, 2012 at 4:55 PM, Nathaniel Smith<njs@...> wrote:
>>>>> On Mon, Apr 30, 2012 at 9:49 PM, Dag Sverre Seljebotn
>>>>> <d.s.seljebotn@...> wrote:
>>>>>> JIT is really the way to go. It is one thing that a JIT could
>>>> optimize the
>>>>>> case where you pass a callback to a function and inline it run-time.
>>>> But
>>>>>> even if it doesn't get that fancy, it'd be great to just be able to
>>>> write
>>>>>> something like "cython.eval(s)" and have that be compiled (I guess
>>>> you could
>>>>>> do that now, but the sheer overhead of the C compiler and all the
>>>> .so files
>>>>>> involved means nobody would sanely use that as the main way of
>>>> stringing
>>>>>> together something like pandas).
>>>>>
>>>>> The overhead of running a fully optimizing compiler over pandas on
>>>>> every import is pretty high, though. You can come up with various
>>>>> caching mechanisms, but they all mean introducing some kind of
>>>> compile
>>>>> time/run time distinction. So I'm skeptical we'll just be able to get
(Continue reading)

Permalink | Reply | 0 comments |
headers
Stefan Behnel | 1 May 2012 11:21
Picon
Favicon

Re: [Cython] [cython-users] Conditional import in pure Python mode

>>> On 29 April 2012 01:33, Ian Bell wrote:
>>>> idiom like
>>>>
>>>> if cython.compiled:
>>>>     cython.import('from libc.math cimport sin')
>>>> else:
>>>>     from math import sin

Actually, in this particular case, I would even accept a solution that
special cases the "math" module internally by automatically cimporting
libc.math as an override (or rather an adapted version as plain "math.pxd").

This CEP describes a general approach:

http://wiki.cython.org/enhancements/overlaypythonmodules

It's partly outdated, so things may have become easier these days.

Stefan
Permalink | Reply | 0 comments |
headers
Stefan Behnel | 1 May 2012 21:14
Picon
Favicon

Re: [Cython] [cython-users] Conditional import in pure Python mode

Ian Bell, 01.05.2012 15:50:
> On Tue, May 1, 2012 at 9:21 PM, Stefan Behnel wrote:
>>>>> On 29 April 2012 01:33, Ian Bell wrote:
>>>>>> idiom like
>>>>>>
>>>>>> if cython.compiled:
>>>>>>     cython.import('from libc.math cimport sin')
>>>>>> else:
>>>>>>     from math import sin
>>
>> Actually, in this particular case, I would even accept a solution that
>> special cases the "math" module internally by automatically cimporting
>> libc.math as an override (or rather an adapted version as plain
>> "math.pxd").
>>
>> This CEP describes a general approach:
>>
>> http://wiki.cython.org/enhancements/overlaypythonmodules
>>
>> It's partly outdated, so things may have become easier these days.
> 
> That is exactly what I was looking for.  If we could implement that, it
> would solve all my problems.  It would meet all my needs - on this front at
> least.

There are two things to do here:

1) Write up a math.pxd that contains declarations equivalent to Python's
math module. Note that this may not be entirely trivial because the math
module does some error handling and type special casing under the hood.
(Continue reading)

Permalink | Reply | 9 comments |
headers
Stefan Behnel | 1 May 2012 21:22
Picon
Favicon

Re: [Cython] Conditional import in pure Python mode

Stefan Behnel, 01.05.2012 21:14:
> 2) Use math.pxd as an override for the math module. I'm not sure yet how
> that would best be made to work, but it shouldn't be all that complex. It
> already works (mostly?) for numpy.pxd, for example, although that's done
> explicitly in user code.

BTW, I think it would be helpful to make the numpy.pxd cimport automatic as
well whenever someone does "import numpy" and friends, right?

Stefan
Permalink | Reply | 7 comments |
headers
mark florisson | 1 May 2012 21:39
Picon
Gravatar

Re: [Cython] Conditional import in pure Python mode

On 1 May 2012 20:22, Stefan Behnel <stefan_ml@...> wrote:
> Stefan Behnel, 01.05.2012 21:14:
>> 2) Use math.pxd as an override for the math module. I'm not sure yet how
>> that would best be made to work, but it shouldn't be all that complex. It
>> already works (mostly?) for numpy.pxd, for example, although that's done
>> explicitly in user code.
>
> BTW, I think it would be helpful to make the numpy.pxd cimport automatic as
> well whenever someone does "import numpy" and friends, right?
>
> Stefan
> _______________________________________________
> cython-devel mailing list
> cython-devel@...
> http://mail.python.org/mailman/listinfo/cython-devel

I'm not sure, it means the user has to have numpy development headers.
Permalink | Reply | 6 comments |
headers
Francesc Alted | 1 May 2012 21:49

Re: [Cython] Conditional import in pure Python mode

On 5/1/12 2:39 PM, mark florisson wrote:
> On 1 May 2012 20:22, Stefan Behnel<stefan_ml@...>  wrote:
>> Stefan Behnel, 01.05.2012 21:14:
>>> 2) Use math.pxd as an override for the math module. I'm not sure yet how
>>> that would best be made to work, but it shouldn't be all that complex. It
>>> already works (mostly?) for numpy.pxd, for example, although that's done
>>> explicitly in user code.
>> BTW, I think it would be helpful to make the numpy.pxd cimport automatic as
>> well whenever someone does "import numpy" and friends, right?
>>
>> Stefan
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel@...
>> http://mail.python.org/mailman/listinfo/cython-devel
> I'm not sure, it means the user has to have numpy development headers.

But if the user is going to compile a NumPy application, it sounds like 
strange to me that she should not be required to install the NumPy 
development headers, right?

--

-- 
Francesc Alted
Permalink | Reply | 4 comments |
headers
Stefan Behnel | 1 May 2012 21:51
Picon
Favicon

Re: [Cython] Conditional import in pure Python mode

mark florisson, 01.05.2012 21:39:
> On 1 May 2012 20:22, Stefan Behnel wrote:
>> Stefan Behnel, 01.05.2012 21:14:
>>> 2) Use math.pxd as an override for the math module. I'm not sure yet how
>>> that would best be made to work, but it shouldn't be all that complex. It
>>> already works (mostly?) for numpy.pxd, for example, although that's done
>>> explicitly in user code.
>>
>> BTW, I think it would be helpful to make the numpy.pxd cimport automatic as
>> well whenever someone does "import numpy" and friends, right?
> 
> I'm not sure, it means the user has to have numpy development headers.

Hmm, right. What about making it an explicit compile time option then?
Something like

  # cython: override_modules = math,numpy

Or should we go for an opt-out?

  # cython: python_modules = math,numpy

Sounds like it would hit the more common case by default.

Stefan
Permalink | Reply | 0 comments |

Gmane