Lisandro Dalcin | 1 Apr 03:11 2011
Picon

Re: [Cython] Cannot profile nogil function. Error or Warn?

On 29 March 2011 21:26, Lisandro Dalcin <dalcinl <at> gmail.com> wrote:
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
>
> cdef int PyMPE_Raise(int ierr) except -1 with gil:
>    __Pyx_Raise(RuntimeError, "MPE logging error [code: %d]" % ierr, NULL)
>    return 0
>
> cdef inline int CHKERR(int ierr) nogil except -1:
>    ^
> ------------------------------------------------------------
>
> /home/dalcinl/Devel/mpi4py-dev/src/MPE/helpers.pxi:22:5: Cannot
> profile nogil function.
>
>
> Do we REALLY want this to be an error? Why not just a warning?
>

OK, I pushed a fix. Without this, using -X profile=True cannot work
with any pyx source that has nogil functions. Enabling profiling
should not force users to change source code.

--

-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
(Continue reading)

Stefan Behnel | 1 Apr 10:11 2011
Picon

Re: [Cython] Interest in contributing to the project

Arthur de Souza Ribeiro, 29.03.2011 09:11:
> Hello everybody,
>
> My name is Arthur de Souza Ribeiro and I'm a fourth-year student of Computer
> Science in Federal University of Campina Grande, Brazil. I'm a python
> programmer and have knowledge of other languages too, like Java, C, C++, Qt,
> Grails and ActionScript (used in Flex framework of Adobe).
>
> I saw Cython project and got really interested in contributing to it. By the
> way, I saw that the project is trying to participate of GSoC under Python
> Software Foundation umbrella. I know the student application period have
> already started, but, I'd really enjoy to participate of GSoC 2011 as a
> Cython's student. Until day 8 I could work really hard to show you that I
> can be selected as a GSoC student for Cython. I looked for an Ideas Page of
> the project but didn't find it, Is there any idea that you have to submit a
> project in GSoC?
>
> If possible, please tell me things that I can start doing to help the
> project.

Hi Arthur,

sorry for the late response and thank you for your application. We are 
always happy about contributions.

The Cython project is currently running a workshop that may yield further 
possible GSoC tasks, but the one we already have identified is IMHO quite a 
nice and self-contained one. The goal is to rewrite modules in CPython's 
standard library in Cython that are currently written in C. The intention 
is a) to simplify the implementation to make it easier for CPython 
(Continue reading)

Lisandro Dalcin | 1 Apr 18:57 2011
Picon

[Cython] implementation of cdef functions with default arguments

Perhaps I'm missing something, but why we need the intermediate
struct? Why not generate a regular C function with all the args, and
then generate the call providing arguments? We could even extend this
to support kwargs for calling functions in Cython,

1 - The implementation would be cleaner, IMHO.
2 - These functions cannot be easily used in external C code (or
course, C code should provide all the args)
3 - We could define default args for "cdef extern" C functions, Cython
would provide the arg values on call.
4 - We could add support to pass values as kwargs (well, we could do
that with the current implementation).
5 - Faster code?

Comments?

--

-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
Robert Bradshaw | 1 Apr 20:50 2011

[Cython] Docs repository

The cython documentation is now part of the main repository, see
https://github.com/cython/cython/tree/master/docs .
https://github.com/cython/cython-docs should no longer be used
(perhaps it should be removed entirely, are there any/many inbound
links?)

- Robert
Arthur de Souza Ribeiro | 2 Apr 03:52 2011
Picon

Re: [Cython] Interest in contributing to the project

HI Stefan, thank you very much for responding my e-mail to cython's list.


About the proposal, I'd be very happy in helping the cython community doing the task 'rewrite modules in CPython's standard library in Cython that are currently written in C'. I didn't think about any special modules, but I'm going to start doing it, in my opinion, both modules you've mentioned are really good examples.

I think this project could be very important, but, I don't know CPython very well, are there any examples you could suggest me to understand CPython better? I think I could do a good effort to understand this as fast as I can and we discuss more the proposal.

Waiting for your reply...

Best Regards..

[]s

Arthur

2011/4/1 Stefan Behnel <stefan_ml <at> behnel.de>
Arthur de Souza Ribeiro, 29.03.2011 09:11:

Hello everybody,

My name is Arthur de Souza Ribeiro and I'm a fourth-year student of Computer
Science in Federal University of Campina Grande, Brazil. I'm a python
programmer and have knowledge of other languages too, like Java, C, C++, Qt,
Grails and ActionScript (used in Flex framework of Adobe).

I saw Cython project and got really interested in contributing to it. By the
way, I saw that the project is trying to participate of GSoC under Python
Software Foundation umbrella. I know the student application period have
already started, but, I'd really enjoy to participate of GSoC 2011 as a
Cython's student. Until day 8 I could work really hard to show you that I
can be selected as a GSoC student for Cython. I looked for an Ideas Page of
the project but didn't find it, Is there any idea that you have to submit a
project in GSoC?

If possible, please tell me things that I can start doing to help the
project.

Hi Arthur,

sorry for the late response and thank you for your application. We are always happy about contributions.

The Cython project is currently running a workshop that may yield further possible GSoC tasks, but the one we already have identified is IMHO quite a nice and self-contained one. The goal is to rewrite modules in CPython's standard library in Cython that are currently written in C. The intention is a) to simplify the implementation to make it easier for CPython developers to maintain their code base and b) to try to make the modules even faster than they are to show off Cython's optimisation capabilities (in that order, I think).

A related task could be to take existing Python modules in the stdlib, to profile them, and to add external type annotations to optimise them when being compiled with Cython.

Both the task of showing Cython's ability to efficiently (and compatibly) implement or compile parts of the stdlib, and the resulting testing of Cython (and bug reporting/fixing) against real world Python code would be very valuable to our project.

If you're interested, you could start by writing a short proposal including the modules that you would like to rewrite and what makes them interesting. Both "itertools" and "math" are certainly hot candidates, but there are definitely others, and your interest may change the priorities.

If you think that's not a good project for you, please bug us again, we may be able to come up with other projects as well.

Stefan
_______________________________________________
cython-devel mailing list
cython-devel <at> python.org
http://mail.python.org/mailman/listinfo/cython-devel

<div>
<p>HI Stefan, thank you very much for responding my e-mail to cython's list.</p>
<div><br></div>
<div>About the proposal, I'd be very happy in helping the cython community doing the task 'r<span>ewrite modules in CPython's standard library in Cython that are currently written in C'. I didn't think about any special modules, but I'm going to start doing it, in my opinion, both modules you've mentioned are really good examples.</span>
</div>

<div><br></div>
<div>I think this project could be very important, but, I don't know CPython very well, are there any examples you could suggest me to understand CPython better? I think I could do a good effort to understand this as fast as I can and we discuss more the proposal.</div>

<div><br></div>
<div>Waiting for your reply...</div>
<div><br></div>
<div>Best Regards..</div>
<div><br></div>
<div>[]s</div>
<div><br></div>
<div>Arthur<br><br><div class="gmail_quote">2011/4/1 Stefan Behnel <span dir="ltr">&lt;<a href="mailto:stefan_ml@..." target="_blank">stefan_ml <at> behnel.de</a>&gt;</span><br><blockquote class="gmail_quote">
Arthur de Souza Ribeiro, 29.03.2011 09:11:<div>
<div></div>
<div>
<br><blockquote class="gmail_quote">
Hello everybody,<br><br>
My name is Arthur de Souza Ribeiro and I'm a fourth-year student of Computer<br>
Science in Federal University of Campina Grande, Brazil. I'm a python<br>
programmer and have knowledge of other languages too, like Java, C, C++, Qt,<br>
Grails and ActionScript (used in Flex framework of Adobe).<br><br>
I saw Cython project and got really interested in contributing to it. By the<br>
way, I saw that the project is trying to participate of GSoC under Python<br>
Software Foundation umbrella. I know the student application period have<br>
already started, but, I'd really enjoy to participate of GSoC 2011 as a<br>
Cython's student. Until day 8 I could work really hard to show you that I<br>
can be selected as a GSoC student for Cython. I looked for an Ideas Page of<br>
the project but didn't find it, Is there any idea that you have to submit a<br>
project in GSoC?<br><br>
If possible, please tell me things that I can start doing to help the<br>
project.<br>
</blockquote>
<br>
</div>
</div>
Hi Arthur,<br><br>
sorry for the late response and thank you for your application. We are always happy about contributions.<br><br>
The Cython project is currently running a workshop that may yield further possible GSoC tasks, but the one we already have identified is IMHO quite a nice and self-contained one. The goal is to rewrite modules in CPython's standard library in Cython that are currently written in C. The intention is a) to simplify the implementation to make it easier for CPython developers to maintain their code base and b) to try to make the modules even faster than they are to show off Cython's optimisation capabilities (in that order, I think).<br><br>
A related task could be to take existing Python modules in the stdlib, to profile them, and to add external type annotations to optimise them when being compiled with Cython.<br><br>
Both the task of showing Cython's ability to efficiently (and compatibly) implement or compile parts of the stdlib, and the resulting testing of Cython (and bug reporting/fixing) against real world Python code would be very valuable to our project.<br><br>
If you're interested, you could start by writing a short proposal including the modules that you would like to rewrite and what makes them interesting. Both "itertools" and "math" are certainly hot candidates, but there are definitely others, and your interest may change the priorities.<br><br>
If you think that's not a good project for you, please bug us again, we may be able to come up with other projects as well.<br><br>
Stefan<br>
_______________________________________________<br>
cython-devel mailing list<br><a href="mailto:cython-devel@..." target="_blank">cython-devel <at> python.org</a><br><a href="http://mail.python.org/mailman/listinfo/cython-devel" target="_blank">http://mail.python.org/mailman/listinfo/cython-devel</a><br>
</blockquote>
</div>
<br>
</div>
</div>
Stefan Behnel | 2 Apr 08:50 2011
Picon

Re: [Cython] Interest in contributing to the project

Hi Arthur,

Arthur de Souza Ribeiro, 02.04.2011 03:52:
> HI Stefan, thank you very much for responding my e-mail to cython's list.
>
> About the proposal, I'd be very happy in helping the cython community doing
> the task 'rewrite modules in CPython's standard library in Cython that are
> currently written in C'. I didn't think about any special modules, but I'm
> going to start doing it, in my opinion, both modules you've mentioned are
> really good examples.

Cool.

> I think this project could be very important, but, I don't know CPython very
> well, are there any examples you could suggest me to understand CPython
> better? I think I could do a good effort to understand this as fast as I can
> and we discuss more the proposal.

The nice thing about this task is that you don't have to be an expert of 
CPython's C-API, nor a core developer of Cython. You will have to read the 
C code of the modules, and you will have to look up and understand what the 
C-API calls in the code are doing, but most of them have rather 
understandable names.

However, you will have to program efficiently in Cython, and write fast 
code in it. Writing Cython code that is easy to read and maintain, and at 
the same time fast enough to replace the existing manually tuned C code is 
the challenging bit here.

So my advice would be to get going in Cython programming (take a look 
through our tutorials), and to start reading the source code of a couple of 
CPython stdlib modules to get an idea of what you need to translate.

It would certainly help your application if you could reimplement one 
reasonably sized and self-contained function in a stdlib C module of your 
choice, and present that on the cython-users mailing list to get feedback. 
A couple of benchmark or profiling results comparing it to the original 
CPython function would round this up very nicely.

Stefan
Arthur de Souza Ribeiro | 3 Apr 04:17 2011
Picon

Re: [Cython] Interest in contributing to the project

Hi Stefan, well, i took a look at CPython's source code and as you said, if we use Cython in there we could get a very more readable code without losing performance (I suppose).


I took a look especially in cmathmodule.c that composes Python 3.2 source code (more recent stable version). As you said, depending on how fast I create the Cython code, we could add more modules (like socket one, for example).

An example on how code would be more readable (in my opinion, please correct me if I'm wrong) is that we wouldn't have to have configuration code about what functions would compose the module, for example, in math module we have: 

static PyMethodDef cmath_methods[] = {
    {"acos",   cmath_acos,  METH_VARARGS, c_acos_doc},
    {"acosh",  cmath_acosh, METH_VARARGS, c_acosh_doc},
    {"asin",   cmath_asin,  METH_VARARGS, c_asin_doc},
    {"asinh",  cmath_asinh, METH_VARARGS, c_asinh_doc},
    {"atan",   cmath_atan,  METH_VARARGS, c_atan_doc},
    {"atanh",  cmath_atanh, METH_VARARGS, c_atanh_doc},
    {"cos",    cmath_cos,   METH_VARARGS, c_cos_doc},
    {"cosh",   cmath_cosh,  METH_VARARGS, c_cosh_doc},
    {"exp",    cmath_exp,   METH_VARARGS, c_exp_doc},
    {"isfinite", cmath_isfinite, METH_VARARGS, cmath_isfinite_doc},
    {"isinf",  cmath_isinf, METH_VARARGS, cmath_isinf_doc},
    {"isnan",  cmath_isnan, METH_VARARGS, cmath_isnan_doc},
    {"log",    cmath_log,   METH_VARARGS, cmath_log_doc},
    {"log10",  cmath_log10, METH_VARARGS, c_log10_doc},
    {"phase",  cmath_phase, METH_VARARGS, cmath_phase_doc},
    {"polar",  cmath_polar, METH_VARARGS, cmath_polar_doc},
    {"rect",   cmath_rect,  METH_VARARGS, cmath_rect_doc},
    {"sin",    cmath_sin,   METH_VARARGS, c_sin_doc},
    {"sinh",   cmath_sinh,  METH_VARARGS, c_sinh_doc},
    {"sqrt",   cmath_sqrt,  METH_VARARGS, c_sqrt_doc},
    {"tan",    cmath_tan,   METH_VARARGS, c_tan_doc},
    {"tanh",   cmath_tanh,  METH_VARARGS, c_tanh_doc},
    {NULL,              NULL}           /* sentinel */
};


static struct PyModuleDef cmathmodule = {
    PyModuleDef_HEAD_INIT,
    "cmath",
    module_doc,
    -1,
    cmath_methods,
    NULL,
    NULL,
    NULL,
    NULL
};

And the init function after it, as I saw in cython (and implemented some examples), we would just have to implement the functions that the compilation would generate the object files that would be imported.

But, I noticed a problem that may likely appears that is the configuration part. I mean, Cython code is compiled differently than CPython's one right? If yes, would you have an idea on how we could work on this?

Another stuff that I'm getting in trouble in this initial part is how we would translate functions like PyArg_ParseTuple, any clue? I'm studing ways to replace too.

As you suggested, I'm practicing Cython to create functions and get more and more familiar with the language, so that I can create a very efficient cython code that would meet our expectations. I'm also reading CPython's code to see where cython can be applied.

Thank you.

Best Regards.

[]s

Arthur

2011/4/2 Stefan Behnel <stefan_ml-KjMAwuNBv5izQB+pC5nmwQ@public.gmane.org>
Hi Arthur,

Arthur de Souza Ribeiro, 02.04.2011 03:52:

HI Stefan, thank you very much for responding my e-mail to cython's list.

About the proposal, I'd be very happy in helping the cython community doing
the task 'rewrite modules in CPython's standard library in Cython that are
currently written in C'. I didn't think about any special modules, but I'm
going to start doing it, in my opinion, both modules you've mentioned are
really good examples.

Cool.



I think this project could be very important, but, I don't know CPython very
well, are there any examples you could suggest me to understand CPython
better? I think I could do a good effort to understand this as fast as I can
and we discuss more the proposal.

The nice thing about this task is that you don't have to be an expert of CPython's C-API, nor a core developer of Cython. You will have to read the C code of the modules, and you will have to look up and understand what the C-API calls in the code are doing, but most of them have rather understandable names.

However, you will have to program efficiently in Cython, and write fast code in it. Writing Cython code that is easy to read and maintain, and at the same time fast enough to replace the existing manually tuned C code is the challenging bit here.

So my advice would be to get going in Cython programming (take a look through our tutorials), and to start reading the source code of a couple of CPython stdlib modules to get an idea of what you need to translate.

It would certainly help your application if you could reimplement one reasonably sized and self-contained function in a stdlib C module of your choice, and present that on the cython-users mailing list to get feedback. A couple of benchmark or profiling results comparing it to the original CPython function would round this up very nicely.

Stefan

<div>
<p>Hi Stefan, well, i took a look at CPython's source code and as you said, if we use Cython in there we could get a very more readable code without losing performance (I suppose).</p>
<div><br></div>
<div>I took a look especially in cmathmodule.c that composes Python 3.2 source code (more recent stable version). As you said, depending on how fast I create the Cython code, we could add more modules (like socket one, for example).</div>

<div><br></div>
<div>An example on how code would be more readable (in my opinion, please correct me if I'm wrong) is that we wouldn't have to have configuration code about what functions would compose the module, for example, in math module we have:&nbsp;</div>

<div><br></div>
<div>
<div>static PyMethodDef cmath_methods[] = {</div>
<div>&nbsp;&nbsp; &nbsp;{"acos", &nbsp; cmath_acos, &nbsp;METH_VARARGS, c_acos_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"acosh", &nbsp;cmath_acosh, METH_VARARGS, c_acosh_doc},</div>
<div>
&nbsp;&nbsp; &nbsp;{"asin", &nbsp; cmath_asin, &nbsp;METH_VARARGS, c_asin_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"asinh", &nbsp;cmath_asinh, METH_VARARGS, c_asinh_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"atan", &nbsp; cmath_atan, &nbsp;METH_VARARGS, c_atan_doc},</div>

<div>&nbsp;&nbsp; &nbsp;{"atanh", &nbsp;cmath_atanh, METH_VARARGS, c_atanh_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"cos", &nbsp; &nbsp;cmath_cos, &nbsp; METH_VARARGS, c_cos_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"cosh", &nbsp; cmath_cosh, &nbsp;METH_VARARGS, c_cosh_doc},</div>

<div>&nbsp;&nbsp; &nbsp;{"exp", &nbsp; &nbsp;cmath_exp, &nbsp; METH_VARARGS, c_exp_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"isfinite", cmath_isfinite, METH_VARARGS, cmath_isfinite_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"isinf", &nbsp;cmath_isinf, METH_VARARGS, cmath_isinf_doc},</div>

<div>&nbsp;&nbsp; &nbsp;{"isnan", &nbsp;cmath_isnan, METH_VARARGS, cmath_isnan_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"log", &nbsp; &nbsp;cmath_log, &nbsp; METH_VARARGS, cmath_log_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"log10", &nbsp;cmath_log10, METH_VARARGS, c_log10_doc},</div>

<div>&nbsp;&nbsp; &nbsp;{"phase", &nbsp;cmath_phase, METH_VARARGS, cmath_phase_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"polar", &nbsp;cmath_polar, METH_VARARGS, cmath_polar_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"rect", &nbsp; cmath_rect, &nbsp;METH_VARARGS, cmath_rect_doc},</div>

<div>&nbsp;&nbsp; &nbsp;{"sin", &nbsp; &nbsp;cmath_sin, &nbsp; METH_VARARGS, c_sin_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"sinh", &nbsp; cmath_sinh, &nbsp;METH_VARARGS, c_sinh_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"sqrt", &nbsp; cmath_sqrt, &nbsp;METH_VARARGS, c_sqrt_doc},</div>

<div>&nbsp;&nbsp; &nbsp;{"tan", &nbsp; &nbsp;cmath_tan, &nbsp; METH_VARARGS, c_tan_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{"tanh", &nbsp; cmath_tanh, &nbsp;METH_VARARGS, c_tanh_doc},</div>
<div>&nbsp;&nbsp; &nbsp;{NULL, &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;NULL} &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; /* sentinel */</div>
<div>};</div>

<div><br></div>
<div><br></div>
<div>static struct PyModuleDef cmathmodule = {</div>
<div>&nbsp;&nbsp; &nbsp;PyModuleDef_HEAD_INIT,</div>
<div>&nbsp;&nbsp; &nbsp;"cmath",</div>
<div>&nbsp;&nbsp; &nbsp;module_doc,</div>
<div>&nbsp;&nbsp; &nbsp;-1,</div>
<div>&nbsp;&nbsp; &nbsp;cmath_methods,</div>

<div>&nbsp;&nbsp; &nbsp;NULL,</div>
<div>&nbsp;&nbsp; &nbsp;NULL,</div>
<div>&nbsp;&nbsp; &nbsp;NULL,</div>
<div>&nbsp;&nbsp; &nbsp;NULL</div>
<div>};</div>
</div>
<div><br></div>
<div>And the init function after it, as I saw in cython (and implemented some examples), we would just have to implement the functions that the compilation would generate the object files that would be imported.</div>

<div><br></div>
<div>But, I noticed a problem that may likely appears that is the configuration part. I mean, Cython code is compiled differently than CPython's one right? If yes, would you have an idea on how we could work on this?</div>

<div><br></div>
<div>Another stuff that I'm getting in trouble in this initial part is how we would translate functions like&nbsp;PyArg_ParseTuple, any clue? I'm studing ways to replace too.</div>
<div><br></div>
<div>As you suggested, I'm practicing Cython to create functions and get more and more familiar with the language, so that I can create a very efficient cython code that would meet our expectations. I'm also reading CPython's code to see where cython can be applied.</div>

<div><br></div>
<div>Thank you.</div>
<div><br></div>
<div>Best Regards.</div>
<div><br></div>
<div>[]s</div>
<div><br></div>
<div>Arthur</div>
<div>
<br><div class="gmail_quote">2011/4/2 Stefan Behnel <span dir="ltr">&lt;<a href="mailto:stefan_ml@...">stefan_ml@...</a>&gt;</span><br><blockquote class="gmail_quote">Hi Arthur,<br><br>
Arthur de Souza Ribeiro, 02.04.2011 03:52:<div class="im">
<br><blockquote class="gmail_quote">
HI Stefan, thank you very much for responding my e-mail to cython's list.<br><br>
About the proposal, I'd be very happy in helping the cython community doing<br>
the task 'rewrite modules in CPython's standard library in Cython that are<br>
currently written in C'. I didn't think about any special modules, but I'm<br>
going to start doing it, in my opinion, both modules you've mentioned are<br>
really good examples.<br>
</blockquote>
<br>
</div>
Cool.<div class="im">
<br><br><br><blockquote class="gmail_quote">
I think this project could be very important, but, I don't know CPython very<br>
well, are there any examples you could suggest me to understand CPython<br>
better? I think I could do a good effort to understand this as fast as I can<br>
and we discuss more the proposal.<br>
</blockquote>
<br>
</div>
The nice thing about this task is that you don't have to be an expert of CPython's C-API, nor a core developer of Cython. You will have to read the C code of the modules, and you will have to look up and understand what the C-API calls in the code are doing, but most of them have rather understandable names.<br><br>
However, you will have to program efficiently in Cython, and write fast code in it. Writing Cython code that is easy to read and maintain, and at the same time fast enough to replace the existing manually tuned C code is the challenging bit here.<br><br>
So my advice would be to get going in Cython programming (take a look through our tutorials), and to start reading the source code of a couple of CPython stdlib modules to get an idea of what you need to translate.<br><br>
It would certainly help your application if you could reimplement one reasonably sized and self-contained function in a stdlib C module of your choice, and present that on the cython-users mailing list to get feedback. A couple of benchmark or profiling results comparing it to the original CPython function would round this up very nicely.<br>
<br>
Stefan<br>
</blockquote>
</div>
<br>
</div>
</div>
Sturla Molden | 4 Apr 01:49 2011
Picon

Re: [Cython] Interest in contributing to the project

Den 03.04.2011 04:17, skrev Arthur de Souza Ribeiro:
>
> static PyMethodDef cmath_methods[] = {
>     {"acos",   cmath_acos,  METH_VARARGS, c_acos_doc},
>     {"acosh",  cmath_acosh, METH_VARARGS, c_acosh_doc},
>     {"asin",   cmath_asin,  METH_VARARGS, c_asin_doc},
>     {"asinh",  cmath_asinh, METH_VARARGS, c_asinh_doc},
>     {"atan",   cmath_atan,  METH_VARARGS, c_atan_doc},
>     {"atanh",  cmath_atanh, METH_VARARGS, c_atanh_doc},
>     {"cos",    cmath_cos,   METH_VARARGS, c_cos_doc},
>     {"cosh",   cmath_cosh,  METH_VARARGS, c_cosh_doc},
>     {"exp",    cmath_exp,   METH_VARARGS, c_exp_doc},
>     {"isfinite", cmath_isfinite, METH_VARARGS, cmath_isfinite_doc},
>     {"isinf",  cmath_isinf, METH_VARARGS, cmath_isinf_doc},
>     {"isnan",  cmath_isnan, METH_VARARGS, cmath_isnan_doc},
>     {"log",    cmath_log,   METH_VARARGS, cmath_log_doc},
>     {"log10",  cmath_log10, METH_VARARGS, c_log10_doc},
>     {"phase",  cmath_phase, METH_VARARGS, cmath_phase_doc},
>     {"polar",  cmath_polar, METH_VARARGS, cmath_polar_doc},
>     {"rect",   cmath_rect,  METH_VARARGS, cmath_rect_doc},
>     {"sin",    cmath_sin,   METH_VARARGS, c_sin_doc},
>     {"sinh",   cmath_sinh,  METH_VARARGS, c_sinh_doc},
>     {"sqrt",   cmath_sqrt,  METH_VARARGS, c_sqrt_doc},
>     {"tan",    cmath_tan,   METH_VARARGS, c_tan_doc},
>     {"tanh",   cmath_tanh,  METH_VARARGS, c_tanh_doc},
>     {NULL,              NULL}           /* sentinel */
> };
>
>
> static struct PyModuleDef cmathmodule = {
>     PyModuleDef_HEAD_INIT,
>     "cmath",
>     module_doc,
>     -1,
>     cmath_methods,
>     NULL,
>     NULL,
>     NULL,
>     NULL
> };
>

Cython will make this, do not care about it. You don't have to set up 
jump tables to make Python get the right function from Cython generated 
C code. If you have 22 Python-callable functions (i.e. declared def or 
cpdef), Cython will make a jump table for those as above.

> Another stuff that I'm getting in trouble in this initial part is how 
> we would translate functions like PyArg_ParseTuple, any clue? I'm 
> studing ways to replace too.
>

Do not care about PyArg_ParseTuple either. It's what C Python needs to 
parse function call arguments from a tuple into C primitives. Cython 
will do this, which is some of the raison d'etre for using Cython.

Also observe that any initialisation done in PyInit_cmathshould go as a 
module level function call in Cython, i.e. PyInit_cmathis called on 
import just like module level Python and Cython code.

I don't have time to implement all of cmathmodule.c, but here is a 
starter (not tested & not complete).

It might actually be that we should use "cdef complex z" instead of 
"cdef double complex z". There might be a distinction in the generated 
C/C++ code between those types, e.g. Py_complex for complex and "double 
_Complex" or "std::complex<double>" for double complex,  even though 
they are binary equivalent. I'm not sure about the state of Cython with 
respect to complex numbers, so just try it and see which works better :-)

Also observe that we do not release the GIL here. That is not because 
these functions are not thread-safe, they are, but yielding the GIL will 
slow things terribly.

Sturla

cimport math
cimport stdlib

cdef extern from "_math.h":
     int Py_IS_FINITE(double)
     int Py_IS_NAN(double)
     double copysign(double,double)

cdef enum special_types:
     ST_NINF  # 0, negative infinity
     ST_NEG   # 1, negative finite number (nonzero)
     ST_NZERO # 2, -0.
     ST_PZERO # 3, +0.
     ST_POS   # 4, positive finite number (nonzero)
     ST_PINF  # 5, positive infinity
     ST_NAN   # 6, Not a Number

cdef inline special_types special_type(double d):
     if (Py_IS_FINITE(d)):
         if (d != 0):
             if (copysign(1., d) == 1.):
                 return ST_POS
             else
                 return ST_NEG
         else:
             if (copysign(1., d) == 1.):
                 return ST_PZERO
             else
                 return ST_NZERO
     if (Py_IS_NAN(d)):
         return ST_NAN
     if (copysign(1., d) == 1.):
         return ST_PINF
     else:
         return ST_NINF

cdef void INIT_SPECIAL_VALUES( double complex *table,
                                double complex arc[][7]):
     stdlib.memcpy(table, &(src[0][0]), 7*7*sizeof(double complex))

cdef inline double complex *SPECIAL_VALUE(double complex z, double 
complex table[][7]):
     if (not Py_IS_FINITE(z.real)) or (not Py_IS_FINITE(z.imag)):
         errno = 0
         return &(table[special_type(z.real)][special_type(z.imag)])
     else:
         return NULL

cdef double complex acosh_special_values[7][7]

INIT_SPECIAL_VALUES( acos_special_values, {
  C(P34,INF) C(P,INF) C(P,INF) C(P,-INF) C(P,-INF) C(P34,-INF) C(N,INF)
  C(P12,INF) C(U,U) C(U,U) C(U,U) C(U,U) C(P12,-INF) C(N,N)
  C(P12,INF) C(U,U) C(P12,0.) C(P12,-0.) C(U,U) C(P12,-INF) C(P12,N)
  C(P12,INF) C(U,U) C(P12,0.) C(P12,-0.) C(U,U) C(P12,-INF) C(P12,N)
  C(P12,INF) C(U,U) C(U,U) C(U,U) C(U,U) C(P12,-INF) C(N,N)
  C(P14,INF) C(0.,INF) C(0.,INF) C(0.,-INF) C(0.,-INF) C(P14,-INF) C(N,INF)
  C(N,INF) C(N,N) C(N,N) C(N,N) C(N,N) C(N,-INF) C(N,N)
  })

cdef double complex cmath_acosh(double complex z):

     cdef double complex s1, s2, r, *psv

     psv = SPECIAL_VALUE(z, acosh_special_values)
     if (psv != NULL):
         return psv[0]

     if (math.fabs(z.real) > CM_LARGE_DOUBLE or math.fabs(z.imag) > 
CM_LARGE_DOUBLE):
         # avoid unnecessary overflow for large arguments
         r.real = math.log(hypot(z.real/2., z.imag/2.)) + M_LN2*2.
         r.imag = math.atan2(z.imag, z.real)
     else:
         s1.real = z.real - 1.
         s1.imag = z.imag
         s1 = cmath_sqrt(s1) # cdef double complex cmath_sqrt(double 
complex z)
         s2.real = z.real + 1.
         s2.imag = z.imag
         s2 = cmath_sqrt(s2)
         r.real = math.asinh(s1.real*s2.real + s1.imag*s2.imag)
         r.imag = 2.*math.atan2(s1.imag, s2.real)
     errno = 0
     return r

def acosh(object arg):
     """acos(x)

Return the arc cosine of x."""
     double complex z
     z = cmath_acosh(<double complex> arg)
     return complex(z)

Sturla Molden | 4 Apr 01:53 2011
Picon

Re: [Cython] Interest in contributing to the project

Den 04.04.2011 01:49, skrev Sturla Molden:
> Also observe that we do not release the GIL here. That is not because 
> these functions are not thread-safe, they are, but yielding the GIL 
> will slow things terribly.

Oh, actually they are not thread-safe because we set errno... Sorry.

Sturla

Dag Sverre Seljebotn | 4 Apr 12:17 2011
Picon
Picon

[Cython] CEP: prange for parallel loops

CEP up at http://wiki.cython.org/enhancements/prange

"""
This spec is the result of a number of discussions at Cython workshop 1. 
Quite a few different ways of expressing parallelism was looked at, and 
finally we decided to split the problem in two:

  * A simple and friendly solution that covers, perhaps, 80% of the 
cases, based on simply replacing range with prange.

  * Less friendly solutions for the remaining cases. These cases may 
well not even require language support in Cython, or only in indirect 
ways (e.g., cdef closures if normal closures are too expensive).

This document focuses exclusively on the former solution and does not 
intend to cover all use-cases for parallel programming, only the most 
common ones.
"""

Note that me and Mark talked some more on the way to the airport, and 
also I got a couple of more ideas afterwards, so everybody interested 
should probably take a read even if you were there for discussions.

Main post-workshop changes:

  * cython.parallel.firstiteration()/lastiteration # for in-loop if-test 
for thread setup/teardown blocks

  * An idea for how to implement numthreads(), so that we can drop the 
rather complex Context idea.

  * More thoughts on firstprivate/lastprivate

Dag Sverre

Gmane