Stéfan van der Walt | 1 Jun 2009 01:46
Picon
Picon
Favicon

[Cython] Profiling Cython under OSX

Hi all,

I needed to profile some Cython code on my mac today.  I am only
familiar with Valgrind, but this tool is not available on OSX (closest
thing is  http://www.sealiesoftware.com/valgrind/).  Luckily, I
discovered that the Activity Monitor is very handy in this regard:

1. Start your long running process
2. Go to the activity monitor, select the process, and click "Sample Process"
3. In the window that pops up, select "Display: Percent of Parent"

And there you have a beautiful tree showing how much time each call took:

http://mentat.za.net/refer/osx_profiling.png

Enjoy!
Stéfan
Amit Sethi | 1 Jun 2009 15:38
Picon

[Cython] Creating python bindings for C library using Cython


hi ,
I guess this needs to go on cython-users but i did not see any such mailing list so I am posting it here ,
I am trying to create python bindings for a library written in C ,
But I am wondering as to how I am suppose to handle pointers.
 
For example

for the C function,
int * analyse_row (int length, unsigned char *data),


Now in this function call how am i supposed to handle *data , can i pass a string in place of *data ., Also please guide be to some projects that use Cython for binding with C
--
A-M-I-T S|S
<div>
<br clear="all">hi , <br>I guess this needs to go on cython-users but i did not see any such mailing list so I am posting it here , <br>I am trying to create python bindings for a library written in C , <br>But I am wondering as to how I am suppose to handle pointers.<br>

&nbsp;<br>For example<br><br>for the C function, <br>int * analyse_row (int length, unsigned char *data), <br><br><br>Now in this function call how am i supposed to handle *data , can i pass a string in place of *data ., Also please guide be to some projects that use Cython for binding with C<br>

-- <br>A-M-I-T S|S<br>
</div>
Lisandro Dalcin | 1 Jun 2009 16:19
Picon
Gravatar

Re: [Cython] Creating python bindings for C library using Cython

On Mon, Jun 1, 2009 at 10:38 AM, Amit Sethi
<amit.pureenergy@...> wrote:
>
> hi ,
> I guess this needs to go on cython-users but i did not see any such mailing
> list so I am posting it here ,
> I am trying to create python bindings for a library written in C ,
> But I am wondering as to how I am suppose to handle pointers.
>
> For example
>
> for the C function,
> int * analyse_row (int length, unsigned char *data),
>
>
> Now in this function call how am i supposed to handle *data , can i pass a
> string in place of *data .

Mmm... better you are sure what the 'data' represent. Anyway, you can
also add explicit cast and call like "analize_row(somelen, <unsigned
char*>somepointer)". I would need some more backgraund from you to
provide a better hint (What is analyse_row() for?)

> Also please guide be to some projects that use
> Cython for binding with C
>

>From my side, you could look at mpi4py, petsc4py, slepc4py (all of
them at Google Code).

--

-- 
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
Lisandro Dalcin | 1 Jun 2009 16:33
Picon
Gravatar

Re: [Cython] Running Cython 0.11.2 tests using MinGW32

On Sat, May 30, 2009 at 12:04 PM, Robert Bradshaw
<robertwb@...> wrote:
> On May 28, 2009, at 3:50 PM, Lisandro Dalcin wrote:
>
>>
>> Anyway, suppose you have installed both MSVC and MinGW, an as you are
>> a serious Cython developer, you want to run the testsuite with both.
>> The easiest way to select GCC would be to pass '--mingw' to
>> runtests.py, right?
>
>
> I hadn't thought of that. Are you saying that you would have a single
> Python install that works both with MSVC and MinGW?
>

Of course!

In my particular case, I have installed stock Python 2.6 (using the
MSI installed from python.org), and I routinely test mpi4py building
it with MSVC Express 2008 and MinGW32, with a couple of different MPI
implementations (DeinoMPI, MPICH2, and very recently Microsoft MPI).
Up to now, all work just fine. Of course, I've been very careful and
my code does not call anything from the C stdlib (which is know to
have issues when building with MinGW, because of possible mismatches
of the C runtime).

OTOH, most of the Cython testsuite pass happily with the stock Python
2.6 and MinGW32. The only offending part are test like 'cdef extern
void foo()', as that end-up with linker errors.

--

-- 
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
Sebastien Binet | 1 Jun 2009 18:11
Picon
Gravatar

[Cython] cimport from __all__

hi there,

I am in the process of applying what I learnt at last SciPy'08 tutorial (yes, 
I am that slow) on an almost-real-life project of mine.
Up to now I am trying to translate an old C++/PyCxx project (which does not 
compile anymore) into Cython: so far so good.

One thing, though: the separation of '.pyx' and '.pxd' files to export 
declarations is a bit too C-ish.

Couldn't the same functionality be achieved by re-using the __all__ module 
attribute ?

## mymodule.pyx ##
__all__ = ( 'Foo', 'bar')

cdef class Foo:
   # implementation elided

cdef int bar():
  return 42
## EOF ##

## test.pyx ##
cimport mymodule
print "bar:",mymodule.bar()
## EOF ##

I suppose I am missing some implementation detail, but from my 20k feet 
overview, it seems that should work :)

cheers,
sebastien.
--

-- 
#########################################
# Dr. Sebastien Binet
# Laboratoire de l'Accelerateur Lineaire
# Universite Paris-Sud XI
# Batiment 200
# 91898 Orsay
#########################################

Dag Sverre Seljebotn | 1 Jun 2009 18:23
Picon
Picon

Re: [Cython] cimport from __all__

Sebastien Binet wrote:
> hi there,
>
> I am in the process of applying what I learnt at last SciPy'08 tutorial (yes, 
> I am that slow) on an almost-real-life project of mine.
> Up to now I am trying to translate an old C++/PyCxx project (which does not 
> compile anymore) into Cython: so far so good.
>
> One thing, though: the separation of '.pyx' and '.pxd' files to export 
> declarations is a bit too C-ish.
>
> Couldn't the same functionality be achieved by re-using the __all__ module 
> attribute ?
>
> ## mymodule.pyx ##
> __all__ = ( 'Foo', 'bar')
>
> cdef class Foo:
>    # implementation elided
>
> cdef int bar():
>   return 42
> ## EOF ##
>
> ## test.pyx ##
> cimport mymodule
> print "bar:",mymodule.bar()
> ## EOF ##
>
> I suppose I am missing some implementation detail, but from my 20k feet 
> overview, it seems that should work :)
>   
I remember that the basic idea of this feature (which would be 
implemented as "generate pxd files automatically from pyx files") has 
been discussed earlier (Sage dev 1 days?) and it's definitely on our 
would-like-to-do-if-we-get-time list :-) The idea about using the 
__all__ attribute to declare what symbols to export was new; I like it!

Dag Sverre
Chris Colbert | 1 Jun 2009 19:37
Picon

Re: [Cython] Creating python bindings for C library using Cython

python strings are automagically converted to char* by cython.
 
You can also look at a wrapper i'm building for the OpenCV library. cython-opencv at google code.
 
Chris

On Mon, Jun 1, 2009 at 6:38 AM, Amit Sethi <amit.pureenergy-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:

hi ,
I guess this needs to go on cython-users but i did not see any such mailing list so I am posting it here ,
I am trying to create python bindings for a library written in C ,
But I am wondering as to how I am suppose to handle pointers.
 
For example

for the C function,
int * analyse_row (int length, unsigned char *data),


Now in this function call how am i supposed to handle *data , can i pass a string in place of *data ., Also please guide be to some projects that use Cython for binding with C
--
A-M-I-T S|S

_______________________________________________
Cython-dev mailing list
Cython-dev <at> codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev


<div>
<div>python strings are automagically converted to char* by cython. </div>
<div>&nbsp;</div>
<div>You can also look at a wrapper i'm building for the OpenCV library. cython-opencv at google code. </div>
<div>&nbsp;</div>
<div>Chris<br><br>
</div>
<div class="gmail_quote">On Mon, Jun 1, 2009 at 6:38 AM, Amit Sethi <span dir="ltr">&lt;<a href="mailto:amit.pureenergy@...">amit.pureenergy@...</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">
<br clear="all">hi , <br>I guess this needs to go on cython-users but i did not see any such mailing list so I am posting it here , <br>
I am trying to create python bindings for a library written in C , <br>But I am wondering as to how I am suppose to handle pointers.<br>&nbsp;<br>For example<br><br>for the C function, <br>int * analyse_row (int length, unsigned char *data), <br><br><br>Now in this function call how am i supposed to handle *data , can i pass a string in place of *data ., Also please guide be to some projects that use Cython for binding with C<br>-- <br>A-M-I-T S|S<br><br>_______________________________________________<br>Cython-dev mailing list<br><a href="mailto:Cython-dev@...">Cython-dev <at> codespeak.net</a><br><a href="http://codespeak.net/mailman/listinfo/cython-dev" target="_blank">http://codespeak.net/mailman/listinfo/cython-dev</a><br><br>
</blockquote>
</div>
<br>
</div>
Robert Kern | 1 Jun 2009 20:18
Picon
Gravatar

Re: [Cython] Profiling Cython under OSX

On 2009-05-31 18:46, Stéfan van der Walt wrote:
> Hi all,
>
> I needed to profile some Cython code on my mac today.  I am only
> familiar with Valgrind, but this tool is not available on OSX (closest
> thing is  http://www.sealiesoftware.com/valgrind/).

Actually, the OS X branch got merged into the trunk a few days ago.

> Luckily, I
> discovered that the Activity Monitor is very handy in this regard:
>
> 1. Start your long running process
> 2. Go to the activity monitor, select the process, and click "Sample Process"
> 3. In the window that pops up, select "Display: Percent of Parent"

Instruments.app is the canonical tool for this on OS X.

--

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

_______________________________________________
Cython-dev mailing list
Cython-dev <at> codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev
Robert Bradshaw | 1 Jun 2009 21:06
Favicon

Re: [Cython] Profiling Cython under OSX

On Jun 1, 2009, at 11:18 AM, Robert Kern wrote:

> On 2009-05-31 18:46, Stéfan van der Walt wrote:
>> Hi all,
>>
>> I needed to profile some Cython code on my mac today.  I am only
>> familiar with Valgrind, but this tool is not available on OSX  
>> (closest
>> thing is  http://www.sealiesoftware.com/valgrind/).
>
> Actually, the OS X branch got merged into the trunk a few days ago.
>
>> Luckily, I
>> discovered that the Activity Monitor is very handy in this regard:
>>
>> 1. Start your long running process
>> 2. Go to the activity monitor, select the process, and click  
>> "Sample Process"
>> 3. In the window that pops up, select "Display: Percent of Parent"
>
> Instruments.app is the canonical tool for this on OS X.

Still, very cool! Could you put this up on the wiki? (Also, any  
valgrind+Cython knowledge you might have.)

- Robert

Dag Sverre Seljebotn | 1 Jun 2009 21:09
Picon
Picon

Re: [Cython] Creating python bindings for C library using Cython

Chris Colbert wrote:
> python strings are automagically converted to char* by cython.
I'll beat Stefan to it and note some very important points here. This 
applies if you actually have a string (e.g. could be stored in "unicode" 
in Py2), not if you have raw byte data ("bytes" in Py3).

1) A C char* is not a string as such, just a sequence of bytes. The C 
library should have a definition of what kind of encoding it wants its 
strings in, and one should call the library like this:

encoded = mystr.encode(encoding)
cdef char* encoded_buf = encoded
c_func(encoded_buf)

2) Note that char* does not say anything about releasing memory etc. 
I.e., the following will likely crash:

c_func(mystr.encode(encoding))

because the temporary object returned from the encode method is 
deallocated. *Always* keep a reference to the original Python object for 
the duration someone could use the char*.

(Perhaps somebody could make this a FAQ entry in the wiki if it is not 
already?)

Dag Sverre

Gmane