headers
Mike Axiak | 22 Jul 2010 01:01
Gravatar

Re: C++ Reference Type Declaring too Early

On Wed, Jul 21, 2010 at 6:58 PM, Robert Bradshaw
<robertwb <at> math.washington.edu> wrote:
> Well, references are problematic as you get into issues of different
> scoping rules between C++ and Python. In C++, your supposed to be able
> to use an object whenever you want a reference and vice-versa, right?
> For performance reasons, you may want to take the address of incoming
> references, and there's no problem storing and passing those around.

Ah, but when I do this, I get hit with: "Taking address of
non-lvalue". I can't assign to a temp variable for obvious reasons!

-Mike
Permalink | Reply | 3 comments |
headers
Mike Axiak | 22 Jul 2010 01:06
Gravatar

Re: C++ Reference Type Declaring too Early

If you figure out a work around, I'll post it to the cplusplus wiki page :)
Maybe I can write a macro to circumvent the lvalue check?

-Mike

On Wed, Jul 21, 2010 at 7:01 PM, Mike Axiak <mike <at> axiak.net> wrote:
> On Wed, Jul 21, 2010 at 6:58 PM, Robert Bradshaw
> <robertwb <at> math.washington.edu> wrote:
>> Well, references are problematic as you get into issues of different
>> scoping rules between C++ and Python. In C++, your supposed to be able
>> to use an object whenever you want a reference and vice-versa, right?
>> For performance reasons, you may want to take the address of incoming
>> references, and there's no problem storing and passing those around.
>
> Ah, but when I do this, I get hit with: "Taking address of
> non-lvalue". I can't assign to a temp variable for obvious reasons!
>
> -Mike
>
Permalink | Reply | 1 comment |
headers
Robert Bradshaw | 22 Jul 2010 01:05
Favicon

Re: C++ Reference Type Declaring too Early

On Wed, Jul 21, 2010 at 4:01 PM, Mike Axiak <mike <at> axiak.net> wrote:
> On Wed, Jul 21, 2010 at 6:58 PM, Robert Bradshaw
> <robertwb <at> math.washington.edu> wrote:
>> Well, references are problematic as you get into issues of different
>> scoping rules between C++ and Python. In C++, your supposed to be able
>> to use an object whenever you want a reference and vice-versa, right?
>> For performance reasons, you may want to take the address of incoming
>> references, and there's no problem storing and passing those around.
>
> Ah, but when I do this, I get hit with: "Taking address of
> non-lvalue". I can't assign to a temp variable for obvious reasons!

And it's declared to return a reference? I had thought I made
references into lvalues for the addressOf operator, but I guess not.
Another item for the todo list.

- Robert
Permalink | Reply | 0 comments |
headers
Robert Bradshaw | 22 Jul 2010 01:10
Favicon

Re: C++ Reference Type Declaring too Early

On Wed, Jul 21, 2010 at 4:06 PM, Mike Axiak <mike <at> axiak.net> wrote:
> If you figure out a work around, I'll post it to the cplusplus wiki page :)
> Maybe I can write a macro to circumvent the lvalue check?

Yes, a macro would be the perfect workaround for now.

> -Mike
>
> On Wed, Jul 21, 2010 at 7:01 PM, Mike Axiak <mike <at> axiak.net> wrote:
>> On Wed, Jul 21, 2010 at 6:58 PM, Robert Bradshaw
>> <robertwb <at> math.washington.edu> wrote:
>>> Well, references are problematic as you get into issues of different
>>> scoping rules between C++ and Python. In C++, your supposed to be able
>>> to use an object whenever you want a reference and vice-versa, right?
>>> For performance reasons, you may want to take the address of incoming
>>> references, and there's no problem storing and passing those around.
>>
>> Ah, but when I do this, I get hit with: "Taking address of
>> non-lvalue". I can't assign to a temp variable for obvious reasons!
>>
>> -Mike
>>
>
Permalink | Reply | 0 comments |
headers
Lionel Data | 22 Jul 2010 09:46
Picon

Re: Explicitly using scope operator in Cython C++ wrapping

2010/7/21 Lisandro Dalcin <dalcinl <at> gmail.com>
On 21 July 2010 12:42, Robert Bradshaw <robertwb <at> math.washington.edu> wrote:
> On Wed, Jul 21, 2010 at 6:55 AM, Mike Axiak <mcaxiak <at> gmail.com> wrote:
>> Hi Lionel,
>>
>> Did you ever figure this out? I'm having this exact problem right now
>> in my code.
>>
>> Mike
>>
>> On Wed, May 19, 2010 at 7:08 AM, Lionel Data <lionel.data <at> gmail.com> wrote:
>>> Hi,
>>>
>>> I've been playing with Cython for a while now, and managed to avoid this
>>> situation a few times, but it seems I'm stuck with it now.
>>>
>>> I have a decl.pxd file stating this:
>>> cdef extern from "myFile.h" namespace "myNamespace":
>>>     ctypedef enum myEnum:
>>>         VALUE1
>>>         VALUE2
>>>         VALUE3
>>>
>>> This, as you would imagine, is itself matching a C++ equivalent being
>>> obviously:
>>> namespace myNamespace {
>>>   enum myEnum {
>>>      VALUE1,
>>>      VALUE2,
>>>      VALUE3
>>>   }
>>> }
>>>
>>> However, it seems I can't correctly reach those values within Cython code. I
>>> tried several syntax tricks, but nothing did the job.
>>>
>>> For instance, I have a cython file called code.pyx, in which I write:
>>> cimport decl
>>>
>>> pyVALUE1 = decl.VALUE1 # Generates C++ code stating "VALUE1" instead of
>>> "myNamespace::VALUE1" which of course results in g++ error
>>> pyVALUE2 = decl.myEnum.VALUE1 # Doesn't even translates to C++ (not
>>> surprising)
>>> pyVALUE3 = decl.myNamespace.VALUE2 # Same as above
>>>
>>> I couldn't find an answer on both http://wiki.cython.org/WrappingCPlusPlus
>>> and http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html
>>> Can someone help me on this ?
>
> This looks like a bug. As a workaround, try
>
>>> cdef extern from "myFile.h" namespace "myNamespace":
>>>     ctypedef enum myEnum:
>>>         VALUE1 "myNamespace::VALUE1"
>

Could any of you test the attached patch?


Seems to work just great :)
Thanks !

Lionel
Permalink | Reply | 2 comments |
headers
Lionel Data | 22 Jul 2010 10:07
Picon

Re: Wrapping C++ classes that refer to each other



2010/7/21 ptao <philliptao <at> gmail.com>
Hi,

What's the proper way to wrap classes that depend on other classes?

For example, in A.h:

#include <B.h>

namespace N {
   class A {
       public:
           B getB();
           bool setB(B b);
   };
}

The cython files for B are something like:

# in cB.pxd
cdef extern from "../include/B.h" namespace "N":
   cdef cppclass B:
       ...

# in B.pyx
cimport cB

cdef class B(object):
   cdef cB.B *ptr;
   ...

How would I go about wrapping the A class?


Hello,

The trick is to manage your pointers correctly.

I would do something like this (didnt test the code, but you get the idea):

cimport cB
cimport cA

from B import B

cdef class A(object):
   cdef cA.A *ptr

   def getB(self):
       cdef b = B()
       b.ptr = self.ptr.getB()
       return b

Try not to leak when changing the pointer of the B wrapper. You should actually do this via a method that will delete the inner pointer before replacing it, and also mark that you should not delete it in the __dealloc__ function. I think you see the problem here. Good luck :)

Lionel

Permalink | Reply | 0 comments |
headers
Lionel Data | 22 Jul 2010 10:19
Picon

Re: What is the right way to cython-ify this code??



2010/7/21 glbarnes <glbarnes1 <at> gmail.com>
Hello All:

I am new to both python and cython so please bear with me!  I am a
post doc in the sciences and have done quite a bit of coding in
fortran, but I have recently become excited by the idea of python
simplicity for most of the code with the performance of compiled
code.  As a small startup project I've been working on a simple
Lennard-Jones molecular dynamics program and comparing it to a fortran
equivalent.  The fortran code is currently quite a bit faster.

I am fairly certain that I am getting bogged down in calculating the
potential.  I've tried adding types but have not seen the tremendous
speed increases that people such as Seljebotn have been able to
achieve.  I'm certain that I am trying to do things that are just
plain wrong or the wrong way to think about the cython process.

Rather than post my failed attempts, I am attaching a scaled down
python code that includes what I believe is a similar operation to my
bottleneck in the full code.  The fortran version of this simple code
runs about 180x faster than the python code (very rough estimate using
linux time utility!).

So my question is what is the right way to apply cython to this code??

Thanks in advance to anyone that can help a new python/cython user
out!

Best, George


class Atom:
   """ define an atom, real code includes other properties,
   ie mass, momentum, etc """
   def __init__(self, xyz=np.zeros(3,np.double) ):
       self.position=xyz

class Molecule(list):
   """ define a molecule as a list of atoms.
   real code adds additional properties of the molecule, ie total
energy, potential energy, etc"""
   def __init__(self,atoms=list()):
       self.atoms=atoms
       self.natom=len(self.atoms)

   def distance_mat(self):
       """ example routine similar to rate limiting step.
       actual code uses R to calculate potential=A/R^12 - B/R^6
       and the force on each atom, ie force+=xyz*(-12A/R^14+6B/
R^8)"""
       for i in xrange(self.natom):
           for j in xrange(i+1,self.natom):
               xyz=self.atoms[j].position-self.atoms[i].position
               R=math.sqrt( (xyz**2).sum() )

Hello,

you might want to look at this very good tutorial:

http://wiki.cython.org/tutorials/numpy

You might also want to rework your code. I explain:

Avoid Python loops (for ... in ... ) when possible if you care about performances, try to use Numpy as much as possible.
For instance, instead of having a list of Atom with A, B and C properties, I would have 3 numpy arrays of A's B's and C's. That is, if you have 2 Atoms with A=1, B=1, C=1, and A=2, B=2, C=2, instead of having a list of 2 atoms, I would go for 3 numpy arrays (that would look like [1, 2] here).

Lionel
Permalink | Reply | 3 comments |
headers
Lisandro Dalcin | 22 Jul 2010 16:55
Picon
Gravatar

Re: Explicitly using scope operator in Cython C++ wrapping

On 22 July 2010 04:46, Lionel Data <lionel.data <at> gmail.com> wrote:
> 2010/7/21 Lisandro Dalcin <dalcinl <at> gmail.com>
>>
>> On 21 July 2010 12:42, Robert Bradshaw <robertwb <at> math.washington.edu>
>> wrote:
>> > On Wed, Jul 21, 2010 at 6:55 AM, Mike Axiak <mcaxiak <at> gmail.com> wrote:
>> >> Hi Lionel,
>> >>
>> >> Did you ever figure this out? I'm having this exact problem right now
>> >> in my code.
>> >>
>> >> Mike
>> >>
>> >> On Wed, May 19, 2010 at 7:08 AM, Lionel Data <lionel.data <at> gmail.com>
>> >> wrote:
>> >>> Hi,
>> >>>
>> >>> I've been playing with Cython for a while now, and managed to avoid
>> >>> this
>> >>> situation a few times, but it seems I'm stuck with it now.
>> >>>
>> >>> I have a decl.pxd file stating this:
>> >>> cdef extern from "myFile.h" namespace "myNamespace":
>> >>>     ctypedef enum myEnum:
>> >>>         VALUE1
>> >>>         VALUE2
>> >>>         VALUE3
>> >>>
>> >>> This, as you would imagine, is itself matching a C++ equivalent being
>> >>> obviously:
>> >>> namespace myNamespace {
>> >>>   enum myEnum {
>> >>>      VALUE1,
>> >>>      VALUE2,
>> >>>      VALUE3
>> >>>   }
>> >>> }
>> >>>
>> >>> However, it seems I can't correctly reach those values within Cython
>> >>> code. I
>> >>> tried several syntax tricks, but nothing did the job.
>> >>>
>> >>> For instance, I have a cython file called code.pyx, in which I write:
>> >>> cimport decl
>> >>>
>> >>> pyVALUE1 = decl.VALUE1 # Generates C++ code stating "VALUE1" instead
>> >>> of
>> >>> "myNamespace::VALUE1" which of course results in g++ error
>> >>> pyVALUE2 = decl.myEnum.VALUE1 # Doesn't even translates to C++ (not
>> >>> surprising)
>> >>> pyVALUE3 = decl.myNamespace.VALUE2 # Same as above
>> >>>
>> >>> I couldn't find an answer on both
>> >>> http://wiki.cython.org/WrappingCPlusPlus
>> >>> and http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html
>> >>> Can someone help me on this ?
>> >
>> > This looks like a bug. As a workaround, try
>> >
>> >>> cdef extern from "myFile.h" namespace "myNamespace":
>> >>>     ctypedef enum myEnum:
>> >>>         VALUE1 "myNamespace::VALUE1"
>> >
>>
>> Could any of you test the attached patch?
>>
>
> Seems to work just great :)
> Thanks !
>
> Lionel
>

http://hg.cython.org/cython-devel/rev/bef247b13c97

--

-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169
Permalink | Reply | 1 comment |
headers
Robert Bradshaw | 22 Jul 2010 17:58
Favicon

Re: Explicitly using scope operator in Cython C++ wrapping

On Thu, Jul 22, 2010 at 7:55 AM, Lisandro Dalcin <dalcinl <at> gmail.com> wrote:
> On 22 July 2010 04:46, Lionel Data <lionel.data <at> gmail.com> wrote:
>> 2010/7/21 Lisandro Dalcin <dalcinl <at> gmail.com>
>>>
>>> On 21 July 2010 12:42, Robert Bradshaw <robertwb <at> math.washington.edu>
>>> wrote:
>>> > On Wed, Jul 21, 2010 at 6:55 AM, Mike Axiak <mcaxiak <at> gmail.com> wrote:
>>> >> Hi Lionel,
>>> >>
>>> >> Did you ever figure this out? I'm having this exact problem right now
>>> >> in my code.
>>> >>
>>> >> Mike
>>> >>
>>> >> On Wed, May 19, 2010 at 7:08 AM, Lionel Data <lionel.data <at> gmail.com>
>>> >> wrote:
>>> >>> Hi,
>>> >>>
>>> >>> I've been playing with Cython for a while now, and managed to avoid
>>> >>> this
>>> >>> situation a few times, but it seems I'm stuck with it now.
>>> >>>
>>> >>> I have a decl.pxd file stating this:
>>> >>> cdef extern from "myFile.h" namespace "myNamespace":
>>> >>>     ctypedef enum myEnum:
>>> >>>         VALUE1
>>> >>>         VALUE2
>>> >>>         VALUE3
>>> >>>
>>> >>> This, as you would imagine, is itself matching a C++ equivalent being
>>> >>> obviously:
>>> >>> namespace myNamespace {
>>> >>>   enum myEnum {
>>> >>>      VALUE1,
>>> >>>      VALUE2,
>>> >>>      VALUE3
>>> >>>   }
>>> >>> }
>>> >>>
>>> >>> However, it seems I can't correctly reach those values within Cython
>>> >>> code. I
>>> >>> tried several syntax tricks, but nothing did the job.
>>> >>>
>>> >>> For instance, I have a cython file called code.pyx, in which I write:
>>> >>> cimport decl
>>> >>>
>>> >>> pyVALUE1 = decl.VALUE1 # Generates C++ code stating "VALUE1" instead
>>> >>> of
>>> >>> "myNamespace::VALUE1" which of course results in g++ error
>>> >>> pyVALUE2 = decl.myEnum.VALUE1 # Doesn't even translates to C++ (not
>>> >>> surprising)
>>> >>> pyVALUE3 = decl.myNamespace.VALUE2 # Same as above
>>> >>>
>>> >>> I couldn't find an answer on both
>>> >>> http://wiki.cython.org/WrappingCPlusPlus
>>> >>> and http://docs.cython.org/src/userguide/wrapping_CPlusPlus.html
>>> >>> Can someone help me on this ?
>>> >
>>> > This looks like a bug. As a workaround, try
>>> >
>>> >>> cdef extern from "myFile.h" namespace "myNamespace":
>>> >>>     ctypedef enum myEnum:
>>> >>>         VALUE1 "myNamespace::VALUE1"
>>> >
>>>
>>> Could any of you test the attached patch?
>>>
>>
>> Seems to work just great :)
>> Thanks !
>>
>> Lionel
>>
>
> http://hg.cython.org/cython-devel/rev/bef247b13c97

Thanks!
Permalink | Reply | 0 comments |
headers
glbarnes | 22 Jul 2010 18:02
Picon

Re: What is the right way to cython-ify this code??

Hi Lionel,

Thanks for the reply!

>
> you might want to look at this very good tutorial:
>
> http://wiki.cython.org/tutorials/numpy
>

Yes, I have certainly looked at this tutorial already.  It didn't deal
much with classes though, which might be part of what is tripping me
up!  I am new python, numpy, and cython so I do have a lot to learn
about their capabilities.

> You might also want to rework your code. I explain:
>
> Avoid Python loops (for ... in ... ) when possible if you care about
> performances, try to use Numpy as much as possible.
> For instance, instead of having a list of Atom with A, B and C properties, I
> would have 3 numpy arrays of A's B's and C's. That is, if you have 2 Atoms
> with A=1, B=1, C=1, and A=2, B=2, C=2, instead of having a list of 2 atoms,
> I would go for 3 numpy arrays (that would look like [1, 2] here).
>

I can certainly see where avoiding loops would speed things up.  A
couple of comments and follow up questions about your suggestion.

In the example that I gave I have the class Molecule which is itself a
list of Atom objects that will have several properties.  For
simplicity I just showed a single property, the position which is
itself a (3,1) Numpy array.  Suppose I instead defined the variable A
to be a (N,3) Numpy array that holds all 3 elements of the position
array.  Is there a Numpy function that allows me to take, for example,
A(2,:)-A(1,:)?  I need to take the difference between A(i,:) and all
other A(j,:) for all i < j, ie the distance in 3D space from a given
atom to all other atoms.  In other words, even if I had a single
matrix how do I avoid for loops?

Also, if I did make three variables for my Molecule class rather than
it being a list of Atom objects how is this any simpler than just
writing in Fortran?  One of the big advantages I see of python is the
object orientated nature...that I can have an Atom object that carries
around all the properties and then define a Molecule to be a
collection of these already defined atoms that I can add or remove
member Atom's from.  With a Numpy array I have to know its size before
I declare it, don't I?  Perhaps I want to have my cake and eat it
too?

Thanks again for taking the time to help a new user out!

Best, George
Permalink | Reply | 2 comments |

Gmane