Alexander Bass | 4 May 16:49 2016
Picon

Exposing a c++ function to python using cython

I am trying to expose a c++ function to python using cython as follows:


from libcpp.vector cimport vector cdef extern from "auction.h": cpdef double util( int q, int sq, double alpha, double beta, double kappa, int s, int ss, int m, const vector[double]& p, double a0);

I am getting the following (not very descriptive) error:

src\RFQLib.pyx:4:21: Cannot convert 'double (int, int, double, double, double, int, int, int, const vector[double] &, double)' to Python object

What's the problem here?

Thanks in advance for all your help.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Pallav Gupta | 2 May 23:07 2016
Picon

Issue with linking order when using both shared/static libraries.

I'm using Cython 0.24 on python 2.7.9 on Suse Enterprise Linux 11. I have a C++ class (rdb2oas) that I am trying to cythonize. This class calls some functions in an external library that is distributed as a static library (libdogs.a). So I write the .pyx file and create the following build script.

1. First, I build rdb2oas using my C++ build environment to create a .so library (libRdb2Oas.so).

2. I then run 'python setup.py build_ext --inplace' on to cythonize rdb2oas. I use 'extra_objects' option to specify the location of the static library. The compile runs OK, and I get a shared module rdb2oas.so.

3. When I try to import this module in python, I get an 'undefined symbol' error.

>>> import rdb2oas
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /nfs/tron/out/em64t_SLES11/shOpt64/libRdb2Oas.so: undefined symbol: _ZN11CPolygonVar4OpenEv

I figured out the issue is with the linking order. The static library (libdogs.a) should be at the end of the gcc command line for things to work. In this case it is not. How can I fix this in the setup.py build script? 

Thanks for any ideas.


/usr/pkgs/gcc/4.7.2/bin/g++ -O2 -fPIC -I/usr/pkgs/python/2.7.9/include -L/usr/pkgs/python/2.7.9/lib64 -L/usr/pkgs/python/2.7.9/lib -pthread -shared -O2 -fPIC -L/usr/pkgs/python/2.7.9/lib64 -L/usr/pkgs/python/2.7.9/lib -Wl,--rpath -Wl,/usr/pkgs/python/2.7.9/lib64:/usr/pkgs/python/2.7.9/lib:/usr/pkgs/python/2.7.9/lib64:/usr/pkgs/python/2.7.9/lib:/usr/pkgs/python/2.7.9/lib:/usr/pkgs/python/2.7.9/lib64: build/temp.linux-x86_64-2.7/rdb2oas.o /nfs/dogs/5.5.8/sles11-icc130-gcc472/lib/libdogs.a -L/nfs/tron/out/em64t_SLES11/shOpt64 -L/nfs/boost/1.58.0_gcc472_64/lib -L/usr/pkgs/python/2.7.9/lib -lRdb2Oas -lUtil -lboost_iostreams -lboost_filesystem -lpython2.7 -o /nfs/tron/cython/cles/rdb2oas.so


from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize

# See https://docs.python.org/2/distutils/apiref.html#distutils.core.Extension
ext_modules=[
    Extension(
        name               = 'rdb2oas',
        sources            = [ 'rdb2oas.pyx' ],
        language           = 'c++',
        include_dirs       = [ util.get_resolved_abspath('$DEVDIR/include'),
                                     util.get_resolved_abspath('$CAD_ROOT/boost/$BOOST_VERSION/include'),
                                   ],
        library_dirs        = [ util.get_resolved_abspath('$DEVDIR/out/$SYS_TYPE/$BLD_MODE') ],
        libraries            = [ 'Rdb2Oas'],
        extra_objects    = [ util.get_resolved_abspath('$CAD_ROOT/dogs/$DOGS_VERSION/lib/libdogs.a') ],
        extra_compile_args = [ '-std=c++11' ],             
    )
]

# See https://docs.python.org/2/distutils/apiref.html#distutils.core.setup
setup(
  ext_modules = cythonize(ext_modules)
)

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Robert Bradshaw | 1 May 05:46 2016
Picon

Re: Wrapping accessor that returns shared_ptr to object

On Fri, Apr 29, 2016 at 8:39 AM, <bpwlist <at> gmail.com> wrote:
Hi All,

I was recently introduced to Cython by a coworker.  I'm still getting my head wrapped around it and have been playing around with it a bit to understand if it would be a good candidate to wrap a large C++ class library we have.  I like the shared_ptr approach highlighted here.

One difference though is that in my library I have C++ accessor methods that will return a shared_ptr to an object.  I'm wondering what is the best way to wrap these efficiently.  Here is an example of what I'm trying to do:

from libcpp.memory cimport shared_ptr
from libcpp.string cimport string
from cython.operator cimport dereference as deref


cdef
extern from "CBar.hpp":
    cdef cppclass
CBar:
       
CBar()

       
string getName() except *

cdef
extern from "CFoo.hpp":
    cdef cppclass
CFoo:
       
CFoo()

       
void setBar(shared_ptr[CBar]) except *
        shared_ptr
[CBar] getBar() except *

cdef
class Bar(object):
    cdef shared_ptr
[CBar] _thisptr

   
def __init__(self):
       
pass

   
def __cinit__(self):
       
print "bar.cinit clled"
       
self._thisptr.reset(new CBar())
   
   
<at> staticmethod
    cdef create
(shared_ptr[CBar] ptr):
        cdef
Bar py_obj = Bar()
        py_obj
._thisptr = ptr
       
return py_obj

    property name
:
       
def __get__(self):
           
return deref(self._thisptr).getName()

cdef
class Foo(object):
    cdef shared_ptr
[CFoo] _thisptr
   
def __cinit__(self):
       
self._thisptr.reset(new CFoo())

    property bar
:
       
def __get__(self):
           
return Bar.create(deref(self._thisptr).getBar())





The issue I have with it, is that if I call the Foo's getBar method from python,  a new "CBar" object gets created in the cinit function, which is then immediately  replaced by the create function.  I'm wonder what is the best way to avoid it.  How have other people handle this?  Am I going down the wrong rabbit hole in my approach?


You can pass an optional boolean argument to leave the shared pointer null rather than allocating a new object.  

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Stefan Behnel | 30 Apr 07:34 2016
Picon

Re: Incorrect unpacking of memoryview shape

Antony Lee schrieb am 29.04.2016 um 04:05:
> The following code
> 
> import numpy as np
> cimport numpy as np
> def mv():
>     cdef:
>         double[:, ::1] view = np.zeros((3, 3))
>         size_t a, b
>     a, b = view.shape
> 
> 
> fails to compile because view.shape is of size 8 (=[3, 3, 0, 0, 0, 0, 0,
> 0]).  This leads to some additional funny issues as well, such as
> np.product(view.shape) == 0, but they are somewhat theoretical whereas
> shape unpacking is pretty common.

I agree. This has annoyed me whenever I ran into it. Patches welcome.

Stefan

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Robert Bradshaw | 30 Apr 05:31 2016
Picon

Re: Re: Pass a python instance back and forth between Python and Cython

On Thu, Apr 28, 2016 at 2:59 PM, Vern Muhr <vern.muhr <at> gmail.com> wrote:
Thanks for the tip Robert.
If I keep a copy of the instance on the python side my test script passes.
Is there a way on the Cython side to keep the instance from being garbage collected? Any examples?

You can keep a reference to the instance anywhere in your Cython file as well. 
 


On Tuesday, April 26, 2016 at 11:38:20 PM UTC-7, Vern Muhr wrote:
How can I create a Cython extension which manages a structure that has a python class instance as a member. Something like:

ctypedef struct my_struct:

    float t
    int priority
    ??? my_instance

What would the Cython functions look like to to create the structure and access the my_instance structure member from Python.
Note that Cython never needs to access any of the attributes of the Python instance. It is just an opaque object.

Thanks for the help!

 



--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Julien Delafontaine | 27 Apr 09:43 2016
Picon
Gravatar

How to check that the build was successful

Hello,
I am building a Cython extension and recently modified my setup.py. Now the build appears successful, but in the end I cannot import my module (ImportError), and I have no clue why.
I see a .o in build/temp.linux-..., and a .so in build/lib.linux-...

My question is: how can I diagnostic a build ?

What exactly is supposed to be created when I do "setup.py build_ext [--inplace]", then "setup.py install" (with a setup config following the cython docs) ? I'd like to check if something is missing but I can't understand what to look for.
How can I clean my environment from previous builds ? (e.g. getting again a "ValueError: numpy.dtype has the wrong size, try recompiling" - I suppose this is because of a build with a previous version).

My setup is the following:

import os
from setuptools import setup, find_packages
from setuptools.extension import Extension
import numpy

try:
    from Cython.Build import cythonize
except ImportError:
    BUILD_CYTHON = False
else:
    BUILD_CYTHON = True

ext = '.pyx' if BUILD_CYTHON else '.c'
extensions = [Extension("apply_bitwise",
    sources=["some/path/apply_bitwise"+ext],
    include_dirs=[numpy.get_include()],
)]
if BUILD_CYTHON:
    extensions = cythonize(extensions, include_path=[numpy.get_include()])

setup(
    packages=find_packages(exclude=['tests*', 'benchmark']),
    ext_modules=extensions,
...

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Vern Muhr | 27 Apr 00:53 2016
Picon

Pass a python instance back and forth between Python and Cython

How can I create a Cython extension which manages a structure that has a python class instance as a member. Something like:

ctypedef struct my_struct:

    float t
    int priority
    ??? my_instance

What would the Cython functions look like to to create the structure and access the my_instance structure member from Python.
Note that Cython never needs to access any of the attributes of the Python instance. It is just an opaque object.

Thanks for the help!

 



--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Zheng Shao | 26 Apr 17:25 2016
Picon
Gravatar

How to disable docstrings generation

Hi, I'm have a question about compilation.

All pyx docstrings will be generated into .c file like,

static char __pyx_doc_foobar[] = 'foobar'

Is there any compile flags or other methods could turn off docstrings generation.
Thank you.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Alex Coventry | 24 Apr 06:00 2016

Is there a way to say "no gil, except for translating function-call arguments"?

I have a group of C++ classes which don't touch python data structures at all, and some cython wrappers for them.  I would like to adjust them so that the gil is released on any call to the methods of these classes.  The options I have considered are

  • Wrapping each call in "with nogil."  This is awkward, because the calls rely on cython's automatic translation of, e.g., python lists to C++ vector<>s, etc., meaning I have to make intermediate variables to do the translation outside the "with nogil" block.
  • A shim class around each C++ class which releases the gil in each wrapped method.
  • Extending the cython compiler with a "nevergil" annotation which if necessary releases and reacquires the gil in the immediate vicinity of each function call, i.e., after all the translated arguments have already been assigned to intermediate variables for me.  I think the right place to output the release and acquisition would be here, inside "if self.nevergil" blocks or something.  then I would just need to annotate the "cdef cpplass" definitions with "nevergil".

I can't be the first person to have faced these choices.  Are there any tools or attempts at option 3 out there already?

Best regards,
Alex

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Agrima Srivastava | 20 Apr 13:14 2016
Picon

Running cython with arguments

This is the main code "helloperson.pyx"import optparse
def main(): p = optparse.OptionParser() p.add_option('--person', '-p', default="world") options, arguments = p.parse_args() print 'Hello %s' % options.person if __name__ == '__main__': main()
The setup.py file is as follows:from distutils.core import setup from Cython.Build import cythonize setup( ext_modules=cythonize("helloperson.pyx"),)


I have used python setup.py build_ext --inplaceto get the helloperson.so fileThis code however takes an argument i.e. the person's name.How do I run the .so file ?

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Mike Busch | 19 Apr 02:31 2016
Picon

Wrapping C code that expects unsigned int **

I'm trying to re-purpose some older C code and use it with my shiny, new Python 3.5 program but passing pointers is giving me headaches. I was able to work my way through a simpler set of function calls but now I seem to have hit a wall.

From the C's .h file:

typedef unsigned long long KEY_T ;
typedef KEY_T * KEY_PTR ;
typedef unsigned int VALUE_T;
typedef VALUE_T * VALUE_PTR ;


and


void createHashTable(KEY_PTR d_keys,VALUE_PTR d_values, unsigned int length, KEY_PTR * d_TK,VALUE_PTR * d_TV,unsigned int * tableLength, unsigned int ** d_bucketSize,unsigned int * bucketCount);


my .pxd:

cdef extern from "common.h":
ctypedef unsigned long long int KEY_T
ctypedef unsigned long long int * KEY_PTR
ctypedef unsigned long long int VALUE_T
ctypedef unsigned long long int * VALUE_PTR

cdef extern from "gpuhash.h":
void createHashTable(
KEY_PTR d_keys,
VALUE_PTR d_values,
unsigned int length,
KEY_PTR * d_TK,
VALUE_PTR * d_TV,
unsigned int * tableLength,
unsigned int ** d_bucketSize,
unsigned int * bucketCount);


and my .pyx:

cimport pygpuhash
#from pygpuhash cimport createHashTable


def create_hash_table(
KEY_T[::1] d_keys,
VALUE_T[::1] d_values,
unsigned int length,
KEY_T[::1] *d_TK, # I've tried a wide variety of things on these 2 lines, including 2D arrays, 2 pointers, etc.
VALUE_T[::1] *d_TV,
unsigned int[::1] tableLength,
unsigned int[:,::1] d_bucketSize,
unsigned int[::1] bucketCount):
return pygpuhash.createHashTable(
&d_keys[0],
&d_values[0],
length,
&d_TK[0], # Can't figure out what this call should be
&d_TV[0],
&tableLength[0],
&d_bucketSize[0,0], # nor this one
&bucketCount[0])

and my errors are:

running build_ext
cythoning pygpuhash
.pyx to pygpuhash.c


Error compiling Cython file:
------------------------------------------------------------
...
       
unsigned int[::1] bucketCount):
   
return pygpuhash.createHashTable(
       
&d_keys[0],
       
&d_values[0],
        length
,
       
&d_TK[0],
       
^
------------------------------------------------------------


pygpuhash
.pyx:20:8: Cannot take address of memoryview slice


Error compiling Cython file:
------------------------------------------------------------
...
   
return pygpuhash.createHashTable(
       
&d_keys[0],
       
&d_values[0],
        length
,
       
&d_TK[0],
       
&d_TV[0],
       
^
------------------------------------------------------------


pygpuhash
.pyx:21:8: Cannot take address of memoryview slice


Error compiling Cython file:
------------------------------------------------------------
...
       
&d_values[0],
        length
,
       
&d_TK[0],
       
&d_TV[0],
       
&tableLength[0],
       
&d_bucketSize[0,0],
       
^
------------------------------------------------------------


pygpuhash
.pyx:23:8: Cannot assign type 'unsigned int *' to 'unsigned int **'






Any help would be appreciated!

Thanks

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Gmane