john | 26 Feb 19:10 2015
Picon

cythonise changes my import paths - erroneously

Hi

I am having trouble with Cython. I have inherited a Cython app and just about kept it building and running for the past year but today it has me absolutely stumped. It has started prepending the current (project root) directory to imports so that when they run the module cannot be found and importing fails. I will give a little more detail below:

projdir/kernel.pyx includes the following import line:


from component cimport Component


In the projdir directory, I run cythonize on kernel.pyx as follows:

extensions = [

Extension("kernel", ["kernel.pyx"],

include_dirs = [get_include()]),

]


setup(

ext_modules = cythonize(extensions)

)


and for the import it yields the followin in kernel.c:


  __pyx_ptype_7bastard_9component_Component = __Pyx_ImportType("projdir.component", "Component", sizeof(struct __pyx_obj_7bastard_9component_Component), 1); if (unlikely(!__pyx_ptype_7bastard_9component_Component)) {__pyx_filename = __pyx_f[1]; __pyx_lineno = 3; __pyx_clineno = __LINE__; goto __pyx_L1_error;}


Note the appearance of "projdir.". When it runs, I get the expected error:


ImportError: No module named projdir.component


I can make this work by changing the string to just "component", recompiling, linking and running the python. But why is this happening? I understand that Cython is trying to support post-compile dynamic importing, but surely it has no business producing any path other than what I have in my original .pyx source file? What is the algorithm in cythonize() that is modifying these import paths? Where is the documentation for it? Why has this been working for ages then suddenly gone wrong?


--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
genadijrazdorov | 26 Feb 08:57 2015
Picon

int argument not working

Hi guys,

I just started with cython, and have I problem with int arguments:


def helloworld(int n):
return n

I am on Anaconda python3 win7 64bit system.
distutils build and install works, but when I call halloworld with integer in IPython I get:

TypeError: __int__ returned non-int (type int)

If I change int declaration to double helloworld is working.

Any suggestions?

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Hai Nguyen | 26 Feb 04:39 2015
Picon

Linking time is too slow

hi all, i have been using Cython for wrapping a C++ lib (for data analysis of bimolecular MD simulation). 

https://github.com/pytraj/pytraj (mostly written in Cython)

I think I am in love with Cython (still prefer Cython over Julia). The only problem is the compiling/linking time is too slow (about 4 times slower than compiling the original C++ lib). It takes about 15 minutes to compile. 

So my question is: is there any trick/flag to make this faster? 

Thanks for paying attention to my naïve question. 

Hai

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Curtis M. Faith | 26 Feb 00:19 2015
Picon

Why is it necessary to call auto-generated Cython import functions more than once?

I'm working on the Python bindings for the open-source project OpenCog which are written in Cython. There are two auto generated import functions:

    import_opencog__atomspace();
    import_opencog__agent_finder();

defined in the corresponding api header files.

I appear to need to call the auto-generated import functions several times to avoid crashes when calling into the Cython code marked as "api". I assumed I'd only need to run the code once upon initialization, however, if I only make the calls on startupm I get crashes in code that calls into the Cython defined api later on. The crashes go away if I put another call to:

    import_opencog__agent_finder();

before the Cython api is called from a C++ function in another linux shared library. Why is it necessary to call these import functions more than once?

- Curtis

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Luc Bourhis | 22 Feb 02:02 2015
Picon

Warning: Non-trivial type declarators in shared declaration

Could someone tell me how to disable this warning?

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Rudra Banerjee | 26 Feb 00:10 2015

compile cython generated C code

Hi,

I am very new cython user. My main purpose of using cython is to run the generated C code with fortran.
hence I have generated the C code from pyx and compiled the C code using distutils to generate .so file.

I have tried that so file to be called directly from fortran, which is not working.

So I tried to check the C code itself, with the hope that I can use fortran's C-interoperability.
But the C code is not compiling, giving long list of undefined reference.

I have compiled it using:

gcc  -pthread -fPIC -fwrapv -O2 -Wall -fno-strict-aliasing -I/usr/include/python2.7/ src/parse.c 
 
https://drive.google.com/file/d/0B02WblgE6NEvSl82dGp5d3FHdDg/view?usp=sharing  is the pyx file in question.

It will be of immense help if someone kindly show me the way.
Regards,
RB

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jeroen Demeyer | 25 Feb 14:40 2015
Picon

Accessing Python long type?

Is there any way in Cython to access the Python "long" type?

Essentially, what I'm trying to do is

cdef extern from "longintrepr.h":
     cdef long PyLong_SHIFT
     ctypedef unsigned int digit
     ctypedef class __builtin__.long [object PyLongObject]:
         cdef digit* digits

cdef extern from "Python.h":
     cdef _PyObject_NewVar(type t, Py_ssize_t size)

def foo(s):
     cdef long L = _PyObject_NewVar(long, s)

The problem is that Cython always thinks that "long" means C long, while 
I really want Python long. Is there any solution for this?

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Curtis M. Faith | 24 Feb 17:47 2015
Picon

Crash calling Cython generated api function from C++

I'm helping fix the Python bindings for the Opencog open source framework and am getting a new crash that I can't figure out. Hopefully one of you has seen something similar before and can point me in the right direction.

In the process of trying to consolidate the Python initialization code into one place, I'm now getting a SIGSEGV fault attempting to deference zero in an existing cython api call that was working fine before. The odd thing is that I haven't changed the calling code, nor the .pyx code which defines the cython function py_atomspace. The crash is coming with just a couple of assembler instruction steps on the callq., so it's not crashing in py_atomspace, it never gets there. Like the dynamic linking isn't being done properly ???

I'm running the standard VM for the docker install using boot2docker in a Mac OS X VirtualBox VM with the standard Dockerfile for Opencog:

Ubuntu 14.04.1 LTS

3.16.7-tinycore64

Python 2.7.6

GCC 4.8.2

Cython version 0.20.1post0


The change I made which surfaced this crash is that I am now calling PythonEval::init() earlier than it was getting called before. 

See: https://github.com/opencog/opencog/blob/master/opencog/cython/PythonEval.cc#L49-L125 for the relevant code lines in Opencog.

The cython function is quite simple. The caller is:

PyObject * pyAtomSpace; if (atomspace) pyAtomSpace = py_atomspace(atomspace);

from:

https://github.com/opencog/opencog/blob/master/opencog/cython/PythonEval.cc#L148

where it crashes on the last line which calls into a cython routine:

(gdb) si 0x00007ffff6354b69 160 pyAtomSpace = py_atomspace(atomspace); 0x00007ffff6354b66 <routine+6>: 48 89 f7 mov %rsi,%rdi => 0x00007ffff6354b69 <routine+9>: ff 15 29 69 20 00 callq *0x206929(%rip) # 0x7ffff655b498 <_ZL40__pyx_f_7opencog_9atomspace_py_atomspace> 0x00007ffff6354b6f <routine+15>: 48 89 c3 mov %rax,%rbx (gdb) si [2015-02-24 01:43:14:072] [INFO] PythonEval atomspace 1 0x0000000000000000 in ?? () => 0x0000000000000000: Cannot access memory at address 0x0

The py_atomspace routine is quite simple, it wraps the C++ object with a python object with the following cython code:
cdef api object py_atomspace(cAtomSpace *c_atomspace) with gil: cdef AtomSpace atomspace = AtomSpace_factory(c_atomspace) return atomspace

from: https://github.com/opencog/opencog/blob/master/opencog/cython/opencog/atomspace_details.pyx#L407-410

Even if I make the call right after the auto generated cython import routine as below, I still get this crash.

    // Import Python atom space routines. Test py_atomspace cython wrapper.
    import_opencog__atomspace
();
   
PyObject* testPyAtomSpace = py_atomspace(NULL);

I've posted a few more details including memory dumps and single steps in assembler into the crash on Stack Overflow:

http://stackoverflow.com/questions/28687248/debugging-crash-in-cython-code-called-from-c

Here's the cython generated .cpp code for py_atomspace which is not getting called:

/* "/home/opencog/src/opencog/opencog/cython/opencog/atomspace_details.pyx":407
 *         self.atomspace.print_list()
 *
 * cdef api object py_atomspace(cAtomSpace *c_atomspace) with gil:             # <<<<<<<<<<<<<<
 *     cdef AtomSpace atomspace = AtomSpace_factory(c_atomspace)
 *     return atomspace
 */



static PyObject *__pyx_f_7opencog_9atomspace_py_atomspace(opencog::AtomSpace *__pyx_v_c_atomspace) {
 
struct __pyx_obj_7opencog_9atomspace_AtomSpace *__pyx_v_atomspace = 0;
 
PyObject *__pyx_r = NULL;
  __Pyx_RefNannyDeclarations
 
PyObject *__pyx_t_1 = NULL;
 
int __pyx_lineno = 0;
 
const char *__pyx_filename = NULL;
 
int __pyx_clineno = 0;
 
#ifdef WITH_THREAD
 
PyGILState_STATE __pyx_gilstate_save = PyGILState_Ensure();
 
#endif
  __Pyx_RefNannySetupContext
("py_atomspace", 0);


 
/* "/home/opencog/src/opencog/opencog/cython/opencog/atomspace_details.pyx":408
 *
 * cdef api object py_atomspace(cAtomSpace *c_atomspace) with gil:
 *     cdef AtomSpace atomspace = AtomSpace_factory(c_atomspace)             # <<<<<<<<<<<<<<
 *     return atomspace
 *
 */

  __pyx_t_1
= __pyx_f_7opencog_9atomspace_AtomSpace_factory(__pyx_v_c_atomspace); if (unlikely(!__pyx_t_1)) {__pyx_filename = __pyx_f[1]; __pyx_lineno = 408; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __Pyx_GOTREF
(__pyx_t_1);
 
if (!(likely(((__pyx_t_1) == Py_None) || likely(__Pyx_TypeTest(__pyx_t_1, __pyx_ptype_7opencog_9atomspace_AtomSpace))))) {__pyx_filename = __pyx_f[1]; __pyx_lineno = 408; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
  __pyx_v_atomspace
= ((struct __pyx_obj_7opencog_9atomspace_AtomSpace *)__pyx_t_1);
  __pyx_t_1
= 0;


 
/* "/home/opencog/src/opencog/opencog/cython/opencog/atomspace_details.pyx":409
 * cdef api object py_atomspace(cAtomSpace *c_atomspace) with gil:
 *     cdef AtomSpace atomspace = AtomSpace_factory(c_atomspace)
 *     return atomspace             # <<<<<<<<<<<<<<
 *
 */

  __Pyx_XDECREF
(__pyx_r);
  __Pyx_INCREF
(((PyObject *)__pyx_v_atomspace));
  __pyx_r
= ((PyObject *)__pyx_v_atomspace);
 
goto __pyx_L0;


 
/* "/home/opencog/src/opencog/opencog/cython/opencog/atomspace_details.pyx":407
 *         self.atomspace.print_list()
 *
 * cdef api object py_atomspace(cAtomSpace *c_atomspace) with gil:             # <<<<<<<<<<<<<<
 *     cdef AtomSpace atomspace = AtomSpace_factory(c_atomspace)
 *     return atomspace
 */



 
/* function exit code */
  __pyx_L1_error
:;
  __Pyx_XDECREF
(__pyx_t_1);
  __Pyx_AddTraceback
("opencog.atomspace.py_atomspace", __pyx_clineno, __pyx_lineno, __pyx_filename);
  __pyx_r
= 0;
  __pyx_L0
:;
  __Pyx_XDECREF
((PyObject *)__pyx_v_atomspace);
  __Pyx_XGIVEREF
(__pyx_r);
  __Pyx_RefNannyFinishContext
();
 
#ifdef WITH_THREAD
 
PyGILState_Release(__pyx_gilstate_save);
 
#endif
 
return __pyx_r;
}

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Michael Hamann | 24 Feb 15:52 2015

Call operator in experimental cpp class definition

Hi,

after discovering the experimental support for defining and implementing cpp 
classes in Cython I thought this could be a really nice feature for wrapping 
Python callbacks in cpp objects so they can be passed to functions accepting 
cpp callbacks.

However either I did not understand how to declare operator() in cpp classes 
or there is a bug/missing feature.

Example:

# distutils: language = c++
# cython: experimental_cpp_class_def=True

cdef cppclass CallbackWrapper:
        void* callback
        __init__(object callback):
                this.callback = <void*>callback
        void operator()(int y):
                cdef object myCallback = <object>callback
                myCallback(y)

This code gives two errors for the line with operator():

Function cannot return a function
Compiler crash in ControlFlowAnalysis

Note that __call__ (I thought maybe this is similar to __init__) doesn't work 
either as it doesn't seem to have any special meaning in this context.

I'm using Cython version 0.22.

I managed to get a working example by replacing "operator()" by some name that 
is replaced by operator() later using a macro:

## test.h:

#include <iostream>

#define cython_call_operator operator()

class TestClass {
public:
        template <typename F>
                void callF(F &f) {
                        std::cout << "Calling f with 5" << std::endl;
                        f(5);
                }
};

## test.pyx:

# distutils: language = c++
# cython: experimental_cpp_class_def=True

from cython.operator import dereference

cdef extern from "test.h":
        cdef cppclass _TestClass "TestClass":
                void callF[F](F &f)

def callbackFunc(y):
        print(y)

cdef cppclass CallbackWrapper:
        void* callback
        __init__(object callback):
                this.callback = <void*>callback
        void cython_call_operator(int y):
                cdef object myCallback = <object>callback
                myCallback(y)

cdef _TestClass myTest

cdef CallbackWrapper* wrapper = new CallbackWrapper(callbackFunc)
myTest.callF(dereference(wrapper))

Btw. this is just a "hello world" example of course, in reality I want to 
provide a Python wrapper for "TestClass" that transparently instantiates the 
wrapper object for the Python callback and passes it to the underlying cpp 
method. Note that memory management is no issue here as the callback won't be 
used anymore after the cpp method has returned.

Thanks in advance,
Michael

PS: What exactly is the status of this "experimental" cpp class definition 
feature?

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Matthew Honnibal | 22 Feb 22:24 2015
Picon

Idea: implicit "self" for function pointers in structs

Hi,

I'd like to propose a new piece of syntactic sugar. I thought I'd see what people thought about this before I put in the effort to write a proper CEP.

A C struct can contain one or more function pointers. This has long been used in "object oriented C" code, as a kind of light-weight class. I find myself sometimes wanting to do this, even though Cython allows cdef classes. The appeal is:

* Simple semantics. There's no interaction with the Python ref counting, vtable, etc. There's no implicit life-cycle.
* You can store the struct instances in C data structures, such as arrays and vectors.
* You can manually manage data locality.

So, the advantage is it's just a function pointer --- nothing special. The downside is the syntax for calling the function:

cdef struct Foo:
    int (*bar)(Foo* self) except -1
    int _data

cdef int bar1(Foo* self) except -1:
    return self._data

cdef int bar2(Foo* self) except -1:
    return self._data ** 2

def main():
    cdef Foo foo1 = Foo(bar=bar1, _data=2)
    cdef Foo foo2 = Foo(bar=bar2, _data=2)

    print foo1.bar(&foo1)
    print foo2.bar(&foo2)


The explicit passing of the "self" argument is verbose, and really stands out in Python code. And it's brittle --- if I swap some variable names around, I might call pass the wrong instance.

My idea is to apply a little bit of syntactic sugar. If a struct has a function pointer member, and the first argument is named "self", and typed as a pointer to the struct type, then Cython will automatically fill in the argument.

The Cython code:

    foo1.bar()

Would generate the C code:

    foo1.bar(&bar)

Advantages
----------------

This small piece of syntactic sugar would address the main situation in which I'm tempted to write a native C++ extension. Lots of small Python objects are a problem for performance sensitive code. I can't put them in an array or vector, and the Python list is slow and memory intensive. If I want to work in a block where I'm trying to release the GIL, Python objects are particularly bad.

But I do want something like a class, in some of these situations. When these performance-sensitive blocks have to execute complicated logic, I need to pair up data and functions in some way. I can elaborate on the situations when I need this, if asked.

The other way to address the need is to allow weak references to Python C extensions. I noticed Stefan raises this idea on his website somewhere. I think my proposal is much more light-weight, but is still enough to get the job done in most situations.

I think the implicit self argument can be implemented with full backwards compatibility. Calls to the function will be unambiguous, because you can't have a variable number of pointer arguments.

Disadvantages
--------------------

It's not Python and it's not C. It's a Cython-specific thing, so therefore may be surprising.
 
There's substantial functionality overlap between this and cdef classes. Extending support for "C with classes" introduces another way to write certain pieces of code.

It might be confusing to Python programmers new to C and Cython. I can imagine finding the distinction between a struct with a function pointer and a C extension type to be quite subtle, at first glance.


Thoughts?

---

Matthew Honnibal
http://honnibal.github.io/spaCy/



 

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Nils Bruin | 22 Feb 10:00 2015
Picon

Improve introspection of cython functions

If you have a python function, you can query it and piece together how many positional arguments it expects and what keyword names it accepts:

sage: def g(a,b=0):pass
sage: g.__code__.co_argcount
2
sage: g.__code__.co_varnames
('a', 'b')
sage: g.func_defaults
(0,)

If the same function is compiled with cython, the restriction on the accepted argument formats is the same, but the data cannot be pried from the <type 'builtin_function_or_method'> object.
Would it be possible to expose this data either in the docstring or on some attributes on cython function objects?

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Gmane