Tom Swirly | 24 Jun 18:50 2016
Gravatar

compiling Cython files in multiple working directories?

In my project, I have a small amount of actual .pyx code and a huge amount of generated code.  It makes grep essentially useless and is slowing me down fairly seriously.


If this were C++, I'd move all the generated code into a separate hierarchy and just add a -I to my compile line.

Unfortunately, I'm not finding a way to do this in Cython.  

Indeed, all the documentation talks about the working directory, implying that all the .pyx files involved in a compilation are always visible from one single directory room.

----

As usual, I have a tiny demo, and it's here:  https://github.com/rec/simple-cython/blob/master/simple.pyx#L3-L4   Works, but fails if I uncomment that line.

Is there a way I can change just setup.py file to also allow that .pyx in the other directory to be found?

Thanks in advance!

--
     /t

http://radio.swirly.com - art music radio 24/7 366/1000

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Kiyoshi Masui | 21 Jun 18:57 2016
Picon
Gravatar

Calling parent cppclass constructors

Hi,

In the "experimental_cpp_class_def" functionality (https://github.com/cython/cython/blob/master/tests/run/cpp_classes_def.pyx) is it possible to call parent-class constructors in 'cppclass's?

Details:

I'm trying to extend c++ classes in cython using the "experimental_cpp_class_def" functionality before wrapping them. Some example code is as follows:

myclass.hpp:
-----------------

namespace mine {

struct myclass
{
//myclass();
myclass(int a_, int b_);
int a;
int b;
};

}


myclass.cpp
-----------------

//myclass::myclass() {}

myclass::myclass(int a_, int b_)
{
this->a = a_;
this->b = b_;
}


cmyclass.pxd:
-------------------

cdef extern from "myclass.hpp" namespace "mine":

        cdef cppclass myclass:
        #myclass()
        myclass(int a_, int b_)
        int a
        int b


myclass_wrapper.pyx
-----------------------------

cimport cmyclass


cdef cppclass _myclass(cmyclass.myclass):
    __init__(PyObject* tup):
        
       # WOULD LIKE TO CALL myclass::myclass(int, int) HERE.

        this.a = int((<object>tup)[0])
        this.b = int((<object>tup)[1])


cdef class MyClass(object):

    cdef _myclass *thisptr

    def __cinit__(self):
        self.thisptr = new _myclass((5, 5))


I know that in this case I could convert from python objects to integers in the wrapper class, but I have a less trivial application going forward.

My main issue is that I would like to call the base class constructor (myclass::myclass(int, int)) from _myclass.__init__() but I can't figure out if cython has syntax for this.

Another minor this is that this code will not compile with the no-argument-constructors commented out as they currently are. When compiling the cythonized .pyx, I get errors about "no matching function for call to 'mine::MyClass::MyClass()'. This seems similar to the issue discussed here (https://groups.google.com/d/topic/cython-users/ko7X_fQ0n9Q/discussion), but I thought that was fixed (I'm using Cython 0.24).

Thanks,
Kiyo

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Joshua Arnott | 21 Jun 18:22 2016
Picon
Gravatar

Issues with Cython.Coverage

Hello,

I'm attempting to use Coverage.py to provide coverage analysis for a Python package that consists mostly of Cython cdef classes. I've stumbled across some issues using the Cython.Coverage module. Not sure if I'm using it incorrectly or if there are some bugs.

I've set up an demonstration project here:

https://github.com/snorfalorpagus/hello_cython

If you run "test_hello.py" it should run coverage and produce the HTML output.

There are two issues:
  1. Coverage doesn't seem to detect `def/cdef/cpdef` statements within a `cdef class` - they are always marked as not run (red)
  2. The ignore statements in .coveragerc seem to be completely ignored for pyx files. This is seen in the example "meow" function in both hello.pyx (not ignored) and notcython.py (ignored correctly).

I would file a bug report but it seems trac.cython.org isn't working? I've very keen to get this working. I don't have any experience hacking on Cython but am willing to learn if someone can point me in the right direction.


Kind regards,


Josh

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Stephan Hügel | 21 Jun 12:56 2016
Picon
Gravatar

Cython extension wrapping a DLL can't be imported in Windows wheel

Hi everyone,
I'm building a Cython extension ("cutil") which wraps an external library I've written (I'm making it available as a DLL in Windows). I'm including the DLL in the same directory as the module:

- setup.py
- module
-- __init__.py
-- cutil.pyx
-- rlib.h
-- module.dll
-- module.lib

When I run setup.py build_ext --inplace, Cython generates cutil.c, produces a cutil.pyd in the working directory, and my tests pass. However, when I run pip wheel. to produce a wheel, the compiler places the Cython extension alongside the source files (in the module directory), as you would expect, but this extension can't be imported:

======================================================================
ERROR: Failure: ImportError (No module named cutil)
----------------------------------------------------------------------
Traceback (most recent call last):
File "c:\python27.10-x64\lib\site-packages\nose\loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "c:\python27.10-x64\lib\site-packages\nose\importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "c:\python27.10-x64\lib\site-packages\nose\importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "C:\projects\convertbng\convertbng\test\test_convertbng.py", line 15, in <module>
from convertbng.cutil import convert_bng as cconvert_bng
ImportError: No module named cutil
 

I'm not experienced with Windows, but as I understand it, there's no equivalent of a relative path ($ORIGIN on Linux, <at> loader_path on OS X ) which I can pass to extra_link_args? So, is there a standard approach for relative import paths on Windows, or bundling DLLs in general?

(I should add that I can build working wheels for OS X and Linux, using the rpath arguments mentioned above)

Complete build output is here: https://ci.appveyor.com/project/urschrei/convertbng/build/job/fd1fql0ykd0c78pe

Thanks,
Steph

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
ashwinD12 | 18 Jun 09:25 2016
Picon

Unable to run cython on Ubuntu 14.04

I need to rebuild cython with icc compiler.

When I run cython on Ubuntu 14.04 - I get this error

How do I do that ?

Traceback (most recent call last):
  File "/usr/local/bin/cython", line 9, in <module>
    load_entry_point('Cython==0.24', 'console_scripts', 'cython')()
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 351, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2363, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2088, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/usr/local/lib/python3.4/dist-packages/Cython/Compiler/Main.py", line 28, in <module>
    from .Scanning import PyrexScanner, FileSourceDescriptor
ImportError: /usr/local/lib/python3.4/dist-packages/Cython/Compiler/Scanning.cpython-34m.so: undefined symbol: __intel_sse2_strchr

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Richard O | 18 Jun 06:55 2016
Picon

Calling function retrieved via ctypes

Hello,

I am attempting to call, in Cython, a function retrieved from an external shared library via ctypes. The reason is that 'sharelib.so' and 'add' are not determined until runtime. First I verify that everything works with ctypes:
from ctypes import CDLL, c_double, CFUNCTYPE
sharelib
= CDLL('/path/to/sharelib.so')
prototype
= CFUNCTYPE(c_double, c_double, c_double)
sharefunc
= prototype(('add', sharelib))
print(sharefunc(6, 10)) # 16.0

Now I want to call sharefunc from inside a cdef function. My thought was to take advantage of ctypes' addressof() function to get a raw address as an integer in Python, and then cast it to a function pointer on the C side. Here is as far as I've gotten:

from ctypes import CDLL, c_double, addressof, CFUNCTYPE
sharelib
= CDLL('/path/to/sharelib.so')
prototype
= CFUNCTYPE(c_double, c_double, c_double)
sharefunc
= prototype(('add', sharelib))
ctypedef
double (*addfunc)(double x, double y)
cdef
double doubler(double a, double b, unsigned long func):
   
return 2.0 * (<addfunc>func)(a, b)
print(doubler(6, 10, addressof(sharefunc))) # segfault

I am pretty new to Cython so I'm not sure what I'm doing wrong. My sharelib.c looks like this:
double add(double, double);
 
double add(double a, double b) {
   
return a + b;
}
The compile directive was "gcc -shared -o sharelib.so -fPIC sharelib.c"

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Daniel Rozeboom | 15 Jun 21:54 2016
Picon

Providing Runtime Implementation for Externally Defined C Function

Hi all,

   Please forgive my noobishness around this topic, my C / library linking knowledge is very spotty. :-P

   I'm using Cython to build a Python program ("client") that invokes functions of a C library ("utility", declared in various .h files and dynamically linked). However, the utility library is not available in my development environment and so I'd like to be able to (with a flag) optionally build my client to use local "stub" implementations of the utility instead.

   I have been able to build and run the client where the external utility library is available, but I haven't been able to get a local Cython stub implementation working. I keep getting an error of "Symbol not found" ... "Expected in: flat namespace." I've tried to simplify the problem down to the minimal case, so here are some sample files showing what I'm trying to do:

utility.h: (C function declaration)
extern int do_something(unsigned long* value);

utility.pxd: (Cython declaration of function in utility.h)
cdef extern from "utility.h":
    int do_something(unsigned long *value)

utility.pyx: (Stub implementation, only want to have this included in development build)
cdef int do_something(unsigned long *value):
    print('In Stub Implementation')

client.pyx: (Calls externally declared C function, which should be either the dynamically loaded "real" library or the Cython stub implementation)
from utility cimport do_something

def execute():
    print("Calling utility...")
    cdef unsigned long value = 0
    do_something(&value)

setup.py: (Cythonizes Extensions for use with build_ext, either including or excluding the stub .pyx file based on an environment variable)
from os import environ
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize


module_list = [Extension('client', ['client.pyx'])]
if environ.get('STUB'):
    print("Adding stub utility implementation to module list")
    module_list.append(Extension('utility', ['utility.pyx']))
else:
    print("Not including stub utility implementation")

setup(ext_modules=cythonize(module_list))

run.py: (Run the thing)
from client import execute

execute()

   So I want the client to be able to include "utility.pxd" for a single function declaration, which is implemented either by my stub (utility.pyx) or the dynamically linked library. The following are the steps I'm taking to build and run this with the 'STUB' flag enabled, where I get an error rather than the stub implementation running:

$ STUB=1 python3 setup.py build_ext --inplace
$ python3 run.py
Traceback (most recent call last):
  File "run.py", line 1, in <module>
    from client import execute
ImportError: dlopen(<path>/client.so, 2): Symbol not found: _do_something
  Referenced from: <path>/client.so
  Expected in: flat namespace
 in <path>/client.so

   I'm using Python 3.4.4, Cython 0.24 and setuptools 23.0.0. Based on other similar questions I've looked at, I'm guessing something is wrong with the way I'm declaring my Extension objects and/or a library or path environment variable problem for executing run.py. Please let me know if this question makes sense and if you can offer any tips as to how to get this working, or let me know if I'm not structuring this in a way that's appropriate for Cython.

   Thanks in advance for any help someone can offer on this!

- Daniel

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
kai jay | 12 Jun 01:10 2016
Picon

c++ class convect to cython occur an error

The Cython code as below

# distutils: language = c++ # distutils: sources = cppcode.cpp from libcpp.string cimport string cdef extern from "cppcode.h" namespace "mynamespace": cppclass LinkedList[T]: LinkedList () void append (T) int getLength () void remove (int) void printList () int tes () cdef class linklist: cdef LinkedList[string] * listt def __cinit__(self): self.listt = new LinkedList[string]() if self.listt == NULL: raise MemoryError() def __dealloc__(self): if self.listt != NULL: del self.listt cpdef int tests(self): return self.listt.tes() cpdef void appendlist(self,value): return self.listt.append(value)The C++ code as belowusing namespace std; template <class T> void mynamespace::LinkedList<T>::append (T value) { if (head == NULL) { head = new ListNode (value); tail = head; } else { tail->next = new ListNode (value); tail = tail->next; } length++; } template <class T> void mynamespace::LinkedList<T>::printList (void) { if (head != NULL) { ListNode * it = head; while (it != NULL) { cout << it->value << endl; it = it->next; } } } template <class T> int mynamespace::LinkedList<T>::getLength (void) { return length; } template <class T> int mynamespace::LinkedList<T>::tes () { return 10; } template <class T> void mynamespace::LinkedList<T>::remove (int idx) { if (head != NULL) { // invalid element to remove... if (idx < 0 || idx > length) return; if (idx == 0) { ListNode * tmp = head; head = tmp->next; delete tmp; length--; } else { ListNode * it = head; ListNode * parent = it; int i; bool found = false; for (i = 0; i < length; ++i) { if (i == idx) { found = true; break; } parent = it; it = parent->next; } if (found) { parent->next = it->next; delete it; length--; } } } }

I have try to create a class by cython , but when I compile it to .pyd file have an error occur. But when I delete the function (test) and (appendlist) it can pass the compile.

The error message as below:

cpycode.obj : warning LNK4197: export 'PyInit_cpycode' specified multiple times; using first specification Creating library build\temp.win-amd64-3.5\Release\cpycode.cp35-win_amd64.lib and object build\temp.win-amd64-3.5\Release\cpycode.cp35-win_amd64.exp cpycode.obj : error LNK2001: unresolved external symbol "public: int __cdecl mynamespace::LinkedList<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > >::tes(void)" (?tes <at> ?$LinkedList <at> V?$basic_string <at> DU?$char_traits <at> D <at> std <at> <at> V?$allocator <at> D <at> 2 <at> <at> std <at> <at> <at> mynamespace <at> <at> QEAAHXZ) C:\pptemplates\cpycode.cp35-win_amd64.pyd : fatal error LNK1120: 1 unresolved externals error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64\\link.exe' failed with exit status 1120

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Tom Swirly | 14 Jun 18:33 2016
Gravatar

thisptr considered harmful?

Sorry for all the traffic, but I'm doing my second Cython project so a lot of ideas are coming together.

It's very weird to see new() in use in modern C++ code - even if that C++ code is Cython.

I'm wondering why the thisptr idiom has continued to flourish, when it's so much easier to directly use C++ types as member elements in Cython classes?



Seems to work fine for me.  Is there some reason this is bad?  For example, I instrumented the destruction in the second link, and it seems to go off reliably just as expected.


--
     /t

http://radio.swirly.com - art music radio 24/7 366/1000

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Tom Swirly | 14 Jun 18:16 2016
Gravatar

Cython setup.py best practices and sample project

Abstract

Writing new Cython projects would be easier for everyone if we provided a sample project with a fairly complete build, test and distribution system for new developers to just copy and go.


Details

As discussed elsewhere on this list, we all fumble with build and distribution, when the actual code to do this is fairly short.

Testing hasn't come up as much - however, integrating Python tests into a setup.py toolchain is pretty easy (example) and it's a good idea to get people writing tests from the very beginning, particularly if we can throw it in for free.


Commands to setup.py

My very tentative suggestion, in rough workflow order, is that our sample setup.py provide these commands:
  • clean
  • generate
  • build
  • test
  • distribute

Comments on these commands
  • build and distribute aren't the standard command names from setup.py, because I don't think they're going to do exactly the standard things (although very close).  I don't have a strong opinion here, really...

  • Many teams seem to have some sort of generated file step, so I included it, though it should be blank by default.  It costs nothing to have it there, unused, and will provide a guideline of "how to do it".  Not entirely sure about this idea either.  :-)

  • Running Python unit tests as part of this is quite easy but the one subtlety is getting the Cython-generated .so into a directory in the PYTHONPATH or sys.path.  There are two or three choices here for a solution which we can discuss.  I'm pretty positive about this part...

  • We already talked a bit about the distribute command, which is sort of like sdist and bdist glued together with the generated .cpp code.  More work is needed!  My ignorance is great here.

Comments overall

We can probably do a pretty good job of this without very much work at all.

I think we should support Python 3 and setuptools only to keep the scope of the project to a minimum.  Perhaps others who still use Python 2 will contribute if we get to that.

--
     /t

http://radio.swirly.com - art music radio 24/7 366/1000

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Colin Pattison | 13 Jun 23:32 2016
Picon

cimport results in struct declaration being copied into another location

Hey Guys - 


I’ve just started using Cython after having handwritten extensions for a while. I’ve run into an issue and would appreciate any insight you guys have - I have a common pxd which wraps our standard data representation. I then have module B built separately that cimports the data representation. This then results in a redefinition error as the struct DataPy appears in the header for the common module and then is redefined in the cpp of module B. Is there any way around this?


A replication of this is attached - build the common module first using the setup.sh, then build the session module using the setup.sh. DataPy will be declared in common.h and then redeclared in session.cpp.


Is there any way to avoid this happening?


Cheers

C

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Attachment (redef.tgz): application/x-gtar, 1130 bytes

Gmane