havsula | 23 Dec 10:38 2015
Picon

Make .lib file for cython with Visual C++ for Python

Hello
I have some c-code I want to include in Python by using Cython. Name of the c-file is cmean.c. I also have a cmean.h which define the function in cmean.c.
Have the following c-wrapper file:

# m.pyx
cdef extern from "cmean.h":
    double mean(int, double*)

from stdlib cimport *
def cmean(a):
    n = len(a)
    cdef double *v
    v = malloc(n*sizeof(double))
    for i in range(n):
        v[i] = float(a[i])
    m = mean(n, v)
    free(v)
    return m



And this setup.py
from setuptools import Extension
from Cython.Build import cythonize
from Cython.Distutils import build_ext

ext_modules=[Extension("lib",
    sources = ["m.pyx"],
    libraries=['cmean']
    )] # Unix-like specific

setup(
    name = "Demos",
    ext_modules = cythonize(ext_modules),
    cmdclass = {'build_ext':build_ext}
    )

But I need to make the cmean.c into a library which Cython can use. I have used the following commands
cl.exe /c cmean.c
lib.exe /out:cmean.lib cmean.obj
This makes a cmean.lib file but when I try to run setup.py I got the following error:

E:\GD\UD\Software\CythonTest\calling-c\test2>python setup.py build_ext --inplace
running build_ext
building 'lib' extension
creating build
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
C:\Users\trofl\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\Anaconda2\include -IC:\Anaconda2\PC /Tccfib.c /Fobuild\temp.win32-2.7\Release\cfib.obj
cfib.c
creating build\lib.win32-2.7
C:\Users\trofl\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\Anaconda2\libs /LIBPATH:C:\Anaconda2\PCbuild /LIBPATH:C:\Anaconda2\PC\VS9.0 /EXPORT:initlib build\temp.win32-2.7\Release\cfib.obj /OUT:build\lib.win32-2.7\lib.pyd /IMPLIB:build\temp.win32-2.7\Release\lib.lib /MANIFESTFILE:build\temp.win32-2.7\Release\lib.pyd.manifest
LINK : error LNK2001: unresolved external symbol initlib
build\temp.win32-2.7\Release\lib.lib : fatal error LNK1120: 1 unresolved externals
error: command 'C:\\Users\\trofl\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\link.exe' failed with exit status 1120

So there is something I do wrong when making the .lib file.

I have no trouble compiling .pyx files.


--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Gerald Dalley | 22 Dec 17:38 2015
Picon

bytecode lambdas?

Is there a way to tell Cython to use bytecode-based lambdas instead of making C lambdas?  I'd be happy with either a whole-file directive or some syntax or comment that could be added to individual lambdas.

Motivation: I'm attempting to Cythonize some pure Python code that uses a distributed processing framework.  We use lambdas and inner functions as convenient ways to (a) make partial functions and (b) specify custom operations that occur on the cluster.  We have ways of pickling up bytecode-based lambdas and inner functions, but unsurprisingly the Cython functions can't be pickled.  

Overly simplified example:

def get_remote_work(obj):
    def aggregate(obj, chunk):
        agg  = obj.identity()
        data = obj.load_data(chunk)
        for datum in data:
            agg += datum
        obj.write_agg(chunk, agg)
    chunk_sum = lambda chunk: obj.load_agg(chunk).sum()
    return aggregate, chunk_sum

def do_stuff(obj, chunks):
    for chunk in chunks:
        for fcn in get_remote_work(chunk):
            # launch_remote_task only takes parameterless functions
            launch_remote_task(lambda fcn: fcn(chunk))

The workarounds that come to mind are:
  • Don't use Cython.
  • Move the lambdas out to non-cythonized pure Python modules (hurts code locality and turns compact lambdas into multi-line declarations + imports).
  • Make the lambdas be top-level classes (adds a lot of code bloat and hurts code locality).
  • Implement lambda pickling in Cython's code generator (rather tricky, esp. when choosing what approximations to use when reconstructing closures at unpickle time if we want to support decorators, recursion, co-recursion, etc.).

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jeroen Demeyer | 22 Dec 12:11 2015
Picon

#define __PYX_HAVE... macros in Cython sources?

Hello,

I am looking at the #includes part of a typical Cython-generated file in 
Sage:

[...]
#define __PYX_HAVE__sage__ext__interpreters__wrapper_rr
#define __PYX_HAVE_API__sage__ext__interpreters__wrapper_rr
#include "string.h"
#include "stdio.h"
[...]

My question is about the first two lines: why are they there?

Those lines are causing compiler problems when the Cython file includes 
C code which needs "cdef public" functions implemented in Cython. 
Removing those 2 lines makes the compilation succeed without problems.

Jeroen.

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Zheng Fang | 21 Dec 13:46 2015
Picon

fatal error: Python.h: No such file or directory


When I was "python setup.py install --user" to setup the cython, I came into the problem:"/home/dir/fz/numpy/Cython-0.23.4/Cython/Plex/Scanners.c:4:20: fatal error: Python.h: No such file or directory"

The error is from "gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c /home/dir/fz/numpy/Cython-0.23.4/Cython/Plex/Scanners.c -o build/temp.linux-x86_64-2.7/home/dir/fz/numpy/Cython-0.23.4/Cython/Plex/Scanners.o"


However, since the dir of "/usr/include/python2.7" is out of my access, my python is installed in another path. I want to change the  parameter of  -I in gcc, but I cannot find the command sentence. Who can help me fix the problem?


Thank you sincerely

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Richard Foltyn | 19 Dec 12:34 2015
Picon

Avoid temporary variables in loops and array acces?

Hi list,

I noticed that in generated C code, the Cython compiler inserts temporary variables for loop variables and array (typed memoryview) access, which
are declared as long and Py_ssize_t, respectively. The contents of the "actual" variable is then copied into the temporary whenever array elements are accessed.

I have code that loops over all indices of 4-dimensional arrays, and thus these copy operations happen a few billion times in total during code execution.
Hence my question is:
1) Should I be worried about these copy operations, or does GCC optimize away the temporary variables anyways (say with -O2) if I declare my variables of type Py_ssize_t?
2) If I declare my indexing variables as Py_ssize_t in my Cython code, is the creation of temporary variables of the same type not redundant and could be skipped entirely
by the Cython compiler?

To illustrate my point, here is a simple example where I use Py_ssize_t types:

import numpy as np

def foo():

    cdef
long N = 10, M = 10
    cdef
Py_ssize_t i, j

    cdef
double[:,::1] arr = np.random.rand(M, N)
    cdef
double s = 0.0

   
for i in range(M):
       
for j in range(N):
            s
+= arr[i, j]

in the generated C code I then find things like:

__pyx_t_12 = __pyx_v_i;
__pyx_t_13
= __pyx_v_j;
__pyx_v_s
= (__pyx_v_s + (*((double *) ( /* dim=1 */ ((char *) (((double *) ( /* dim=0 */ (__pyx_v_arr.data + __pyx_t_12 * __pyx_v_arr.strides[0]) )) + __pyx_t_13)) ))));

Thanks,
Richard

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
zhe qu | 19 Dec 11:02 2015
Picon

char[] in the struct

hi,all,this is my code:
from libc cimport stdlib
from libc.string cimport strncpy

cdef struct DemoField:
char[10] Field

def func():
cdef DemoField *df
b = "abcddd"
strncpy(df.Field,b.encode(),2)

why I got a Segmentation fault? thank you!

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Sergei Lebedev | 17 Dec 16:21 2015
Picon
Gravatar

Segfault when using memoryviews with NumPy and nogil

Hello, 

I've encountered a SIGSEGV when passing NumPy array to a nogil-function which accepts a memory view. The minimal crashing example looks like this:

    cimport numpy as np

    import numpy as np

    ctypedef np.float64_t dtype_t


    cdef dtype_t f(dtype_t[:] X) nogil:
        pass


    def main():
        cdef np.ndarray[dtype_t, ndim=1] work_buffer = np.zeros(4)
        with nogil:
            f(work_buffer)

Changing the type of ``work_buffer`` to ``dtype_t[:]`` resolves the issue.

I'm using Python3.4.3 on OS X. NumPy 1.10.1. Cython 0.23.4. 

Cheers,
Sergei

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
zhe qu | 17 Dec 15:33 2015
Picon

PyThreadState_Get: no current thread

hi,I try to call a DLL file use cython, all dll api have independent working threads.
when I use time.sleep in my code after call the dll api, the python environment crash down, and show this error:Fatal Python error: PyThreadState_Get: no current thread
how to avoid this, thank you very much!

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Björn Dahlgren | 18 Dec 00:17 2015
Picon
Gravatar

Handling unique_ptr factories in C++ class

So I have a C++ class which may produce unique_ptr instances based on other instances.
I am trying to wrap my head around how to best interface this from Cython, here is an attempt which (apparently) works:

https://gist.github.com/anonymous/fa348100c58052044c7c

But perhaps there is a more elegant solution? Note that the clone() function in reality is considerably more complex (makes zero sense for this trivial example).
Any thoughts?

Best regards,
Björn

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Antony Lee | 16 Dec 22:00 2015
Picon

cythonize should recompile on dependency changes

Currently, I believe that cythonize only tracks changes to the main .pyx file to decide whether to recompile a .pyx into a .c.  It would be nice if it also tracked changes to cimported pxds (as they can certainly affect the result of the compilation).
Thanks,
Antony

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Antony Lee | 16 Dec 21:58 2015
Picon

tempita

Hi,
I see that Cython vendorizes the Tempita templating engine, but I didn't see any reference to it other than a short discussion on scipy-dev in 2011 and a use in statsmodels (using an ad hoc version of cythonize).
It could probably help me solve some issues, so any help about how to run it would be welcome.
Antony

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Gmane