Alex V. | 14 May 08:25 2015
Picon

how to wrap Qt class using cython

Having a Qt class that declares some signals and slots how do I wrap those using cython?

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Debesh Mohanty | 14 May 06:57 2015
Picon

cython: defined but not used error

Hello

I new to cython. I am facing a problem in a very simple code and I need help as I am not able to figure out what the error is

I wrote a file t3.pyx as

from libc.stdlib cimport atoi

cdef int fun(char *s):
    return atoi(s)

The setup.py is

from distutils.core import setup
from Cython.Build import cythonize

setup(ext_modules=cythonize("t3.pyx"))


The warning I get when I rin setup.py with the command ' python setup.py build_ext --inplace' is

Cythonizing t3.pyx
running build_ext
building 't3' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c t3.c -o build/temp.linux-x86_64-2.7/t3.o
t3.c:556:12: warning: ‘__pyx_f_2t3_fun’ defined but not used [-Wunused-function]
 static int __pyx_f_2t3_fun(char *__pyx_v_s) {
            ^
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/t3.o -o /home/debesh/Documents/cython/t3/t3.so



If I still run the code in python interpreter I get

>>> import t3
>>> t3.fun('123')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'fun'
>>>

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Nikolaj van Omme | 13 May 16:06 2015
Picon

Re: How to deal with several types in a class?


On 15-05-13 12:45 AM, Robert Bradshaw wrote:
> On Tue, May 12, 2015 at 11:37 AM, Nikolaj van Omme
> <nikolaj.van.omme <at> gmail.com> wrote:
>> Hello,
>>
>> I'm writing a library that implements its own sparse matrices. We
>> decided to implement everything in Cython.
>>
>> We have several extension classes for different sparse matrix types. For
>> instance:
>>
>> cdef class LLSparseMatrix(SparseMatrix):
>>     cdef:
>>          FLOAT64_t * val
>>          INT32_t * col
>>
>>     def __cinit__(self, **kwargs):
>>          val = <FLOAT64_t *> PyMem_Malloc(self.size_hint *
>> sizeof(FLOAT64_t))
>>             if not val:
>>                 raise MemoryError()
>>             self.val = val
>>
>>             col = <INT32_t *> PyMem_Malloc(self.size_hint * sizeof(INT32_t))
>>             if not col:
>>                 raise MemoryError()
>>             self.col = col
>>     ...
>>
>> where FLOAT64_t and INT32_t are ctypedef types.
>>
>> This LLSparseMatrix has INT32_t for index type and FLOAT64_t type for
>> its element but we would like to offer the possibility to use other
>> types at run time.For instance, INT64_t for indices and UINT64_t for the
>> element types. The user could give the index type "itype" and element
>> type "dtype" as arguments to the constructor:
>>
>> my_matrix = LLSparseMatrix(itype=INT32_t, dtype=FLOAT64_t, ...)
>>
>> Access to the elements of the internal "val" and "col" arrays in Cython
>> should be efficient.
>>
>> Here is my question: what is the best (most efficient (wrt time) and at
>> the same time not too difficult to maintain) solution if I want to use
>> different types at run time?
>>
>> We can use a template language (like Jinja2) to generate Cython code, we
>> can use numpy arrays for "val" and "col" and use numpy types, we
>> **don't** want to implement the library in C++ with templates.
>>
>> Any help must appreciated. Thanks in advance.
> Personally, I would go with NumPy.
>
Thanks for your reply.

How would you use the NumPy arrays?

Something along these lines?

==================
cimport numpy as cnp
import numpy as np

cnp.import_array()

cdef Py_ssize_t SIZE_HINT = 100

cdef class LLSparseMatrix:
    cdef:
        cnp.ndarray  val
        cnp.ndarray col     

    def __cinit__(self, **kwargs):
        size_hint = kwargs.get('size_hint', SIZE_HINT)
        dtype = kwargs.get('dtype', np.double)
        itype = kwargs.get('itype', np.int)
        self.val = np.empty(dtype=dtype, shape=(size_hint,))
        self.col = np.empty(dtype=itype, shape=(size_hint,))

==================

and the user would be able to create a matrix like this:

==================
matrix = LLSparseMatrix(dtype=np.float, size_hint=10000)
==================

and fill the matrix later on. Attached, you'll find a minimal example.
And here is my test.py script:

==================
import base_case as bc

import numpy as np

DTYPE = np.int

size = 60

B = bc.LLSparseMatrix(nrow=size, ncol=size, dtype=np.double, itype=np.int)

for i in xrange(size):
    for j in xrange(size):
        B[i,j] = 9.99

==================

I've tried exactly this approach and it's about 300 times slower than
with C arrays for the basic operations we need. Maybe, I'm not using the
NumPy arrays correctly/efficiently?

Thank you in advance,

Nikolaj

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
cimport numpy as cnp
import numpy as np

cnp.import_array()

LL_MAT_INCREASE_FACTOR = 1.5

cdef class LLSparseMatrix:
    cdef:
        free      # index to first element in free chain
        cnp.ndarray  val       
        double  *ival      # pointer to array of imaginary values NOT USED here
        cnp.ndarray col       
        cnp.ndarray link      
        cnp.ndarray root      

        public size_hint
        public nalloc
        public nnz
        public bint is_symmetric
        public bint is_complex
        public bint store_zeros
        public nrow
        public ncol

        
    def __cinit__(self, **kwargs):

        self.nrow = kwargs['nrow']
        self.ncol = kwargs['ncol']
        size_hint = kwargs.get('size_hint', 510)
        self.size_hint = size_hint
        self.nalloc = size_hint
        self.free = -1
        self.nnz = 0

        self.val = np.empty(dtype=np.double, shape=(size_hint,))
        self.col = np.empty(dtype=np.int, shape=(size_hint,))
        self.link = np.empty(dtype=np.int, shape=(size_hint,))
        self.root = np.empty(dtype=np.int, shape=(self.nrow, ))
        self.root.fill(-1)

        self.is_symmetric = False
        self.is_complex = False
        self.store_zeros = False

         
    cdef _realloc(self, nalloc_new):
        self.val.resize(nalloc_new)
        self.col.resize(nalloc_new)
        self.link.resize(nalloc_new)

        self.nalloc = nalloc_new

    cdef _realloc_expand(self):

        assert LL_MAT_INCREASE_FACTOR > 1.0
        real_new_alloc = int(LL_MAT_INCREASE_FACTOR * self.nalloc) + 1

        return self._realloc(real_new_alloc)

    cdef put(self, i, j, value, imaginary = 0.0):
        if self.is_symmetric and i < j:
            raise IndexError('Write operation to upper triangle of symmetric matrix not allowed')

        # Find element to be set (or removed)
        col = last = -1
        k = self.root[i]
        while k != -1:
            col = self.col[k]
            # TODO: check this
            if col >= j:
                break
            last = k
            k = self.link[k]

        # Store value
        if self.store_zeros or (value != 0.0 or imaginary != 0.0):
            if col == j:
                # element already exist
                self.val[k] = value
                if self.is_complex:
                    self.ival[k] = imaginary
            else:
                # new element
                # find location for new element
                if self.free != -1:
                    # use element from the free chain
                    new_elem = self.free
                    self.free = self.link[new_elem]

                else:
                    # append new element to the end
                    new_elem = self.nnz

                # test if there is space for a new element
                if self.nnz == self.nalloc:
                    # we have to reallocate some space
                    self._realloc_expand()

                self.val[new_elem] = value
                if self.is_complex:
                    self.ival[new_elem] = imaginary

                self.col[new_elem] = j
                self.link[new_elem] = k

                if last == -1:
                    self.root[i] = new_elem
                else:
                    self.link[last] = new_elem

                self.nnz += 1

        else:
            # value == 0.0 and imaginary == 0.0:
            # if element exists but we don't store zero elements
            # we need to "zeroify" this element
            if col == j:
                # relink row i
                if last == -1:
                    self.root[i] = self.link[k]
                else:
                    self.link[last] = self.link[k]

                # add element to free list
                self.link[k] = self.free
                self.free = k

                self.nnz -= 1

    cdef safe_put(self, i, j, value, imaginary = 0.0):
        if i < 0 or i >= self.nrow or j < 0 or j >= self.ncol:
            raise IndexError('Indices out of range')

        self.put(i, j, value, imaginary)

    def __setitem__(self, tuple key, value):
        i = key[0]
        j = key[1]

        if self.is_complex:
            pass
        else:
            self.safe_put(i, j, value)

Vighnesh Birodkar | 13 May 08:24 2015
Picon

Function with no parameters returning fused type

Hello

Why can't I define a function in Cython with no fused types are arguments ?

eg :

cdef fused_type fused_function():
   
......
   
......

# usage
cdef
int i = fused_function[int]()



Will this be possible in later versions ?

Thanks
Vighnesh

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Hai Nguyen | 13 May 06:01 2015
Picon

prange with gil?

Hi all,

I am working with `prange` and am just wondering about motivation to specify "nogil"

for i in prange(n, nogil=True).

If nogil=False, or don't specify it, Cython will complain. So why not make default as 'nogil' and don't need to explicitly specify it? Or I miss something behind the scene here? thanks

Hai

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Nikolaj van Omme | 12 May 20:37 2015
Picon

How to deal with several types in a class?

Hello,

I'm writing a library that implements its own sparse matrices. We
decided to implement everything in Cython.

We have several extension classes for different sparse matrix types. For
instance:

cdef class LLSparseMatrix(SparseMatrix):
    cdef:
         FLOAT64_t * val
         INT32_t * col

    def __cinit__(self, **kwargs):
         val = <FLOAT64_t *> PyMem_Malloc(self.size_hint *
sizeof(FLOAT64_t))
            if not val:
                raise MemoryError()
            self.val = val

            col = <INT32_t *> PyMem_Malloc(self.size_hint * sizeof(INT32_t))
            if not col:
                raise MemoryError()
            self.col = col
    ...

where FLOAT64_t and INT32_t are ctypedef types.

This LLSparseMatrix has INT32_t for index type and FLOAT64_t type for
its element but we would like to offer the possibility to use other
types at run time.For instance, INT64_t for indices and UINT64_t for the
element types. The user could give the index type "itype" and element
type "dtype" as arguments to the constructor:

my_matrix = LLSparseMatrix(itype=INT32_t, dtype=FLOAT64_t, ...)

Access to the elements of the internal "val" and "col" arrays in Cython
should be efficient.

Here is my question: what is the best (most efficient (wrt time) and at
the same time not too difficult to maintain) solution if I want to use
different types at run time?

We can use a template language (like Jinja2) to generate Cython code, we
can use numpy arrays for "val" and "col" and use numpy types, we
**don't** want to implement the library in C++ with templates.

Any help must appreciated. Thanks in advance.

Nikolaj

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ted Fujimoto | 12 May 19:36 2015
Picon

Any Cython-based Graph/Network analysis packages?

Hi,

Anyone know of any Cython-based Graph/Network analysis packages?

Thanks.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Antony Lee | 12 May 11:04 2015
Picon

Casting fftw_complex pointer (aka double[2]) to complex memoryview

Hi,
I am trying to use FFTW for some Cython code.  I am running into issues when trying to cast a pointer to a fftw_complex (a typedef to double[2]) to a complex memoryview, namely (minimal example)

cdef extern from "fftw3.h":
    ctypedef double fftw_complex[2]
    fftw_complex* fftw_alloc_complex(int N)

cdef foo(complex[::1] input):
    complex_ny = fftw_alloc_complex(input.size)
    (<complex[:self.ny]>complex_ny)[:] = input
    # actual call to FFTW follows

fails at the cast: "hmm.pyx:8:34: Pointer base type does not match cython.array base type" (perhaps it can't be expected for Cython to know that double[2] can be converted to complex but... what else can I do?)

PS: If someone has experience using the pyFFTW wrapper directly from Cython, I'm interested too.

Any help would be very welcome.
Thanks,
Antony

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jeffrey Bush | 10 May 08:36 2015

New to Cython - a few bugs I have found and my wishlist of features

Hi,

I have been enjoying using Cython recently and had some great success with it. I really like coding in C and Python and Cython has been a great merging of those!

However, I have run into a few bugs. The most recent one I thought I would report is if a template function being externed from a C++ file returns a pointer the file fails to compile.

Here are a few examples of non-working ones:

cdef extern from * nogil:
    cdef T
* example[T](T*) # results in "Template parameter not a type" for the [T] and "'T' is not a type identifier" for both T* in this definition

cdef
extern from * nogil:
    cdef
void* example[T](T*) # results in "Template parameter not a type" for the [T] and "'T' is not a type identifier" for the T* in this definition

cdef
extern from * nogil:
    cdef
void* example[T](void*) # results in "Template parameter not a type" for the [T]



However, all of the following work fine:

cdef extern from * nogil:
    cdef
void example[T](T*)

cdef
extern from * nogil:
    cdef size_t example
[T](T*)

cdef
extern from * nogil:
    cdef size_t example
[T](void*)

cdef
extern from * nogil:
    cdef
void* example(void*)


The workaround I came up with is the ctypedef a void* pointer and then cast whenever I need to, for example:

ctypedef void* pointer
cdef extern from "_label.h" nogil:
    cdef pointer example[T](T*)
# and when using it cast the return value to T*


I am using Cython 0.21.1 which I know is a version out, but the change log doesn't mention this at all. Some other bugs I have found (but don't remember the details at the moment) include compiler crashes when using the wrong fused type inside a function body (should give an error, not a crash) and when using a fused type pointer as a template parameter for a class std::map/std::unordered_map unless you remove the return value of the insert function (it doesn't like the pair with a fused type pointer?).



Now onto my wishlist. They mostly revolve around fused types
  • Fused types with fused types. As in I want to group all integers into one fused type and all floating-point numbers into another fused type, and then make a fused type that has all of both of those without copying the whole list. For example:
    • ctypedef fused Integral:
         char
         
      short
         
      int
         
      long
      ctypedef fused
      FloatingPoint:
         
      float
         
      double
      ctypedef fused
      Real:
         
      Integral # equivalent to listing out char, short, int, long here
         
      FloatingPoint # equivalent to listing out float, double here
  • Numpy fused types. The same groupings as per http://docs.scipy.org/doc/numpy/reference/arrays.scalars.html (at least under "number") AND this could be done to include all the platform-specific types in the proper groups when on the proper platforms (like float128) so that on the implementation side if I want to make a function for all available floating-point types (or unsigned types, or numbers) on the current platform I get them all for this platform.
  • Support for integral template arguments (in C++: template <typename T, size_t n> class array;, but not possible to reference in Cython)
  • The complex types cannot be used directly with comparisons (there is no one way to order them, I know) but even if I have defined an order I like (in my case I mimic Numpy's behavior) and have defined the necessary functions on the C++ side, I cannot use them with Cython. There should be a way to either tell Cython "I know that normally these are unordered, but let me use them anyway (I have the operators defined in C++)" or a way to define the operators in Cython and then have Cython allow you to use them. I know that with the first way it can be tricky since if it isn't compiled with C++ the operators can't be easily defined, which makes the second choice better.
  • The range(start, stop, step) function with a runtime step be able to create white code. I understand that it needs to be known whether step is positive or negative to make the loop use the right comparator, but even if step is an unsigned integer, it still doesn't realize that it must be positive. So unsigned steps could be used to signify the positive direction. Additionally, the unary + could be used in front of the step to signify positive direction. The negative direction is a bit harder to make a symbol for, but it is also rarer.

Lastly I am wondering if there is a better way to do the following. Basically I wrap all of my functions this way and it gets really long...

cimport numpy as np
def example(np.ndarray a):
    cdef size_t size
= a.size
    cdef
int n = a.dtype.num if a.dtype.isnative else -1
   
if n == 5 or n == 6: n = np.dtype(a.dtype.name).num
   
if   n == 1:  __example(<np.int8_t*>a.data, size)
   
elif n == 2:  __example(<np.uint8_t*>a.data, size)
   
elif n == 3:  __example(<np.int16_t*>a.data, size)
   
elif n == 4:  __example(<np.uint16_t*>a.data, size)
   
elif n == 7:  __example(<np.int32_t*>a.data, size)
   
elif n == 8:  __example(<np.uint32_t*>a.data, size)
   
elif n == 9:  __example(<np.int64_t*>a.data, size)
   
elif n == 10: __example(<np.uint64_t*>a.data, size)
   
elif n == 11: __example(<np.float32_t*>a.data, size)
   
elif n == 12: __example(<np.float64_t*>a.data, size)
   
elif n == 14: __example(<np.complex64_t*>a.data, size)
   
elif n == 15: __example(<np.complex128_t*>a.data, size)
   
else: __example_fallback(a, size)

Where __example is fused type cdef function. It would also be nice if it automatically change for changes to the fused type. Looking at https://github.com/scipy/scipy/blob/master/scipy/ndimage/src/_ni_label.pyx they do it by using the auto lookup for Python function fused types and returns the function pointer to the real C function. However this has some drawbacks, it requires the auto-lookup feature which is slow compared to using the integer base method I use. Also, I sometimes have problems using the the square brackets to get a particular fused type function if the function is a C function (always has worked if the function is a Python function).


I know this was long, but its the summary of my first week with Cython. Overall I have really enjoyed it and hope to see it continue to grow!

Jeff

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
pauld11718 | 8 May 21:35 2015
Picon

numpy-cython

Unable to compile :

import numpy as np
cimport numpy as np
import math as m

DTYPE = np.float
ctypedef np.float_t DTYPE_t

def visc1(float t, float dcal):
    cdef float h, tr, trinv, rhor
    cdef float eta0, sumi, i, sumj, im1, jm1, eta
   
    cdef np.ndarray vb = np.array([1.00000, 0.940695, 0.578377, -0.202044], dtype = DTYPE)
   
    cdef np.ndarray[DTYPE_t, ndim=2] va = np.array([[.4864192, .3509007, -.2847572, .07013759,.0164122, -.01163815,.0],
        [-.2448372,1.315436, -1.037026, .4660127, -.02884911,-.008239587,.0],
        [-.8702035, 1.297752, -1.287846, .2292075, .0,  .0, .0],
        [.8716056, 1.353448,  .0,  -.4857462, .1607171,.0,    -.003886659],
        [-1.051126, .0,   .0,   .0,   .0,      .0,   .0],
        [.3458395, .0,  -.02148229, .0, -.009603846, .004559914,.0]], dtype=DTYPE, ndim = 2)
   

    h=55.2651e-06;
    tr = t/643.89;
    trinv=643.89/t;
    rhor=dcal/0.358;

    eta0 = h*(m.pow(tr,0.5))/(vb[0] + vb[1]/tr + vb[2]/(tr*tr) + vb[3]/(tr**3));
    sumi=0.0
    for i in range(6):
        sumj=va[i,0]
        for j in range(2,7):
            jm1=j-1;
            sumj=sumj+va[i,j]*((rhor-1.0)**jm1);
           
        im1 = i-1
        sumi = sumi+sumj*((trinv-1.0)**im1);
       
    eta = eta0*m.exp(rhor*sumi)
    return eta

Error :
Compiling visco.pyx because it changed.
Cythonizing visco.pyx
running build_ext
building 'visco' extension
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/deepraj/miniconda3/envs/venv1/include/python3.4m -c visco.c -o build/temp.linux-x86_64-3.4/visco.o
In file included from /usr/include/numpy/ndarraytypes.h:1761:0,
                 from /usr/include/numpy/ndarrayobject.h:17,
                 from /usr/include/numpy/arrayobject.h:4,
                 from visco.c:258:
/usr/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it by " \
  ^
gcc -pthread -shared build/temp.linux-x86_64-3.4/visco.o -L/home/abcd/miniconda3/envs/venv1/lib -lpython3.4m -o /media/abcd/Man_UTD/pythoncode/venv1/visco.cpython-34m.so

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Omer Katz | 8 May 19:49 2015
Picon

stdlib logging module not faster after cythonizing

I used the cystdlib script from the Cython repository in order to try to speed up the logging module.
I did a simple microbenchmark:
import logging
import sys

root
= logging.getLogger()
root
.setLevel(logging.DEBUG)
ch
= logging.StreamHandler(sys.stdout)
ch
.setLevel(logging.DEBUG)
formatter
= logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch
.setFormatter(formatter) root.addHandler(ch)

%timeit -r 100 root.info("hello")

The results are pretty much the same.
The benchmark execution time is between 41.1 µs per loop and 65 µs per loop.
The benchmark was performed with 10000 loops, best of 100.

Can you explain why isn't the logging module faster?

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Gmane