Daiyue Weng | 1 Jun 16:54 2016
Picon

install cython using anaconda and run cython files

Hi, I have installed Cython using Anaconda. I also use PyCharm with Python 3.5. I have also installed Visual Studio 2015. But when I tried to build the helloworld.pyx from the basic tutorial, I get the following error:

error: Unable to find vcvarsall.bat

I have been followed the instructions given at

https://github.com/cython/cython/wiki/InstallingOnWindows

https://github.com/cython/cython/wiki/CythonExtensionsOnWindows

It seems Python 3.5 works fine with Visual Studio 2015 when compiling Python extensions on Windows. But it didn't work out straight away for me. So how to resolve the problem?


Many thanks

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Kane | 31 May 08:26 2016
Picon

Basic MySQL header use

Hi all,

I want to do is get a dump of a MySQL table into python as quickly as possible, but most python mysql connectors seem to be relatively slow. So I'm trying Cython.

In theory, the C side has a nice API and is very simple: see "retrieving data from the database" section of here: http://zetcode.com/db/mysqlc/. What I'm finding hard (as a C/Cython newbie) is getting all of the libraries/links correct, and even the most basic thing to run.

So far, I have the following mymysql.pyx

cdef extern from "mysql.h":
    pass

    ctypedef struct MYSQL:
        #NET net;
        unsigned char *connector_fd;
        char *host,*user,*passwd,*unix_socket,*server_version,*host_info;
        char *info, *db;
        #const struct charset_info_st *charset;
        #MYSQL_FIELD *fields;
        #MEM_ROOT field_alloc;
        #my_ulonglong affected_rows;
        #my_ulonglong insert_id;
        #my_ulonglong extra_info;
        unsigned long thread_id;
        unsigned long packet_length;
        unsigned int port;
        unsigned long client_flag,server_capabilities;
        unsigned int protocol_version;
        unsigned int field_count;
        unsigned int server_status;
        unsigned int server_language;
        unsigned int warning_count;
        #struct st_mysql_options options;
        #enum mysql_status status;
        #my_bool free_me;
        #my_bool reconnect;
        #char scramble[SCRAMBLE_LENGTH+1];
        #my_bool unused1;
        void *unused2, *unused3, *unused4, *unused5;
        #LIST *stmts;
        #const struct st_mysql_methods *methods;
        void *thd;
        #my_bool *unbuffered_fetch_owner;
        char *info_buffer;
        void *extension;

    MYSQL * mysql_init(MYSQL *mysql)

def do():
    cdef MYSQL *con = mysql_init(NULL)

which compiles using:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
extensions = [
        Extension("mymysql", ["mymysql.pyx"],
            extra_compile_args=["-L/usr/lib64", "-lmysqlclient", "-lpthread", "-lz", "-lm", "-lssl", "-lcrypto", "-ldl", "-I/usr/include/mysql"] # from $(mysql_config --libs --cflags)
            )
        ]
setup(ext_modules=cythonize(extensions))

However, when I try to import the basic 'do' function, I get:

ImportError: mymysql.cpython-34m.so: undefined symbol: mysql_init

Is the first place to start completing the ctypedef of MYSQL? I haven't because I don't even know if I need to include it at all, and it's proving quite painful as there seem to be quite a few other things I need to define to fully define MYSQL.

Tips would be most appreciated,

Kane

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Mathieu Virbel | 31 May 00:26 2016
Picon
Gravatar

New feature: use full qualified name as init entrypoint

Hi everyone!

I'm a fellow user of Cython from a long time, and used it in Kivy.org many years ago, which is used for creating desktop and mobile (iOS / Android) application in Python. 

# Background

I would like to talk about few issues i've seen, which used the same workaround (more or less):
  1. Android platform was limited to 64 dynamic libraries loaded per applications: i let you imagine how fast you can reach this issue on big application. To get around this limitation:
    1. our toolchain (python-for-android) doesn't link any the Python extension
    2. it generate an empty .so
    3. it save the reference to the .o and link options in the path
    4. at the end, it make a single "libpymodules.so" with all the .o and link options we found.
    5. Python is patched to search the dlsym in our "libpymodules.so" before looking at the real .so
  2. iOS platform don't accept dynamic library, so the solution 1 is a must, and the instead of creating libpymodules.so, all the .o are statically linked to the executable
  3. Recently, i needed to compile a full python application with Cython (mostly to prevent easy decompilation with uncompyle). So same approach as 1...
 
Issue: we faced the issue of name conflict (imagine pygame.event and kivy.event clash if both are used, as both Python extension would have a "initevent" entrypoint).
Solution: search all the occurences in the Cython generated source code for "initevent", and replace with the full qualified name: "initkivy_event" + inject a import hook to search for full qualified name module prior to standard import.


# Feature: full qualified name as entrypoint

I would like to propose a new option that i called: "use_fqdn_entrypoint". If activated, the source code generated by Cython will use the full qualified name of a module as entrypoint. For example, a Cython file like "kivy/graphics/opengl.pyx" will be compiled with an entrypoint like "initkivy_graphics_opengl".

This features require a bootloader in the application, but i think it's out of the scope of Cython.

Here is an example of a setup.py with the options activated:

file_list = find_files("myapp", "py")
extensions = cythonize(file_list, nthreads=4,
                       compiler_directives={
                           "use_fqdn_entrypoint": True})
sources = [ext.sources[0] for ext in extensions]
 
# required for Windows otherwise the dll wont have all the entrypoints 
def to_export_symbol(filename):
    return "init{}".format(filename.rpartition(".")[0].replace(sep, "_"))
export_symbols = map(to_export_symbol, sources)
export_symbols.remove("initmyapp___init__")

setup(name="myapp",
      ext_modules=[Extension("myapp",
                             sources,
                             export_symbols=export_symbols)])


The current implementation look like that: https://github.com/cython/cython/compare/master...tito:feature-fqdn-entrypoint

# Questions

How do you feel about this contribution?
Would it be acceptable in this form?


Thanks a lot for your time,

Mathieu



--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Aidin Hasanzadeh | 30 May 10:20 2016
Picon

Setting up theano on Ubuntu 16.04

Hi,


I am trying to configure and run my first theano code with gpu on a Ubuntu Mate 16.04 platform. My video device is Cuda-Capable and I have manged and installed Cuda. Now, I am trying to run my first theano code using gpu but always run on cpu. Here are all the steps taken so far:


0. System info
- Verify if GPU is CUDA-Capable (Yes)   

    $ lspci | grep -i nvidia
   
01:00.0 VGA compatible controller: NVIDIA Corporation GK107GL [Quadro K600] (rev a1)
   
01:00.1 Audio device: NVIDIA Corporation GK107 HDMI Audio Controller (rev a1)

- Verify if the systme is spported version of Linux   

    $ uname -m && cat /etc/*release
    x86_64
    DISTRIB_ID=Ubuntu
    DISTRIB_RELEASE=16.04
    DISTRIB_CODENAME=xenial
    DISTRIB_DESCRIPTION="Ubuntu 16.04 LTS"
    NAME="Ubuntu"
    VERSION="16.04 LTS (Xenial Xerus)"
    ID=ubuntu
    ID_LIKE=debian
    PRETTY_NAME="Ubuntu 16.04 LTS"
    VERSION_ID="16.04"
    HOME_URL="http://www.ubuntu.com/"
    SUPPORT_URL="http://help.ubuntu.com/"
    BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
    UBUNTU_CODENAME=xenia

- Verify gcc version   

    $ gcc --version
    gcc
(Ubuntu 4.9.3-13ubuntu2) 4.9.3
   
Copyright (C) 2015 Free Software Foundation, Inc.
   
This is free software; see the source for copying conditions.  There is NO
    warranty
; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

1. Setup additional drivers, configure the display adapter to (proprietary,tested) driver.


2. Install CUDA

- Download CUDA 7.5.18 run file for NVIDIA officical website.

    $ wget developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda_7.5.18_linux.run    

 - Reboot into text mode runlevel 3.
 - make the file execuatbale

    $ chmod 755 cuda_7.5.18_linux.run

 - install CUDA (do not install NVIDIA Accelerated Graphics Driver)

    $ sudo ./cuda_7.5.18_linux.run --override
- Force CUDA to work with gcc 5 (optional)

- Setup environment variables

    $ export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib:$LD_LIBRARY_PATH
    $
export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH
    $ sudo source
~/.bashrc

3. Setup CUDNN
- Download the latest CUDNN

    $ wget https://developer.nvidia.com/rdp/assets/cudnn-7.5-linux-x64-v5.0-ga-tgz
- extract files to the desired directory
    $ tar -xf cudnn-7.5-linux-x64-v5.0-ga.tgz ~/cudnn-7.5
- add the extracted directory for LD_LIBRARY_PATH
    $ export LD_LIBRARY_PATH=/home/aidin/software/cudnn-7.5/
- Perform the below copies
    $ sudo cp ~/cudnn-7.5/include/cudnn.h /usr/local/cuda-7.5/incl
    $ sudo cp
~/cudnn-7.5/lib64/libcudnn* /usr/local/cuda-7.5/lib64/

4. Check nvidia-smi    

    $ nvidia-smi
   
Mon May 30 10:51:40 2016      
   
+------------------------------------------------------+                      
   
| NVIDIA-SMI 361.42     Driver Version: 361.42         |                      
   
|-------------------------------+----------------------+----------------------+
   
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
   
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
   
|===============================+======================+======================|
   
|   0  Quadro K600         Off  | 0000:01:00.0      On |                  N/A |
   
| 26%   53C    P0    N/A /  N/A |    609MiB /  1023MiB |      0%      Default |
   
+-------------------------------+----------------------+----------------------+
                                                                               
   
+-----------------------------------------------------------------------------+
   
| Processes:                                                       GPU Memory |
   
|  GPU       PID  Type  Process name                               Usage      |
   
|=============================================================================|
   
|    0      1299    G   /usr/lib/xorg/Xorg                             383MiB |
   
|    0      1830    G   compiz                                         215MiB |
   
+-----------------------------------------------------------------------------+

5. Test theano with gpu (sample code from deeplearning.net)
    sudo THEANO_FLAGS=device=gpu0,nvcc.flags=-D_FORCE_INLINES python check1.py
   
Using gpu device 0: Quadro K600 (CNMeM is disabled, cuDNN 5005)
   
[Elemwise{exp,no_inplace}(<TensorType(float64, vector)>)]
   
Looping 1000 times took 28.682443 seconds
   
Result is [ 1.23178032  1.61879341  1.52278065 ...,  2.20771815  2.29967753
     
1.62323285]
   
Used the cpu


Please correct me if there is sth wrong or missing.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Javier Panadero Martínez | 30 May 00:50 2016
Picon

Error accesing to a method AttributeError: 'fib.Edge' object has no attribute 'getOrigin'

Hi guys, I'm developing a cython code in order to improve the performance of my python code. When I try to access to a method of an instance inside in a list, I obtain the next message error: xx.getOrigin() AttributeError: 'fib.Edge' object has no attribute 'getOrigin' I attach a part of the pyx code, the python and output. Any of you could help me?



code pyx

#######################################################################
###########################Clase Edge #################################
#######################################################################
cdef class Edge:
#INSTANCE FIELDS & CONSTRUCTOR */
cdef Node origin #origin node
cdef Node end #end node



def __cinit__(self, originNode, endNode):
self.origin = originNode
self.end = endNode


#GET METHODS */
cdef Node getOrigin(self):
return self.origin






#######################################################################
###########################Clase Route ################################
#######################################################################
cdef class Route:
#INSTANCE FIELDS & CONSTRUCTOR */
cdef double costs #route total costs
cdef float demand #route total demand
cdef list edges #edges list
cdef c_array.array center #(x-bar, y-bar) for all (x,y) in the route


def __cinit__(self):
self.edges = []
self.center = array('f', [0,1])



#SET METHODS
cdef void setEdges(self,list e):
self.edges = e



#LIST GET METHODS*/
cdef list getEdges(self):
return self.edges


code python
def prueba():
print "llega"
nodo1 = Node(1, 2, 3, 4)
nodo2 = Node(2, 2, 3, 4)
nodo3 = Node(3, 2, 3, 4)

lazo = Edge(nodo1,nodo2)
lazo2 = Edge(nodo1,nodo3)

d = lazo2.getOrigin()
print d

#pruebo rutas
ruta = Route()
lista = []

lista.append(lazo)
lista.append(lazo2)

ruta.setEdges(lista)

listaa = ruta.getEdges()
print lista


#lista
for xx in listaa:
print "sss "
print xx
n = xx.getOrigin()




output

MBP-de-javier:cython javier$ python mydebug.py 

llega

<fib.Node object at 0x102546668>

[<fib.Edge object at 0x102546738>, <fib.Edge object at 0x1025467a0>]

sss 

<fib.Edge object at 0x102546738>

Traceback (most recent call last):

  File "mydebug.py", line 3, in <module>

    fib.prueba()

  File "fib.pyx", line 317, in fib.prueba (fib.c:5016)

    xx.getOrigin()

AttributeError: 'fib.Edge' object has no attribute 'getOrigin'


--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ecolss Logan | 28 May 12:49 2016
Picon

typed memoryview is deprecated when working with numpy?

I use typed memoryview primarily or as much as possible, but if I need to use numpy's functions or the broadcasting features, I think I'd better not use typed memoryview but go with numpy.ndarray.

For example, suppose I have a function which takes memoryview as arg:

> cdef void fn (double[:] a, double b):    #  add b to a
>    numpy.add(a, b, out = a)                 # this line will fail, since numpy.add requires `out` to be ArrayType, not memoryview

the above simple code is just an example to demonstrate my idea, since it'll fail, I could change it to:

> cdef void fn (double[:] a, double b): 
>    cdef np.ndarray[double, ndim=1] c = a    # create a numpy array from the memoryview, btw, is `np.ndarray` use is deprecated nowadays?
>    numpy.add(a, b, out = a)

OK, if I call `fn` many times, I think the creation of `np.ndarray` could make a difference in terms of time.
So, is memoryview not the right static type to use if we need to work with numpy ?

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Ecolss Logan | 28 May 11:07 2016
Picon

make numpy.dot faster by pre-allocating output array

Hey guys,

I'm writing a numeric computation library which will make lots of matrix linear algebra computation.
Since in my code, numpy.dot is called repeatedly and in order to make it faster, I thought it would be helpful to pre-allocate the output array. for example:

> numpy.dot(matA, matB, outArray)

so numpy won't create that outArray any longer, and as long as the output shape is fixed, I could use outArray repeatedly which could save lots of time, right.

However, in my code, the matA and matB might vary, but I know for sure that the output shape will be within (10,10) ~ (100,100), so I wonder is there any way that  I pre-allocate an outArray with shape (100,100), and use it repeatedly?

If it's not possible, do you think I should go with direct BLAS calls to scipy.linalg.cython_blas ?


Thanks

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
David Hannasch | 28 May 00:24 2016
Picon

Do typed memoryviews prevent garbage collection?

I cannot find an answer in http://docs.cython.org/src/userguide/memoryviews.html
--- do typed memoryviews count as references to the underlying memory, preventing garbage collection?

As a simple example, consider:
cdef numpy.float64_t[:] return_memoryview():
 
return numpy.array([1,2,3])
When this function is called, the numpy array will be immediately discarded and then garbage-collected...but what about the underlying memory that the typed memoryview now points to?

Obviously, I *have* kept memoryviews long after the numpy array has been discarded, and it works fine, but I worry that this is only because the underlying memory hasn't happened to be overwritten and that it might be corrupted at any moment.
I would think the documentation would have a warning if you needed a separate reference to prevent garbage collection, but on the other hand you can't create a typed memoryview without first creating the underlying array by other means, which suggests it might be dependent...does anyone know the true answer?

Thank you,
David Hannasch

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Antony Lee | 28 May 00:17 2016
Picon

Easily avoidable warning in module compilation on Windows

Consider an empty `foo.pyx`.  On Windows, calling `cythonize -b foo.pyx` results in the following warning at link time:

foo.obj : warning LNK4197: export 'PyInit_foo' specified multiple times; using first specification

This is because the source contains the lines

#if PY_MAJOR_VERSION < 3
PyMODINIT_FUNC initfoo(void); /*proto*/
PyMODINIT_FUNC initfoo(void)
#else
PyMODINIT_FUNC PyInit_foo(void); /*proto*/
PyMODINIT_FUNC PyInit_foo(void)
#endif
{ // definition follows

While the warning is only a minor annoyance, would it be possible to get rid of the prototype line and thus of the warning at the same time?

Antony

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Joshua Klein | 26 May 16:58 2016
Picon
Gravatar

Mirroring Documentation between Pure Python and Cython implementations


Short:
Is there a way to override the docstring on extension classes or cdef functions at build time or run time?

Long:

Hello,

I've been using Cython regularly in my work, thank you for all of the effort involved in maintaining so many language variants and compiler schemes.

I often write a Pure Python implementation for testing purposes before writing an equivalent Cython implementation. Then I maintain both implementations such that my libraries can be used by people without C compilers available. Since most of my code has the same API between Python and Cython, I usually copy-paste docstrings, but I am curious if there is a better way. I tried assigning to `__doc__` on a `cdef class` and got an exception, `AttributeError: attribute '__doc__' of 'builtin_function_or_method' objects is not writable`

Is there a way to override the docstring at build time or run time? If it's not yet possible, is it something that the Python interpreter would support with changes to Cython, and if so could you point in the right starting direction so I can try to solve it and submit a patch?

Thank you

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Paul Londino | 26 May 03:59 2016
Picon

Extension module in different folder than .pxd

Hi,

I am creating a Cython module, but I have a growing number of pyx and pxd files. I want to place the .pyx and .pxd files in a separate directory and compile them into extension modules that live in a parent directory. For example, a module laid out as follows:

package_a/
  __init__.py
  pure_python.py
  cython_files/
    rect.pxd
    rect.pyx
    cy_module.pyx

I want to create the following extension modules:

ext_modules = [
    Extension('package_a.rect', sources=['package_a/cython_files/rect.pyx']),
    Extension('package_a.cy_module', sources=['package_a/cython_files/cy_module.pyx'])
]

However, this fails on import of package_a.cy_module which cimports rect.pxd:

>>> import package_a.cy_module
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "package_a\cython_files\rect.pxd", line 10, in init package_a.cython_files.cy_module(package_a/cython_files/cy_module.cpp:1070)
    cdef class PyRectangle:
ImportError: No module named rect


 I tried various combinations of relative c imports and absolute c imports, but the only thing that worked was changing the extension modules to:

ext_modules = [
    Extension('package_a.cython_files.rect', sources=['package_a/cython_files/rect.pyx']),
    Extension('package_a.cython_files.cy_module', sources=['package_a/cython_files/cymodule.pyx'])
]

Is there any way to do what I'm trying to do and still cimport the .pxd of another module?

Thanks,
Paul

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Gmane