Emmanuel Gil Peyrot | 4 May 13:46 2015
Picon

Specify C attributes on ctypedef struct

Hi,

I noticed gcc produces sub-optimal code when auto-vectorizing some of
my matrix- and vector-manipulating functions, which is fixed when
__attribute__((aligned(16))) is defined on struct it is using.

This is a gcc extension[1], also supported by clang; on MSVC it would
be __declspec(align(16)) from what I gathered on the Internet[2].

I’d like a way to optionally specify that kind of extension in a
ctypedef struct in cython, something like that:

 ctypedef struct Vector[align=16]:
     float a, b, c, d

For easily-vectorizable code this improves performances a lot.

Thanks. :)

[1] https://gcc.gnu.org/onlinedocs/gcc/Common-Type-Attributes.html
[2] https://msdn.microsoft.com/en-us/library/83ythb65.aspx

-- 
Emmanuel Gil Peyrot

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
(Continue reading)

Stefan Behnel | 3 May 11:29 2015
Picon

fixing jedi typing support with Jedi 0.9

Hi,

due to changes in Jedi 0.9 [1], the automatic jedi typing support is now
broken. Since I remember there being some interest in automatic static
Python code typing, I'm asking here if anyone's interested in fixing it, or
really maintaining it.

The latest Jedi release is aiming towards better integration with external
tools, so there should be way to implement this in a cleaner way than what
I hacked up last year. The current code is here:

https://github.com/cython/cython/blob/master/Tools/jedi-typer.py

The new intended starting point (instead of jedi.Script) seems to be
jedi.names() for static code analysis, which then returns a list of
"Definition" objects.

http://jedi.jedidjah.ch/en/latest/docs/plugin-api-classes.html

Anyone interested? Please keep further discussion on the cython-devel
mailing list (not cython-users).

Stefan

[1] http://jedi.jedidjah.ch/

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
(Continue reading)

Logan Page | 2 May 23:26 2015
Picon

Best package / method to profile memory usage of Cython extensions

Hi All

I could do with a few tips on what is the best package / method for profiling memory usage of Cython extentions.

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
bndp | 2 May 18:22 2015
Picon

trying get a faster random float.

I thought this would be faster than pythons random.random()
and it is minutely faster if called sparingly. In the program I want to use it in
it is called around 5 times per loop and the loop can execute several million times, and pythons random seems to overtake it by a few seconds .
Am I doing something blatantly wrong here? 
Why would it slow down over time?
is it the overhead from initializing the class in separate modules with 
rng = mwc_float().nxt 


here is the pyx

cimport cython
cdef extern from "stdint.h": 
    ctypedef int uint32_t

<at> cython.final
cdef class mwc_float:
    cdef uint32_t z,w
    def __cinit__(self):
        cdef uint32_t z=362436069,w=521288629
        self.z = z
        self.w = w
    cdef double _nxt(self)nogil:
        cdef double x
        self.z = 36969 * (self.z & 65535) + (self.z >> 16)
        self.w = 18000 * (self.w & 65535) + (self.w >> 16)
        x = ((self.z << 16) + self.w)* 2.328306e-10
        return x
    def nxt(self):
        return self._nxt()

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
ihsanahsan | 2 May 08:25 2015
Picon

calling routines in scipy binaries directly from cython

The scipy interpolation routines call routines found in _fitpack.so  and dfitpack.so  which are generated through cython (correct me if I am wrong). I was looking at __fitpack.h and I can see that for example, the c-interface to the fortran routine SPLEV is called via a pyobject. Since I am working with cython files I want to avoid this overhead and call SPLEV directly without incurring the pyobject overhead. I was wondering if someone else has successfully done something similar? Note that while scipy routines are fast when called with numpy arrays just once, but when repeatedly called the overhead becomes significant.

cheers

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
michael | 1 May 23:37 2015

How to fake exception stack traces when translating a C++ exception?

Hi,

my project consists of a C++ library, python code and Cython glue code.

The C++ library has an exception type that contains - at the very least - the file name and line number where the exception has originated, and, under certain circumstances, an entire stack trace.

Now I'm writing a custom (except+) exception translator, which
 - calls a C++ function that returns the C++ exception object
 - analyzes the exception object
 - raises an analogous Python exception

ideally, that Python exception object should have the backtrace information from the original C++ exception, but there doesn't seem to be a way to "fake" Python stack traces.

Any ideas?

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Topi Wala | 1 May 01:29 2015
Picon

Using Cython for packaging large python project

Hi folks,

I have a fairly large python project with nested directories multiple levels deep with many python files. I have a requirement to now distribute this as a standalone binary, for which I'm using Cython.
I have converted all my python files to shared objects, except the entry-level file {the one I chose was the one specified by the entry_point keyword in my setup}, that I used the --embed option to generate main, and linked against ALL the other so files. GCC does produce a binary for me.

However, on running, I encounter the error:

pkg_resources.DistrbutionNotFound: [executable-name == 0.9.0]

I have this version number in my setup.py. I'd like to keep it for internal consumption [where we'll be using the python files]. 

How do I overcome this? Is there any compiler_directives that I could add to my main entry point python file which will satisfy the runtime?

Thanks,
Topi

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Curtis M. Faith | 1 May 02:37 2015
Picon

Advanced AGI Tech Ideas

I've got some ideas for AGI I've been working on for years. Didn't know they were for AGI at first as I was trying to solve the global mind problem, i.e. come up with a distribution and messaging and flow architecture for recursive algorithms for composition and decomposition of anything at an abstract level.

Please check out my LinkedIn and see if we have any common friends.

I am willing to let anyone here already as of the time of this post join my private group to discuss this. The group has to be private until the patents are filed to avoid public disclosure invalidation. Read up on that and confirm that you understand what that means both for U.S. and international treaty patent law first.

Then you have to agree to donate all your future IP from future derivative products to a common IP / Patent pool with distribution algorithms to be decided using super super majority liquid democracy rules. Please look that up and don't ask me about it because you'll find good stuff in the search for liquid democracy if you search yourself.

Welcome all GSoC selectees. You are going to make magic this summer...

Hope you decide to join me in the pursuit of the next generation net

or as I prefer in Español:

RED

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Pedro Camargo | 1 May 01:06 2015
Picon

Writting to an hdf5 file in parallel

Hi there,

            I recently finished the first stage of developing an open source software for transportation planning (www.aequilibrae.com), but I am struggling with the next step.

        In a nutshell, what I got so far was to calculate the paths generated by running shortest paths functions (Each origin or set of origins in a thread) using memory maps and properly releasing the GIL (big thanks to Sturla Molden).

       Right now I need to save these results to disk in an efficient format to be analyzed later by the user (the rest of the math, which is traffic assignment, is already done and presented).  Of all I read, it seems that HDF5 is my best option, but that brings a couple of questions:

1 - Can I write to hdf5 without acquiring the GIL?
2 - If so, any suggestions on wheter using h5py, pytables or to write my own wrapping?
3 - Can I access an hdf file to append different tables at the same time?

Hope this all makes sense.

Thanks,

Pedro

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Zahra Sheikh | 29 Apr 23:12 2015
Picon

Reading and writing an array in a binary file using C functions in cython

I am instantiating a class in cython. I want to declare one of the instance which is an array of float values once by computing it for the first time with a given function and save it in a binary file using c functions. For the further call of my class if the input file does already exist, the instance would be declared by reading the array from the file otherwise it would compute it again.

Test.pyx

import numpy as np
cimport numpy as np
cimport cython
from libc.stdio cimport FILE, fopen, fwrite, fscanf, fclose, fprintf
from numpy cimport float64_t
from libc.stdlib cimport malloc, free
cdef extern from "math.h":
    double exp(double) nogil
    double log(double) nogil

cdef class Hit(object):
    cdef public double[::1] zs,  Da
    cdef char* path

    <at> cython.boundscheck(False)
    <at> cython.cdivision(True)
    <at> cython.wraparound(False)
    <at> cython.nonecheck(False)
    def __cinit__(self, path=None, zs=None):
        if path is None:
           raise ValueError("Could not find a path to the file which contains the table of distances")
        else:
           self.path=path

        if zs is None:
           raise ValueError("You must give an array which contains the steps!")
        self.zs=zs

        cdef Py_ssize_t i, N
        N=len(np.ascontiguousarray(self.zs))
        cdef FILE *ptr_fr
        cdef FILE *ptr_fw
        cdef double *ptr_d= <double *>malloc(N * sizeof(double))
        ptr_fr = fopen(self.path, "rb")        
        if (ptr_fr==NULL):
           print "I/O Error: cannot open file {}".format( self.path)  
           for i from N > i >= 0:
               ptr_d[i]=log(self.zs[i]+1.) /(1- self.zs[i])**0.5
               print ptr_d[i]
           ptr_fw = fopen(self.path, "wb")
           if (ptr_fw == NULL):
               print "Unable to open file!\n"

           else:

               print "Opened file successfully for writing.\n"
               fwrite(ptr_d, sizeof(double), N, ptr_fw)
               fclose(ptr_fw)
               self.Da = <double[:N]>ptr_d
        else: 
           for i from N > i >= 0:
               fscanf(ptr_fr,"%Lf", ptr_d[i])
               print ptr_d[i]
           fclose(ptr_fr)
           self.Da = <double[:N]>ptr_d
        free(ptr_d)
        print np.ascontiguousarray(self.Da)
        print type(self.Da)


When I run it in the python like this :

import Test
import numpy as np        
Test.Hit('test.cat',np.arange(0.01,1,0.01))

Basically the result is an empty array:

[  5.43123598e-317   6.91697796e-310               nan   5.76958004e-317
   9.88131292e-324   6.91698188e-310   1.03753786e-322   6.91697796e-310
   6.91697955e-310   6.91698199e-310   6.91697955e-310   0.00000000e+000
   5.76983695e-317   0.00000000e+000   0.00000000e+000   0.00000000e+000
   0.00000000e+000   0.00000000e+000   3.69736595e-317   7.27844556e-312
   1.82092564e-157   4.98693282e+173   5.67133142e-308   6.19632855e-119
   4.99659416e+173   7.89640541e-308   2.10851421e-080   3.41964447e+125
   9.67385631e-308   5.83294368e-302   3.39404469e+125   1.07863949e-307
   1.55969698e-239   1.13426633e-307   1.02216421e-234   1.10753424e+217
   2.56242041e-308   1.88556458e-215   1.10861151e+217   3.11868890e-308
   3.47826090e-196   1.10968879e+217   3.67495740e-308   6.41627392e-177
   1.11076607e+217   4.23122590e-308   1.18359641e-157   1.11184334e+217
   5.83403743e-302   3.41305005e+125   1.18989302e-307   6.91698191e-310
   6.91698189e-310   6.91698189e-310   6.91698189e-310   6.91698196e-310
   6.91698198e-310   6.91698189e-310   6.91698189e-310   0.00000000e+000
   0.00000000e+000   0.00000000e+000   0.00000000e+000   6.91697955e-310
   6.91698191e-310   6.91698189e-310   6.91698189e-310   6.91698189e-310
   6.91698196e-310   6.91698198e-310   6.91697955e-310   6.91698189e-310
   1.24335033e-307   1.03259720e-321   5.21840040e-317   6.91698122e-310
   3.50702979e-191   3.28799181e+178   2.10230602e-309   8.16544004e-201
   5.01949268e+173   1.63273486e-307   2.42143943e-008   3.97882525e+202
   5.16074548e-310   1.58691454e-003   4.94710808e+173   3.35724048e-295
   1.15708147e-306   3.97978206e+202   1.93477168e-307  -3.18152442e-294
   2.57444430e-003   4.94712695e+173   2.04645986e-307   2.92734580e+016
   3.28089961e+178   4.88908397e-309   1.02765654e-321]
<type 'Test._memoryviewslice'>
<Test.Hit object at 0x7f548e8e38c8>

I am guessing one of the problems is the way I assign my pointer to the class instance which is declared as a memoryview. 


I appreciate if you will point out my mistakes.

Cheers,
Zahra

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Logan Page | 27 Apr 12:56 2015
Picon

Cython traceback on crash with typo for a keyword argument

Hi All

Here is a simple example that causes the compiler to crash and give traceback info if there is a simple typo with one of the keyword arguments.
Perhaps you can add this to a wishlist to output an error message instead of compiler crash traceback.


In .pxd file:
cdef void do_something(double*, int, tuple foo=*)



In .pyx file:
cdef void do_something(double* x, int n, tuple foO=None):
   
print foO

This compiler error occurs at the point where the mis-typed name (foO) is used in the function.

The compiler traceback is:
Traceback (most recent call last):
  File "/home/logan/.virtualenvs/pyCFD/bin/cython", line 9, in <module>
    load_entry_point('Cython==0.21.2', 'console_scripts', 'cython')()
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 637, in setuptools_main
    return main(command_line = 1)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 654, in main
    result = compile(sources, options)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 629, in compile
    return compile_multiple(source, options)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 607, in compile_multiple
    result = run_pipeline(source, options, context=context)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 441, in run_pipeline
    err, enddata = Pipeline.run_pipeline(pipeline, source)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/Pipeline.py", line 328, in run_pipeline
    data = phase(data)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/Pipeline.py", line 54, in generate_pyx_code_stage
    module_node.process_implementation(options, result)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/ModuleNode.py", line 111, in process_implementation
    self.generate_c_code(env, options, result)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/ModuleNode.py", line 330, in generate_c_code
    self.body.generate_function_definitions(env, code)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/Nodes.py", line 398, in generate_function_definitions
    stat.generate_function_definitions(env, code)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/Nodes.py", line 1819, in generate_function_definitions
    self.generate_argument_parsing_code(env, code)
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/Nodes.py", line 2456, in generate_argument_parsing_code
    self.type.opt_arg_cname(declarator.name)))
  File "/home/logan/.virtualenvs/pyCFD/lib/python2.7/site-packages/Cython/Compiler/PyrexTypes.py", line 2576, in opt_arg_cname
    return self.op_arg_struct.base_type.scope.lookup(arg_name).cname
AttributeError: 'NoneType' object has no attribute 'cname'

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Gmane