Re: _import_umath error when compiling code

On Fri, Nov 4, 2011 at 4:19 PM, Jon Herman <jfc.herman <at> gmail.com> wrote:
> Robert, I think I'm declaring everything correctly, but this is where my
> inexperience might come in.
>
> Xtemp1 and kX are defined as np.ndarray
> b is a function input and defined as a np.ndarray(np.double_t, ndim=2)
> All counters are defined as ints.
>
> I uploaded a new annotated version here:
> https://blackboard.tudelft.nl/bbcswebdav/xid-4043100_1
> Clearly, there is still a lot of python code in there (the loop I mentioned
> here is at line 198/199 of the .pyx file).
>
> Do you see anything I might be declaring incorrectly?

You need to type them as numpy.ndarray[dtype, ndim=...]. Cython can't
do fast indexing if it doesn't know the type and dimension count (it
wouldn't know what the return type is).

You'll also want to make sure everything on line 203 is typed.

> On Fri, Nov 4, 2011 at 2:23 PM, Robert Bradshaw
> <robertwb <at> math.washington.edu> wrote:
>>
>> On Fri, Nov 4, 2011 at 2:17 PM, Jon Herman <jfc.herman <at> gmail.com> wrote:
>> > Robert,
>> >
>> > for now I'm just focusing on the loops, but I'm actually getting code
>> > that
>> > is about 6 times slower. To be concise, I am replacing (for example) the

https://github.com/markflorisson88/cython.git ?

Re: Re: Segmentation fault when importing a module with embedded python3

Adam, 04.11.2011 16:54:
> I figured out the problem.
>
> During initialization (PyInit_crash()), Cython creates a Python module
> named 'crash', and sets a global pointer.  This global pointer is
> *also* the return value of the initialization function.  If the cython
> file (crash.pyx) ignores the return value, then Cython accounts for
> this by decrementing the reference count of the initialization return
> value. The reference count of the module is now zero, and Python
> destroys the newly created module.  The global pointer now points to
> garbage.
>
> There are two ways to get around this.
>
> 1. Fix Cython so that it increments the reference count of the module
> to 2 (one for the global reference, one for the returned reference)
> during initialization.
> 2. Store the return value of the initialization as a global variable
> in the cython module (crash.pyx) itself. (See example below.)

3. properly register the module with CPython.

If you want to do this manually, you should use the CPython import C-API to 
get the module stored in sys.modules.

Stefan

https://github.com/markflorisson88/cython.git ?

On 5 November 2011 00:32, Paul Leopardi <paul.leopardi <at> iinet.net.au> wrote:
> Here's another one:
> Crash:
>
> leopardi <at> linfinit:~/src/Working/Working-0.5.2/glucat/pyclical-templates>
> make
> rm -f PyCliCal.so PyCliCal.cpp PyCliCal.html
> CXXFLAGS="-DNDEBUG -D_GLUCAT_USE_TR1_UNORDERED_MAP
> -D_GLUCAT_USE_DENSE_MATRICES -D_GLUCAT_USE_QD -D_GLUCAT_USE_EIGENVALUES
> -D_GLUCAT_USE_ALGLIB " INCLUDES="-I. -I..
> -I/home/leopardi/usr/local/include" LDFLAGS="-L/home/leopardi/usr/local/lib
> -lqd -lalglib" python setup.py build_ext --inplace
> running build_ext
> cythoning PyCliCal.pyx to PyCliCal.cpp
> Traceback (most recent call last):
>   File "setup.py", line 21, in <module>
>     ext_modules = [ext]
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/distutils/core.py",
> line 152, in setup
>     dist.run_commands()
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/distutils/dist.py",
> line 953, in run_commands
>     self.run_command(cmd)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/distutils/dist.py",
> line 972, in run_command
>     cmd_obj.run()
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Distutils/build_ext.py",
> line 125, in run
>     _build_ext.build_ext.run(self)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/distutils/command/build_ext.py",
> line 340, in run
>     self.build_extensions()
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Distutils/build_ext.py",
> line 132, in build_extensions
>     ext.sources = self.cython_sources(ext.sources, ext)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Distutils/build_ext.py",
> line 275, in cython_sources
>     full_module_name=module_name)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Main.py",
> line 630, in compile
>     return compile_single(source, options, full_module_name)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Main.py",
> line 575, in compile_single
>     return run_pipeline(source, options, full_module_name)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Main.py",
> line 467, in run_pipeline
>     err, enddata = Pipeline.run_pipeline(pipeline, source)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Pipeline.py",
> line 313, in run_pipeline
>     data = phase(data)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Pipeline.py",
> line 51, in generate_pyx_code_stage
>     module_node.process_implementation(options, result)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/ModuleNode.py",
> line 107, in process_implementation
>     self.generate_c_code(env, options, result)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/ModuleNode.py",
> line 335, in generate_c_code
>     self.body.generate_function_definitions(env, code)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Nodes.py",
> line 349, in generate_function_definitions
>     stat.generate_function_definitions(env, code)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Nodes.py",
> line 349, in generate_function_definitions
>     stat.generate_function_definitions(env, code)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Nodes.py",
> line 1427, in generate_function_definitions
>     with_pymethdef = with_pymethdef)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Nodes.py",
> line 2073, in generate_function_header
>     header = self.return_type.declaration_code(entity,
> dll_linkage=dll_linkage)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/PyrexTypes.py",
> line 2910, in declaration_code
>     for param in self.templates]
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/PyrexTypes.py",
> line 1135, in declaration_code
>     raise Exception("This may never happen, please report a bug")
> Exception: This may never happen, please report a bug
> make: *** [PyCliCal.so] Error 1
> leopardi <at> linfinit:~/src/Working/Working-0.5.2/glucat/pyclical-templates> cd
> /home/leopardi/src/Working/Working-0.5.2/glucat/pyclical-templates
> leopardi <at> linfinit:~/src/Working/Working-0.5.2/glucat/pyclical-templates> cd
> /home/leopardi/src/Working/Working-0.5.2/glucat/pyclical-templates
> leopardi <at> linfinit:~/src/Working/Working-0.5.2/glucat/pyclical-templates>
> make
> rm -f PyCliCal.so PyCliCal.cpp PyCliCal.html
> CXXFLAGS="-DNDEBUG -D_GLUCAT_USE_TR1_UNORDERED_MAP
> -D_GLUCAT_USE_DENSE_MATRICES -D_GLUCAT_USE_QD -D_GLUCAT_USE_EIGENVALUES
> -D_GLUCAT_USE_ALGLIB " INCLUDES="-I. -I..
> -I/home/leopardi/usr/local/include" LDFLAGS="-L/home/leopardi/usr/local/lib
> -lqd -lalglib" python setup.py build_ext --inplace
> running build_ext
> cythoning PyCliCal.pyx to PyCliCal.cpp
> Traceback (most recent call last):
>   File "setup.py", line 21, in <module>
>     ext_modules = [ext]
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/distutils/core.py",
> line 152, in setup
>     dist.run_commands()
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/distutils/dist.py",
> line 953, in run_commands
>     self.run_command(cmd)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/distutils/dist.py",
> line 972, in run_command
>     cmd_obj.run()
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Distutils/build_ext.py",
> line 125, in run
>     _build_ext.build_ext.run(self)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/distutils/command/build_ext.py",
> line 340, in run
>     self.build_extensions()
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Distutils/build_ext.py",
> line 132, in build_extensions
>     ext.sources = self.cython_sources(ext.sources, ext)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Distutils/build_ext.py",
> line 275, in cython_sources
>     full_module_name=module_name)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Main.py",
> line 630, in compile
>     return compile_single(source, options, full_module_name)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Main.py",
> line 575, in compile_single
>     return run_pipeline(source, options, full_module_name)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Main.py",
> line 467, in run_pipeline
>     err, enddata = Pipeline.run_pipeline(pipeline, source)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Pipeline.py",
> line 313, in run_pipeline
>     data = phase(data)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Pipeline.py",
> line 51, in generate_pyx_code_stage
>     module_node.process_implementation(options, result)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/ModuleNode.py",
> line 107, in process_implementation
>     self.generate_c_code(env, options, result)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/ModuleNode.py",
> line 335, in generate_c_code
>     self.body.generate_function_definitions(env, code)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Nodes.py",
> line 349, in generate_function_definitions
>     stat.generate_function_definitions(env, code)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Nodes.py",
> line 349, in generate_function_definitions
>     stat.generate_function_definitions(env, code)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Nodes.py",
> line 1427, in generate_function_definitions
>     with_pymethdef = with_pymethdef)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/Nodes.py",
> line 2073, in generate_function_header
>     header = self.return_type.declaration_code(entity,
> dll_linkage=dll_linkage)
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/PyrexTypes.py",
> line 2910, in declaration_code
>     for param in self.templates]
>   File
> "/home/leopardi/opt/Enthought/epd-7.1-1-rh5-x86_64/lib/python2.7/site-packages/Cython/Compiler/PyrexTypes.py",
> line 1135, in declaration_code
>     raise Exception("This may never happen, please report a bug")
> Exception: This may never happen, please report a bug
> make: *** [PyCliCal.so] Error 1
>
> pyclical.pyx:
>
> # -*- coding: utf-8 -*-
> import types
> from PyCliCal cimport *
> ctypedef multivector[scalar_t] multivector_scalar_t
> cdef class clifford
> cdef inline multivector[scalar_t] toClifford(obj):
>     return clifford(obj).instance[0]
> cdef class clifford:
>     cdef multivector[scalar_t] *instance # hold a C++ instance which we're
> wrapping
>     cdef inline wrap(clifford self, multivector[scalar_t] other):
>         self.instance[0] = other
>         return self
>     cdef inline multivector[scalar_t] unwrap(clifford self):
>         return self.instance[0]
>     cdef inline copy(clifford self):
>         return clifford(self)
>     def __cinit__(self, other = 0):
>         t = type(other)
>         if   t == clifford:
>             self.instance = new
> multivector[scalar_t]((<clifford>other).unwrap())
>         elif t == types.IntType or t == types.LongType or t ==
> types.FloatType:
>             self.instance =  < multivector_scalar_t* > new
> multivector[double](<double>other)
>         else:
>             s = repr(other)
>             self.instance = new multivector[scalar_t](<char *>s)
>     def __dealloc__(self):
>         del self.instance
>     def __richcmp__(lhs, rhs, int op):
>         if op == 2: # ==
>             return bool( toClifford(lhs) == toClifford(rhs) )
>         elif op == 3: # !=
>             return bool( toClifford(lhs) != toClifford(rhs) )
>         else:
>             return NotImplemented
>     def __coerce__(lhs, rhs):
>         return clifford(lhs), clifford(rhs)
>     def __repr__(self):
>         return clifford_to_repr( self.unwrap() ).c_str()
>
>

I added some more bubble wrap to the compiler, you can retry with the
latest cython master.

Re: Numpy support

Robert

Have just got our Python/Numpy program to work under Cython without
any modification.  The performance is just a little better.  The next
step is to add static typing and hence the above questions.

Are you suggesting to try:

cdef int* indices = ...
cdef np.ndarray[int, dim=3] A
cdef int value = A[indices]  # Perhaps require A[indices[:3]]...

Or, are you saying it would be interesting if Cython supported this in
the future?

https://github.com/markflorisson88/cython.git ?

Re: _import_umath error when compiling code

>> Xtemp1 and kX are defined as np.ndarray
>> b is a function input and defined as a np.ndarray(np.double_t, ndim=2)
>> All counters are defined as ints.

you can get better performance if you type variables you are using to 
index as:

unsigned int

See:

http://wiki.cython.org/tutorials/numpy#Tuningindexingfurther

-Chris

--

-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker <at> noaa.gov