Newbie Questions about speeding up some Numpy calculations

Hi All,

I am pretty new to Cython, so please bare with me. I am at the moment trying to speed up some code that I use a lot
in calculations, so  thought this might be a good idea to learn how to use Cython when I have already some
working Numpy code. 

After searching the tutorials I came up with the attached solution, however compared with the original
Numpy code this is getting slightly slower rather than faster, so obviously I am doing something very
wrong. Could somebody please give me some pointers where I can find examples that are closer to what I am
trying to speed up than what I can find in the tutorials?

Attached are the original numpy code, as well as the Cython code I have tried to develop. If I run this on my
machine, I get:

Python 2.7.2 |EPD 7.2-2 (32-bit)| (default, Sep  7 2011, 09:16:50) 
Type "copyright", "credits" or "license" for more information.

IPython 0.12 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import be

In [2]: timeit be.referencecase(samples=250000)
1 loops, best of 3: 2.12 s per loop

In [3]: import be1

Large C Arrays to Python lists

Cython based 2D Game Engine

Templates in Sage

Hi everyone,
I've been using Cython in Sage for many years, but haven't posted to
this list before.

Sage currently uses .pxi files to model C++'s templates in Cython.
The code was written about 3 years ago, and I want to apply the same
paradigm elsewhere in the Sage library.  But William Stein (lead
developer of Sage) suggested that including .pxi files may not be a
good choice since that practice was discouraged by the Cython
community.  In case you're not familiar with the relevant code in
Sage, let me summarize what it does, and then I have a few questions.

The idea is that it allows multiple C library back-ends to be plugged
into the same Cython code (implementing polynomials in this case).  In
the following snippets celement is typedefed at the appropriate place
to allow compile-time linkage between the polynomial template and the
back-end znpoly library.  For the full code see
http://hg.sagemath.org/sage-main/file/c239be1054e0/sage/rings/polynomial.

sage/libs/flint/zmod_poly_linkage.pxi --
...
cdef inline int celement_add(zmod_poly_t res, zmod_poly_t a,
zmod_poly_t b, unsigned long n) except -2:
    zmod_poly_add(res, a, b)
...

sage/rings/polynomial/polynomial_template_header.pxi
...
cef class Polynomial_template(Polynomial):
    celement x
...

sage/rings/polynomial/polynomial_template.pxi
...
cdef class Polynomial_template(Polynomial):
    ...
    def __add__(self, other):
        ...
        celement_add(ans.x, self.x, other.x, n)
...

sage/rings/polynomial/polynomial_zmod_flint.pxd
...
ctypedef zmod_poly_struct celement
include "polynomial_template_header.pxi"
cdef class Polynomial_zmod_flint(Polynomial_template):
...

sage/rings/polynomial/polynomial_zmod_flint.pyx
...
include "../../libs/flint/zmod_poly_linkage.pxi"
include "polynomial_template.pxi"
cdef class Polynomial_zmod_flint(Polynomial_template):
...

I want to reuse this paradigm to implement something else in Sage.  So
now for the questions.

* Is there a reason why using includes in this way is a bad idea?
* Is there a way to get compile time linking with different C-library
backends in cython without including pxi files?
* Can one ctypedef a cython extension class?  I seem to get errors
when doing:
cdef class A:
    pass
ctypedef A my_class

Thanks for the help,
David
*

Make enumeration "public" to Python?

Is there a way to make enumerations available to Python automatically?
This would be a real time-saver as it is almost always necessary to do
this. Something similiar to this would be nice:

cdef extern from *:
    public enum:
        foo
        bar

It should basically replace the following:

cdef extern from *:
    enum:
        cfoo "foo"
        cbar "bar"

foo = cfoo
bar = cbar

-Niklas

Fast indexing of a boolean numpy array

cdef public class and vtab export

I am pretty new to Cython but have wrapped C/C++ using other methods
(C api, PyCXX, Boost::Python, and ctypes.) I'm really enjoying Cython
so far, but have the following question:

I have a c++ class which needs to invoke a bound Python method. The
cleanest way I see is to have the usual Cython extension class
(declared 'cdef public class..) with a thisptr to the c++ object, and
have the c++ object have a void *py_obj which holds a pointer to the
extension class struct. So far, so good - except that when the c++
wants to call a bound method in the Cython extension class, it needs
the virtual function table which Cython emits in the .cpp source, but
not in the .h file (generated because I declared the class public.) I
find it odd that the vtab struct is not exported into the .h file,
because the extension class object structure which gets exported
includes a member called __pyx_vtab, but the actual structure of the
vtab isn't in the .h file; so I can *almost* get everything I need! Am
I missing something? A keyword perhaps, which causes the vtab to be
exported?

===== cython code
cdef public class MyTestClass(object)[object MyTestClassObject, type
MyTestType] :
    cdef TPyBathControl *ths      # hold a C++ instance which we're
wrapping
              # This is the class whose c++ method wants to call this
object's cb method.

    def __cinit__(self):
        self.ths = new TPyBathControl()
        self.ths.py_obj = <void *>self   # put reference to this
object in the cpp object

    def __dealloc__(self):
        del self.ths

    cdef void cb(MyTestClass self):
        print 'called python'

===== generated .h file snippet
struct MyTestClassObject {
  PyObject_HEAD
  struct __pyx_vtabstruct_11basf_ext_cy_MyTestClass *__pyx_vtab;
  TPyBathControl *ths;
};
===== What's missing from .h file?
struct __pyx_vtabstruct_11basf_ext_cy_MyTestClass ...

Thanks in advance for any advice.

problem running embedded cython program

Hi all,

I managed to compile the embedded.pyx program into a *.exe using the --
embed feature of cython.  The program runs fine at the compiled
location.  But after I move the *.exe file to a different folder on
the same computer, it won't run, with the error box:

"The program can't start because MSVCR90.dll' is missing from your
compute.  Try reinstalling the program to fix this problem."

I am on windows 7 64bit.

Any one has suggestions?

Thanks

cythonize and support files (c/cpp)

Calling free() from numpy.ndarray.__del__

I create numpy.ndarray objects from dynamic allocated arrays of
structs.
How can I tell these ndarray objects to deallocate such memory by
themselves?

pseudo code:
cimport numpy
from libc.stdlib cimport malloc, free
cdef extern from "my_struct.h"
	struct MY_STRUCT:
		(some fieldd here)
	MY_STRUCT* data
(in a function):
	data = <MY_STRUCT *>malloc( my_size )
	array = numpy.asarray(data)

I need to call 'free(data)' when the object 'array' is no longer
referenced anywhere.
I would solve providing a custom __del__ method to the instance
'array', but it doesn't work: "AttributeError: 'numpy.ndarray' object
has no attribute '__del__'." when doing something like array.__del__ =
types.MethodType(my_del_func, array, array.__class__).

Cython trunk
NumPy 1.5.1
Python 2.7.1

Thanks,
alef

c++ template trouble

Hello all,

i'm having trouble compiling https://github.com/bshanks/kf . The exact
error message is at the bottom of that webpage (at the bottom of the
README). In summary,

1) The C++ code i am wrapping is used like this:

  KF<> kf;
  vector<double> x(1);
  x(0) = 1.0;
  kf(0.0, x);

  The template parameter is a size_t of the vector, e.g. for a length
2 vector:

  KF<2> kf(.999, vector<double>(2, .999));

2) Here's an excerpt from the underlying C++ code (written by Max
Dama, not by me):

  template <std::size_t p = 1>
  class KF
  {
  public:
    KF(double beta = .99, const vector<double>& Delta = vector<double>
(p, .99))

3) Here's an excerpt from my Cython wrapper (the whole thing can be
found at https://github.com/bshanks/kf ):

  cdef extern from "kf.cpp":
      cdef cppclass KF[size_t]:
          KF(double, vector[double])

  cdef class PyKf:
      cdef KF[size_t] *kf

      def __cinit__(self, double beta = 0.99, delta = [.99]):
          cdef vector[double] delta_vec

          for i in range(len(delta)):
              delta_vec.push_back(delta[i])
          self.kf = new KF(beta, delta_vec)

4) Here's an excerpt from the error message:

  PyKf.cpp:318:12: error: type/value mismatch at argument 1 in
template parameter list for ‘template<long unsigned int p> class KF’
  PyKf.cpp:318:12: error:   expected a constant of type ‘long unsigned
int’, got ‘size_t {aka long unsigned int}’

thanks,
  bayle