Richard Hansen | 30 Jan 08:17 2015
Picon

typed memoryviews: support for read-only buffer objects?

Hi all,

How can I write a Cython function that takes a byte string object (a
normal byte string, a bytearray, or another object that follows the
buffer protocol [1]) as a typed memoryview [2]?  I'm using Python 2.7.

According to the Unicode and Passing Strings Cython tutorial page [3],
the following should work:

    cpdef object printbuf(unsigned char[:] buf):
        chars = [chr(x) for x in buf]
        print repr(''.join(chars))

Unfortunately it doesn't work for normal byte strings and other
read-only buffer objects:

    $ python -c 'import test; test.printbuf("test\0ing")'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "test.pyx", line 1, in test.printbuf (test.c:1417)
      File "stringsource", line 614, in
View.MemoryView.memoryview_cwrapper (test.c:6795)
      File "stringsource", line 321, in
View.MemoryView.memoryview.__cinit__ (test.c:3341)
    BufferError: Object is not writable.

But it does work for bytearrays and other writable buffers:

    $ python -c 'import test; test.printbuf(bytearray("test\0ing"))'
    'test\x00ing'
(Continue reading)

Toni Barth | 29 Jan 17:23 2015

Accessing python functions in callbacks with return values

Hello Cython users,

i'm currently facing a quite interesting problem.
I already wrote a callback function (declared as cdef) which gets called 
by a binary c file I linked to. This callback, without a return value 
(means void) can access python functions, means it can call a user 
defined python function, so it can be used as callback bridge to call a 
python function as real callback, just like the cython example.
But when this C callback function gets a return value, things get much 
stranger. Since the cython source doesn't contain a demo with return 
value, I wasn't able to check it out.
Each time the C binary wants to execute my c function which now has a 
return value (which is compatible to the binary's definitions), the code 
causes a segfault at the point where a pythonic function is called.
I also tried commenting this step out and try to find a solution, but 
then the code crashes at the next pythonic function call, so it seems to 
have problems calling into the python namespace when called by the 
binary callback thread.
I also read that a c library which wants to call cython code needs to 
execute Py_Initialize() first, but since it's a binary i'm unable to do so.
It's also curious that a cython callback without return value is able to 
call into the python namespace, but a cython callback with return value 
isn't.
Since my c callback with return value needs to call a python function 
(since it's a callback bridge again) there is no way to get around 
calling a pythonic function.

Did anyone of you face that problem before and maybe found a, if 
possible, cython-python-only solution?

(Continue reading)

Jacob Schreiber | 30 Jan 03:24 2015
Picon

Looking for help writing a setup.py

Hello all!

I've been writing a package in cython which compiles and works locally if the pyx and pxd files are in the appropriate folder in the pythonpath, but when I try to install it using pip it fails. I assume that the problem is that I've messed up writing the setup.py. I have found limited documentation on writing setup.pys for larger packages as opposed to single scripts, so I thought I'd ask you guys!

The package is located here; https://github.com/jmschrei/pomegranate

The discussion of the issue, including logs of what happens when I try to install it on a Windows machine and another user tries to install it on a Ubuntu is here: https://github.com/jmschrei/pomegranate/issues/1

Any advice is extremely appreciated.

Jacob

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Valcortez | 30 Jan 00:27 2015
Picon

Importing cython generated cpp extension after importing c extension causes a crash on exit.

Hi guys, there's a similar issue reported before (https://groups.google.com/forum/#!topic/cython-users/pLiLrbdZpMo), but there was no responses so I'm posting with a fully standalone example replicating the issue. If there's a mailing list more appropriate, please let me know where to ask.

File structure:

    src/
        cypp_playground.pyx
        cyplayground.pyx
    setup.py


cypp_playground.pyx contents:

class MyClassA(object):

   
pass




cyplayground.pyx contents:

class MyClassB(object):

   
pass




setup.py contents:

from distutils.core import setup
from distutils.extension import Extension
import Cython.Compiler.Options
#Cython.Compiler.Options.annotate = True
from Cython.Distutils import build_ext
from os.path import join, sep, dirname, abspath


def get_modulename_from_file(filename):
    filename
= filename.replace(sep, '/')
    pyx
= '.'.join(filename.split('.')[:-1])
    pyxl
= pyx.split('/')
   
while pyxl[0] != 'src':
        pyxl
.pop(0)
   
if pyxl[1] == 'src':
        pyxl
.pop(0)
   
return '.'.join(pyxl)


def expand(*args):
   
return abspath(join(dirname(__file__), 'src', *args))


def get_extensions_from_sources():
    ext_modules
= []
    pyx
= expand('cyplayground.pyx')
    ext_modules
.append(Extension(get_modulename_from_file(pyx), [pyx]))
    pyx
= expand('cypp_playground.pyx')
    ext_modules
.append(
       
Extension(get_modulename_from_file(pyx), [pyx], language='c++'))
   
return ext_modules

ext_modules
= get_extensions_from_sources()

for e in ext_modules:
    e
.cython_directives = {'embedsignature': True,
                           
'c_string_encoding': 'utf-8'}

setup
(
    name
='Playground',
    version
='.1',
    author
='Matthew Einhorn',
    ext_modules
=ext_modules,
    cmdclass
={'build_ext': build_ext},
    packages
=['src']
     
)


Then we compile with `python setup.py build_ext --inplace`.

Finally, we do:

G:\Python\libs\Playground\src>python
Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cyplayground
>>> import cypp_playground
>>> exit()

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

G:\Python\libs\Playground\src>python
Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cypp_playground
>>> import cyplayground
>>> exit()

G:\Python\libs\Playground\src>

The difference between cyplayground and cypp_playground is that the latter is compiled with language='c++', while the former is c. The result is if you first import the c compiled extension and then c++ compiled extension, when exiting it crashes, but it doesn't happen when importing in reverse order.

Also, this only happens with more recent versions of mingw (I think, but not 100%). Here's my system info:

Win7

G:\Python\libs\Playground\src>python --version
Python 2.7.8

G
:\Python\libs\Playground\src>gcc -v
Using built-in specs.
COLLECT_GCC
=gcc
COLLECT_LTO_WRAPPER
=e:/mingw/bin/../libexec/gcc/mingw32/4.8.1/lto-wrapper.exe
Target: mingw32
Configured with: ../gcc-4.8.1/configure --prefix=/mingw --host=mingw32 --build=m
ingw32 --without-pic --enable-shared --enable-static --with-gnu-ld --enable-lto
--enable-libssp --disable-multilib --enable-languages=c,c++,fortran,objc,obj-c++
,ada --disable-sjlj-exceptions --with-dwarf2 --disable-win32-registry --enable-l
ibstdcxx-debug --enable-version-specific-runtime-libs --with-gmp=/
usr/src/pkg/gm
p
-5.1.2-1-mingw32-src/bld --with-mpc=/usr/src/pkg/mpc-1.0.1-1-mingw32-src/bld --
with-mpfr= --with-system-zlib --with-gnu-as --enable-decimal-float=yes --enable-
libgomp
--enable-threads --with-libiconv-prefix=/mingw32 --with-libintl-prefix=/
mingw
--disable-bootstrap LDFLAGS=-s CFLAGS=-D_USE_32BIT_TIME_T
Thread model: win32
gcc version
4.8.1 (GCC)

G
:\Python\libs\Playground\src>pip show cython
---
Name: Cython
Version: 0.21.2
Location: e:\python27\lib\site-packages
Requires:



--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Marcel Martin | 28 Jan 12:33 2015
Picon

Using cythonize conflicts with Cython-free builds

Hi,

when writing a Cython module, the recommendation is to specify distutils 
options such as 'sources', 'language', 'extra_compile_args' etc. within 
the pyx file itself using "# distutils: ..." directives. This was also 
one of the rationales given in CEP 201 for the existence of cythonize(), 
see 
<https://github.com/cython/cython/wiki/enhancements-distutils_preprocessing>.

Then there also exists the recommendation of shipping the .c/.cpp 
sources along with the package so that Cython isn't required for package 
installation. But this also means that no cythonize command is available 
and, in particular, that the source .pyx file with the C compiler 
options is never read. I've now included the compiler options in the 
setup.py file, but I think these conflicting recommendations should be 
reconciled. What’s the best practice here?

Regards,
Marcel

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Toni Barth | 27 Jan 19:18 2015

Problems with string conversion and buffer storage

Hi Folks,

i've got a kind of unresolvable problem, at least for me. I've got a c 
library which defines functions which accept callbacks which take a 
buffer of a given length as parameter and are ment to fill the buffer 
with relevant data, in a given format. Here the documentation excerpt 
relevant for the parameters:
Parameters
handle
The stream that needs writing.
buffer
Pointer to the buffer to write the sample data in. The data should be as 
follows: 8-bit samples are unsigned, 16-bit samples are signed, 32-bit 
floating-point samples range from -1 to +1.
length
The maximum number of bytes to write.
user
The user instance data given when BASS_StreamCreate was called.

I don't know if these information are relevant, but anyway.
At the moment i'm that far that I wrote a c callback function which 
wraps a python function, the actual callback function, which the 
python-user can define. This function should return the requested data. 
It should work just like reading from a file, passing these data as 
return value and the c function should fill the c buffer correctly and 
yeah... all should work fine. My callback python function looks like this:
def cb(stream,length,user):
  print 'cb fired!'
  print length
  print stream
  d=f.read(length)
  return d
f=open('test','rb')

So, the c callback takes the python return value and converts it to the 
type 'bytes', which should be fine, since a pythonic string is returned. 
It then checks the string length and shortens it if it is too long. It 
then casts the bytes object into a char * and memmoves the data to the 
buffer. It just looks like this:
cdef DWORD CSTREAMPROC(HSTREAM handle,void *buffer,DWORD length,void *user):
  cdef object cb,pythonf
  cdef char *cbuf
  cdef int pos,i
  cdef bytes pythonbuf
  cdef STREAM stream
  stream=STREAM(handle)
  pos=<int>user
  cb=callbacks.Callbacks.GetCallback(pos)
  pythonf=cb['function']
  pythonbuf=pythonf(stream,length,cb['user'])
  print 'got buf'
  blen=<DWORD>len(pythonbuf)
  print blen
  if blen>length:
   pythonbuf=pythonbuf[:length]
   blen=length
   print 'length is too large'
  cbuf=<char *>pythonbuf
  memmove(buffer,<const void*>cbuf,blen)
  print 'copying around'
  return blen

Yep, the c function must return the written bytes as DWORD. So, my 
current state looks like this. Calling the callback works one time. Then 
the program, which is a audio playback library as obvious, plays a 
whitenoise and the program crashes, indicating that something went wrong 
when filling the buffer with the relevant data, so i'm now starting to 
guess, since any debug attempts fail (Visual Studio 2012 doesn't return 
much and gdb under debian 8 just says 'warning: .cygdbinit: no such file 
or directory' when calling cygdb and returning some errors in the plain 
c code).
One thing is that file.read() doesn't return the data as expected/the 
right way, at least not as the c equivalent fread() does, because with 
this function it all works fine. Another idea is that I'll have to 
convert the data into the concerned bit formats before filling the buffer.
So anyway, I don't have any clue.

So, to help my by taking a look into the whole source code, just clone 
https://github.com/Timtam/Bass4Py.git
and checkout the cython branch. By calling the following calls, you'll 
have the full environment setup and are about to test:
import Bass4Py
bass=Bass4Py.BASS()
device=bass.GetDevice(-1)
device.Init(48000,0,0)
stream=bass.StreamCreate(48000,2,0,cb,0) # cb is the callback function 
concerned above.
stream.Channel.Play(False)
while True:
  # do something

This should do the trick. The relevant c callback function can be found 
in Bass4Py/bassstream.pyx

Honestly, it would be really nice if someone has some ideas to help me 
out. I'm quite new to c (actually learning it while studying for over 2 
or 3 months now) and therefore not good at it, so it might be that i'm 
missing something really easy.
Thanks for everyone who red through my brainfreeze and tries to look 
into that stuff. A feedback would be just great!

Best Regards.

Toni Barth

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Jeroen Demeyer | 26 Jan 11:48 2015
Picon

Defining custom __new__() for extension types?

When I try to redefine __new__() in an extension type I get

sage/rings/integer.pyx:506:4: __new__ method of extension type will 
change semantics in a future version of Pyrex and Cython. Use __cinit__ 
instead.

But I *really* want __new__ (more precisely, the tp_new slot) because I 
want to customize the allocation part. Is that possible in Cython?

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Richard Foltyn | 25 Jan 12:31 2015
Picon

Compiler directives precedence

Hi list,

I am wondering about the precedence of compiler directives specified at different locations.
The compilation documentation states that
"Directives passed on the command line will override directives set in header comments."

In fact, on cython 0.21.2 I cannot reproduce this behavior. For example, when using this demo code

# cython: boundscheck = True
# cython: cdivision = False

cdef double func1(double[::1] mv):
    cdef double s = 1.0
    cdef unsigned int i
    for i in range(mv.shape[0]):
        s /= mv[i]

    return s

and running

cython -3 -v -a -X boundscheck=False,cdivision=True cdirectives.pyx

the annotated HTML file shows that the resulting C code checks for both index errors and division by zero.
The command line directives are ignored, as are cython_directives or compiler_directives specified in a
setup.py file.

So is the documentation outdated, or am I doing something wrong? To me the actual precedence seems to be

1) locally specified directives via decorators
2) headers in pyx files
3) command-line options or options in setup.py

Is there then any way to override directives in headers using options in setup.py?

Thanks,
Richard

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Richard Foltyn | 25 Jan 12:37 2015
Picon

Disable nogil globally

Hi list,

is there some (potentially not documented) way to disable 'nogil' globally at compile time?

The use case is this: I have many cdef functions distributed over a dozen pyx/pxd files that are declared 'nogil'.

Unfortunately, using the profile=True compiler directive does not seem to work on nogil functions.
Hence every time I want to profile the code, I have to manually hunt down quite a lot or cdef function definitions/declarations
and delete the 'nogil', and reverse everything once done profiling, which is tedious.

Is there some smarter way to do this?

Thanks,
Richard

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Mathieu Courtois | 25 Jan 18:53 2015
Picon

how to generate -> operator ?

Hello,

My class C1 stores ptr2 a (shared pointer) to an object of class C2.
The operator -> is overloaded in C1. It returns the pointer ptr2.

So, I can write something as:
    obj1 = C1();
    obj1->method2();

that calls (obj1.ptr2)->method2().

How can I make cython generate a such code (with obj1->method2() ) ?
I'd like to avoid to declare the C2 class in cython.

I tried:
    ( <void *> (self.cppObj) ).method2( )

but it fails with error:
    Object of type 'void' has no attribute 'method2'


Any idea ?
Tell me if it's not clear.

Thanks in advance.
MC

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Jeroen Demeyer | 24 Jan 19:28 2015
Picon

Disabling GC for a type (custom tp_flags)

Hello,

is there any way in Cython to define an extension type without 
Py_TPFLAGS_HAVE_GC in its tp_flags? If it's not directly possible, what 
would you consider the least hackish way to accomplish this?

I am asking because Sage has a hack to remove this flag from its Integer 
class at runtime and I wonder if there is no cleaner way.

Jeroen.

--

-- 

--- 
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users+unsubscribe <at> googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Gmane