Daniele Nicolodi | 25 Oct 03:36 2014
Picon

[Cython] [PATCH] explain how to compile C++ extensions up to Cython 0.21

---
 docs/src/userguide/wrapping_CPlusPlus.rst | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/docs/src/userguide/wrapping_CPlusPlus.rst b/docs/src/userguide/wrapping_CPlusPlus.rst
index 59c89a1..5119b63 100644
--- a/docs/src/userguide/wrapping_CPlusPlus.rst
+++ b/docs/src/userguide/wrapping_CPlusPlus.rst
 <at>  <at>  -141,6 +141,21  <at>  <at>  Note that the ``language`` option has no effect on user provided Extension
 objects that are passed into ``cythonize()``.  It is only used for modules
 found by file name (as in the example above).

+The ``cythonize()`` function in Cython versions up to 0.21 does not
+recognize the ``language`` option and it needs to be specified as an
+option to an :class:`Extension` that describes your extension and that
+is then handled by ``cythonize()`` as follows::
+
+   from distutils.core import setup, Extension
+   from Cython.Build import cythonize
+
+   setup(ext_modules = cythonize(Extension(
+              "rect",                                # the extesion name
+              sources=["rect.pyx", "Rectangle.cpp"], # the Cython source and
+                                                     # additional C++ source files
+              language="c++",                        # generate and compile C++ code
+         )))
+
 The options can also be passed directly from the source file, which is
 often preferable (and overrides any global option).  Starting with
 version 0.17, Cython also allows to pass external source files into the
(Continue reading)

Shang Yu | 20 Oct 00:54 2014
Picon

[Cython] Fwd: wrong c file

Sorry . I've found it's because of the following line

#cython:callspec="__hello",language_level=2

---------- Forwarded message ----------
From: Shang Yu <yusunn@...>
Date: 2014-10-19 22:10 GMT+08:00
Subject: wrong c file
To: cython-devel@...

Hi dear all,
I'm trying cyhotn(0.21) with following code (hello.pyx)

cdef void cfun():
print "hello world"

if __name__ == "__main__":
cfun()

the generated c file has following line

static void "__hello" __pyx_f_5hello_cfun(void) {

which can't compiled by c compiler , what's wrong with this ?
Many thanks !!!
Shang Yu | 19 Oct 16:10 2014
Picon

[Cython] wrong c file

Hi dear all,
I'm trying cyhotn(0.21) with following code (hello.pyx)

cdef void cfun():
print "hello world"

if __name__ == "__main__":
cfun()

the generated c file has following line

static void "__hello" __pyx_f_5hello_cfun(void) {

which can't compiled by c compiler , what's wrong with this ?
Many thanks !!!
Patrick Snape | 15 Oct 20:27 2014
Picon

[Cython] Strange bug? with np.int64_t

System information:

    Ubuntu 14.04 x64
    gcc: Version 4.8.2 (Ubuntu 4.8.2-19ubuntu1), Target: x86_64-linux-gnu


To reproduce:

    # distutils: language = c++
    import numpy as np
    cimport numpy as np
    cimport cython


    cdef test_np_int64(const np.int64_t test_var):
        cdef np.int64_t error = test_var / 2


This fails to compile with the error:

    In function ‘const __pyx_t_5numpy_int64_t     __Pyx_div___pyx_t_5numpy_int64_t__const__(__pyx_t_5numpy_int64_t, __pyx_t_5numpy_int64_t)’:
    error: assignment of read-only variable ‘q’
         q -= ((r != 0) & ((r ^ b) < 0));


Using just a regular int type:

    # distutils: language = c++
    import numpy as np
    cimport numpy as np
    cimport cython


    cdef test_int(const int test_var):
        cdef int no_error = test_var / 2

Works as expected.

Fixing this is as simple as removing the const that is being complained about. I assume this is a bug? I can't see why I wouldn't be allowed to perform that operation?

Regards,

Patrick
<div><div dir="ltr">
<div>System information:</div>
<div><br></div>
<div>&nbsp; &nbsp; Ubuntu 14.04 x64</div>
<div>&nbsp; &nbsp; gcc: Version 4.8.2 (Ubuntu 4.8.2-19ubuntu1), Target: x86_64-linux-gnu</div>
<div><br></div>
<div><br></div>To reproduce:<div><br></div>
<div>
<div>&nbsp; &nbsp; # distutils: language = c++</div>
<div>&nbsp; &nbsp;&nbsp;import numpy as np</div>
<div>&nbsp; &nbsp;&nbsp;cimport numpy as np</div>
<div>&nbsp; &nbsp;&nbsp;cimport cython</div>
<div><br></div>
<div><br></div>
<div>&nbsp; &nbsp;&nbsp;cdef test_np_int64(const np.int64_t test_var):</div>
<div>&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;cdef np.int64_t error = test_var / 2</div>
<div><br></div>
<div><br></div>
</div>
<div>This fails to compile with the error:</div>
<div><br></div>
<div>
<div>&nbsp; &nbsp;&nbsp;In function &lsquo;const __pyx_t_5numpy_int64_t&nbsp;&nbsp; &nbsp;&nbsp;__Pyx_div___pyx_t_5numpy_int64_t__const__(__pyx_t_5numpy_int64_t, __pyx_t_5numpy_int64_t)&rsquo;:</div>
<div>&nbsp; &nbsp;&nbsp;error: assignment of read-only variable &lsquo;q&rsquo;</div>
<div>&nbsp; &nbsp; &nbsp;&nbsp; &nbsp;&nbsp;q -= ((r != 0) &amp; ((r ^ b) &lt; 0));</div>
</div>
<div><br></div>
<div><br></div>
<div>Using just a regular int type:</div>
<div><br></div>
<div>
<div>
<div>&nbsp; &nbsp; # distutils: language = c++</div>
<div>&nbsp; &nbsp;&nbsp;import numpy as np</div>
<div>&nbsp; &nbsp;&nbsp;cimport numpy as np</div>
<div>&nbsp; &nbsp;&nbsp;cimport cython</div>
</div>
<div><br></div>
<div><br></div>
<div>&nbsp; &nbsp; cdef test_int(const int test_var):</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; cdef int no_error = test_var / 2</div>
</div>
<div><br></div>
<div>Works as expected.</div>
<div><br></div>
<div>Fixing this is as simple as removing the const that is being complained about. I assume this is a bug? I can't see why I wouldn't be allowed to perform that operation?</div>
<div><br></div>
<div>Regards,</div>
<div><br></div>
<div>Patrick</div>
</div></div>
Stefan Behnel | 12 Oct 06:52 2014
Picon

[Cython] accidental breakage of 0.20.x branch

Hi,

I accidentally updated the 0.20.x branch to recent 0.21.x (typo), and then
had to force push it over to fix it. Sorry for that, I hope it doesn't
cause too much trouble.

Stefan
Николай | 10 Oct 15:04 2014
Picon

[Cython] compiler bug in 0.21

I found a bug in cython v0.21
Test source is attached.

regards,
Niki
--
|  (\_/)  This is Bunny. Copy and paste
| (='.'=) Bunny into your signature to help
| (")_(") him gain world domination
Attachment (cython_err.pyx): text/x-python, 182 bytes
Attachment (cy.trace): application/octet-stream, 6139 bytes
<div><div dir="ltr">I found a bug in cython v0.21<div>Test source is attached.<br clear="all"><div><br></div>
<div>regards,</div>
<div>Niki</div>-- <br>|&nbsp; (\_/)&nbsp; This is Bunny. Copy and paste<br>| (='.'=) Bunny into your signature to help<br>| (")_(") him gain world domination
</div>
</div></div>
Joel Nothman | 14 Sep 13:06 2014
Picon

[Cython] Variable shadows type name in posix/time.pxd

a struct and a double are defined with the name 'timezone' in posix/time.pxd. Hence

from posix.time cimport timeval, timezone, gettimeofday
cdef timeval tv
cdef timezone tz
gettimeofday(&tv, &tz)

fails with 'timezone' is not a type identifier

It would be nice if there were a test for duplicate names in pxd files.

Thanks for the awesome tool, Joel


<div><div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">a <a href="https://github.com/cython/cython/blob/master/Cython/Includes/posix/time.pxd#L48" target="_blank">struct</a> and a <a href="https://github.com/cython/cython/blob/master/Cython/Includes/posix/time.pxd#L105" target="_blank">double</a> are defined with the name 'timezone' in posix/time.pxd. Hence<div><br></div>
<div>from posix.time cimport timeval, timezone, gettimeofday</div>
<div>cdef timeval tv</div>
<div>cdef timezone tz</div>
<div>gettimeofday(&amp;tv, &amp;tz)</div>
<div><br></div>
<div>fails with 'timezone' is not a type identifier</div>
<div><br></div>
<div>It would be nice if there were a test for duplicate names in pxd files.</div>
<div><br></div>
<div>Thanks for the awesome tool, Joel</div>
<div><br></div>
</div>
</div>
<br>
</div></div>
Stefan Behnel | 5 Sep 21:39 2014
Picon

[Cython] preparing a release candidate, pending issues

Hi,

current master looks good enough to plan for a release candidate next week.
There are only a couple of pending issues that I'm aware of that may or may
not get looked into for the release.

One thing is cygdb and its tests - are they working again? They seem to
work for me, but I don't know what the last status was there. Given how
long they must have been broken, it doesn't look all too critical, though.

This one should be resolved now:

"build failure on windows with 0.21b2 windows py27 x64"
http://thread.gmane.org/gmane.comp.python.cython.devel/15424

This isn't, but the solution isn't currently obvious (and I think it can
wait until the first bugfix release if we can't find a fix in time):

"Buffer dtype mismatch error"
http://thread.gmane.org/gmane.comp.python.cython.user/11603

These are open, but don't feel like they should block the release:

"error LNK2001: unresolved external symbol PyInit_init"
http://thread.gmane.org/gmane.comp.python.cython.devel/15392/focus=15415

"Memory leak with vector of memoryviews"
http://thread.gmane.org/gmane.comp.python.cython.user/11704/focus=11705

"how to have hashable extension types in pure python mode ? (__eq__ error)"
http://thread.gmane.org/gmane.comp.python.cython.user/11731/focus=11732

This isn't finished but is a new feature which can wait:

"Automatic conversion with fixed-size C arrays"
https://github.com/cython/cython/pull/308

This hasn't lead anywhere, apparently, but can wait as well:

"OpenMP thread private variable not recognized (bug report + discussion)"
http://thread.gmane.org/gmane.comp.python.cython.devel/15382

So, nothing critical, but I might have forgotten something. If nothing else
comes up, I'll just wait a couple of more days and then start preparing RC
and final release.

Stefan
Dirk Rothe | 1 Sep 09:55 2014
Picon

[Cython] build failure on windows with 0.21b2 windows py27 x64

Hello Stefan,

the Problem reported by Ian Bell on v0.21b1 seems still to be there. With
v0.21a1 from https://github.com/cython/cython/releases/tag/0.21a1
everything seems to be fine. So maybe it's related to the method call
optimisations, you've mentioned.

I've also tested with windows+py27+x64 and I get the same errors;

configobj.c(42437) : error C2121: '#' : invalid character : possibly the
result of a macro expansion
configobj.c(42437) : error C2146: syntax error : missing ')' before
identifier 'ifdef'
configobj.c(42437) : error C2121: '#' : invalid character : possibly the
result of a macro expansion
configobj.c(42439) : error C2059: syntax error : 'else'
configobj.c(42449) : error C2059: syntax error : '}'
configobj.c(42463) : error C2121: '#' : invalid character : possibly the
result of a macro expansion
configobj.c(42463) : error C2121: '#' : invalid character : possibly the
result of a macro expansion
error: command '"C:\Program Files (x86)\Microsoft Visual Studio
9.0\VC\BIN\amd64\cl.exe"' failed with exit status 2

I've cythonized the following Code:
http://bpaste.net/show/a10f8b244c2d

with a setup.py like:
---------------------

import sys
   from setuptools import setup
   from setuptools.extension import Extension

   from Cython.Distutils import build_ext

extensions = [Extension("configobj", ["configobj.py"])]
setup_info = dict(ext_modules=extensions, cmdclass={'build_ext':
build_ext})
sys.argv.extend(["build_ext", "-i"])
setup(**setup_info)

--dirk
Alexandru Ionut Grama | 26 Aug 17:45 2014
Picon

[Cython] [Desired feature] Process .pxd before .py

Hello to all

I've start to use cython in order to use call some API modules written in python from C/C++. Those modules written in python must remain untouched, and all .py files should be "decorated" with their counterpart .pxd files.
I would like to mark as public some API functions from .py files, following the indications from http://docs.cython.org/src/tutorial/pxd_files.html. I've started to write a little example:

suma_alex.py:
def suma_Alex(a,b):
        return a+b

suma_alex.pxd:
cdef public int suma_Alex(int a,int b)


The combination of those two files should turn into something like this for the compiler:

cdef public int suma_Alex(int a,int b):
      return a+b

On generated .h file I should obtain a function prototype like this:
__PYX_EXTERN_C DL_IMPORT(int) suma_Alex(int, int);

Instead of that, I obtain a prototype like this:
__PYX_EXTERN_C DL_IMPORT(int) __pyx_f_10suma_alex_suma_Alex(int, int);

Digging into code and executing cythonize with pdb, I followed that is executed a pipeline for .py file before .pxd file's pipeline. Correct me if I'm wrong, but this makes the name of .c function be generated according to py file instead of pxd file. That makes an add of string "__pyx_f_10suma_alex_" before function name, because the compiler doesn't know about "public" key (on python doesn't exists). Including public key on a pxd file for this function makes the compiler remove "static" key and generate a .h file, but doesn't change the name prototype into a simple "suma_Alex".

If I use cython on a pyx file containing the code below, it works perfectly as expected.
cdef public int suma_Alex(int a,int b):
      return a+b

The questions are:
- I'm doing what I want to do on a right way?
- Is this feature included on cython development version? I've tried on 0.20.2
- If is not implemented, I want to colaborate implementing it. Could you please provide me some guidelines with the methods/modules that I should modify?

King regards,
Alexandru.

--
---------------------------------------------------------------
Alexandru Ionut Grama
email: gramaalexandruionut-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
<div><div dir="ltr"><div>
<div>
<div>Hello to all</div>
<div><br></div>
<div>I've start to use cython in order to use call some API modules written in python from C/C++. Those modules written in python must remain untouched, and all .py files should be "decorated" with their counterpart .pxd files.</div>

<div>I would like to mark as public some API functions from .py files, following the indications from <a href="http://docs.cython.org/src/tutorial/pxd_files.html">http://docs.cython.org/src/tutorial/pxd_files.html</a>. I've started to write a little example:</div>

<div><br></div>
<div>suma_alex.py:</div>
<div>def suma_Alex(a,b):</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; return a+b</div>
<div><br></div>
<div>suma_alex.pxd:</div>
<div>cdef public int suma_Alex(int a,int b)</div>
<div><br></div>
<div><br></div>
<div>

The combination of those two files should turn into something like this for the compiler:</div>
<div><br></div>
<div>cdef public int suma_Alex(int a,int b):</div>
<div>&nbsp; &nbsp; &nbsp; return a+b</div>
<div><br></div>
<div>On generated .h file I should obtain a function prototype like this:</div>

<div>__PYX_EXTERN_C DL_IMPORT(int) suma_Alex(int, int);</div>
<div><br></div>
<div>Instead of that, I obtain a prototype like this:</div>
<div>__PYX_EXTERN_C DL_IMPORT(int) __pyx_f_10suma_alex_suma_Alex(int, int);</div>
<div>

<br>
</div>
<div>Digging into code and executing cythonize with pdb, I followed that is executed a pipeline for .py file before .pxd file's pipeline. Correct me if I'm wrong, but this makes the name of .c function be generated according to py file instead of pxd file. That makes an add of string "__pyx_f_10suma_alex_" before function name, because the compiler doesn't know about "public" key (on python doesn't exists). Including public key on a pxd file for this function makes the compiler remove "static" key and generate a .h file, but doesn't change the name prototype into a simple "suma_Alex".</div>

<div><br></div>
<div>If I use cython on a pyx file containing the code below, it works perfectly as expected.</div>
<div>cdef public int suma_Alex(int a,int b):</div>
<div>&nbsp; &nbsp; &nbsp; return a+b</div>
<div><br></div>
<div>The questions are:</div>

<div>- I'm doing what I want to do on a right way?</div>
<div>- Is this feature included on cython development version? I've tried on 0.20.2</div>
<div>- If is not implemented, I want to colaborate implementing it. Could you please provide me some guidelines with the methods/modules that I should modify?</div>

<div><br></div>
<div>King regards,</div>
<div>Alexandru.</div>
</div>
<div><br></div>-- <br>---------------------------------------------------------------<br>Alexandru Ionut Grama<br>email: <a href="mailto:gramaalexandruionut@..." target="_blank">gramaalexandruionut@...</a><br>
</div></div></div>
Robert Bradshaw | 24 Aug 07:46 2014
Picon

[Cython] Towards a formal Cython grammar

I've started playing around with writing a grammar for Cython. As well
as formally defining the language (in particular, with respect to
Python) this should allow us to eventually move to using
parser-generators rather than ad-hoc hand written code and be useful
to  external tools (e.g. IDEs, linters, syntax highlighting, etc.)

I've posted what I have at
https://github.com/robertwb/cython/tree/grammar , in particular

https://github.com/robertwb/cython/blob/grammar/Cython/Parser/Grammar
https://github.com/robertwb/cython/compare/robertwb:3aa9056943f83a68cc9d9335f8a9c81e9a6f3f91...363bc162fd626203f832a33b6c736ff8b10f6086#diff-8b69afcfc588fde3d763f2ec670e42c2L1

Nothing beyond generating the raw parse tree is hooked up yet, and
even that requires an extra directive (formal_grammar=True). Building
and using the parser requires a source-built Python (it looks for the
pgen artifact Python uses to compile the grammar). We may or may not
want to stick with this approach long term (though if we do we might
ship the generated .c files). This parse tree is still pretty
low-level, we might want to also create something like Python.asdl to
give us a (closer) AST.

The grammar isn't complete, but should cover a most of the language
(over 3/4 of the test suite passes, and that explores a lot of the
corner cases). The most notable omissions are that it's using Python's
lexer, so doesn't have a token for '?' or the additional literal
string prefixes/int literal suffixes. Also, as the lexer clearly
doesn't understand includes, these are handled by inlining in a
preparsing step (which messes with line numbers).

The grammar could be tightened up as well. For example, this grammar
doesn't distinguish between valid pxd vs. pyx constructs, and allows
cdef statements within if statements (or even normal class
declarations).

I tried to restrict the existing grammar as little as possible, in
particular the only new illegal identifiers are 'cdef', 'ctypedef' and
(for ambiguity reasons new' and 'sizeof'). Also, the "cython" keywords
may not be used for identifiers that might be typed (e.g. "def
foo(int): ..." is not allowed).

The most significant departure from the existing "grammar" is that
rather than using C-style declarators, cdef declarations are of the
form "cdef [type] name." Thus one write the (already legal)

    cdef double[3] loc

and

    cdef double* a, b

declares two pointers. How to handle function pointers is still up in
the air, but I wouldn't be opposed to moving to a new syntax (e.g.
"(double, void*) -> double" inspired by Py3) for those. It disallows
empty "declarators" for parameters of function declarations (though we
could consider adding this back).

i think this would also be a nice chance to simplify the grammar, so
there are some intentional ommisions. Most notably, there are several
modifiers (e.g the __cdecl, __stdcall, __fastcall callspecs, maybe
inline, maybe even "with nogil", and the "cdef class Foo [object
object_struct_name, type type_object_name ]" spec for external
classes) that would make more sense as decorators. This would be
backwards incompatible, but these are not commonly used features and
fair warning (or even translators, I included a sed script to deal
with the common case of declarators mentioned above) could be given
and I think could be worth the simplification.

Thoughts?

- Robert

Gmane