Raniere Silva | 8 Feb 15:09 2016
Picon

[Cython] Google Summer of Code 2016, GSoC2016

Hi all,

Since Cython is a NumFOCUS affiliated project
and NumFOCUS will apply to be a mentoring organization on GSoC
I want to know (1) if Cython is planning to apply this year
and (2) if want to apply under NumFOCUS umbrella.

Cython is welcome and encouraged to apply as separate mentoring
organizations directly with Google. We're happy to help you fill out your
application and improve your ideas pages, as well as link your page to help
students find you. We may also be able to be a reference for you.
It is totally fine if you want to use the NumFOCUS umbrella org
as a backup plan in case you don't get selected and we do!

Cheers,
Raniere
Hi all,

Since Cython is a NumFOCUS affiliated project
and NumFOCUS will apply to be a mentoring organization on GSoC
I want to know (1) if Cython is planning to apply this year
and (2) if want to apply under NumFOCUS umbrella.

Cython is welcome and encouraged to apply as separate mentoring
organizations directly with Google. We're happy to help you fill out your
application and improve your ideas pages, as well as link your page to help
students find you. We may also be able to be a reference for you.
It is totally fine if you want to use the NumFOCUS umbrella org
(Continue reading)

Nils Werner | 5 Feb 21:43 2016
Picon
Gravatar

[Cython] import-free setuptools integration

I have implemented some very simple setuptools integration that circumvents
the requirement to `import cython` in `setup.py` entirely, doing away with the
century-old problem of having to importing requirements during installation:


This implementation is inspired by CFFI's setuptools integration:

They do not require you to `import cffi` but simply define an additional keyword
argument `cffi_modules` for `setup()`. This additional keyword argument _does not
raise errors when cffi is not installed yet_ but can be used once cffi is there
(as defined in `setup_requires`). Later setuptools will call cffi and have it do
whatever it wants with the contents of the argument.

Cython can do the exact same. Below you have the usual `setup.py`, with the
chicken and egg `import` problem and a few `Extensions` that need to be `cythonize`d:

    import setuptools
    from Cython.Build import cythonize  # this will fail if cython is not present prior to running `pip install this`
    from setuptools.extension import Extension

    extensions = [
        Extension("fib", ["fib.pyx"]),
        Extension("fib2", ["fib2.pyx"]),
    ]

    setuptools.setup(
        name='example',
        version="1.0.0",
        packages=setuptools.find_packages(),
        setup_requires=['cython'],
        install_requires=['cython'],
        ext_modules=cythonize(extensions),
    )

With the changes I am proposing the usual

    ext_modules=cythonize(extensions),

can be replaced with

    cython_modules=extensions,

removing the need for `from Cython.Build import cythonize` and solving the import problem:

    import setuptools
    from setuptools.extension import Extension

    extensions = [
        Extension("fib", ["fib.pyx"]),
        Extension("fib2", ["fib2.pyx"]),
    ]

    setuptools.setup(
        name='example',
        version="1.0.0",
        setup_requires=['cython'],  # we must later require a Cython version that has this kind of integration
        install_requires=['cython'],
        cython_modules=extensions,
    )

This allows a nicer automated installation of tools that depend on cython and
also allows end users to keep their `setup.py` cleaner and leaner.

Additionally, I have also extended `cythonize()` a bit and thus allow for definitions like:

    import setuptools
    from setuptools.extension import Extension

    extensions = [
        Extension("fib", ["fib.c"]),
        Extension("fib2", ["fib2.c"]),
    ]

    setuptools.setup(
        name='example',
        version="1.0.0",
        extras_require={'cython': ['cython']},
        ext_modules=extensions,
        cython_modules=extensions,
    )

This will automatically compile pyx->c if cython is installed and a re-compilation is
needed. Otherwise it will merely compile c->so.

This allows devs to ship the generated C-code (as you suggest per your docs) but
takes the heuristic to find out whether to compile pyx->C->so or only C->so
completely out of `setup.py` and under control of cython.

The change is written to be backwards compatible:

 - Using `cython_modules` is entirely optional and not using it is sideeffect-free
 - Having `cythonize()` internally automatically compile *.pyx files when *.c files
   are provided is disabled by default and must be enabled using the `replace_extension`
   keyword argument (`cython_modules` enables that option).
<div><div dir="ltr">
<div>I have implemented some very simple setuptools integration that circumvents</div>
<div>the requirement to `import cython` in `setup.py` entirely, doing away with the</div>
<div>century-old problem of having to importing requirements during installation:</div>
<div><br></div>
<div>&lt;<a href="https://github.com/cython/cython/pull/477" target="_blank">https://github.com/cython/cython/pull/477</a>&gt;</div>
<div><br></div>
<div>This implementation is inspired by CFFI's setuptools integration:</div>
<div>&lt;<a href="https://cffi.readthedocs.org/en/latest/cdef.html#distutils-setuptools" target="_blank">https://cffi.readthedocs.org/en/latest/cdef.html#distutils-setuptools</a>&gt;.</div>
<div><br></div>
<div>They do not require you to `import cffi` but simply define an additional keyword</div>
<div>argument `cffi_modules` for `setup()`. This additional keyword argument _does not</div>
<div>raise errors when cffi is not installed yet_ but can be used once cffi is there</div>
<div>(as defined in `setup_requires`). Later setuptools will call cffi and have it do</div>
<div>whatever it wants with the contents of the argument.</div>
<div><br></div>
<div>Cython can do the exact same. Below you have the usual `setup.py`, with the</div>
<div>chicken and egg `import` problem and a few `Extensions` that need to be `cythonize`d:</div>
<div><br></div>
<div>&nbsp; &nbsp; import setuptools</div>
<div>&nbsp; &nbsp; from Cython.Build import cythonize &nbsp;# this will fail if cython is not present prior to running `pip install this`</div>
<div>&nbsp; &nbsp; from setuptools.extension import Extension</div>
<div><br></div>
<div>&nbsp; &nbsp; extensions = [</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; Extension("fib", ["fib.pyx"]),</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; Extension("fib2", ["fib2.pyx"]),</div>
<div>&nbsp; &nbsp; ]</div>
<div><br></div>
<div>&nbsp; &nbsp; setuptools.setup(</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; name='example',</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; version="1.0.0",</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; packages=setuptools.find_packages(),</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; setup_requires=['cython'],</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; install_requires=['cython'],</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; ext_modules=cythonize(extensions),</div>
<div>&nbsp; &nbsp; )</div>
<div><br></div>
<div>With the changes I am proposing the usual</div>
<div><br></div>
<div>&nbsp; &nbsp; ext_modules=cythonize(extensions),</div>
<div><br></div>
<div>can be replaced with</div>
<div><br></div>
<div>&nbsp; &nbsp; cython_modules=extensions,</div>
<div><br></div>
<div>removing the need for `from Cython.Build import cythonize` and solving the import problem:</div>
<div><br></div>
<div>&nbsp; &nbsp; import setuptools</div>
<div>&nbsp; &nbsp; from setuptools.extension import Extension</div>
<div><br></div>
<div>&nbsp; &nbsp; extensions = [</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; Extension("fib", ["fib.pyx"]),</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; Extension("fib2", ["fib2.pyx"]),</div>
<div>&nbsp; &nbsp; ]</div>
<div><br></div>
<div>&nbsp; &nbsp; setuptools.setup(</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; name='example',</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; version="1.0.0",</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; setup_requires=['cython'], &nbsp;# we must later require a Cython version that has this kind of integration</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; install_requires=['cython'],</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; cython_modules=extensions,</div>
<div>&nbsp; &nbsp; )</div>
<div><br></div>
<div>This allows a nicer automated installation of tools that depend on cython and</div>
<div>also allows end users to keep their `setup.py` cleaner and leaner.</div>
<div><br></div>
<div>Additionally, I have also extended `cythonize()` a bit and thus allow for definitions like:</div>
<div><br></div>
<div>&nbsp; &nbsp; import setuptools</div>
<div>&nbsp; &nbsp; from setuptools.extension import Extension</div>
<div><br></div>
<div>&nbsp; &nbsp; extensions = [</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; Extension("fib", ["fib.c"]),</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; Extension("fib2", ["fib2.c"]),</div>
<div>&nbsp; &nbsp; ]</div>
<div><br></div>
<div>&nbsp; &nbsp; setuptools.setup(</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; name='example',</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; version="1.0.0",</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; extras_require={'cython': ['cython']},</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; ext_modules=extensions,</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; cython_modules=extensions,</div>
<div>&nbsp; &nbsp; )</div>
<div><br></div>
<div>This will automatically compile pyx-&gt;c if cython is installed and a re-compilation is</div>
<div>needed. Otherwise it will merely compile c-&gt;so.</div>
<div><br></div>
<div>This allows devs to ship the generated C-code (as you suggest per your docs) but</div>
<div>takes the heuristic to find out whether to compile pyx-&gt;C-&gt;so or only C-&gt;so</div>
<div>completely out of `setup.py` and under control of cython.</div>
<div><br></div>
<div>The change is written to be backwards compatible:</div>
<div><br></div>
<div>&nbsp;- Using `cython_modules` is entirely optional and not using it is sideeffect-free</div>
<div>&nbsp;- Having `cythonize()` internally automatically compile *.pyx files when *.c files</div>
<div>&nbsp; &nbsp;are provided is disabled by default and must be enabled using the `replace_extension`</div>
<div>&nbsp; &nbsp;keyword argument (`cython_modules` enables that option).</div>
</div></div>
Neal Becker | 4 Feb 13:04 2016
Picon

[Cython] Cython-0.23.4 build failing on gcc6 for upcoming fedora 24

Anyone able to shed any light on this?   Here are the logs:

https://kojipkgs.fedoraproject.org//work/tasks/4202/12804202/build.log

Marvin Ritter | 1 Feb 23:56 2016
Picon

[Cython] Bug: Typed Memoryview and unicode string typ

Hi,

First off all thanks to all of you for this awesome tool, it is working great and I love it :)
Also the idea looks cool, I haven't been able to use typed memories views as they seem broken for my setup:

Cython 0.23.4
Python 3.4 (and 3.5)
Compiler directive: c_string_type=unicode (which should be the same as str in Python 3.x)

Steps:
cdef int* ptr = <int*>malloc(sizeof(int) * 10)
memView = <int[:n]>ptr 

Result:
Compiles, but will raise a TypeError('expected bytes, str found') at runtime

I attached a minimal example.
Note that if you change the c_string_type to 'bytes' it will work fine.

Best
Marvin
Attachment (cython-string_type.zip): application/zip, 1945 bytes
<div><div dir="ltr">
<span>Hi,</span><div><br></div>
<div>First off all thanks to all of you for this awesome tool, it is working great and I love it :)</div>
<div>Also the idea looks cool, I haven't been able to use typed memories views as they seem broken for my setup:<br>
</div>
<div><br></div>
<div>Cython 0.23.4</div>
<div>Python 3.4 (and 3.5)</div>
<div>Compiler directive: c_string_type=unicode (which should be the same as str in Python 3.x)</div>
<div><br></div>
<div>Steps:</div>
<div>cdef int* ptr = &lt;int*&gt;malloc(sizeof(int) * 10)</div>
<div>memView = &lt;int[:n]&gt;ptr&nbsp;<br>
</div>
<div><br></div>
<div>Result:</div>
<div>Compiles, but will raise a TypeError('expected bytes, str found') at runtime</div>
<div><br></div>
<div>I attached a minimal example.</div>
<div>Note that if you change the c_string_type to 'bytes' it will work fine.</div>
<div><br></div>
<div>Best</div>
<div>Marvin</div>
</div></div>
Marvin Ritter | 2 Feb 00:22 2016
Picon

[Cython] NumPy C API Status?

Hi,

I have to use the NumPy C API for my current project and it seems that Cython 0.23.4 it only shipping definitions for on all version of the API. But I need some of the newer methods.

What are your plans on updating? Is there anything that prevents you?
And if not, how complicated is it? Could a novice like me do it?

Thanks
Marvin
<div><div dir="ltr">Hi,<div><br></div>
<div>I have to use the NumPy C API for my current project and it seems that Cython 0.23.4 it only shipping definitions for on all version of the API. But I need some of the newer methods.</div>
<div><br></div>
<div>What are your plans on updating? Is there anything that prevents you?</div>
<div>And if not, how complicated is it? Could a novice like me do it?</div>
<div><br></div>
<div>Thanks</div>
<div>Marvin</div>
</div></div>
Kevin Thornton | 2 Feb 02:03 2016
Picon

[Cython] C++ structs, namespace, and auto-conversion of struct to dict

Hi,

I recently ran into an issue where I found that Cython will not generate code to auto-convert structs do dicts if the struct is declared in a C++ namespace.


While there are obvious workarounds, none of them are ideal.

Is there a road map for addressing this issue?  If not, what would it take to do so, and where in the code should one start looking?

Thanks, 

Kevin


<div><div dir="ltr">Hi,<div><br></div>
<div>I recently ran into an issue where I found that Cython will not generate code to auto-convert structs do dicts if the struct is declared in a C++ namespace.</div>
<div><br></div>
<div>The issue has also been discussed here:&nbsp;<a href="http://stackoverflow.com/questions/29603562/auto-conversion-of-structs-to-dicts-in-cython">http://stackoverflow.com/questions/29603562/auto-conversion-of-structs-to-dicts-in-cython</a>
</div>
<div><br></div>
<div>While there are obvious workarounds, none of them are ideal.</div>
<div><br></div>
<div>Is there a road map for addressing this issue?&nbsp; If not, what would it take to do so, and where in the code should one start looking?</div>
<div><br></div>
<div>Thanks,&nbsp;</div>
<div><br></div>
<div>Kevin<br><div><br></div>
<div><br></div>
</div>
</div></div>
Elizabeth Fischer | 29 Jan 20:02 2016

[Cython] Cython 0.23.4: Crash with Template Stuff

Cython is crashing on me when I compile.  See below.

-- Elizabeth

cython --version

Cython version 0.23.4


Error compiling Cython file:
------------------------------------------------------------
...
#        cdef int flags

def test_double_blitz(a):
cdef vector[int] v
cdef cblitz.Array[double,1] a_b
a_b = cibmisc.np_to_blitz[double,1](a, b'var', [-1])
                         ^
------------------------------------------------------------

/Users/rpfische/git/ibmisc/pylib/ibmisc.pyx:38:26: Compiler crash in AnalyseExpressionsTransform

ModuleNode.body = StatListNode(ibmisc.pyx:1:0)
StatListNode.stats[3] = StatListNode(ibmisc.pyx:35:0)
StatListNode.stats[0] = DefNode(ibmisc.pyx:35:0,
    modifiers = [...]/0,
    name = 'test_double_blitz',
    num_required_args = 1,
    py_wrapper_required = True,
    reqd_kw_flags_cname = '0',
    used = True)
File 'Nodes.py', line 430, in analyse_expressions: StatListNode(ibmisc.pyx:36:1)
File 'Nodes.py', line 4775, in analyse_expressions: SingleAssignmentNode(ibmisc.pyx:38:36)
File 'Nodes.py', line 4887, in analyse_types: SingleAssignmentNode(ibmisc.pyx:38:36)
File 'ExprNodes.py', line 4640, in analyse_types: SimpleCallNode(ibmisc.pyx:38:36,
    analysed = True,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 3109, in analyse_types: IndexNode(ibmisc.pyx:38:26,
    is_called = 1,
    is_subscript = True,
    result_is_used = True,
    use_managed_ref = True)
File 'ExprNodes.py', line 3419, in analyse_base_and_index_types: IndexNode(ibmisc.pyx:38:26,
    is_called = 1,
    is_subscript = True,
    result_is_used = True,
    use_managed_ref = True)

Compiler crash traceback from this point on:
  File "/Users/rpfische/macports/mpgompi-4.9.3/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Cython/Compiler/ExprNodes.py", line 3419, in analyse_base_and_index_types
    elif len(base_type.templates) != len(self.type_indices):
TypeError: object of type 'NoneType' has no len()
make[2]: *** [pylib/ibmisc.cpp] Error 1
make[2]: *** Deleting file `pylib/ibmisc.cpp'
make[1]: *** [pylib/CMakeFiles/ibmisc_so.dir/all] Error 2
make: *** [all] Error 2

<div><div dir="ltr">Cython is crashing on me when I compile.&nbsp; See below.<div><br></div>
<div>-- Elizabeth<br><div><br></div>
<div>

<p class="">cython --version</p>
<p class="">Cython version 0.23.4</p>
</div>
<div><br></div>
<div>
<div>Error compiling Cython file:</div>
<div>------------------------------------------------------------</div>
<div>...</div>
<div># &nbsp; &nbsp; &nbsp; &nbsp;cdef int flags</div>
<div><br></div>
<div>def test_double_blitz(a):</div>
<div>
<span class="">	</span>cdef vector[int] v</div>
<div>
<span class="">	</span>cdef cblitz.Array[double,1] a_b</div>
<div>
<span class="">	</span>a_b = cibmisc.np_to_blitz[double,1](a, b'var', [-1])</div>
<div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;^</div>
<div>------------------------------------------------------------</div>
<div><br></div>
<div>/Users/rpfische/git/ibmisc/pylib/ibmisc.pyx:38:26: Compiler crash in AnalyseExpressionsTransform</div>
<div><br></div>
<div>ModuleNode.body = StatListNode(ibmisc.pyx:1:0)</div>
<div>StatListNode.stats[3] = StatListNode(ibmisc.pyx:35:0)</div>
<div>StatListNode.stats[0] = DefNode(ibmisc.pyx:35:0,</div>
<div>&nbsp; &nbsp; modifiers = [...]/0,</div>
<div>&nbsp; &nbsp; name = 'test_double_blitz',</div>
<div>&nbsp; &nbsp; num_required_args = 1,</div>
<div>&nbsp; &nbsp; py_wrapper_required = True,</div>
<div>&nbsp; &nbsp; reqd_kw_flags_cname = '0',</div>
<div>&nbsp; &nbsp; used = True)</div>
<div>File 'Nodes.py', line 430, in analyse_expressions: StatListNode(ibmisc.pyx:36:1)</div>
<div>File 'Nodes.py', line 4775, in analyse_expressions: SingleAssignmentNode(ibmisc.pyx:38:36)</div>
<div>File 'Nodes.py', line 4887, in analyse_types: SingleAssignmentNode(ibmisc.pyx:38:36)</div>
<div>File 'ExprNodes.py', line 4640, in analyse_types: SimpleCallNode(ibmisc.pyx:38:36,</div>
<div>&nbsp; &nbsp; analysed = True,</div>
<div>&nbsp; &nbsp; result_is_used = True,</div>
<div>&nbsp; &nbsp; use_managed_ref = True)</div>
<div>File 'ExprNodes.py', line 3109, in analyse_types: IndexNode(ibmisc.pyx:38:26,</div>
<div>&nbsp; &nbsp; is_called = 1,</div>
<div>&nbsp; &nbsp; is_subscript = True,</div>
<div>&nbsp; &nbsp; result_is_used = True,</div>
<div>&nbsp; &nbsp; use_managed_ref = True)</div>
<div>File 'ExprNodes.py', line 3419, in analyse_base_and_index_types: IndexNode(ibmisc.pyx:38:26,</div>
<div>&nbsp; &nbsp; is_called = 1,</div>
<div>&nbsp; &nbsp; is_subscript = True,</div>
<div>&nbsp; &nbsp; result_is_used = True,</div>
<div>&nbsp; &nbsp; use_managed_ref = True)</div>
<div><br></div>
<div>Compiler crash traceback from this point on:</div>
<div>&nbsp; File "/Users/rpfische/macports/mpgompi-4.9.3/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Cython/Compiler/ExprNodes.py", line 3419, in analyse_base_and_index_types</div>
<div>&nbsp; &nbsp; elif len(base_type.templates) != len(self.type_indices):</div>
<div>TypeError: object of type 'NoneType' has no len()</div>
<div>make[2]: *** [pylib/ibmisc.cpp] Error 1</div>
<div>make[2]: *** Deleting file `pylib/ibmisc.cpp'</div>
<div>make[1]: *** [pylib/CMakeFiles/ibmisc_so.dir/all] Error 2</div>
<div>make: *** [all] Error 2</div>
</div>
<div><br></div>
</div>
</div></div>
Todd Rme | 20 Jan 19:36 2016
Picon

[Cython] cython 0.23.4 test failure, async_iter_pep492 (), asyncio_generators, test_async_def_future.py

On 13 January 2016 at 17:50, Orion Poplawski <orion at cora.nwra.com> wrote: > Dimitri John Ledkov <xnox at ...> writes: > >> >> Hello, >> >> Building 0.23.4 fails on ubuntu with python 3.5. >> >> Full build log is at: >> > https://launchpadlibrarian.net/228728126/buildlog_ubuntu-xenial > -amd64.cython_0.23.4-0ubuntu1_BUILDING.txt.gz >> >> The what i think relevant portion of the build log is as follows: >> >> End-to-end asyncio_generators ... FAIL >> runTest (__main__.CythonRunTestCase) >> compiling (c) and running attr ... attr () >> Doctest: attr ... /usr/bin/python3.5 test_async_def_future.py >> >> Traceback (most recent call last): >> File "test_async_def_future.py", line 16, in <module> >> runloop() >> File "test_async_def_future.py", line 14, in runloop >> assert events == expected, events >> AssertionError: ['setup', 'setval', None] >> >> Any help resolving this would help. Is this a broken python3.5? or >> cython? or both? > > We are seeing the same thing building cython 0.23.4 on Fedora rawhide for > python 3.5.1: > > https://kojipkgs.fedoraproject.org//work/tasks/5210/12535210/build.log

We are seeing the same failure on openSUSE, and I think I have worked out part of the reason why.

I have been playing around with the failing unit test using travis and my own github fork.  If I enable testing on python 3.5.1 or 3.5-dev the problem appears there too.  Cython doesn't currently test against these, which is why the issue wasn't discovered earlier.

I have discovered that the change in behaviour between 3.5 and 3.5.1 occurs in line 224 of  cython/tests/run/asyncio_generators.srctree 

In this line, in python 3.5 the "await fut" statement returned the result of "fut", 123.  In python 3.5.1+, however, it returns None instead.  The result value is successfully getting set, as seen by checking "fut.get_result()" after the await, but the await is not returning it.

You would think this would be an upstream Python problem, so I tried it in a standalone python interpreter.  However, the problem didn't happen, it worked as it did in 3.5.  So I went back to the test and changed the extension of "async_def_future" from "pyx" to "py" and the problem disappeared!

So somehow cython is doing something different with await than normal python, and this is causing await to return None in python 3.5.1+.  However, I do not know specifically what this is, or why it behaves differently in 3.5 and 3.5.1

<div>
<p dir="ltr">On 13 January 2016 at 17:50, Orion Poplawski &lt;<a href="https://mail.python.org/mailman/listinfo/cython-devel">orion at</a><a href="https://mail.python.org/mailman/listinfo/cython-devel"> cora.nwra.com</a>&gt; wrote: &gt; Dimitri John Ledkov &lt;<a href="https://mail.python.org/mailman/listinfo/cython-devel">xnox</a><a href="https://mail.python.org/mailman/listinfo/cython-devel"> at ...</a>&gt; writes: &gt; &gt;&gt; &gt;&gt; Hello, &gt;&gt; &gt;&gt; Building 0.23.4 fails on ubuntu with python 3.5. &gt;&gt; &gt;&gt; Full build log is at: &gt;&gt; &gt;<a href="https://launchpadlibrarian.net/228728126/buildlog_ubuntu-xenial"> https</a><a href="https://launchpadlibrarian.net/228728126/buildlog_ubuntu-xenial">://</a><a href="https://launchpadlibrarian.net/228728126/buildlog_ubuntu-xenial">launchpadlibrarian.net</a><a href="https://launchpadlibrarian.net/228728126/buildlog_ubuntu-xenial">/228728126/≤/a><a href="https://launchpadlibrarian.net/228728126/buildlog_ubuntu-xenial">buildlog</a><a href="https://launchpadlibrarian.net/228728126/buildlog_ubuntu-xenial">_</a><a href="https://launchpadlibrarian.net/228728126/buildlog_ubuntu-xenial">ubuntu-xenial</a> &gt; -amd64.cython_0.23.4-0ubuntu1_BUILDING.txt.gz &gt;&gt; &gt;&gt; The what i think relevant portion of the build log is as follows: &gt;&gt; &gt;&gt; End-to-end asyncio_generators ... FAIL &gt;&gt; runTest (__main__.CythonRunTestCase) &gt;&gt; compiling (c) and running attr ... attr () &gt;&gt; Doctest: attr ... /usr/bin/python3.5 test_async_def_future.py &gt;&gt; &gt;&gt; Traceback (most recent call last): &gt;&gt; File "test_async_def_future.py", line 16, in &lt;module&gt; &gt;&gt; runloop() &gt;&gt; File "test_async_def_future.py", line 14, in runloop &gt;&gt; assert events == expected, events &gt;&gt; AssertionError: ['setup', 'setval', None] &gt;&gt; &gt;&gt; Any help resolving this would help. Is this a broken python3.5? or &gt;&gt; cython? or both? &gt; &gt; We are seeing the same thing building cython 0.23.4 on Fedora rawhide for &gt; python 3.5.1: &gt; &gt;<a href="https://kojipkgs.fedoraproject.org//work/tasks/5210/12535210/build.log"> https://</a><a href="https://kojipkgs.fedoraproject.org//work/tasks/5210/12535210/build.log">kojipkgs.fedoraproject.org</a><a href="https://kojipkgs.fedoraproject.org//work/tasks/5210/12535210/build.log">//work/tasks/5210/12535210/≤/a><a href="https://kojipkgs.fedoraproject.org//work/tasks/5210/12535210/build.log">build.log</a></p>
<p dir="ltr">We are seeing the same failure on openSUSE, and I think I have worked out part of the reason why.</p>
<p dir="ltr">I have been playing around with the failing unit test using travis and my own github fork.&nbsp; If I enable testing on python 3.5.1 or 3.5-dev the problem appears there too.&nbsp; Cython doesn't currently test against these, which is why the issue wasn't discovered earlier.</p>
<p dir="ltr">I have discovered that the change in behaviour between 3.5 and 3.5.1 occurs in line 224 of&nbsp; cython/tests/run/asyncio_generators.srctree&nbsp;</p>
<p dir="ltr">In this line, in python 3.5 the "await fut" statement returned the result of "fut", 123.&nbsp; In python 3.5.1+, however, it returns None instead.&nbsp; The result value is successfully getting set, as seen by checking "fut.get_result()" after the await, but the await is not returning it.</p>
<p dir="ltr">You would think this would be an upstream Python problem, so I tried it in a standalone python interpreter.&nbsp; However, the problem didn't happen, it worked as it did in 3.5.&nbsp; So I went back to the test and changed the extension of "async_def_future" from "pyx" to "py" and the problem disappeared!</p>
<p dir="ltr">So somehow cython is doing something different with await than normal python, and this is causing await to return None in python 3.5.1+.&nbsp; However, I do not know specifically what this is, or why it behaves differently in 3.5 and 3.5.1</p>
</div>
Serhiy Storchaka | 11 Jan 21:32 2016
Picon

[Cython] Cython creates non-pickleable classes

Sorry, I'm not very familiar with Cython, but it looks to me that Cython 
creates extension classes that can't be correctly copied and unpickled. 
Since default __reduce_ex__ implementation knows nothing about Cython 
class structure, it's result doesn't contain the state of the object. 
The object could be pickled, but unpickled object doesn't contain 
original values of fields, it is just non-initialized. In recent CPython 
releases was added new restrictions for pickling objects that can't be 
correctly unpickled. This had added incompatibility with Cython classes. 
Building some packages now is failed, because Cython makes a deep copy 
that actually doesn't work.

See for original changes and discussion about the regression: 
http://bugs.python.org/issue22995 .

Besides the fact of Cython regression, it looks to me, that Cython also 
needs to be fixed. It can be done either by generating default 
pickle-related methods (__getnewargs__/__getnewargs_ex__, or 
__getstate__ + __setstate__, or __reduce__/__reduce_ex__) for all Cython 
classes, or implementing pickling or deepcopying only for NameAssignment 
and like.

B. Clausius | 26 Dec 11:59 2015
Picon
Picon
Gravatar

[Cython] strange error message

Hi cython devs,

a strange error message, showing cython internals:

$ cat gldraw.pxd
ctypedef float vec4[4]
ctypedef vec4 mat4[4]

$ cat glarea.pyx
cimport gldraw

cdef struct Data:
    gldraw.mat4 matrix

cdef Data data

cdef void sync():
    data.changed = True  # <- this wrong line produces the error

$ cython glarea.pyx

Error compiling Cython file:
------------------------------------------------------------
...
    void PyTuple_SET_ITEM(object  p, Py_ssize_t pos, object o)
    void PyList_SET_ITEM(object  p, Py_ssize_t pos, object o)

 <at> cname("__Pyx_carray_to_py_vec4")
cdef inline list __Pyx_carray_to_py_vec4(vec4 *v, Py_ssize_t length):
                                        ^
------------------------------------------------------------

carray.to_py:112:41: 'vec4' is not a type identifier

Error compiling Cython file:
------------------------------------------------------------
...
        PyList_SET_ITEM(l, i, value)
    return l

 <at> cname("__Pyx_carray_to_tuple_vec4")
cdef inline tuple __Pyx_carray_to_tuple_vec4(vec4 *v, Py_ssize_t length):
                                            ^
------------------------------------------------------------

carray.to_py:124:45: 'vec4' is not a type identifier

$ cython --version
Cython version 0.23.3

Best regards,

B.C.

Dimitri John Ledkov | 16 Dec 06:18 2015
Gravatar

[Cython] cython 0.23.4 test failure, async_iter_pep492 (), asyncio_generators, test_async_def_future.py

Hello,

Building 0.23.4 fails on ubuntu with python 3.5.

Full build log is at:
https://launchpadlibrarian.net/228728126/buildlog_ubuntu-xenial-amd64.cython_0.23.4-0ubuntu1_BUILDING.txt.gz

First with 2.7, run.asyncio_generators are skipped. Then there is a
successful run of asyncio_generators (?!) and then a failed one (?!)

The what i think relevant portion of the build log is as follows:

async_iter_pep492 ()
Doctest: async_iter_pep492 ... ok
test_aiter_raises (async_iter_pep492)
Doctest: async_iter_pep492.test_aiter_raises ... ok
test_broken_anext (async_iter_pep492)
Doctest: async_iter_pep492.test_broken_anext ... ok
test_for_1 (async_iter_pep492)
Doctest: async_iter_pep492.test_for_1 ... ok
test_for_2 (async_iter_pep492)
Doctest: async_iter_pep492.test_for_2 ... ok
test_for_3 (async_iter_pep492)
Doctest: async_iter_pep492.test_for_3 ... ok
test_with_for (async_iter_pep492)
Doctest: async_iter_pep492.test_with_for ... ok
runTest (__main__.EndToEndTest)
End-to-end asyncio_generators ... FAIL
runTest (__main__.CythonRunTestCase)
compiling (c) and running attr ... attr ()
Doctest: attr ... /usr/bin/python3.5 test_async_def_future.py

Traceback (most recent call last):
  File "test_async_def_future.py", line 16, in <module>
    runloop()
  File "test_async_def_future.py", line 14, in runloop
    assert events == expected, events
AssertionError: ['setup', 'setval', None]

ok

Any help resolving this would help. Is this a broken python3.5? or
cython? or both?

--

-- 
Regards,

Dimitri.

Gmane