Lisandro Dalcin | 2 Oct 12:11 2015
Picon

[Cython] Python 3 wheels including bytecode files.

After getting AppVeyor to work, I noticed that Python 3 wheels do
include __pycache__ directory. This also happens with in OS X as well
as Linux.

I think the root of the issue resides in setup.py, in the build_ext
command class workaround that seems to be a legacy thing from the time
Cython required 2to3. BTW, in setup.py, compile_cython_modules()
follows quite a different path in Python 2 versus 3.

Unless some of you believe we should keep things that way, I'll try to
fix setup.py to be as much independent as possible of the major Python
version.

--

-- 
Lisandro Dalcin
============
Research Scientist
Computer, Electrical and Mathematical Sciences & Engineering (CEMSE)
Numerical Porous Media Center (NumPor)
King Abdullah University of Science and Technology (KAUST)
http://numpor.kaust.edu.sa/

4700 King Abdullah University of Science and Technology
al-Khawarizmi Bldg (Bldg 1), Office # 4332
Thuwal 23955-6900, Kingdom of Saudi Arabia
http://www.kaust.edu.sa

Office Phone: +966 12 808-0459
Hogan, Christopher | 30 Sep 00:03 2015
Picon

[Cython] Bug: llabs not defined in vs2008

Hello,

 

I was trying to compile numpy v1.9.2 with Visual Studio 2008 and Cython 0.23.1.  This worked fine with Cython 0.22.x, but in 0.23.1 there are some changes in Cython/Utility/TypeConversion.c that seem to cause this error:

 

mtrand.obj : error LNK2019: unresolved external symbol llabs referenced in function __pyx_pf_6mtrand_11RandomState_24 choice

 

llabs is not available in vs2008, but it ends up taking that “#elif” branch in TypeConversion.c and defining __Pyx_sst_abs to be llabs.  I was able to fix this by swapping the order of the “#elif defined (_MSC_VER) …”  block with the “#elif defined (__STDC_VERSION__) … “ block.  That probably won’t fix the problem in all cases though.

 

Thanks,

 

Chris Hogan

Scripting Tools Engineer

Scripting, Analyzers and Tools

Intel Corporation

 

<div>
<div class="WordSection1">
<p class="MsoNormal">Hello,<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">I was trying to compile numpy v1.9.2 with Visual Studio 2008 and Cython 0.23.1.&nbsp; This worked fine with Cython 0.22.x, but in 0.23.1 there are some changes in Cython/Utility/TypeConversion.c that seem to cause this error:<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal"><span>mtrand.obj : error LNK2019: unresolved external symbol llabs referenced in function __pyx_pf_6mtrand_11RandomState_24 choice<p></p></span></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">llabs is not available in vs2008, but it ends up taking that &ldquo;#elif&rdquo; branch in TypeConversion.c and defining __Pyx_sst_abs to be llabs.&nbsp; I was able to fix this by swapping the order of the &ldquo;#elif defined (_MSC_VER) &hellip;&rdquo; &nbsp;block with the &ldquo;#elif
 defined (__STDC_VERSION__) &hellip; &ldquo; block.&nbsp; That probably won&rsquo;t fix the problem in all cases though.<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">Thanks,<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
<p class="MsoNormal">Chris Hogan<p></p></p>
<p class="MsoNormal">Scripting Tools Engineer<p></p></p>
<p class="MsoNormal">Scripting, Analyzers and Tools<p></p></p>
<p class="MsoNormal">Intel Corporation<p></p></p>
<p class="MsoNormal"><p>&nbsp;</p></p>
</div>
</div>
Mirth Hickford | 27 Sep 21:30 2015
Picon

[Cython] Publish Windows wheels to PyPI

Hi Cython. Please consider publishing wheels for Windows to PyPI [1]. Wheels [2] are a package format that installs more reliably than source distributions, especially on Windows.

Meanwhile a workaround is to download wheels from Christoph Gohlke's site [3] a godsend.

<div><div dir="ltr">Hi Cython. Please consider publishing wheels for Windows to PyPI [1]. Wheels [2] are a package format that installs more reliably than source distributions, especially on Windows.<div><br></div>
<div>Meanwhile a workaround is to download wheels from&nbsp;Christoph Gohlke's site [3] a godsend.</div>
<div>
<br><div>
<div>-M</div>
<div><br></div>
<div>[1]&nbsp;<a href="https://pypi.python.org/pypi/Cython/">https://pypi.python.org/pypi/Cython/</a>
</div>
<div>[2]&nbsp;<a href="http://pythonwheels.com/">http://pythonwheels.com/</a>
</div>
<div>[3] <a href="http://www.lfd.uci.edu/~gohlke/pythonlibs/#cython">http://www.lfd.uci.edu/~gohlke/pythonlibs/#cython</a><br>
</div>
</div>
</div>
<div><br></div>
</div></div>
Yaroslav Halchenko | 21 Sep 16:26 2015
Gravatar

[Cython] ct_DEF.__test__.large_nums doctest failing in 32bit

was reported in Debian against 0.23.2 so tried blindly freshier snapshot
from 0.23.x branch and that one is no go too:

======================================================================
FAIL: large_nums (line 95) (ct_DEF.__test__)
Doctest: ct_DEF.__test__.large_nums (line 95)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/doctest.py", line 2226, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for ct_DEF.__test__.large_nums (line 95)
  File "/tmp/buildd/cython-0.23.2+git12-g2c9d175/build/work-dir/run/cpp/ct_DEF/ct_DEF.so",
line unknown line number, in large_nums (line 95)

----------------------------------------------------------------------
File "/tmp/buildd/cython-0.23.2+git12-g2c9d175/build/work-dir/run/cpp/ct_DEF/ct_DEF.so",
line ?, in ct_DEF.__test__.large_nums (line 95)
Failed example:
    print_large_number(n64)
Expected:
    -4294967295
Got:
    1

--

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        
Nathaniel Smith | 14 Sep 14:33 2015

[Cython] Is it easy to do an "AST grep" of Cython code?

I have a few hundred files worth of Cython code, and I'd like to find
all instances where a variable/expression has a certain cdef class
type. (More specifically, I'd like to find all accesses to the cdef
fields of a particular cdef class, but even just finding all 'cdef
MYTYPE x" and "<MYTYPE>x" statements would probably get me pretty
close.) Is there any easy way to do this?

(The files are "every cython file on github or searchcode.com that
mentions the word "ufunc"", and I'm trying to find code that does
direct field access to the internals of the PyUFuncObject struct.)

--

-- 
Nathaniel J. Smith -- http://vorpus.org
Jelle Zijlstra | 13 Sep 21:00 2015
Picon

[Cython] CPython incompatibility in string literal concatenation

While implementing PEP 498 I found a minor incompatibility between
CPython and Cython: CPython lets you concatenate u-prefixed and
non-prefixed string literals, while Cython throws an error.

jelle <at> devjelle:~/cython$ cat concat.py
print(u'foo' 'bar')

jelle <at> devjelle:~/cython$ python concat.py
foobar
jelle <at> devjelle:~/cython$ cython concat.py

Error compiling Cython file:
------------------------------------------------------------
...
print(u'foo' 'bar')
            ^
------------------------------------------------------------

concat.py:1:13: Cannot mix string literals of different types,
expected u'', got ''

I tried both CPython 2.7 and 3.5. The documentation at
https://docs.python.org/2/reference/lexical_analysis.html#string-literal-concatenation
is ambiguous as to whether this behavior is intentional.

As part of implementing PEP 498, I can make Cython's behavior match
CPython's, unless there's a good reason to keep the incompatibility.
Fabrizio Messina | 7 Sep 12:23 2015
Picon

Re: [Cython] call for contribution: PEP 498 - Literal String Interpolation

Hello, I would like to give a look on that on my spare time.

On 5 Sep 2015 20:32, "Stefan Behnel" <stefan_ml-KjMAwuNBv5izQB+pC5nmwQ@public.gmane.org> wrote:
Hi!

It looks like PEP 498 (f-strings) is going to be accepted soon. Anyone
interested in implementing it for Cython?

https://www.python.org/dev/peps/pep-0498/

It's certainly a bit of work, but it's also very much Python-level
functionality and most of it can be handled in the parser, so I would count
it as an entry-level project for working on Cython. There is already an
initial implementation for CPython (in C) which should provide an answer to
the tricky details of the functionality itself, as well as tests for it
(eventually).

https://bugs.python.org/issue24965

So, if this feature, and if extending the Cython language and working on a
real compiler sounds tempting to you, please speak up on the cython-devel
mailing list.

Stefan
_______________________________________________
cython-devel mailing list
cython-devel-+ZN9ApsXKcEdnm+yROfE0A@public.gmane.org
https://mail.python.org/mailman/listinfo/cython-devel
<div>
<p dir="ltr">Hello, I would like to give a look on that on my spare time.</p>
<div class="gmail_quote">On 5 Sep 2015 20:32, "Stefan Behnel" &lt;<a href="mailto:stefan_ml@...">stefan_ml@...</a>&gt; wrote:<br type="attribution"><blockquote class="quote">Hi!<br><br>
It looks like PEP 498 (f-strings) is going to be accepted soon. Anyone<br>
interested in implementing it for Cython?<br><br><a href="https://www.python.org/dev/peps/pep-0498/" rel="noreferrer" target="_blank">https://www.python.org/dev/peps/pep-0498/</a><br><br>
It's certainly a bit of work, but it's also very much Python-level<br>
functionality and most of it can be handled in the parser, so I would count<br>
it as an entry-level project for working on Cython. There is already an<br>
initial implementation for CPython (in C) which should provide an answer to<br>
the tricky details of the functionality itself, as well as tests for it<br>
(eventually).<br><br><a href="https://bugs.python.org/issue24965" rel="noreferrer" target="_blank">https://bugs.python.org/issue24965</a><br><br>
So, if this feature, and if extending the Cython language and working on a<br>
real compiler sounds tempting to you, please speak up on the cython-devel<br>
mailing list.<br><br>
Stefan<br>
_______________________________________________<br>
cython-devel mailing list<br><a href="mailto:cython-devel@...">cython-devel@...</a><br><a href="https://mail.python.org/mailman/listinfo/cython-devel" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/cython-devel</a><br>
</blockquote>
</div>
</div>
Jeroen Demeyer | 5 Sep 12:17 2015
Picon

[Cython] All DEF constants are now unsigned?

Hello Cython-devel,

I noticed that now unsigned constants like

DEF foo = 1

become 1U in the C source code.

This change (i.e. commit ea6414cd293ba2134c3df5b37b158d8e3db79083) broke 
Sage. I haven't managed yet to actually isolate why this change broke 
something.

Was this change of 1 to 1U intentional? Why was it done?

Jeroen.
Jeroen Demeyer | 5 Sep 11:01 2015
Picon

[Cython] Bootstrapping broken in master

Checking out Cython from master (the 0.23.x branch works fine) and 
running "python setup.py install" gives me

Unable to find pgen, not compiling formal grammar.
Compiling module Cython.Compiler.Scanning ...

ERROR: 'module' object has no attribute 'CCodeConfig'

Extension module compilation failed, looks like Cython cannot run
properly on this system.  To work around this, pass the option
"--no-cython-compile".  This will install a pure Python version of
Cython without compiling its own sources.

Traceback (most recent call last):
   File "setup.py", line 289, in <module>
     compile_cython_modules(cython_profile, cython_compile_more, 
cython_with_refnanny)
   File "setup.py", line 233, in compile_cython_modules
     result = compile(pyx_source_file)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/Main.py", line 
677, in compile
     return compile_single(source, options, full_module_name)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/Main.py", line 
630, in compile_single
     return run_pipeline(source, options, full_module_name)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/Main.py", line 
487, in run_pipeline
     err, enddata = Pipeline.run_pipeline(pipeline, source)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/Pipeline.py", 
line 334, in run_pipeline
     data = phase(data)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/Pipeline.py", 
line 53, in generate_pyx_code_stage
     module_node.process_implementation(options, result)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/ModuleNode.py", line 
118, in process_implementation
     self.generate_c_code(env, options, result)
   File 
"/usr/local/src/sage-git/local/src/cython/Cython/Compiler/ModuleNode.py", line 
317, in generate_c_code
     c_code_config = Code.CCodeConfig(
AttributeError: 'module' object has no attribute 'CCodeConfig'
Nathan Goldbaum | 3 Sep 21:29 2015
Picon

[Cython] cygdb: no debug files were found

Hi all,

I'm following the directions in the link below to debug some cython code using cygdb:


Unfortunately, when I try to actually debug my program using cygdb, I get a "no debug files were found" error.

It looks like the error is getting raised here: https://github.com/cython/cython/blob/master/Cython/Debugger/Cygdb.py#L33

So for some reason the "cython_debug" folder isn't being generated at build time. I tried to follow the directions closely, you can see the notes I put together during the installation process here: https://bpaste.net/show/6b11f815d449

Unfortunately the library I'm trying to debug (yt: bitbucket.org/yt_analysis/yt) uses a somewhat nonstandard build setup based on numpy distutils, and I suspect that is breaking some of the assumptions that went into the cygdb instructions in the cython docs.

Does anyone have any idea what went wrong here?

For now I'm going to fall back to regular gdb, and debug the autogenerated cython sources.

Thanks for your help,

Nathan Goldbaum
<div><div dir="ltr">Hi all,<div><br></div>
<div>I'm following the directions in the link below to debug some cython code using cygdb:</div>
<div><br></div>
<div>
<a href="http://docs.cython.org/src/userguide/debugging.html">http://docs.cython.org/src/userguide/debugging.html</a><br>
</div>
<div><br></div>
<div>Unfortunately, when I try to actually debug my program using cygdb, I get a "no debug files were found" error.</div>
<div><br></div>
<div>It looks like the error is getting raised here:&nbsp;<a href="https://github.com/cython/cython/blob/master/Cython/Debugger/Cygdb.py#L33">https://github.com/cython/cython/blob/master/Cython/Debugger/Cygdb.py#L33</a>
</div>
<div><br></div>
<div>So for some reason the "cython_debug" folder isn't being generated at build time. I tried to follow the directions closely, you can see the notes I put together during the installation process here:&nbsp;<a href="https://bpaste.net/show/6b11f815d449">https://bpaste.net/show/6b11f815d449</a>
</div>
<div><br></div>
<div>Unfortunately the library I'm trying to debug (yt: <a href="http://bitbucket.org/yt_analysis/yt">bitbucket.org/yt_analysis/yt</a>) uses a somewhat nonstandard build setup based on numpy distutils, and I suspect that is breaking some of the assumptions that went into the cygdb instructions in the cython docs.</div>
<div><br></div>
<div>Does anyone have any idea what went wrong here?</div>
<div><br></div>
<div>For now I'm going to fall back to regular gdb, and debug the autogenerated cython sources.</div>
<div><br></div>
<div>Thanks for your help,</div>
<div><br></div>
<div>Nathan Goldbaum</div>
</div></div>
Jakub Wilk | 2 Sep 08:16 2015
Picon

[Cython] array->memoryview typecast causes compiler crash in AnalyseExpressionsTransform

In https://bugs.debian.org/793129 Dan Greene reported that trying to 
cast a pointer to a memoryview object causes a compiler crash:

$ python -c 'import cython; cython.inline("<int[:12]>NULL")'
Compiling /home/jwilk/.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx
because it changed.
[1/1] Cythonizing /home/jwilk/.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx

Error compiling Cython file:
------------------------------------------------------------
...

def __invoke():
    <int[:12]>NULL
   ^
------------------------------------------------------------

.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4: Compiler crash in AnalyseExpressionsTransform

ModuleNode.body = StatListNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:4:0)
StatListNode.stats[0] = DefNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:4:0,
    modifiers = [...]/0,
    name = u'__invoke',
    py_wrapper_required = True,
    reqd_kw_flags_cname = '0',
    used = True)
File 'Nodes.py', line 430, in analyse_expressions: StatListNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4,
    is_terminator = True)
File 'Nodes.py', line 4741, in analyse_expressions: ExprStatNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4)
File 'ExprNodes.py', line 460, in analyse_expressions: CythonArrayNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4,
    is_temp = True,
    mode = 'c',
    use_managed_ref = True)
File 'ExprNodes.py', line 9567, in analyse_types: CythonArrayNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4,
    is_temp = True,
    mode = 'c',
    use_managed_ref = True)
File 'ExprNodes.py', line 9582, in get_cython_array_type: CythonArrayNode(_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx:5:4,
    is_temp = True,
    mode = 'c',
    use_managed_ref = True)

Compiler crash traceback from this point on:
  File "/home/jwilk/.local/lib/python2.7/site-packages/Cython/Compiler/ExprNodes.py", line
9582, in get_cython_array_type
    return env.global_scope().context.cython_scope.viewscope.lookup("array").type
AttributeError: 'CythonScope' object has no attribute 'viewscope'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/jwilk/.local/lib/python2.7/site-packages/Cython/Shadow.py", line 114, in inline
    return cython_inline(f, *args, **kwds)
  File "/home/jwilk/.local/lib/python2.7/site-packages/Cython/Build/Inline.py", line 212, in cython_inline
    build_extension.extensions = cythonize([extension], include_path=cython_include_dirs, quiet=quiet)
  File "/home/jwilk/.local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 877,
in cythonize
    cythonize_one(*args)
  File "/home/jwilk/.local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 997,
in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: /home/jwilk/.cython/inline/_cython_inline_ad8e8c5018441f8a5417cab08ec65d13.pyx

--

-- 
Jakub Wilk

Gmane