[Cython] patch for bad buffer creation

Hi,

I tracked down an issue that was giving me segfaults with a poor 
__getbuffer__ implementation and modified the compiler to catch this case.

I couldn't figure out how to create an account on Trac, so instead of 
submitting a patch there, I'm just sending this email. Included is a 
testcase added to tests/run/bufaccess.pyx. This is a patch against 
cython-devel and I release this under the PSF license (like the rest of 
Cython).

I had to use "hg diff -b" to create this patch while ignoring whitespace 
because I have emacs configured to strip trailing whitespace, which 
caused lots of false positives on the code. So, hopefully, this patch 
will still apply cleanly.

(The patch also includes a tiny fix to the ErrorBuffer implementation in 
tests/run/bufaccess.pyx.)

-Andrew

Hi,

I tracked down an issue that was giving me segfaults with a poor 
__getbuffer__ implementation and modified the compiler to catch this case.

I couldn't figure out how to create an account on Trac, so instead of 
submitting a patch there, I'm just sending this email. Included is a 

[Cython] python version for conditional compilation?

Hi, looking at the Cython Sphinx documentation, I don't see the Python 
version as a variable I can use for conditional compilation. This would 
be handy in the case of newly added features in the Python-C API, for 
example. Would this be feasible to add (alongside UNAME_SYSNAME and 
friends)?

(The location of the docs I was looking at: 
http://www.mudskipper.ca/cython-doc/docs/language_basics.html#compile-time-definitions 
)

[Cython] Repository usage

There's been some talk already but now seems the time to make votes etc.

I propose that bugfixes that a) are relatively simple, and b) doesn't rely
on or interfere features implemented since the last release, are coded
against and pushed to "cython" and then merged into "cython-devel".

"cython" should still be considered relatively stable, so every commit
there is like a micro-patch-release of Cython.

This should take some pressure off cython-devel (although keeping it as
stable as possible is still good). Cython 0.9.8.2 may then come entirely
from "cython" if it is primarily a bugfix release, or from "cython-devel"
if it stabilizes.

One should think about some of the things that may be done to cython-devel
soon, especially import Greg's result_code-calculation-in-generation-phase
changes, and the other refactorings related to/made possible by that.
These are mildly dangerous (at least they lead to changes "everywhere")
but also needs to be in the main stream of development.

An alternative would perhaps be renaming "dagss" to "experimental" (but
IMO all new features as well as new framework changes so go into it then).

Thoughts?

Dag Sverre

[Cython] Cython 0.9.8.1 Released

Sources are up in their usual places. The largest addition was Dag  
Seljebotn's buffer support which is amazing. Kudos to him for all his  
hard work (and to Enthought/Google for funding him this summer). Part  
of this effort involved moving to a pipeline/transformation based  
compilation system, which has allowed for many other nice features  
(e.g. long if statements get translated into c switch statements if  
possible).

Another big inclusion is Paul Prescod's pyximport. Now one can simply  
start up Python and type

     import pyximport; pyximport.install()
     import foo

which will compile foo.pyx for you. We would like to be able to  
(optionally) specify all options via directives in the files  
themselves, which would make this even more useful.

There are numerous fixes and improvements by Stefan Behnel, Dag  
Seljebotn, and Robert Bradshaw that are not listed here. Among them  
are better support for Py3, unicode, C++, literals, testing  
framework, etc. We also merged patches by Jim Kleckner, Hoyt Koepke,  
Marcus Bitzl, Kirill Smelkov, and Carl Witty.

Also, a (far from exhaustive) summary of the tickets closed, see:

http://trac.cython.org/cython_trac/query?status=closed&milestone=0.9.8.1

- Robert

[Cython] Any thoughts on a DEFINED keyword?

Does anyone else have any thoughts on this: http://trac.cython.org/ 
cython_trac/ticket/27

- Robert

[Cython] print statement under Py3k

Hi,

Dag Sverre Seljebotn wrote:
> There's a problem if you do an empty print (newline
> print) right after a non-newline print. The attached test case works in
> Py2 but fails in Py3 (the line with "2 2 2" is output as "2 2 2 " instead).

> for i in range(3):
>        print 2,
> print

Right, that makes sense. The Py3 print() function (that we map the print
statement to when running under Py3) has an explicit 'end' keyword that keeps
it from having to do any "softspace" special casing. So this is a difference
that is hard to emulate in Py3, which (as opposed to Py2) does not expose the
softspace hint at all.

http://www.python.org/dev/peps/pep-3105/

Open poll:

a) would it be acceptable if this difference persisted, i.e. if Cython code
   that uses something like the example above would behave different in Py3
b) should Cython try to special case this in Py3 to emulate the Py2 behaviour
   in common cases
c) should Cython stick to the Py3 behaviour also in Py2 (thus breaking the
   complete semantics of the print statement)

?

[Cython] Request for review: runtests.py and NumPy issues

As we're so close to a release I'm going to post a patch here instead of 
commiting it straight away.

The patch fixes an issue with runtests.py with a lot of releases of 
NumPy. The latest svn version doesn't have the problem but the one 
shipped with the latest Ubuntu does, so I expect a lot of users may 
encounter it. The problem is that NumPy doesn't like 
distutils.unixccompiler being imported already.

One could think that the test environment should be further "sandboxed" 
and that all modules should be removed, but I don't know anything about 
sys.modules really so I'll leave that to somebody else.

Something else: The "cimport" triggers a module import so the numpy 
testcase fails on systems where numpy is not installed. This isn't all 
that serious I feel (a failing numpy testcase will communicate the right 
thing to users not having numpy installed) so I think we should wait 
until after the release (and then add some kind of "_USES" special 
testcase variable).

--

-- 
Dag Sverre

As we're so close to a release I'm going to post a patch here instead of 
commiting it straight away.

The patch fixes an issue with runtests.py with a lot of releases of 
NumPy. The latest svn version doesn't have the problem but the one 

[Cython] Another annotator bug

Hello,

In continuing to play around with the annotation function, I've found
another bug (and I think I have it patched as well).  For me, it shows
up when there is an error in my code.  It doesn't seem to matter what
that error is, as I've fixed several and still get the same backtrace.
 This is in r1071

After running cython -a transforms.pyx, it prints out an error message
and then raises

Traceback (most recent call last):
  File "/home/hoytak/sysroot/bin/cython", line 8, in <module>
    main(command_line = 1)
  File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",
line 698, in main
    result = compile(sources, options)
  File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",
line 675, in compile
    return compile_multiple(source, options)
  File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",
line 645, in compile_multiple
    result = run_pipeline(source, options)
  File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",
line 507, in run_pipeline
    err, enddata = context.run_pipeline(pipeline, source)
  File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",
line 169, in run_pipeline
    data = phase(data)
  File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",

[Cython] Boundchecking question

I've been playing with the (very cool!) buffer stuff Dag did getting  
ready for the SciPy conference. I've seen a 20% increase in speed not  
adjusting for negative indices, but making sure the indices are  
unsigned is a pain (especially if there is arithmetic involved). One  
proposal that has been brought up is to make positive integer  
constants unsigned, which could lead to complications elsewhere and  
would still require care. Another proposal that I'd like to throw out  
there is to have negative index realigning part of bounds checking,  
i.e. negative indices are not checked for if bounds checking is off.

Thoughts?

- Robert

[Cython] Fwd: some errors annotating code

Hello,

I've been playing around with the cython code annotation feature, and
I think I've found a couple easily fixable bugs in the html code
annotation feature.  I'm using the latest version from the mercurial
repository (1053:e5976a417edf).

First bug.  I get the following backtrace:

hoytak <at> ubuntu804desktop:~/workspace/clsynth-cython/clsynth$ cython -a
transforms.pyx
Traceback (most recent call last):
 File "/home/hoytak/sysroot/bin/cython", line 8, in <module>
   main(command_line = 1)
 File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",
line 694, in main
   result = compile(sources, options)
 File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",
line 671, in compile
   return compile_multiple(source, options)
 File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",
line 641, in compile_multiple
   result = run_pipeline(source, options)
 File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",
line 503, in run_pipeline
   err, enddata = context.run_pipeline(pipeline, source)
 File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",
line 169, in run_pipeline
   data = phase(data)
 File "/home/hoytak/sysroot/lib/python2.5/site-packages/Cython/Compiler/Main.py",

[Cython] [Fwd: [Numpy-discussion] VS 2003 problems with cython-generated code]

I remember this being discussed here:

http://codespeak.net/pipermail/cython-dev/2008-May/000844.html

Was this resolved back then?

-------- Original Message --------
Subject: [Numpy-discussion] VS 2003 problems with cython-generated code
Date: Fri, 15 Aug 2008 11:49:28 -0500
From: David Cournapeau <cournape@...>
Reply-To: Discussion of Numerical Python <numpy-discussion@...>
To: Discussion of Numerical Python <numpy-discussion@...>

Hi,

I noticed this morning that numpy 1.2 is not buildable with VS 2003
(the one you have to use for official python releases for at least
python 2.5, and maybe 2.4). When we generate C code, both with
internal code (numpy/core/code_generator) and with external tools
(cython/pyrex for mtrand), the string literals generated for
docstrings are too long for visual studio. We have to break them (e.g.
"foo bar" becomes "foo""bar"), but doing so with cython-generated code
is only doable by changing cython itself.

So I did patch cython to break those, and regenerate the mtrand.c.
This is done in the vs_longstring branch. Is it ok to put this for 1.2
? Without it, I don't see a way to have numpy 1.2 buildable with VS.

cheers,