Re: Cython 0.9.6.10

On Dec 31, 2007 6:04 PM, Robert Bradshaw
<robertwb@...> wrote:
> I did
>
> Robert-Bradshaws-Laptop:~/sage/cython/cython-0.9.6.10/Cython robert$
> python ../cython.py
> Cython (http://cython.org) is a compiler for code written in the
> Cython language.  Cython is based on Pyrex by Greg Ewing.
>
> Usage: cython [options] sourcefile.pyx ...
>
> Options:
>    -v, --version                  Display version number of cython
> compiler
>    -l, --create-listing           Write error messages to a listing file
>    -I, --include-dir <directory>  Search for include files in named
> directory
>                                   (multiply include directories are
> allowed).
>    -o, --output-file <filename>   Specify name of generated C file
>    -p, --embed-positions          If specified, the positions in
> Cython files of each
>                                   function definition is embedded in
> its docstring.
>    -z, --pre-import <module>      If specified, assume undeclared
> names in this
>                                   module. Emulates the behavior of
> putting
>                                   "from <module> import *" at the top
> of the file.

Re: Cython Distutils bug

Stefan Behnel wrote:
> I just noticed this bug in Cython.Distutils.build_ext which kept the
> "pyrex_c_in_temp" option from working, i.e. generated C files always ended up
> next to the source file.

... actually, there were two bugs and my fix made the second one come out,
which now prevents Distutils from putting the C sources where the .pyx sources
were (including not finding header files, etc...). Here is the fix for the fix.

Stefan

# HG changeset patch
# User Stefan Behnel <scoder@...>
# Date 1199181173 -3600
# Node ID 81f10bc7c32c68d2b79c950e32a6906dc186ce8a
# Parent  ceb176a73e7393204f0724da072231faa50e8799
another Distutils fix: keep source directory as target if not requested otherwise

diff -r ceb176a73e73 -r 81f10bc7c32c Distutils/build_ext.py
--- a/Distutils/build_ext.py	Tue Jan 01 10:40:46 2008 +0100
+++ b/Distutils/build_ext.py	Tue Jan 01 10:52:53 2008 +0100
@@ -153,12 +153,13 @@ class build_ext(_build_ext.build_ext):
                 or getattr(extension, 'pyrex_c_in_temp', 0)):
             target_dir = os.path.join(self.build_temp, "pyrex")
         else:
-            target_dir = ""
+            target_dir = None

unsigned long conversion

Hi,

while building up the pile of test cases, I stumbled over this one:

==================================
__doc__ = """
    >>> test(0)
    0L
    >>> test(1)
    1L
    >>> 2**36
    68719476736L
    >>> test(2**36)
    0L
    >>> test(2L**36)
    0L
"""

def test(k):
    cdef unsigned long m
    m = k
    return m
==================================

The doctest shows the actual results. Is this considered correct?

The generated code (minus error handling) looks like this:

  __pyx_1 = PyInt_AsUnsignedLongMask(__pyx_v_k);
  __pyx_v_m = __pyx_1;

Error handling/temp Cleanup / 3 Bug Fixes

Hello Robert,

well, I have been working with Cython 0.9.6.9 and attached you will find the 
patch that I have so far come up with. I would like you to review it and merge 
the parts that you see fit in.

It contains an improvement to the temp variable allocation algorithm that 
avoids that 

8 in a
8 in a

generates different code for the two lines, which makes testing ridicolously 
harder, if code changes just because of statements added before. Please merge 
that part.

Furthermore, I have found 3 alledged bugs.

Bug #1 was something I found checking generated code for what, I forgot. It 
striked me though, that it was testing !self after using it. That surely 
always worked, and exceptions on the other hand were missed.
Bug #2 is more serious. The code generated for

8 not in a

is wrong. This shows the benefit of having error checks separated from the 
actual operation.

Bug #3 is that print exceptions are not checked.

Re: Error handling/temp Cleanup / 3 Bug Fixes

Hi Kay,

Kay Hayen wrote:
> Note the use of enumerate. I wonder why it wasn't used before

Hmm, it came with Python 2.3. AFAICT, Cython generated code does necessarily
require Python 2.3 to run, so we should be careful with using these code in
the compiler that keeps it from running on older versions than what we can
support as C code. Robert?

> Here comes the utility code for the sequence test:
>  
> #------------------------------------------------------------------------------------
> +sequence_membership_test_ultity_code = [

Mind the typo.

> +"""
> +static int __Pyx_Sequence_Contains(PyObject *seq, PyObject *obj, int 
> negate); /*proto*/
> +""","""
> +static int __Pyx_Sequence_Contains(PyObject *seq, PyObject *obj, int negate)
> +{
> +    int result = PySequence_Contains(seq,obj);
> +
> +    /* Negate the result, unless it indicates an exception (result<0) */
> +    if (negate && result >= 0)
> +        result = !result;
> +
> +    return result;

PyLint - static error checking.

Hello,

I forgot to mention, I was looking at ExprNodes.py a lot, and one more bug is 
fixed, there was code like this inside:

    def generate_result_code(self, code):
        if self.is_py_attr:
            if Options.intern_names:
                code.putln(
                    '%s = PyObject_GetAttr(%s, %s); %s' % (
                        self.result_code,
                        self.obj.py_result(),
                        self.interned_attr_cname,
                        code.error_goto_if_null(self.result_code, self.pos)))
            else:
                code.putln(
                    '%s = PyObject_GetAttrString(%s, "%s"); %s' % (
                        self.result_code,
                        self.objpy_result(),
                        self.attribute,
                        code.error_goto_if_null(self.result_code, self.pos)))

Has an absolutely obvious typo. "objpy_result -> obj.py_result". Please merge 
that part.

Other than that, I think it shows that Pyline is not used before release. And 
it makes it questionable, if that option "not Options.intern_names" gets 
used. As it largely uglifies things, I would recommend to remove it if that's 
the case.

Re: Error handling/temp Cleanup / 3 Bug Fixes

Hello Stefan,

> Kay Hayen wrote:
> > Note the use of enumerate. I wonder why it wasn't used before
>
> Hmm, it came with Python 2.3. AFAICT, Cython generated code does
> necessarily require Python 2.3 to run, so we should be careful with using
> these code in the compiler that keeps it from running on older versions
> than what we can support as C code. Robert?

The generated code will still compile against the same versions as before. But 
using Python 2.2 in Cython compiler itself is not going to help making Cython 
itself readable Python.

Is it a goal to have the compiler run under 2.2?!

> > Here comes the utility code for the sequence test:
> >
> > #------------------------------------------------------------------------
> >------------ +sequence_membership_test_ultity_code = [
>
> Mind the typo.

Thanks, corrected.

> > +"""
> > +static int __Pyx_Sequence_Contains(PyObject *seq, PyObject *obj, int
> > negate); /*proto*/
> > +""","""

Re: Error handling/temp Cleanup / 3 Bug Fixes

> > Kay Hayen wrote:
> > > Note the use of enumerate. I wonder why it wasn't used before
> >
> > Hmm, it came with Python 2.3. AFAICT, Cython generated code does
> > necessarily require Python 2.3 to run, so we should be careful with using
> > these code in the compiler that keeps it from running on older versions
> > than what we can support as C code. Robert?
>
> The generated code will still compile against the same versions as before. But
> using Python 2.2 in Cython compiler itself is not going to help making Cython
> itself readable Python.
>
> Is it a goal to have the compiler run under 2.2?!

Even Debian uses python2.4 now. :)

Ondrej

Re: Suggested FAQ entry additions (please review)

Hello Chris,

thanks for the great code example. I have just added that information to the 
Wiki FAQ:

> cdef extern from "Python.h":
>     object PyString_FromStringAndSize(char *v, int len)
>     int PyString_AsStringAndSize(object obj, char **buffer,
> Py_ssize_t* length) except -1
>
> #copy a string by converting it to C values and back
> cdef copyString(object input_string):
>   cdef char *buffer
>   cdef Py_ssize_t len
>   PyString_AsStringAndSize(input_string, &buffer, &len)
>   return PyStringFromStringAndSize(buffer, len) #return a new string object

Also my personal thanks.

> Python argument unpacking has a format for calling this implicitly but
> since it requires mapping 2 C variables to a single Python argument
> you'd need new Cython syntax to use it. I'm not sure what that syntax
> could look like.

I have seen that as well. I would like (for now) a syntax like "bytes" on 
which a call of len() would do the right thing automatically. No need to have 
second variable IMHO. But then again, I am not too deep into Cython C syntax, 
I will come up with a standard syntax proposal later on anyway.

FAQ Update done

Hello,

living up to the promise I made, I have added all suggested FAQ entries, 
except the one about performance. These are results of your answers to my 
questions. 

For the performance question, I was considering to first test popular 
benchmarks like PyStone to have numbers.

So please review:
http://wiki.cython.org/FAQ

Best regards,
Kay Hayen