General discussion about the Cython source code generator for Python extensions

Stefan Behnel | 3 Jan 2011 08:41

Picon

[Cython] Fixing #602 - type inference for byte string literals

Hi,

I've been working on a fix for ticket #602, negative indexing for inferred 
char*.

http://trac.cython.org/cython_trac/ticket/602

Currently, when you write this:

     s = b'abc'

s is inferred as char*. This has several drawbacks. For one, we loose the 
length information, so "len(s)" becomes O(n) instead of O(1). Negative 
indexing fails completely because it will use pointer arithmetic, thus 
leaving the allocated memory area of the string. Also, code like the 
following is extremely inefficient because it requires multiple conversions 
from a char* of unknown length to a Python bytes object:

     s = b'abc'
     a = s1 + s
     b = s2 + s

I came to the conclusion that the right fix is to stop letting byte string 
literals start off as char*. This immediately fixes these issues and 
improves Python compatibility while still allowing automatic coercion, but 
it also comes with its own drawbacks.

In nogil blocks, you will have to explicitly declare a variable as char* 
when assigning a byte string literal to it, otherwise you'd get a compile 
time error for a Python object assignment. I think this is a minor issue as

(Continue reading)

Lisandro Dalcin | 3 Jan 2011 13:01

Picon

Re: [Cython] Fixing #602 - type inference for byte string literals

On 3 January 2011 04:41, Stefan Behnel <stefan_ml <at> behnel.de> wrote:
> Hi,
>
> I've been working on a fix for ticket #602, negative indexing for inferred
> char*.
>
> http://trac.cython.org/cython_trac/ticket/602
>
> Currently, when you write this:
>
>     s = b'abc'
>
> s is inferred as char*. This has several drawbacks. For one, we loose the
> length information, so "len(s)" becomes O(n) instead of O(1). Negative
> indexing fails completely because it will use pointer arithmetic, thus
> leaving the allocated memory area of the string. Also, code like the
> following is extremely inefficient because it requires multiple conversions
> from a char* of unknown length to a Python bytes object:
>
>     s = b'abc'
>     a = s1 + s
>     b = s2 + s
>
> I came to the conclusion that the right fix is to stop letting byte string
> literals start off as char*. This immediately fixes these issues and
> improves Python compatibility while still allowing automatic coercion, but
> it also comes with its own drawbacks.
>
> In nogil blocks, you will have to explicitly declare a variable as char*
> when assigning a byte string literal to it, otherwise you'd get a compile

(Continue reading)

Robert Bradshaw | 3 Jan 2011 18:35

Re: [Cython] Fixing #602 - type inference for byte string literals

On Mon, Jan 3, 2011 at 4:01 AM, Lisandro Dalcin <dalcinl@...> wrote:
> On 3 January 2011 04:41, Stefan Behnel <stefan_ml@...> wrote:
>> Hi,
>>
>> I've been working on a fix for ticket #602, negative indexing for inferred
>> char*.
>>
>> http://trac.cython.org/cython_trac/ticket/602
>>
>> Currently, when you write this:
>>
>>     s = b'abc'
>>
>> s is inferred as char*. This has several drawbacks. For one, we loose the
>> length information, so "len(s)" becomes O(n) instead of O(1). Negative
>> indexing fails completely because it will use pointer arithmetic, thus
>> leaving the allocated memory area of the string. Also, code like the
>> following is extremely inefficient because it requires multiple conversions
>> from a char* of unknown length to a Python bytes object:
>>
>>     s = b'abc'
>>     a = s1 + s
>>     b = s2 + s
>>
>> I came to the conclusion that the right fix is to stop letting byte string
>> literals start off as char*. This immediately fixes these issues and
>> improves Python compatibility while still allowing automatic coercion, but
>> it also comes with its own drawbacks.
>>
>> In nogil blocks, you will have to explicitly declare a variable as char*

(Continue reading)

Dag Sverre Seljebotn | 3 Jan 2011 19:10

Picon

Picon

Re: [Cython] Fixing #602 - type inference for byte string literals

On 01/03/2011 06:35 PM, Robert Bradshaw wrote:
> On Mon, Jan 3, 2011 at 4:01 AM, Lisandro Dalcin<dalcinl@...>  wrote:
>    
>> On 3 January 2011 04:41, Stefan Behnel<stefan_ml@...>  wrote:
>>      
>>> Hi,
>>>
>>> I've been working on a fix for ticket #602, negative indexing for inferred
>>> char*.
>>>
>>> http://trac.cython.org/cython_trac/ticket/602
>>>
>>> Currently, when you write this:
>>>
>>>      s = b'abc'
>>>
>>> s is inferred as char*. This has several drawbacks. For one, we loose the
>>> length information, so "len(s)" becomes O(n) instead of O(1). Negative
>>> indexing fails completely because it will use pointer arithmetic, thus
>>> leaving the allocated memory area of the string. Also, code like the
>>> following is extremely inefficient because it requires multiple conversions
>>> from a char* of unknown length to a Python bytes object:
>>>
>>>      s = b'abc'
>>>      a = s1 + s
>>>      b = s2 + s
>>>
>>> I came to the conclusion that the right fix is to stop letting byte string
>>> literals start off as char*. This immediately fixes these issues and
>>> improves Python compatibility while still allowing automatic coercion, but

(Continue reading)

Stefan Behnel | 4 Jan 2011 08:45

Picon

Re: [Cython] Fixing #602 - type inference for byte string literals

Dag Sverre Seljebotn, 03.01.2011 19:10:
> On 01/03/2011 06:35 PM, Robert Bradshaw wrote:
>> On Mon, Jan 3, 2011 at 4:01 AM, Lisandro Dalcin<dalcinl@...>   wrote:
>>
>>> On 3 January 2011 04:41, Stefan Behnel<stefan_ml@...>   wrote:
>>>
>>>> Hi,
>>>>
>>>> I've been working on a fix for ticket #602, negative indexing for inferred
>>>> char*.
>>>>
>>>> http://trac.cython.org/cython_trac/ticket/602
>>>>
>>>> Currently, when you write this:
>>>>
>>>>       s = b'abc'
>>>>
>>>> s is inferred as char*. This has several drawbacks. For one, we loose the
>>>> length information, so "len(s)" becomes O(n) instead of O(1). Negative
>>>> indexing fails completely because it will use pointer arithmetic, thus
>>>> leaving the allocated memory area of the string. Also, code like the
>>>> following is extremely inefficient because it requires multiple conversions
>>>> from a char* of unknown length to a Python bytes object:
>>>>
>>>>       s = b'abc'
>>>>       a = s1 + s
>>>>       b = s2 + s
>>>>
>>>> I came to the conclusion that the right fix is to stop letting byte string
>>>> literals start off as char*. This immediately fixes these issues and

(Continue reading)

Daniel Norberg | 6 Jan 2011 12:20

Picon

[Cython] Cython 0.14 Bugreport

Hi folks,

I'm getting a curious error in Cython 0.14 when trying to compile this:

  def bar(foo):
      qux = foo
      quux = foo[qux.baz]

The error message:

	$ cython bar.py

	Error compiling Cython file:
	------------------------------------------------------------
	...
	def bar(foo):
		qux = foo
		quux = foo[qux.baz]
	              ^
	------------------------------------------------------------

	/Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '<unspecified>' has no attribute 'baz'

Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502) and
it fails.

Regards,
Daniel

Stefan Behnel | 6 Jan 2011 19:30

Picon

Re: [Cython] Cython 0.14 Bugreport

Daniel Norberg, 06.01.2011 12:20:
> I'm getting a curious error in Cython 0.14 when trying to compile this:
>
>    def bar(foo):
>        qux = foo
>        quux = foo[qux.baz]
>
> The error message:
>
> 	$ cython bar.py
>
> 	Error compiling Cython file:
> 	------------------------------------------------------------
> 	...
> 	def bar(foo):
> 		qux = foo
> 		quux = foo[qux.baz]
> 	              ^
> 	------------------------------------------------------------
>
> 	/Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '<unspecified>' has no attribute 'baz'
>
> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502) and
it fails.

I can reproduce this. From a quick test, it seems like the type inference 
machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before 
'qux'. Anyone interested in taking a closer look?

Stefan

(Continue reading)

Robert Bradshaw | 6 Jan 2011 19:45

Re: [Cython] Cython 0.14 Bugreport

On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel <stefan_ml@...> wrote:
> Daniel Norberg, 06.01.2011 12:20:
>> I'm getting a curious error in Cython 0.14 when trying to compile this:
>>
>>    def bar(foo):
>>        qux = foo
>>        quux = foo[qux.baz]
>>
>> The error message:
>>
>>       $ cython bar.py
>>
>>       Error compiling Cython file:
>>       ------------------------------------------------------------
>>       ...
>>       def bar(foo):
>>               qux = foo
>>               quux = foo[qux.baz]
>>                     ^
>>       ------------------------------------------------------------
>>
>>       /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '<unspecified>' has no
attribute 'baz'
>>
>> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502)
and it fails.
>
> I can reproduce this. From a quick test, it seems like the type inference
> machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before
> 'qux'. Anyone interested in taking a closer look?

(Continue reading)

Vitja Makarov | 6 Jan 2011 19:48

Picon

Re: [Cython] Cython 0.14 Bugreport

2011/1/6 Robert Bradshaw <robertwb@...>:
> On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel <stefan_ml@...> wrote:
>> Daniel Norberg, 06.01.2011 12:20:
>>> I'm getting a curious error in Cython 0.14 when trying to compile this:
>>>
>>>    def bar(foo):
>>>        qux = foo
>>>        quux = foo[qux.baz]
>>>
>>> The error message:
>>>
>>>       $ cython bar.py
>>>
>>>       Error compiling Cython file:
>>>       ------------------------------------------------------------
>>>       ...
>>>       def bar(foo):
>>>               qux = foo
>>>               quux = foo[qux.baz]
>>>                     ^
>>>       ------------------------------------------------------------
>>>
>>>       /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '<unspecified>' has no
attribute 'baz'
>>>
>>> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502)
and it fails.
>>
>> I can reproduce this. From a quick test, it seems like the type inference
>> machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before

(Continue reading)

Group

gmane.comp.python.cython.devel.

Project Web Page

General discussion about the Cython source code generator for Python extensions

Search Archive

Page

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 21

Articles in period: 200

January 2011

Language

Change language

Options

Current view: Threads only / Showing only 20 lines / Not hiding cited text.
Change to All messages, whole messages, or hide cited text.

Post a message
NNTP Newsgroup
Classic Gmane web interface
XML

XML

RSS Feed
List Information

About Gmane

Recent entries

[Cython] How should be C-clobal deletion handled? (24 May 2013)
[Cython] Compiler crash fix (22 May 2013)
[Cython] CF based type inference (21 May 2013)
[Cython] nogil doesn't seem to work when defined on extern cpp functio (8 May 2013)
[Cython] [cython] Order cdef classes by inheritance before generating (6 May 2013)
[Cython] [cython] Hide Cython utility classes (like memoryview) from P (5 May 2013)
[Cython] any more fixes for 0.19.1 ? (3 May 2013)
[Cython] Regresion in 0.19 (1 May 2013)
[Cython] Biggest Fake Conference in Computer Science (1 May 2013)
[Cython] -O3 causes MinGW to segfault python (29 Apr 2013)
[Cython] A little bugtracker cleanup (26 Apr 2013)
[Cython] Fwd: Re: Version of Cython packages in Ubuntu PPA (26 Apr 2013)
[Cython] Version of Cython packages in Ubuntu PPA (26 Apr 2013)
[Cython] regarding ticket #810 (segfault in Sage) (26 Apr 2013)
[Cython] Cython code producing different, incorrect results under Pyth (24 Apr 2013)
[Cython] Unit testing Cython extensions (23 Apr 2013)
[Cython] libc.string msvc incompatibility (23 Apr 2013)
[Cython] Surprising behaviour wrt. generated tp_clear and tp_dealloc f (21 Apr 2013)
[Cython] gsoc-kurt-regraft branch and ticket #469 (19 Apr 2013)
[Cython] [cython] Autodoc improvements (#216) (19 Apr 2013)
[Cython] bint and autodoc signatures (14 Apr 2013)

Archives

37	May 2013
129	April 2013
87	March 2013
75	February 2013
31	January 2013
30	December 2012
33	November 2012
45	October 2012
56	September 2012
154	August 2012
128	July 2012
124	June 2012
279	May 2012
235	April 2012
158	March 2012
146	February 2012
125	January 2012
20	December 2011
43	November 2011
196	October 2011
116	September 2011
184	August 2011
245	July 2011
72	June 2011
220	May 2011
294	April 2011
217	March 2011
105	February 2011
200	January 2011
237	December 2010
377	November 2010
62	October 2010
124	September 2010
232	August 2010
288	July 2010
88	June 2010
135	May 2010
230	April 2010
222	March 2010
181	February 2010
111	January 2010
253	December 2009
245	November 2009
607	October 2009
360	September 2009
219	August 2009
206	July 2009
357	June 2009
639	May 2009
527	April 2009
678	March 2009
406	February 2009
181	January 2009
274	December 2008
291	November 2008
418	October 2008
256	September 2008
320	August 2008
215	July 2008
284	June 2008
705	May 2008
398	April 2008
223	March 2008
65	February 2008
133	January 2008
137	December 2007
82	November 2007
49	October 2007
151	September 2007
32	August 2007

Design

Zawodny | Right Menu | Left Menu

Your Own Design

Paste the URL of your CSS below. Download CSS template