Stefan Behnel | 3 Jan 08:41 2011
Picon

[Cython] Fixing #602 - type inference for byte string literals

Hi,

I've been working on a fix for ticket #602, negative indexing for inferred 
char*.

http://trac.cython.org/cython_trac/ticket/602

Currently, when you write this:

     s = b'abc'

s is inferred as char*. This has several drawbacks. For one, we loose the 
length information, so "len(s)" becomes O(n) instead of O(1). Negative 
indexing fails completely because it will use pointer arithmetic, thus 
leaving the allocated memory area of the string. Also, code like the 
following is extremely inefficient because it requires multiple conversions 
from a char* of unknown length to a Python bytes object:

     s = b'abc'
     a = s1 + s
     b = s2 + s

I came to the conclusion that the right fix is to stop letting byte string 
literals start off as char*. This immediately fixes these issues and 
improves Python compatibility while still allowing automatic coercion, but 
it also comes with its own drawbacks.

In nogil blocks, you will have to explicitly declare a variable as char* 
when assigning a byte string literal to it, otherwise you'd get a compile 
time error for a Python object assignment. I think this is a minor issue as 
(Continue reading)

Lisandro Dalcin | 3 Jan 13:01 2011
Picon

Re: [Cython] Fixing #602 - type inference for byte string literals

On 3 January 2011 04:41, Stefan Behnel <stefan_ml <at> behnel.de> wrote:
> Hi,
>
> I've been working on a fix for ticket #602, negative indexing for inferred
> char*.
>
> http://trac.cython.org/cython_trac/ticket/602
>
> Currently, when you write this:
>
>     s = b'abc'
>
> s is inferred as char*. This has several drawbacks. For one, we loose the
> length information, so "len(s)" becomes O(n) instead of O(1). Negative
> indexing fails completely because it will use pointer arithmetic, thus
> leaving the allocated memory area of the string. Also, code like the
> following is extremely inefficient because it requires multiple conversions
> from a char* of unknown length to a Python bytes object:
>
>     s = b'abc'
>     a = s1 + s
>     b = s2 + s
>
> I came to the conclusion that the right fix is to stop letting byte string
> literals start off as char*. This immediately fixes these issues and
> improves Python compatibility while still allowing automatic coercion, but
> it also comes with its own drawbacks.
>
> In nogil blocks, you will have to explicitly declare a variable as char*
> when assigning a byte string literal to it, otherwise you'd get a compile
(Continue reading)

Robert Bradshaw | 3 Jan 18:35 2011

Re: [Cython] Fixing #602 - type inference for byte string literals

On Mon, Jan 3, 2011 at 4:01 AM, Lisandro Dalcin <dalcinl@...> wrote:
> On 3 January 2011 04:41, Stefan Behnel <stefan_ml@...> wrote:
>> Hi,
>>
>> I've been working on a fix for ticket #602, negative indexing for inferred
>> char*.
>>
>> http://trac.cython.org/cython_trac/ticket/602
>>
>> Currently, when you write this:
>>
>>     s = b'abc'
>>
>> s is inferred as char*. This has several drawbacks. For one, we loose the
>> length information, so "len(s)" becomes O(n) instead of O(1). Negative
>> indexing fails completely because it will use pointer arithmetic, thus
>> leaving the allocated memory area of the string. Also, code like the
>> following is extremely inefficient because it requires multiple conversions
>> from a char* of unknown length to a Python bytes object:
>>
>>     s = b'abc'
>>     a = s1 + s
>>     b = s2 + s
>>
>> I came to the conclusion that the right fix is to stop letting byte string
>> literals start off as char*. This immediately fixes these issues and
>> improves Python compatibility while still allowing automatic coercion, but
>> it also comes with its own drawbacks.
>>
>> In nogil blocks, you will have to explicitly declare a variable as char*
(Continue reading)

Dag Sverre Seljebotn | 3 Jan 19:10 2011
Picon
Picon

Re: [Cython] Fixing #602 - type inference for byte string literals

On 01/03/2011 06:35 PM, Robert Bradshaw wrote:
> On Mon, Jan 3, 2011 at 4:01 AM, Lisandro Dalcin<dalcinl@...>  wrote:
>    
>> On 3 January 2011 04:41, Stefan Behnel<stefan_ml@...>  wrote:
>>      
>>> Hi,
>>>
>>> I've been working on a fix for ticket #602, negative indexing for inferred
>>> char*.
>>>
>>> http://trac.cython.org/cython_trac/ticket/602
>>>
>>> Currently, when you write this:
>>>
>>>      s = b'abc'
>>>
>>> s is inferred as char*. This has several drawbacks. For one, we loose the
>>> length information, so "len(s)" becomes O(n) instead of O(1). Negative
>>> indexing fails completely because it will use pointer arithmetic, thus
>>> leaving the allocated memory area of the string. Also, code like the
>>> following is extremely inefficient because it requires multiple conversions
>>> from a char* of unknown length to a Python bytes object:
>>>
>>>      s = b'abc'
>>>      a = s1 + s
>>>      b = s2 + s
>>>
>>> I came to the conclusion that the right fix is to stop letting byte string
>>> literals start off as char*. This immediately fixes these issues and
>>> improves Python compatibility while still allowing automatic coercion, but
(Continue reading)

Stefan Behnel | 4 Jan 08:45 2011
Picon

Re: [Cython] Fixing #602 - type inference for byte string literals

Dag Sverre Seljebotn, 03.01.2011 19:10:
> On 01/03/2011 06:35 PM, Robert Bradshaw wrote:
>> On Mon, Jan 3, 2011 at 4:01 AM, Lisandro Dalcin<dalcinl@...>   wrote:
>>
>>> On 3 January 2011 04:41, Stefan Behnel<stefan_ml@...>   wrote:
>>>
>>>> Hi,
>>>>
>>>> I've been working on a fix for ticket #602, negative indexing for inferred
>>>> char*.
>>>>
>>>> http://trac.cython.org/cython_trac/ticket/602
>>>>
>>>> Currently, when you write this:
>>>>
>>>>       s = b'abc'
>>>>
>>>> s is inferred as char*. This has several drawbacks. For one, we loose the
>>>> length information, so "len(s)" becomes O(n) instead of O(1). Negative
>>>> indexing fails completely because it will use pointer arithmetic, thus
>>>> leaving the allocated memory area of the string. Also, code like the
>>>> following is extremely inefficient because it requires multiple conversions
>>>> from a char* of unknown length to a Python bytes object:
>>>>
>>>>       s = b'abc'
>>>>       a = s1 + s
>>>>       b = s2 + s
>>>>
>>>> I came to the conclusion that the right fix is to stop letting byte string
>>>> literals start off as char*. This immediately fixes these issues and
(Continue reading)

Daniel Norberg | 6 Jan 12:20 2011
Picon

[Cython] Cython 0.14 Bugreport

Hi folks,

I'm getting a curious error in Cython 0.14 when trying to compile this:

  def bar(foo):
      qux = foo
      quux = foo[qux.baz]

The error message:

	$ cython bar.py

	Error compiling Cython file:
	------------------------------------------------------------
	...
	def bar(foo):
		qux = foo
		quux = foo[qux.baz]
	              ^
	------------------------------------------------------------

	/Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '<unspecified>' has no attribute 'baz'

Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502) and
it fails.

Regards,
Daniel
Stefan Behnel | 6 Jan 19:30 2011
Picon

Re: [Cython] Cython 0.14 Bugreport

Daniel Norberg, 06.01.2011 12:20:
> I'm getting a curious error in Cython 0.14 when trying to compile this:
>
>    def bar(foo):
>        qux = foo
>        quux = foo[qux.baz]
>
> The error message:
>
> 	$ cython bar.py
>
> 	Error compiling Cython file:
> 	------------------------------------------------------------
> 	...
> 	def bar(foo):
> 		qux = foo
> 		quux = foo[qux.baz]
> 	              ^
> 	------------------------------------------------------------
>
> 	/Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '<unspecified>' has no attribute 'baz'
>
> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502) and
it fails.

I can reproduce this. From a quick test, it seems like the type inference 
machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before 
'qux'. Anyone interested in taking a closer look?

Stefan
(Continue reading)

Robert Bradshaw | 6 Jan 19:45 2011

Re: [Cython] Cython 0.14 Bugreport

On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel <stefan_ml@...> wrote:
> Daniel Norberg, 06.01.2011 12:20:
>> I'm getting a curious error in Cython 0.14 when trying to compile this:
>>
>>    def bar(foo):
>>        qux = foo
>>        quux = foo[qux.baz]
>>
>> The error message:
>>
>>       $ cython bar.py
>>
>>       Error compiling Cython file:
>>       ------------------------------------------------------------
>>       ...
>>       def bar(foo):
>>               qux = foo
>>               quux = foo[qux.baz]
>>                     ^
>>       ------------------------------------------------------------
>>
>>       /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '<unspecified>' has no
attribute 'baz'
>>
>> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502)
and it fails.
>
> I can reproduce this. From a quick test, it seems like the type inference
> machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before
> 'qux'. Anyone interested in taking a closer look?
(Continue reading)

Vitja Makarov | 6 Jan 19:48 2011
Picon

Re: [Cython] Cython 0.14 Bugreport

2011/1/6 Robert Bradshaw <robertwb@...>:
> On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel <stefan_ml@...> wrote:
>> Daniel Norberg, 06.01.2011 12:20:
>>> I'm getting a curious error in Cython 0.14 when trying to compile this:
>>>
>>>    def bar(foo):
>>>        qux = foo
>>>        quux = foo[qux.baz]
>>>
>>> The error message:
>>>
>>>       $ cython bar.py
>>>
>>>       Error compiling Cython file:
>>>       ------------------------------------------------------------
>>>       ...
>>>       def bar(foo):
>>>               qux = foo
>>>               quux = foo[qux.baz]
>>>                     ^
>>>       ------------------------------------------------------------
>>>
>>>       /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '<unspecified>' has no
attribute 'baz'
>>>
>>> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502)
and it fails.
>>
>> I can reproduce this. From a quick test, it seems like the type inference
>> machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before
(Continue reading)

Robert Bradshaw | 6 Jan 19:54 2011

Re: [Cython] Cython 0.14 Bugreport

On Thu, Jan 6, 2011 at 10:48 AM, Vitja Makarov
<vitja.makarov@...> wrote:
> 2011/1/6 Robert Bradshaw <robertwb@...>:
>> On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel <stefan_ml@...> wrote:
>>> Daniel Norberg, 06.01.2011 12:20:
>>>> I'm getting a curious error in Cython 0.14 when trying to compile this:
>>>>
>>>>    def bar(foo):
>>>>        qux = foo
>>>>        quux = foo[qux.baz]
>>>>
>>>> The error message:
>>>>
>>>>       $ cython bar.py
>>>>
>>>>       Error compiling Cython file:
>>>>       ------------------------------------------------------------
>>>>       ...
>>>>       def bar(foo):
>>>>               qux = foo
>>>>               quux = foo[qux.baz]
>>>>                     ^
>>>>       ------------------------------------------------------------
>>>>
>>>>       /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '<unspecified>' has no
attribute 'baz'
>>>>
>>>> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502)
and it fails.
>>>
(Continue reading)


Gmane