headers
Richie Ward | 1 Oct 2007 01:11
Picon

OpenSSL httplib bug

I was using httplib to power my xml rpc script.

I had problems when I wanted to use SSL and I got this error:
  File "/usr/lib/python2.5/httplib.py", line 1109, in recv
    return self._ssl.read(len)
socket.sslerror: (8, 'EOF occurred in violation of protocol')

I figured out this was because of poor error handling in python.

May I suggest this as a fix to this bug:
$ diff /usr/lib/python2.5/httplib.py /usr/lib/python2.5/httplib.py~
1109,1112c1109
<         try:
<             return self._ssl.read(len)
<         except socket.sslerror:
<             return
---
>         return self._ssl.read(len)

Just a note. I am by no means a python expert, just good enough to get
my work done.
I use Ubuntu gutsy.
--

-- 
Thanks, Richie Ward
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
(Continue reading)

Permalink | Reply | 752 comments |
headers
Greg Ewing | 1 Oct 2007 02:17
Picon
Picon
Favicon

Re: [python] Re: New lines, carriage returns, and Windows

Nick Maclaren wrote:
> I don't know PRECISELY what you mean by "universal newlines mode"

I mean precisely what Python means by the term: any of
"\r", "\n" or "\r\n" represent a newline, and no distinction
is made between them.

You only need to use that if you don't know what convention
is being used by the file you're reading. And if you don't
know that, you've already lost information about what the
contents of the file means, and there's nothing that any
I/O system can do to get it back.

> Model 1:  certain characters can be used only in combination.
 > ...

That's all fine if you know the file adheres to those
conventions. Just open it in binary mode and go for it.

The I/O systems of C and/or Python are designed for
environments where the files *don't* adhere to conventions
as helpful as that. They're making the best of what they're
given.

> Note that the above is what the program sees - what is written
> to the outside world and how input is read is another matter.
> 
> But I can assure you, from my own and many other people's experience,
> that neither of the above models cause the confusion being shown by
> the postings in this thread.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Brett Cannon | 1 Oct 2007 02:17
Favicon
Gravatar

Re: OpenSSL httplib bug

On 9/30/07, Richie Ward <richies <at> gmail.com> wrote:
> I was using httplib to power my xml rpc script.
>
> I had problems when I wanted to use SSL and I got this error:
>   File "/usr/lib/python2.5/httplib.py", line 1109, in recv
>     return self._ssl.read(len)
> socket.sslerror: (8, 'EOF occurred in violation of protocol')
>
> I figured out this was because of poor error handling in python.
>
> May I suggest this as a fix to this bug:
> $ diff /usr/lib/python2.5/httplib.py /usr/lib/python2.5/httplib.py~
> 1109,1112c1109
> <         try:
> <             return self._ssl.read(len)
> <         except socket.sslerror:
> <             return
> ---
> >         return self._ssl.read(len)
>
> Just a note. I am by no means a python expert, just good enough to get
> my work done.
> I use Ubuntu gutsy.

If you could, Richie, please open a bug report at bugs.python.org and
attach your patch to it?  That way it won't get lost and it can be
attended to properly.

-Brett
_______________________________________________
(Continue reading)

Permalink | Reply | 0 comments |
headers
Greg Ewing | 1 Oct 2007 02:37
Picon
Picon
Favicon

Re: New lines, carriage returns, and Windows

Nick Maclaren wrote:
> I have implemented both of those two models
> on systems that are FAR more different than most people can imagine.
> Both work, and neither causes confusion.  The C/Unix/Python one does.

Now I'm not sure what *you* mean by the C/Unix/Python
model. As far as newlines are concerned, the internal
model is fine as far as I can see.

> a mismatch between
>>the world of Python strings which use "\n" and .NET
>>library code expecting strings which use "\r\n".
> 
> That's an I/O problem :-)

If you define passing a string to/from any .NET function
as I/O, I suppose that's true, but it's not what people
normally mean by the term.

> the REASON it causes trouble is the inconsistency
> in the basic C/Unix/Python text I/O model.  Let's consider just
> \f, \r and \n,

But we're not talking about \f or anything else here, only
newlines. I've never heard anyone complain about getting
confused over the handling of \f in Python. That may be
because nobody uses \f for anything these days, but whatever
the reason, it seems to be a non-issue.

In any case, it still doesn't mean that you "don't get
(Continue reading)

Permalink | Reply | 0 comments |
headers
Greg Ewing | 1 Oct 2007 02:48
Picon
Picon
Favicon

Re: New lines, carriage returns, and Windows

skip <at> pobox.com wrote:
> I've been thinking about this some more (in lieu of actually writing up any
> sort of proposal ;-) and I'm not so sure it would be all that useful.

Yes, despite being the one who suggested it, I've come to
the same conclusion myself. The problem should really be
addressed at the source, which is the Python/.NET boundary.
Anything else would just lead to ambiguity.

So I'm voting -1 on my own proposal here.

--

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing <at> canterbury.ac.nz	   +--------------------------------------+
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 1 comment |
headers
Greg Ewing | 1 Oct 2007 02:57
Picon
Picon
Favicon

Re: [python] Re: New lines, carriage returns, and Windows

Michael Foord wrote:
> We stick to using the .NET file I/O and so don't 
> have a problem. The only time it is an issue for us is our tests, where 
> we have string literals in our test code (where new lines are obviously 
> '\n')

If you're going to do that, you really need to be consistent
about and have IronPython use \r\n internally for line endings
*everywhere*, including string literals.

> It is just slightly ironic that the time Python 'gets it wrong' (for 
> some value of wrong) is when you are using text mode for I/O :-)

I would say IronPython is getting it wrong by using inconsistent
internal representations of line endings.

--

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing <at> canterbury.ac.nz	   +--------------------------------------+
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 2 comments |
headers
Guido van Rossum | 1 Oct 2007 04:14
Favicon

Re: [python] Re: New lines, carriage returns, and Windows

On 9/30/07, Greg Ewing <greg.ewing <at> canterbury.ac.nz> wrote:
> Michael Foord wrote:
> > We stick to using the .NET file I/O and so don't
> > have a problem. The only time it is an issue for us is our tests, where
> > we have string literals in our test code (where new lines are obviously
> > '\n')
>
> If you're going to do that, you really need to be consistent
> about and have IronPython use \r\n internally for line endings
> *everywhere*, including string literals.

I don't know what you mean by "internally". There's lots of portable
code that uses the \n character in string literals (either to generate
line endings or to recognize them). That code can't suddenly be made
invalid. And changing all string literals that say "\n" to secretly
become "\r\n" would be worse than the \r <--> \n swap that some old
Apple tools used to do. (If len("\n") == 2, what would len("\r\n")
be?)

> > It is just slightly ironic that the time Python 'gets it wrong' (for
> > some value of wrong) is when you are using text mode for I/O :-)
>
> I would say IronPython is getting it wrong by using inconsistent
> internal representations of line endings.

Honestly, I find it hard to see much merit in this discussion. A
number of Python libraries, including print() and io.py, use \n to
represent line endings in memory, and translate these to/from
platform-appropriate line endings when reading/writing text files.
OTOH, some other APIs, for example, sockets talking various internet
(Continue reading)

Permalink | Reply | 1 comment |
headers
Justin Tulloss | 1 Oct 2007 08:43
Picon

GC Changes

Hello,

I've been doing some tests on removing the GIL, and it's becoming clear that some basic changes to the garbage collector may be needed in order for this to happen efficiently. Reference counting as it stands today is not very scalable.

I've been looking into a few options, and I'm leaning towards the implementing IBMs recycler GC ( http://www.research.ibm.com/people/d/dfb/recycler-publications.html ) since it is very similar to what is in place now from the users' perspective. However, I haven't been around the list long enough to really understand the feeling in the community on GC in the future of the interpreter. It seems that a full GC might have a lot of benefits in terms of performance and scalability, and I think that the current gc module is of the mark-and-sweep variety. Is the trend going to be to move away from reference counting and towards the mark-and-sweep implementation that currently exists, or is reference counting a firmly ingrained tradition?

On a more immediately relevant note, I'm not certain I understand the full extent of the gc module. From what I've read, it sounds like it's fairly close to a fully functional GC, yet it seems to exist only as a cycle-detecting backup to the reference counting mechanism. Would somebody care to give me a brief overview on how the current gc module interacts with the interpreter, or point me to a place where that is done? Why isn't the mark-and-sweep mechanism used for all memory management?

Thanks a lot!

Justin 
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 27 comments |
headers
Nick Maclaren | 1 Oct 2007 10:27
Picon
Picon

Re: [python] Re: New lines, carriage returns, and Windows

Greg Ewing <greg.ewing <at> canterbury.ac.nz> wrote:
>
> > I don't know PRECISELY what you mean by "universal newlines mode"
> 
> I mean precisely what Python means by the term: any of
> "\r", "\n" or "\r\n" represent a newline, and no distinction
> is made between them.

Excellent.  While this over-simplifies the issue, let's stick to
the over-simplified form, as we may be able to get somewhere.

The question is independent of what the outside system believes a
text file should look like, and is solely what Python believes a
sequence of characters should mean.  For example, does 'A\r\nB'
mean that B is separated from A by one newline or two?

The point is that, once we know that, we can design a translator
to and from Python's conventions to any reasonable system (and,
as I say, I have done it many times).  But, if Python's own
interpretation is ambiguous, it is a sure recipe for different
translators being incompatible, even on the same system.  Which
is what has happened here.

So, damn the outside system, EXACTLY what does Python mean by
such characters, and EXACTLY what uses of them are discouraged
as having unspecified meanings?  If we could get an answer to
that precisely enough to write a parse tree with all terminals
explicit, this problem would go away.

And that is all that I say can or should be done.  The details
of how to write the translators to other file systems are then
a separate matter.

Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 <at> cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 4 comments |
headers
Gustavo Carneiro | 1 Oct 2007 13:10
Picon
Gravatar

Re: GC Changes

On 01/10/2007, Justin Tulloss <tulloss2 <at> uiuc.edu> wrote:

Hello,

I've been doing some tests on removing the GIL, and it's becoming clear that some basic changes to the garbage collector may be needed in order for this to happen efficiently. Reference counting as it stands today is not very scalable.

I've been looking into a few options, and I'm leaning towards the implementing IBMs recycler GC ( http://www.research.ibm.com/people/d/dfb/recycler-publications.html ) since it is very similar to what is in place now from the users' perspective. However, I haven't been around the list long enough to really understand the feeling in the community on GC in the future of the interpreter. It seems that a full GC might have a lot of benefits in terms of performance and scalability, and I think that the current gc module is of the mark-and-sweep variety. Is the trend going to be to move away from reference counting and towards the mark-and-sweep implementation that currently exists, or is reference counting a firmly ingrained tradition?

On a more immediately relevant note, I'm not certain I understand the full extent of the gc module. From what I've read, it sounds like it's fairly close to a fully functional GC, yet it seems to exist only as a cycle-detecting backup to the reference counting mechanism. Would somebody care to give me a brief overview on how the current gc module interacts with the interpreter, or point me to a place where that is done? Why isn't the mark-and-sweep mechanism used for all memory management?

The cyclic GC is just too slow to react and makes programmers mad.

For instance, in PyGtk we had a traditional problem with gtk.gdk.Pixbuf, which is basically an object that wraps a raw RGB image.  When users deleted such an object, which could sometimes comprise tens or hundreds of megabytes, the memory was not relased until much much later.  That kind of code ended up having to manually call gc.collect() to fix what was perceived by most programmers as a "memory leak", which kind of defeats the purpose of a garbage collector.  This happened because PyGtk used to rely on the cyclic GC doing its work.  Thankfully we moved away from that and now simple reference counting can free a Pixbuf in most cases.

The cyclic GC is a very useful system, but it should only be used in addition to, not instead of, reference counting.  At least that's my personal opinion...

--
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert 
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 0 comments |

Gmane