headers
P.J. Eby | 4 Nov 2010 00:19
Gravatar

Should PEP 3333 be Python 3-only? What about transcoding?

As I've been tidying up wsgiref in the stdlib for PEP 3333, I've been 
noticing that there's a bit of an issue with the PEP as far as CGI variables.

Currently, the CGI example is the same as it is in PEP 3333, which 
means that it's correct code for Python 2.x, but wrong for 3.x due to 
the environment transcoding issue.  (See 
http://bugs.python.org/issue10155 for details.)

There are other code sample differences, too.  In effect, PEP 3333 is 
still using Python 2 code samples, because it's trying to cover every 
version of Python from 2.1 through 3.2.

Should we ditch that, and say, "hey, if you want Python 2.x code 
samples, go see PEP 333?"

That will simplify a couple of things, but still won't address the 
transcoding issue.

Specifically, the problem is that on Python 3, os.environ contains 
*unicode*, not bytes masquerading as unicode.  Unfortunately, this 
means that it very possibly contains garbage for CGI variables, as 
the web server puts bytes in the environment, then Python converts 
those bytes to unicode using the system encoding + surrogateescape.

To get back to bytes, then, we have to decode using the same 
combination, then re-encode with latin-1 to get back to a 
WSGI-compatible string.

The hitch is this: not everything in os.environ comes from an HTTP 
request, and therefore may not be decodable in such a fashion.  For
(Continue reading)

Permalink | Reply | 2 comments |
headers
and-py | 4 Nov 2010 02:26
Favicon

Re: Should PEP 3333 be Python 3-only? What about transcoding?

On Wed, 2010-11-03 at 19:19 -0400, P.J. Eby wrote:
> Should we ditch that, and say, "hey, if you want Python 2.x code 
> samples, go see PEP 333?"

That seems reasonable to me: if there is indeed never going to be a
Python 2.8, there is no way the PEP can ever be accepted for a Python 2
release anyway.

Given this, I might go further and suggest dropping all mention of
Python 2. It might make the wording issues easier. (Although, ahem, that
would mean a bunch of rewriting for some poor soul.)

> 2. Make the CGI sample in 3333 do an indiscriminate transcode (which 
> only takes a few lines) and add a note to indicate that a robust CGI 
> implementation should only do it to CGI variables

Or go straight for unmolested os.environ, as long as there is that note
that it's not really the Right Thing. If we're going to be wrong for
some cases either way, might as well go for the simplest. The PEP code
needs to be illustrative more than it needs to be 100% correct.

--

-- 
And Clover
mailto:and@... http://www.doxdesk.com
skype:uknrbobince gtalk:chat?jid=bobince@...

_______________________________________________
Web-SIG mailing list
Web-SIG@...
Web SIG: http://www.python.org/sigs/web-sig
(Continue reading)

Permalink | Reply | 1 comment |
headers
Lennart Regebro | 4 Nov 2010 06:16
Picon
Gravatar

Re: Should PEP 3333 be Python 3-only? What about transcoding?

On Thu, Nov 4, 2010 at 02:26, and-py <and-py@...> wrote:
> On Wed, 2010-11-03 at 19:19 -0400, P.J. Eby wrote:
>> Should we ditch that, and say, "hey, if you want Python 2.x code
>> samples, go see PEP 333?"
>
> That seems reasonable to me: if there is indeed never going to be a
> Python 2.8, there is no way the PEP can ever be accepted for a Python 2
> release anyway.

That doesn't mean it should not work on Python 2. Major rule in Python
3 porting: Don't change your API's. ;-)
_______________________________________________
Web-SIG mailing list
Web-SIG@...
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org
Permalink | Reply | 0 comments |
headers
Alice Bevan-McGregor | 21 Nov 2010 12:12
Gravatar

PEP 444

(A version of this is is available at http://web-core.org/2.0/pep-0444/ — links are links, code may be
easier to read.)

PEP 444 is quite exciting to me.  So much so that I’ve been spending a few days writing a high-performance
(C10K, 10Krsec) Py2.6+/3.1+ HTTP/1.1 server which implements much of the proposed standard.  The server
is functional (less web3.input at the time of this writing), but differs from PEP 444 in several ways.  It
also adds several features I feel should be part of the spec.

Source for the server is available on GitHub:

	https://github.com/pulp/marrow.server.http

I have made several notes about the PEP 444 specification during implementation of the above, and concern
over some implementation details:

First, async is poorly defined:

> If the origin server advertises that it has the web3.async capability, a Web3 application callable used
by the server is permitted to return a callable that accepts no arguments. When it does so, this callable is
to be called periodically by the origin server until it returns a non-None response, which must be a normal
Web3 response tuple.

Polling is not true async.  I believe that it should be up to the server to define how async is utilized, and
that the specification should be clarified on this point.  (“Called periodically” is too vague.) 
“Callable” should likely be redefined as “generator” (a callable that yields) as most
applications require holding on to state and wrapping everything in functools.partial() is somewhat
ugly.  Utilizing generators would improve support for existing Python async frameworks, and allow four
modes of operation: yield None (no response, keep waiting), yield response_tuple (standard response),
return / raise StopIteration (close the async connection) and allow for data to be passed back to the async
callable by the higher-level async framework.
(Continue reading)

Permalink | Reply | 13 comments |
headers
Chris McDonough | 21 Nov 2010 17:55
Favicon
Gravatar

Re: PEP 444

PEP 444 has no champion currently.  Both Armin and I have basically left
it behind.  It would be great if you wanted to be its champion.

- C

On Sun, 2010-11-21 at 03:12 -0800, Alice Bevan-McGregor wrote:
> (A version of this is is available at http://web-core.org/2.0/pep-0444/ — links are links, code may be
easier to read.)
> 
> PEP 444 is quite exciting to me.  So much so that I’ve been spending a few days writing a high-performance
(C10K, 10Krsec) Py2.6+/3.1+ HTTP/1.1 server which implements much of the proposed standard.  The server
is functional (less web3.input at the time of this writing), but differs from PEP 444 in several ways.  It
also adds several features I feel should be part of the spec.
> 
> Source for the server is available on GitHub:
> 
> 	https://github.com/pulp/marrow.server.http
> 
> I have made several notes about the PEP 444 specification during implementation of the above, and concern
over some implementation details:
> 
> First, async is poorly defined:
> 
> > If the origin server advertises that it has the web3.async capability, a Web3 application callable used
by the server is permitted to return a callable that accepts no arguments. When it does so, this callable is
to be called periodically by the origin server until it returns a non-None response, which must be a normal
Web3 response tuple.
> 
> Polling is not true async.  I believe that it should be up to the server to define how async is utilized, and
that the specification should be clarified on this point.  (“Called periodically” is too vague.)
(Continue reading)

Permalink | Reply | 12 comments |
headers
Alice Bevan-McGregor | 21 Nov 2010 18:32
Gravatar

Re: PEP 444

> PEP 444 has no champion currently.  Both Armin and I have basically left it behind.  It would be great if you
wanted to be its champion.

Done.

As I already have a functional, performant HTTP server[1] and example filter[2] (compression) utilizing
a slightly modified version of PEP 444, and hope to be giving a presentation on its design and related
utilities[3] early next year, I’d love to have the opportunity to directly shape its future.  My server
may be a bit large to be a reference implementation, but until it has its first user I have the benefit of
being able to experiment whole-heartedly with features and proposals.

Since Python 3 was released I haven’t heard of much forward-progress in getting web frameworks
compatible.  The largest complaint I’ve heard is that there are too few things already ported, which is a
chicken and the egg problem.  This is one scenario where re-inventing the wheel may be the only way to see
forward movement.  So far, I seem to be buckling down and Getting Things Done™ in this regard.

How would I go about getting access to the PEP in order to fix the issues I’ve been catching up on?  (I’ve
been reading through quite a bit of old mailing list traffic these last few hours in-between writing docs
and unit tests for the compression egress filter.)

Now I’m even more excited.  I’ll make a separate post to confirm and get some input on the issues I’ve
encountered thus far.

	— Alice.

[1] https://github.com/pulp/marrow.server.http
[2] https://github.com/pulp/marrow.wsgi.egress.compression — full documentation included
[3] http://web-core.org/marrow/confoo/ — input welcome; the deadline for modification is the 26th
_______________________________________________
Web-SIG mailing list
(Continue reading)

Permalink | Reply | 11 comments |
headers
Chris McDonough | 22 Nov 2010 08:40
Favicon
Gravatar

Re: PEP 444

On Sun, 2010-11-21 at 09:32 -0800, Alice Bevan-McGregor wrote:
> > PEP 444 has no champion currently.  Both Armin and I have basically left it behind.  It would be great if you
wanted to be its champion.
> 
> Done.
> 
> As I already have a functional, performant HTTP server[1] and example filter[2] (compression)
utilizing a slightly modified version of PEP 444, and hope to be giving a presentation on its design and
related utilities[3] early next year, I’d love to have the opportunity to directly shape its future.  My
server may be a bit large to be a reference implementation, but until it has its first user I have the benefit
of being able to experiment whole-heartedly with features and proposals.
> 
> Since Python 3 was released I haven’t heard of much forward-progress in getting web frameworks
compatible.  The largest complaint I’ve heard is that there are too few things already ported, which is a
chicken and the egg problem.  This is one scenario where re-inventing the wheel may be the only way to see
forward movement.  So far, I seem to be buckling down and Getting Things Done™ in this regard.
> 
> How would I go about getting access to the PEP in order to fix the issues I’ve been catching up on?  (I’ve
been reading through quite a bit of old mailing list traffic these last few hours in-between writing docs
and unit tests for the compression egress filter.)

Georg Brandl has thus far been updating the canonical PEP on python.org.
I don't know how you get access to that.  My working copy is at
https://github.com/mcdonc/web3 .

- C

_______________________________________________
Web-SIG mailing list
Web-SIG <at> python.org
(Continue reading)

Permalink | Reply | 10 comments |
headers
Alice Bevan-McGregor | 22 Nov 2010 09:08
Gravatar

Re: PEP 444

Would you prefer to give me collaboration permissions on your repo, or should I fork it?

This message was sent from a mobile device. Please excuse any terseness and spelling or grammatical errors. If additional information is indicated it will be sent from a desktop computer as soon as possible. Thank you.

On 2010-11-21, at 11:40 PM, Chris McDonough <chrism-ccARneWBNkgAvxtiuMwx3w@public.gmane.org> wrote:

Georg Brandl has thus far been updating the canonical PEP on python.org.
I don't know how you get access to that.  My working copy is at
https://github.com/mcdonc/web3 . 
_______________________________________________
Web-SIG mailing list
Web-SIG@...
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org
Permalink | Reply | 8 comments |
headers
Chris McDonough | 22 Nov 2010 09:12
Favicon
Gravatar

Re: PEP 444

On Mon, 2010-11-22 at 00:08 -0800, Alice Bevan-McGregor wrote:
> Would you prefer to give me collaboration permissions on your repo, or
> should I fork it?

Please fork it or create another repository entirely. I have no plans to
do more work on it personally, so I don't think it should really be
associated with me.  To that end, I think I'd prefer my name to either
be off the PEP entirely or just listed as a helper or typist or
something. ;-)

- C

> 
> This message was sent from a mobile device. Please excuse any
> terseness and spelling or grammatical errors. If additional
> information is indicated it will be sent from a desktop computer as
> soon as possible. Thank you.
> 
> On 2010-11-21, at 11:40 PM, Chris McDonough <chrism@...> wrote:
> 
> 
> > Georg Brandl has thus far been updating the canonical PEP on
> > python.org.
> > I don't know how you get access to that.  My working copy is at
> > https://github.com/mcdonc/web3 .
> 

_______________________________________________
Web-SIG mailing list
Web-SIG@...
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org
Permalink | Reply | 7 comments |
headers
Alice Bevan-McGregor | 22 Nov 2010 09:18
Gravatar

Re: PEP 444

I’ve forked it, now available at:

	https://github.com/GothAlice/wsgi2

Re-naming it to wsgi2 will be my first order of business during the week, altering your association the
second.  I’ll post change descriptions for discussion as I go.

	— Alice.

On 2010-11-22, at 12:12 AM, Chris McDonough wrote:

>> Would you prefer to give me collaboration permissions on your repo, or
>> should I fork it?
> 
> Please fork it or create another repository entirely. I have no plans to
> do more work on it personally, so I don't think it should really be
> associated with me.  To that end, I think I'd prefer my name to either
> be off the PEP entirely or just listed as a helper or typist or
> something. ;-)

_______________________________________________
Web-SIG mailing list
Web-SIG <at> python.org
Web SIG: http://www.python.org/sigs/web-sig
Unsubscribe: http://mail.python.org/mailman/options/web-sig/gcpw-web-sig%40m.gmane.org
Permalink | Reply | 6 comments |

Gmane