headers
M.-A. Lemburg | 2 Jan 2007 23:41
Favicon

Re: pep-3108.txt

On 2007-01-02 01:02, brett.cannon wrote:
> Author: brett.cannon
> Date: Tue Jan  2 01:02:41 2007
> New Revision: 53204
> 
> Added:
>    peps/trunk/pep-3108.txt   (contents, props changed)
> Modified:
>    peps/trunk/pep-0000.txt
> Log:
> Add PEP 3108: Standard Library Reorganization.
> 
>...
>
> +Open Issues
> +===========
> +
> +Consolidate dependent modules together into a single module or package?
> ...
> +Consolidate certain modules with similar themes together in a package?
> +----------------------------------------------------------------------
> ...

If you do follow this route, please take the chance to place
the whole Python stdlib under a single package. That way we'll
avoid name clashes with existing packages and modules now and
in the future.

Together with absolute imports this also improves the readability
of modules since it becomes immediately clear where the imported code
(Continue reading)

Permalink | Reply | 10 comments |
headers
Brett Cannon | 2 Jan 2007 23:54
Favicon
Gravatar

Re: pep-3108.txt



On 1/2/07, M.-A. Lemburg <mal <at> egenix.com> wrote:
On 2007-01-02 01:02, brett.cannon wrote:
> Author: brett.cannon
> Date: Tue Jan  2 01:02:41 2007
> New Revision: 53204
>
> Added:
>    peps/trunk/pep-3108.txt   (contents, props changed)
> Modified:
>    peps/trunk/pep-0000.txt
> Log:
> Add PEP 3108: Standard Library Reorganization.
>
>...
>
> +Open Issues
> +===========
> +
> +Consolidate dependent modules together into a single module or package?
> ...
> +Consolidate certain modules with similar themes together in a package?
> +----------------------------------------------------------------------
> ...

If you do follow this route, please take the chance to place
the whole Python stdlib under a single package. That way we'll
avoid name clashes with existing packages and modules now and
in the future.

That has been suggested before (including by me) and Guido has always shot it down.  That's why I left it out of this proposal.

Together with absolute imports this also improves the readability
of modules since it becomes immediately clear where the imported code
is coming from.

Note that as side-effect of this it becomes a lot harder to manipulate
PYTHONPATH to trick Python into loading a standard module from a
non-standard location, improving security and robustness of the
Python installations.

> +Packages are often used to group together modules that have a similar
> +theme but do not have any direct relationship or dependency upon each
> +other.  For Python 3.0 obvious groupings could be done since renaming
> +of various modules is already occurring.
> +
> +* collections
> +    + heapq
> +    + Queue
> +    + sets
> +    + UserDist
> +    + UserList
> +    + What to do with UserString?
> +        - Have a package for Python implementations of built-in types
> +          instead of putting the User* modules into 'collections'?
> +* mac
> +    + Various Mac-specific modules.
> +    + Same can be done for other platform-specific code.
> +* Profiling
> +    + cProfile
> +    + profile
> +    + hotshot
> +    + pstats
> +* email
> +    + mailbox
> +    + mhlib
> +* Databases
> +    + anydbm
> +    + dbhash
> +    + dbm
> +    + bsddb
> +    + dumbdbm
> +    + gdbm
> +    + whichdb
> +* Audio
> +    + aifc
> +    + audioop
> +    + chunk
> +    + ossaudiodev
> +    + sunau
> +    + wave
> +    + winsound
> +* Servers
> +    + BaseHTTPServer
> +    + CGIHTTPServer
> +    + DocXMLRPCServer
> +    + SimpleHTTPServer
> +    + SimpleXMLRPCServer
> +    + SocketServer

The package names should probably be converted to lower-case to
follow PEP 8.

Oops, I should have clarified that was not package name suggestsions beyond 'collections'.  It was just meant to act as what the type of grouping was.

-Brett

_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 6 comments |
headers
M.-A. Lemburg | 3 Jan 2007 00:02
Favicon

Re: pep-3108.txt

On 2007-01-02 23:54, Brett Cannon wrote:
> On 1/2/07, M.-A. Lemburg <mal <at> egenix.com> wrote:
>>
>> On 2007-01-02 01:02, brett.cannon wrote:
>> > Author: brett.cannon
>> > Date: Tue Jan  2 01:02:41 2007
>> > New Revision: 53204
>> >
>> > Added:
>> >    peps/trunk/pep-3108.txt   (contents, props changed)
>> > Modified:
>> >    peps/trunk/pep-0000.txt
>> > Log:
>> > Add PEP 3108: Standard Library Reorganization.
>> >
>> >...
>> >
>> > +Open Issues
>> > +===========
>> > +
>> > +Consolidate dependent modules together into a single module or
>> package?
>> > ...
>> > +Consolidate certain modules with similar themes together in a package?
>> > +----------------------------------------------------------------------
>> > ...
>>
>> If you do follow this route, please take the chance to place
>> the whole Python stdlib under a single package. That way we'll
>> avoid name clashes with existing packages and modules now and
>> in the future.
> 
> 
> That has been suggested before (including by me) and Guido has always shot
> it down.  That's why I left it out of this proposal.

Even if it is shot down again, it still deserves to be documented
together with the reasons for being shot down.

This is a one-in-a-lifetime chance, so it would be sad if it were
not taken into account.

The extra effort would be minimal - the renaming would have to be
done using a script anyway and adding an extra 'from py import '
prefix to the modules wouldn't really make the renaming more
complicated ;-)

> Together with absolute imports this also improves the readability
>> of modules since it becomes immediately clear where the imported code
>> is coming from.
>>
>> Note that as side-effect of this it becomes a lot harder to manipulate
>> PYTHONPATH to trick Python into loading a standard module from a
>> non-standard location, improving security and robustness of the
>> Python installations.

--

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 02 2007)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 5 comments |
headers
Brett Cannon | 3 Jan 2007 00:09
Favicon
Gravatar

Re: pep-3108.txt



On 1/2/07, M.-A. Lemburg <mal <at> egenix.com> wrote:
On 2007-01-02 23:54, Brett Cannon wrote:
> On 1/2/07, M.-A. Lemburg <mal <at> egenix.com> wrote:
>>
>> On 2007-01-02 01:02, brett.cannon wrote:
>> > Author: brett.cannon
>> > Date: Tue Jan  2 01:02:41 2007
>> > New Revision: 53204
>> >
>> > Added:
>> >    peps/trunk/pep-3108.txt   (contents, props changed)
>> > Modified:
>> >    peps/trunk/pep-0000.txt
>> > Log:
>> > Add PEP 3108: Standard Library Reorganization.
>> >
>> >...
>> >
>> > +Open Issues
>> > +===========
>> > +
>> > +Consolidate dependent modules together into a single module or
>> package?
>> > ...
>> > +Consolidate certain modules with similar themes together in a package?
>> > +----------------------------------------------------------------------
>> > ...
>>
>> If you do follow this route, please take the chance to place
>> the whole Python stdlib under a single package. That way we'll
>> avoid name clashes with existing packages and modules now and
>> in the future.
>
>
> That has been suggested before (including by me) and Guido has always shot
> it down.  That's why I left it out of this proposal.

Even if it is shot down again, it still deserves to be documented
together with the reasons for being shot down.

Aw, but that means I have to go find why Guido didn't like it.  =)  But yes, it should be either an open issue or rejected idea.

This is a one-in-a-lifetime chance, so it would be sad if it were
not taken into account.

The extra effort would be minimal - the renaming would have to be
done using a script anyway and adding an extra 'from py import '
prefix to the modules wouldn't really make the renaming more
complicated ;-)

=)

> Together with absolute imports this also improves the readability
>> of modules since it becomes immediately clear where the imported code
>> is coming from.
>>
>> Note that as side-effect of this it becomes a lot harder to manipulate
>> PYTHONPATH to trick Python into loading a standard module from a
>> non-standard location, improving security and robustness of the
>> Python installations.

Good point.  Could actually have the py namespace be special and use a separate path list ( e.g., sys.library_path) for that specific namespace.

-Brett 

_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 0 comments |
headers
Barry Warsaw | 3 Jan 2007 00:35
Favicon

Re: pep-3108.txt


On Jan 2, 2007, at 5:41 PM, M.-A. Lemburg wrote:

> Note that as side-effect of this it becomes a lot harder to manipulate
> PYTHONPATH to trick Python into loading a standard module from a
> non-standard location, improving security and robustness of the
> Python installations.

Sometimes though you want to do this, as when you want your  
application to ensure it gets a particular version of a standard  
library module, regardless of the version of Python being used.  And  
now we're back to application-specific site-packages ;).

-Barry
Permalink | Reply | 2 comments |
headers
Brett Cannon | 3 Jan 2007 01:42
Favicon
Gravatar

Re: pep-3108.txt



On 1/2/07, M.-A. Lemburg <mal <at> egenix.com> wrote:
On 2007-01-02 23:54, Brett Cannon wrote:
> On 1/2/07, M.-A. Lemburg <mal <at> egenix.com> wrote:
>>
>> On 2007-01-02 01:02, brett.cannon wrote:
>> > Author: brett.cannon
>> > Date: Tue Jan  2 01:02:41 2007
>> > New Revision: 53204
>> >
>> > Added:
>> >    peps/trunk/pep-3108.txt   (contents, props changed)
>> > Modified:
>> >    peps/trunk/pep-0000.txt
>> > Log:
>> > Add PEP 3108: Standard Library Reorganization.
>> >
>> >...
>> >
>> > +Open Issues
>> > +===========
>> > +
>> > +Consolidate dependent modules together into a single module or
>> package?
>> > ...
>> > +Consolidate certain modules with similar themes together in a package?
>> > +----------------------------------------------------------------------
>> > ...
>>
>> If you do follow this route, please take the chance to place
>> the whole Python stdlib under a single package. That way we'll
>> avoid name clashes with existing packages and modules now and
>> in the future.
>
>
> That has been suggested before (including by me) and Guido has always shot
> it down.  That's why I left it out of this proposal.

Even if it is shot down again, it still deserves to be documented
together with the reasons for being shot down.

This is a one-in-a-lifetime chance, so it would be sad if it were
not taken into account.

The extra effort would be minimal - the renaming would have to be
done using a script anyway and adding an extra 'from py import '
prefix to the modules wouldn't really make the renaming more
complicated ;-)

I was about to start writing an open issue on this since the biggest objection from Guido I could find on this topic is http://mail.python.org/pipermail/python-dev/2002-July/026409.html , but then it started to feel like a separate PEP to me.  So I think I am going to pass on taking on this topic and let someone else tackle it in a PEP.  Sorry, MAL, but I need to worry about my sanity on this one.  =)

-Brett
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 3 comments |
headers
dustin | 3 Jan 2007 06:27
Picon
Gravatar

smarter temporary file object (SF #415692)

I came across a complaint that PEP 0042 had become a graveyard of
neglected ideas, and decided to have a look through and implement
something.  Creating a smarter temporary file object seemed simple
enough.

Oddly, even after GvR re-opened it, I can't post an attachment to that
tracker item (it's under "Feature Requests" -- does it need to get moved
to "Patches" first?), but the implementation is short, so it's included
below.  This is intended to be appended to Lib/tempfile.py (and thus
assumes that module's globals are present).

I would appreciate it if the gurus of python-dev could take a peek and
let me know if this is unsuitable or incorrect for any reason.  It's not
the most straightforward implementatio -- I used the optimization
techniques I found in TemporaryFile.

If this looks good, I'll prepare a patch against trunk, including an
additional chunk of documentation and a unit test.

Dustin

                     -----cut-here-----

try:
  from cStringIO import StringIO
except:
  from StringIO import StringIO

class SpooledTemporaryFile:
    """Temporary file wrapper, specialized to switch from
    StringIO to a real file when it exceeds a certain size or
    when a fileno is needed.
    """
    _rolled = False

    def __init__(self, max_size=0, mode='w+b', bufsize=-1,
                 suffix="", prefix=template, dir=None):
        self._file = StringIO()
        self._max_size = max_size
        self._TemporaryFileArgs = (mode, bufsize, suffix, prefix, dir)

    def _check(self, file):
        if self._rolled: return
        if file.tell() > self.__dict__['_max_size']:
            self._rollover(file)

    def _rollover(self, file):
        args = self.__dict__['_TemporaryFileArgs']
        self.__dict__.clear() # clear attributes cached by __getattr__
        newfile = self._file = TemporaryFile(*args)
        newfile.write(file.getvalue())
        newfile.seek(file.tell(), 0)
        self._rolled = True

        # replace patched functions with the new file's methods
        self.write = newfile.write
        self.writelines = newfile.writelines
        self.fileno = newfile.fileno

    def write(self, s):
        file = self.__dict__['_file']
        rv = file.write(s)
        self._check(file)
        return rv

    def writelines(self, iterable):
        file = self.__dict__['_file']
        rv = file.writelines(iterable)
        self._check(file)
        return rv

    def fileno(self):
        self._rollover(self.__dict__['_file'])
        return self.fileno()

    def __getattr__(self, name):
        file = self.__dict__['_file']
        a = getattr(file, name)
        if type(a) != type(0):
            setattr(self, name, a)
        return a
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 2 comments |
headers
Neal Norwitz | 3 Jan 2007 07:07
Picon

Re: smarter temporary file object (SF #415692)

On 1/2/07, dustin <at> v.igoro.us <dustin <at> v.igoro.us> wrote:
> I came across a complaint that PEP 0042 had become a graveyard of
> neglected ideas, and decided to have a look through and implement
> something.  Creating a smarter temporary file object seemed simple
> enough.
>
> Oddly, even after GvR re-opened it, I can't post an attachment to that
> tracker item

Hi Dustin.

Thanks for your patch!

Unfortunately, SF doesn't allow anyone except developers and the
tracker item owner to submit attachments.  The best thing to do would
be to create a new patch.  Then you can make a comment in the old RFE
that references your new patch.  Something like this is sufficient:

  Patch available here:  http://python.org/sf/NEW_PATCH_NUMBER

Cheers,
n
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 1 comment |
headers
M.-A. Lemburg | 3 Jan 2007 12:07
Favicon

Re: pep-3108.txt

On 2007-01-03 00:35, Barry Warsaw wrote:
> On Jan 2, 2007, at 5:41 PM, M.-A. Lemburg wrote:
> 
>> Note that as side-effect of this it becomes a lot harder to manipulate
>> PYTHONPATH to trick Python into loading a standard module from a
>> non-standard location, improving security and robustness of the
>> Python installations.
> 
> Sometimes though you want to do this, as when you want your application
> to ensure it gets a particular version of a standard library module,
> regardless of the version of Python being used.  And now we're back to
> application-specific site-packages ;).

Well, I guess that's a rather particular use case and can probably
only be safely implemented by the maintainer of the module or package
in question ;-)

In such (rare) cases, it should be possible to use one of the harder
ways to achieve this:

 * monkey patching the package
 * using package.__path__ to redirect the in-package search
 * creating a private copy of the whole package which then has
   the modified modules and packages in place

Regarding application specific package setups:

In my experience it's better to have an application specific
sys.path setup function that manages this, rather than trying
to manipulate PYTHONPATH or trying to tweak Python's stdlib
site.py into using some particular way of setting up application
specific paths which then makes interop harder for all
applications using Python, rather than just the few that
require such setups.

The application can then call this path setup function early
on in the startup phase to make sure that the rest of the
startup and the application's main code then imports the
right modules and packages.

--

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 03 2007)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
_______________________________________________
Python-Dev mailing list
Python-Dev <at> python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org
Permalink | Reply | 1 comment |
headers
Barry Warsaw | 3 Jan 2007 14:12
Favicon

Re: pep-3108.txt


On Jan 3, 2007, at 6:07 AM, M.-A. Lemburg wrote:

> Regarding application specific package setups:
>
> In my experience it's better to have an application specific
> sys.path setup function that manages this, rather than trying
> to manipulate PYTHONPATH or trying to tweak Python's stdlib
> site.py into using some particular way of setting up application
> specific paths which then makes interop harder for all
> applications using Python, rather than just the few that
> require such setups.
>
> The application can then call this path setup function early
> on in the startup phase to make sure that the rest of the
> startup and the application's main code then imports the
> right modules and packages.

Oh, I totally agree MAL.  It's what I've done in Mailman for ages.   
What makes it more complicated is when you have dozens of entry  
points (read: command line scripts), but it's solvable.

I guess when I read "PYTHONPATH" I also read "sys.path", so now that  
I've slept a little, never mind!

-Barry
Permalink | Reply | 0 comments |

Gmane