s.krah | 1 Mar 20:45 2015

Wg: Re: [Python-checkins] cpython (3.4): Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer



== == == == == == Weitergeleitete Nachricht == == == == == ==
Absender : Stefan Krah<stefan <at> bytereef.org>
Empfänger : "Victor Stinner"<victor.stinner <at> gmail.com>
Datum : So, 01 Mrz 2015 18:58:43 +0000
Betreff : Re: [Python-Dev] [Python-checkins] cpython (3.4): Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer
== == == == == == Weitergeleitete Nachricht == == == == == ==
On Mon, Feb 16, 2015 at 10:14:59PM +0100, Victor Stinner wrote:
> 2015-02-16 17:34 GMT+01:00 Stefan Krah <stefan <at> bytereef.org>:
> >
> > On Mon, Feb 16, 2015 at 11:35:52AM +0000, serhiy.storchaka wrote:
> >> diff --git a/Modules/_testbuffer.c b/Modules/_testbuffer.c
> >> --- a/Modules/_testbuffer.c
> >> +++ b/Modules/_testbuffer.c
> >> <at> <at> -850,7 +850,7 <at> <at>
> >> Py_ssize_t *dest;
> >> Py_ssize_t x, i;
> >>
> >> - dest = PyMem_Malloc(len * (sizeof *dest));
> >> + dest = PyMem_New(Py_ssize_t, len);
> >> if (dest == NULL) {
> >> PyErr_NoMemory();
> >> return NULL;
> >
> > This, too, was already protected by len == ndim <= 64.
>
> I don't understand why you don't want to use PyMem_New() even if it
> cannot overflow. PyMem_New() is more readable no?

It's readable, but I don't see a reason to change code that already has an
overflow analysis, especially in 3.4.

As I wrote in http://bugs.python.org/issue23446#msg235770 , people need to
see that one *can* make certain assumptions about PEP-3118 buffers (otherwise
one would go insane with overflow checks when doing arithmetic on the buffers.


So, in a sense, this commit removes information for the reader. I would
prefer an "assert(len <= 64)" for documentation purposes while keeping the
multiplication.



Stefan Krah




<div><div>
<br><div class="zmail_extra">
<div>
<br> == == == == == == Weitergeleitete Nachricht == == == == == == <br>Absender : Stefan Krah&lt;stefan <at> bytereef.org&gt;<br>Empf&auml;nger : "Victor Stinner"&lt;victor.stinner <at> gmail.com&gt;<br>Datum : So, 01 Mrz 2015 18:58:43 +0000<br>Betreff : Re: [Python-Dev] [Python-checkins] cpython (3.4): Issue #23446: Use PyMem_New instead of PyMem_Malloc to avoid possible integer<br> == == == == == == Weitergeleitete Nachricht == == == == == == <br>
</div>
<blockquote>On Mon, Feb 16, 2015 at 10:14:59PM +0100, Victor Stinner wrote: <br>&gt; 2015-02-16 17:34 GMT+01:00 Stefan Krah &lt;<a href="mailto:stefan <at> bytereef.org" target="_blank">stefan <at> bytereef.org</a>&gt;: <br>&gt; &gt; <br>&gt; &gt; On Mon, Feb 16, 2015 at 11:35:52AM +0000, serhiy.storchaka wrote: <br>&gt; &gt;&gt; diff --git a/Modules/_testbuffer.c b/Modules/_testbuffer.c <br>&gt; &gt;&gt; --- a/Modules/_testbuffer.c <br>&gt; &gt;&gt; +++ b/Modules/_testbuffer.c <br>&gt; &gt;&gt;  <at>  <at>  -850,7 +850,7  <at>  <at>  <br>&gt; &gt;&gt;      Py_ssize_t *dest; <br>&gt; &gt;&gt;      Py_ssize_t x, i; <br>&gt; &gt;&gt; <br>&gt; &gt;&gt; -    dest = PyMem_Malloc(len * (sizeof *dest)); <br>&gt; &gt;&gt; +    dest = PyMem_New(Py_ssize_t, len); <br>&gt; &gt;&gt;      if (dest == NULL) { <br>&gt; &gt;&gt;          PyErr_NoMemory(); <br>&gt; &gt;&gt;          return NULL; <br>&gt; &gt; <br>&gt; &gt; This, too, was already protected by len == ndim &lt;= 64. <br>&gt;  <br>&gt; I don't understand why you don't want to use PyMem_New() even if it <br>&gt; cannot overflow. PyMem_New() is more readable no? <br><br>It's readable, but I don't see a reason to change code that already has an <br>overflow analysis, especially in 3.4. <br><br>As I wrote in <a href="http://bugs.python.org/issue23446#msg235770" target="_blank">http://bugs.python.org/issue23446#msg235770</a> , people need to <br>see that one *can* make certain assumptions about PEP-3118 buffers (otherwise <br>one would go insane with overflow checks when doing arithmetic on the buffers. <br><br><br>So, in a sense, this commit removes information for the reader.  I would <br>prefer an "assert(len &lt;= 64)" for documentation purposes while keeping the <br>multiplication. <br><br><br><br>Stefan Krah <br><br><br>
</blockquote>
<br>
</div>
<br>
</div></div>
Guido van Rossum | 27 Feb 20:38 2015

PEP 485 review (isclose())

I think it's time to accept PEP 485. I've re-read it once more, and it looks like the text is in great shape. (My only recommendation would be to update the Abstract to state that we're specifically adding math.isclose().)

A wording question: "This implementation has a flag that lets the user select which relative tolerance test to apply -- this PEP does not suggest that that be retained, but rather than the weak test be selected." -- I think this was meant to say "... rather *that* the weak test be selected", right? (It would be nice if the sample implementation defaulted to the choice in the PEP.)

However, those are just minor edits, and I hereby approve the PEP. Thanks Chris and everyone else for the fruitful discussion (and thanks especially to Chris for eventually ending the bikeshedding and writing a PEP that explains each of the choices). Congrats!

--
--Guido van Rossum (python.org/~guido)
<div><div dir="ltr">
<div>
<div>I think it's time to accept PEP 485. I've re-read it once more, and it looks like the text is in great shape. (My only recommendation would be to update the Abstract to state that we're specifically adding math.isclose().)<br><br>
</div>A wording question: "This implementation has a flag that lets the user select which relative tolerance test to apply -- this PEP does not suggest that that be retained, but rather than the weak test be selected." -- I think this was meant to say "... rather *that* the weak test be selected", right? (It would be nice if the sample implementation defaulted to the choice in the PEP.)<br><br>
</div>However, those are just minor edits, and I hereby approve the PEP. Thanks Chris and everyone else for the fruitful discussion (and thanks especially to Chris for eventually ending the bikeshedding and writing a PEP that explains each of the choices). Congrats!<br><div><div><div>
<br>-- <br><div class="gmail_signature">--Guido van Rossum (<a href="http://python.org/~guido">python.org/~guido</a>)</div>
</div></div></div>
</div></div>
Python tracker | 27 Feb 18:08 2015

Summary of Python tracker Issues


ACTIVITY SUMMARY (2015-02-20 - 2015-02-27)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4788 (+18)
  closed 30510 (+26)
  total  35298 (+44)

Open issues with patches: 2245 

Issues opened (34)
==================

#23458: [2.7] random: make the file descriptor non-inheritable (on POS
http://bugs.python.org/issue23458  reopened by haypo

#23494: adding timedelta to datetime object is not timezone aware
http://bugs.python.org/issue23494  opened by gilsho

#23495: The writer.writerows method should be documented as accepting 
http://bugs.python.org/issue23495  opened by Steven.Barker

#23496: Steps for Android Native Build of Python 3.4.2
http://bugs.python.org/issue23496  opened by chaselton

#23498: Expose http.cookiejar.split_header_words()
http://bugs.python.org/issue23498  opened by vadmium

#23500: Argument Clinic: multiple macro definition
http://bugs.python.org/issue23500  opened by serhiy.storchaka

#23501: Argument Clinic: generate code into separate files by default
http://bugs.python.org/issue23501  opened by serhiy.storchaka

#23502: pprint: added support for mapping proxy
http://bugs.python.org/issue23502  opened by serhiy.storchaka

#23503: undefined behavior in Objects/obmalloc.c
http://bugs.python.org/issue23503  opened by dwight.guth

#23504: Add __all__ into types
http://bugs.python.org/issue23504  opened by serhiy.storchaka

#23505: Urlparse insufficient validation leads to open redirect
http://bugs.python.org/issue23505  opened by yaaboukir

#23507: Tuple creation is too slow
http://bugs.python.org/issue23507  opened by serhiy.storchaka

#23508: Document & unittest the subprocess.getstatusoutput() status va
http://bugs.python.org/issue23508  opened by gregory.p.smith

#23509: Speed up Counter operators
http://bugs.python.org/issue23509  opened by joern

#23510: multiprocessing bug SyncManager and 'with'
http://bugs.python.org/issue23510  opened by Cezary.Wagner

#23512: List of builtins is not alphabetical on https://docs.python.or
http://bugs.python.org/issue23512  opened by edsouza

#23513: Add support for classes/object model in multiprocessing/pickle
http://bugs.python.org/issue23513  opened by Cezary.Wagner

#23514: multiprocessing documentation - little more examples for paral
http://bugs.python.org/issue23514  opened by Cezary.Wagner

#23516: pip: urllib3 does not encode userinfo section of URL with auth
http://bugs.python.org/issue23516  opened by leotan

#23517: datetime.utcfromtimestamp parses timestamps incorrectly
http://bugs.python.org/issue23517  opened by tbarbugli

#23520: test_os failed (python-3.4.3)
http://bugs.python.org/issue23520  opened by avoevodkin

#23521: OverflowError from timedelta * float in datetime.py
http://bugs.python.org/issue23521  opened by belopolsky

#23522: Misleading note in Statistics module documentation
http://bugs.python.org/issue23522  opened by Journeyman08

#23524: Use _set_thread_local_invalid_parameter_handler in posixmodule
http://bugs.python.org/issue23524  opened by steve.dower

#23525: isbuiltin, isroutine, etc.
http://bugs.python.org/issue23525  opened by abarnert

#23526: Silence resource warnings in test_httplib
http://bugs.python.org/issue23526  opened by ashkop

#23527: test_smtpnet uses incorrect port for STARTTLS
http://bugs.python.org/issue23527  opened by ashkop

#23528: Limit decompressed data when reading from GzipFile
http://bugs.python.org/issue23528  opened by vadmium

#23529: Limit decompressed data when reading from LZMAFile and BZ2File
http://bugs.python.org/issue23529  opened by vadmium

#23530: os and multiprocessing.cpu_count do not respect cpuset/affinit
http://bugs.python.org/issue23530  opened by jtaylor

#23531: SSL operations cause entire process to hang
http://bugs.python.org/issue23531  opened by johnkw

#23532: add example of 'first match wins' to regex "|"  documentation?
http://bugs.python.org/issue23532  opened by Rick Otten

#23534: `test_longdouble` fails on Mac when using system libffi (versi
http://bugs.python.org/issue23534  opened by Maxime Belanger

#23536: Add explicit information on config file format not supporting 
http://bugs.python.org/issue23536  opened by piotr.dobrogost

Most recent 15 issues with no replies (15)
==========================================

#23536: Add explicit information on config file format not supporting 
http://bugs.python.org/issue23536

#23527: test_smtpnet uses incorrect port for STARTTLS
http://bugs.python.org/issue23527

#23525: isbuiltin, isroutine, etc.
http://bugs.python.org/issue23525

#23522: Misleading note in Statistics module documentation
http://bugs.python.org/issue23522

#23520: test_os failed (python-3.4.3)
http://bugs.python.org/issue23520

#23512: List of builtins is not alphabetical on https://docs.python.or
http://bugs.python.org/issue23512

#23504: Add __all__ into types
http://bugs.python.org/issue23504

#23503: undefined behavior in Objects/obmalloc.c
http://bugs.python.org/issue23503

#23502: pprint: added support for mapping proxy
http://bugs.python.org/issue23502

#23501: Argument Clinic: generate code into separate files by default
http://bugs.python.org/issue23501

#23498: Expose http.cookiejar.split_header_words()
http://bugs.python.org/issue23498

#23487: argparse: add_subparsers 'action' broken
http://bugs.python.org/issue23487

#23485: PEP 475: handle EINTR in the select and selectors module
http://bugs.python.org/issue23485

#23470: OpenBSD buildbot uses wrong stdlib
http://bugs.python.org/issue23470

#23469: Delete Misc/*.wpr files
http://bugs.python.org/issue23469

Most recent 15 issues waiting for review (15)
=============================================

#23529: Limit decompressed data when reading from LZMAFile and BZ2File
http://bugs.python.org/issue23529

#23528: Limit decompressed data when reading from GzipFile
http://bugs.python.org/issue23528

#23527: test_smtpnet uses incorrect port for STARTTLS
http://bugs.python.org/issue23527

#23526: Silence resource warnings in test_httplib
http://bugs.python.org/issue23526

#23524: Use _set_thread_local_invalid_parameter_handler in posixmodule
http://bugs.python.org/issue23524

#23521: OverflowError from timedelta * float in datetime.py
http://bugs.python.org/issue23521

#23509: Speed up Counter operators
http://bugs.python.org/issue23509

#23507: Tuple creation is too slow
http://bugs.python.org/issue23507

#23504: Add __all__ into types
http://bugs.python.org/issue23504

#23502: pprint: added support for mapping proxy
http://bugs.python.org/issue23502

#23501: Argument Clinic: generate code into separate files by default
http://bugs.python.org/issue23501

#23496: Steps for Android Native Build of Python 3.4.2
http://bugs.python.org/issue23496

#23492: Argument Clinic: improve generated parser for 1-argument funct
http://bugs.python.org/issue23492

#23491: PEP 441 - Improving Python Zip Application Support
http://bugs.python.org/issue23491

#23488: Random objects twice as big as necessary on 64-bit builds
http://bugs.python.org/issue23488

Top 10 most discussed issues (10)
=================================

#23491: PEP 441 - Improving Python Zip Application Support
http://bugs.python.org/issue23491  26 msgs

#23496: Steps for Android Native Build of Python 3.4.2
http://bugs.python.org/issue23496  24 msgs

#23521: OverflowError from timedelta * float in datetime.py
http://bugs.python.org/issue23521  17 msgs

#23524: Use _set_thread_local_invalid_parameter_handler in posixmodule
http://bugs.python.org/issue23524  14 msgs

#23493: optimize sort_keys in json module by using operator.itemgetter
http://bugs.python.org/issue23493  12 msgs

#23517: datetime.utcfromtimestamp parses timestamps incorrectly
http://bugs.python.org/issue23517  12 msgs

#23492: Argument Clinic: improve generated parser for 1-argument funct
http://bugs.python.org/issue23492   8 msgs

#20323: Argument Clinic: docstring_prototype output causes build failu
http://bugs.python.org/issue20323   7 msgs

#23314: Disabling CRT asserts in debug build
http://bugs.python.org/issue23314   7 msgs

#23458: [2.7] random: make the file descriptor non-inheritable (on POS
http://bugs.python.org/issue23458   7 msgs

Issues closed (25)
==================

#5700: io.FileIO calls flush() after file closed
http://bugs.python.org/issue5700  closed by serhiy.storchaka

#6639: turtle: _tkinter.TclError: invalid command name ".10170160"
http://bugs.python.org/issue6639  closed by serhiy.storchaka

#15955: bz2, lzma: add option to limit output size
http://bugs.python.org/issue15955  closed by pitrou

#20780: Shadowed (duplicate name but different body) test in test_stat
http://bugs.python.org/issue20780  closed by benjamin.peterson

#21934: OpenBSD has no /dev/full device
http://bugs.python.org/issue21934  closed by serhiy.storchaka

#22113: memoryview and struct.pack_into
http://bugs.python.org/issue22113  closed by serhiy.storchaka

#22435: socketserver.TCPSocket leaks socket to garbage collector if se
http://bugs.python.org/issue22435  closed by berker.peksag

#23152: fstat64 required on Windows
http://bugs.python.org/issue23152  closed by steve.dower

#23215: MemoryError with custom error handlers and multibyte codecs
http://bugs.python.org/issue23215  closed by serhiy.storchaka

#23245: urllib2: urlopen() gets exception(kwargs bug?)
http://bugs.python.org/issue23245  closed by Douman

#23374: pydoc 3.x raises UnicodeEncodeError on sqlite3 package
http://bugs.python.org/issue23374  closed by serhiy.storchaka

#23476: SSL cert verify fail for "www.verisign.com"
http://bugs.python.org/issue23476  closed by pitrou

#23489: atexit handlers are not executed when using multiprocessing.Po
http://bugs.python.org/issue23489  closed by davin

#23490: allocation (and overwrite) of a 0 byte buffer
http://bugs.python.org/issue23490  closed by serhiy.storchaka

#23497: textwrap.dedent doesn't work properly with embedded strings co
http://bugs.python.org/issue23497  closed by ned.deily

#23499: Minor grammar issue in mimetypes.rst
http://bugs.python.org/issue23499  closed by ned.deily

#23506: Data Structure of Dict not changing its value as expected
http://bugs.python.org/issue23506  closed by zach.ware

#23511: Broken code example in email module documentation
http://bugs.python.org/issue23511  closed by berker.peksag

#23515: Bad logic in timsort's merge_collapse
http://bugs.python.org/issue23515  closed by python-dev

#23518: Misplaced diagnostic caret for some SyntaxErrors
http://bugs.python.org/issue23518  closed by zach.ware

#23519: using asyncio.iscoroutinefunction() on a functools.partial obj
http://bugs.python.org/issue23519  closed by gvanrossum

#23523: cmath.atanh has wrong output
http://bugs.python.org/issue23523  closed by benjamin.peterson

#23533: MacOSX Python child process running urlopen crashed when tkMes
http://bugs.python.org/issue23533  closed by ned.deily

#23535: os.path.join() wrong concatenation of "C:" on Windows
http://bugs.python.org/issue23535  closed by eric.smith

#23537: BaseSubprocessTransport includes two unused methods
http://bugs.python.org/issue23537  closed by haypo
Paul Moore | 26 Feb 18:05 2015
Picon

Re: Request for Pronouncement: PEP 441 - Improving Python ZIP Application Support

On 24 February 2015 at 18:24, Guido van Rossum <guido <at> python.org> wrote:
> Here's my review. I really like where this is going but I have a few
> questions and suggestions (I can't help myself :-).

OK, I've updated both the PEP and the patch based on follow-up
discussions. I think (again!) it is ready to go.

I've included the updated PEP inline below, it'll be available at
peps.python.org as soon as someone has a chance to upload it.

Thanks to everyone for the various comments. If I've missed anything
that someone thinks I'd said I would change, please let me know.

Paul

PEP: 441
Title: Improving Python ZIP Application Support
Version: $Revision$
Last-Modified: $Date$
Author: Daniel Holth <dholth <at> gmail.com>,
        Paul Moore <p.f.moore <at> gmail.com>
Discussions-To:
https://mail.python.org/pipermail/python-dev/2015-February/138277.html
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 30 March 2013
Post-History: 30 March 2013, 1 April 2013, 16 February 2015

Improving Python ZIP Application Support
========================================

Python has had the ability to execute directories or ZIP-format
archives as scripts since version 2.6 [1]_.  When invoked with a zip
file or directory as its first argument the interpreter adds that
directory to sys.path and executes the ``__main__`` module.  These
archives provide a great way to publish software that needs to be
distributed as a single file script but is complex enough to need to
be written as a collection of modules.

This feature is not as popular as it should be mainly because it was
not promoted as part of Python 2.6 [2]_, so that it is relatively
unknown, but also because the Windows installer does not register a
file extension (other than ``.py``) for this format of file, to associate
with the launcher.

This PEP proposes to fix these problems by re-publicising the feature,
defining the ``.pyz`` and ``.pyzw`` extensions as "Python ZIP Applications"
and "Windowed Python ZIP Applications", and providing some simple
tooling to manage the format.

A New Python ZIP Application Extension
======================================

The terminology "Python Zip Application" will be the formal term used
for a zip-format archive that contains Python code in a form that can
be directly executed by Python (specifically, it must have a
``__main__.py`` file in the root directory of the archive).  The
extension ``.pyz`` will be formally associated with such files.

The Python 3.5 installer will associate ``.pyz`` and ``.pyzw`` "Python
Zip Applications" with the platform launcher so they can be executed.
A ``.pyz`` archive is a console application and a ``.pyzw`` archive is a
windowed application, indicating whether the console should appear
when running the app.

On Unix, it would be ideal if the ``.pyz`` extension and the name
"Python Zip Application" were registered (in the mime types database?).
However, such an association is out of scope for this PEP.

Python Zip applications can be prefixed with a ``#!`` line
pointing to the correct Python interpreter and an optional
explanation::

    #!/usr/bin/env python3
    #  Python application packed with zipapp module
    (binary contents of archive)

On Unix, this allows the OS to run the file with the correct
interpreter, via the standard "shebang" support.  On Windows, the
Python launcher implements shebang support.

However, it is always possible to execute a ``.pyz`` application by
supplying the filename to the Python interpreter directly.

As background, ZIP archives are defined with a footer containing
relative offsets from the end of the file.  They remain valid when
concatenated to the end of any other file.  This feature is completely
standard and is how self-extracting ZIP archives and the bdist_wininst
installer format work.

Minimal Tooling: The zipapp Module
==================================

This PEP also proposes including a module for working with these
archives.  The module will contain functions for working with Python
zip application archives, and a command line interface (via ``python
-m zipapp``) for their creation and manipulation.

More complete tools for managing Python Zip Applications are
encouraged as 3rd party applications on PyPI.  Currently, pyzzer [5]_
and pex [6]_ are two such tools.

Module Interface
----------------

The zipapp module will provide the following functions:

``create_archive(source, target=None, interpreter=None, main=None)``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Create an application archive from *source*.  The source can be any
of the following:

* The name of a directory, in which case a new application archive
  will be created from the content of that directory.
* The name of an existing application archive file, in which case the
  file is copied to the target.  The file name should include the
  ``.pyz`` extension, if required.
* A file object open for reading in bytes mode.  The content of the
  file should be an application archive, and the file object is
  assumed to be positioned at the start of the archive.

The *target* argument determines where the resulting archive will be
written:

* If it is the name of a file, the archive will be written to that
  file.
* If it is an open file object, the archive will be written to that
  file object, which must be open for writing in bytes mode.
* If the target is omitted (or None), the source must be a directory
  and the target will be a file with the same name as the source, with
  a ``.pyz`` extension added.

The *interpreter* argument specifies the name of the Python
interpreter with which the archive will be executed.  It is written as
a "shebang" line at the start of the archive.  On Unix, this will be
interpreted by the OS, and on Windows it will be handled by the Python
launcher.  Omitting the *interpreter* results in no shebang line being
written.  If an interpreter is specified, and the target is a
filename, the executable bit of the target file will be set.

The *main* argument specifies the name of a callable which will be
used as the main program for the archive.  It can only be specified if
the source is a directory, and the source does not already contain a
``__main__.py`` file.  The *main* argument should take the form
"pkg.module:callable" and the archive will be run by importing
"pkg.module" and executing the given callable with no arguments.  It
is an error to omit *main* if the source is a directory and does not
contain a ``__main__.py`` file, as otherwise the resulting archive
would not be executable.

If a file object is specified for *source* or *target*, it is the
caller's responsibility to close it after calling create_archive.

When copying an existing archive, file objects supplied only need
``read`` and ``readline``, or ``write`` methods.  When creating an
archive from a directory, if the target is a file object it will be
passed to the ``zipfile.ZipFile`` class, and must supply the methods
needed by that class.

``get_interpreter(archive)``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Returns the interpreter specified in the shebang line of the
*archive*.  If there is no shebang, the function returns ``None``.
The *archive* argument can be a filename or a file-like object open
for reading in bytes mode.

Command Line Usage
------------------

The zipapp module can be run with the python ``-m`` flag.  The command
line interface is as follows::

    python -m zipapp directory [options]

        Create an archive from the given directory.  An archive will
        be created from the contents of that directory.  The archive
        will have the same name as the source directory with a .pyz
        extension.

        The following options can be specified:

        -o archive / --output archive

            The destination archive will have the specified name.  The
            given name will be used as written, so should include the
            ".pyz" extension.

        -p interpreter / --python interpreter

            The given interpreter will be written to the shebang line
            of the archive.  If this option is not given, the archive
            will have no shebang line.

        -m pkg.mod:fn / --main pkg.mod:fn

            The source directory must not have a __main__.py file. The
            archiver will write a __main__.py file into the target
            which calls fn from the module pkg.mod.

The behaviour of the command line interface matches that of
``zipapp.create_archive()``.

In addition, it is possible to use the command line interface to work
with an existing archive::

    python -m zipapp app.pyz --show

        Displays the shebang line of an archive.  Output is of the
        form

            Interpreter: /usr/bin/env
        or
            Interpreter: <none>

        and is intended for diagnostic use, not for scripts.

    python -m zipapp app.pyz -o newapp.pyz [-p interpreter]

        Copy app.pyz to newapp.pyz, modifying the shebang line based
        on the -p option (as for creating an archive, no -p option
        means remove the shebang line).  Specifying a destination is
        mandatory.

        In-place modification of an archive is *not* supported, as the
        risk of damaging archives is too great for a simple tool.

As noted, the archives are standard zip files, and so can be unpacked
using any standard ZIP utility or Python's zipfile module.  For this
reason, no interfaces to list the contents of an archive, or unpack
them, are provided or needed.

FAQ
---

Are you sure a standard ZIP utility can handle ``#!`` at the beginning?
    Absolutely.  The zipfile specification allows for arbitrary data to
    be prepended to a zipfile.  This feature is commonly used by
    "self-extracting zip" programs.  If your archive program can't
    handle this, it is a bug in your archive program.

Isn't zipapp just a very thin wrapper over the zipfile module?
    Yes.  If you prefer to build your own Python zip application
    archives using other tools, they will work just as well.  The
    zipapp module is a convenience, nothing more.

Why not use just use a ``.zip`` or ``.py`` extension?
    Users expect a ``.zip`` file to be opened with an archive tool, and
    expect a ``.py`` file to contain readable text.  Both would be
    confusing for this use case.

How does this compete with existing package formats?
    The sdist, bdist and wheel formats are designed for packaging of
    modules to be installed into an existing Python installation.
    They are not intended to be used without installing.  The
    executable zip format is specifically designed for standalone use,
    without needing to be installed.  They are in effect a multi-file
    version of a standalone Python script.

Rejected Proposals
==================

Convenience Values for Shebang Lines
------------------------------------

Is it worth having "convenience" forms for any of the common
interpreter values? For example, ``-p 3`` meaning the same as ``-p
"/usr/bin/env python3"``.  It would save a lot of typing for the
common cases, as well as giving cross-platform options for people who
don't want or need to understand the intricacies of shebang handling
on "other" platforms.

Downsides are that it's not obvious how to translate the
abbreviations.  For example, should "3" mean "/usr/bin/env python3",
"/usr/bin/python3", "python3", or something else?  Also, there is no
obvious short form for the key case of "/usr/bin/env python" (any
available version of Python), which could easily result in scripts
being written with overly-restrictive shebang lines.

Overall, this seems like there are more problems than benefits, and as
a result has been dropped from consideration.

Registering ``.pyz`` as a Media Type
------------------------------------

It was suggested [3]_ that the ``.pyz`` extension should be registered
in the Unix database of extensions.  While it makes sense to do this
as an equivalent of the Windows installer registering the extension,
the ``.py`` extension is not listed in the media types database [4]_.
It doesn't seem reasonable to register ``.pyz`` without ``.py``, so
this idea has been omitted from this PEP.  An interested party could
arrange for *both* ``.py`` and ``.pyz`` to be registered at a future
date.

Default Interpreter
-------------------

The initial draft of this PEP proposed using ``/usr/bin/env python``
as the default interpreter.  Unix users have problems with this
behaviour, as the default for the python command on many distributions
is Python 2, and it is felt that this PEP should prefer Python 3 by
default.  However, using a command of ``python3`` can result in
unexpected behaviour for Windows users, where the default behaviour of
the launcher for the command ``python`` is commonly customised by users,
but the behaviour of ``python3`` may not be modified to match.

As a result, the principle "in the face of ambiguity, refuse to guess"
has been invoked, and archives have no shebang line unless explicitly
requested.  On Windows, the archives will still be run (with the
default Python) by the launcher, and on Unix, the archives can be run
by explicitly invoking the desired Python interpreter.

Command Line Tool to Manage Shebang Lines
-----------------------------------------

It is conceivable that users would want to modify the shebang line for
an existing archive, or even just display the current shebang line.
This is tricky to do so with existing tools (zip programs typically
ignore prepended data totally, and text editors can have trouble
editing files containing binary data).

The zipapp module provides functions to handle the shebang line, but
does not include a command line interface to that functionality.  This
is because it is not clear how to provide one without the resulting
interface being over-complex and potentially confusing.  Changing the
shebang line is expected to be an uncommon requirement.

Reference Implementation
========================

A reference implementation is at http://bugs.python.org/issue23491.

References
==========

.. [1] Allow interpreter to execute a zip file
   (http://bugs.python.org/issue1739468)

.. [2] Feature is not documented
   (http://bugs.python.org/issue17359)

.. [3] Discussion of adding a .pyz mime type on python-dev
   (https://mail.python.org/pipermail/python-dev/2015-February/138338.html)

.. [4] Register of media types
   (http://www.iana.org/assignments/media-types/media-types.xhtml)

.. [5] pyzzer - A tool for creating Python-executable archives
   (https://pypi.python.org/pypi/pyzzer)

.. [6] pex - The PEX packaging toolchain
   (https://pypi.python.org/pypi/pex)

The discussion of this PEP took place on the python-dev mailing list,
in the thread starting at
https://mail.python.org/pipermail/python-dev/2015-February/138277.html

Copyright
=========

This document has been placed into the public domain.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:
Guido van Rossum | 25 Feb 20:42 2015

PEP 448 review

I'm back, I've re-read the PEP, and I've re-read the long thread with "(no subject)".

I think Georg Brandl nailed it:

"""
I like the "sequence and dict flattening" part of the PEP, mostly because it
is consistent and should be easy to understand, but the comprehension syntax
enhancements seem to be bad for readability and "comprehending" what the code
does.

The call syntax part is a mixed bag on the one hand it is nice to be
consistent with the extended possibilities in literals (flattening),
but on the other hand there would be small but annoying inconsistencies
anyways (e.g. the duplicate kwarg case above).
"""

Greg Ewing followed up explaining that the inconsistency between dict flattening and call syntax is inherent in the pre-existing different rules for dicts vs. keyword args: {'a':1, 'a':2} results in {'a':2}, while f(a=1, a=2) is an error. (This form is a SyntaxError; the dynamic case f(a=1, **{'a': 1}) is a TypeError.)

For me, allowing f(*a, *b) and f(**d, **e) and all the other combinations for function calls proposed by the PEP is an easy +1 -- it's a straightforward extension of the existing pattern, and anybody who knows what f(x, *a) does will understand f(x, *a, y, *b). Guessing what f(**d, **e) means shouldn't be hard either. Understanding the edge case for duplicate keys with f(**d, **e) is a little harder, but the error messages are pretty clear, and it is not a new edge case.

The sequence and dict flattening syntax proposals are also clean and logical -- we already have *-unpacking on the receiving side, so allowing *x in tuple expressions reads pretty naturally (and the similarity with *a in argument lists certainly helps). From here, having [a, *x, b, *y] is also natural, and then the extension to other displays is natural: {a, *x, b, *y} and {a:1, **d, b:2, **e}. This, too, gets a +1 from me.

So that leaves comprehensions. IIRC, during the development of the patch we realized that f(*x for x in xs) is sufficiently ambiguous that we decided to disallow it -- note that f(x for x in xs) is already somewhat of a special case because an argument can only be a "bare" generator expression if it is the only argument. The same reasoning doesn't apply (in that form) to list, set and dict comprehensions -- while f(x for x in xs) is identical in meaning to f((x for x in xs)), [x for x in xs] is NOT the same as [(x for x in xs)] (that's a list of one element, and the element is a generator expression).

The basic premise of this part of the proposal is that if you have a few iterables, the new proposal (without comprehensions) lets you create a list or generator expression that iterates over all of them, essentially flattening them:

    >>> xs = [1, 2, 3]
    >>> ys = ['abc', 'def']
    >>> zs = [99]
    >>> [*xs, *ys, *zs]
    [1, 2, 3, 'abc', 'def', 99]
    >>>

But now suppose you have a list of iterables:

    >>> xss = [[1, 2, 3], ['abc', 'def'], [99]]
    >>> [*xss[0], *xss[1], *xss[2]]
    [1, 2, 3, 'abc', 'def', 99]
    >>>

Wouldn't it be nice if you could write the latter using a comprehension?

    >>> xss = [[1, 2, 3], ['abc', 'def'], [99]]
    >>> [*xs for xs in xss]
    [1, 2, 3, 'abc', 'def', 99]
    >>>

This is somewhat seductive, and the following is even nicer: the *xs position may be an expression, e.g.:

    >>> xss = [[1, 2, 3], ['abc', 'def'], [99]]
    >>> [*xs[:2] for xs in xss]
    [1, 2, 'abc', 'def', 99]
    >>>

On the other hand, I had to explore the possibilities here by experimenting in the interpreter, and I discovered some odd edge cases (e.g. you can parenthesize the starred expression, but that seems a syntactic accident).

All in all I am personally +0 on the comprehension part of the PEP, and I like that it provides a way to "flatten" a sequence of sequences, but I think very few people in the thread have supported this part. Therefore I would like to ask Neil to update the PEP and the patch to take out the comprehension part, so that the two "easy wins" can make it into Python 3.5 (basically, I am accepting two-thirds of the PEP :-). There is some time yet until alpha 2.

I would also like code reviewers (Benjamin?) to start reviewing the patch, taking into account that the comprehension part needs to be removed.

--
--Guido van Rossum (python.org/~guido)

<div><div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>I'm back, I've re-read the PEP, and I've re-read the long thread with "(no subject)".<br><br>
</div>I think Georg Brandl nailed it:<br><br>"""<br>I like the "sequence and dict flattening" part of the PEP, mostly because it<br>is consistent and should be easy to understand, but the comprehension syntax<br>enhancements seem to be bad for readability and "comprehending" what the code<br>does.<br><br>The call syntax part is a mixed bag on the one hand it is nice to be<br>
consistent with the extended possibilities in literals (flattening),<br>
but on the other hand there would be small but annoying inconsistencies<br>
anyways (e.g. the duplicate kwarg case above).<br>"""<br><br>
</div>Greg Ewing followed up explaining that the inconsistency between dict flattening and call syntax is inherent in the pre-existing different rules for dicts vs. keyword args: {'a':1, 'a':2} results in {'a':2}, while f(a=1, a=2) is an error. (This form is a SyntaxError; the dynamic case f(a=1, **{'a': 1}) is a TypeError.)<br><br>
</div>For me, allowing f(*a, *b) and f(**d, **e) and all the other combinations for function calls proposed by the PEP is an easy +1 -- it's a straightforward extension of the existing pattern, and anybody who knows what f(x, *a) does will understand f(x, *a, y, *b). Guessing what f(**d, **e) means shouldn't be hard either. Understanding the edge case for duplicate keys with f(**d, **e) is a little harder, but the error messages are pretty clear, and it is not a new edge case.<br><br>
</div>The sequence and dict flattening syntax proposals are also clean and logical -- we already have *-unpacking on the receiving side, so allowing *x in tuple expressions reads pretty naturally (and the similarity with *a in argument lists certainly helps). From here, having [a, *x, b, *y] is also natural, and then the extension to other displays is natural: {a, *x, b, *y} and {a:1, **d, b:2, **e}. This, too, gets a +1 from me.<br><br>
</div>So that leaves comprehensions. IIRC, during the development of the patch we realized that f(*x for x in xs) is sufficiently ambiguous that we decided to disallow it -- note that f(x for x in xs) is already somewhat of a special case because an argument can only be a "bare" generator expression if it is the only argument. The same reasoning doesn't apply (in that form) to list, set and dict comprehensions -- while f(x for x in xs) is identical in meaning to f((x for x in xs)), [x for x in xs] is NOT the same as [(x for x in xs)] (that's a list of one element, and the element is a generator expression).<br><br>
</div>The basic premise of this part of the proposal is that if you have a few iterables, the new proposal (without comprehensions) lets you create a list or generator expression that iterates over all of them, essentially flattening them:<br><br>
</div>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; xs = [1, 2, 3]<br>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; ys = ['abc', 'def']<br>
</div>
<div>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; zs = [99]<br>
</div>
<div>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; [*xs, *ys, *zs]<br>&nbsp;&nbsp;&nbsp; [1, 2, 3, 'abc', 'def', 99]<br>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; <br><br>
</div>But now suppose you have a list of iterables:<br><br>
</div>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; xss = [[1, 2, 3], ['abc', 'def'], [99]]<br>
</div>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; [*xss[0], *xss[1], *xss[2]]<br>&nbsp;&nbsp;&nbsp; [1, 2, 3, 'abc', 'def', 99]<br>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; <br><br>
</div>Wouldn't it be nice if you could write the latter using a comprehension?<br><br>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; xss = [[1, 2, 3], ['abc', 'def'], [99]]<br>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; [*xs for xs in xss]<br>&nbsp;&nbsp;&nbsp; [1, 2, 3, 'abc', 'def', 99]<br>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; <div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><div>
<div class="gmail_extra"><br></div>
<div class="gmail_extra">This is somewhat seductive, and the following is even nicer: the *xs position may be an expression, e.g.:<br><br>
</div>
<div class="gmail_extra">&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; xss = [[1, 2, 3], ['abc', 'def'], [99]]<br>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt; [*xs[:2] for xs in xss]<br>&nbsp;&nbsp;&nbsp; [1, 2, 'abc', 'def', 99]<br>&nbsp;&nbsp;&nbsp; &gt;&gt;&gt;<br><br>
</div>
<div class="gmail_extra">On the other hand, I had to explore the possibilities here by experimenting in the interpreter, and I discovered some odd edge cases (e.g. you can parenthesize the starred expression, but that seems a syntactic accident).<br><br>
</div>
<div class="gmail_extra">All in all I am personally +0 on the comprehension part of the PEP, and I like that it provides a way to "flatten" a sequence of sequences, but I think very few people in the thread have supported this part. Therefore I would like to ask Neil to update the PEP and the patch to take out the comprehension part, so that the two "easy wins" can make it into Python 3.5 (basically, I am accepting two-thirds of the PEP :-). There is some time yet until alpha 2.<br><br>
</div>
<div class="gmail_extra">I would also like code reviewers (Benjamin?) to start reviewing the <a href="http://bugs.python.org/issue2292">patch</a>, taking into account that the comprehension part needs to be removed.<br>
</div>
<div class="gmail_extra">
<br>-- <br><div class="gmail_signature">--Guido van Rossum (<a href="http://python.org/~guido">python.org/~guido</a>)<br><br>
</div>
</div>
</div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div></div>
Ryan Gonzalez | 25 Feb 19:17 2015
Picon

Working on issue 23496: should I use a macro test or an edit to configure.ac?

So...

There was a recent discussion here on porting Python to Android. Well, for those of you who saw too many unread messages and marked the whole thread as read like I usually do, I helped Cyd figure out some patches that make it work. Cyd then opened Issue 23496. Now, I'm going to try to redo the patches against HEAD (or tip in Mercurial language).

Which leads me to the question. See, of course, the patches should only be enabled if Python is being built targeting Android, but I'm not sure how that should be detected.

I know that the Android target triple is arm-linux-androideabi. Should I test for the __ANDROID__ macro in the source file, or should I modify configure.ac to detect Android and define its own macro? Also, if a configure.ac edit is preferred, where should it be? It's slightly painful to scan through all 4000 lines of Python's configure script. ;)

--
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23( <at> #Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. - srean
Check out my website: http://kirbyfan64.github.io/
<div><div dir="ltr">So...<div><br></div>
<div>There was a recent discussion here on porting Python to Android. Well, for those of you who saw too many unread messages and marked the whole thread as read like I usually do, I helped Cyd figure out some patches that make it work. Cyd then opened&nbsp;<a href="http://bugs.python.org/issue23496">Issue 23496</a>. Now, I'm going to try to redo the patches against HEAD (or tip in Mercurial language).</div>
<div><br></div>
<div>Which leads me to the question. See, of course, the patches should only be enabled if Python is being built targeting Android, but I'm not sure how that should be detected.</div>
<div><br></div>
<div>I know that the Android target triple is arm-linux-androideabi. Should I test for the __ANDROID__ macro in the source file, or should I modify <a href="http://configure.ac">configure.ac</a> to detect Android and define its own macro? Also, if a <a href="http://configure.ac">configure.ac</a> edit is preferred, where should it be? It's slightly painful to scan through all 4000 lines of Python's configure script. ;)</div>
<div>
<div><br></div>-- <br><div class="gmail_signature"><div dir="ltr">Ryan<div><div>If anybody ever asks me why I prefer C++ to C, my answer will be simple: "It's becauseslejfp23( <at> #Q*(E*EIdc-SEGFAULT. Wait, I don't think that was nul-terminated."</div></div>Personal reality distortion fields are immune to contradictory evidence. - srean<div>Check out my website:&nbsp;<a href="http://kirbyfan64.github.io/" target="_blank">http://kirbyfan64.github.io/</a>
</div>
</div></div>
</div>
</div></div>
Gregory P. Smith | 25 Feb 02:56 2015

how to inspect if something includes a bound first param

inspect.getargspec(method) and inspect.signature(method) both include the 'self' parameter but how are we to figure out from method itself that it is actually bound and that its first parameter is expected to be a bound instance?

So far I've come up with:
 inspect.ismethod(method) or inspect.ismethoddescriptor(method)

But that is still not quite right as it remains False in 3.4 for the simple case of:

>>> class A:
...  def x(self):
...    pass
...
>>> inspect.ismethod(A.x)
False
>>> inspect.ismethoddescriptor(A.x)
False
>>> inspect.signature(A.x).parameters
mappingproxy(OrderedDict([('self', <Parameter at 0x7fdd8188eae8 'self'>)]))
>>> inspect.getargspec(A.x)
ArgSpec(args=['self'], varargs=None, keywords=None, defaults=None)

The above works if use it on object.__init__, but not on a method of an uninstantiated class.  (it also works on the instantiated A().x)

Checking if (inspect.isfunction(method) and method.__qualname__ != method.__name__) perhaps but that seems questionably hacky. Is there a right way to do this via the inspect module that I've missed?

It seems like that'd be nice, but I don't feel like I know enough to write up a feature request for it yet.  (granted I hope nobody wants to write code that does this...)

-gps
<div><div dir="ltr">inspect.getargspec(method) and inspect.signature(method) both include the 'self' parameter but how are we to figure out from method itself that it is actually bound and that its first parameter is expected to be a bound instance?<div><br></div>
<div>So far I've come up with:</div>
<div><span>&nbsp;inspect.ismethod(method) or inspect.ismethoddescriptor(method)</span></div>
<div><br></div>
<div>But that is still not quite right as it remains False in 3.4 for the simple case of:</div>
<div><br></div>
<div>&gt;&gt;&gt; class A:</div>
<div>... &nbsp;def x(self):</div>
<div>... &nbsp; &nbsp;pass</div>
<div>...</div>
<div>&gt;&gt;&gt; inspect.ismethod(A.x)</div>
<div>False</div>
<div>&gt;&gt;&gt; inspect.ismethoddescriptor(A.x)</div>
<div>False</div>
<div>
<div>&gt;&gt;&gt; inspect.signature(A.x).parameters</div>
<div>mappingproxy(OrderedDict([('self', &lt;Parameter at 0x7fdd8188eae8 'self'&gt;)]))</div>
<div>&gt;&gt;&gt; inspect.getargspec(A.x)</div>
<div>ArgSpec(args=['self'], varargs=None, keywords=None, defaults=None)</div>
</div>
<div><br></div>
<div>The above works if use it on object.__init__, but not on a method of an uninstantiated class. &nbsp;(it also works on the instantiated A().x)</div>
<div><br></div>
<div>Checking if (inspect.isfunction(method) and method.__qualname__ != method.__name__)&nbsp;perhaps but that seems questionably hacky. Is there a right way to do this via the inspect module that I've missed?</div>
<div><br></div>
<div>It seems like that'd be nice, but I don't feel like I know enough to write up a feature request for it yet. &nbsp;(granted I hope nobody wants to write code that does this...)</div>
<div><br></div>
<div>-gps</div>
</div></div>
GARRY M CRANE | 25 Feb 02:15 2015
Picon

?s re documentation of Python features

I am trying to learn Python for use in computational biology.  I am using the interesting book: "Computing for Biologists; Python Programming and Principles" (by Ran Libeskind-Hadas and Eliot Bush).  It has an interesting and useful set of programming exercises at www.cs.hmc.edu/CFB.  I am actually enjoying solving (doing) the example problems.  However, I find some of the instructions non-functional for me.  For example the import function does not work,  nor f=open("filename.txt").  I have saved files per instructions in the programming exercise inside the Python34 folder (I am using Python 3.4 in Windows 7).  But use of the f=open() command produces an error message that the requested file does not exist. I assume I have chosen a wrong location for the saved file within that Python34 folder, but trial and error has not led to a successful use of these functions.  import simply leaves a blank line .. no suggestion about the problem.

 
Asking questions in Google and Ask about  where to save Python-related files that can be used subsequently have not led to answers - just details about structuring or formatting things to be written/saved/use of the \n at end of each line, etc.  Important details, but of no help.   I am finding Python to be very handy at many biologic things such as working with DNA strings, etc. but I find the documentation and indexing for finding how to use many Python features exasperating.  I am writing to you based on a READ ME file in my Python folder - generated when I installed Python.
 
FYI, I asked a few questions of one of the authors of the interesting book - who politely replied he was too busy to answer right now - the book and problems were meant for a class ... though neither the book nor problems say so.  The professor hopes to get around to issues of use by non-students sometime - but not now.
 
Another feature I have come across so far that does not work is importation of matplotlib.  I copy computed results (that otherwise would go to your plotting routine)  then go to Excel and with manipulation produce a usable chart there - but at a cost of time and energy.
 
Your Python tool has many intriguing features - but some of the most basic functions do not work for me (even though many features do, e.g., import random does work).  The failure of these features - so far as I can tell - is  because of lack of description (for the general non-expert public) about where/how to install various features.  Perhaps I need to reinstall from the ground up???  If so, just what should I do?  If there is a less drastic solution, can you tell me about it?
 
Thank you for any help ... and if you could provide me a lead regarding WHERE to ask subsequent questions I would be most grateful.  Sometimes, Google or Ask or a U Tube tutorial does a good job - but if one does not know the 'proper' name or term for something, it often is frustrating or impossible to get an answer.  I have not heard about any comprehensive handbook for Python34 aimed at one who wants to use Python for creating programs (functions) that work - and is not an expert at back-room structure of files and folders.... have I simply missed it?  So far, I have not found a local Python expert to ask for help.  I am sure some are in the greater Seattle area where I live- but I don't know how to find even one at this time.
 
Garry Crane
gandkcrane <at> msn.com
<div><div dir="ltr">
<p align="left">I am trying to learn Python&nbsp;for use in computational biology.&nbsp; I am using the interesting book: "Computing for Biologists; Python Programming and Principles" (by Ran Libeskind-Hadas and Eliot Bush).&nbsp; It has an interesting and useful set of programming exercises at <a href="http://www.cs.hmc.edu/CFB">www.cs.hmc.edu/CFB</a>.&nbsp; I am actually enjoying solving (doing) the example problems.&nbsp; However, I find some of the instructions non-functional for me.&nbsp; For example the import function does not work,&nbsp; nor f=open("filename.txt").&nbsp; I have saved files per instructions in the programming exercise&nbsp;inside the Python34 folder (I am using Python 3.4 in Windows 7).&nbsp; But use of the f=open() command produces an error message that the requested file does not exist. I assume I have chosen a wrong location for the saved file&nbsp;within that Python34 folder, but trial and error has not led to a successful use of these functions.&nbsp; import simply leaves a blank line .. no suggestion about the problem.</p>&nbsp;<br>Asking questions in Google and Ask about&nbsp; where to save Python-related files&nbsp;that can be used subsequently&nbsp;have not led to answers - just details about structuring or formatting things to be written/saved/use of the \n at end of each line, etc.&nbsp; Important details, but of no help.&nbsp;&nbsp; I am finding Python to be very handy at many biologic things such as working with DNA strings, etc. but I find the documentation and indexing for finding how to use many Python features exasperating.&nbsp; I am writing to you based on a READ ME file in my Python folder - generated when I installed Python.<br>&nbsp;<br>FYI, I asked a few questions of one of the authors of the interesting book - who politely replied he was too busy to answer right now - the book and problems were meant for a class ... though neither the book nor problems say so.&nbsp; The professor hopes to get around to issues of use by non-students sometime - but not now.<br>&nbsp;<br>Another feature I have come across so far that does not work is importation of matplotlib.&nbsp; I copy computed results (that otherwise would go to your plotting routine)&nbsp; then go to Excel and with manipulation produce a usable chart there - but at a cost of time and energy.<br>&nbsp;<br>Your Python tool has many intriguing features - but some of the most basic functions do not work for me (even though many features do, e.g., import random does work).&nbsp; The failure of these features - so far as I can tell - is&nbsp;&nbsp;because of lack of description (for the general non-expert public) about where/how to install various features.&nbsp; Perhaps I need to reinstall from the ground up???&nbsp; If so, just what should I do?&nbsp; If there is a less drastic solution, can you tell me about it?<br>&nbsp;<br>Thank you for any help ... and if you could provide me a lead regarding WHERE to ask subsequent questions I would be most grateful.&nbsp; Sometimes, Google or Ask or a U Tube tutorial does a good job - but if one does not know the 'proper' name or term for something, it often is frustrating or impossible to get an answer.&nbsp; I have not heard about any comprehensive handbook for Python34 aimed at one who wants to use Python for creating programs (functions) that work - and is not an expert at back-room structure of files and folders.... have I simply missed it?&nbsp; So far, I have not found a local Python expert to ask for help.&nbsp; I am sure&nbsp;some are in the greater Seattle area where I live- but I don't know how to find even one at this time.<br>&nbsp;<br>Garry Crane<br><a href="mailto:gandkcrane <at> msn.com">gandkcrane <at> msn.com</a><br>
</div></div>
Guido van Rossum | 24 Feb 22:35 2015

Re: Emit SyntaxWarning on unrecognized backslash escapes?

[Adding back python-dev]

Actually, I wasn't proposing to change repr() -- my sentiments are similar to Isaac Morland's. Only the error message for the most basic file open() call should be tweaked.

No solution is perfect -- but it's probably common enough for someone to create a file or folder named C:\test and being stumped by "cannot open 'C:\test'."

On Tue, Feb 24, 2015 at 1:25 PM, Glenn Linderman <v+python <at> g.nevcal.com> wrote:
On 2/24/2015 10:49 AM, Guido van Rossum wrote:
I like the \x07 solution.

The more I think about it, the more I think this is a sufficient solution.  People that use repr to get a Python-compatible string syntax for further program use get one. People that see them in an error message are much less likely to be fooled into thinking it is two characters, because they are thinking of it as two characters to start with.

On the other hand, I have a directory full of "throw away" experimental source files named  x01, x02, x03, ...  :)

And I suppose extensive use of certain characters in repr in intermediate or archived data could make such data grow in size.

And while \t and \n are very commonly used escapes, maybe string repr could have a special function called .I_promise_not_to_report_issues_when_I_use_backslash_escapes_in_character_literals_and_get_confused() which would switch back from \x07 to \t and \x0a to \n, etc.  This would be callable only from __main__ :)



--
--Guido van Rossum (python.org/~guido)
<div>
<div dir="ltr">
<div>[Adding back python-dev]<br><br>Actually, I wasn't proposing to change repr() -- my sentiments are similar to Isaac Morland's. Only the error message for the most basic file open() call should be tweaked.<br><br>
</div>No solution is perfect -- but it's probably common enough for someone to create a file or folder named C:\test and being stumped by "cannot open 'C:\test'."<br>
</div>
<div class="gmail_extra">
<br><div class="gmail_quote">On Tue, Feb 24, 2015 at 1:25 PM, Glenn Linderman <span dir="ltr">&lt;<a href="mailto:v+python <at> g.nevcal.com" target="_blank">v+python <at> g.nevcal.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">

    

  <div bgcolor="#FFFFFF" text="#330033">
<span class="">
    <div>On 2/24/2015 10:49 AM, Guido van Rossum
      wrote:<br>
</div>
    <blockquote type="cite">I like the \x07 solution.</blockquote>
    <br></span>
    The more I think about it, the more I think this is a sufficient
    solution.&nbsp; People that use repr to get a Python-compatible string
    syntax for further program use get one. People that see them in an
    error message are much less likely to be fooled into thinking it is
    two characters, because they are thinking of it as two characters to
    start with.<br><br>
    On the other hand, I have a directory full of "throw away"
    experimental source files named&nbsp; x01, x02, x03, ...&nbsp; :)<br><br>
    And I suppose extensive use of certain characters in repr in
    intermediate or archived data could make such data grow in size.<br><br>
    And while \t and \n are very commonly used escapes, maybe string
    repr could have a special function called
    .I_promise_not_to_report_issues_when_I_use_backslash_escapes_in_character_literals_and_get_confused()
    which would switch back from \x07 to \t and \x0a to \n, etc.&nbsp; This
    would be callable only from __main__ :)<br>
</div>

</blockquote>
</div>
<br><br clear="all"><br>-- <br><div class="gmail_signature">--Guido van Rossum (<a href="http://python.org/~guido">python.org/~guido</a>)</div>
</div>
</div>
Gregory P. Smith | 24 Feb 20:33 2015

getstatusoutput()'s behavior changed in 3.3.4 and 3.4

While porting some code from 2.7 to 3.4 I discovered that command.getstatusoutput() (renamed to subprocess.getstatusoutput() in 3.x) had changed. Surprise!

The code was working under an earlier version of 3.3 but broke when I ran it on 3.4.  Nowhere was this documented that I could find. Tracking down what changed, I discovered it was unintentional due to the meaning of the returned status int never being tested. http://bugs.python.org/issue23508 filed...

Given it has shipped in several stable 3.x releases and as part of some major Linux distros, reverting the behavior change seems wrong. The new behavior is nicer and more consistent with the rest of the subprocess module. I suggest just documenting it and moving on. It seems too late to fix this mistake without causing additional headaches. Anyone disagree?

-gps
<div><div dir="ltr">While porting some code from 2.7 to 3.4 I discovered that command.getstatusoutput() (renamed to subprocess.getstatusoutput() in 3.x) had changed. Surprise!<div><br></div>
<div>The code was working under an earlier version of 3.3 but broke when I ran it on 3.4.&nbsp; Nowhere was this documented that I could find. Tracking down what changed, I discovered it was unintentional due to the meaning of the returned status int never being tested. <span><a href="http://bugs.python.org/issue23508">http://bugs.python.org/issue23508</a> filed...</span><div><br></div>
<div>Given it has shipped in several stable 3.x releases and as part of some major Linux distros, reverting the behavior change seems wrong. The new behavior is nicer and more consistent with the rest of the subprocess module. I suggest just documenting it and moving on. It seems too late to fix this mistake without causing additional headaches. Anyone disagree?</div>
</div>
<div><br></div>
<div>-gps</div>
</div></div>
Guido van Rossum | 24 Feb 19:58 2015

Fwd: Request for Pronouncement: PEP 441 - Improving Python ZIP Application Support

[Sorry, accidentally dropped the list from this message.]

Here's my review. I really like where this is going but I have a few questions and suggestions (I can't help myself :-).

[I sneaked a peek at the update you sent to peps <at> python.org.]

"Currently, pyyzer [5] and pex [6] are two tools known to exist." -> "... are two such tools."

It's not stated whether the archive names include the .pyz[w] extension or not (though I'm guessing it's not -- this should be stated).

The naming of the functions feels inconsistent -- maybe pack(directory, target) -> create_archive(directory, archive), and set_interpreter() -> copy_archive(archive, new_archive)?

Why no command-line equivalent for the other two methods?  I propose the following interface: if there's only one positional argument, we're asking to print its shebang line; if there are two and the input position is an archive instead of a directory, we're copying.  (In the future people will want an option to print more stuff, e.g. the main function or even a full listing.)

I've not seen the pkg.mod:fn notation before.  Where is this taken from?  Why can't it be pkg.mod.fn?

I'd specify that when the output argument is a file open for writing, it is the caller's responsibility to close the file.  Also, can the file be a pipe?  (I.e. are we using seek()/tell() or not?)  And what about the input archive?  Can that be a file open for reading?

--
--Guido van Rossum (python.org/~guido)
<div><div dir="ltr">[Sorry, accidentally dropped the list from this message.]<br><div>
<div class="gmail_quote">
<br><div dir="ltr">Here's my review. I really like where this is going but I have a few questions and suggestions (I can't help myself :-).<br><br>[I sneaked a peek at the update you sent to <a href="mailto:peps <at> python.org" target="_blank">peps <at> python.org</a>.]<br><br>"Currently, pyyzer [5] and pex [6] are two tools known to exist." -&gt; "... are two such tools."<br><br>It's not stated whether the archive names include the .pyz[w] extension or not (though I'm guessing it's not -- this should be stated).<br><br>The naming of the functions feels inconsistent -- maybe pack(directory, target) -&gt; create_archive(directory, archive), and set_interpreter() -&gt; copy_archive(archive, new_archive)?<br><br>Why no command-line equivalent for the other two methods?&nbsp; I propose the following interface: if there's only one positional argument, we're asking to print its shebang line; if there are two and the input position is an archive instead of a directory, we're copying.&nbsp; (In the future people will want an option to print more stuff, e.g. the main function or even a full listing.)<br><br>I've not seen the pkg.mod:fn notation before.&nbsp; Where is this taken from?&nbsp; Why can't it be pkg.mod.fn?<br><br>I'd specify that when the output argument is a file open for writing, it is the caller's responsibility to close the file.&nbsp; Also, can the file be a pipe?&nbsp; (I.e. are we using seek()/tell() or not?)&nbsp; And what about the input archive?&nbsp; Can that be a file open for reading?<br clear="all">
</div>
</div>
<br>-- <br><div class="gmail_signature">--Guido van Rossum (<a href="http://python.org/~guido">python.org/~guido</a>)</div>
</div>
</div></div>

Gmane