kernc | 6 Feb 16:54 2016

[issue26303] Shared execution context between doctests in a module


New submission from kernc:

The doctest execution context documentation [0] says the tests get shallow *copies* of module's globals,
so one test can't mingle with results of another. This makes it impossible to make literate modules such as:

    """
    This module is about reusable doctests context.

    Examples
    --------
    Let's prepare something the later examples can work with:

    >>> import foo
    >>> result = foo.Something()
    2

    """
    class Bar:
        """
        Class about something.

        >>> bar = Bar(foo)
        >>> bar.uses(foo)
        True

        """
        def baz(self):
            """
            Returns 3.
(Continue reading)

Jason R. Coombs | 6 Feb 06:18 2016

[issue26302] cookies module allows commas in keys


New submission from Jason R. Coombs:

Commas aren't legal characters in cookie keys, yet in Python 3.5, they're allowed:

>>> bool(http.cookies._is_legal_key(','))
True

The issue lies in the use of _LegalChars constructing a regular expression.

"Some people, when confronted with a problem, think 'I know, I'll use regular expressions.' Now they have
two problems."

The issue arises in this line:

_is_legal_key = re.compile('[%s]+' % _LegalChars).fullmatch

Which was added in 88e1151e8e0242 referencing issue2211.

The problem is that in a regular expression, and in a character class in particular, the '-' character has a
special meaning if not the first character in the class, which is "span all characters between the leading
and following characters". As a result, the pattern has the unintended effect of including the comma in
the pattern:

>>> http.cookies._is_legal_key.__self__
re.compile("[abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!#$%&'*+-.^_`|~:]+")
>>> pattern = _
>>> pattern.fullmatch(',')
<_sre.SRE_Match object; span=(0, 1), match=','>
>>> ord('+')
(Continue reading)

STINNER Victor | 6 Feb 02:40 2016

[issue26301] ceval.c: reintroduce fast-path for list[index] in BINARY_SUBSCR


New submission from STINNER Victor:

Copy of msg222985 by Raymond Hettinger from issue #21955:

"There also used to be a fast path for binary subscriptions with integer indexes.  I would like to see that
performance regression fixed if it can be done cleanly."

----------
messages: 259708
nosy: haypo, rhettinger, yselivanov
priority: normal
severity: normal
status: open
title: ceval.c: reintroduce fast-path for list[index] in BINARY_SUBSCR
type: performance
versions: Python 3.6

_______________________________________
Python tracker <report <at> bugs.python.org>
<http://bugs.python.org/issue26301>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/python-python-bugs-list%40m.gmane.org

Andrew Barnert | 5 Feb 23:17 2016

[issue26300] "unpacked" bytecode


New submission from Andrew Barnert:

Currently, the compiler starts with a list of arrays of instructions, packs them to 1/3/6-bytes-apiece
bytecodes, fixes up all the jumps, and then calls PyCode_Optimize on the result. This makes the peephole
optimizer much more complicated. Assuming PEP 511 is accepted, it will also make plug-in bytecode
optimizers much more complicated (and probably wasteful--they'll each be repeating the same work to
re-do the fixups).

The simplest alternative (as suggested by Serhiy on -ideas) is to expose an "unpacked" bytecode to the
optimizer (in the code parameter and return value and lnotab_obj in-out parameter for PyCode_Optimize,
and similarly for PEP 511) where each instruction takes a fixed 4 bytes. This is much easier to process.
After the optimizer returns, the compiler packs opcodes into the usual 1/3/6-byte format, removing
NOPs, retargeting jumps, and adjusting the lnotab as it goes. (Note that it already pretty much has code to
do all of this except the NOP removal; it's just doing it before the optimizer instead of after.)

Negatives:

 * Arguments can now only go up to 2**23 instead of 2**31. I don't think that's a problem (has anyone ever
created a code object with 4 million instructions?).

 * A bit more work for the compiler; we'd need to test to make sure there's no measurable performance impact.

We could also expose this functionality through C API PyCode_Pack/Unpack and Python
dis.pack_code/unpack_code functions (and also make the dis module know how to parse unpacked code),
which would allow import hooks, post-processing decorators, etc. to be simplified as well. This would
remove some, but not all, of the need for things like byteplay. I think this may be worth doing, but I'm not
sure until I see how complicated it is.

We could even allow code objects with unpacked bytecode to be executed, but I think that's unnecessary
(Continue reading)

Samwyse | 5 Feb 19:40 2016

[issue26299] wsgiref.util FileWrapper raises ValueError: I/O operation on closed file.


New submission from Samwyse:

While developing, I am using wsgiref.simple_server.  Using to serve static content isn't working.  The
attached program demonstrates the issue.  Run it and connect to http://127.0.0.1:8000/.  You will see
three buttons.  Clicking on 'wsgi.filewrapper' causes the FileWrapper class found in wsgiref.util to be
used, while clicking on 'PEP 0333' uses code suggested in PEP 0333
(https://www.python.org/dev/peps/pep-0333/#id36).  Both of these fail.  Clicking on 'slurp' causes
the entire file to loaded and returned in a list.  This works.

When an application returns an instance of environ['wsgi.file_wrapper'], or creates it's own iterator,
an error is raised during the initial read of the file.  Reading the entire file and returning a
single-element list (the 'slurp' technique) works, but is impractical for larger files.

I've tested this with both Python 2.7.9 and 3.4.3 under Windows 7 (my laptop) and 3.4.3 under Alpine Linux (a
Docker instance).

----------
components: Library (Lib)
files: wsgitest.py
messages: 259685
nosy: samwyse
priority: normal
severity: normal
status: open
title: wsgiref.util FileWrapper raises ValueError: I/O operation on closed file.
versions: Python 2.7, Python 3.4
Added file: http://bugs.python.org/file41828/wsgitest.py

_______________________________________
(Continue reading)

STINNER Victor | 5 Feb 18:43 2016

[issue26298] Split ceval.c into small files


New submission from STINNER Victor:

Attached patch splits the huge "switch (opcode)" of ceval.c into smaller ceval_xxx.h files. New files:

   93 Python/ceval_stack.h
  142 Python/ceval_condjump.h
  155 Python/ceval_misc.h
  162 Python/ceval_fast.h
  180 Python/ceval_module.h
  238 Python/ceval_ctx.h
  249 Python/ceval_func.h
  262 Python/ceval_iter.h
  268 Python/ceval_build.h
  384 Python/ceval_number.h

Maybe we can put more files per .h file, maybe less. I don't really care.

It will allow to keep the code readable even with new optimizations like the issue #21955.

What do you think?

----------
files: split_ceval.patch
keywords: patch
messages: 259682
nosy: haypo, pitrou, serhiy.storchaka, yselivanov
priority: normal
severity: normal
status: open
(Continue reading)

Serhiy Storchaka | 5 Feb 18:34 2016

[issue26297] Move constant folding to AST level


New submission from Serhiy Storchaka:

For using more efficient bytecode (either specialized 8-bit opcodes or 16-bit opcodes) we need to move
some optimizations from bytecode level to AST level, since LOAD_CONST variants could have variable
size. Now with the Constant node this should be easy.

----------
components: Interpreter Core
messages: 259680
nosy: abarnert, benjamin.peterson, brett.cannon, georg.brandl, haypo, ncoghlan, serhiy.storchaka, yselivanov
priority: normal
severity: normal
stage: needs patch
status: open
title: Move constant folding to AST level
type: enhancement
versions: Python 3.6

_______________________________________
Python tracker <report <at> bugs.python.org>
<http://bugs.python.org/issue26297>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/python-python-bugs-list%40m.gmane.org

Mats Luspa | 5 Feb 15:00 2016

[issue26296] colorys rgb_to_hls algorithm error


New submission from Mats Luspa:

In the colorsys library function rgb_to_hls the algorithm is not implemented quite correctly.

According to algorithm the correct implementation should be (the error is in the condition r == maxc). In
the current code g<b is not tested:

def rgb_to_hls(r, g, b):
    maxc = max(r, g, b)
    minc = min(r, g, b)
    # XXX Can optimize (maxc+minc) and (maxc-minc)
    l = (minc+maxc)/2.0
    if minc == maxc:
        return 0.0, l, 0.0
    if l <= 0.5:
        s = (maxc-minc) / (maxc+minc)
    else:
        s = (maxc-minc) / (2.0-maxc-minc)
    rc = (maxc-r) / (maxc-minc)
    gc = (maxc-g) / (maxc-minc)
    bc = (maxc-b) / (maxc-minc)
    if r == maxc:
        if (g<b):
            h = bc-gc+6
        else:
            h = bc-gc
    elif g == maxc:
        h = 2.0+rc-bc
    else:
(Continue reading)

STINNER Victor | 5 Feb 10:15 2016

[issue26295] Random failures when running test suite in parallel (-m test -j0) caused by test_regrtest


New submission from STINNER Victor:

test_regrtest creates temporary test files called test_regrtest_pid_xxx.py in Lib/test/. The problem
is that some tests like test___all__ and test_zipfile haves test relying on the list of Lib/test/test_*.py.

When tests are run in parallel, test_regrtest can creates temporary test_regrtest_pid_xxx.py files,
test_zipfile sees them, test_regrtest removes them, and then test_zipfiles fails.

The best would be to write these temporary files into a temporary directory, but I failed to fix
Lib/test/regrtest.py to load tests from a different directory. In theory, Python 3 supports packages
with files splitted into multiple directories, in practice it doesn't seem to work :-p

Maybe test_regrtest should use a test submodule like Lib/test/temp_regrtest/ ?

test_regrtest started to create temporary test_xxx.py files since issue #25220. (Other changes to
test_regrtest: issues #18174, #22806, #25260, #25306,  #25369, #25373, #25694).

======================================================================
ERROR: test_all (test.test___all__.AllTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/haypo/prog/python/default/Lib/test/test___all__.py", line 102, in test_all
    with open(path, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/haypo/prog/python/default/Lib/test/test_regrtest_25743_noop20.py'

----------------------------------------------------------------------

----------
components: Tests
(Continue reading)

Frank Millman | 5 Feb 10:00 2016

[issue26294] Queue().unfinished_tasks not in docs - deliberate?


New submission from Frank Millman:

dir(queue.Queue()) shows an attribute 'unfinished_tasks'.

It appears to be the counter referred to in the docs to 'join()', but it is not documented itself.

I would like to make use of it, but I don't know if it is part of the official API for this module.

Please advise.

----------
assignee: docs <at> python
components: Documentation
messages: 259645
nosy: docs <at> python, frankmillman
priority: normal
severity: normal
status: open
title: Queue().unfinished_tasks not in docs - deliberate?
versions: Python 3.4, Python 3.5, Python 3.6

_______________________________________
Python tracker <report <at> bugs.python.org>
<http://bugs.python.org/issue26294>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/python-python-bugs-list%40m.gmane.org

(Continue reading)

spoo | 5 Feb 09:52 2016

[issue26293] Embedded zipfile fields dependent on absolute position


New submission from spoo:

Example:

from zipfile import ZipFile
with open('a.zipp', 'wb') as base:
    base.write(b'old\n')
    with ZipFile(base, 'a') as myzip:
        myzip.write('eggs.txt')

If the embedded zip portion of the file is extracted (first four bytes deleted), some fields will be
incorrect in the resultant file - commenting out line 3 produces a file that can serve as a comparison. 
These differences cause issues opening with some zip library implementations.

My best guess is that this is related to this line: https://github.com/python/cpython/blob/master/Lib/zipfile.py#L1459

----------
components: Library (Lib)
messages: 259642
nosy: spoo
priority: normal
severity: normal
status: open
title: Embedded zipfile fields dependent on absolute position
type: behavior
versions: Python 3.6

_______________________________________
Python tracker <report <at> bugs.python.org>
(Continue reading)


Gmane