Python tracker | 3 Jul 18:08 2015

Summary of Python tracker Issues


ACTIVITY SUMMARY (2015-06-26 - 2015-07-03)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4922 (+14)
  closed 31399 (+30)
  total  36321 (+44)

Open issues with patches: 2255 

Issues opened (27)
==================

#3530: ast.NodeTransformer doc bug
http://bugs.python.org/issue3530  reopened by terry.reedy

#24400: Awaitable ABC incompatible with functools.singledispatch
http://bugs.python.org/issue24400  reopened by r.david.murray

#24458: Documentation for PEP 489
http://bugs.python.org/issue24458  reopened by ncoghlan

#24518: json.dumps should accept key function for ``sort_keys``
http://bugs.python.org/issue24518  opened by catherinedevlin

#24520: Stop using deprecated floating-point environment functions on 
(Continue reading)

Chris Moller | 30 Jun 21:40 2015

Capturing PyRun_String stdout

I expect this has been asked before, but I can't find out much about it...

I'm trying to embed Python as a scripting language and I need to capture the output of PyRun_String(), PyEval_EvalCode(), or whatever as a char * (or wchar_t * or whatever) rather than have it go to stdout.

Python 3.3.2 under plain C, not C++

And, while I'm interrupting everyone's afternoon, another question: if I  pass Py_single_input to PyRun_String() or Py_CompileString()/PyEval_EvalCode(), it accepts statements like "a=10" and can then properly do stuff like "print(a)".  If I use Py_eval_input instead, I get error messages.  In both cases, I'm using the same global_dict and local_dict, if that makes any difference.  What am I doing wrong?

Thanks,
Chris Moller
<div>
    I expect this has been asked before, but I can't find out much about
    it...<br><br>
    I'm trying to embed Python as a scripting language and I need to
    capture the output of PyRun_String(), PyEval_EvalCode(), or whatever
    as a char * (or wchar_t * or whatever) rather than have it go to
    stdout.<br><br>
    Python 3.3.2 under plain C, not C++<br><br>
    And, while I'm interrupting everyone's afternoon, another question:
    if I&nbsp; pass Py_single_input to PyRun_String() or
    Py_CompileString()/PyEval_EvalCode(), it accepts statements like
    "a=10" and can then properly do stuff like "print(a)".&nbsp; If I use
    Py_eval_input instead, I get error messages.&nbsp; In both cases, I'm
    using the same global_dict and local_dict, if that makes any
    difference.&nbsp; What am I doing wrong?<br><br>
    Thanks,<br>
    Chris Moller<br>
</div>
Robert Collins | 29 Jun 06:28 2015
Picon

Not getting mail from the issue tracker

Firstly, a big sorry to all those unittest issues I haven't commented on.

Turns out I simply don't get mail from the issue tracker. :(.

Who should I speak to to try and debug this?

In the interim, if you want me to look at an issue please ping me on
IRC (lifeless) or mail me directly.

-Rob

--

-- 
Robert Collins <rbtcollins <at> hp.com>
Distinguished Technologist
HP Converged Cloud
Python tracker | 26 Jun 18:08 2015

Summary of Python tracker Issues


ACTIVITY SUMMARY (2015-06-19 - 2015-06-26)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4908 (+16)
  closed 31369 (+27)
  total  36277 (+43)

Open issues with patches: 2254 

Issues opened (29)
==================

#15014: smtplib: add support for arbitrary auth methods
http://bugs.python.org/issue15014  reopened by barry

#24475: The docs never define what a pool "task" is
http://bugs.python.org/issue24475  opened by Zahari.Dim

#24477: In argparse subparser's option goes to parent parser
http://bugs.python.org/issue24477  opened by py.user

#24479: Support LMMS project files in mimetypes.guess_type
http://bugs.python.org/issue24479  opened by Andreas Nilsson

#24481: hotshot pack_string Heap Buffer Overflow
http://bugs.python.org/issue24481  opened by JohnLeitch

#24482: itertools.tee causes segfault in a multithreading environment,
http://bugs.python.org/issue24482  opened by Dmitry Odzerikho

#24483: Avoid repeated hash calculation in C implementation of functoo
http://bugs.python.org/issue24483  opened by serhiy.storchaka

#24484: multiprocessing cleanup occasionally throws exception
http://bugs.python.org/issue24484  opened by Jorge Herskovic

#24485: Function source inspection fails on closures
http://bugs.python.org/issue24485  opened by malthe

#24488: ConfigParser.getboolean fails on boolean options
http://bugs.python.org/issue24488  opened by andreas-h

#24492: using custom objects as modules: AttributeErrors new in 3.5
http://bugs.python.org/issue24492  opened by arigo

#24493: subprocess with env=os.environ doesn't preserve environment va
http://bugs.python.org/issue24493  opened by The Compiler

#24494: Can't specify encoding with fileinput and inplace=True
http://bugs.python.org/issue24494  opened by lilydjwg

#24498: Shoudl ptags and eptags be removed from repo?
http://bugs.python.org/issue24498  opened by r.david.murray

#24499: Python Installer text piles up during installation process
http://bugs.python.org/issue24499  opened by Zach “The Quantum Mechanic” W

#24500: provide context manager to redirect C output
http://bugs.python.org/issue24500  opened by Zahari.Dim

#24501: configure does not find (n)curses in /usr/local/libs
http://bugs.python.org/issue24501  opened by petepdx

#24502: OS X 2.7 package has zeros for version numbers in sub-packages
http://bugs.python.org/issue24502  opened by Jim Zajkowski

#24503: csv.writer fails when within csv.reader
http://bugs.python.org/issue24503  opened by ezzieyguywuf

#24505: shutil.which wrong result on Windows
http://bugs.python.org/issue24505  opened by bobjalex

#24506: make fails with gcc 4.9 due to fatal warning of unused variabl
http://bugs.python.org/issue24506  opened by krichter

#24507: CRLF issues
http://bugs.python.org/issue24507  opened by RusiMody

#24508: Backport 3.5's Windows build project files to 2.7
http://bugs.python.org/issue24508  opened by zach.ware

#24510: Make _PyCoro_GetAwaitableIter a public API
http://bugs.python.org/issue24510  opened by yselivanov

#24511: Add methods for async protocols
http://bugs.python.org/issue24511  opened by yselivanov

#24512: multiprocessing should log a warning when forking multithreade
http://bugs.python.org/issue24512  opened by trcarden

#24514: tarfile fails to extract archive (handled fine by gnu tar and 
http://bugs.python.org/issue24514  opened by pombreda

#24515: docstring of isinstance
http://bugs.python.org/issue24515  opened by Luc Saffre

#24516: SSL create_default_socket purpose insufficiently documented
http://bugs.python.org/issue24516  opened by messa

Most recent 15 issues with no replies (15)
==========================================

#24512: multiprocessing should log a warning when forking multithreade
http://bugs.python.org/issue24512

#24499: Python Installer text piles up during installation process
http://bugs.python.org/issue24499

#24498: Shoudl ptags and eptags be removed from repo?
http://bugs.python.org/issue24498

#24481: hotshot pack_string Heap Buffer Overflow
http://bugs.python.org/issue24481

#24477: In argparse subparser's option goes to parent parser
http://bugs.python.org/issue24477

#24475: The docs never define what a pool "task" is
http://bugs.python.org/issue24475

#24466: extend_path explanation in documentation is ambiguous
http://bugs.python.org/issue24466

#24424: xml.dom.minidom: performance issue with Node.insertBefore()
http://bugs.python.org/issue24424

#24417: Type-specific documentation for __format__ methods
http://bugs.python.org/issue24417

#24415: SIGINT always reset to SIG_DFL by Py_Finalize()
http://bugs.python.org/issue24415

#24414: MACOSX_DEPLOYMENT_TARGET set incorrectly by configure
http://bugs.python.org/issue24414

#24407: Use after free in PyDict_merge
http://bugs.python.org/issue24407

#24381: Got warning when compiling ffi.c on Mac
http://bugs.python.org/issue24381

#24364: Not all defects pass through email policy
http://bugs.python.org/issue24364

#24352: Provide a way for assertLogs to optionally not hide the loggin
http://bugs.python.org/issue24352

Most recent 15 issues waiting for review (15)
=============================================

#24514: tarfile fails to extract archive (handled fine by gnu tar and 
http://bugs.python.org/issue24514

#24508: Backport 3.5's Windows build project files to 2.7
http://bugs.python.org/issue24508

#24506: make fails with gcc 4.9 due to fatal warning of unused variabl
http://bugs.python.org/issue24506

#24483: Avoid repeated hash calculation in C implementation of functoo
http://bugs.python.org/issue24483

#24468: Expose C level compiler flag constants to Python code
http://bugs.python.org/issue24468

#24467: bytearray pop and remove Buffer Over-read
http://bugs.python.org/issue24467

#24465: Make tarfile have deterministic sorting
http://bugs.python.org/issue24465

#24464: Got warning when compiling sqlite3 module on Mac OS X
http://bugs.python.org/issue24464

#24462: bytearray.find Buffer Over-read
http://bugs.python.org/issue24462

#24458: Documentation for PEP 489
http://bugs.python.org/issue24458

#24456: audioop.adpcm2lin Buffer Over-read
http://bugs.python.org/issue24456

#24452: Make webbrowser support Chrome on Mac OS X
http://bugs.python.org/issue24452

#24451: Add metrics to future objects (concurrent or asyncio?)
http://bugs.python.org/issue24451

#24450: Add cr_await calculated property to coroutine object
http://bugs.python.org/issue24450

#24440: Move the buildslave setup information from the wiki to the dev
http://bugs.python.org/issue24440

Top 10 most discussed issues (10)
=================================

#24483: Avoid repeated hash calculation in C implementation of functoo
http://bugs.python.org/issue24483  15 msgs

#24129: Incorrect (misleading) statement in the execution model docume
http://bugs.python.org/issue24129  11 msgs

#24484: multiprocessing cleanup occasionally throws exception
http://bugs.python.org/issue24484  10 msgs

#20387: tokenize/untokenize roundtrip fails with tabs
http://bugs.python.org/issue20387   8 msgs

#24263: unittest cannot load module whose name starts with Unicode
http://bugs.python.org/issue24263   8 msgs

#23883: __all__ lists are incomplete
http://bugs.python.org/issue23883   7 msgs

#15014: smtplib: add support for arbitrary auth methods
http://bugs.python.org/issue15014   6 msgs

#24325: Speedup types.coroutine()
http://bugs.python.org/issue24325   6 msgs

#24514: tarfile fails to extract archive (handled fine by gnu tar and 
http://bugs.python.org/issue24514   6 msgs

#12210: test_smtplib: intermittent failures on FreeBSD
http://bugs.python.org/issue12210   5 msgs

Issues closed (27)
==================

#13213: generator.throw() behavior
http://bugs.python.org/issue13213  closed by vadmium

#23246: distutils fails to locate vcvarsall with Visual C++ Compiler f
http://bugs.python.org/issue23246  closed by steve.dower

#23684: urlparse() documentation does not account for default scheme
http://bugs.python.org/issue23684  closed by berker.peksag

#24244: Python exception on strftime with %f on Python 3 and Python 2 
http://bugs.python.org/issue24244  closed by python-dev

#24309: string.Template should be using str.format and/or deprecated
http://bugs.python.org/issue24309  closed by wau

#24400: Awaitable ABC incompatible with functools.singledispatch
http://bugs.python.org/issue24400  closed by yselivanov

#24426: re.split performance degraded significantly by capturing group
http://bugs.python.org/issue24426  closed by serhiy.storchaka

#24436: _PyTraceback_Add has no const qualifier for its char * argumen
http://bugs.python.org/issue24436  closed by serhiy.storchaka

#24439: Feedback for awaitable coroutine documentation
http://bugs.python.org/issue24439  closed by yselivanov

#24447: tab indentation breaks in tokenize.untokenize
http://bugs.python.org/issue24447  closed by terry.reedy

#24448: Syntax highlighting marks multiline comments as strings
http://bugs.python.org/issue24448  closed by terry.reedy

#24457: audioop.lin2adpcm Buffer Over-read
http://bugs.python.org/issue24457  closed by serhiy.storchaka

#24474: Accidental exception chaining in inspect.Signature.bind()
http://bugs.python.org/issue24474  closed by ncoghlan

#24476: Statically link vcruntime140.dll
http://bugs.python.org/issue24476  closed by steve.dower

#24478: asyncio: segfault in test_env_var_debug() on non-debug Windows
http://bugs.python.org/issue24478  closed by zach.ware

#24480: Python 2.7.10
http://bugs.python.org/issue24480  closed by yliu120

#24486: http/client.py block indefinitely on line 308 in _read_status
http://bugs.python.org/issue24486  closed by Julien.Palard

#24487: Change asyncio.async() → ensure_future()
http://bugs.python.org/issue24487  closed by yselivanov

#24489: cmath.polar() can raise due to pre-existing errno
http://bugs.python.org/issue24489  closed by pitrou

#24490: DeprecationWarning hidden even after warnings.filterwarnings('
http://bugs.python.org/issue24490  closed by ned.deily

#24491: inspect.getsource can't get source code if provided function i
http://bugs.python.org/issue24491  closed by ned.deily

#24495: asyncio.ensure_future() AttributeError with “async def” co
http://bugs.python.org/issue24495  closed by yselivanov

#24496: Non-idiomatic examples in gzip docs
http://bugs.python.org/issue24496  closed by berker.peksag

#24497: test_decimal.py contains a dead link
http://bugs.python.org/issue24497  closed by ned.deily

#24504: os.listdir() error if the last folder starts not with the capi
http://bugs.python.org/issue24504  closed by Djarbore

#24509: Undocumented features of asyncio: call_at, call_later
http://bugs.python.org/issue24509  closed by yselivanov

#24513: decimal test version mismatch
http://bugs.python.org/issue24513  closed by skrah

ACTIVITY SUMMARY (2015-06-19 - 2015-06-26)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4908 (+16)
  closed 31369 (+27)
  total  36277 (+43)

Open issues with patches: 2254 

Issues opened (29)
==================

#15014: smtplib: add support for arbitrary auth methods
http://bugs.python.org/issue15014  reopened by barry

#24475: The docs never define what a pool "task" is
http://bugs.python.org/issue24475  opened by Zahari.Dim

#24477: In argparse subparser's option goes to parent parser
http://bugs.python.org/issue24477  opened by py.user

#24479: Support LMMS project files in mimetypes.guess_type
http://bugs.python.org/issue24479  opened by Andreas Nilsson

#24481: hotshot pack_string Heap Buffer Overflow
http://bugs.python.org/issue24481  opened by JohnLeitch

#24482: itertools.tee causes segfault in a multithreading environment,
http://bugs.python.org/issue24482  opened by Dmitry Odzerikho

#24483: Avoid repeated hash calculation in C implementation of functoo
http://bugs.python.org/issue24483  opened by serhiy.storchaka

#24484: multiprocessing cleanup occasionally throws exception
http://bugs.python.org/issue24484  opened by Jorge Herskovic

#24485: Function source inspection fails on closures
http://bugs.python.org/issue24485  opened by malthe

#24488: ConfigParser.getboolean fails on boolean options
http://bugs.python.org/issue24488  opened by andreas-h

#24492: using custom objects as modules: AttributeErrors new in 3.5
http://bugs.python.org/issue24492  opened by arigo

#24493: subprocess with env=os.environ doesn't preserve environment va
http://bugs.python.org/issue24493  opened by The Compiler

#24494: Can't specify encoding with fileinput and inplace=True
http://bugs.python.org/issue24494  opened by lilydjwg

#24498: Shoudl ptags and eptags be removed from repo?
http://bugs.python.org/issue24498  opened by r.david.murray

#24499: Python Installer text piles up during installation process
http://bugs.python.org/issue24499  opened by Zach “The Quantum Mechanic” W

#24500: provide context manager to redirect C output
http://bugs.python.org/issue24500  opened by Zahari.Dim

#24501: configure does not find (n)curses in /usr/local/libs
http://bugs.python.org/issue24501  opened by petepdx

#24502: OS X 2.7 package has zeros for version numbers in sub-packages
http://bugs.python.org/issue24502  opened by Jim Zajkowski

#24503: csv.writer fails when within csv.reader
http://bugs.python.org/issue24503  opened by ezzieyguywuf

#24505: shutil.which wrong result on Windows
http://bugs.python.org/issue24505  opened by bobjalex

#24506: make fails with gcc 4.9 due to fatal warning of unused variabl
http://bugs.python.org/issue24506  opened by krichter

#24507: CRLF issues
http://bugs.python.org/issue24507  opened by RusiMody

#24508: Backport 3.5's Windows build project files to 2.7
http://bugs.python.org/issue24508  opened by zach.ware

#24510: Make _PyCoro_GetAwaitableIter a public API
http://bugs.python.org/issue24510  opened by yselivanov

#24511: Add methods for async protocols
http://bugs.python.org/issue24511  opened by yselivanov

#24512: multiprocessing should log a warning when forking multithreade
http://bugs.python.org/issue24512  opened by trcarden

#24514: tarfile fails to extract archive (handled fine by gnu tar and 
http://bugs.python.org/issue24514  opened by pombreda

#24515: docstring of isinstance
http://bugs.python.org/issue24515  opened by Luc Saffre

#24516: SSL create_default_socket purpose insufficiently documented
http://bugs.python.org/issue24516  opened by messa

Most recent 15 issues with no replies (15)
==========================================

#24512: multiprocessing should log a warning when forking multithreade
http://bugs.python.org/issue24512

#24499: Python Installer text piles up during installation process
http://bugs.python.org/issue24499

#24498: Shoudl ptags and eptags be removed from repo?
http://bugs.python.org/issue24498

#24481: hotshot pack_string Heap Buffer Overflow
http://bugs.python.org/issue24481

#24477: In argparse subparser's option goes to parent parser
http://bugs.python.org/issue24477

#24475: The docs never define what a pool "task" is
http://bugs.python.org/issue24475

#24466: extend_path explanation in documentation is ambiguous
http://bugs.python.org/issue24466

#24424: xml.dom.minidom: performance issue with Node.insertBefore()
http://bugs.python.org/issue24424

#24417: Type-specific documentation for __format__ methods
http://bugs.python.org/issue24417

#24415: SIGINT always reset to SIG_DFL by Py_Finalize()
http://bugs.python.org/issue24415

#24414: MACOSX_DEPLOYMENT_TARGET set incorrectly by configure
http://bugs.python.org/issue24414

#24407: Use after free in PyDict_merge
http://bugs.python.org/issue24407

#24381: Got warning when compiling ffi.c on Mac
http://bugs.python.org/issue24381

#24364: Not all defects pass through email policy
http://bugs.python.org/issue24364

#24352: Provide a way for assertLogs to optionally not hide the loggin
http://bugs.python.org/issue24352

Most recent 15 issues waiting for review (15)
=============================================

#24514: tarfile fails to extract archive (handled fine by gnu tar and 
http://bugs.python.org/issue24514

#24508: Backport 3.5's Windows build project files to 2.7
http://bugs.python.org/issue24508

#24506: make fails with gcc 4.9 due to fatal warning of unused variabl
http://bugs.python.org/issue24506

#24483: Avoid repeated hash calculation in C implementation of functoo
http://bugs.python.org/issue24483

#24468: Expose C level compiler flag constants to Python code
http://bugs.python.org/issue24468

#24467: bytearray pop and remove Buffer Over-read
http://bugs.python.org/issue24467

#24465: Make tarfile have deterministic sorting
http://bugs.python.org/issue24465

#24464: Got warning when compiling sqlite3 module on Mac OS X
http://bugs.python.org/issue24464

#24462: bytearray.find Buffer Over-read
http://bugs.python.org/issue24462

#24458: Documentation for PEP 489
http://bugs.python.org/issue24458

#24456: audioop.adpcm2lin Buffer Over-read
http://bugs.python.org/issue24456

#24452: Make webbrowser support Chrome on Mac OS X
http://bugs.python.org/issue24452

#24451: Add metrics to future objects (concurrent or asyncio?)
http://bugs.python.org/issue24451

#24450: Add cr_await calculated property to coroutine object
http://bugs.python.org/issue24450

#24440: Move the buildslave setup information from the wiki to the dev
http://bugs.python.org/issue24440

Top 10 most discussed issues (10)
=================================

#24483: Avoid repeated hash calculation in C implementation of functoo
http://bugs.python.org/issue24483  15 msgs

#24129: Incorrect (misleading) statement in the execution model docume
http://bugs.python.org/issue24129  11 msgs

#24484: multiprocessing cleanup occasionally throws exception
http://bugs.python.org/issue24484  10 msgs

#20387: tokenize/untokenize roundtrip fails with tabs
http://bugs.python.org/issue20387   8 msgs

#24263: unittest cannot load module whose name starts with Unicode
http://bugs.python.org/issue24263   8 msgs

#23883: __all__ lists are incomplete
http://bugs.python.org/issue23883   7 msgs

#15014: smtplib: add support for arbitrary auth methods
http://bugs.python.org/issue15014   6 msgs

#24325: Speedup types.coroutine()
http://bugs.python.org/issue24325   6 msgs

#24514: tarfile fails to extract archive (handled fine by gnu tar and 
http://bugs.python.org/issue24514   6 msgs

#12210: test_smtplib: intermittent failures on FreeBSD
http://bugs.python.org/issue12210   5 msgs

Issues closed (27)
==================

#13213: generator.throw() behavior
http://bugs.python.org/issue13213  closed by vadmium

#23246: distutils fails to locate vcvarsall with Visual C++ Compiler f
http://bugs.python.org/issue23246  closed by steve.dower

#23684: urlparse() documentation does not account for default scheme
http://bugs.python.org/issue23684  closed by berker.peksag

#24244: Python exception on strftime with %f on Python 3 and Python 2 
http://bugs.python.org/issue24244  closed by python-dev

#24309: string.Template should be using str.format and/or deprecated
http://bugs.python.org/issue24309  closed by wau

#24400: Awaitable ABC incompatible with functools.singledispatch
http://bugs.python.org/issue24400  closed by yselivanov

#24426: re.split performance degraded significantly by capturing group
http://bugs.python.org/issue24426  closed by serhiy.storchaka

#24436: _PyTraceback_Add has no const qualifier for its char * argumen
http://bugs.python.org/issue24436  closed by serhiy.storchaka

#24439: Feedback for awaitable coroutine documentation
http://bugs.python.org/issue24439  closed by yselivanov

#24447: tab indentation breaks in tokenize.untokenize
http://bugs.python.org/issue24447  closed by terry.reedy

#24448: Syntax highlighting marks multiline comments as strings
http://bugs.python.org/issue24448  closed by terry.reedy

#24457: audioop.lin2adpcm Buffer Over-read
http://bugs.python.org/issue24457  closed by serhiy.storchaka

#24474: Accidental exception chaining in inspect.Signature.bind()
http://bugs.python.org/issue24474  closed by ncoghlan

#24476: Statically link vcruntime140.dll
http://bugs.python.org/issue24476  closed by steve.dower

#24478: asyncio: segfault in test_env_var_debug() on non-debug Windows
http://bugs.python.org/issue24478  closed by zach.ware

#24480: Python 2.7.10
http://bugs.python.org/issue24480  closed by yliu120

#24486: http/client.py block indefinitely on line 308 in _read_status
http://bugs.python.org/issue24486  closed by Julien.Palard

#24487: Change asyncio.async() → ensure_future()
http://bugs.python.org/issue24487  closed by yselivanov

#24489: cmath.polar() can raise due to pre-existing errno
http://bugs.python.org/issue24489  closed by pitrou

#24490: DeprecationWarning hidden even after warnings.filterwarnings('
http://bugs.python.org/issue24490  closed by ned.deily

#24491: inspect.getsource can't get source code if provided function i
http://bugs.python.org/issue24491  closed by ned.deily

#24495: asyncio.ensure_future() AttributeError with “async def” co
http://bugs.python.org/issue24495  closed by yselivanov

#24496: Non-idiomatic examples in gzip docs
http://bugs.python.org/issue24496  closed by berker.peksag

#24497: test_decimal.py contains a dead link
http://bugs.python.org/issue24497  closed by ned.deily

#24504: os.listdir() error if the last folder starts not with the capi
http://bugs.python.org/issue24504  closed by Djarbore

#24509: Undocumented features of asyncio: call_at, call_later
http://bugs.python.org/issue24509  closed by yselivanov

#24513: decimal test version mismatch
http://bugs.python.org/issue24513  closed by skrah
Elizabeth Shashkova | 25 Jun 19:23 2015
Picon

Is it a Python bug that the main thread of a process created in a daemon thread is a daemon itself?

Hello everybody!

When I call fork() inside a daemon thread, the main thread in the child process has the "daemon" property set to True. This is very confusing, since the program keeps running while the only thread is a daemon. According to the docs, if all the threads are daemons the program should exit. Here is an example:
import os
import threading


def child():
assert not threading.current_thread().daemon # This shouldn't fail


def parent():
new_pid = os.fork()
if new_pid == 0:
child()
else:
os.waitpid(new_pid, 0)


t = threading.Thread(target=parent)
t.setDaemon(True)
t.start()
t.join()
Is it a bug in the CPython implementation? 
Also let's assume the second example. I have another non-daemon thread in the child process and want to detect this situation. Does anybody know a way to find such fake daemon threads that are really main threads? Best regards, Elizaveta Shashkova.

<div><div dir="ltr">Hello everybody!<br><br>When I call fork() inside a daemon thread, the main thread in the child process has the "daemon" property set to True. This is very confusing, since the program keeps running while the only thread is a daemon. According to the docs, if all the threads are daemons the program should exit.&nbsp;Here is an example:<br><span>

import </span>os<br><span>import </span>threading<br><br><br><span>def </span>child():<br><span>assert not </span>threading.current_thread().daemon  <span># This shouldn't fail<br></span><span><br></span><span><br></span><span>def </span>parent():<br>    new_pid = os.fork()<br><span>if </span>new_pid == <span>0</span>:<br>        child()<br><span>else</span>:<br>        os.waitpid(new_pid, <span>0</span>)<br><br><br>t = threading.Thread(<span>target</span>=parent)<br>t.setDaemon(<span>True</span>)<br>t.start()<br>t.join()

<span><br>Is it a bug in the CPython implementation?&nbsp;<br></span>Also let's assume the second example. I have another non-daemon thread in the child process and want to detect this situation. Does anybody know a way to find such fake daemon threads that are really main threads? 

Best regards,
Elizaveta Shashkova.<div><br></div>
</div></div>
Arc Riley | 25 Jun 04:36 2015
Picon

Adding c-api async protocol support

A type slot for tp_as_async has already been added (which is good!) but we do not currently seem to have protocol functions for awaitable types.

I would expect to find an Awaitable Protocol listed under Abstract Objects Layer, with functions like PyAwait_Check, PyAwaitIter_Check, and PyAwaitIter_Next, etc.

Specifically its currently difficult to test whether an object is awaitable or an awaitable iterable, or use said objects from the c-api without relying on method testing/calling mechanisms.


<div><div dir="ltr">A type slot for tp_as_async has already been added (which is good!) but we do not currently seem to have protocol functions for awaitable types.<div><br></div>
<div>I would expect to find an Awaitable Protocol listed under Abstract Objects Layer, with functions like&nbsp;PyAwait_Check, PyAwaitIter_Check, and PyAwaitIter_Next, etc.</div>
<div><br></div>
<div>Specifically its currently difficult to test whether an object is awaitable or an awaitable iterable, or use said objects from the c-api without relying on method testing/calling mechanisms.</div>
<div><br></div>
<div><br></div>
</div></div>
Sven R. Kunze | 24 Jun 20:14 2015
Picon

Importance of "async" keyword

Hi everybody,

recently, I stumbled over the new 3.5 release and in doing so over PEP 0492.

After careful consideration and after reading many blog posts of various 
coders, I first would like to thank Yury Selivanov and everybody else 
who brought PEP 0492 to its final state. I therefore considered usage 
within our projects, however, still find hazy items in PEP 0492. So, I 
would like to contribute my thoughts on this in order to either increase 
my understanding or even improve Python's async capability.

In order to do this, I need a clarification regarding the rationale 
behind the async keyword. The PEP rationalizes its introduction with:

"If useful() [...] would become a regular python function, [...] 
important() would be broken."

What bothers me is, why should important() be broken in that case?

Regards,
Sven R. Kunze
Rustom Mody | 24 Jun 12:22 2015
Picon

CRLF problems in repo

Hello folks

<context>
Along with a few students we were preparing to poke around inside the CPython sources.
Of course it would be neat if we submitted useful patches... but since I dont expect to get there so fast I thought I'd start by setting up with git which I am more familiar with than mercurial.
That resulted in some adventures... including CRLF issues.
</context>

Some suggestions on both python mentors list and the general user list did not seem to think these completely irrelevant so reporting here.

Can submit a bug-report if it looks ok

-------------------------

Mixed file -- both LF and CRLF (line 29 LF rest CRLF)
Lib/venv/scripts/nt/Activate.ps1

Lib/test/decimaltestdata is a directory with mixed up files -- ie some CRLF some LF files

PCbuild/readme.txt: CRLF (explicit)
This is fine. Many of the following should (ideally/IMHO) be likewise CRLF (not BIN)

*.sln: CRLF but marked as BIN in hgeol

PC/example_nt/example.vcproj
Emacs shows as Dos file; But not picked up by file; maybe other such

Missing BIN pattern from .hgeol
*.pdf
Note that Doc/library/turtle-star.pdf exists
This is most likely a bug

Existent file-types with non-existent files in hgeol; Probably harmless
*.vsprops
*.mk
*.dsp
*.dsw

These seem straightforward CRLF files to me; why BIN in hgeol?
Lib/test/test_email/data/msg_26.txt
Lib/test/coding20731.py

Regards,
Rusi

<div><div dir="ltr">
<div>
<div>
<div>
<div>Hello folks<br><br>
</div>
<div>&lt;context&gt;<br>
</div>Along with a few students we were preparing to poke around inside the CPython sources.<br>
</div>Of course it would be neat if we submitted useful patches... but since I dont expect to get there so fast I thought I'd start by setting up with git which I am more familiar with than mercurial.<br>
</div>That resulted in some adventures... including CRLF issues.<br>
</div>&lt;/context&gt;<br><div>
<br><div><div><div>
<div>Some suggestions on both python mentors list and the general user list did not seem to think these completely irrelevant so reporting here.<br><br>
</div>
<div>Can submit a bug-report if it looks ok <br>
</div>
<div>
<br>-------------------------<br><br>Mixed file -- both LF and CRLF (line 29 LF rest CRLF)<br>Lib/venv/scripts/nt/Activate.ps1<br><br>Lib/test/decimaltestdata
is a directory with mixed up files -- ie some CRLF some LF files<br><br>PCbuild/readme.txt: CRLF (explicit)<br>This is fine. Many of the following should (ideally/IMHO) be likewise CRLF (not BIN)<br><br>*.sln: CRLF but marked as BIN in hgeol<br><br>PC/example_nt/example.vcproj<br>Emacs shows as Dos file; But not picked up by file; maybe other such<br><br>Missing BIN pattern from .hgeol<br>*.pdf<br>Note that Doc/library/turtle-star.pdf exists<br>
</div>
<div>This is most likely a bug<br>
</div>
<div>
<br>Existent file-types with non-existent files in hgeol; Probably harmless<br>*.vsprops <br>*.mk<br>*.dsp<br>*.dsw<br><br>These seem straightforward CRLF files to me; why BIN in hgeol?<br>Lib/test/test_email/data/msg_26.txt<br>Lib/test/coding20731.py<br><br>
</div>
<div>Regards,<br>
</div>
<div>Rusi<br>
</div>
<div>
<div><br></div>
</div>
</div></div></div>
</div>
</div></div>
Zachary Ware | 22 Jun 19:03 2015
Picon

Backporting the 3.5+ Windows build project files to 2.7

Hi,

As you may know, Steve Dower put significant effort into rewriting the
project files used by the Windows build as part of moving to VC14 as
the official compiler for Python 3.5.  Compared to the project files
for 3.4 (and older), the new project files are smaller, cleaner,
simpler, more easily extensible, and in some cases quite a bit more
correct.

I'd like to backport those new project files to 2.7, and Intel is
willing to fund that work as part of making Python ICC compilable on
all platforms they support since it makes building Python 2.7 with ICC
much easier.  I have no intention of changing the version of MSVC used
for official 2.7 builds, it *will* remain at MSVC 9.0 (VS 2008) as
determined the last time we had a thread about it.  VS 2010 and newer
can access older compilers (back to 2008) as a 'PlatformToolset' if
they're installed, so all we have to do is set the PlatformToolset in
the projects at 'v90'.  Backporting the projects would make it easier
to build 2.7 with a newer compiler, but that's already possible if
somebody wants to put enough work into it, the default will be the old
compiler, and we can emit big scary warnings if somebody does use a
compiler other than v90.

With the stipulation that the officially supported compiler won't
change, I want to make sure there's no major opposition to replacing
the old project files in PCbuild.  The old files would move to
PC\VS9.0, so they'll still be available and usable if necessary.

Using the backported project files to build 2.7 would require two
versions of Visual Studio to be installed; VS2010 (or newer) would be
required in addition to VS2008.  All Windows core developers should
already have VS2010 for Python 3.4 (and/or VS2015 for 3.5) and I
expect that anyone else who cares enough to still have VS2008 probably
has (or can easily get) one of the free editions of VS 2010 or newer,
so I don't consider this to be a major issue.

The backported files could be added alongside the old files in
PCbuild, in a better-named 'NewPCbuild' directory, or in a
subdirectory of PC. I would rather replace the old project files in
PCbuild, though; I'd like for the backported files to be the
recommended way to build, complete with support from PCbuild/build.bat
which would make the new project files the default for the buildbots.

I have a work-in-progress branch with the backported files in PCbuild,
which you can find at
https://hg.python.org/sandbox/zware/log/project_files_backport.  There
are still a couple bugs to work out (and a couple unrelated changes to
PC/pyconfig.h), but most everything works as expected.

Looking forward to hearing others' opinions,
--

-- 
Zach
Ivan Levkivskyi | 20 Jun 08:51 2015
Picon

Unbound locals in class scopes

Hello,

There appeared a question in the discussion on http://bugs.python.org/issue24129 about documenting the behavior that unbound local variables in a class definition do not follow the normal rules.

Guido said 13 years ago that this behavior should not be changed: https://mail.python.org/pipermail/python-dev/2002-April/023428.html, however, things changed a bit in Python 3.4 with the introduction of the LOAD_CLASSDEREF opcode. I just wanted to double-check whether it is still a desired/expected behavior. 
<div><div dir="ltr">
<div>Hello,<br><br>
</div>There appeared a question in the discussion on <a href="http://bugs.python.org/issue24129">http://bugs.python.org/issue24129</a> about documenting the behavior that unbound local variables in a class definition do not follow the normal rules.<br><div><div>
<br>Guido said 13 years ago that this behavior should not be changed: <a href="https://mail.python.org/pipermail/python-dev/2002-April/023428.html">https://mail.python.org/pipermail/python-dev/2002-April/023428.html</a>, however, things changed a bit in Python 3.4 with the introduction of the LOAD_CLASSDEREF opcode. I just wanted to double-check whether it is still a desired/expected behavior.&nbsp; <br>
</div></div>
</div></div>
Victor Stinner | 20 Jun 09:30 2015
Picon

PEP 490: Chain exceptions at C level

Hi,

I didn't get much feedback on this PEP. Since the Python 3.6 branch is
open (default), it's probably better to push such change in the
beginning of the 3.6 cycle, to catch issues earlier.

Are you ok to chain exceptions at C level by default?

Relatedi issue: https://bugs.python.org/issue23763

Victor

PEP: 490
Title: Chain exceptions at C level
Version: $Revision$
Last-Modified: $Date$
Author: Victor Stinner <victor.stinner <at> gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 25-March-2015
Python-Version: 3.6

Abstract
========

Chain exceptions at C level, as already done at Python level.

Rationale
=========

Python 3 introduced a new killer feature: exceptions are chained by
default, PEP 3134.

Example::

    try:
        raise TypeError("err1")
    except TypeError:
        raise ValueError("err2")

Output::

    Traceback (most recent call last):
      File "test.py", line 2, in <module>
        raise TypeError("err1")
    TypeError: err1

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "test.py", line 4, in <module>
        raise ValueError("err2")
    ValueError: err2

Exceptions are chained by default in Python code, but not in
extensions written in C.

A new private ``_PyErr_ChainExceptions()`` function was introduced in
Python 3.4.3 and 3.5 to chain exceptions. Currently, it must be called
explicitly to chain exceptions and its usage is not trivial.

Example of ``_PyErr_ChainExceptions()`` usage from the ``zipimport``
module to chain the previous ``OSError`` to a new ``ZipImportError``
exception::

    PyObject *exc, *val, *tb;
    PyErr_Fetch(&exc, &val, &tb);
    PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);
    _PyErr_ChainExceptions(exc, val, tb);

This PEP proposes to also chain exceptions automatically at C level to
stay consistent and give more information on failures to help
debugging. The previous example becomes simply::

    PyErr_Format(ZipImportError, "can't open Zip file: %R", archive);

Proposal
========

Modify PyErr_*() functions to chain exceptions
----------------------------------------------

Modify C functions raising exceptions of the Python C API to
automatically chain exceptions: modify ``PyErr_SetString()``,
``PyErr_Format()``, ``PyErr_SetNone()``, etc.

Modify functions to not chain exceptions
----------------------------------------

Keeping the previous exception is not always interesting when the new
exception contains information of the previous exception or even more
information, especially when the two exceptions have the same type.

Example of an useless exception chain with ``int(str)``::

    TypeError: a bytes-like object is required, not 'type'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: int() argument must be a string, a bytes-like object or
a number, not 'type'

The new ``TypeError`` exception contains more information than the
previous exception. The previous exception should be hidden.

The ``PyErr_Clear()`` function can be called to clear the current
exception before raising a new exception, to not chain the current
exception with a new exception.

Modify functions to chain exceptions
------------------------------------

Some functions save and then restore the current exception. If a new
exception is raised, the exception is currently displayed into
sys.stderr or ignored depending on the function.  Some of these
functions should be modified to chain exceptions instead.

Examples of function ignoring the new exception(s):

* ``ptrace_enter_call()``: ignore exception
* ``subprocess_fork_exec()``: ignore exception raised by enable_gc()
* ``t_bootstrap()`` of the ``_thread`` module: ignore exception raised
  by trying to display the bootstrap function to ``sys.stderr``
* ``PyDict_GetItem()``, ``_PyDict_GetItem_KnownHash()``: ignore
  exception raised by looking for a key in the dictionary
* ``_PyErr_TrySetFromCause()``: ignore exception
* ``PyFrame_LocalsToFast()``: ignore exception raised by
  ``dict_to_map()``
* ``_PyObject_Dump()``: ignore exception. ``_PyObject_Dump()`` is used
  to debug, to inspect a running process, it should not modify the
  Python state.
* ``Py_ReprLeave()``: ignore exception "because there is no way to
  report them"
* ``type_dealloc()``: ignore exception raised by
  ``remove_all_subclasses()``
* ``PyObject_ClearWeakRefs()``: ignore exception?
* ``call_exc_trace()``, ``call_trace_protected()``: ignore exception
* ``remove_importlib_frames()``: ignore exception
* ``do_mktuple()``, helper used by ``Py_BuildValue()`` for example:
  ignore exception?
* ``flush_io()``: ignore exception
* ``sys_write()``, ``sys_format()``: ignore exception
* ``_PyTraceback_Add()``: ignore exception
* ``PyTraceBack_Print()``: ignore exception

Examples of function displaying the new exception to ``sys.stderr``:

* ``atexit_callfuncs()``: display exceptions with
  ``PyErr_Display()`` and return the latest exception, the function
  calls multiple callbacks and only returns the latest exception
* ``sock_dealloc()``: log the ``ResourceWarning`` exception with
  ``PyErr_WriteUnraisable()``
* ``slot_tp_del()``: display exception with
  ``PyErr_WriteUnraisable()``
* ``_PyGen_Finalize()``: display ``gen_close()`` exception with
  ``PyErr_WriteUnraisable()``
* ``slot_tp_finalize()``: display exception raised by the
  ``__del__()`` method with ``PyErr_WriteUnraisable()``
* ``PyErr_GivenExceptionMatches()``: display exception raised by
  ``PyType_IsSubtype()`` with ``PyErr_WriteUnraisable()``

Backward compatibility
======================

A side effect of chaining exceptions is that exceptions store
traceback objects which store frame objects which store local
variables.  Local variables are kept alive by exceptions. A common
issue is a reference cycle between local variables and exceptions: an
exception is stored in a local variable and the frame indirectly
stored in the exception. The cycle only impacts applications storing
exceptions.

The reference cycle can now be fixed with the new
``traceback.TracebackException`` object introduced in Python 3.5. It
stores informations required to format a full textual traceback without
storing local variables.

The ``asyncio`` is impacted by the reference cycle issue. This module
is also maintained outside Python standard library to release a
version for Python 3.3.  ``traceback.TracebackException`` will maybe
be backported in a private ``asyncio`` module to fix reference cycle
issues.

Alternatives
============

No change
---------

A new private ``_PyErr_ChainExceptions()`` function is enough to chain
manually exceptions.

Exceptions will only be chained explicitly where it makes sense.

New helpers to chain exceptions
-------------------------------

Functions like ``PyErr_SetString()`` don't chain automatically
exceptions. To make the usage of ``_PyErr_ChainExceptions()`` easier,
new private functions are added:

* ``_PyErr_SetStringChain(exc_type, message)``
* ``_PyErr_FormatChain(exc_type, format, ...)``
* ``_PyErr_SetNoneChain(exc_type)``
* ``_PyErr_SetObjectChain(exc_type, exc_value)``

Helper functions to raise specific exceptions like
``_PyErr_SetKeyError(key)`` or ``PyErr_SetImportError(message, name,
path)`` don't chain exceptions.  The generic
``_PyErr_ChainExceptions(exc_type, exc_value, exc_tb)`` should be used
to chain exceptions with these helper functions.

Appendix
========

PEPs
----

* `PEP 3134 -- Exception Chaining and Embedded Tracebacks
  <https://www.python.org/dev/peps/pep-3134/>`_ (Python 3.0):
  new ``__context__`` and ``__cause__`` attributes for exceptions
* `PEP 415 - Implement context suppression with exception attributes
  <https://www.python.org/dev/peps/pep-0415/>`_ (Python 3.3):
  ``raise exc from None``
* `PEP 409 - Suppressing exception context
  <https://www.python.org/dev/peps/pep-0409/>`_
  (superseded by the PEP 415)

Python C API
------------

The header file ``Include/pyerror.h`` declares functions related to
exceptions.

Functions raising exceptions:

* ``PyErr_SetNone(exc_type)``
* ``PyErr_SetObject(exc_type, exc_value)``
* ``PyErr_SetString(exc_type, message)``
* ``PyErr_Format(exc, format, ...)``

Helpers to raise specific exceptions:

* ``PyErr_BadArgument()``
* ``PyErr_BadInternalCall()``
* ``PyErr_NoMemory()``
* ``PyErr_SetFromErrno(exc)``
* ``PyErr_SetFromWindowsErr(err)``
* ``PyErr_SetImportError(message, name, path)``
* ``_PyErr_SetKeyError(key)``
* ``_PyErr_TrySetFromCause(prefix_format, ...)``

Manage the current exception:

* ``PyErr_Clear()``: clear the current exception,
  like ``except: pass``
* ``PyErr_Fetch(exc_type, exc_value, exc_tb)``
* ``PyErr_Restore(exc_type, exc_value, exc_tb)``
* ``PyErr_GetExcInfo(exc_type, exc_value, exc_tb)``
* ``PyErr_SetExcInfo(exc_type, exc_value, exc_tb)``

Others function to handle exceptions:

* ``PyErr_ExceptionMatches(exc)``: check to implement
  ``except exc:  ...``
* ``PyErr_GivenExceptionMatches(exc1, exc2)``
* ``PyErr_NormalizeException(exc_type, exc_value, exc_tb)``
* ``_PyErr_ChainExceptions(exc_type, exc_value, exc_tb)``

Python Issues
-------------

Chain exceptions:

* `Issue #23763: Chain exceptions in C
  <http://bugs.python.org/issue23763>`_
* `Issue #23696: zipimport: chain ImportError to OSError
  <http://bugs.python.org/issue23696>`_
* `Issue #21715: Chaining exceptions at C level
  <http://bugs.python.org/issue21715>`_: added
  ``_PyErr_ChainExceptions()``
* `Issue #18488: sqlite: finalize() method of user function may be
  called with an exception set if a call to step() method failed
  <http://bugs.python.org/issue18488>`_
* `Issue #23781: Add private _PyErr_ReplaceException() in 2.7
  <http://bugs.python.org/issue23781>`_
* `Issue #23782: Leak in _PyTraceback_Add
  <http://bugs.python.org/issue23782>`_

Changes preventing to loose exceptions:

* `Issue #23571: Raise SystemError if a function returns a result with an
  exception set <http://bugs.python.org/issue23571>`_
* `Issue #18408: Fixes crashes found by pyfailmalloc
  <http://bugs.python.org/issue18408>`_

Copyright
=========

This document has been placed in the public domain.

..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:

Gmane