PyPy 2.0.2 released

Hi all,

The bugfix PyPy 2.0.2 has been released (on all platforms but OS/X
which should come later in the day).

=========================
PyPy 2.0.2 - Fermi Panini
=========================

We're pleased to announce PyPy 2.0.2.  This is a stable bugfix release
over 2.0 and 2.0.1.  You can download it here:

    http://pypy.org/download.html

It fixes a crash in the JIT when calling external C functions (with
ctypes/cffi) in a multithreaded context.

What is PyPy?
=============

PyPy is a very compliant Python interpreter, almost a drop-in replacement for
CPython 2.7. It's fast (pypy 2.0 and cpython 2.7.3 performance comparison:
http://speed.pypy.org) due to its integrated tracing JIT compiler.

This release supports x86 machines running Linux 32/64, Mac OS X 64 or
Windows 32.  Support for ARM is progressing but not bug-free yet.

Highlights
==========

Is cpyext dead forever?

cpyext is currently disabled on default. Is there a branch to revive it?
Or are we expecting cffi to replace all interaction with external c code?
I'm not sure what will be more painful, getting the world to rewrite all 
modules or supporting cpyext, both seem non-trivial.
(I'm opening the discussion here since I'm not sure if everyone was 
heard in the discussion on IRC)
Matti

pypy performance on fractal terrain generator.

_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
http://mail.python.org/mailman/listinfo/pypy-dev

PyPy 2.0.1 released

==============================
PyPy 2.0.1 - Bohr Smørrebrød
==============================

We're pleased to announce PyPy 2.0.1.  This is a stable bugfix release
over 2.0.  You can download it here:

    http://pypy.org/download.html

The fixes are mainly about fatal errors or crashes in our stdlib.  See
below for more details.

What is PyPy?
=============

PyPy is a very compliant Python interpreter, almost a drop-in replacement for
CPython 2.7. It's fast due to its integrated tracing JIT compiler.

(pypy 2.0 and cpython 2.7.3 performance comparison: http://speed.pypy.org)

This release supports x86 machines running Linux 32/64, Mac OS X 64 or
Windows 32.  Support for ARM is progressing but not bug-free yet.

Highlights
==========

- fix an occasional crash in the JIT that ends in `RPython Fatal error:
  NotImplementedError` (https://bugs.pypy.org/issue1482).

- `id(x)` is now always a positive number (except on int/float/long/complex).
  This fixes an issue in `_sqlite.py` (mostly for 32-bit Linux).

- fix crashes of callback-from-C-functions (with cffi) when used together with
  Stackless features, on asmgcc (i.e. Linux only).  Now gevent should work
  better (http://mail.python.org/pipermail/pypy-dev/2013-May/011362.html).

- work around an eventlet issue with `socket._decref_socketios()`
  (https://bugs.pypy.org/issue1468).

Cheers,

arigo et. al. for the PyPy team

performance issue with context managers

Hi list,

I was toying around a bit with writing a statistical profiler in python,
and came up with https://gist.github.com/jap/5584946

For reference, the main routine is:

with Profiler() as p:
    with ProfilerContext("c1"):
        s = ""
        for t in range(100000):
            with ProfilerContext("c2"):
                s = s + "a"
            s = s + "b"
        print p.get_data()

When running it on my local machine, this gives the following output:

spaans <at> spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py
Counter({'c1': 638})
6.42user 0.42system 0:07.06elapsed 97%CPU (0avgtext+0avgdata
30160maxresident)k
0inputs+0outputs (0major+8383minor)pagefaults 0swaps
spaans <at> spaans-e6500:/tmp$ /usr/bin/time python pmp.py
Counter({'c1': 18, 'c2': 3})
0.23user 0.02system 0:00.25elapsed 98%CPU (0avgtext+0avgdata
8200maxresident)k
0inputs+0outputs (0major+2226minor)pagefaults 0swaps

So, two things seem to happen: the pypy version is almost 30 times
slower than the python version (but hey, string appending has poor
performance), and it somehow does not trigger the "c2" context..

Is the c2 context supposed to disappear?

If I change the main loop to

        for t in range(100000):
            with ProfilerContext("c2"):
                s = s + "a"

The output is still limited to
spaans <at> spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py
Counter({'c1': 156})
1.61user 0.14system 0:01.78elapsed 98%CPU (0avgtext+0avgdata
30040maxresident)k

which seems credible, because when I change it to

        for t in range(100000):
            s = s + "a"

it's suddenly fast in pypy as well:

0.03user 0.01system 0:00.05elapsed 96%CPU (0avgtext+0avgdata
8024maxresident)k

Removing all the threading.local stuff gives me the following
performance data:

spaans <at> spaans-e6500:/tmp$ /usr/bin/time python pmp.py
Counter({'c1': 12, 'c2': 6})
0.20user 0.01system 0:00.22elapsed 98%CPU (0avgtext+0avgdata
8192maxresident)k
0inputs+0outputs (0major+2224minor)pagefaults 0swaps

spaans <at> spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py
Counter({'c1': 621})
6.18user 0.42system 0:06.76elapsed 97%CPU (0avgtext+0avgdata
30084maxresident)k

which does not seem to differ that much.

Finally, to rule out object creation issues, main was changed to:

with Profiler() as p:
    with ProfilerContext("c1"):
        p2 = ProfilerContext("c2")
        s = ""
        for t in range(100000):
            with p2:
                s = s + "a"
            s = s + "b"
        print p.get_data()

but that still behaves similar to the previous runs:

spaans <at> spaans-e6500:/tmp$ /usr/bin/time ~/xsrc/pypy-2.0/bin/pypy pmp.py
Counter({'c1': 624})
6.25user 0.38system 0:06.69elapsed 99%CPU (0avgtext+0avgdata
29792maxresident)k

spaans <at> spaans-e6500:/tmp$ /usr/bin/time python pmp.py
Counter({'c2': 6, 'c1': 6})
0.14user 0.02system 0:00.16elapsed 98%CPU (0avgtext+0avgdata
8188maxresident)k

(all of this on my trusty old Dell E6500 running Ubuntu 13.04 on amd64)

Could someone help me get "c2" registered, or is this expected behaviour? :)

Cheers,
Jasper

--

-- 
 /\____/\   ir. Jasper Spaans // Lead Developer DetACT
 \   (_)/   Fox-IT - For a more secure society!
  \    X    T: +31-15-2847999       
   \  / \   M: +31-6-41588725
    \/      KvK Haaglanden 27301624

Include .so modules generated with f2py from fortran script

_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
http://mail.python.org/mailman/listinfo/pypy-dev

Issue 1475 (add_memory_pressure())

Hi everybody,

First, let me introduce myself. I'm Bart Wiegmans and I would like to
work for pypy via GSoC this summer. To that end I proposed adding a
concurrent / incremental garbage collector to PyPy.

Reasonably, Fijal requested me to bring in some code before I could be
accepted, and since issue 1475 (https://bugs.pypy.org/issue1475) is
rather memory-related, it seems like a good fit. However, I've failed
to make any progress on it (despite advice). In fact, I'd say that at
this point I simply don't understand PyPy well enough to add this
feature, and I'm not sure what to do next.

Anyway, this way you know that I've tried, at least. If anyone has any
more advice I'll happily try again, of course.
Thank you for your time.

Kind regards,
Bart Wiegmans

Segfault with pypy-2.0, gevent, dnspython

Hi List,

I discovered a segementation fault with pypy-2.0 and gevent, while
playing with the dnspython library. I compiled pypy from the source
tarball downloaded from the pypy website with jit enabled and installed
gevent with pypy support as described in the pypycore repository.

Below is a log of the simple steps needed to reproduce the problem.
From the generated core file, it seems, the problem occurs in the
minimark gc. I also attached the core file to this mail.

Regards,
Emil

(pypy-gevent)emil <at> descartes:~/Play/pypy-gevent/test$ ulimit -c unlimited
(pypy-gevent)emil <at> descartes:~/Play/pypy-gevent/test$ python
Python 2.7.3 (b9c3566aa0170aaa736db0491d542c309ec7a5dc, May 12 2013, 11:22:30)
[PyPy 2.0.0 with GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``All problems in computer science
can be solved by another level of indirection. --Butler Lampson''
>>>> from gevent import monkey; monkey.patch_all()
>>>> from dns.resolver import query
>>>> a = query('www.google.de')
*** using ev loop
>>>> dir(a)
Speicherzugriffsfehler (Speicherabzug geschrieben)
(pypy-gevent)emil <at> descartes:~/Play/pypy-gevent/test$ gdb --core core --args python
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /home/emil/.virtualenvs/pypy-gevent/bin/python...(no debugging symbols found)...done.
[New LWP 3216]

warning: Can't read pathname for load map: Eingabe-/Ausgabefehler.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `python'.
Program terminated with signal 11, Segmentation fault.
#0  0x000000000110b2a4 in pypy_g_walk_to_parent_frame ()
(gdb) bt
#0  0x000000000110b2a4 in pypy_g_walk_to_parent_frame ()
#1  0x000000000110b75c in pypy_g_walk_stack_from ()
#2  0x000000000110b899 in pypy_g__asm_callback ()
#3  0x000000000130b725 in pypy_asm_stackwalk ()
#4  0x0000000001104f32 in pypy_g_MiniMarkGC_minor_collection.part.1 ()
#5  0x00000000011099f8 in pypy_g_MiniMarkGC_collect_and_reserve ()
#6  0x0000000000ee4482 in pypy_g_invalidate_loop ()
#7  0x000000000104f85f in pypy_g_QuasiImmut_invalidate ()
#8  0x0000000000cdbdc3 in pypy_g_ModuleDictStrategy_setitem_str ()
#9  0x0000000000419539 in pypy_g___mm_setitem_W_DictMultiObject_W_Root_W_Root ()
#10 0x0000000000c9a3ff in pypy_g_displayhook ()
#11 0x0000000000849ede in pypy_g_BuiltinCode1_fastcall_1 ()
#12 0x00000000008475b0 in pypy_g_funccall_valuestack__AccessDirect_None ()
#13 0x000000000086a86c in pypy_g_CALL_FUNCTION__AccessDirect_None ()
#14 0x000000000087357f in pypy_g_dispatch_bytecode__AccessDirect_None ()
#15 0x0000000000876583 in pypy_g_handle_bytecode__AccessDirect_None ()
#16 0x0000000000c827a2 in pypy_g_portal_4 ()
#17 0x0000000001067113 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter ()
#18 0x000000000085e826 in pypy_g_PyFrame_run ()
#19 0x000000000080a67b in pypy_g_call_function__star_1 ()
#20 0x0000000000872295 in pypy_g_dispatch_bytecode__AccessDirect_None ()
#21 0x0000000000876583 in pypy_g_handle_bytecode__AccessDirect_None ()
#22 0x0000000000c827a2 in pypy_g_portal_4 ()
#23 0x0000000001067113 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter ()
#24 0x000000000085e826 in pypy_g_PyFrame_run ()
#25 0x000000000086cff4 in pypy_g_EXEC_STMT__AccessDirect_None ()
#26 0x00000000008726aa in pypy_g_dispatch_bytecode__AccessDirect_None ()
#27 0x0000000000876583 in pypy_g_handle_bytecode__AccessDirect_None ()
#28 0x0000000000c827a2 in pypy_g_portal_4 ()
#29 0x0000000001067113 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter ()
#30 0x000000000085e826 in pypy_g_PyFrame_run ()
#31 0x0000000000cda8ba in pypy_g_CALL_METHOD__AccessDirect_star_1 ()
#32 0x0000000000873867 in pypy_g_dispatch_bytecode__AccessDirect_None ()
#33 0x0000000000876583 in pypy_g_handle_bytecode__AccessDirect_None ()
#34 0x0000000000c827a2 in pypy_g_portal_4 ()
#35 0x0000000001067113 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter ()
#36 0x000000000085e826 in pypy_g_PyFrame_run ()
#37 0x0000000000cda8ba in pypy_g_CALL_METHOD__AccessDirect_star_1 ()
---Type <return> to continue, or q <return> to quit---
#38 0x0000000000873867 in pypy_g_dispatch_bytecode__AccessDirect_None ()
#39 0x0000000000876583 in pypy_g_handle_bytecode__AccessDirect_None ()
#40 0x0000000000c827a2 in pypy_g_portal_4 ()
#41 0x0000000001067113 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter ()
#42 0x000000000085e826 in pypy_g_PyFrame_run ()
#43 0x0000000000cda8ba in pypy_g_CALL_METHOD__AccessDirect_star_1 ()
#44 0x0000000000873867 in pypy_g_dispatch_bytecode__AccessDirect_None ()
#45 0x0000000000876583 in pypy_g_handle_bytecode__AccessDirect_None ()
#46 0x0000000000c827a2 in pypy_g_portal_4 ()
#47 0x0000000001067113 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter ()
#48 0x000000000085e826 in pypy_g_PyFrame_run ()
#49 0x000000000086a86c in pypy_g_CALL_FUNCTION__AccessDirect_None ()
#50 0x000000000087357f in pypy_g_dispatch_bytecode__AccessDirect_None ()
#51 0x0000000000876583 in pypy_g_handle_bytecode__AccessDirect_None ()
#52 0x0000000000c827a2 in pypy_g_portal_4 ()
#53 0x0000000001067113 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter ()
#54 0x000000000085e826 in pypy_g_PyFrame_run ()
#55 0x0000000000cb71fd in pypy_g_call_args ()
#56 0x000000000086a74e in pypy_g_call_function__AccessDirect_None ()
#57 0x000000000087375c in pypy_g_dispatch_bytecode__AccessDirect_None ()
#58 0x0000000000876583 in pypy_g_handle_bytecode__AccessDirect_None ()
#59 0x0000000000c827a2 in pypy_g_portal_4 ()
#60 0x0000000001067113 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter ()
#61 0x000000000085e826 in pypy_g_PyFrame_run ()
#62 0x00000000008473d3 in pypy_g_funccall_valuestack__AccessDirect_None ()
#63 0x000000000086a86c in pypy_g_CALL_FUNCTION__AccessDirect_None ()
#64 0x000000000087357f in pypy_g_dispatch_bytecode__AccessDirect_None ()
#65 0x0000000000876583 in pypy_g_handle_bytecode__AccessDirect_None ()
#66 0x0000000000c827a2 in pypy_g_portal_4 ()
#67 0x0000000001067113 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter ()
#68 0x000000000085e826 in pypy_g_PyFrame_run ()
#69 0x0000000000cb71fd in pypy_g_call_args ()
#70 0x000000000086a74e in pypy_g_call_function__AccessDirect_None ()
#71 0x000000000087370f in pypy_g_dispatch_bytecode__AccessDirect_None ()
#72 0x0000000000876583 in pypy_g_handle_bytecode__AccessDirect_None ()
#73 0x0000000000c827a2 in pypy_g_portal_4 ()
#74 0x0000000001067113 in pypy_g_ll_portal_runner__Unsigned_Bool_pypy_interpreter ()
#75 0x000000000085e826 in pypy_g_PyFrame_run ()
---Type <return> to continue, or q <return> to quit---
#76 0x000000000080a9ce in pypy_g_call_function__star_2 ()
#77 0x0000000000763aea in pypy_g_entry_point ()
#78 0x0000000001306234 in pypy_main_function ()
#79 0x00007f84c489476d in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#80 0x0000000000411c91 in _start ()
(gdb) quit

-- 
Emil Kroymann
VoIP Services Engineer

Email: emil.kroymann <at> isaco.de
Tel: +49-30-203899885
Mobile: +49-151-62820588

ISACO GmbH
Kurfürstenstraße 79
10787 Berlin
Germany

Amtsgericht Charlottenburg, HRB 112464B
Geschäftsführer: Daniel Frommherz

_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
http://mail.python.org/mailman/listinfo/pypy-dev

cffi wrapper for libeinspline, a cubic B-spline library, runs on Pypy

Hi group!

I've been working on wrapping libeinspline, a comprehensive C library 
for creating and evaluating cubic B-splines in 1-3 dimensions and real 
and complex data types. The wrap is done using cffi, and it runs with 
Pypy and Cpython. I've now put an alpha version online at

https://bitbucket.org/kfj/python-bspline

I am pointing you to this project since it actually works well with Pypy 
and has a few bits in it which you may find inspiring. I'd welcome 
comments. I wrote the wrapper because I wanted B-splines for a Pypy pet 
project of mine and couldn't find anything suitable. It may be of use 
while a Pypy version of scipy hasn't materialized, and even has a few 
bits which I haven't found in scipy.

Kay F. Jahnke

Using "rpath"

Hi all,

I think I know how to solve the two linking issues we were discussing
on irc yesterday: using the "rpath" (thanks squeaky_pl for originally
pointing it out).  See for example
http://stackoverflow.com/a/6323222/1556290 .

1) This could let us distribute more self-contained executables by
putting the libffi.so, libssl.so, etc. along it in the same directory,
and specifying an rpath of "$ORIGIN".

2) This could be used for building in "--shared" mode, with a small
executable and a big libpypy.so in the same directory, again using the
rpath "$ORIGIN".

A bientôt,

Armin.

SSL version?

I get an error when trying to run pypy 2.0 on a Debian Wheezy system:

dstromberg <at> deskie:~/src/home-svn/backshift/trunk$ /usr/local/pypy-2.0/bin/pypy
/usr/local/pypy-2.0/bin/pypy: error while loading shared libraries:
libssl.so.0.9.8: cannot open shared object file: No such file or
directory
dstromberg <at> deskie:~/src/home-svn/backshift/trunk$

Isn't libssl 0.9.8 getting kind of old?  I don't see it in synaptic.