Andy Baker | 1 Jul 10:58 2015
Picon

pypy 2.2.1 performance on Raspian

I am trying pypy as a performance increase over CPython.  My code uses standard python libraries (not numpy), plus 2 'C' libraries accessing Raspberry Pi GPIO hardware with python wrappers.  I have compiled both with pypy (sudo pypy setup.py install).  The pypy version runs with no problems but is 2.5 times slower than CPython.  From the performance stats, I believe 2.2.1 should be on average 6 times faster.  I presume it's my 'C' python libraries which are the cause of the reduced performance.  What do I need to do to make these private libraries pypy compatible?
 
Thanks
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Ozan Çağlayan | 29 Jun 15:02 2015
Picon

A simple file reading is 2x slow wrt CPython

Hi,

I just downloaded PyPy 2.6.0 just to play with it.

I have a simple line-by-line file reading example where the file is 324MB.

Code:

# Not doing this import crashes PyPy with MemoryError??
from io import open

a = 0
f = open(fname)
for line in f.readlines():
  a += len(line)
f.close()

PyPy:
Python 2.7.9 (295ee98b69288471b0fcf2e0ede82ce5209eb90b, Jun 01 2015, 17:30:13)
[PyPy 2.6.0 with GCC 4.9.2] on linux2

real 0m6.068s
user 0m4.582s
sys 0m0.846s

CPython (2.7.10)

real 0m3.799s
user 0m2.851s
sys 0m0.860s

Am I doing something wrong or is this expected?

Thanks!

--

-- 
Ozan Çağlayan
Research Assistant
Galatasaray University - Computer Engineering Dept.
http://www.ozancaglayan.com
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Yicong Huang | 25 Jun 12:34 2015
Picon

pypy blocked at RPyGilAcquire ()

Hi,

We embedded Pypy in C++ code, and found out if Pypy was initialized twice it would block at function RpyGilAcquire().
Here is the  case:

rpython_startup_code();
res = pypy_setup_home("...")
res = pypy_execute_source_ptr(...)
//Some code here
rpython_startup_code();
res = pypy_setup_home("...")
res = pypy_execute_source_ptr(...)

We found pypy blocked at second pypy_execute_source_pty(...), the detail call stack are
#0  0x0000003fea40b1c0 in pthread_cond_timedwait <at> <at> GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f8543087a1d in RPyGilAcquire () from /usr/ali/odps-pypy/libpypy-c.so
#2  0x00007f85425a5f8d in pypy_g_pypy_execute_source_ptr () from /usr/ali/odps-pypy/libpypy-c.so
#3  0x00007f85423a6743 in pypy_execute_source_ptr () from /usr/ali/odps-pypy/libpypy-c.so

The reason that we initialized pypy twice is that we would like to get a clean environment to execute without any global variables' pollution from the first run. 
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
lac | 24 Jun 22:58 2015
Picon

[PSF-Board-Public] PSF Meeting Minutes - 2015-06-08 (fwd)

------- Forwarded Message

I've trimmed the minutes.

Return-Path: <psf-board-public-bounces+lac=openend.se <at> python.org>
Received: from mail.python.org (mail.python.org [82.94.164.166])
From: Diana Clarke <diana.joan.clarke <at> gmail.com>
To: Public PSF Board List <psf-board-public <at> python.org>,
        PSF Members List <psf-members <at> python.org>
Subject: [PSF-Board-Public] PSF Meeting Minutes - 2015-06-08

Hi folks:

The first PSF board meeting for the 2015/2016 term was held on June 8,
2015. Those meeting minutes can be found here:

    https://www.python.org/psf/records/board/minutes/2015-06-08/

<snip>

The PSF also formally recognized the Scientific Python Working Group.
More information about the working group can be found here:

    https://wiki.python.org/psf/ScientificWG
    https://pyfound.blogspot.com/2015/05/psfnumfocus-joint-working-group-on.html

For more updates from the PSF, please check out our blog and follow us
on Twitter.

    https://twitter.com/ThePSF
    https://pyfound.blogspot.com/

Cheers,

- --diana
_______________________________________________
PSF-Board-Public mailing list
PSF-Board-Public <at> python.org
https://mail.python.org/mailman/listinfo/psf-board-public

------- End of Forwarded Message

Do we have any presence in that wg?  We should probably have somebody
be a member just to be on top of what the PSF, at any rate, thinks is
scientific python ....

Laura
Laura Creighton | 23 Jun 13:53 2015
Picon

Final CFP: 2015 Workshop on Exascale Multi/many Core Computing Systems (E-MuCoCoS) (fwd)

This showed up in python-list, of all places, apparantly posted to
comp.lang.python

In case somebody wants to go to Portugal in September.

Note: I am still a little hazy on what Exascale means, and therefore
if we do it. :)

Laura

------- Forwarded Message

From: SP <sabri.pllana <at> gmail.com>
Injection-Date: Tue, 23 Jun 2015 09:51:23 +0000

To: python-list <at> python.org

CALL FOR PAPERS

2015 Workshop on Exascale Multi/many Core Computing Systems (E-MuCoCoS)

- - Paper submission deadline: June 30, 2015 (firm deadline)
- - Workshop proceedings are published by the IEEE

* CONTEXT

Exascale computing will revolutionize computational science and
engineering by providing 1000x the capabilities of currently available
computing systems, while having a similar power footprint. The HPC
community is working towards the development of the first Exaflop
computer (expected around 2020) after reaching the Petaflop milestone
in 2008. There are concerns that computer designs based on existing
multi-core and many-core solutions will not scale to Exascale
considering technical challenges (such as, productivity, energy
consumption or reliability) and reasonable economic constraints.
Therefore, novel multi-core and many-core solutions are required to
reach Exascale.

E-MuCoCoS will be organized in conjunction with the 18th IEEE
Conference on Computational Science and Engineering (CSE 2015), Porto,
Portugal, October 21 - 23, 2015.

Previous editions of MuCoCoS workshop include: MuCoCoS 2014 (Porto,
PT), MuCoCoS 2013 (Edinburgh, UK), MuCoCoS 2012 (SLC, US), MuCoCoS
2011 (Seoul, KR), MuCoCoS 2010 (Krakow, PL), MuCoCoS 2009 (Fukuoka,
JP), MuCoCoS 2008 (Barcelona, ES).

* TOPICS OF INTEREST

E-MuCoCoS focuses on multi/many core languages, system software and
architectural solutions towards Exascale computing systems. The topics
of the workshop include but are not limited to:

:: Methods and tools for preparing applications for Exascale
:: Programming models, languages, libraries and compilation techniques
:: Run-time systems and hardware support
:: Patterns, algorithms and data structures
:: Performance analysis, modeling, optimization and tuning

* SUBMISSION GUIDELINES

- - The papers should be prepared using the IEEE format, and no longer
than 6 pages. Submitted papers will be carefully evaluated based on
originality, significance to workshop topics, technical soundness, and
presentation quality.
- - Submission of the paper implies that should the paper be accepted,
at least one of the authors will register and present the paper at the
workshop.
- - Please submit your paper (as PDF) electronically using the online
submission system https://easychair.org/conferences/?conf=emucocos2015

* IMPORTANT DATES

- - Submission: June 30, 2015 (firm deadline)
- - Notification: July 28, 2015
- - Camera ready: September 4, 2015
- - Registration: September 4, 2015
- - Workshop: October 20, 2015

* PROGRAM COMMITTEE

- - Erika Abraham, RWTH Aachen University (DE)
- - Siegfried Benkner, University of Vienna (AT)
- - Eduardo Cesar, UAB (ES)
- - Jiri Dokulil, University of Vienna, (AT)
- - Norbert Eicker, Jlich Supercomputing Centre (DE)
- - Franz Franchetti, Carnegie Mellon University (US)
- - Samir Genaim, Universidad Complutense de Madrid (ES)
- - Houcine Hassan, Universitat Politecnica de Valencia, (ES)
- - Atsushi Hori, RIKEN AICS (JP)
- - Einar Broch Johnsen, University of Oslo, (NO)
- - Christos Kartsaklis, Oak Ridge National Laboratory (US)
- - Jrg Keller, FernUniversitt in Hagen (DE)
- - Christoph Kessler, Linkping University (SE)
- - Joanna Kolodziej, Cracow University of Technology (PL)
- - Ivan Kondov, Karlsruhe Institute of Technology (DE)
- - Erwin Laure, KTH/PDC (SE)
- - Renato Miceli, SENAI CIMATEC (BR)
- - Lasse Natvig, Norwegian University of Science and Technology (NO)
- - Dana Petcu, West University of Timisoara (RO)
- - Uwe Schwiegelshohn, TU Dortmund University (DE)
- - Achim Streit, Karlsruhe Institute for Technology (DE)
- - Osamu Tatebe, University of Tsukuba, (JP)
- - Josef Weidendorfer, TUM (DE)

* WORKSHOP CHAIR

- - Sabri Pllana, Linnaeus University, Sweden
http://homepage.lnu.se/staff/saplaa/

- -- 
https://mail.python.org/mailman/listinfo/python-list

------- End of Forwarded Message
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Derek Lockhart | 22 Jun 17:17 2015
Picon

RFFI Usage and Documentation

After a bit of a struggle I finally got RFFI working for a shared libary I had compiled. I found a few strange discrepancies in the behavior of ExternalCompilitionInfo on OSX I thought may benefit from fixes to make behavior more consistent.

=== RPython Translation vs. Python Execution Strangeness ===

Given a shared library I had compiled named libsoftfloat.so:

- ExternalCompilationInfo( libraries = ['libsoftfloat'] ) and libname == libsoftfloat.so
  - python execution: Error: "cannot find library libsoftfloat"
  - rpython translation: Error: "ld: library not found for -llibsoftfloat"

- ExternalCompilationInfo( libraries = ['softfloat'] ) and libname == libsoftfloat.so
  - python execution: Error: "cannot find library softfloat"
  - rpython translation: Works!

The problem with RFFI during Python execution appears to be that on OSX it checks for the library suffix of .dylib, **not** .so:

- ExternalCompilationInfo( libraries = ['libsoftfloat'] ) and libname == libsoftfloat.dylib
  - python execution: Works!
  - rpython translation: Error: "ld: library not found for -llibsoftfloat"

- ExternalCompilationInfo( libraries = ['softfloat'] ) and libname == libsoftfloat.dylib
  - python execution: Works!
  - rpython translation: Works!

Some naming schemes work on just python execution, some on just rpython translation.  It would be nice if these failed/worked consistently in both modes.  In particular, I think on OSX the library lookup for RFFI should look for both .so and .dylib, not just .dylib since it seems to be fairly common practice to generate .so files?

Also, the libraries naming convention passed to ExternalCompilationInfo is a bit inconsistent/foreign for someone coming from CFFI and using ffi.dlopen().

=== Documentation ===

I really couldn't figure out how to use RFFI from the RPython docs or from the PyPy source.  I started off using these resources:


The only way I really figured out how to use it was from this patch submitted by the libgccjit guy that had some simple examples on how to use RFFI:


What would be really nice is some example of how to write a simple function in C, compile it into a shared library, and use it via RFFI. Even better, an example mapping a CFFI use case to the RFFI interface.  I personally learn how to use APIs much more easily with concrete examples; for example, I created a little git repo as a self-demonstration of how to use the new CFFI APIs with a non-trivial library:


I'd be willing to help a bit with the docs, if it's agreed updates would be appreciated.

Derek
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Armin Rigo | 21 Jun 12:33 2015

tkinter tests broken

Hi Amaury,

In fd331e4bf733 you did an untested change to the tkinter library; as
it turns out, the CPython tests find problems:

http://buildbot.pypy.org/summary/longrepr?testname=unmodified&builder=pypy-c-jit-linux-x86-32&build=2727&mod=lib-python.2.7.test.test_tcl

Can you fix the situation?  Thanks!

Armin
Laura Creighton | 21 Jun 11:52 2015
Picon

Paper on the HOPE JIT for Python

http://www.sciencedirect.com/science/article/pii/S2213133714000687

I think some of our poor performance is because they never let the
jit warm up, but should we grab some of their benchmarks and
see if we can do better?

Laura
Boxiang Sun | 21 Jun 01:33 2015
Picon

Questions about build new extension with RPython

Hi all,

I am a newbie in PyPy. But with some experience in CPython, and also OpenCV-Python binding and OpenCV-Julia binding.I want to bring OpenCV into PyPy. Then use PyPy to do my research.

First, I read some PyPy and RPython document in recent days. I want to wrap OpenCV in PyPy in "low level". Like OpenCV-CPython, it use Python-C API and Numpy C-API to wrap function and class. Then write a autowrap script to wrap it. In this way, OpenCV-CPython can communicate with NumPy array type.

If my understanding is correct. NumPy-PyPy wrote in RPython, wrap the corresponding C and Fortran function. 

I want wrap OpenCV in PyPy just like OpenCV-CPython. Let NumPy ndarray hold the image matrix. But RPython documentation said  "It(Write with RPython) cannot be used by any 3rd-party module". Does it mean if wrap OpenCV in PyPy with cffi. OpenCV-PyPy could not use ndarray?

I plan to use RPython, but RPython documentation said it need rebuild PyPy it self, and the module write in RPython will become a built-in module. Rebuild PyPy seems unacceptable in OpenCV-PyPy.

Is there has a way to wrap OpenCV in PyPy that could communicate with NumPy-PyPy and without rebuild PyPy it self?

Please correct me if my understanding was wrong.

Regards,
Sun
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Glenn Chin | 20 Jun 20:42 2015

"ImportError: No module named httplib2"

On my Ubuntu machine, there’s an  httplib2  directory under

/usr/lib/python2.7/dist-packages .

 

httplib2  imports and works fine in a Python script.  However, the above error message appears when I try to run the same script using  PyPy 2.6 .  I suspect it’s a configuration problem.  Any suggestions?

 

 

Glenn

_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
anatoly techtonik | 19 Jun 17:12 2015
Picon

rogue CFFI speedups #.... <at> k....$...#

Hi,

Just wanted to let you know of awesome 3x
speedup after python-tdl was ported to CFFI.

python-tdl is a libtcod binding for Python.

The wrapped libtcod is the very rad “Doryen Library"
used to build roguelikes in C world. I am sure that
having it in Python will pave a road to more pleasant
text mode interfaces. =)

There were some issues during the porting and
packaging process. I think you might be interested
to check them out:

https://github.com/HexDecimal/python-tdl/issues/5

--

-- 
anatoly t.
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Gmane