Matti Picus | 30 Jan 11:09 2015
Picon

pypy-c failing to find libpypy-c.so on freebsd

The freebsd builds are failing since we changed to --shared by default. 
While translation succeeds, the resulting pypy-c cannot find 
libpypy-c.so even though it seems to be copied properly. See for instance

http://buildbot.pypy.org/builders/pypy-c-jit-freebsd-9-x86-64/builds/462/steps/shell_1/logs/stdio

It seems the $ORIGIN flag is somehow not functioning properly. I found 
this link

http://stackoverflow.com/questions/6324131/rpath-origin-not-having-desired-effect

which would seems to suggest we need to add "-z origin" as well as 
-rpath=$ORIGIN to the linker flags. Could someone (Tobias is the admin 
of the buildbot, but anyone else is welcome to try) with a freebsd 
platform try to track this down? It should be enough to run

python pytest.py rpython/translator/c/test/test_standalone.py -k shared 
--verbose -s

on a pypy default repo after commit fa382e9b1c95, the test should fail. 
Then try to mess with the rpath_flags in 
rpython/translator/platform/posix.py till it passes

Thanks
Matti
Matti Picus | 29 Jan 06:22 2015
Picon

starting 2.5 release cycle, help needed with macos

Buildbots for linux are green (arm and x86), windows seems as good as it 
gets.
I have looked at the open issues, none seem like blockers.
My personal baby, the ufuncapi branch, seems to be functioning after I 
found the "last bug"
So I guess it is time to start the 2.5 release cycle, unless I missed 
something.

We have a persistent crash with macos nightly builds in the 
_continuation module, help is needed to track it down
http://buildbot.pypy.org/summary?builder=pypy-c-jit-macosx-x86-64

Armin suggested maybe it was shadowstack, I think I ruled that out by 
translating on x86 linux:
https://gist.github.com/mattip/8407b7fa7dbe1cc2f786

Any help/criticism/comments are welcome
Matti
Ho33e5 | 25 Jan 00:05 2015
Picon

rpython and pep 484

Hi everybody,

firstly, this is just an email for personal interest and has nothing to do directly
with development so this mailing list may not be quite the right place (I am going
to hang around on #pypy...). I am a student and generally interested with the pypy
development, especially with the rpython language, and I have some general questions:

What is your view on the new typing/mypy things that are happening on python-dev
(pep 484)? What I mean is will this make the typing system of rpython evolve? Could
RTyper be adapted to work on pep 484 annotations (would it actually be useful)? I
read a bit of the paper about rpython listed on the docs and i had the feeling that
your typing is a bit more low level. The quite different goals and contraints that the
2 type system have may explain that they look different, but could there be an
interaction (in one way or another)?

An other question that is related: it's maybe early to think about that but could it be
reasonable to expect that pypy will better optimize pep-484-annotated python programs?
The thrusting of these user annotations is indeed a problem, so a pypy option could
specify that we want it to thrust the type annotations. It may then be worth just writing
programs in rpython directly. 

These questions are quite hypothetical so I don't expect concrete answers, just thoughts!
If someone wants to react to this or point me to other (theoretical) ressources about
rpython... :)

Bonsoir,
Peio
Armin Rigo | 23 Jan 10:28 2015

errno and GetLastError in RPython

Hi all,

I recently merged the "errno-again" branch.  This branch moves the
reading/saving of errno (and on Windows Get/SetLastError) closer to
the actual function call.  It should avoid bugs about rarely getting
the wrong value for errno, in case "something special" happened: for
example, it was not impossible that a call to malloc would invoke the
GC at precisely the wrong spot, which might need to ask the OS for
more memory, which would overwrite the current value of errno.  The
bug actually showed up on the Windows buildbots for GetLastError(),
which would in some cases incorrectly return 0 just if the code
happened to be JIT-traced (not before and not after).  This is now
fixed.

It means any RPython project needs to be updated when it upgrades to
the trunk version of the RPython translation toolchain.  The fix is
rather mechanical.

Replace rposix.get_errno() with rposix.get_saved_errno().
Importantly, review each place that you change.  You need to make sure
which external function call is done before (usually in the few lines
before).  Once you're sure which function's errno is being checked, go
to the declaration of that function, which should be using
rffi.llexternal().  Add the keyword argument
"save_err=rffi.RFFI_SAVE_ERRNO".  This forces errno to be saved
immediately after the function call, into the so-called "saved errno".
This "saved errno" is another thread-local variable, which
rposix.get_saved_errno() returns.

Similarly with rwin32.GetLastValue() -> rwin32.GetLastValue_saved() +
(Continue reading)

Armin Rigo | 21 Jan 07:23 2015

OrderedDict.move_to_end()

Hi all,

About the new OrderedDict and how to support `move_to_end(last=False)`
in the py3k branch: an implementation of the correct complexity is
possible.  It would piggy-back on the part of `lookup_function_no`
that acts as counter for how many entries at the start are known to be
deleted.  This number is present to allow for a good implementation of
`popitem(last=False)`, so that it doesn't have to scan a larger and
larger area of deleted items.

So you can use the same number in reverse.  As long as this number is
greater than zero, you can insert the new item at position "this
number minus one".  When it is zero, you resize and reindex the
dictionary by adding an extra argument to the relevant functions which
would force it to artificially reserve n free entries at the start.
If n is proportional to "num_live_items", maybe 1/8 or 1/16 of it, it
should be enough to give amortized constant time to the operation.

A bientôt,

Armin.
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Steven Jackson | 21 Jan 01:57 2015
Picon

Numpy Topics


Hey I'd like to know if the proposed numpy projects list at https://bitbucket.org/pypy/extradoc/src/extradoc/planning/micronumpy.txt is still up to date, and if so what is meant by "a good sort function."
If it's just a matter of implementing a known good algorithm, that seems like a good way to start contributing to pypy.

The advice on http://doc.pypy.org/en/latest/project-ideas.html suggested posting this question to #pypy on IRC, which I attempted to do through http://webchat.freenode.net/ but I never got a response. It was my first time trying to communicate over IRC, so I'm not sure if I did something incorrectly while trying to join the channel (I saw buildbot messages but no one else speaking) or if the lack of activity was simply due to time difference (I'm on USA east coast time, while I'm aware that much of the pypy-dev community is located in Europe).

Any help with either the original question or joining the IRC discussion would be greatly appreciated :)

--
Steven Jackson
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Timothy Baldridge | 20 Jan 15:59 2015
Picon

Sudden failures during compile-c

Recently my builds on linux with --opt=jit have started failing with the following error:

 
[translation:info] Error:

[translation:info]    File "/home/travis/build/pixie-lang/externals/pypy/rpython/translator/goal/translate.py", line 316, in main

[translation:info]     drv.proceed(goals)

[translation:info]    File "/home/travis/build/pixie-lang/externals/pypy/rpython/translator/driver.py", line 539, in proceed

[translation:info]     return self._execute(goals, task_skip = self._maybe_skip())

[translation:info]    File "/home/travis/build/pixie-lang/externals/pypy/rpython/translator/tool/taskengine.py", line 114, in _execute

[translation:info]     res = self._do(goal, taskcallable, *args, **kwds)

[translation:info]    File "/home/travis/build/pixie-lang/externals/pypy/rpython/translator/driver.py", line 276, in _do

[translation:info]     res = func()

[translation:info]    File "/home/travis/build/pixie-lang/externals/pypy/rpython/translator/driver.py", line 505, in task_compile_c

[translation:info]     cbuilder.compile(**kwds)

[translation:info]    File "/home/travis/build/pixie-lang/externals/pypy/rpython/translator/c/genc.py", line 375, in compile

[translation:info]     extra_opts)

[translation:info]    File "/home/travis/build/pixie-lang/externals/pypy/rpython/translator/platform/posix.py", line 198, in execute_makefile

[translation:info]     self._handle_error(returncode, stdout, stderr, path.join('make'))

[translation:info]    File "/home/travis/build/pixie-lang/externals/pypy/rpython/translator/platform/__init__.py", line 151, in _handle_error

[translation:info]     raise CompilationError(stdout, stderr)

[translation:ERROR] CompilationError: CompilationError(err="""

[translation:ERROR] pixie_vm_threads.c: In function ‘pypy_g_do_yield_thread’:

[translation:ERROR] pixie_vm_threads.c:475:21: error: ‘pypy_g_do_yield_thread_reload’ undeclared (first use in this function)

[translation:ERROR] pixie_vm_threads.c:475:21: note: each undeclared identifier is reported only once for each function it appears in

[translation:ERROR] make[1]: *** [pixie_vm_threads.gcmap] Error 1

[translation:ERROR] """)

[translation] start debugger...

> /home/travis/build/pixie-lang/externals/pypy/rpython/translator/platform/__init__.py(151)_handle_error()

-> raise CompilationError(stdout, stderr)

(Pdb+)

I find this odd as it works just fine without the JIT, and compiles fine on OS X. The code in question is basically a copy-and-paste from PyPy's code: https://github.com/pixie-lang/pixie/blob/master/pixie/vm/threads.py#L90

Any ideas why this would suddenly have started failing recently? I normally build against a pretty recent version of PyPy master, so did something change in the pypy source?

Thanks again for any help,

Timothy
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Omer Katz | 20 Jan 14:14 2015
Picon

cppyy questions

I'm trying to use protobuf with PyPy and I've been quite successful doing so with cppyy.
I generated the protobuf in C++ and used reflex to generate the bindings.
I've encountered some problems that I don't know how to deal with and the documentation doesn't describe what you can do to resolve them.

I discovered that If you don't specify --deep when generating the reflex bindings and you try to pass a string to the C++ side you get a segfault.
I'm guessing that's a bug.

I can't catch exceptions that are being raised from C++ (it's also undocumented).
I ensured that protobuf's FatalException has reflex bindings but the process crashes on an exception.

e.SerializeAsString()
[libprotobuf FATAL google/protobuf/message_lite.cc:273] CHECK failed: IsInitialized(): Can't serialize message of type "MyProtobufType" because it is missing required fields: header
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  CHECK failed: IsInitialized(): Can't serialize message of type "MyProtobufType" because it is missing required fields: header
Aborted (core dumped)

How can I catch that exception?


The documentation is unclear how you can pass a pointer to a Python variable e.g.:
str = ""

e.SerializeToString(str)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-993880892d74> in <module>()
----> 1 e.SerializeToString(str)

TypeError: none of the 5 overloaded methods succeeded. Full details:
  bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
    TypeError: cannot pass str as basic_string<char>
  bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
    TypeError: cannot pass str as basic_string<char>
  bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
    TypeError: cannot pass str as basic_string<char>
  bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
    TypeError: cannot pass str as basic_string<char>
  bool google::protobuf::MessageLite::SerializeToString(std::string*) =>
    TypeError: cannot pass str as basic_string<char>

Best Regards,
Omer Katz.
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Mike Kaplinskiy | 20 Jan 05:26 2015
Picon

RFC: Copy-on-write list slices

Hey folks,

https://bitbucket.org/mikekap/pypy/commits/b774ae0be11b2012852a175f4bae44841343f067 has an implementation of list slicing that copies the data on write. (The third idea from http://doc.pypy.org/en/latest/project-ideas.html .) It's a first pass (and also my first time working on the pypy codebase), so I wanted to solicit some feedback. I'm curious if this was even the right direction and if I'm actually breaking/slowing something down without realizing it.

Also would anyone happen to know some representative stress/performance tests I could run? I ran some simple tests myself (some things got slightly faster), but I doubt that's enough :)

Thanks,
Mike.

(Aside: there is a pull request <at>  https://bitbucket.org/pypy/pypy/pull-request/282/add-a-copy-on-write-slice-list-strategy/diff for this commit, but I clearly messed something up with hg - the diff is from an earlier copy and bitbucket doesn't seem to want to pick it up.)
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev
Matti Picus | 19 Jan 21:37 2015
Picon

pypy2.5 with stdlib-2.7.9?

I would like to start a release cycle of pypy 2.5.0 which seems to be 
quite a jump from 2.4 in terms of performance. The major blocker for me 
is stdlib-2.7.9, especially the improved ssl support. Could we get a 
show of hands for:
- yes I will make an effort to help finish stdlib-2.7.9
- nah, just give up and release with stdlib-2.7.8

It would be nice to get the version out before the sprint

Matti
If there are other blocker issues and/or branches, now is the time to 
mention them

Send desktop notifications to everyone that downloads your software.

Be like AVG, Zone Alarm and Microsoft and send Desktop Notifications to everyone that downloads your software.
 
AVG, Zone Alarm and Microsoft and many other major software sellers now send full colour desktop notifications to everyone that downloads their software.
 
These notifications appear directly onto their desktops informing them about upgrades, new products and server notifications and more.
 
These companies can communicate with everyone that downloads their software.
 
You have probably seen them and wondered how they did that.
 
As a result they have found that their sales have increased and they have a high customer satisfaction rate.
 
You can now do the same!
 
Using our Dymantex desktop messaging system that is even better than the ones used by the major companies you can send full colour interactive desktop notifications so that you too can offer upgrades, tell them about special offers and give technical support to everyone that downloads your software!
 
We offer you a Free, no obligation trial of our
Dymantex Desktop Notification system.
 
Please Press Here for more information and we send it to you.
 
Dymantex
The Desktop Notification System
 
 
This is a B2B comminication. If received in error please accept our apologise
_______________________________________________
pypy-dev mailing list
pypy-dev <at> python.org
https://mail.python.org/mailman/listinfo/pypy-dev

Gmane