Andrew Francis | 1 Nov 14:26 2008
Picon

Rpython-mode Re: Problems getting Translate.py and Pylint to work


Hello Maciej and Adrien:

I am new to PyPy so excuse me for silly questions....

> > I am trying to get RPython to work. I get the
> following problem:

...

> You cannot use a compiler package when writing rpython
> code. I'm not
> sure what you're trying to do, but if your goal is to
> translate
> existing python program using rpython toolchain, this will
> not work
> (almost certainly).

I was testing out RPython by following Christopher Armstrong's tutorial "How to do Something in RPython"
(http://radix.twistedmatrix.com/2006/12/how-to-do-something-with-rpython-part-1.html).
Have things changed?

> This is indeed not the right place to ask :-). Try
> python-projects <at> logilab.org instead for any pylint related
> question.
> Juste to make a quick answer, your problem is probably just
> that you didn't install the logilab.common package
> (http://www.logilab.org/project/logilab-common)

Thanks for the hint. I'll ask future questions in the Logilab mailing list. The main reason I asked here is
(Continue reading)

Armin Rigo | 1 Nov 18:05 2008

Re: Problems getting Translate.py and Pylint to work

Hi Andrew,

On Fri, Oct 31, 2008 at 12:45:00PM -0700, Andrew Francis wrote:
> [translation:ERROR]     from compiler import parse, ast, pycodegen

"from compiler import parse" works for me on all Python versions 2.2 to
2.6.  Either your Python installation is broken, or you have a module
called "compiler.py" that hides the package "compiler" from the stdlib.

Armin
_______________________________________________
pypy-dev <at> codespeak.net
http://codespeak.net/mailman/listinfo/pypy-dev

Andrew Francis | 3 Nov 22:32 2008
Picon

Questions about Moving from Stackless to PyPy

Hi Folks:

I am really new to PyPy. However I have been using Stackless. I would like to do two things. First convert a
pre-existing Python programme, A scaled down WS-BPEL processor to RPython and in the process use the
Stackless module. I would want the end-product to be a C programme or a C library. I am interested stuff like
1) Will the Picking still work? 2) How to glue this to Twisted?

The second thing I would like to do if the first part works - is convert WS-BPEL code (WS-BPEL is an XML based
programming language) into a C library too (I have write a very crude WS-BPEL to Python translator).

Is what I want to do viable? What are some of things the pitfalls? Thanks in advance.

Cheers,
Andrew
Andrew Francis | 3 Nov 21:34 2008
Picon

Re: Problems getting Translate.py and Pylint to work

Hello Armin:

Thanks! Pylint now runs. The problem was I had a compiler package. I also managed to get translate to compile
the Christopher Armstrong example. Sweet!

Cheers,
Andrew 

--- On Sat, 11/1/08, Armin Rigo <arigo <at> tunes.org> wrote:

> From: Armin Rigo <arigo <at> tunes.org>
> Subject: Re: [pypy-dev] Problems getting Translate.py and Pylint to work
> To: "Andrew Francis" <andrewfr_ice <at> yahoo.com>
> Cc: pypy-dev <at> codespeak.net
> Date: Saturday, November 1, 2008, 10:05 AM
> Hi Andrew,
> 
> On Fri, Oct 31, 2008 at 12:45:00PM -0700, Andrew Francis
> wrote:
> > [translation:ERROR]     from compiler import parse,
> ast, pycodegen
> 
> "from compiler import parse" works for me on all
> Python versions 2.2 to
> 2.6.  Either your Python installation is broken, or you
> have a module
> called "compiler.py" that hides the package
> "compiler" from the stdlib.
> 
> 
(Continue reading)

Daniel Furrer | 14 Nov 15:48 2008
Picon

compiler optimizations: collecting ideas

[This message has been posted to python-dev as well. Sorry for the duplication to those who read both.]

Hello everybody,

As part of an advanced compiler design course at our university (ETH Zurich), we have to implement an optimization (or various thereof).
I've spoken with a couple of people who are, like me, very fascinated by Python.
So I would just like to ask if somebody has any ideas on what we could do?

Our constraints:
- We are 4 persons, each willing to potentially spend around 4-6 full days of work.
- We've been working on generating Control Flow Graphs, generating Static Single Assignment Form (inserting phi-nodes) and removing it again. We've also implemented techniques like constant folding, copy propagation, Common Subexpression Elimination etc.

We're looking for:
- Hints as to which python compiler would be best to work on. (The official Python compiler package seems very nice to work with actually... [and so does PyPy.])
- If there are any requests in the area of optimization that we could have a look at

Any feedback would be highly appreciated as well as pointers to specific bugs in the tracker and other relevant discussions.

Best regards,
Daniel

_______________________________________________
pypy-dev <at> codespeak.net
http://codespeak.net/mailman/listinfo/pypy-dev
Bartosz SKOWRON | 16 Nov 02:01 2008
Picon

sprint in Wrocław (PL)

Hello folks!

Fijal asked me to write about possibility of making a sprint in Wroclaw.

There is a possibility to set a sprint at Wroclaw University of
Technology campus for free. But we have a chance to reserve a class
only during winter break. This period is between 30.01 - 22.02. I got
a permission to make a sprint from uni's boss but i have to set the
dates. My proposition is 9.02-15.02 2009. What about you?
Are you interested in this sprint? The town is quite cheap (as whole
Eastern Europe is) and there are a lot of hostels/hotels etc. Also
it's very near to Germany and Czech Rep.

Let me know if you have any questions.

I should reserve a class asap but I need the dates and your opinion :)

cheers!
bart/getxsick
_______________________________________________
pypy-dev <at> codespeak.net
http://codespeak.net/mailman/listinfo/pypy-dev

Carl Friedrich Bolz | 17 Nov 14:21 2008
Picon
Picon

Re: sprint in Wroc?aw (PL)

Hi!

Bartosz SKOWRON wrote:
> Fijal asked me to write about possibility of making a sprint in Wroclaw.
> 
> There is a possibility to set a sprint at Wroclaw University of
> Technology campus for free. But we have a chance to reserve a class
> only during winter break. This period is between 30.01 - 22.02. I got
> a permission to make a sprint from uni's boss but i have to set the
> dates. My proposition is 9.02-15.02 2009. What about you?
> Are you interested in this sprint? The town is quite cheap (as whole
> Eastern Europe is) and there are a lot of hostels/hotels etc. Also
> it's very near to Germany and Czech Rep.
> 
> Let me know if you have any questions.
> 
> I should reserve a class asap but I need the dates and your opinion :)

I would definitely like to have a sprint in Wroclaw! However, for me it
would fit better if it were a week later, since our lecture goes till
the 6th of february and the exams are in the week afterwards. You think
that would be possible too?

Cheers,

Carl Friedrich
_______________________________________________
pypy-dev <at> codespeak.net
http://codespeak.net/mailman/listinfo/pypy-dev

Carl Friedrich Bolz | 17 Nov 14:26 2008
Picon
Picon

Re: compiler optimizations: collecting ideas

Hi Daniel,

sorry for the late response.

Daniel Furrer wrote:
> As part of an advanced compiler design course at our university (ETH
> Zurich), we have to implement an optimization (or various thereof).
> I've spoken with a couple of people who are, like me, very fascinated by
> Python.
> So I would just like to ask if somebody has any ideas on what we could do?
> 
> Our constraints:
> - We are 4 persons, each willing to potentially spend around 4-6 full
> days of work.
> - We've been working on generating Control Flow Graphs, generating
> Static Single Assignment Form (inserting phi-nodes) and removing it
> again. We've also implemented techniques like constant folding, copy
> propagation, Common Subexpression Elimination etc.
> 
> We're looking for:
> - Hints as to which python compiler would be best to work on. (The
> official Python compiler package seems very nice to work with
> actually... [and so does PyPy.])
> - If there are any requests in the area of optimization that we could
> have a look at
> 
> Any feedback would be highly appreciated as well as pointers to specific
> bugs in the tracker and other relevant discussions.

I guess the problem for "classical" compiler optimizations applied to a
compiler producing Python bytecode is that most of them are potentially
unsafe. E.g. you cannot do CSE in Python, because any expression can
have arbitrary side-effects. Therefore it is very limited which
optimizations can be applied at all.

PyPy's translation toolchain works in SSI as an intermediate
representation (which is a sub-set of SSA). However, many of the
straightforward optimizations (constant-folding, copy-progragation,
inlining, a form of escape analysis) have already been implemented. Some
things are not done yet, like common subexpression elimination. We
didn't bother to implement them yet, because they are not as useful for
PyPy since we target C, and most C compilers can do theses things for us.

Cheers,

Carl Friedrich
_______________________________________________
pypy-dev <at> codespeak.net
http://codespeak.net/mailman/listinfo/pypy-dev

Paolo Giarrusso | 17 Nov 14:51 2008
Picon

Re: compiler optimizations: collecting ideas

Hi, first I'd like to state I'm not part of the Zurich group, so what
follows are just my 2 cents.

On Mon, Nov 17, 2008 at 14:26, Carl Friedrich Bolz <cfbolz <at> gmx.de> wrote:
> Hi Daniel,

> Daniel Furrer wrote:
>> We're looking for:
>> - Hints as to which python compiler would be best to work on. (The
>> official Python compiler package seems very nice to work with
>> actually... [and so does PyPy.])

> I guess the problem for "classical" compiler optimizations applied to a
> compiler producing Python bytecode is that most of them are potentially
> unsafe. E.g. you cannot do CSE in Python, because any expression can
> have arbitrary side-effects. Therefore it is very limited which
> optimizations can be applied at all.

I guess that is because an object can define a custom '+' operator,
for instance, right?
But aren't those optimizations possible anyway after specialization
and inlining? And anyway, who said that CSE should be applied to the
Python bytecode? In fact, I think both Java and .NET do it inside the
VM.
Then, the next point is that you do not have yet a JIT. But still, I
think it is possible to specialize bytecode to eliminate dynamic
lookups and enable optimizations such as CSE. But the benefit of
specialized bytecode can be significant, I guess, only if the
interpreter is really fast (either a threaded one, or a code-copying
one). Is the PyPy interpreter threaded?

> PyPy's translation toolchain works in SSI as an intermediate
> representation (which is a sub-set of SSA). However, many of the
> straightforward optimizations (constant-folding, copy-progragation,
> inlining, a form of escape analysis) have already been implemented. Some
> things are not done yet, like common subexpression elimination. We
> didn't bother to implement them yet, because they are not as useful for
> PyPy since we target C, and most C compilers can do theses things for us.

--

-- 
Paolo Giarrusso
_______________________________________________
pypy-dev <at> codespeak.net
http://codespeak.net/mailman/listinfo/pypy-dev

Antonio Cuni | 17 Nov 15:05 2008
Picon

Re: compiler optimizations: collecting ideas

Paolo Giarrusso wrote:

> specialized bytecode can be significant, I guess, only if the
> interpreter is really fast (either a threaded one, or a code-copying
> one). Is the PyPy interpreter threaded?

sometime ago I tried to measure if/how much we can gain with a threaded 
interpreter. I manually modified the produced C code to make the main 
loop threaded, but we didn't gain anything; I think there are three 
possible reasons:

1) in Python a lot of opcodes are quite complex and time-consuming, so 
the time spent to dispatch to them is a little percentage of the total 
time spent for the execution

2) due to Python's semantics, it's not possible to just jump from one 
opcode to the next, as we need to do a lot of bookkeeping, like 
remembering what was the last line executed, etc. This means that the 
trampolines at the end of each opcode contains a lot code duplication, 
leading to a bigger main loop, with possibly bad effects on the cache 
(didn't measure this, though)

3) it's possible that I did something wrong, so in that case my 
measurements are completely useless :-).  If anyone wants to try again, 
it cannot hurt.

ciao,
Anto
_______________________________________________
pypy-dev <at> codespeak.net
http://codespeak.net/mailman/listinfo/pypy-dev


Gmane