Re: [Simpy-developer] Fwd: Pickling of generators

Steve,
First of all, many thanks for approaching Brett Cannon. I am glad that he is
not opposed to pickling generators. There certainly is a requirement for it.

Unfortunately, I have no insight into the inner workings of Python at
runtime. I therefore have no idea of a design for pickling generators.

My spec is really just "pickle the generator state (not the code) and
generate code which allows branching (based on state) to the point (yield)
where the generator was at pickling time".

Anybody out there who can help?

I believe that this is worth pursuing, as we have a valid requirement which
can be met.

Klaus Müller

> -----Original Message-----
> From: simpy-developer-bounces <at> lists.sourceforge.net 
> [mailto:simpy-developer-bounces <at> lists.sourceforge.net] On 
> Behalf Of Steven H. Rogers
> Sent: Saturday, February 24, 2007 4:04 AM
> To: Gregory Smith
> Cc: simpy-users <at> lists.sourceforge.net; Klaus Muller; 
> simpy-developer <at> lists.sourceforge.net
> Subject: Re: [Simpy-developer] [Simpy-users] Fwd: Pickling of 
> generators
> 
> Thanks Greg:

Re: [Simpy-developer] Fwd: Pickling of generators

G'day Klaus:

Klaus Muller wrote:
> Steve,
> First of all, many thanks for approaching Brett Cannon. I am glad that he is
> not opposed to pickling generators. There certainly is a requirement for it.
> 
You're quite welcome.  Yes, it would obviously be a useful feature.
> Unfortunately, I have no insight into the inner workings of Python at
> runtime. I therefore have no idea of a design for pickling generators.
> 
I believe the design issue is that to pickle a generator, you need to 
capture everything on the C stack that the generator uses.  This is 
non-trivial.
> My spec is really just "pickle the generator state (not the code) and
> generate code which allows branching (based on state) to the point (yield)
> where the generator was at pickling time".
For a generator, the state can be the entire C stack.  For SimPy this is 
probably not true, but it may still require multiple stack frames.
> 
> Anybody out there who can help?
> 
> I believe that this is worth pursuing, as we have a valid requirement which
> can be met.
Yes, but how.  It appears to me that pickling generators requires 
implementing much, if not all, of Stackless Python.  Why not use it?

Regards,
Steve

Re: [Simpy-developer] Fwd: Pickling of generators

Steve,
I might be happy to use Stackless for SimPy, if it were not for the fact
that there are no binary installers for other platforms than Windows. Not
everybody who wants to use SimPy knows how to access SVN and then compile
and install from source. I don't want to lose the SimPy applications
programmers on Linux and Mac.

Klaus

> -----Original Message-----
> From: Steven H. Rogers [mailto:steve <at> shrogers.com] 
> Sent: Tuesday, March 06, 2007 6:01 AM
> To: Klaus Muller
> Cc: 'Gregory Smith'; simpy-users <at> lists.sourceforge.net; 
> simpy-developer <at> lists.sourceforge.net
> Subject: Re: [Simpy-developer] [Simpy-users] Fwd: Pickling of 
> generators
> 
> G'day Klaus:
> 
> Klaus Muller wrote:
> > Steve,
> > First of all, many thanks for approaching Brett Cannon. I 
> am glad that 
> > he is not opposed to pickling generators. There certainly 
> is a requirement for it.
> > 
> You're quite welcome.  Yes, it would obviously be a useful feature.
> > Unfortunately, I have no insight into the inner workings of 
> Python at 

Re: [Simpy-developer] Fwd: Pickling of generators

Not the entire C stack. generators can call arbitrary functions
(including C extensions), but when they have reached a yield statement,
none of that is
active and Python only has the one stack frame for the coroutine. This
is what allows generators in the first place - yield is treated as a
'return' statement (to the caller of .next()) but the frame of the
active generator is preserved instead of being destroyed as in a normal
return. The frame includes the local variables, current execution
position, and block stack (while,for, try blocks etc).

Multiple threads have multiple C stacks - but generators run on the same
C stack as whichever thread has called 'next()'.

Look at it this way - you can call 15 levels of python (building up a
big
C stack in the process), create a generator, call next() once, then
return
that generator through 15 levels of python, and it still works. Clearly
it's
not tied to the 'under' stack, since that can be reduced arbitrarily and
it
still works - and it's only tied to the 'over' stack when it's actually
being executed (while next() is being called). There is no 'over' stack
when next() returns. 

There is a flag in the generator object which indicates whether the gen.
is running; this is set when next() is active, and is presumably to
detect a case where the generator somehow calls its own 'next' method,
or if another thread tries to call next(), both of which would be
disastrous since you can't share a frame object like that. It would only

Re: [Simpy-developer] Fwd: Pickling of generators

Dear Klaus,

I just compiled the greenlet c module on Windows using Python 2.5 and
MinGW, and it seems to work fine. Could we distribute a compiled version
of just the greenlet module with SimPy (we'd have to include at least
some of the source)?

Best
Simon

Klaus Muller wrote:
> Steve,
> I might be happy to use Stackless for SimPy, if it were not for the fact
> that there are no binary installers for other platforms than Windows. Not
> everybody who wants to use SimPy knows how to access SVN and then compile
> and install from source. I don't want to lose the SimPy applications
> programmers on Linux and Mac.
>
> Klaus
>
>
>
>   
>> -----Original Message-----
>> From: Steven H. Rogers [mailto:steve <at> shrogers.com] 
>> Sent: Tuesday, March 06, 2007 6:01 AM
>> To: Klaus Muller
>> Cc: 'Gregory Smith'; simpy-users <at> lists.sourceforge.net; 
>> simpy-developer <at> lists.sourceforge.net
>> Subject: Re: [Simpy-developer] [Simpy-users] Fwd: Pickling of 

Re: [Simpy-developer] Fwd: Pickling of generators

Gregory Smith wrote:
> Not the entire C stack. generators can call arbitrary functions
> (including C extensions), but when they have reached a yield statement,
> none of that is
> active and Python only has the one stack frame for the coroutine. This
> is what allows generators in the first place - yield is treated as a
> 'return' statement (to the caller of .next()) but the frame of the
> active generator is preserved instead of being destroyed as in a normal
> return. The frame includes the local variables, current execution
> position, and block stack (while,for, try blocks etc).
> 
> Multiple threads have multiple C stacks - but generators run on the same
> C stack as whichever thread has called 'next()'.
> 
> Look at it this way - you can call 15 levels of python (building up a
> big
> C stack in the process), create a generator, call next() once, then
> return
> that generator through 15 levels of python, and it still works. Clearly
> it's
> not tied to the 'under' stack, since that can be reduced arbitrarily and
> it
> still works - and it's only tied to the 'over' stack when it's actually
> being executed (while next() is being called). There is no 'over' stack
> when next() returns. 
> 
> There is a flag in the generator object which indicates whether the gen.
> is running; this is set when next() is active, and is presumably to
> detect a case where the generator somehow calls its own 'next' method,
> or if another thread tries to call next(), both of which would be

Re: [Simpy-developer] Fwd: Pickling of generators

Simon Frost wrote:
> Dear Klaus,
> 
> I just compiled the greenlet c module on Windows using Python 2.5 and
> MinGW, and it seems to work fine. Could we distribute a compiled version
> of just the greenlet module with SimPy (we'd have to include at least
> some of the source)?
> 

Seems to work fine with Python 2.4.4 on Fedora Core 6 as well.

# Steve

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

User interface simulation

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
Simpy-users mailing list
Simpy-users <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/simpy-users

Re: User interface simulation

Shmuel Kessler wrote:
> 
> Does anyone have any comments on whether SimPy is a good candidate to 
> use for a simulation of a user interface of a mobile communications device?
> 

Hi Shmuel:

That depends upon what you want to accomplish with the simulations. 
Could you elaborate?

# Steve

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

Re: [Simpy-developer] Fwd: Pickling of generators

Steven H. Rogers wrote:
> Simon Frost wrote:
>> Dear Klaus,
>>
>> I just compiled the greenlet c module on Windows using Python 2.5 and
>> MinGW, and it seems to work fine. Could we distribute a compiled version
>> of just the greenlet module with SimPy (we'd have to include at least
>> some of the source)?
>>
> 
> Seems to work fine with Python 2.4.4 on Fedora Core 6 as well.
> 

I just found a fly in the greenlet ointment.  According to this post 
(http://www.stackless.com/pipermail/stackless/2007-March/002344.html) 
greenlets don't support pickling.

On the bright side, Stackless in pure Python + greenlets raises the 
possibility of porting SimPy to Stackless without requiring the custom 
interpreter.  Those who need to suspend a running simulation to disc or 
migrate it from one processor to another could use it on full Stackless. 
  Others without this requirement and are unable to use the custom 
Stackless interpreter could use the pure Python + greenlets version. 
This might be a tough choice for some, but I think it would cover most 
use cases.

# Steve

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV