headers
Tim Peters | 1 Jul 1999 06:30
Picon

RE: Fake threads (was ActiveState & fork & Perl)

[Guido]
> I guess it's all in the perspective.  99.99% of all thread apps I've
> ever written use threads primarily to overlap I/O -- if there wasn't
> I/O to overlap I wouldn't use a thread.  I think I share this
> perspective with most of the thread community (after all, threads
> originate in the OS world where they were invented as a replacement
> for I/O completion routines).

Different perspective indeed!  Where I've been, you never used something as
delicate as a thread to overlap I/O, you instead used the kernel-supported
asynch Fortran I/O extensions <0.7 wink>.

Those days are long gone, and I've adjusted to that.  Time for you to leave
the past too <wink>:  by sheer numbers, most of the "thread community"
*today* is to be found typing at a Windows box, where cheap & reliable
threads are a core part of the programming culture.  They have better ways
to overlap I/O there too.  Throwing explicit threads at this is like writing
a recursive Fibonacci number generator in Scheme, but building the recursion
yourself by hand out of explicit continuations <wink>.

> ...
> As far as I can tell, all the examples you give are easily done using
> coroutines.  Can we call whatever you're asking for coroutines instead
> of fake threads?

I have multiple agendas, of course.  What I personally want for my own work
is no more than Icon's generators, formally "semi coroutines", and easily
implemented in the interpreter (although not the language) as it exists
today.
(Continue reading)

Permalink | Reply | 24 comments |
headers
Tim Peters | 1 Jul 1999 09:45
Picon

RE: ob_refcnt access

[Mark Hammond]
> Im a little unhappy as this [stackless Python] will break the Active
> Debugging stuff ...
> ...
> So the solution MS came up with was, surprise surprise, the machine stack!
> :-)  The assumption is that all languages will make _some_ use of the
> stack, so they ask a language to report its "stack base address"
> and "stack size".  Using this information, the debugger sorts into the
> correct call sequence.

Mark, you can't *really* believe Chris is incapable of hacking around this,
right?

It's not even clear there's something to be hacked around, since Python is
only Python and there's nothing Christian can do to stop other languages
that call into Python from using the machine stack, or to call other
languages from Python without using the machine stack.  So Python "shows up
on the stack" no matter what, cross-language.

> ...
> Bit I also understand completely the silence on this issue.  When the
> thread started, there was much discussion about exactly what the
> hell these continuation/coroutine thingies even were.

The Fuchs paper Sam referenced explained it in simple C terms:  a
continuation is exactly what C setjmp/longjmp would do if setjmp saved (&
longjmp restored) the C stack *in addition* to the program counter and
machine registers (which they already save/restore).  That's all there is to
it, at heart:  objects capture data state, continuations capture control
flow state.  Whenever the OS services an interrupt and drops into kernel
(Continue reading)

Permalink | Reply | 1 comment |
headers
Mark Hammond | 1 Jul 1999 10:18
Picon
Favicon

RE: ob_refcnt access

[Tim tells me it will all be obvious if I just think a little harder
<wink>]

Your points about "acedemic examples" is well taken.  The reality is that,
even given these simple examples (which I dared deride as acedemic), the
simple fact is Im not seeing "the point".

I seriously dont doubt all all you say.  However, as Sam and Chris have
said many times, it is just a matter of changing the way to you think.

Interestingly:
Chris said it recently, and continues to say it.
Sam said it to me _years_ ago, and said it repeatedly, but hasnt said it
recently.
Tim hasnt really said it yet :-)

This is almost certainly because when your brain does switch, it is a
revelation, and really not too hard at all.  But after a while, you forget
the switch ever took place.

Closest analogy I can think of is OO programming.  In my experience trying
to _learn_ OO programming from a few misc examples and texts was pointless
and very hard.  You need a language to play with it in.  And when you have
one, your brain makes the switch, you see the light, and you can't see what
was ever mysterious about it.  And you tell everyone its easy; "just change
the way you think about data" :-)
But to all us here, OO programming is just so obvious it goes without
saying.  Occasionaly a newbie will have trouble with OO concepts in Python,
and I personally have trouble seeing what could _possibly_ be difficult
about understanding these very simple concepts.  So Im just as guilty, just
(Continue reading)

Permalink | Reply | 0 comments |
headers
Jack Jansen | 1 Jul 1999 18:05
Picon

Re: Paul Prescod: add Expat to 1.6

Recently, Skip Montanaro <skip <at> mojam.com> said:
> 
>     Andrew> My personal leaning is that we can get more bang for the buck by
>     Andrew> working on the Distutils effort, so that installing a package
>     Andrew> like PyExpat becomes much easier, rather than piling more things
>     Andrew> into the core distribution.
> 
> Amen to that.  See Guido's note and my response regarding soundex in the
> Doc-SIG.  Perhaps you could get away with a very small core distribution
> that only contained the stuff necessary to pull everything else from the net 
> via http or ftp...

I don't know whether this subject belongs on the python-dev list (is
there a separate distutils list?), but let's please be very careful
with this. The Perl people apparently think that their auto-install
stuff is so easy to use that if you find a tool on the net that needs
Perl they'll just give you a few incantations you need to build the
"correct" perl to run the tool, but I've never managed to do so. My
last try was when I spent 2 days to try and get the perl-based Palm
software for unix up and running. With various incompatilble versions
of perl installed in /usr/local by the systems staff and knowing
nothing about perl I had to give up at some point, because it was
costing far more time (and diskspace:-) than the whole thing was
worth.

Something like mailman is (afaik) easy to install for non-pythoneers
because it only depends on a single, well-defined Python distribution.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen <at> oratrix.com | ++++ if you agree copy these lines to your sig ++++
(Continue reading)

Permalink | Reply | 2 comments |
headers
Skip Montanaro | 1 Jul 1999 21:54

Re: Paul Prescod: add Expat to 1.6

    Skip> Amen to that.  See Guido's note and my response regarding soundex
    Skip> in the Doc-SIG.  Perhaps you could get away with a very small core
    Skip> distribution that only contained the stuff necessary to pull
    Skip> everything else from the net via http or ftp...

    Jack> I don't know whether this subject belongs on the python-dev list
    Jack> (is there a separate distutils list?), but let's please be very
    Jack> careful with this. The Perl people apparently think that their
    Jack> auto-install stuff is so easy to use ...

I suppose I should have added a <0.5 wink> to my note.  Still, knowing what
Guido does and doesn't feel comfortable with in the core distribution would
be a good start at seeing where we might like the core to wind up.

Skip Montanaro	| http://www.mojam.com/
skip <at> mojam.com  | http://www.musi-cal.com/~skip/
518-372-5583
Permalink | Reply | 0 comments |
headers
Tim Peters | 2 Jul 1999 04:33
Picon

RE: Paul Prescod: add Expat to 1.6

[large vs small distributions]

[Jack Jansen]
> I don't know whether this subject belongs on the python-dev list (is
> there a separate distutils list?), but let's please be very careful
> with this. [and recounts his problems with Perl]

I must say the idea of a minimal distribution sounds very appealing.  But
then I consider that Guido never got me to even try Tk until he put it into
the std Windows distribution, and I've never given anyone any code that
won't work with a fresh-from-the-box distribution either.  FrankS's snappy
"batteries included" wouldn't carry quite the same punch if it got reduced
to "coupons for batteries hidden in the docs" <wink>.

OTOH, I've got about as much use for XML as MarkH has for continuations
<twist>, and here-- as in many other places --we've been saved so far by
Guido's good judgment about what goes in & what stays out.

So it's a good thing he can't ever resign this responsibility <wink>.

if-20%-of-users-need-something-i'd-include-it-else-not-ly y'rs  - tim
Permalink | Reply | 0 comments |
headers
Guido van Rossum | 4 Jul 1999 03:56
Picon
Picon

Re: Fake threads (was ActiveState & fork & Perl)

> [Guido]
> > I guess it's all in the perspective.  99.99% of all thread apps I've
> > ever written use threads primarily to overlap I/O -- if there wasn't
> > I/O to overlap I wouldn't use a thread.  I think I share this
> > perspective with most of the thread community (after all, threads
> > originate in the OS world where they were invented as a replacement
> > for I/O completion routines).

[Tim]
> Different perspective indeed!  Where I've been, you never used something as
> delicate as a thread to overlap I/O, you instead used the kernel-supported
> asynch Fortran I/O extensions <0.7 wink>.
> 
> Those days are long gone, and I've adjusted to that.  Time for you to leave
> the past too <wink>:  by sheer numbers, most of the "thread community"
> *today* is to be found typing at a Windows box, where cheap & reliable
> threads are a core part of the programming culture.

No quibble so far...

> They have better ways to overlap I/O there too.

Really?  What are they?  The non-threaded primitives for overlapping
I/O look like Medusa to me: very high performance, but a pain to use
-- because of the event-driven programming model!  (Or worse, callback
functions.)  But maybe there are programming techniques that I'm not
even aware of?

(Maybe I should define what I mean by overlapping I/O -- basically
every situation where disk or network I/O or GUI event handling goes
(Continue reading)

Permalink | Reply | 19 comments |
headers
Gordon McMillan | 4 Jul 1999 05:41

Re: Fake threads (was ActiveState & fork & Perl)

Hmmm. I jumped back into this one, but never saw my post show up...

Threads (real or fake) are useful when more than one thing is 
"driving" your processing. It's just that in the real world (a place 
Tim visited, once, but didn't like - or was it vice versa?) those 
"drivers" are normally I/O.

Guido complained that to do it right would require gathering up all 
the fds and doing a select. I don't think that's true (at least, for 
a decent fake thread). You just have to select on the one (to see if 
the I/O will work) and swap or do it accordingly. Also makes it a bit 
easier for portability (I thought I heard that Mac's select is 
limited to sockets).

I see 2 questions. First, is there enough of an audience (Mac, 
mostly, I think) without native threads to make them worthwhile? 
Second, do we want to introduce yet more possibilities for 
brain-explosions by enabling coroutines / continuations / generators 
or some such? There is practical value there (as Sam has pointed out, 
and I now concur, watching my C state machine grow out of control 
with each new client request).

I think the answer to both is probably "yes", and though they have a
lot in common technically, they have totally different rationales.

- Gordon
Permalink | Reply | 3 comments |
headers
Tim Peters | 4 Jul 1999 10:46
Picon

RE: Fake threads (was ActiveState & fork & Perl)

[Gordon McMillan]
> Hmmm. I jumped back into this one, but never saw my post show up...

Me neither!  An exclamation point because I see there's a recent post of
yours in the Python-Dev archives, but I didn't get it in the mail either.

> Threads (real or fake) are useful when more than one thing is
> "driving" your processing. It's just that in the real world (a place
> Tim visited, once, but didn't like - or was it vice versa?) those
> "drivers" are normally I/O.

Yes, but that's the consensus view of "real", and so suffers from "ten
billion flies can't be wrong" syndrome <wink>.  If you pitch a parallel
system to the NSA, they give you a long list of problems and ask you to
sketch the best way to solve them on your platform; as I recall, none had
anything to do with I/O even under Guido's definition; instead tons of
computation with difficult twists, and enough tight coupling to make threads
the natural approach in most cases.  If I said any more they'd terminate me
with extreme prejudice, and the world doesn't get any realer than that
<wink>.

> Guido complained that to do it right would require gathering up all
> the fds and doing a select. I don't think that's true (at least, for
> a decent fake thread). You just have to select on the one (to see if
> the I/O will work) and swap or do it accordingly. Also makes it a bit
> easier for portability (I thought I heard that Mac's select is
> limited to sockets).

Can you flesh out the "swap" part more?  That is, we're in the middle of
some C code, so the C stack is involved in the state that's being swapped,
(Continue reading)

Permalink | Reply | 1 comment |
headers
Tim Peters | 4 Jul 1999 10:45
Picon

RE: Fake threads (was ActiveState & fork & Perl)

[Guido and Tim, Guido and Tim]

Ouch!  This is getting contentious.  Let's unwind the "you said, I said, you
said" business a bit.

Among the three {coroutines, fake threads, continuations}, I expect any
could be serviceably simulated via either of the others.  There:  just saved
a full page of sentence diagramming <wink>.  All offer a strict superset of
generator semantics.

It follows that, *given* either coroutines or continuations, I indeed see no
semantic hole that would be plugged by fake threads.  But Python doesn't
have any of the three now, and there are two respects in which fake threads
may have an advantage over the other two:

1) Pedagogical, a friendlier sandbox for learning "real threads".

2) Python already has *a* notion of threads.  So fake threads could be seen
as building on that (variation of an existing concept, as opposed to
something unprecedented).

I'm the only one who seems to see merit in #2, so I won't mention it again:
fake threads may be an aid to education, but other than that they're useless
crap, and probably cause stains if not outright disk failure <wink>.

About newbies, I've seen far too many try to learn threads to entertain the
notion that they're easier than I think.  Threads != parallel programming,
though!  Approaches like Gelertner's Linda, or Klappholz's "refined
languages", *are* easy for newbies to pick up, because they provide clear
abstractions that prevent the worst parallelism bugs by offering primitives
(Continue reading)

Permalink | Reply | 18 comments |

Gmane