headers
Paul Moore | 5 Mar 2004 21:45
Picon
Favicon

PEAK design pointers

I've just discovered PEAK, by a fairly convoluted route. So far, I've
read the tutorial on the Wiki (which is excellent, BTW!) and the
manual. I've not started delving into the code to understand how to
use the various bits, but I expect to.

However, the area I'm having trouble getting to grips with is how to
design an application based around PEAK. I come from a procedural/OOP
background, and there's nothing difficult per se with the concepts in
PEAK, but I'm just having difficulty getting my concept of what I'm
trying to achieve to "break up" into components in a way that seems to
work...

Any pointers on things to read would be much appreciated!

I do have a specific application in mind - I'll describe it here as a
concrete example, but I'm definitely *not* expecting someone else to
do my design for me...

What I'm trying to write is an application (more precisely, a
framework) for database monitoring. I have a central repository
database, and a large number (100+) of target databases. My
application needs to connect to each target, run a query, collect the
results and save them in a table in the repository. I'll ultimately
have a number of different instances of this application, each
collecting the results of a different query, and possibly handling the
results in slightly different ways.

Objects (components?) I see a need for include:

    * A "target database"
(Continue reading)

Permalink | Reply | 7 comments |
headers
Phillip J. Eby | 6 Mar 2004 00:23
Gravatar

Re: PEAK design pointers

At 08:45 PM 3/5/04 +0000, Paul Moore wrote:

>What I'm trying to write is an application (more precisely, a
>framework) for database monitoring. I have a central repository
>database, and a large number (100+) of target databases. My
>application needs to connect to each target, run a query, collect the
>results and save them in a table in the repository. I'll ultimately
>have a number of different instances of this application, each
>collecting the results of a different query, and possibly handling the
>results in slightly different ways.
>
>Objects (components?) I see a need for include:
>
>     * A "target database"
>     * A "repository"

I assume these are DBAPI SQL databases?

>     * A "thread pool" (too many targets to query in sequence - I need
>       to do the queries in the background).

I'll probably be adding one of these to PEAK eventually, but you might want 
to just use Twisted's threadpool support, i.e. 
'reactor.deferToThread()'.  The resulting Deferred objects are usable with 
'peak.events' to control "tasks".

>     * A "report" (to pack up the results of a run for publishing)
>     * A "mailer" (reports are typically sent by mail)
>
>But I can't work out how to connect them all together (well, I can,
(Continue reading)

Permalink | Reply | 6 comments |
headers
Paul Moore | 6 Mar 2004 11:24
Picon
Favicon

Re: PEAK design pointers

"Phillip J. Eby" <pje@...> writes:

>>But I can't work out how to connect them all together (well, I can,
>>but that's when all the nice modularity and flexibility breaks
>>down...)
>
> Well, if I understand you correctly, I'd probably write something that
> would spawn an events.Task for each of the databases to be monitored,
> and create another Task that accepted the results of each of the other
> tasks in order to prepare the consolidated report and mail it out.

Right - it sounds like events.Task is the place for me to look. That's
excellent. As I said, it's pointers to the right places to look which
I was lacking...

> But, I don't really know enough about what you're trying to even know
> if that's really appropriate, or if something else would be better.

That's not surprising, I was a little short on detail in my
descriptions. Now that I have the starter, I'll play around a bit
myself, and if I have any specific questions, I'll ask again.

> The other thing I imagine I'd do is tie the components together with
> ZConfig, creating a top-level schema with sections to describe the
> individual databases, etc.  Then, my app would literally be assembled
> by a config document, like this:

[...]

> All of the DB things would get put into a list attribute on the
(Continue reading)

Permalink | Reply | 5 comments |
headers
Phillip J. Eby | 6 Mar 2004 13:51
Gravatar

Re: Re: PEAK design pointers

At 10:24 AM 3/6/04 +0000, Paul Moore wrote:
>"Phillip J. Eby" <pje@...> writes:
>
> >>But I can't work out how to connect them all together (well, I can,
> >>but that's when all the nice modularity and flexibility breaks
> >>down...)
> >
> > Well, if I understand you correctly, I'd probably write something that
> > would spawn an events.Task for each of the databases to be monitored,
> > and create another Task that accepted the results of each of the other
> > tasks in order to prepare the consolidated report and mail it out.
>
>Right - it sounds like events.Task is the place for me to look. That's
>excellent. As I said, it's pointers to the right places to look which
>I was lacking...

Note that normally the way you'll do tasks is to wrap an object method with 
events.taskFactory.  E.g.:

     def someMethod(self,...):
         # blah blah
         yield something; events.resume()
         # ...

     someMethod = events.taskFactory(someMethod)

Then, calling 'someMethod(...)' will return a running 'events.Task' for 
that invocation of the method.

Often, you'll have objects that want a per-instance task to do something
(Continue reading)

Permalink | Reply | 4 comments |
headers
Tom Schwaller | 6 Mar 2004 14:35
Picon

Dependency Injection Examples with PEAK?

After reading Martin Fowlers Article about Dependecy Injection (aka 
Inversion of Control (IoC)

http://www.martinfowler.com/articles/injection.html

I wonder how to compare PEAK with picocontainer.org / nanocontainer.org,
springframework.org and avalon.apache.org. 

Is it possible to write code with PEAK the way e.g. springframework.org
(new kid on the Java block) does it? I mean: complete separation of
Components and Configuration/Composition of Components. Looking at the
PEAK examples in the Dev-Wiki I have the impression that by using PEAK I
have to stick it deeply in my code and will never be able to get rid of
it. Maybe someone has a PEAK example which "emulates" the
Springframework approach (Setter Injection beeing the prefered IoC
approach there)

--

-- 
Tom Schwaller
http://www.python.de
Permalink | Reply | 1 comment |
headers
Phillip J. Eby | 6 Mar 2004 18:39
Gravatar

Re: Dependency Injection Examples with PEAK?

At 02:35 PM 3/6/04 +0100, Tom Schwaller wrote:
>After reading Martin Fowlers Article about Dependecy Injection (aka
>Inversion of Control (IoC)
>
>http://www.martinfowler.com/articles/injection.html
>
>I wonder how to compare PEAK with picocontainer.org / nanocontainer.org,
>springframework.org and avalon.apache.org.
>
>Is it possible to write code with PEAK the way e.g. springframework.org
>(new kid on the Java block) does it? I mean: complete separation of
>Components and Configuration/Composition of Components. Looking at the
>PEAK examples in the Dev-Wiki I have the impression that by using PEAK I
>have to stick it deeply in my code and will never be able to get rid of
>it.

Why would you want to be rid of it?  ;)

Seriously, the point of putting Obtain and Make bindings in your components 
is to allow them to provide sensible defaults.  There is nothing stopping 
you from overriding those bindings via keyword arguments, subclassing, or 
direct attribute assignment.  And, if you use ZConfig, you can even 
assemble your complete application from data.

A lot of the differences between the Java frameworks and PEAK have to do 
with the differences between Python and Java.  In Java, you get 
type/interface metadata "for free", so just by creating a class with the 
right signatures, you have a "bean", and can then do things with it.

In Python, attributes and method signatures do not have any type
(Continue reading)

Permalink | Reply | 0 comments |
headers
Paul Moore | 7 Mar 2004 16:29
Picon
Favicon

Re: PEAK design pointers

"Phillip J. Eby" <pje@...> writes:

>> > Well, if I understand you correctly, I'd probably write something that
>> > would spawn an events.Task for each of the databases to be monitored,
>> > and create another Task that accepted the results of each of the other
>> > tasks in order to prepare the consolidated report and mail it out.
>>
>>Right - it sounds like events.Task is the place for me to look. That's
>>excellent. As I said, it's pointers to the right places to look which
>>I was lacking...
>
> Note that normally the way you'll do tasks is to wrap an object method
> with events.taskFactory.  E.g.:
>
>      def someMethod(self,...):
>          # blah blah
>          yield something; events.resume()
>          # ...
>
>      someMethod = events.taskFactory(someMethod)
>
> Then, calling 'someMethod(...)' will return a running 'events.Task'
> for that invocation of the method.

Hmm, I've read the events module, and as far as I can see it's aimed
at a cooperative multitasking model. That doesn't seem to fit my
application structure at all.

On the other hand, your suggestion of using Twisted's deferToThread
does help a lot. (I never thought of using Twisted for a non-network
(Continue reading)

Permalink | Reply | 1 comment |
headers
Paul Moore | 7 Mar 2004 18:39
Picon
Favicon

Silly basic problem: commands.NoSuchSubCommand

This is a silly trivial issue. I'm reasonably sure I had this working
before.

Application structure:

monitor\main.py

   defines class Main(commands.Bootstrap)
           class CmdTest(commands.AbstractCommand) 

monitor\monitor.ini

    [peak.running]
    app = importString("monitor.main:Main")

    [peak.running.shortcuts]
    test = importString("monitor.main:CmdTest")
    * = commands.NoSuchSubCommand

This is basically from the tutorial.

Now, peak runIni monitor\monitor.ini a generates a traceback saying
AttributeError: 'module' object has no attribute 'NoSuchSubCommand'.

I thought the idea of NoSuchSubCommand was to give a friendly error
when an unexpected command was entered?

As I say, I thought I'd had this working, so have I done something
obviously wrong? Originally, monitor\main.py was called
monitor\commands.py - I changed the name in case there was a lookup
(Continue reading)

Permalink | Reply | 1 comment |
headers
Paul Moore | 7 Mar 2004 22:16
Picon
Favicon

Re: Silly basic problem: commands.NoSuchSubCommand

Paul Moore <pf_moore@...> writes:

> This is a silly trivial issue.
[...]
>     * = commands.NoSuchSubCommand

----------------------------^

AAAAAAARGH.

Sorry for the waste of bandwidth.
Paul
--

-- 
This signature intentionally left blank
Permalink | Reply | 0 comments |
headers
Phillip J. Eby | 7 Mar 2004 23:10
Gravatar

Re: Re: PEAK design pointers

At 03:29 PM 3/7/04 +0000, Paul Moore wrote:

>Hmm, I've read the events module, and as far as I can see it's aimed
>at a cooperative multitasking model. That doesn't seem to fit my
>application structure at all.

Actually, it does, since it's just a higher-level view of Twisted-style 
callbacks.  However, for your specific application, if you are skilled with 
Twisted already, and the tasks you want to achieve are relatively simple, 
it may not be worth you learning the 'peak.events' model.

Most of my uses of 'peak.events' are for things that have complex 
interactions and "temporal rules" like "if we get lots of requests, and we 
have less than the maximum number of processes started, then start more 
processes up to a goal level, but no more than one every N seconds.  If any 
processes die while we're trying to get to the goal level, keep starting 
processes.  If we reach the goal level, reset the goal level to the minimum 
number of processes."  That sort of logic is very straightforward to 
express as events.Task objects, but very very hard to do correctly with 
just plain callbacks.

If your top-level framework needs to do that kind of task management, you 
will probably sooner or later want to look at peak.events in more detail.
Permalink | Reply | 0 comments |

Gmane