headers
Wilson Freitas | 8 Feb 2008 04:10
Picon
Gravatar

self-referencing table

Hi all,

Is it possible to use storm to handle self-referencing tables ?
I have made a trial with the following code:

class Category(object):
    __storm_table__ = "category"
    id = Int(name="id_category", primary=True)
    name = Unicode()
    id_parent = Int()
    parent = Reference(id_parent, Category.id)

    def __init__(self, name):
        self.name = name

But I have got a NameError, because Category was already not defined.
I also have tried this code:

class Category(object):
    __storm_table__ = "category"
    id = Int(name="id_category", primary=True)
    name = Unicode()
    id_parent = Int()
    parent = None

    def __init__(self, name):
        self.name = name
        self.parent = Reference(self.id_parent, Category.id)

and it does not worked too.
(Continue reading)

Permalink | Reply | 2 comments |
headers
Stephen Waterbury | 8 Feb 2008 04:29
Picon
Favicon

Re: self-referencing table

Wilson Freitas wrote:
> Hi all,
> 
> Is it possible to use storm to handle self-referencing tables ?

Sure -- look at the tutorial section "The Storm base class".
Inherit from that, and define the self-references using the
stringified version of the class name.

<https://storm.canonical.com/Tutorial#head-bcaf591032d0efda072f3c51adc3649714407552>

Steve
Permalink | Reply | 0 comments |
headers
Gustavo Niemeyer | 8 Feb 2008 18:28
Favicon
Gravatar

Re: self-referencing table

Hey Wilson,

> Is it possible to use storm to handle self-referencing tables ?
> I have made a trial with the following code:
> 
> class Category(object):
>     __storm_table__ = "category"
>     id = Int(name="id_category", primary=True)
>     name = Unicode()
>     id_parent = Int()
>     parent = Reference(id_parent, Category.id)
> 
>     def __init__(self, name):
>         self.name = name
(...)
> Is there any way to make this work?

Yep. Try this:

    parent = Reference(id_parent, id)

--

-- 
Gustavo Niemeyer
http://niemeyer.net
Permalink | Reply | 0 comments |
headers
James Henstridge | 11 Feb 2008 15:40
Picon
Gravatar

Select results randomisation

Hi everyone,

I've put together an in-progress branch to add select results
randomisation to Storm:
    https://bugs.launchpad.net/storm/+bug/94983

The patch adds a Store.set_randomise_order(value) method that can be
used to turn on the feature.  When turned on, the Select() expressions
generated by Store.find() will include RANDOM() as the last item (or
RAND() on MySQL), to randomise the order of the results (modulo any
requested ordering constraints).

This is essentially a port of an SQLObject feature we wrote to help
test Launchpad.  The problem is that when creating data for a test,
the data is generally inserted in a fixed order.  When running select
queries without any ORDER BY clause on the table, you will generally
get the results back in this same order.  This can easily lead to bugs
related to insufficient result sorting, since production data rarely
has such a nice natural ordering.

By adding RANDOM() to the end of the ORDER BY clause, we can make sure
that the tests do not depend on this natural ordering, so must perform
any required sorting.

As it stands, the branch works for simple queries.  I need to add a
few more tests for a few more cases:
 * interaction with set operations
 * distinct queries
 * aggregates
(Continue reading)

Permalink | Reply | 0 comments |
headers
Carlos Garcia Campos | 12 Feb 2008 11:51
Picon

storm performance

Hi all, 

I've been using storm in some projects for some time now without
problems, however I have a project that needs to do a lot of insertions
and I've noticed a huge performance problem. The project is now much
slower than the old versions that didn't use storm. 

So, I've written some small test cases to make sure no other things in
my project are causing the problem. Here are some results:

$ time ./sqlite.py 

real	0m47.519s
user	0m44.607s
sys	0m2.120s

$ time ./stormm.py 

real	9m37.220s
user	9m24.123s
sys	0m10.809s

both scripts insert 700000 records in the database. I've done the same
test with mysql and the results are quite similar. 

Of course I knew that using storm could introduce a performance penalty,
but it's too big in this case. Any idea what could be the problem? maybe
storm isn't thought to be used like this? or am I doing something wrong?

The scripts I used can be found here: http://gsyc.es/~carlosgc/files/
(Continue reading)

Permalink | Reply | 1 comment |
headers
James Henstridge | 12 Feb 2008 12:10
Picon
Gravatar

Re: storm performance

On 12/02/2008, Carlos Garcia Campos <carlosgc@...> wrote:
> Hi all,
>
> I've been using storm in some projects for some time now without
> problems, however I have a project that needs to do a lot of insertions
> and I've noticed a huge performance problem. The project is now much
> slower than the old versions that didn't use storm.
>
> So, I've written some small test cases to make sure no other things in
> my project are causing the problem. Here are some results:
>
> $ time ./sqlite.py
>
> real    0m47.519s
> user    0m44.607s
> sys     0m2.120s
>
> $ time ./stormm.py
>
> real    9m37.220s
> user    9m24.123s
> sys     0m10.809s
>
>
> both scripts insert 700000 records in the database. I've done the same
> test with mysql and the results are quite similar.

The two tests should result in pretty much identical SQL, so the
overhead is likely to be related to object creation.  This is likely
to be true of any ORM for this sort of workload.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Sean Upton | 13 Feb 2008 21:39
Picon
Gravatar

storm.zope in a minimal zope CA environment

I'm writing applications in minimal zope component architecture
environment, and I'd like to use storm.zope.  To date, this is not
possible, but only for cosmetic reasons (likely optional
dependencies).  I am using zope.component, zope.event, zope.schema,
and the newly liberated transaction package from PYPI.  I am NOT using
ZODB nor zope.app.* packages (thus not zope.security).  I've made
local changes to work around these limitations.  All tests pass for my
changes, with some modification to the storm.zope doctest README.txt.

Changes I found necessary:
1. Fallback to importing TransactionFailedError in zstorm.py from
transaction if import from ZODB fails.  This works with recent (<3mo.)
transaction in svn and pypi.  This is a second choice, so it should
not interfere with previous deployments of earlier ZODB versions.

2. Optionally import zope.security proxy stuff in __init__.py.

3. Make zope.security proxy doctest stuff optional, dependent on
whether zope.security can be imported.

I plan to create a branch in launchpad for my changes, but I would
like some comment/review from the list before I do this (diff pasted
inline below).  Specifically, my changes to the doctests to deal with
possibly optional dependencies and conditional imports needs
review/thought.

bzr diff output is below my sig...

Any comments/ideas appreciated. Thanks,
Sean Upton
(Continue reading)

Permalink | Reply | 1 comment |
headers
Tres Seaver | 13 Feb 2008 22:42
Gravatar

Re: storm.zope in a minimal zope CA environment


Sean Upton wrote:
> I'm writing applications in minimal zope component architecture
> environment, and I'd like to use storm.zope.  To date, this is not
> possible, but only for cosmetic reasons (likely optional
> dependencies).  I am using zope.component, zope.event, zope.schema,
> and the newly liberated transaction package from PYPI.  I am NOT using
> ZODB nor zope.app.* packages (thus not zope.security).  I've made
> local changes to work around these limitations.  All tests pass for my
> changes, with some modification to the storm.zope doctest README.txt.
> 
> Changes I found necessary:
> 1. Fallback to importing TransactionFailedError in zstorm.py from
> transaction if import from ZODB fails.  This works with recent (<3mo.)
> transaction in svn and pypi.  This is a second choice, so it should
> not interfere with previous deployments of earlier ZODB versions.

It would likely be more forward-compatible (avoiding a potential
deprecation warning) to try importing first from 'transaction', and then
fall back to 'ZODB.POSExcception'.

> 2. Optionally import zope.security proxy stuff in __init__.py.

Rather than using a flag, I think I would be tempted to do the work in
the 'else:' clause of the try-wrapped import.  That's just style,
though:  your way should work fine.

> 3. Make zope.security proxy doctest stuff optional, dependent on
> whether zope.security can be imported.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Brad Allen | 19 Feb 2008 22:39
Favicon

Storm presentation at Python user group

Hello Stormers,

I am planning to give a brief presentation about Storm at the DFW Python 
user group this coming Saturday, and am looking for some ideas.

Here is my current list of topic ideas:

* briefly review the tutorial for those who missed it in prior meeting
* source code overview, discussion of architecture
* look at test runner
* examine individual tests to see examples of using Storm
* look at TODO list, README, and NEWS released with .12
* look at branches posted in LaunchPad (such as Oracle backend)
* find some interesting tracepoints step through using pdb (such as 
Store.flush() )
* discuss exception injection, which injects Storm exceptions into dbapi 
modules exception inheritance path
* lead into a discussion of using bzr (most of us use SVN)

Can anyone add to this list of ideas, or provide any testimonials about 
why they find Storm interesting or practical?

Also, are there any existing open source projects using Storm, so I can 
show some examples of where it is being used?

Closed source projects using Storm would also be good to hear about. I 
have heard that Canonical Landscape uses Storm, but I am not sure. Also, 
I have not heard any news about whether LaunchPad has successfully 
transitioned from SQLObject to Storm.
(Continue reading)

Permalink | Reply | 14 comments |
headers
Duncan McGreggor | 19 Feb 2008 22:56
Picon
Gravatar

Re: Storm presentation at Python user group

On Feb 19, 2008 3:39 PM, Brad Allen <ballen@...> wrote:

> Hello Stormers,
>
> I am planning to give a brief presentation about Storm at the DFW Python
> user group this coming Saturday

Cool!

> Can anyone add to this list of ideas, or provide any testimonials about
> why they find Storm interesting or practical?

I like storm for the API. It's simple and it makes sense. It doesn't
try to do too much for me either. To be fair, I also like SQLAlchemy
for the API, but that's because other ORMs just didn't fit my brain
and SQLAlchemy was the closest I could get.

> Also, are there any existing open source projects using Storm, so I can
> show some examples of where it is being used?

I use Storm for work and play. I've used it to generate stats from
trac, as well as for writing miscellaneous scripts that have to query
databases and manage relationships. The pymon project (Twisted-based
systems/network monitoring) is in the process of converting to using a
database, and storm is the orm we're using (storm.twisted, to be
precise; the branch that supports async operations with deferreds).

> Closed source projects using Storm would also be good to hear about.

I'm the architect for a new set of services that will be provided by
(Continue reading)

Permalink | Reply | 12 comments |

Gmane