Up-to-date list of methods and attributes

Does anybody maintain a reasonably accurate list of methods and 
attributes, in a reference type format for SQLObject?  The various 
tutorials and release notes are nice, but a consolidated list would save 
a lot of searching around.

Thanks in advance.

Tom

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Up-to-date list of methods and attributes

List escribió:
> Does anybody maintain a reasonably accurate list of methods and 
> attributes, in a reference type format for SQLObject?  The various 
> tutorials and release notes are nice, but a consolidated list would save 
> a lot of searching around.
>
> Thanks in advance.
>   

I use epydoc[1] to generate the API documentation and consult it 
locally. This is my command:

epydoc -o /home/jonhattan/doc/sqlobject-api --name "SQLObject API Doc" 
-v /usr/lib/python2.5/site-packages/SQLObject-0.9.0-py2.5.egg/sqlobject/

note -o is the outputdir and -v your sqlobject location

I've put the generated api doc online at [2] in order to show you the 
results.

epydoc generates the documentation based on the python docstrings, that 
unfortunatelly are not very present in sqlobject code. Anyway it's a 
very useful to have the list of methods and attrs.

I'm very new to sqlobject. I think I can do my contribution by writing 
docstrings.

What about having the epydoc-generated-api-doc in a non-personal web 
location and linked from the sqlobject mainpage? (I can provide webspace 
at http://software.opentech.es if it's needed)

sqlite and threads

Hi all,

I've a little program to perform a massive import of data into a sqlite 
database. As the data I process come from internet connections,  I've 
opted to implement it with threads, to not get stalled waiting for each 
internet request to return the data.

The memory grows and grows indefinitely. If I change to postgres the 
problem dissapear. I've searched a lot about sqlobject+sqlite and 
threading without finding a solution.

I've read in main sqlobject doc: "SQLite may have concurrency issues, 
depending on your usage in a multi-threaded environment."

I'll explain my code with a simplified example:

class mySQLObject(SQLObject):
    _connection = ConnectionForUri("sqlite:test.db")
    item = UnicodeCol()

class get_page(Thread):
    def __init__(self, id):
        self.id = id
        Thread.__init__(self)

    def run()
        f = urlopen("http://example.com/article/%d" % id)
        mySQLObject(item = f.read())

# main

Re: sqlite and threads

On 6/5/07, jonhattan <jonathan <at> opentechsl.com> wrote:
> The memory grows and grows indefinitely. If I change to postgres the
> problem dissapear. I've searched a lot about sqlobject+sqlite and
> threading without finding a solution.

I imagine that the important difference between Postgres and Sqlite is
that Sqlite connections are only usable on the thread they're created
on. So if you're starting 10 000 threads, SQLObject has to create 10
000 connections to the SQLite database while in the Postgres case
threads can share connections.

It's possible that this alone is causing your problems (tests in our
work code reliably trigger Sqlite problems with only 20 threads
writing concurrently).

You might also want  to check whether turning off caching in SQLObject
helps at all (it's vaguely possible that the extra SQLite connections
result in many more objects being cached). See
http://www.sqlobject.org/module-sqlobject.cache.html.

If turning off caching doesn't help, I suggest simply rate limiting
the threads, storing the results in a temporary array and then having
the main thread do all the writes to sqlite (this shouldn't be any
slower than having lots of threads write). Something like:

import time

class get_page(Thread):
     def __init__(self, id, results):
         self.id = id

Re: sqlite and threads

Simon Cross escribió:
> On 6/5/07, jonhattan <jonathan <at> opentechsl.com> wrote:
>> The memory grows and grows indefinitely. If I change to postgres the
>> problem dissapear. I've searched a lot about sqlobject+sqlite and
>> threading without finding a solution.
>
> I imagine that the important difference between Postgres and Sqlite is
> that Sqlite connections are only usable on the thread they're created
> on. So if you're starting 10 000 threads, SQLObject has to create 10
> 000 connections to the SQLite database while in the Postgres case
> threads can share connections.
I figure out that connections are destroyed when the thread finish.. So 
it could be increase the load of the system, not memory consuming. Isn't 
it ?

> It's possible that this alone is causing your problems (tests in our
> work code reliably trigger Sqlite problems with only 20 threads
> writing concurrently).
actually I have a limit of 20 threads:

        while threading.activeCount()  > 19:
            sleep(5)

What I see is that threads are not being 'freed' as they finish. Perhaps 
the question is 'how can I destroy all references to a SQLObject?'
If I do:

o = mySQLObject(item = xxx)
sys.getrefcount(o) - 1    # the value is 2
del o                                # still one reference so the object 

Selecting from multiple tables/classes.

Hi all,

I have two classes (simplified here):

class ContactDetails(SQLObject):
     email = UnicodeCol(alternateID=True)

class VIPContactDetails(ContactDetails):
     pass

Is there any way I can perform a select across both classes, something 
like (in my head):
(ContactDetails & VIPContactDetails).select(...)

The alternative seems to be to just perform a select on each class and 
merge them together in Python, which isn't particularly convenient when 
I intend to pass orderBy and so on into select().

Thanks,

Nick Murdoch

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Selecting from multiple tables/classes.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

_______________________________________________
sqlobject-discuss mailing list
sqlobject-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss

Re: Selecting from multiple tables/classes.

Hi,

That doesn't seem to be quite what I need. I basically have two tables 
with identical headers in the SQL database (not the best design, I know, 
but it evolved that way from something a lot simpler.)

I've managed to work out what the SQL statement (Postgres) would be to 
do what I want, if that's useful:

SELECT * from (SELECT * FROM vip_comp_details UNION SELECT * FROM 
contact_details ) AS people where date > '2007-06-06' and date < 
'2007-06-07' order by date desc;

The relevant part there is SELECT * from (SELECT * FROM vip_comp_details 
UNION SELECT * FROM contact_details ), I guess.

I've not been able to find anything in the SQLObject docs for how to do 
this, so any help would be much appreciated :)

Thanks,

Nick Murdoch

TiNo wrote:
> Do a (left) join. (About joins: http://www.w3schools.com/sql/sql_join.asp)
> And on how to do it with SQLObject: 
> http://www.sqlobject.org/FAQ.html#how-can-i-do-a-left-join and 
> http://www.sqlobject.org/SQLObject.html#left-join-and-other-joins
> 
> TiNo
> 
> 2007/6/8, Nick Murdoch <nmurdoch <at> locayta.com <mailto:nmurdoch <at> locayta.com>>:
> 
>     Hi all,
> 
>     I have two classes (simplified here):
> 
>     class ContactDetails(SQLObject):
>          email = UnicodeCol(alternateID=True)
> 
>     class VIPContactDetails(ContactDetails):
>          pass
> 
> 
>     Is there any way I can perform a select across both classes, something
>     like (in my head):
>     (ContactDetails & VIPContactDetails).select(...)
> 
>     The alternative seems to be to just perform a select on each class and
>     merge them together in Python, which isn't particularly convenient when
>     I intend to pass orderBy and so on into select().
> 
>     Thanks,
> 
>     Nick Murdoch
> 
>     -------------------------------------------------------------------------
>     This SF.net email is sponsored by DB2 Express
>     Download DB2 Express C - the FREE version of DB2 express and take
>     control of your XML. No limits. Just data. Click to get it now.
>     http://sourceforge.net/powerbar/db2/
>     _______________________________________________
>     sqlobject-discuss mailing list
>     sqlobject-discuss <at> lists.sourceforge.net
>     <mailto:sqlobject-discuss <at> lists.sourceforge.net>
>     https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
> 
> 

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

Re: Selecting from multiple tables/classes.

On Fri, Jun 08, 2007 at 06:11:11PM +0100, Nick Murdoch wrote:
> The relevant part there is SELECT * from (SELECT * FROM vip_comp_details 
> UNION SELECT * FROM contact_details ), I guess.

   UNION is meaningless for high-level SQLObject API. The only way to do
UNION is:

connection.queryAll("SELECT * ...")

Oleg.
--

-- 
     Oleg Broytmann            http://phd.pp.ru/            phd <at> phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/

LEFTJOINOn and InheritableSQLObject weirdness

Multiple LEFTJOINOn queries seem to work if all classes subclass
SQLObject, but not if they subclass a subclass of
InheritableSQLObject. This is what I mean:

################################################################
class zoo( SQLObject ):
    name = StringCol( )
    cages = MultipleJoin( 'cage' )

class cage( SQLObject ):
    name = StringCol( )
    animals = MultipleJoin( 'animal' )
    zoo = ForeignKey( 'zoo' )

class animal( SQLObject ):
    name = StringCol( )
    cage = ForeignKey( 'cage' )
################################################################

So there are zoos, each zoo has a number of cages and each cage has a
number of animals. The following query selects the total number of
animals in a given zoo:

################################################################
joins = [ ]
joins.append( LEFTJOINOn( None, cage, animal.q.cageID==cage.q.id ) )
joins.append( LEFTJOINOn( None, zoo, cage.q.zooID==zoo.q.id ) )

print animal.select( zoo.q.name=='myzoo', join=joins ).count( )
################################################################

This is all fine, but if the model is changed slightly to use inheritance:

################################################################
class named( InheritableSQLObject ):
    name = StringCol( )

class zoo( named ):
    cages = MultipleJoin( 'cage' )

class cage( named ):
    animals = MultipleJoin( 'animal' )
    zoo = ForeignKey( 'zoo' )

class animal( named ):
    cage = ForeignKey( 'cage' )
################################################################

then the exact same query stops working. Why is that? If it is a
feature and not a bug, what should be the correct query for the
getting the total number of animals?

Daniel
python 2.5, SQLObject-0.10dev_r2660, sqlite.

-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/