Discussion of Python object-relational mapper SQLObject

Re: memcache

Hello. I have returned to investigate the issue. Are you still interested?

On Fri, Jul 31, 2009 at 01:10:34PM -0400, Stef Telford wrote:
>    Okay.. so.. assuming I can get a writeLock again after del'ing it..  
> we should be good there. If you can let me know exactly -what- in the  
> sqlmeta blows up, that may also help matters. The connection surely  
> explodes (And rightly so I think).

   The problem with sqlmeta is that it's a class generated on the fly. It
doesn't even have a module - and pickle protocol prefers stable classes,
with proper modules and names.
   But I think it's easy to deal with it anyway. We don't need to pickle
sqlmeta *class* - the class is in the corresponding SQLObject class. We
probably even don't need to pickle sqlmeta instance - we can just recreate
it on unpickling 'cause its only instance data is a reference to SQLObject
instance and we know the instance - at the time of unpickling it's 'self'
so we can reconstruct a sqlmeta instance using the known class and known
instance data. Just for the (rare) case a user would have put some data
into the instance I can save sqlmeta.__dict__ and restore it on unpickling.
   I am going to add the corresponding __getstate__ and __setstate__ to
SQLObject and test them a bit.

Oleg.
--

-- 
     Oleg Broytman            http://phd.pp.ru/            phd <at> phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business

(Continue reading)

headers

Stef Telford | 1 Feb 19:40

Re: memcache

Morning Oleg, Everyone,
     I found that to get memcache working, I had do two things;

     *) get rid of the cache writelocks inside main.py (eg; remark out 
any val._SO_writeLock.acquire()) mostly due to explosions whilst pickling
     *) subclass the SQLObject into another class (which I called 
CompassObject, since our reservation system is called Compass ;) and 
overloaded _SO_setValue to do a delete inside memcache on change and 
overloaded 'get' such that it looks up memcache. I have to del/create 
_SO_writeLock, sqlmeta and _connection on each object when it's returned.

     Note that if someone was calling a series of 'setValue' instead of 
one large 'set', this would cause a lot of cache deletes and then misses 
when it tries to set. Thankfully, we have already audited the codebase 
for that (it was about a 3-4% "win" for us to change it over).

     Also, in a totally random thought, you will probably hit the 1mb 
memcache limit when storing objects. Since we are storing objects 
without blobs, it works. The only real workaround if people start to 
complain about memcache 'not storing their objects' is to either go 
Tokyo Tyrant or Redis. If you go Redis, you also end up with more speed 
and m->s replication for free ;).

     I am more than willing to help and/or bounce ideas off if you want. 
The solution I have may not be 'perfect' but it's definitely performant 
enough for our needs :D

     Regards
     Stef

(Continue reading)

headers

Petr Jakeš | 1 Feb 20:40

ForeignKey and MultipleJoin on legacy (existing) database

Hi,
I am trying to connect via SQLObject to the existing database (MySQL - Prestashop database)

The database has different naming style than SQLObject expect.

I know it is possible to use

class sqlmeta:
idName = 'id_order'

to get primary key working properly.

What I really can not figure out is how to get ForeignKey and MultipleJoin working.

Thanks for your comments

Petr Jakes

------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com

_______________________________________________
sqlobject-discuss mailing list
sqlobject-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss

headers

Oleg Broytman | 1 Feb 21:33

Re: ForeignKey and MultipleJoin on legacy (existing) database

On Mon, Feb 01, 2010 at 08:40:48PM +0100, Petr Jake?? wrote:
> I am trying to connect via SQLObject to the existing database

   You didn't mention fromDatabase but I am sure that what you meant,
right? Because it's certainly possible to connect to an existing database
without it.

> What I really can not figure out is how to get ForeignKey and MultipleJoin
> working.

   By declaring them yourself:

class MyTable(SQLObject):
    class sqlmeta:
        fromDatabase = True
    fk = ForeignKey('AnotherTable')
    mj = MultipleJoin('AnotherTable', joinColumn='at_id')

   Explicit declarations are compatible with fromDatabase. And of course
you are not obliged to use fromDatabase at all - just declare all you
columns and joins yourself.
   Of course IWBN if your patch fromDatabase machinery (methods
columnsFromSchema and guessClass in Connection class) to recognize at least
ForeignKeys.

Oleg.
--

-- 
     Oleg Broytman            http://phd.pp.ru/            phd <at> phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com

headers

Oleg Broytman | 1 Feb 21:43

Re: memcache

On Mon, Feb 01, 2010 at 01:40:51PM -0500, Stef Telford wrote:
>      *) subclass the SQLObject into another class (which I called 
> CompassObject, since our reservation system is called Compass ;) and 
> overloaded _SO_setValue to do a delete inside memcache on change and 

   To delete the instance from the cache to make the cache coherent?

> overloaded 'get' such that it looks up memcache. I have to del/create 

   I am going to do it in steps. The first step - to make SQLObject
instances pickleable. MemcachedCache would be the next. But I am going to
develop a separate cache class, not hack into .get machinery.

> _SO_writeLock, sqlmeta and _connection on each object when it's returned.

   _connection? Do you mean per-instance connection? In general, the
connection is stored in the SQLObject table class and it doesn't need to be
pickled or restored. But how do you restore per-instance connection?

>      Also, in a totally random thought, you will probably hit the 1mb 
> memcache limit when storing objects.

   What's the limit?

Oleg.
--

-- 
     Oleg Broytman            http://phd.pp.ru/            phd <at> phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com

headers

Petr Jakeš | 1 Feb 22:09

Re: ForeignKey and MultipleJoin on legacy (existing) database

> I am trying to connect via SQLObject to the existing database

You didn't mention fromDatabase but I am sure that what you meant,
right? Because it's certainly possible to connect to an existing database
without it.

> What I really can not figure out is how to get ForeignKey and MultipleJoin
> working.

By declaring them yourself:

class MyTable(SQLObject):
class sqlmeta:
fromDatabase = True
fk = ForeignKey('AnotherTable')
mj = MultipleJoin('AnotherTable', joinColumn='at_id')

Thanks for the syntax, it works great.

Explicit declarations are compatible with fromDatabase. And of course
you are not obliged to use fromDatabase at all - just declare all you
columns and joins yourself.

I have used fromDatabase because my lazybones only.

Of course IWBN if your patch fromDatabase machinery (methods
columnsFromSchema and guessClass in Connection class) to recognize at least
ForeignKeys.

IWBN, I would like to. But, to be honest, I think my Pythonic skills are to low for this task (it is REALLY not about the time or laziness).

Thanks and Best Regards

Petr

------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com

_______________________________________________
sqlobject-discuss mailing list
sqlobject-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss

headers

John Bresnahan | 4 Feb 00:37

setConnection()

I am trying to change the connection with which a given object is associate for some needed 
transactional behavior.  In an API function a user hands me an SQLObject called xfer.  I want to 
update a value in that object and automically update the value of a row in another table.

To achieve this I created a transaction with

trans = sqlhub.processConnection.transaction()

and then i call xfer.setConnection(trans).

This does not seem to have the effects I assumed so I have a few questions:

1) Is there a better way to achieve the results I am looking for?

2) at what point is an object associated with a DB connection?

3) is it at all possible to change the DB connection associated with a given object?

Any help is greatly appreciated.

Thanks,

John

------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com

headers

Oleg Broytman | 4 Feb 14:53

Re: per-instance connection (was: setConnection)

On Wed, Feb 03, 2010 at 01:37:08PM -1000, John Bresnahan wrote:
> 2) at what point is an object associated with a DB connection?
> 3) is it at all possible to change the DB connection associated with a given object?

   Objects (SQLObject instances that correspond to rows in a table) are
seldom associated with a connection (though it is certainly possible).
   SQLObject classes (that correspond to tables) are usually associated with
a connection; .setConnection() is a class method.
   To associate an object with a connection you have to SELECT it via that
connection:

> trans = sqlhub.processConnection.transaction()

and now do

   xfer = XferTable.get(id, connection=trans)
or XferTable.select(condition, connection=trans).
   Most SQLObject methods accept 'connection' parameter.

Oleg.
--

-- 
     Oleg Broytman            http://phd.pp.ru/            phd <at> phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com

headers

John Bresnahan | 4 Feb 21:45

Re: per-instance connection

> On Wed, Feb 03, 2010 at 01:37:08PM -1000, John Bresnahan wrote:
>> 2) at what point is an object associated with a DB connection?
>> 3) is it at all possible to change the DB connection associated with a given object?
> 
>    Objects (SQLObject instances that correspond to rows in a table) are
> seldom associated with a connection (though it is certainly possible).
>    SQLObject classes (that correspond to tables) are usually associated with
> a connection; .setConnection() is a class method.
>    To associate an object with a connection you have to SELECT it via that
> connection:
> 
>> trans = sqlhub.processConnection.transaction()
> 
> and now do
> 
>    xfer = XferTable.get(id, connection=trans)
> or XferTable.select(condition, connection=trans).
>    Most SQLObject methods accept 'connection' parameter.

Thanks much!  Is there a way to change the connection associated with an object you have already 
acquired via select?

------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com

headers

Petr Jakeš | 4 Feb 21:46

how to set dbEncoding for the whole table/database, not for the row?

Hi,
I know below code example works. I would like to know, if it is possible to set the encoding for the whole table at once (or for the connection/database) instead to define it for each column.

Petr

====== code example ============

class Customer(SQLObject):
    _connection = connection
    class sqlmeta:
        idName = 'id_customer'
#        fromDatabase = True
    orders = MultipleJoin('Orders', joinColumn = "id_customer")
    lastname = UnicodeCol(dbEncoding = "1250")
    firstname = UnicodeCol(dbEncoding = "1250")

------------------------------------------------------------------------------
The Planet: dedicated and managed hosting, cloud storage, colocation
Stay online with enterprise data centers and the best network in the business
Choose flexible plans and management services without long-term contracts
Personal 24x7 support from experience hosting pros just a phone call away.
http://p.sf.net/sfu/theplanet-com

_______________________________________________
sqlobject-discuss mailing list
sqlobject-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss

Mon	Tue	Wed	Thu	Fri	Sat	Sun

1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28

5	February 2012
12	January 2012
16	December 2011
17	November 2011
12	August 2011
29	July 2011
9	June 2011
27	May 2011
26	April 2011
36	March 2011
44	February 2011
15	January 2011
30	December 2010
11	November 2010
26	October 2010
12	September 2010
23	August 2010
37	July 2010
13	June 2010
14	May 2010
71	April 2010
23	March 2010
63	February 2010
24	January 2010
11	December 2009
10	November 2009
20	October 2009
38	September 2009
57	August 2009
36	July 2009
30	June 2009
38	May 2009
19	April 2009
22	March 2009
46	February 2009
69	January 2009
65	December 2008
41	November 2008
37	October 2008
21	September 2008
30	August 2008
66	July 2008
31	June 2008
82	May 2008
69	April 2008
55	March 2008
139	February 2008
82	January 2008
54	December 2007
137	November 2007
101	October 2007
31	September 2007
81	August 2007
118	July 2007
82	June 2007
129	May 2007
120	April 2007
115	March 2007
102	February 2007
152	January 2007
135	December 2006
128	November 2006
330	October 2006
142	September 2006
161	August 2006
94	July 2006
160	June 2006
254	May 2006
218	April 2006
228	March 2006
157	February 2006
235	January 2006
235	December 2005
315	November 2005
466	October 2005
303	September 2005
336	August 2005
265	July 2005
166	June 2005
193	May 2005
152	April 2005
145	March 2005
258	February 2005
264	January 2005
177	December 2004
169	November 2004
60	October 2004
78	September 2004
65	August 2004
52	July 2004
67	June 2004
65	May 2004
88	April 2004
123	March 2004
92	February 2004
38	January 2004
61	December 2003
95	November 2003
85	October 2003
88	September 2003
64	August 2003
129	July 2003
107	June 2003
206	May 2003
204	April 2003
43	March 2003
2	February 2003