Sophana K | 3 Sep 10:26
Picon

This transaction has already gone through ROLLBACK

Hello

I'm using transactions in sqlobjects since years in my web application.
But since some months, I receive these exceptions:

in update
    hubNoCache.doInTransaction(updateSubscr,self.id)
  File "/usr/lib/python2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/dbconnection.py",
line 856, in doInTransaction
    conn = old_conn.transaction()
  File "/usr/lib/python2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/dbconnection.py",
line 754, in __getattr__
    self.assertActive()
  File "/usr/lib/python2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/dbconnection.py",
line 678, in assertActive
    assert not self._obsolete, "This transaction has already gone
through ROLLBACK; begin another transaction"
AssertionError: This transaction has already gone through ROLLBACK;
begin another transaction

It looks like its related with the mysql server being overloaded. But
I still don't understand why.
I've been looking at the source and don't understand how this can
happen. I never call rollback in my source, and always make
transactions through doInTransaction.

In the traceback I don't understand why there are missing step between
conn = old_conn.transaction() and self.assertActive(). It should go
through the Transaction constructor.

(Continue reading)

Oleg Broytmann | 3 Sep 11:06
X-Face
Picon
Favicon

Re: This transaction has already gone through ROLLBACK

On Thu, Sep 03, 2009 at 10:26:03AM +0200, Sophana K wrote:
>   File "/usr/lib/python2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/dbconnection.py",
> line 856, in doInTransaction
>     conn = old_conn.transaction()
>   File "/usr/lib/python2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/dbconnection.py",
> line 754, in __getattr__
>     self.assertActive()
>   File "/usr/lib/python2.5/site-packages/SQLObject-0.10.2-py2.5.egg/sqlobject/dbconnection.py",
> line 678, in assertActive
>     assert not self._obsolete, "This transaction has already gone
> through ROLLBACK; begin another transaction"
> AssertionError: This transaction has already gone through ROLLBACK;
> begin another transaction
> 
> It looks like its related with the mysql server being overloaded. But

   I think you mean "restarted", not "overloaded", right?

> I still don't understand why.
> I've been looking at the source and don't understand how this can
> happen. I never call rollback in my source, and always make
> transactions through doInTransaction.

   doInTransaction() calls rollback() on any exception.

> In the traceback I don't understand why there are missing step between
> conn = old_conn.transaction() and self.assertActive(). It should go
> through the Transaction constructor.

   What connection have you assigned to hubNoCache? It seems it's a (rolled
(Continue reading)

Sophana K | 3 Sep 13:33
Picon

Re: This transaction has already gone through ROLLBACK


> It looks like its related with the mysql server being overloaded. But

  I think you mean "restarted", not "overloaded", right?

No, the mysql server is very loaded.
 
> I still don't understand why.
> I've been looking at the source and don't understand how this can
> happen. I never call rollback in my source, and always make
> transactions through doInTransaction.

  doInTransaction() calls rollback() on any exception.

I understand this.
rollBack is called on a Transaction object by doInTransaction.
But it just throws away the object just after, doesn't it?
This is why I don't understand how it can happen if I don't explicitely call rollback myself.
Could it happen during the contruction of the Transaction object itself?
As you might see in the traceback, the error happens while the Transaction is created.
My code isn't called yet...

I forgot to say. It looks that once this happened once, all upcoming transactions fail the same way.

In the Transaction constructor:
class Transaction(object):
    def __init__(self, dbConnection):
        # this is to skip __del__ in case of an exception in this __init__
        self._obsolete = True
        self._dbConnection = dbConnection
        self._connection = dbConnection.getConnection()
        self._dbConnection._setAutoCommit(self._connection, 0)
        self.cache = CacheSet(cache=dbConnection.doCache)
        self._deletedCache = {}
        self._obsolete = False

_obsolete is set to True.
Then you have self._connection accessed.
So the __getattr__ method should be called, doesn't it?
__getattr__ first calls self.assertActive() which checks that _obsolete is False.
So I wonder how this can work?

 
> In the traceback I don't understand why there are missing step between
> conn = old_conn.transaction() and self.assertActive(). It should go
> through the Transaction constructor.

  What connection have you assigned to hubNoCache? It seems it's a (rolled
back) transaction, not a simple connection.


hubNoCache=ConnectionHub()
hubNoCache.processConnection = connectionForURI(database+'&cache=')


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
sqlobject-discuss mailing list
sqlobject-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Oleg Broytmann | 3 Sep 14:57
X-Face
Picon
Favicon

Re: This transaction has already gone through ROLLBACK

On Thu, Sep 03, 2009 at 01:33:53PM +0200, Sophana K wrote:
> > > It looks like its related with the mysql server being overloaded. But
> >
> >    I think you mean "restarted", not "overloaded", right?
> >
> > No, the mysql server is very loaded.

   Then how it's related to SQLObject? Timeouts between clients and the
server?

> > > I still don't understand why.
> > > I've been looking at the source and don't understand how this can
> > > happen. I never call rollback in my source, and always make
> > > transactions through doInTransaction.
> >
> >    doInTransaction() calls rollback() on any exception.
> >
> > I understand this.
> rollBack is called on a Transaction object by doInTransaction.
> But it just throws away the object just after, doesn't it?
> This is why I don't understand how it can happen if I don't explicitely call
> rollback myself.
> Could it happen during the contruction of the Transaction object itself?
> As you might see in the traceback, the error happens while the Transaction
> is created.

   The only place I could suspect is the call dbConnection.getConnection()
in Transaction.__init__().

> I forgot to say. It looks that once this happened once, all upcoming
> transactions fail the same way.
> 
> In the Transaction constructor:
> class Transaction(object):
>     def __init__(self, dbConnection):
>         # this is to skip __del__ in case of an exception in this __init__
>         self._obsolete = True
>         self._dbConnection = dbConnection
>         self._connection = dbConnection.getConnection()
>         self._dbConnection._setAutoCommit(self._connection, 0)
>         self.cache = CacheSet(cache=dbConnection.doCache)
>         self._deletedCache = {}
>         self._obsolete = False
> 
> _obsolete is set to True.
> Then you have self._connection accessed.
> So the __getattr__ method should be called, doesn't it?
> __getattr__ first calls self.assertActive() which checks that _obsolete is
> False.

   No, self._connection is assigned:
self._connection = dbConnection.getConnection(). After this __getattr__ is
never called on accessing the self._connection. __getattr__ is called on
accessing non-existing attributes. In the Transaction class it is used to
delegate attribute access to the underlying connection.

> >    What connection have you assigned to hubNoCache? It seems it's a (rolled
> > back) transaction, not a simple connection.
> >
> >
> hubNoCache=ConnectionHub()
> hubNoCache.processConnection = connectionForURI(database+'&cache=')

   Looks good, nothing strange here. Could it be an old transaction is
stuck in  hubNoCache?

> > > In the traceback I don't understand why there are missing step between
> > > conn = old_conn.transaction() and self.assertActive(). It should go
> > > through the Transaction constructor.

   I recommend you to investigate it deeper: add a lot of debugging output
to the dbconnection.py to find out what is the connection, what is the
transaction and what is the attribute that cause the problem.

Oleg.
--

-- 
     Oleg Broytmann            http://phd.pp.ru/            phd <at> phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
Tobias Weber | 6 Sep 09:54
Picon
Gravatar

Inheritance looses column

Hi,
as I understand the documentation, sub-sub-classing  
InheritableSQLObject should be fine. But it messes with introspection,  
as demonstrated by the short attached script.

When there is no C and you create and retrieve B, everything works.  
Introduce C, and running it with an empty database (rm /tmp/x) still  
works:

<class '__main__.A'>
<class '__main__.B'>
X
<class '__main__.C'>
X

But run it again, so that the first list() actually does retrieve  
something, and it will raise:

<class '__main__.A'>
<class '__main__.B'>
X
<class '__main__.C'>
Traceback (most recent call last):
   File "desperate.py", line 23, in <module>
     list(B.select())
   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/ 
lib/python2.6/site-packages/SQLObject-0.12dev_r3970-py2.6.egg/ 
sqlobject/sresults.py", line 179, in __iter__
     return iter(list(self.lazyIter()))
   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/ 
lib/python2.6/site-packages/SQLObject-0.12dev_r3970-py2.6.egg/ 
sqlobject/inheritance/iteration.py", line 43, in next
     childResults=childResults, connection=self.dbconn)
   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/ 
lib/python2.6/site-packages/SQLObject-0.12dev_r3970-py2.6.egg/ 
sqlobject/inheritance/__init__.py", line 309, in get
     selectResults=childResults)
   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/ 
lib/python2.6/site-packages/SQLObject-0.12dev_r3970-py2.6.egg/ 
sqlobject/inheritance/__init__.py", line 309, in get
     selectResults=childResults)
   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/ 
lib/python2.6/site-packages/SQLObject-0.12dev_r3970-py2.6.egg/ 
sqlobject/inheritance/__init__.py", line 290, in get
     val = super(InheritableSQLObject, cls).get(id, connection,  
selectResults)
   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/ 
lib/python2.6/site-packages/SQLObject-0.12dev_r3970-py2.6.egg/ 
sqlobject/main.py", line 893, in get
     val._init(id, connection, selectResults)
   File "/Users/towb/Desktop/desperate.py", line 10, in _init
     print getattr(self, 'name')
   File "<string>", line 1, in <lambda>
AttributeError: 'NoneType' object has no attribute 'name'

I couldn't find out what actually happens as the code is full of  
eval()...
Attachment (desperate.py): text/x-python-script, 602 bytes
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
sqlobject-discuss mailing list
sqlobject-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/sqlobject-discuss
Oleg Broytmann | 6 Sep 15:33
X-Face
Picon
Favicon

Re: Inheritance looses column

On Sun, Sep 06, 2009 at 09:54:43AM +0200, Tobias Weber wrote:
> class A(InheritableSQLObject):
>     def _init(self, *args, **kargs):
>         InheritableSQLObject._init(self, *args, **kargs)
>         print self.__class__
>         if 'name' in dir(self):
>             # This should never fail
>             print getattr(self, 'name')
> 
> class B(A):
>     name = sqlobject.StringCol()
> 
> class C(B):
>     pass
> 
> sqlobject.sqlhub.processConnection = sqlobject.connectionForURI('sqlite:/tmp/x')
> 
> for klass in (A, B, C):
>     klass.createTable(ifNotExists = True)

   Works for me:

print list(B.select())
C(name = 'X')
print list(B.select())

   prints

 1/QueryOne:  SELECT tbl_name FROM sqlite_master WHERE type='table' AND tbl_name = 'a'
 1/QueryR  :  SELECT tbl_name FROM sqlite_master WHERE type='table' AND tbl_name = 'a'
 2/Query   :  CREATE TABLE a (
    id INTEGER PRIMARY KEY,
    child_name VARCHAR (255)
)
 2/QueryR  :  CREATE TABLE a (
    id INTEGER PRIMARY KEY,
    child_name VARCHAR (255)
)
 3/QueryOne:  SELECT tbl_name FROM sqlite_master WHERE type='table' AND tbl_name = 'b'
 3/QueryR  :  SELECT tbl_name FROM sqlite_master WHERE type='table' AND tbl_name = 'b'
 4/Query   :  CREATE TABLE b (
    id INTEGER PRIMARY KEY,
    name TEXT,
    child_name VARCHAR (255)
)
 4/QueryR  :  CREATE TABLE b (
    id INTEGER PRIMARY KEY,
    name TEXT,
    child_name VARCHAR (255)
)
 5/QueryOne:  SELECT tbl_name FROM sqlite_master WHERE type='table' AND tbl_name = 'c'
 5/QueryR  :  SELECT tbl_name FROM sqlite_master WHERE type='table' AND tbl_name = 'c'
 6/Query   :  CREATE TABLE c (
    id INTEGER PRIMARY KEY,
    child_name VARCHAR (255)
)
 6/QueryR  :  CREATE TABLE c (
    id INTEGER PRIMARY KEY,
    child_name VARCHAR (255)
)
 7/Select  :  SELECT a.id, a.child_name FROM a WHERE ((a.child_name) = ('B'))
 7/QueryR  :  SELECT a.id, a.child_name FROM a WHERE ((a.child_name) = ('B'))
[]
 8/QueryIns:  INSERT INTO a (child_name) VALUES ('B')
 8/QueryR  :  INSERT INTO a (child_name) VALUES ('B')
 9/QueryOne:  SELECT child_name FROM a WHERE ((a.id) = (1))
 9/QueryR  :  SELECT child_name FROM a WHERE ((a.id) = (1))
<class '__main__.A'>
10/QueryIns:  INSERT INTO b (id, name, child_name) VALUES (1, 'X', 'C')
10/QueryR  :  INSERT INTO b (id, name, child_name) VALUES (1, 'X', 'C')
11/QueryOne:  SELECT name, child_name FROM b WHERE ((b.id) = (1))
11/QueryR  :  SELECT name, child_name FROM b WHERE ((b.id) = (1))
<class '__main__.B'>
X
12/QueryIns:  INSERT INTO c (id, child_name) VALUES (1, NULL)
12/QueryR  :  INSERT INTO c (id, child_name) VALUES (1, NULL)
13/QueryOne:  SELECT child_name FROM c WHERE ((c.id) = (1))
13/QueryR  :  SELECT child_name FROM c WHERE ((c.id) = (1))
<class '__main__.C'>
X
14/Select  :  SELECT a.id, a.child_name FROM a WHERE ((a.child_name) = ('B'))
14/QueryR  :  SELECT a.id, a.child_name FROM a WHERE ((a.child_name) = ('B'))
14/Select children of the class B:  SELECT b.id, b.name, b.child_name FROM b WHERE ((b.id) = (1))
14/QueryR  :  SELECT b.id, b.name, b.child_name FROM b WHERE ((b.id) = (1))
[<C 1 name='X'>]

   No exception.

Oleg.
--

-- 
     Oleg Broytmann            http://phd.pp.ru/            phd <at> phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
Oleg Broytmann | 7 Sep 00:10
X-Face
Picon
Favicon

Re: Inheritance looses column

[Answering to the list...]

On Sun, Sep 06, 2009 at 11:27:12PM +0200, Tobias Weber wrote:
> On 06.09.2009, at 15:33, Oleg Broytmann wrote:
>
>> No exception.
>
> That's 'cause you didn't run it twice. I wrote that the problem only  
> manifests when retrieving records from the database that do not yet  
> exist in memory.

   Aha, now I see. I got the exception for the second time. 'name' is in
dir(self) because it came from the class, but it's not in self.__dict__.
Seems like a bug.

Oleg.
--

-- 
     Oleg Broytmann            http://phd.pp.ru/            phd <at> phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
Oleg Broytmann | 7 Sep 00:29
X-Face
Picon
Favicon

Re: Inheritance looses column

On Mon, Sep 07, 2009 at 02:10:42AM +0400, Oleg Broytmann wrote:
>    Aha, now I see. I got the exception for the second time. 'name' is in
> dir(self) because it came from the class, but it's not in self.__dict__.
> Seems like a bug.

   Well, got it. Not a bug in SQLObject, but a "bug" in your expectations. :)
Let' see.

 1/Select  :  SELECT a.id, a.child_name FROM a WHERE ((a.child_name) = ('B'))
 1/QueryR  :  SELECT a.id, a.child_name FROM a WHERE ((a.child_name) = ('B'))
 1/Select children of the class B:  SELECT b.id, b.name, b.child_name FROM b WHERE ((b.id) IN (1, 2, 3))
 1/QueryR  :  SELECT b.id, b.name, b.child_name FROM b WHERE ((b.id) IN (1, 2, 3))
<class '__main__.A'>
<class '__main__.B'>
X
 2/QueryOne:  SELECT child_name FROM c WHERE ((c.id) = (1))
 2/QueryR  :  SELECT child_name FROM c WHERE ((c.id) = (1))
 2/COMMIT  :  auto
<class '__main__.C'>
Traceback (most recent call last):
   ...

   You see - SQLObject SELECTs data for the class C. But the table for the
class is empty, 'name' is in class B. During _init() the class is not yet
fully drawn from the database, so there is no 'name' in it yet.
   Let's wait and see if SQLObject SELECTs the additional data for the
class C from the tables for its parent classes. Let's comment out your
getattr (and replace it with 'pass'):

 1/Select  :  SELECT a.id, a.child_name FROM a WHERE ((a.child_name) = ('B'))
 1/QueryR  :  SELECT a.id, a.child_name FROM a WHERE ((a.child_name) = ('B'))
 1/Select children of the class B:  SELECT b.id, b.name, b.child_name FROM b WHERE ((b.id) = (1))
 1/QueryR  :  SELECT b.id, b.name, b.child_name FROM b WHERE ((b.id) = (1))
<class '__main__.A'>
<class '__main__.B'>
 2/QueryOne:  SELECT child_name FROM c WHERE ((c.id) = (1))
 2/QueryR  :  SELECT child_name FROM c WHERE ((c.id) = (1))
 2/COMMIT  :  auto
<class '__main__.C'>
 1/COMMIT  :  auto
[<C 1 name='X'>]

   Now, much later after _init(), the instance of class C got the attribute
'name'.

Oleg.
--

-- 
     Oleg Broytmann            http://phd.pp.ru/            phd <at> phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
Tobias Weber | 7 Sep 09:12
Picon
Gravatar

Re: Inheritance looses column

On 07.09.2009, at 00:29, Oleg Broytmann wrote:

> Now, much later after _init(), the instance of class C got the  
> attribute 'name'.

I know. The problem is that dir() lists it before that, which, as I  
wrote, breaks any code using introspection. SQLObject passes  
internally inconsistent objects to _init.

In my case, a library implementing the observer pattern looks for a  
certain kind of method using a module that comes with Python:

inspect.getmembers(self, predicate=lambda m: inspect.ismethod(m))

Internally that iterates over dir() and does isinstance(getattr). Code  
like this should never fail.

The ugly part is that the reason dir() mentions 'name' is because it's  
in the __dict of the parent class, as a StringCol instance. Normally  
getattr does search there and would just return that StringCol. I  
guess the error stems from SQLObject overriding __getattribute__ or  
something with an implementation that is aware of columns, but at the  
time of _init is not defined. I think it should at all times behave  
like a normal Python object and just return StringCol if it doesn't  
have a value from the table yet. Or also override __dir__ and only  
list magic members when they actually work.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
Oleg Broytmann | 7 Sep 11:15
X-Face
Picon
Favicon

Re: Inheritance looses column

On Mon, Sep 07, 2009 at 09:12:14AM +0200, Tobias Weber wrote:
> On 07.09.2009, at 00:29, Oleg Broytmann wrote:
> 
> > Now, much later after _init(), the instance of class C got the  
> > attribute 'name'.
> 
> I know. The problem is that dir() lists it before that, which, as I  
> wrote, breaks any code using introspection.

   dir() didn't list *it* - dir() listed a property from the class - print
self.__class__.__dict__ and see it's there. But having a property in the
class doesn't guarantee it wouldn't raise an exception when called. In case
of properties dir() is not a substitute for hasattr().

> SQLObject passes  
> internally inconsistent objects to _init.

   I don't think it's inconsistent. Any property is free to raise any
exception. Think about a property that raises UnicodeError, or
socket.error, or KeyboardInterrupt.

> In my case, a library implementing the observer pattern looks for a  
> certain kind of method using a module that comes with Python:
> 
> inspect.getmembers(self, predicate=lambda m: inspect.ismethod(m))
> 
> Internally that iterates over dir() and does isinstance(getattr).

   Looks like a bug in that library. It must distinguish plain attributes
from properties.

> Code  
> like this should never fail.

   Really? Even if dir() detects a property that raises an exception? Any
exception?

> The ugly part is that the reason dir() mentions 'name' is because it's  
> in the __dict of the parent class, as a StringCol instance. Normally  
> getattr does search there and would just return that StringCol. I  
> guess the error stems from SQLObject overriding __getattribute__ or  
> something with an implementation that is aware of columns, but at the  
> time of _init is not defined. I think it should at all times behave  
> like a normal Python object and just return StringCol if it doesn't  
> have a value from the table yet.

   I don't think so. After you have declared 'name' is a StringCol
self.name is supposed to be a property that accepts and returns strings,
not the StringCol instance. What are you going to do with those StringCol
instances?

> Or also override __dir__ and only  
> list magic members when they actually work.

   Impossible now - __dir__ is only used in Python 2.6+, and SQLObject still
supports 2.4.
   And I don't think it's necessary. Any property is free to raise any
exception.

Oleg.
--

-- 
     Oleg Broytmann            http://phd.pp.ru/            phd <at> phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july

Gmane