headers
James Henstridge | 11 May 2010 14:56
Picon
Gravatar

Re: Questions about DeferredStores on twisted-storm branch

On Tue, Apr 27, 2010 at 6:03 AM, Oscar Campos
<oscar.campos@...> wrote:
> Ty very much for the quick reply.
>
> I see.
>
> Btw, is DeferredStore meant to be threadsafe?

The DeferredStore does all its database access in a thread, so calls
from multiple threads will be serialised.  With that in mind though,
it doesn't mean that using a single store for unrelated concurrent
work is a good idea.  Things will be serialised on the connection, so
if you want parallelism you really need multiple connections (and
hence multiple DeferredStores).

James.
Permalink | Reply | 3 comments |
headers
Oscar Campos | 11 May 2010 15:48
Gravatar

Re: Questions about DeferredStores on twisted-storm branch

Yeah, that makes sense to me, so I already implemented it on this way.

In my application I have twisted web resources loaded as "controllers", each resource can have a model object (storm object) attached on it that represents a database table, each model object owns a DeferredStore (or a Store) that depends on application configuration.

I'm developing a Web Applications Framework using Twisted, Storm, Evoque and ExtJS so I think I'll ask a lot around here and twisted lists also.

I get some troubles with DeferredResults but I'm busy now implementing Authentication, and ACL, so I'll ask when I'll back to the code that gave me the problems.

Btw, do you guys know any decent ACL implementation (not based on file system like POSIX one) in Python?, I'm developing it from zero so I'm not a very big fan of reinventing the wheel.

Best Regards.

2010/5/11 James Henstridge <james-whk3OXXaEJgXC2x5gXVKYQ@public.gmane.org>
On Tue, Apr 27, 2010 at 6:03 AM, Oscar Campos
<oscar.campos <at> open-phoenix.com> wrote:
> Ty very much for the quick reply.
>
> I see.
>
> Btw, is DeferredStore meant to be threadsafe?

The DeferredStore does all its database access in a thread, so calls
from multiple threads will be serialised.  With that in mind though,
it doesn't mean that using a single store for unrelated concurrent
work is a good idea.  Things will be serialised on the connection, so
if you want parallelism you really need multiple connections (and
hence multiple DeferredStores).

James.



--
Oscar Campos Ruiz-Adame -oscar.campos <at> open-phoenix.com
Consultor - Consultant
Open Phoenix IT Consultoría Tecnológica - IT Consulting
_________________________________________________
Tel. +34 644569088
http://www.open-phoenix.com
<div>
<p>Yeah, that makes sense to me, so I already implemented it on this way.<br><br>In my application I have twisted web resources loaded as "controllers", each resource can have a model object (storm object) attached on it that represents a database table, each model object owns a DeferredStore (or a Store) that depends on application configuration.<br><br>I'm developing a Web Applications Framework using Twisted, Storm, Evoque and ExtJS so I think I'll ask a lot around here and twisted lists also.<br><br>I get some troubles with DeferredResults but I'm busy now implementing Authentication, and ACL, so I'll ask when I'll back to the code that gave me the problems.<br><br>Btw, do you guys know any decent ACL implementation (not based on file system like POSIX one) in Python?, I'm developing it from zero so I'm not a very big fan of reinventing the wheel.<br><br>Best Regards.<br><br></p>
<div class="gmail_quote">2010/5/11 James Henstridge <span dir="ltr">&lt;<a href="mailto:james@...">james@...</a>&gt;</span><br><blockquote class="gmail_quote">
<div class="im">On Tue, Apr 27, 2010 at 6:03 AM, Oscar Campos<br>
&lt;<a href="mailto:oscar.campos@...">oscar.campos <at> open-phoenix.com</a>&gt; wrote:<br>
&gt; Ty very much for the quick reply.<br>
&gt;<br>
&gt; I see.<br>
&gt;<br>
&gt; Btw, is DeferredStore meant to be threadsafe?<br><br>
</div>The DeferredStore does all its database access in a thread, so calls<br>
from multiple threads will be serialised. &nbsp;With that in mind though,<br>
it doesn't mean that using a single store for unrelated concurrent<br>
work is a good idea. &nbsp;Things will be serialised on the connection, so<br>
if you want parallelism you really need multiple connections (and<br>
hence multiple DeferredStores).<br><br>
James.<br>
</blockquote>
</div>
<br><br clear="all"><br>-- <br>Oscar Campos Ruiz-Adame -<a href="mailto:oscar.campos@...">oscar.campos <at> open-phoenix.com</a><br>Consultor - Consultant<br>Open Phoenix IT Consultor&iacute;a Tecnol&oacute;gica - IT Consulting<br>
_________________________________________________<br>Tel. +34 644569088<br><a href="http://www.open-phoenix.com">http://www.open-phoenix.com</a><br>
</div>
Permalink | Reply | 2 comments |
headers
James Henstridge | 12 May 2010 04:23
Picon
Gravatar

Re: Questions about DeferredStores on twisted-storm branch

On Tue, May 11, 2010 at 9:48 PM, Oscar Campos
<oscar.campos@...> wrote:
> Yeah, that makes sense to me, so I already implemented it on this way.
>
> In my application I have twisted web resources loaded as "controllers", each
> resource can have a model object (storm object) attached on it that
> represents a database table, each model object owns a DeferredStore (or a
> Store) that depends on application configuration.

Hmm.  That doesn't sound like the best way to structure things.

Usually I'd associate the store with the current request (the
twisted-integration branch provides a store pool implementation which
you could use this for).  You'd then use that store for all the
database access done by the request.  Having a single request make
changes to a database via multiple connections simultaneously almost
always ends in tears :)

James.
Permalink | Reply | 1 comment |
headers
Oscar Campos | 12 May 2010 05:31
Gravatar

Re: Questions about DeferredStores on twisted-storm branch

Thanks for the advice, I think you're right.
I'll move the stores from model to controller and associate it with requests instead of models.

Best Regards.

2010/5/12 James Henstridge <james <at> jamesh.id.au>
On Tue, May 11, 2010 at 9:48 PM, Oscar Campos
<oscar.campos-Pgw/qM3viRlHp/TnHMKxog@public.gmane.org> wrote:
> Yeah, that makes sense to me, so I already implemented it on this way.
>
> In my application I have twisted web resources loaded as "controllers", each
> resource can have a model object (storm object) attached on it that
> represents a database table, each model object owns a DeferredStore (or a
> Store) that depends on application configuration.

Hmm.  That doesn't sound like the best way to structure things.

Usually I'd associate the store with the current request (the
twisted-integration branch provides a store pool implementation which
you could use this for).  You'd then use that store for all the
database access done by the request.  Having a single request make
changes to a database via multiple connections simultaneously almost
always ends in tears :)

James.



--
Oscar Campos Ruiz-Adame -oscar.campos <at> open-phoenix.com
Consultor - Consultant
Open Phoenix IT Consultoría Tecnológica - IT Consulting
_________________________________________________
Tel. +34 644569088
http://www.open-phoenix.com
<div>
<p>Thanks for the advice, I think you're right.<br>I'll move the stores from model to controller and associate it with requests instead of models.<br><br>Best Regards.<br><br></p>
<div class="gmail_quote">2010/5/12 James Henstridge <span dir="ltr">&lt;<a href="mailto:james@...">james <at> jamesh.id.au</a>&gt;</span><br><blockquote class="gmail_quote">On Tue, May 11, 2010 at 9:48 PM, Oscar Campos<br><div class="im">&lt;<a href="mailto:oscar.campos@...">oscar.campos@...</a>&gt; wrote:<br>
</div>
<div class="im">&gt; Yeah, that makes sense to me, so I already implemented it on this way.<br>
&gt;<br>
&gt; In my application I have twisted web resources loaded as "controllers", each<br>
&gt; resource can have a model object (storm object) attached on it that<br>
&gt; represents a database table, each model object owns a DeferredStore (or a<br>
&gt; Store) that depends on application configuration.<br><br>
</div>Hmm. &nbsp;That doesn't sound like the best way to structure things.<br><br>
Usually I'd associate the store with the current request (the<br>
twisted-integration branch provides a store pool implementation which<br>
you could use this for). &nbsp;You'd then use that store for all the<br>
database access done by the request. &nbsp;Having a single request make<br>
changes to a database via multiple connections simultaneously almost<br>
always ends in tears :)<br><br>
James.<br>
</blockquote>
</div>
<br><br clear="all"><br>-- <br>Oscar Campos Ruiz-Adame -<a href="mailto:oscar.campos@...">oscar.campos <at> open-phoenix.com</a><br>Consultor - Consultant<br>Open Phoenix IT Consultor&iacute;a Tecnol&oacute;gica - IT Consulting<br>
_________________________________________________<br>Tel. +34 644569088<br><a href="http://www.open-phoenix.com">http://www.open-phoenix.com</a><br>
</div>
Permalink | Reply | 0 comments |
headers
Jorge Sierra | 13 May 2010 14:52
Picon

hey guys

I was doing some research to run storm on android.
I removed support for mysql, postgres, all ok.
Now I found cextensions.c, compiled to cextensions.pyd.

I have to take a look at that file, but there is any way I can convert 
this file to a .py file
migrate this c code to python code.

If I have 100% pure python storm module, it will be great, so I can 
start experimenting with
android-python-sqlite3.

Thanks
Jorge Sierra
http://sierramobiledeveloper.blogspot.com
Permalink | Reply | 5 comments |
headers
Jamu Kakar | 13 May 2010 16:17
Picon
Gravatar

Re: hey guys

Hi Jorge,

On Thu, May 13, 2010 at 2:52 PM, Jorge Sierra <asierra01@...> wrote:
> I was doing some research to run storm on android.
> I removed support for mysql, postgres, all ok.
> Now I found cextensions.c, compiled to cextensions.pyd.
>
> I have to take a look at that file, but there is any way I can convert
> this file to a .py file
> migrate this c code to python code.
>
> If I have 100% pure python storm module, it will be great, so I can
> start experimenting with
> android-python-sqlite3.

Nice!  The C extensions are in place purely for performance reasons.
Storm runs just fine as pure Python.  The extensions are imported by
default, unless the STORM_CEXTENSIONS environment variable is
explicitly set to '0'.  You can see the behaviour in place by
looking at the storm/__init__.py file.

Let us know how it works out for you!

Thanks,
J.
Permalink | Reply | 4 comments |
headers
asierra01 | 13 May 2010 16:26
Picon

Re: hey guys

WHAT, ok this is good news,

I will remove the use of extension. I dont really need a fast storm performance. I just need the ORM
features. I have, the java way of accessing sqlite3 databases under android, or or I can use python+sqlite3
or even better python+sqlite3+storm so my fields mapping will be easy/almost seamlessly.

I will even post my results here or somewhere else, if someone wants to use storm under android.

I renamed the module storm to storm_android,

from storm_android.locals import *
from storm_android.info import *
from storm_android.properties import *

I can put it back, I will write how to do some test and put it on the emulator, and run it.

<div><p>WHAT, ok this is good news,
<br><br>  I will remove the use of extension. I dont really need a fast storm performance. I just need the ORM
<br>  features. I have, the java way of accessing sqlite3 databases under android, or or I can use python+sqlite3
<br>  or even better python+sqlite3+storm so my fields mapping will be easy/almost seamlessly.
<br><br>  I will even post my results here or somewhere else, if someone wants to use storm under android.
<br><br>  I renamed the module storm to storm_android, 
<br><br>from storm_android.locals import *
<br>from storm_android.info import *
<br>from storm_android.properties import *
<br><br>  I can put it back, I will write how to do some test and put it on the emulator, and run it.</p></div>
Permalink | Reply | 3 comments |
headers
Vernon Cole | 13 May 2010 16:30
Picon
Gravatar

Re: hey guys

Great!
Now I HAVE to upgrade by Windows Mobile phone!
;-)
--
Vernon Cole

On Thu, May 13, 2010 at 8:26 AM, <asierra01-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
WHAT, ok this is good news,

I will remove the use of extension. I dont really need a fast storm performance. I just need the ORM
features. I have, the java way of accessing sqlite3 databases under android, or or I can use python+sqlite3
or even better python+sqlite3+storm so my fields mapping will be easy/almost seamlessly.

I will even post my results here or somewhere else, if someone wants to use storm under android.

I renamed the module storm to storm_android,

from storm_android.locals import *
from storm_android.info import *
from storm_android.properties import *

I can put it back, I will write how to do some test and put it on the emulator, and run it.
--
storm mailing list
storm-33AaDErTWvAAkhNd+W/JZ0EOCMrvLtNR@public.gmane.org
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/storm


<div>
<p>Great! <br>Now I HAVE to upgrade by Windows Mobile phone!<br>;-)<br>--<br>Vernon Cole<br><br></p>
<div class="gmail_quote">On Thu, May 13, 2010 at 8:26 AM,  <span dir="ltr">&lt;<a href="mailto:asierra01@...">asierra01@...</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">WHAT, ok this is good news,
<br><br>  I will remove the use of extension. I dont really need a fast storm performance. I just need the ORM
<br>  features. I have, the java way of accessing sqlite3 databases under android, or or I can use python+sqlite3
<br>  or even better python+sqlite3+storm so my fields mapping will be easy/almost seamlessly.
<br><br>  I will even post my results here or somewhere else, if someone wants to use storm under android.
<br><br>  I renamed the module storm to storm_android, 
<br><br>from storm_android.locals import *
<br>from <a href="http://storm_android.info" target="_blank">storm_android.info</a> import *
<br>from storm_android.properties import *
<br><br>  I can put it back, I will write how to do some test and put it on the emulator, and run it.<br>--<br>
storm mailing list<br><a href="mailto:storm@...">storm@...</a><br>
Modify settings or unsubscribe at: <a href="https://lists.ubuntu.com/mailman/listinfo/storm" target="_blank">https://lists.ubuntu.com/mailman/listinfo/storm</a><br><br>
</blockquote>
</div>
<br>
</div>
Permalink | Reply | 2 comments |
headers
Vernon Cole | 13 May 2010 16:32
Picon
Gravatar

Re: hey guys

Damn! Typo when making a flippant comment.
... _MY_ Windows mobile ...
--
Vernon

On Thu, May 13, 2010 at 8:30 AM, Vernon Cole <vernondcole-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Great!
Now I HAVE to upgrade by Windows Mobile phone!
;-)
--
Vernon Cole

On Thu, May 13, 2010 at 8:26 AM, <asierra01-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
WHAT, ok this is good news,

I will remove the use of extension. I dont really need a fast storm performance. I just need the ORM
features. I have, the java way of accessing sqlite3 databases under android, or or I can use python+sqlite3
or even better python+sqlite3+storm so my fields mapping will be easy/almost seamlessly.

I will even post my results here or somewhere else, if someone wants to use storm under android.

I renamed the module storm to storm_android,

from storm_android.locals import *
from storm_android.info import *
from storm_android.properties import *

I can put it back, I will write how to do some test and put it on the emulator, and run it.
--

storm mailing list
storm <at> lists.canonical.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/storm



<div>
<p>Damn! Typo when making a flippant comment.<br>... _MY_ Windows mobile ...<br>--<br>Vernon<br><br></p>
<div class="gmail_quote">On Thu, May 13, 2010 at 8:30 AM, Vernon Cole <span dir="ltr">&lt;<a href="mailto:vernondcole <at> gmail.com">vernondcole@...</a>&gt;</span> wrote:<br><blockquote class="gmail_quote">Great! <br>Now I HAVE to upgrade by Windows Mobile phone!<br>;-)<br>--<br>Vernon Cole<br><br><div class="gmail_quote">
<div>
<div></div>
<div class="h5">On Thu, May 13, 2010 at 8:26 AM,  <span dir="ltr">&lt;<a href="mailto:asierra01@..." target="_blank">asierra01@...</a>&gt;</span> wrote:<br>
</div>
</div>
<blockquote class="gmail_quote">
<div>
<div></div>
<div class="h5">WHAT, ok this is good news,
<br><br>  I will remove the use of extension. I dont really need a fast storm performance. I just need the ORM
<br>  features. I have, the java way of accessing sqlite3 databases under android, or or I can use python+sqlite3
<br>  or even better python+sqlite3+storm so my fields mapping will be easy/almost seamlessly.
<br><br>  I will even post my results here or somewhere else, if someone wants to use storm under android.
<br><br>  I renamed the module storm to storm_android, 
<br><br>from storm_android.locals import *
<br>from <a href="http://storm_android.info" target="_blank">storm_android.info</a> import *
<br>from storm_android.properties import *
<br><br>  I can put it back, I will write how to do some test and put it on the emulator, and run it.<br>
</div>
</div>--<div class="im">
<br>
storm mailing list<br><a href="mailto:storm@..." target="_blank">storm <at> lists.canonical.com</a><br>
Modify settings or unsubscribe at: <a href="https://lists.ubuntu.com/mailman/listinfo/storm" target="_blank">https://lists.ubuntu.com/mailman/listinfo/storm</a><br><br>
</div>
</blockquote>
</div>
<br>
</blockquote>
</div>
<br>
</div>
Permalink | Reply | 0 comments |
headers
Jorge Sierra | 13 May 2010 16:38
Picon

Re: hey guys

You can take a look at my project  here

http://sierramobiledeveloper.blogspot.com

I can release/publish my C++ ( MFC C++, cppsqlite3, sqlite3 C engine) 
code, 70+ tables
I am migrating the database code to android, and is experimental now.

But If I can run storm, this will help my migration from 1-2 months to 1 
week.

All non GUI related code python-storm
All GUI related code I plan to use their Java GUI classes and make the 
python code, interact with GUI
using some json-rpc mechanism.

crazy ?
Permalink | Reply | 0 comments |

Gmane