Jason Baker | 3 Aug 15:46 2009

How does storm handle dependencies for DELETEs?

I've been working on the following class for some of our integration
tests (these are the relevant parts):

class RowTracker(object):
    def __init__(self, store):
        self.store = store
        self.tracked_objs = []

    def track(self, obj):
        if obj not in self.tracked_objs:
            self.tracked_objs.append(obj)

    def add(self, obj):
        self.store.add(obj)
        self.track(obj)

    def revert(self):
        self.tracked_objs.reverse()
        for obj in self.tracked_objs:
            self.store.remove(obj)
        try:
            self.store.commit()
        except:
            print "++++ exception on reverting changes ++++"
            print "Tracked objects:  %s" % self.tracked_objs
            raise
        self.tracked_objs = []

The idea being that when I add an object to it, it will be created in
the database.  When I call the revert method, the objects should be
(Continue reading)

Stuart Bishop | 4 Aug 11:55 2009

All bugs for 0.15 fix committed - release time?

https://launchpad.net/storm/+milestone/0.15

All bugs that have been targetted to 0.15 have been addressed.

Time to squeeze out that release?

--

-- 
Stuart Bishop <stuart@...>
http://www.stuartbishop.net/

Jamu Kakar | 4 Aug 18:09 2009
Picon

Re: All bugs for 0.15 fix committed - release time?

Hi Stuart,

On Tue, Aug 4, 2009 at 2:55 AM, Stuart
Bishop<stuart.bishop@...> wrote:
> https://launchpad.net/storm/+milestone/0.15
>
> All bugs that have been targetted to 0.15 have been addressed.
>
> Time to squeeze out that release?

Ooh nice, thanks for the heads up.  I'll prepare an 0.15 release
this week.

Thanks,
J.

akm | 4 Aug 19:38 2009
Picon

join and subquery

Hi All,

I have a query that looks like this: Is there any way to convert this to storm ?

 SELECT *
 FROM Table1 me
 LEFT JOIN (
                             SELECT MAX(crt_dt) crt_dt,
                                    test_idn
                             FROM Table2
                             WHERE test_cd = 'Completed'
                             GROUP BY test_idn)  inner_query1
                  ON inner_query1.test_idn = me.test_idn
 LEFT JOIN (
                             SELECT MAX(crt_dt) crt_dt,
                                    test_idn
                             FROM Table2
                             WHERE test_cd = 'Closed'
                             GROUP BY test_idn)  inner_query2
                  ON inner_query2.test_idn = me.test_idn

akm <at> akm-laptop:~$ sqlite3
SQLite version 3.4.2
Enter ".help" for instructions
sqlite> Create table Table1( test_idn numeric(18) );
sqlite> insert into Table1 values (1);
sqlite> insert into Table1 values (2);
sqlite> Create table Table2( test_idn numeric(18), test_cd
varchar(50), crt_dt datetime );
sqlite> insert into Table2 values (1, 'Closed', date('now'));
(Continue reading)

James Henstridge | 5 Aug 08:27 2009
Picon

Re: Limit cache size by memory usage?

On Thu, Jul 30, 2009 at 12:05 PM, Stuart
Bishop<stuart.bishop@...> wrote:
> On Thu, Jul 30, 2009 at 1:29 AM, Gustavo Niemeyer<gustavo@...> wrote:
>>> Generally, trying to manage memory yourself inside an application can
>>> work against you. See the ArchitectNotes for Varnish for a general
>>> overview of why and how:
>>
>> Yeah, I'm kind of concerned about it too, since there are so many
>> details in the way (Python's internal memory buffers, C library
>> allocation behavior, the application's use of objects, etc).
>> Promising an auto-tweaking cache which behaves poorly would be worse
>> than advising developers to tweak their cache size to fit their needs.
>
> I agree that auto-tweaking caches that behave poorly are not a good
> idea :-) I'm fishing for ideas on how to get one that doesn't behave
> poorly, or at least better than my first experiment. It may be
> impossible.

Perhaps it would be possible to implement some kind of profiling cache
implementation would be worthwhile then: something that would record
cache hits and misses, and some utilities to analyse that data.

> I do feel that if I had a half decent implementation and could tell a
> program 'Use up to 2GB of RAM', that would be useful for almost all of
> the code we deploy. Our current mechanism is to make an educated guess
> as to the cache size and forget about it unless performance or RAM
> consumption is an issue (multiple bits of code sharing a server means
> swapping is bad). Even if went to the trouble of calculating the ideal
> cache size, soon it would no longer be ideal as our data is constantly
> changing - the optimal cache size for importing a pofile with small
(Continue reading)

James Henstridge | 5 Aug 08:30 2009
Picon

Re: All bugs for 0.15 fix committed - release time?

On Wed, Aug 5, 2009 at 12:09 AM, Jamu Kakar<jkakar@...> wrote:
> Hi Stuart,
>
> On Tue, Aug 4, 2009 at 2:55 AM, Stuart
> Bishop<stuart.bishop@...> wrote:
>> https://launchpad.net/storm/+milestone/0.15
>>
>> All bugs that have been targetted to 0.15 have been addressed.
>>
>> Time to squeeze out that release?
>
> Ooh nice, thanks for the heads up.  I'll prepare an 0.15 release
> this week.

It would be nice to get lp:~jamesh/storm/bug-268151 merged.  The C
extension doesn't currently build under Windows, and this branch has
been reported to fix that problem.

James.

--

-- 
storm mailing list
storm@...
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/storm

Stuart Bishop | 5 Aug 09:30 2009

Re: Limit cache size by memory usage?



On Wed, Aug 5, 2009 at 1:27 PM, James Henstridge<james <at> jamesh.id.au> wrote:
> On Thu, Jul 30, 2009 at 12:05 PM, Stuart
> Bishop<stuart.bishop <at> canonical.com> wrote:
>> On Thu, Jul 30, 2009 at 1:29 AM, Gustavo Niemeyer<gustavo <at> niemeyer.net> wrote:
>>>> Generally, trying to manage memory yourself inside an application can
>>>> work against you. See the ArchitectNotes for Varnish for a general
>>>> overview of why and how:
>>>
>>> Yeah, I'm kind of concerned about it too, since there are so many
>>> details in the way (Python's internal memory buffers, C library
>>> allocation behavior, the application's use of objects, etc).
>>> Promising an auto-tweaking cache which behaves poorly would be worse
>>> than advising developers to tweak their cache size to fit their needs.
>>
>> I agree that auto-tweaking caches that behave poorly are not a good
>> idea :-) I'm fishing for ideas on how to get one that doesn't behave
>> poorly, or at least better than my first experiment. It may be
>> impossible.
>
> Perhaps it would be possible to implement some kind of profiling cache
> implementation would be worthwhile then: something that would record
> cache hits and misses, and some utilities to analyse that data.

Hmm... yes. That might be a better approach.


>> I do feel that if I had a half decent implementation and could tell a
>> program 'Use up to 2GB of RAM', that would be useful for almost all of
(Continue reading)

Jamu Kakar | 6 Aug 08:53 2009
Picon

Re: All bugs for 0.15 fix committed - release time?

Hi James,

On Tue, Aug 4, 2009 at 11:30 PM, James Henstridge<james <at> jamesh.id.au> wrote:
> It would be nice to get lp:~jamesh/storm/bug-268151 merged.  The C
> extension doesn't currently build under Windows, and this branch has
> been reported to fix that problem.

I think this is important enough to hold up the release.  I've added
the bug to the 0.15 milestone.  Have you (or anybody) else tested
the patch on Windows?  I've reviewed the code and tested it on
Ubuntu and it at least doesn't break anything here.  I don't have a
Windows environment to test it in, or I'd do it myself.

Thanks,
J.

--

-- 
storm mailing list
storm <at> lists.canonical.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/storm
Jason Baker | 6 Aug 15:36 2009

All bugs for 0.15 fix committed - release time?

On Thu, Aug 6, 2009 at 1:53 AM, Jamu Kakar<jkakar@...> wrote:
> Hi James,
>
> On Tue, Aug 4, 2009 at 11:30 PM, James Henstridge<james@...> wrote:
>> It would be nice to get lp:~jamesh/storm/bug-268151 merged.  The C
>> extension doesn't currently build under Windows, and this branch has
>> been reported to fix that problem.
>
> I think this is important enough to hold up the release.  I've added
> the bug to the 0.15 milestone.  Have you (or anybody) else tested
> the patch on Windows?  I've reviewed the code and tested it on
> Ubuntu and it at least doesn't break anything here.  I don't have a
> Windows environment to test it in, or I'd do it myself.
>
> Thanks,
> J.
>
> --
> storm mailing list
> storm@...
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/storm
>

If I build this branch on Windows, I get output that looks like this.
I'm sure it's something that I have misconfigured.  Unfortunately, I'm
not very skilled at troubleshooting gcc errors. Any advice?

C:\MinGW\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.5\Release\storm\c
extensions.o build\temp.win32-2.5\Release\storm\cextensions.def -LC:\Python25\li
bs -LC:\Python25\PCbuild -lpython25 -lmsvcr71 -o build\lib.win32-2.5\storm\cexte
(Continue reading)

Brad Allen | 6 Aug 17:16 2009

Re: All bugs for 0.15 fix committed - release time?

On Thu, Aug 6, 2009 at 12:23 PM, Jamu Kakar <jkakar-kvr/BIsI4Rg@public.gmane.org> wrote:

Hi James,

On Tue, Aug 4, 2009 at 11:30 PM, James Henstridge<james-whk3OXXaEJgXC2x5gXVKYQ@public.gmane.org> wrote:
> It would be nice to get lp:~jamesh/storm/bug-268151 merged.  The C
> extension doesn't currently build under Windows, and this branch has
> been reported to fix that problem.

I think this is important enough to hold up the release.  I've added
the bug to the 0.15 milestone.  Have you (or anybody) else tested
the patch on Windows?  I've reviewed the code and tested it on
Ubuntu and it at least doesn't break anything here.  I don't have a
Windows environment to test it in, or I'd do it myself.

If you're holding up the release a few days, maybe it's worth considering getting the Oracle backend included. Jason Baker has gotten all the unit tests to pass with a branch taken from trunk a couple of months ago, and we are getting closer to putting it into production use at ZeOmega. We just need to take a new branch from trunk, add the Oracle backend (along with some additional tests and test modifications) and see if tests are still passing.

Alternately, if you don't want to hold up the release of 0.15, hopefully we can still have it added to the trunk soon after.

--
ZeOmega
Open Minds' Open Solutions
3010 Gaylord Parkway, Ste. 210
Frisco TX, 75034
http://www.zeomega.com

Brad Allen
214-618-9880 (ext. 8006)
<div>
<p>On Thu, Aug 6, 2009 at 12:23 PM, Jamu Kakar <span dir="ltr">&lt;<a href="mailto:jkakar@...">jkakar@...</a>&gt;</span> wrote:<br></p>
<div class="gmail_quote">
<blockquote class="gmail_quote">
Hi James,<br><div class="im">
<br>
On Tue, Aug 4, 2009 at 11:30 PM, James Henstridge&lt;<a href="mailto:james <at> jamesh.id.au">james@...</a>&gt; wrote:<br>
&gt; It would be nice to get lp:~jamesh/storm/bug-268151 merged. &nbsp;The C<br>
&gt; extension doesn't currently build under Windows, and this branch has<br>
&gt; been reported to fix that problem.<br><br>
</div>I think this is important enough to hold up the release. &nbsp;I've added<br>
the bug to the 0.15 milestone. &nbsp;Have you (or anybody) else tested<br>
the patch on Windows? &nbsp;I've reviewed the code and tested it on<br>
Ubuntu and it at least doesn't break anything here. &nbsp;I don't have a<br>
Windows environment to test it in, or I'd do it myself.<br>
</blockquote>
<div>
<br>If you're holding up the release a few days, maybe it's worth considering getting the Oracle backend included. Jason Baker has gotten all the unit tests to pass with a branch taken from trunk a couple of months ago, and we are getting closer to putting it into production use at ZeOmega. We just need to take a new branch from trunk, add the Oracle backend (along with some additional tests and test modifications) and see if tests are still passing.<br>
</div>
</div>
<br>Alternately, if you don't want to hold up the release of 0.15, hopefully we can still have it added to the trunk soon after.<br clear="all"><br>-- <br>ZeOmega<br>Open Minds' Open Solutions<br>3010 Gaylord Parkway, Ste. 210<br>
Frisco TX, 75034<br><a href="http://www.zeomega.com">http://www.zeomega.com</a><br><br>Brad Allen<br>214-618-9880 (ext. 8006)<br>
</div>

Gmane