headers
Brad Allen | 1 Jul 2009 03:54
Favicon

Re: Should storm have separate String and Blob datatypes?

This issue came up because our app only supports ASCII strings, not
unicode. Currently we use the RawStr property for attributes mapped to
SQL Server VARCHAR fields, which has been working fine for us with the
MS SQL backend.

Howevever, we're working toward Oracle compatibility now for our app
(while still continuing MS SQL support), and we're finding that RawStr
isn't appropriate for VARCHAR since RawStr needs to be hexified to
support BLOB. We're discussing creating an Ascii property to use for
mapping to VARCHAR.

Is that the best approach, or does anyone have a different recommendation?

Thanks!

--

-- 
Brad Allen
ZeOmega
Open Minds' Open Solutions
3010 Gaylord Parkway, Ste. 210
Frisco TX, 75034
http://www.zeomega.com
Permalink | Reply | 9 comments |
headers
James Henstridge | 1 Jul 2009 05:42
Picon
Gravatar

Re: Should storm have separate String and Blob datatypes?

On Wed, Jul 1, 2009 at 9:54 AM, Brad Allen<ballen@...> wrote:
> This issue came up because our app only supports ASCII strings, not
> unicode. Currently we use the RawStr property for attributes mapped to
> SQL Server VARCHAR fields, which has been working fine for us with the
> MS SQL backend.

Okay.  That is not a recommended way to use Storm.  Storm is working
with a model close to Python 3: use unicode for textual data, byte
strings for binary data and don't automatically coerce one to the
other.

Given the naming that Python eventually settled on, it might be worth
renaming RawStr() to Bytes() to cut down on confusion.

> Howevever, we're working toward Oracle compatibility now for our app
> (while still continuing MS SQL support), and we're finding that RawStr
> isn't appropriate for VARCHAR since RawStr needs to be hexified to
> support BLOB. We're discussing creating an Ascii property to use for
> mapping to VARCHAR.
>
> Is that the best approach, or does anyone have a different recommendation?

There has been suggestions for an "encoded string" property that would
present byte strings to Python and do the relevant encode/decode when
going to/from the database.  That would handle the ASCII case as well
as other encodings.

James.
(Continue reading)

Permalink | Reply | 8 comments |
headers
James Henstridge | 1 Jul 2009 06:33
Picon
Gravatar

Re: Beginner question: adding referenced objects to store

On Wed, Jul 1, 2009 at 2:43 AM, Neil de Carteret<storm-list@...> wrote:
> I've come unstuck while trying to learn the basics of Storm by writing
> a basic wiki:
>
> I have a simple many-to-many relationship, between page and tags. I
> have Page class, a Tag class, and a PageTag class, and three matching
> tables. When I set the tags used by a page, I'd like to able to say:
>
>>>> page.tags = [Tag(u"mytag"), Tag(u"othertag"), Tag(u"thirdtag")]

This syntax isn't supported.  The fact that you can execute it with no
error sounds like a bug (probably need to add a ReferenceSet.__set__()
method).  Could you file a bug about this?

> where "page" is an instance of my Page storm class, and Tag is the
> storm class for tags (with an appropriate __init__.)
>
> The problem I'm having is that the tag objects are not getting
> persisted when I "store.commit()".
>
> If I explicitly say:
>
>>>> mytag = Tag(u"mytag")
>>>> store.add(mytag)
>>>> ...etc
>
> Then the tags are saved. Which is a start, but still nothing goes into
> the linker table. And because I don't have any actual PageTag
> instances to play with, I can't store.add() them by hand.
>
(Continue reading)

Permalink | Reply | 502 comments |
headers
Gustavo Niemeyer | 1 Jul 2009 14:35
Favicon
Gravatar

Re: Should storm have separate String and Blob datatypes?

> Given the naming that Python eventually settled on, it might be worth
> renaming RawStr() to Bytes() to cut down on confusion.

Agreed, we should implement support for Bytes, except I think we
shouldn't rename RawStr to it, but rather use Bytes exclusively to
represent the new "bytes" type, offering people the chance of a smooth
migration (Python 2.6 has both strings and bytes).

Out of curiosity, the name of RawStr is also like that to avoid
confusion with the upcoming naming conventions.  "str" in Python 3.0
is actually unicode, so it'll be natural to have Str in Storm some day
to represent it.

--

-- 
Gustavo Niemeyer
http://niemeyer.net
Permalink | Reply | 7 comments |
headers
James Henstridge | 1 Jul 2009 15:50
Picon
Gravatar

Re: Should storm have separate String and Blob datatypes?

On Wed, Jul 1, 2009 at 8:35 PM, Gustavo Niemeyer<gustavo@...> wrote:
>> Given the naming that Python eventually settled on, it might be worth
>> renaming RawStr() to Bytes() to cut down on confusion.
>
> Agreed, we should implement support for Bytes, except I think we
> shouldn't rename RawStr to it, but rather use Bytes exclusively to
> represent the new "bytes" type, offering people the chance of a smooth
> migration (Python 2.6 has both strings and bytes).

Actually, Python 2.6 doesn't have a separate bytes type.  It is just
an alias for the existing byte string type:

    $ python
    Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
    [GCC 4.3.3] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> bytes
    <type 'str'>
    >>> type(bytes('foo'))
    <type 'str'>
    >>> type(b'foo')
    <type 'str'>

This is pretty close to Python 3 where stringobject.c was renamed
bytesobject.c: they have never existed in parallel.  So I don't think
we want separate Bytes() and RawStr().

For what it is worth, we seem to have another alias for RawStr().  In
storm/properties.py we have:
(Continue reading)

Permalink | Reply | 6 comments |
headers
Jason Baker | 1 Jul 2009 16:37
Favicon

Oracle back end now passing all tests in tests.store and tests.database

As of rev 339, the Oracle backend we've been working on is passing all
the tests in both tests.store and tests.databases.  I've been working
in lp:~zeomega/storm/zeomega_storm, which is mainly a somewhat heavily
modified branch of lp:~kov-alfaiati/storm/oracle-support with most of
Drew Smather's patches (lp:~djfroofy/storm/oracle-support-patches)
added.  It's definitely far from being in a perfect state (as you can
see from the bugs that are open), but it's getting better.  :-)

That said, here are some issues that may pop up if you're switching
from the current Oracle branch:

 * Binary data is no longer encoded as base64.  Instead, the data will
be hexlified, as Oracle expects binary data.
 * The back end now requires cx_Oracle >= 5.  This is because
cx_Oracle 5 makes type handling much easier by way of input type
handlers and output type handlers.
 * For ORA-08177 (a DatabaseError), the back end will now raise an
OperationalError with the message 'database is locked' as storm
expects.
 * The back end now keeps its own store of reserved words and escapes
them in mostly the same manner as storm itself does.  I did this
because the reserved word list in storm is a bit dated (from SQL 92)
and contains 159 words that aren't reserved in Oracle 10g.  For
reference, Oracle 10g has about 66 reserved words.

Some other things I've added:

 * Support for connecting using a TNS name.
 * "Isolation contexts" - essentially a way to temporarily put the
database in read committed mode temporarily using a with statement.  I
(Continue reading)

Permalink | Reply | 1 comment |
headers
Gustavo Niemeyer | 1 Jul 2009 16:44
Favicon
Gravatar

Re: Should storm have separate String and Blob datatypes?

> Actually, Python 2.6 doesn't have a separate bytes type.  It is just
> an alias for the existing byte string type:

Ouch.. from the discussions during development I understood that it'd
actually be the real bytes type.

> This is pretty close to Python 3 where stringobject.c was renamed
> bytesobject.c: they have never existed in parallel.  So I don't think
> we want separate Bytes() and RawStr().

It wasn't just renamed it's a very differently behaving type:

>>> bytes(b"foo")[0]
102

So I still think we should make Bytes support the real bytes type, and
keep RawStr on the old str.   This will likely yield startup
exceptions rather than late runtime ones, and will help people
upgrading it smoothly.

> For what it is worth, we seem to have another alias for RawStr().  In
> storm/properties.py we have:
>
>    # OBSOLETE RawStr was Chars in 0.9. This will die soon.
>    Chars = RawStr
>
> We can probably get rid of that one now.

+1!
(Continue reading)

Permalink | Reply | 5 comments |
headers
James Henstridge | 1 Jul 2009 17:48
Picon
Gravatar

Re: Should storm have separate String and Blob datatypes?

On Wed, Jul 1, 2009 at 10:44 PM, Gustavo Niemeyer<gustavo@...> wrote:
>> Actually, Python 2.6 doesn't have a separate bytes type.  It is just
>> an alias for the existing byte string type:
>
> Ouch.. from the discussions during development I understood that it'd
> actually be the real bytes type.
>
>> This is pretty close to Python 3 where stringobject.c was renamed
>> bytesobject.c: they have never existed in parallel.  So I don't think
>> we want separate Bytes() and RawStr().
>
> It wasn't just renamed it's a very differently behaving type:
>
>>>> bytes(b"foo")[0]
> 102

I know that there were behaviour changes.  What I'm getting at is that
there are no versions of Python that have both a "str" and "bytes"
type

> So I still think we should make Bytes support the real bytes type, and
> keep RawStr on the old str.   This will likely yield startup
> exceptions rather than late runtime ones, and will help people
> upgrading it smoothly.

I'm not sure how that would work.  The only versions of Python with a
dedicated "bytes" type are 3.x releases, which we won't be able to
support from the same code base as the one for Python 2.x.

The upgrade path in 2.6 is to treat "str" as the "bytes" type, so
(Continue reading)

Permalink | Reply | 4 comments |
headers
Drew Smathers | 1 Jul 2009 20:34
Picon

Re: Oracle back end now passing all tests in tests.store and tests.database

On Wed, Jul 1, 2009 at 10:37 AM, Jason Baker<jbaker <at> zeomega.com> wrote:
> As of rev 339, the Oracle backend we've been working on is passing all
> the tests in both tests.store and tests.databases.  I've been working
> in lp:~zeomega/storm/zeomega_storm, which is mainly a somewhat heavily
> modified branch of lp:~kov-alfaiati/storm/oracle-support with most of
> Drew Smather's patches (lp:~djfroofy/storm/oracle-support-patches)
> added.  It's definitely far from being in a perfect state (as you can
> see from the bugs that are open), but it's getting better.  :-)
>
> That said, here are some issues that may pop up if you're switching
> from the current Oracle branch:
>
>  * Binary data is no longer encoded as base64.  Instead, the data will
> be hexlified, as Oracle expects binary data.
>  * The back end now requires cx_Oracle >= 5.  This is because
> cx_Oracle 5 makes type handling much easier by way of input type
> handlers and output type handlers.
>  * For ORA-08177 (a DatabaseError), the back end will now raise an
> OperationalError with the message 'database is locked' as storm
> expects.
>  * The back end now keeps its own store of reserved words and escapes
> them in mostly the same manner as storm itself does.  I did this
> because the reserved word list in storm is a bit dated (from SQL 92)
> and contains 159 words that aren't reserved in Oracle 10g.  For
> reference, Oracle 10g has about 66 reserved words.
>
> Some other things I've added:
>
>  * Support for connecting using a TNS name.
>  * "Isolation contexts" - essentially a way to temporarily put the
(Continue reading)

Permalink | Reply | 0 comments |
headers
Neil de Carteret | 1 Jul 2009 20:48

Re: Beginner question: adding referenced objects to store

Thanks James. Bug duly filed.

I had read the tutorial, but I was hoping for a quick way to clear and
set all the related doodads at once, and tripped up because assigning
a sequence directly does work in another well-known Python ORM ;)

On Wed, Jul 1, 2009 at 5:33 AM, James Henstridge<james@...> wrote:
> This syntax isn't supported.  The fact that you can execute it with no
> error sounds like a bug (probably need to add a ReferenceSet.__set__()
> method).  Could you file a bug about this?

--

-- 
storm mailing list
storm@...
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/storm
Permalink | Reply | 2 comments |

Gmane