headers
Tony Bibbs | 15 Apr 2009 22:24
Gravatar

DAO for Propel with Named Query support.

Ok, I've managed to get around to polishing off a DAO implementation
for Propel.  What makes this implementation unique is it adds a
powerful alternative to Criteria in what is dubbed Named Queries.
Named Queries are nothing more than XML files containing SQL queries
that are given names.  Those names can then be used to execute a
specific query.  Why, would anybody want this?

- It's a good middle ground between portability and the desire to
prevent having to learn a new way to write queries (e.g. Criteria or
something like HQL for you Hibernate fans).
- Allows a single SQL query to be reused in multiple parts of your application.
- It keeps your application clean by taking explicit SQL calls out of your code
- Allows for quick regression testing.  Because SQL is centralized
into one (or more) files you can easily write a unit test to
regression test all your queries after a database upgrade.
- Just like Criteria it returns Propel objects OR raw data.

How does it work?

- First, all Named Query files are validated against the included XSD
- XML is compiled into a PHP array for faster, subsequent access.
- Calls to a specific query cause the query to be pulled from the
array and then ran through a PDO prepared statement.

There are two parts to this implementation:

1) Generator (optional): we add a new PEER builder class that is DAO
aware.  The only thing this builder does special is it adds a
__callStatic() method (new to PHP 5.3) that knows how to call named
queries by name (example later).  To use it simply change set the
(Continue reading)

Permalink | Reply | 4 comments |
headers
Cameron Brunner | 15 Apr 2009 22:49
Picon

Re: DAO for Propel with Named Query support.

I've seen this mentioned a few times before and I don't think it would
be a 'bad' addition to propel but reading the existing code... there's
more than a couple things that need touching up (read: slow) and it
would need to hook into a caching module instead of having apc_*
hardcoded which means creating an interface for the caching module and
making a null/xcache/apc/memcache object to suit the interface as
well, probably some more things, it was only a quick glance at the
code. Personally I would rarely (probably never) use this as every
time I see this I think about the overhead this will create when you
get 1k+ queries in your xml file (too many people write bad code even
with the best help) but that's just me.

On Thu, Apr 16, 2009 at 6:24 AM, Tony Bibbs <tony <at> tonybibbs.com> wrote:
> Ok, I've managed to get around to polishing off a DAO implementation
> for Propel.  What makes this implementation unique is it adds a
> powerful alternative to Criteria in what is dubbed Named Queries.
> Named Queries are nothing more than XML files containing SQL queries
> that are given names.  Those names can then be used to execute a
> specific query.  Why, would anybody want this?
>
> - It's a good middle ground between portability and the desire to
> prevent having to learn a new way to write queries (e.g. Criteria or
> something like HQL for you Hibernate fans).
> - Allows a single SQL query to be reused in multiple parts of your application.
> - It keeps your application clean by taking explicit SQL calls out of your code
> - Allows for quick regression testing.  Because SQL is centralized
> into one (or more) files you can easily write a unit test to
> regression test all your queries after a database upgrade.
> - Just like Criteria it returns Propel objects OR raw data.
>
(Continue reading)

Permalink | Reply | 3 comments |
headers
Tony Bibbs | 15 Apr 2009 23:51
Gravatar

Re: DAO for Propel with Named Query support.

FWIW, you can have any number of XML files to address that concern.  
With the exception of VERY large apps I'd be surprised if you had  
anything approaching 1,000 queries because of the reuse you gain. Also  
because the XML is compiled into PHP you get the benefit of opcode  
caching. We're using this implementation on a number of production  
sites with no noticeable performance issues, though it would be worth  
running a few benchmarks against Criteria.

--Tony

On Apr 15, 2009, at 3:49 PM, Cameron Brunner  
<cameron.brunner <at> gmail.com> wrote:

> I've seen this mentioned a few times before and I don't think it would
> be a 'bad' addition to propel but reading the existing code... there's
> more than a couple things that need touching up (read: slow) and it
> would need to hook into a caching module instead of having apc_*
> hardcoded which means creating an interface for the caching module and
> making a null/xcache/apc/memcache object to suit the interface as
> well, probably some more things, it was only a quick glance at the
> code. Personally I would rarely (probably never) use this as every
> time I see this I think about the overhead this will create when you
> get 1k+ queries in your xml file (too many people write bad code even
> with the best help) but that's just me.
>
> On Thu, Apr 16, 2009 at 6:24 AM, Tony Bibbs <tony <at> tonybibbs.com>  
> wrote:
>> Ok, I've managed to get around to polishing off a DAO implementation
>> for Propel.  What makes this implementation unique is it adds a
>> powerful alternative to Criteria in what is dubbed Named Queries.
(Continue reading)

Permalink | Reply | 2 comments |
headers
Cameron Brunner | 16 Apr 2009 00:20
Picon

Re: DAO for Propel with Named Query support.

The first thing I noticed and the only reason I mentioned performance
is that you are doing multiple xpath queries in at least one of the
pieces of code where you could just as easily be doing one and doing
some more method calls instead, compared to Criteria it probably is
faster for a lot of things, it's just something I noticed (I don't
even know if it was in the code to generate the cache or what, I just
remember seeing it)

On Thu, Apr 16, 2009 at 7:51 AM, Tony Bibbs <tony <at> tonybibbs.com> wrote:
> FWIW, you can have any number of XML files to address that concern.
> With the exception of VERY large apps I'd be surprised if you had
> anything approaching 1,000 queries because of the reuse you gain. Also
> because the XML is compiled into PHP you get the benefit of opcode
> caching. We're using this implementation on a number of production
> sites with no noticeable performance issues, though it would be worth
> running a few benchmarks against Criteria.
>
> --Tony
>
>
>
> On Apr 15, 2009, at 3:49 PM, Cameron Brunner
> <cameron.brunner <at> gmail.com> wrote:
>
>> I've seen this mentioned a few times before and I don't think it would
>> be a 'bad' addition to propel but reading the existing code... there's
>> more than a couple things that need touching up (read: slow) and it
>> would need to hook into a caching module instead of having apc_*
>> hardcoded which means creating an interface for the caching module and
>> making a null/xcache/apc/memcache object to suit the interface as
(Continue reading)

Permalink | Reply | 1 comment |
headers
Tony Bibbs | 16 Apr 2009 04:03
Gravatar

Re: DAO for Propel with Named Query support.

Yeah, that code has room for improvement but keep in mind once the XML  
is compiled into PHP form we don't use xpath (or XML)...we just load  
the generated PHP file. We only go back to the XML file if you change  
it and, again, we simply recompile it to PHP.

On Apr 15, 2009, at 5:20 PM, Cameron Brunner  
<cameron.brunner <at> gmail.com> wrote:

> The first thing I noticed and the only reason I mentioned performance
> is that you are doing multiple xpath queries in at least one of the
> pieces of code where you could just as easily be doing one and doing
> some more method calls instead, compared to Criteria it probably is
> faster for a lot of things, it's just something I noticed (I don't
> even know if it was in the code to generate the cache or what, I just
> remember seeing it)
>
> On Thu, Apr 16, 2009 at 7:51 AM, Tony Bibbs <tony <at> tonybibbs.com>  
> wrote:
>> FWIW, you can have any number of XML files to address that concern.
>> With the exception of VERY large apps I'd be surprised if you had
>> anything approaching 1,000 queries because of the reuse you gain.  
>> Also
>> because the XML is compiled into PHP you get the benefit of opcode
>> caching. We're using this implementation on a number of production
>> sites with no noticeable performance issues, though it would be worth
>> running a few benchmarks against Criteria.
>>
>> --Tony
>>
>>
(Continue reading)

Permalink | Reply | 0 comments |
headers
Tony Bibbs | 16 Apr 2009 17:59
Gravatar

Fwd: Moving settings from from schema.xml to property files

Fellow devs,

With the looming PHP5.3 release there is one thing yet to be cleaned
up in the Propel codebase.  As many of you know I committed namespace
support into the trunk some time ago.  You can now set the default
namespace for OM, Peer, Maps, etc.  The only thing missing is the
ability to override a namespace on a per-table basis.

My original thought was to just add a "namespace" attribute on the
<table> tag in schema.xml.  This, however, is a bit confusing when
used in conjunction with the "package" attribute which, to new Propel
users, might seem like the same thing.  My original way to fix this
was to simply deprecate the use of the "package" attribute and use
something like an "outputDIrectory" attribute instead.  In an IRC
chat with Ron, he came up with I felt was a good way to not only fix
this specific problem but to clean things up a bit: put it all in
property files.

The idea is that schema.xml would only contain meta data describing
the database.  This may not seem like a deviation from today but the
notable change in philosophy is that anything related to how
files get built or where files are outputted to would be moved to
property files. The idea then is we'd have settings like:

table.blogpost.namespace = \MyApp\Plugins\Blog
table.blogpost.outputDirectory = /path/to/MyApp/plugins/Blog/

Only draw back I see is in the case where you rename a table.  That'd
mean you'd have to touch both the schema.xml and (possibly) the
properties file.  I say possibly there because we'd have default
(Continue reading)

Permalink | Reply | 8 comments |
headers
Alan Pinstein | 16 Apr 2009 18:56
Picon
Gravatar

Re: Fwd: Moving settings from from schema.xml to property files

Tony-

I have long advocated a 2-stage approach to metadata, one for purely  
DATA description, and the other for ORM metadata for said DATA.

I think ultimately there should be 2 xml files; schema.xml which is  
just the database setup, and schema-orm.xml which would have a similar  
"gross" xml structure but would allow addition of ORM-specific  
metadata such as you suggest.

This would allow schema.xml to migrate over time to a standard "DDL"  
schema which would give propel users better integration with toolchain  
workflows for database management while allowing us to do whatever we  
want in Propel to aid in the ORM layer. Even your "named queries"  
could potentially go in schema-orm.xml.

I don't really like build.properties for several reasons:
1) It's hard to control its use b/c it's included from phing internals
2) It doesn't support hierarchical stuff very well (for individual  
tables for example)
3) It is hard to manage for multiple environments (dev/staging/ 
production)

If you agree with the concepts behind my architecture, I'd see if I  
can help in building it.

LMK!

Alan
(Continue reading)

Permalink | Reply | 7 comments |
headers
Even André Fiskvik | 16 Apr 2009 19:00
Picon

Re: Fwd: Moving settings from from schema.xml to property files

+1 from me. I'm really looking forward to namespace support (and the  
final release of 5.3 of course),
hopefully I'll be able to clean up my codebase a bit more :)

Even André Fiskvik

On 16. april. 2009, at 18.56, Alan Pinstein wrote:

> Tony-
>
> I have long advocated a 2-stage approach to metadata, one for purely
> DATA description, and the other for ORM metadata for said DATA.
>
> I think ultimately there should be 2 xml files; schema.xml which is
> just the database setup, and schema-orm.xml which would have a similar
> "gross" xml structure but would allow addition of ORM-specific
> metadata such as you suggest.
>
> This would allow schema.xml to migrate over time to a standard "DDL"
> schema which would give propel users better integration with toolchain
> workflows for database management while allowing us to do whatever we
> want in Propel to aid in the ORM layer. Even your "named queries"
> could potentially go in schema-orm.xml.
>
> I don't really like build.properties for several reasons:
> 1) It's hard to control its use b/c it's included from phing internals
> 2) It doesn't support hierarchical stuff very well (for individual
> tables for example)
> 3) It is hard to manage for multiple environments (dev/staging/
> production)
(Continue reading)

Permalink | Reply | 0 comments |
headers
Tony Bibbs | 20 Apr 2009 20:51
Gravatar

Re: Fwd: Moving settings from from schema.xml to property files

Alan, sounds like you mostly agree with what Ron and I were tossing
around.  Some thoughts on your post below.

Hans, I'd be interested in your $0.02 when you get a minute to put
that Python code down ;-)

...same for some of you other, more seasoned, Propel developers.

On Thu, Apr 16, 2009 at 11:56 AM, Alan Pinstein <apinstein <at> mac.com> wrote:

> I don't really like build.properties for several reasons:
> 1) It's hard to control its use b/c it's included from phing internals

Can you expand on this?  The use of Phing is sort of key to the entire
architecture so decoupling now would be a bit of a pain.

> 2) It doesn't support hierarchical stuff very well (for individual
> tables for example)

Agreed.

> 3) It is hard to manage for multiple environments (dev/staging/
> production)

To me this is a non-issue because you only really work with the
property files during development.  The only useful configuration file
out of Propel for dev/staging/production is the runtime stuff.  Am I
missing something (maybe in the details of how your organization uses
Propel)?
(Continue reading)

Permalink | Reply | 5 comments |
headers
Alan Pinstein | 20 Apr 2009 21:24
Picon
Gravatar

Re: Fwd: Moving settings from from schema.xml to property files

>> I don't really like build.properties for several reasons:
>> 1) It's hard to control its use b/c it's included from phing  
>> internals
>
> Can you expand on this?  The use of Phing is sort of key to the entire
> architecture so decoupling now would be a bit of a pain.
>
>> 2) It doesn't support hierarchical stuff very well (for individual
>> tables for example)
>
> Agreed.
>

>> 3) It is hard to manage for multiple environments (dev/staging/
>> production)
>
> To me this is a non-issue because you only really work with the
> property files during development.  The only useful configuration file
> out of Propel for dev/staging/production is the runtime stuff.  Am I
> missing something (maybe in the details of how your organization uses
> Propel)?

[I'll hit all these together...]

I understand that build.properties part of phing and am not suggesting  
to stop using it.

I am only suggesting not to use it for ORM metadata as much as possible.

FWIW I do build from multiple development machines (and developers) so
(Continue reading)

Permalink | Reply | 3 comments |

Gmane