Pierre Minnieur | 16 May 17:04
Picon
Gravatar

[phpdba] DBAL definition

Hi,
 
I have some questions regarding the terms from the DBAL world, respective their roles, meanings and essentials in PHP. What I want to know is what's essential for a DBAL and what's useless and should be implemented elsewhere, because it has nothing to do with an abstraction layer or it is better abrogated elsewhere.
 
DBAL - Database Abstraction Layer:
 
Description:
A database abstraction layer provides a common API to access data located in a RDBMS.
 
 
Meaning:
  1. A common API for connecting and disconnecting to a RDBMS.
    We don't have to remember how mysqli can connect to a server, neither how PDO connections are established.
  2. A common API for sending queries to the database.
    This may include implementations for the different kinds of queries, e.g. SELECT, INSERT, UPDATE, DELETE to provide later usage of the results (e.g. return the last inserted id, the number of affected rows or the result set).
  3. A common API for transaction handling.
    Methods to begin, commit and rollback a transaction, including nested transactions.
  4. Features/Quoting/Wildcards/etc
    A list of features the connected RDBMS supports, e.g. if auto_increment or sequences are supported, how table/field names and their values have to be quoted/escaped, which wildcards can be used for several query operations (e.g. MySQL's LIKE) and so on. They could be used by other components to build queries.
  5. Primary Key handling (auto_increment/sequences)
    Some kind of an ID generator like Creole has, so we can easily handle IDs without knowing which mechanism is used by the RDBMS.
  6. Charset conversation.
    Ensures that the charset used is always correct and no wrong encoded data will find its way into the database.

Don'ts:

  • Field type conversation. Any piece of data is handled as it is delivired by the RDBMS.
  • Query builder. A DBAL only accepts plain text (strings) which will be send to the RDBMS directly.
  • RDBMS specific query syntax checking. The queries will be executed as provided.
I'd appreciate if you would correct me if I understand anything wrong, if you have suggestions or if something is missing in that definition.
 
Regards,
Pierre
Lukas Kahwe Smith | 8 May 16:41

[phpdba] database schema evolution

Hi,

I was hoping that posting to planet php would have generated a bit more 
feedback, so I will spam this list in the hopes of some more thoughts on 
the topic:
http://pooteeweet.org/blog/688

In that blog post I compare the "Ruby migrations" and "MDB2_Schema" 
approaches to schema evolution.

regards,
Lukas

Lukas Kahwe Smith | 24 Mar 18:06

[phpdba] [Fwd: Re: [Hackers] Portable schema definition format]

FYI

-------- Original Message --------
Subject: Re: [Hackers] Portable schema definition format
Date: Sat, 24 Mar 2007 09:41:09 -0700
From: Craig L Russell <Craig.Russell@...>
CC: Hackers@...
References: <46053576.5070300@...>

For a practical implementation, you might also want to take a look at
the DDLUtils open source project at Apache[1]. And there is a
standard for schema if you like standards. [2], [3], and [4] are the
result of years of investment by big players in the metadata
interchange arena. You will find everything there, but perhaps not
the simple, approachable, or easily implemented format that is the
hallmark of FOSS projects.

Craig

[1] http://db.apache.org/ddlutils/
[2] http://www.omg.org/technology/cwm/
[3] http://xml.coverpages.org/omg-cwmi.html
[4] http://www.omg.org/docs/formal/00-04-03.pdf

On Mar 24, 2007, at 7:28 AM, Lukas Kahwe Smith wrote:

> Hello,
>
> I recently opened a list to discuss topics related to database  
> independence within the PHP community [1]. One of the topics that  
> quickly materialized was in relation to finding a portable format  
> for managing database schema. The obvious choice for a portable  
> format would be XML, since its easy to parse and extend.
>
> There is currently a fairly well established XML format originally  
> defined many years ago by Manuel Lemos [2]. There is a DTD [3], XSD  
> [4] and XLT [5] available. However aside from support in the no  
> longer developed DBDesigner [6], only PHP packages support it [7][8] 
> [9]. It also has some unique aspects (like not using attributes and  
> originally being most focused on the MySQL 3.x feature set).
>
> Is there any such standard? Atleast we are not aware of anything in  
> that direction and non PHP people also seem to be looking [10]
>
> Anyways the question that was brought up was:
> What does it take to create a real industry standard for this?
>
> I would be very grateful for any feedback, pointers ..
>
> regards,
> Lukas
>
> [1] http://pooteeweet.org/blog/611
> [2] http://cvs.php.net/viewcvs.cgi/pear/MDB2_Schema/docs/ 
> xml_schema_documentation.html?view=co
> [3] http://cvs.php.net/viewcvs.cgi/pear/MDB2_Schema/docs/MDB.dtd? 
> view=co
> [4] http://cvs.php.net/viewcvs.cgi/pear/MDB2_Schema/docs/MDB.xsd? 
> view=co
> [5] http://cvs.php.net/viewcvs.cgi/pear/MDB2_Schema/docs/MDB.xsl? 
> view=co
> [6] http://fabforce.net/dbdesigner4/
> [7] http://www.phpclasses.org/browse/package/20.html
> [8] http://pear.php.net/MDB2_Schema
> [9] http://ez.no/doc/components/view/latest/(file)/ 
> introduction_DatabaseSchema.html
> [10] http://www.jgp.net/index.php? 
> option=com_content&task=view&id=45&Itemid=1
> _______________________________________________
> Hackers mailing list
> Hackers@...
> http://lists.osdbconsortium.org/mailman/listinfo/hackers

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@...
P.S. A good JDO? O, Gasp!

Adam Banko | 21 Mar 19:26
Picon

[phpdba] native ORM extension for php

Hi!

I'm building a native object persistence extension for PHP, and
interested in your opinion about it's API.

Here's the source: http://dbobj.sourceforge.net/bzr/dbobj/

I don't have any good documentation, but there are some working
examples, so please take a look at those (dbobj.php, forum.php,
bench.php) and tell my how do you like the API.

There's lot of discussion of this project in the php-dev list, but you,
ORM developers, are joint here.

Thanks,
	Adam

Thomas Koch | 15 Mar 16:45
Picon

[phpdba] eZ Components

Yes, I know there are many frameworks and I should not hype anyone in this 
list, but this email is also a response to Torsten Raudssus.

Hi Torsten,

you seem to live in Duesseldorf, so you're next to the PHPUG Dortmund, to eZ 
Systems and to me.
Because you talked about ORM, I'd like to recommend the eZ Components to you 
(and all the others). I'm not a developer, but only a fan of them. I'd 
appreciate very much, if they'd become a standard in the PHP world.

I like their testing, documenting and coding quality which isn't met by any 
other PHP project, as far as I know.

Best regards,

Thomas Koch

Torsten Raudssus | 14 Mar 18:18
Picon
Favicon

[phpdba] my 2 cent

Hi,

I've read lots of messages now about lots of themes, actually the 2 big 
ones: Database Schema and Database Abstraction API. Actually both are 
stuff that is not needed by anyone. I dont think you cant force the 
people to anything, so that discussion level is in my eyes very useless. 
I will never implement any Database Schema/Generation Stuff to DB_DO2, 
probably DB_DO3 or as Extra Stuff, because i see no deeper sense in it, 
because the database is always developed at work, and there are hundreds 
of cases that could happens (how do we need an update of a field, does 
it have a specific value, that is calculated out of the other fields, 
does the description kills old fields too). And the API.... i mean what 
is the reason that there are so many ORMs? Cause there are so many ways 
of doing that API. A standard would just block individual advantages. So 
i would like to open a discussion about the stuff that is clear for 
all.... i mean stuff that never blocks out anything. And i guess there 
are not so many points ;) We just can try to "animate" each other for 
new ideas.

My 2 Cent.... Thank you :)

Manuel Lemos | 13 Mar 04:56
Picon
Favicon
Gravatar

[phpdba] PHP DB Abstraction API standard specification

Hello,

on 03/08/2007 04:57 AM Lukas Kahwe Smith said the following:
>> For instance, what database API do you think Delphi for PHP supports?
> 
> Actually I was wondering the same thing, when I was looking over their
> press releases the other day. But again, I do not hope to consolidate
> the available solutions with this list. The goal is to pool know how to
> at least prevent each of us wasting time on figuring out things one of
> us has already gotten to work.

No, consolidating all solutions will never happen, it is too late. I
have a better idea.

There is something good that worked very well in the Java world: JCP -
Java Community Process. Vendors (People and companies) with common
interests sit down and agree on API specifications. When the
specification is finished, each vendor is free to implement their own
version of their specification.

The advantage of this is that application developers are not tied to a
specific implementation. They develop applications using the same APIs
that may run with different API implementations.

I have written more about this below in this blog post. Look for the
section about "PHP lacks of formal standard specifications".

http://www.phpclasses.org/blog/post/52-Recommended-PHP-frameworks.html

The article is about general purpose frameworks, but restricting
ourselves to database abstractions, it would be the almost the same
writing a specification of PHP DBC .

Once such specification was ready, each database abstraction developer
could write wrappers on even full implementations from scratch that
would implement the specified API.

There would be a single base of unit test scripts that could be used to
certify the compliance of all implementations.

Once you have this all working, then all database applications and
components could be more interoperable.

Let me give a simple but concrete example. Remember the SQL LIKE pattern
matching abstraction that I implemented in Metabase and then you adapted
to PEAR::MDB2?

With that I implemented an auto-complete plug-in for forms text inputs
that pulls completion texts from any database.

That works with Metabase API. To support PEAR::MDB2 you developed an
adapter to use PEAR::MDB2 API for the same purpose. There is also an
adapter for MySQL with the native API. But what about other database APIs?

If there was a common API specifications, all database abstractions that
provide an implementation for that API would have been supported.

The auto-complete plug-in is just an example that can be seen in action
here:
http://www.meta-language.net/forms-examples.html?example=test_auto_complete

But there are more, like for instance the plug-in to link multiple form
select inputs here:
http://www.meta-language.net/forms-examples.html?example=test_linked_select

And there will be more and more.

So my idea can be summarized as gathering all developers that implement
database abstraction packages, give up any personal differences, and sit
down to work on a common database API specification from which we all
can benefit.

--

-- 

Regards,
Manuel Lemos

Metastorage - Data object relational mapping layer generator
http://www.metastorage.net/

PHP Classes - Free ready to use OOP components written in PHP
http://www.phpclasses.org/

Manuel Lemos | 9 Mar 19:50
Picon
Favicon
Gravatar

[phpdba] Re: so who's there?

Hello,

on 03/08/2007 04:57 AM Lukas Kahwe Smith said the following:
>> Any particular reason why the list was not created on a more capable
>> list service, like Yahoo Groups, for instance?
> 
> Not really, well I guess I personally prefer this over the services of a
> large organization, where people have to create accounts.

Erm... in Yahoo groups you do not have to create accounts. You can just
subscribe and unsubscribe by e-mail. Accounts are only needed to access
Web resources associated to the group.

Maybe you were thinking of Google Groups. If there is a thing that
Google does not lead yet, it is in the mailing lists/discussion groups
field.

>>> However, over time I guess this list will hopefully will define itself
>>> as we discuss topics that come up. So with that I encourage all of you
>>> to post about challenging new features you are currently working on.
>>
>> I am not working in any new feature for Metabase right now. The last
>> feature that I developed, as you know was the pattern matching
>> abstraction. It is still not perfectly solved because different
>> databases may handle patterns with case sensitiveness or not. I have not
>> invested time in finding a solution either.
> 
> Yeah, I remember. One thing that we found out back then is that we would
> probably write a callback for SQLite IIRC:

I was not thinking of SQLite. I was thinking that MySQL LIKE operator is
case insensitive (which is what most people want) but in other databases
LIKE is case sensitive by default.

>> Maybe I got the wrong impression, but it felt there wasn't much will to
>> really cooperate and reach a consensus. This is a shame for the PHP
>> community because we loose the opportunity to have a single solution
>> that gathers greater adoption and more contributions.
> 
> Like I said above, in the particular case you mentioned, I think it was
> just that the topic kinda fell of the radar, I presume because of time
> constraints, rather than Igor "actively" ignoring your proposal.

I don't know, but he mailed several people including me to give
opinions. He proposed an approach. I proposed something different more
consistent with the previous definition of indexes and schemas, and he
insisted on his approach. Maybe I got it wrong, but it felt pretty much
that there was not interest to reach a consensus.

> So the purpose of the list is not to change today's reality of a
> multitude of packages, but to at least foster the exchange of ideas (and
> maybe even some code). Back when I was active on MDB/MDB2 I frequently
> looked at other layers, like on going development of Metabase (pattern
> matching etc) and PEAR::DB (error code abstraction), as well as ADODB
> ("smart" transaction), Creole (sequence/autoincrement tweaks). I am
> hoping that with this list, we will have a big bigger exchange of ideas.
> That when someone is trying to tackle a new tricky topic, that he can
> ask here for advice etc.

If you want an honest opinion, I am afraid that goal will hardly happen,
I mean most people will not share good ideas.

While there is a sense of competition, many packages for the same
purposes, people with good ideas already implemented will act
defensively and avoid sharing the details those ideas to not encourage
plagiarism. They will come to the list but remain on spy mode, i.e.
watching others bragging about good ideas of their packages.

Nothing wrong with this, it is just a consequence of ego driven
motivation that fuels the development of many open source projects.

>> For instance, what database API do you think Delphi for PHP supports?
> 
> Actually I was wondering the same thing, when I was looking over their
> press releases the other day. But again, I do not hope to consolidate
> the available solutions with this list. The goal is to pool know how to
> at least prevent each of us wasting time on figuring out things one of
> us has already gotten to work.

No, consolidating all solutions will never happen, it is too late. I
have a better idea.

There is something good that worked very well in the Java world: JCP -
Java Community Process. Vendors (People and companies) with common
interests sit down and agree on API specifications. When the
specification is finished, each vendor is free to implement their own
version of their specification.

The advantage of this is that application developers are not tied to a
specific implementation. They develop applications using the same APIs
that may run with different API implementations.

I have written more about this below in this blog post. Look for the
section about "PHP lacks of formal standard specifications".

http://www.phpclasses.org/blog/post/52-Recommended-PHP-frameworks.html

The article is about general purpose frameworks, but restricting
ourselves to database abstractions, it would be the almost the same
writing a specification of PHP DBC .

Once such specification was ready, each database abstraction developer
could write wrappers on even full implementations from scratch that
would implement the specified API.

There would be a single base of unit test scripts that could be used to
certify the compliance of all implementations.

Once you have this all working, then all database applications and
components could be more interoperable.

Let me give a simple but concrete example. Remember the SQL LIKE pattern
matching abstraction that I implemented in Metabase and then you adapted
to PEAR::MDB2?

With that I implemented an auto-complete plug-in for forms text inputs
that pulls completion texts from any database.

That works with Metabase API. To support PEAR::MDB2 you developed an
adapter to use PEAR::MDB2 API for the same purpose. There is also an
adapter for MySQL with the native API. But what about other database APIs?

If there was a common API specifications, all database abstractions that
provide an implementation for that API would have been supported.

The auto-complete plug-in is just an example that can be seen in action
here:
http://www.meta-language.net/forms-examples.html?example=test_auto_complete

But there are more, like for instance the plug-in to link multiple form
select inputs here:
http://www.meta-language.net/forms-examples.html?example=test_linked_select

And there will be more and more.

So my idea can be summarized as gathering all developers that implement
database abstraction packages, give up any personal differences, and sit
down to work on a common database API specification from which we all
can benefit.

--

-- 

Regards,
Manuel Lemos

Metastorage - Data object relational mapping layer generator
http://www.metastorage.net/

PHP Classes - Free ready to use OOP components written in PHP
http://www.phpclasses.org/

Lukas Kahwe Smith | 10 Mar 08:22

Re: [phpdba] Database Schema XML

Thomas Koch wrote:

> well, the nice thing with standards is, that there are so many to choose 
> from...
> That was exactly my problem: I searched ONE canonical format to save such a 
> basic thing like a database schema, so that it could be read and written by 
> many programs.

Well the Metabase format is supported by Metabase, MDB, MDB2_Schema,
ezc::DatabaseSchema as well as the DBDesigner ER Tool. I am not aware of
anything equally supported .. at least in the PHP scene.

You can also validate against a DTD or XSD .. plus there is a nice XSLT
to view the schemas:
http://cvs.php.net/viewvc.cgi/pear/MDB2_Schema/docs/

regards,
Lukas

Thomas Koch | 10 Mar 08:17
Picon

[phpdba] Hosting an overview page anyone?

Wouldn't anyone like to host a page to give an overview about the existing 
DB-Abstraction libraries, so that we all can add our informations?

Some infos:

Name
Ancestor projects
PHP4/5
Maintainers (number and names, maybe company)
Used by projects:
UnitTests Y/N
Documentation (good / great / exists / WTF )

Capabilities:
* ... (you know better)

regards, Thomas Koch

Thomas Koch | 10 Mar 08:00
Picon

[phpdba] Database Schema XML

Good morning,

I've recently searched the net to find a standard XML-Format to save Database 
Schemes. And guess how suprised I've been, that there actualy isn't any!

Any of our applications needs to store it's schema for the setup process. 
Couldn't we have at least for PHP a standard XML-Format as XML should be and 
maybe even a validation DTD (or whatever) for it?

The other thing related to this is, that there is nothing like an ERD-Editor. 
There are some one-man-show projects and some oldtimers, but nothing useable 
to quickly create, visualize (and modify!) some tables. 

(Some links from my search: http://www.linuxführer.de/index.php/Datenbanken)

Best regards,

Thomas Koch


Gmane