Re: 4.0b5 – Monday, June 28

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first

_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers

Plone Tests: 5 OK

Summary of messages to the testbot list.
Period Wed Jun 30 12:00:00 2010 UTC to Thu Jul  1 12:00:00 2010 UTC.
There were 5 messages: 1 from ATContentTypes Tests, 1 from Archetypes Tests, 1 from Plone Libraries Tests,
1 from Plone Products Tests, 1 from Plone Tests.

Tests passed OK
---------------

Subject: OK : Plone-3.3 Zope-2.10 Python-2.4.6
From: Plone Products Tests
Date: Thu Jul  1 04:06:12 CEST 2010
URL: http://lists.plone.org/pipermail/testbot/2010-July/016341.html

Subject: OK : Plone-3.3 Zope-2.10 Python-2.4.6
From: Plone Libraries Tests
Date: Thu Jul  1 04:11:12 CEST 2010
URL: http://lists.plone.org/pipermail/testbot/2010-July/016342.html

Subject: OK : AT-1.5 Plone-3.3 Zope-2.10 Python-2.4.6
From: Archetypes Tests
Date: Thu Jul  1 04:17:28 CEST 2010
URL: http://lists.plone.org/pipermail/testbot/2010-July/016343.html

Subject: OK : ATCT-1.3 Plone-3.3 Zope-2.10 Python-2.4.6
From: ATContentTypes Tests
Date: Thu Jul  1 04:23:55 CEST 2010
URL: http://lists.plone.org/pipermail/testbot/2010-July/016344.html

Subject: OK (97 packages) : Plone-4.0 Zope-2.12 Python-2.6.5
From: Plone Tests
Date: Thu Jul  1 04:28:56 CEST 2010
URL: http://lists.plone.org/pipermail/testbot/2010-July/016345.html

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first

Versioning in Zope and Plone revisited

Andreas' blog doesn't have comments, so I thought I'd follow up here
to his post:

http://www.zopyx.de/blog/versioning-in-zope-and-plone-revisited-part-i

Overall I think it's an interesting approach (and a problem we really
should solve) but I think it's a little too fixed on serializing
objects to json, I have a couple of problems with it though:

 * For a ZODB backed storage this seems an unnecessary overhead.

 * For a relational database backed storage, you may want to use a
scheme that inserts a new row for each version, leading to a schema
something like:

#content_uid (PK), current_version (FK)
(1, 3)
(2, 5)

#content_uid (FK), version (PK), data...
(1, 1, "object 1", "title")
(1, 2, "object 1", "changed title)
(1, 3, "object 1", "another title)
(2, 4, "object 2", "title")
(2, 5, "object 2", "changed title")

This is what I used for simple versioning of data with collective.tin.

I think supporting these other options means focussing first on what
the history and edit views need:

class IHistory(zope.interface.common.mapping.IIterableMapping):
    """"version_id -> object implementing IReadVersionData"""

class IReadVersionData(Interface):
    object = Attribute(
        """The objects state at save time.

        To avoid temporal problems (by changing the history) this
        object has to be cloned before any change
(IRestorableVersionData.restore)
        """)

    metadata = Attribute(
        """Metadata stored alongside when the objects state was saved.

        Metadata has to be cloned before any write change to avoid
        temporal problems (by changing the history).
        """)

class IPurgableHistory(IHistory):
    def __delitem__(key):
        """delete a revision"""

    def clear():
        """delete all revisions"""

class IRestorableVersionData(IReadVersionData):
    def restore(metadata=None):
        """Restore the version, supplying new versioning metadata
        Returns an editable copy of the object.
        """

class IVersionedItem(Interface):
   current_version_id = Attribute("current version id")

    def getCurrentVersionData():
        """Returns the version data representing the current version.

        Must call restore() to get an editable version
        """

This allows us to simply support staging, as it is just a matter of
creating future history.

class IStagingItem(IVersionedItem):
    working_version_id = Attribute("The latest version, may not yet be
published")

The IHistory adapter then allows the storage mechanism to be
implemented pretty much how you like.

By enforcing a call to restore() before applying changes from an edit
form, storage implementation becomes a lot simpler - no need to create
unnecessary copies of old versions.

Laurence

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first

Re: Versioning in Zope and Plone revisited

Laurence Rowe wrote:
> Andreas' blog doesn't have comments, so I thought I'd follow up here
> to his post:
> 
> http://www.zopyx.de/blog/versioning-in-zope-and-plone-revisited-part-i
> 
> Overall I think it's an interesting approach (and a problem we really
> should solve) but I think it's a little too fixed on serializing
> objects to json, I have a couple of problems with it though:
> 
>  * For a ZODB backed storage this seems an unnecessary overhead.

Partly agreed - except that I don't want to deal with specific datatypes
like DateTime on a layer other than the application layer
(mean inside the application (Plone) and on the adapter level).

> 
>  * For a relational database backed storage, you may want to use a
> scheme that inserts a new row for each version, leading to a schema
> something like:
> 
> #content_uid (PK), current_version (FK)
> (1, 3)
> (2, 5)
> 
> #content_uid (FK), version (PK), data...
> (1, 1, "object 1", "title")
> (1, 2, "object 1", "changed title)
> (1, 3, "object 1", "another title)
> (2, 4, "object 2", "title")
> (2, 5, "object 2", "changed title")
> 

That's roughly the same approach I took for the MongoDB
storage.

> 
> 
> I think supporting these other options means focussing first on what
> the history and edit views need:
> 

Agreed that the UI views may need some special API for doing
efficient introspection on the version storage. I think that this should
not be a big deal - as long as the underlying storage
provides efficient search capabilities (ootb available with a
RDBMS or MongoDB - a ZODB-based version storage like needs
something on top of zope.index and zope.catalog).

Andreas

-- 
ZOPYX Limited           | zopyx group
Charlottenstr. 37/1     | The full-service network for Zope & Plone
D-72070 Tübingen        | Produce & Publish
www.zopyx.com           | www.produce-and-publish.com
------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first

_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers

Re: Versioning in Zope and Plone revisited

Hi,

Glad to see this discussion happen. I think a simpler, leaner
versioning solution should be on our wish list. CMFEditions works, but
I don't think many people understand it or know how to customise or
maintain it.

On 1 July 2010 22:46, Laurence Rowe <l <at> lrowe.co.uk> wrote:
> Andreas' blog doesn't have comments, so I thought I'd follow up here
> to his post:

I emailed Andreas in private, but I'll rehash some thoughts here too.

In short:

1) Perhaps we should consider using a specialised version control
system to control version data? One option may be to use svn (for
which there are Python bindings and ore.svn apparently provides a nice
API among other packages). The main file data could be checked in and
versioned as a file. Metadata could perhaps be stored in svn
properties.

Some possible advantages:

 - You can use other tools to inspect and manage the version storage.

 - Subversion should be quite efficient at storing version data.

 - We could support branching and merging of content, although I don't
think that should be in the base product - it's more of a specialised
use case that requires specialised UI.

2) If we want to do this, we can use zope.filerepresentation's
interfaces as an abstraction for "reading" a content item as a file
(on versioning) and "writing" such a file back to an object (on
restore). Dexterity supports this already, and happens to use
plone.rfc822 to format the fiel data, but it's really quite generic,
and could store almost anything (including JSON or XML).

Possible advantages:

 - We would have the same "file representation" for WebDAV,
externalised versioning and possibly other use cases like web
services.

 - There's already pretty good, reciprocal read/write stuff for
Dexterity, and the existing AT marshallers could be used via a simple
shim to make it work for AT content items.

 - zope.filerepresentation is a decent API.

Anyway, this is not to detract from what Andreas has done or what
Laurence is suggesting - just some ideas I've been thinking about for
a few days that are maybe worth discussing.

Cheers,
Martin

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first

Re: Versioning in Zope and Plone revisited

I think we are mixing up things. First we have do deal
serialization issues on two levels:

a) between app and "archiver"
b) between "archiver" and storage backend

For a) we need a somewhat common and neutral format. That's why I
decided on JSON..as mentioned earlier I do not want to see Zope specific
data-types on the archiver level.

For b) you can of course choose a different serialization format
depending on the backend. RFC822 clearly makes sense for using some kind
of version control system as backend - however it would not make sense
to convert JSON to RFC822 and stuff it into MongoDB.

Am I wrong with those assumptions?

Andreas

Martin Aspeli wrote:
> Hi,
> 
> Glad to see this discussion happen. I think a simpler, leaner
> versioning solution should be on our wish list. CMFEditions works, but
> I don't think many people understand it or know how to customise or
> maintain it.
> 
> On 1 July 2010 22:46, Laurence Rowe <l <at> lrowe.co.uk> wrote:
>> Andreas' blog doesn't have comments, so I thought I'd follow up here
>> to his post:
> 
> I emailed Andreas in private, but I'll rehash some thoughts here too.
> 
> In short:
> 
> 1) Perhaps we should consider using a specialised version control
> system to control version data? One option may be to use svn (for
> which there are Python bindings and ore.svn apparently provides a nice
> API among other packages). The main file data could be checked in and
> versioned as a file. Metadata could perhaps be stored in svn
> properties.
> 
> Some possible advantages:
> 
>  - You can use other tools to inspect and manage the version storage.
> 
>  - Subversion should be quite efficient at storing version data.
> 
>  - We could support branching and merging of content, although I don't
> think that should be in the base product - it's more of a specialised
> use case that requires specialised UI.
> 
> 2) If we want to do this, we can use zope.filerepresentation's
> interfaces as an abstraction for "reading" a content item as a file
> (on versioning) and "writing" such a file back to an object (on
> restore). Dexterity supports this already, and happens to use
> plone.rfc822 to format the fiel data, but it's really quite generic,
> and could store almost anything (including JSON or XML).
> 
> Possible advantages:
> 
>  - We would have the same "file representation" for WebDAV,
> externalised versioning and possibly other use cases like web
> services.
> 
>  - There's already pretty good, reciprocal read/write stuff for
> Dexterity, and the existing AT marshallers could be used via a simple
> shim to make it work for AT content items.
> 
>  - zope.filerepresentation is a decent API.
> 
> Anyway, this is not to detract from what Andreas has done or what
> Laurence is suggesting - just some ideas I've been thinking about for
> a few days that are maybe worth discussing.
> 
> Cheers,
> Martin

-- 
ZOPYX Limited           | zopyx group
Charlottenstr. 37/1     | The full-service network for Zope & Plone
D-72070 Tübingen        | Produce & Publish
www.zopyx.com           | www.produce-and-publish.com
------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first

_______________________________________________
Plone-developers mailing list
Plone-developers@...
https://lists.sourceforge.net/lists/listinfo/plone-developers

Plone Sprint in Germany in September

Hi all,

there is a new chance to have a Plone sprint in Germany after the annual
german Zope-Conference (DZUG-Tagung
http://www.zope.de/tagung/Dresden_2010) on the 18th and 19th of
September in Dresden.

If you intend to visit the conference, you can simply add two days to
sprint and work with other on your favorite cms.

If you are interested to sprint for more than two days we perhaps can
get a room for you during the conference.

So if you are interested, please wave your hands.

I am not sure whether I'll sprint myself, but I am willing to help with
the coordination.

Cheers!
juh

--

-- 
Jan Ulrich Hasecke
DZUG e.V. (Deutschsprachige Zope User Group)

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first

Re: Versioning in Zope and Plone revisited

On 1 July 2010 16:50, Andreas Jung <lists@...> wrote:
> I think we are mixing up things. First we have do deal
> serialization issues on two levels:
>
> a) between app and "archiver"
> b) between "archiver" and storage backend
>
> For a) we need a somewhat common and neutral format. That's why I
> decided on JSON..as mentioned earlier I do not want to see Zope specific
> data-types on the archiver level.
>
> For b) you can of course choose a different serialization format
> depending on the backend. RFC822 clearly makes sense for using some kind
> of version control system as backend - however it would not make sense
> to convert JSON to RFC822 and stuff it into MongoDB.
>
> Am I wrong with those assumptions?

There are (at least) two ways of approaching versioning. Martin and
you are describing an observer based pattern, where the versioning
system essentially hooks into changes in the content object and stores
a copy in a repository. This is how CMFEditions works, or content
mirror (though that does not do versioning).

The other approach is to use an inherently versioning content type.
This is how collective.tin works - the content object is an ORM mapped
object from the database, when versioning is enabled it just inserts
new data rows for each version and updates the content-id ->
current-version-id table. It should also be possible to build an in
ZODB repository based content type, where data is stored directly in
the repository rather than being copied there (as CMFEditions does,
causing database bloat). Or even a self contained versioning content
type.

This second approach does not require any serialisation. I want to
make sure that we support this approach with the same UI, I think that
means we need to ensure the the 'view' logic is separate from the
'controller' logic. In some ways it is a similar intention to that in
the "Unified interface for folder listing" PLIP at
http://dev.plone.org/plone/ticket/9327

Laurence

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first

out of the office

I will be out of the office starting  07/01/2010 and will not return until
07/06/2010.

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first

Re: Versioning in Zope and Plone revisited

On 02/07/2010, at 5:35 AM, Laurence Rowe wrote:

> On 1 July 2010 16:50, Andreas Jung <lists@...> wrote:
>> I think we are mixing up things. First we have do deal
>> serialization issues on two levels:
>>
>> a) between app and "archiver"
>> b) between "archiver" and storage backend
>>
>> For a) we need a somewhat common and neutral format. That's why I
>> decided on JSON..as mentioned earlier I do not want to see Zope  
>> specific
>> data-types on the archiver level.
>>
>> For b) you can of course choose a different serialization format
>> depending on the backend. RFC822 clearly makes sense for using some  
>> kind
>> of version control system as backend - however it would not make  
>> sense
>> to convert JSON to RFC822 and stuff it into MongoDB.
>>
>> Am I wrong with those assumptions?
>
> There are (at least) two ways of approaching versioning. Martin and
> you are describing an observer based pattern, where the versioning
> system essentially hooks into changes in the content object and stores
> a copy in a repository. This is how CMFEditions works, or content
> mirror (though that does not do versioning).
>
> The other approach is to use an inherently versioning content type.
> This is how collective.tin works - the content object is an ORM mapped
> object from the database, when versioning is enabled it just inserts
> new data rows for each version and updates the content-id ->
> current-version-id table. It should also be possible to build an in
> ZODB repository based content type, where data is stored directly in
> the repository rather than being copied there (as CMFEditions does,
> causing database bloat). Or even a self contained versioning content
> type.
>
> This second approach does not require any serialisation. I want to
> make sure that we support this approach with the same UI, I think that
> means we need to ensure the the 'view' logic is separate from the
> 'controller' logic. In some ways it is a similar intention to that in
> the "Unified interface for folder listing" PLIP at
> http://dev.plone.org/plone/ticket/9327

Obviously technical considerations trump everything else but what  
would be very nice is that if we did make a new archiving solution it  
could have the features to comply to legislated records keep acts of  
governments around the world.
Plone's used for a lot of government websites and I imagine that most  
governments have similar laws to that here in Australia which says  
roughly speaking that there must be a way to show exactly what a  
website said on a given date since it is on the public record.  
Governments have to fall back on some pretty expensive clunky systems  
(I better not mention names) to provide that compliance. Some  
solutions that compete with plone offer bridges to those systems.  
Inbuilt Plone compliance would be potentially a big win.
Perhaps if you're involved in government work and know about your  
local legislation email me and we could work out a set of features  
that would enable Plone archiving to be compliant world wide and  
hopefully that would be compatible with the technical approach?

>
> Laurence
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Plone-developers mailing list
> Plone-developers@...
> https://lists.sourceforge.net/lists/listinfo/plone-developers

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first