headers
Paul Burba | 1 Feb 02:44
Picon

Re: [RFC] Server Dictated Configuration

On Tue, Jan 31, 2012 at 12:48 PM, Julian Foad
<julianfoad <at> btopenworld.com> wrote:
> Paul Burba wrote:
>
>> Julian Foad wrote:
>>>  The ability to see the inherited value and then merge in a child-defined
>>> value (adding/subtracting/overriding semantic sub-elements within the
>>> property value) is essential if we're going to implement these features
>>> using properties with semantics like the existing 'svn:ignores'.
>>
>> Why do we need to subtract and override?  [...]
>>
>> 5) [...] we take a path's inherited (or explicit) svn:i:ignores
>> property value, the svn:ignore property (if any) on a path's
>> parent directory, and the global-ignores runtime config value and
>> append all three together to get the final answer on what to ignore.
>>
>> I take it you view this as insufficient?
>
> This hybrid approach to defining the ignore patterns by means of one inheritable property and a different
non-inheritable property gives us in total the ability to both override and append.
>
> Overriding is done by setting a new value for the inheritable property svn:i:ignore, like this:
>
>   /subversion               svn:i:ignore = *.o *.obj *.a *.lib ...
>   /subversion/trunk/tests   svn:i:ignore =     *.obj *.a *.lib ...
>
> ... which can be done hierarchically; but every such override at a subtree level duplicates much of the
information that was provided at the '/subversion' level, which means that whenever we modify the base
setting we probably want to look through the whole repository and modify all the subtree settings in the
(Continue reading)

Permalink | Reply | 7 comments |
headers
Greg Stein | 1 Feb 05:08
Picon

Ev2, RA, Commit Process (was: Editor v2 - suggestions and queries)

I've included an email from back in November for the basis of the
"next steps" for Ev2 tweaks. As I mentioned yesterday, Hyrum and I got
a chance to talk at length about Ev2 issues and design. Some extra
conversations with Philip and Julian also helped out here.

These next steps feel Good, but have some big implications around
client/server interaction. I'd like to hear any thoughts and concerns.

See below:

On Sat, Nov 5, 2011 at 19:56, Greg Stein <gstein <at> gmail.com> wrote:
> On Fri, Nov 4, 2011 at 11:16, Julian Foad <julian.foad <at> wandisco.com> wrote:
>...
>> Huh?  We're now talking about a single call that sets the target and
>> the properties together.  If we take this approach, I suggest naming
>> the three calls 'set_symlink' (like 'add_symlink'), 'set_file' (like
>> 'add_file'), and 'set_dir_props' (which is not quite like
>> 'add_directory').
>
> It was never intended to be set_dir_props(). You could set properties
> on any node. That should stick around, cuz it kind of sucks for the
> caller to have to know the node type just to set some properties.
>
> But I do like where you're going with this. Rather than set_$kind,
> let's go with alter_file(), alter_symlink(), and alter_directory(). I
> chose "alter" rather than "change" since change_directory might throw
> people off with some implied stateful semantics. Each of the alter_*
> functions would provide for changing the properties, symlink target,
> file contents, etc. Maybe we eliminate alter_directory() since that
> would be exactly the same as set_props() [though maybe that becomes
(Continue reading)

Permalink | Reply | 13 comments |
headers
Hyrum K Wright | 1 Feb 05:27
Favicon

Re: Ev2, RA, Commit Process (was: Editor v2 - suggestions and queries)

On Tue, Jan 31, 2012 at 10:08 PM, Greg Stein <gstein <at> gmail.com> wrote:
> I've included an email from back in November for the basis of the
> "next steps" for Ev2 tweaks. As I mentioned yesterday, Hyrum and I got
> a chance to talk at length about Ev2 issues and design. Some extra
> conversations with Philip and Julian also helped out here.
>
> These next steps feel Good, but have some big implications around
> client/server interaction. I'd like to hear any thoughts and concerns.
>
> See below:
>
> On Sat, Nov 5, 2011 at 19:56, Greg Stein <gstein <at> gmail.com> wrote:
>> On Fri, Nov 4, 2011 at 11:16, Julian Foad <julian.foad <at> wandisco.com> wrote:
>>...
>>> Huh?  We're now talking about a single call that sets the target and
>>> the properties together.  If we take this approach, I suggest naming
>>> the three calls 'set_symlink' (like 'add_symlink'), 'set_file' (like
>>> 'add_file'), and 'set_dir_props' (which is not quite like
>>> 'add_directory').
>>
>> It was never intended to be set_dir_props(). You could set properties
>> on any node. That should stick around, cuz it kind of sucks for the
>> caller to have to know the node type just to set some properties.
>>
>> But I do like where you're going with this. Rather than set_$kind,
>> let's go with alter_file(), alter_symlink(), and alter_directory(). I
>> chose "alter" rather than "change" since change_directory might throw
>> people off with some implied stateful semantics. Each of the alter_*
>> functions would provide for changing the properties, symlink target,
>> file contents, etc. Maybe we eliminate alter_directory() since that
(Continue reading)

Permalink | Reply | 3 comments |
headers
Greg Stein | 1 Feb 05:55
Picon

Re: Ev2, RA, Commit Process (was: Editor v2 - suggestions and queries)

On Tue, Jan 31, 2012 at 23:27, Hyrum K Wright <hyrum.wright <at> wandisco.com> wrote:
>...
> One of the purposes the "commit plan" could accomplish is establishing
> a priori what content the server has, and what content we need to
> send, identified via sha1.  Thus, if a commit is something like a
> merge, where most of the content is already on the server, the client
> doesn't have to transmit that information again.  (Though maybe this
> is already served by the lazy-load semantics of the Ev2 contents
> streams.)

Ah. Right. Forgot about that part.

Yes, the server can say "don't send C1, C2, C3 to me". Typically, that
list will be empty, but sure: sometimes we can save delivery of
content. Also note that the list of checksums [to not send] needs to
be preserved from Step 1 to Step 2 of the commit process. Some sort of
stateful commit process object needs to be carried between the two RA
steps.

> Another (complimentary?) approach might be to totally divorce content
> from structure.  Just remove contents streams from Ev2 completely.

I'm not sure that I like this approach. That goes back to the "save
state across two API invocations" that we had before. It also means
every Ev2 receiver will need to have some out-of-band way to receive
the content or manage an interaction with a "checksum to content"
mapping subsystem.

Sounds yucky, or I'm not understanding your suggestion.
(Continue reading)

Permalink | Reply | 2 comments |
headers
Greg Stein | 1 Feb 06:01
Picon

Re: svn commit: r1238931 - /subversion/trunk/subversion/include/svn_editor.h

On Tue, Jan 31, 2012 at 22:50,  <hwright <at> apache.org> wrote:
>...
> +++ subversion/trunk/subversion/include/svn_editor.h Wed Feb  1 03:50:40 2012
> @@ -885,6 +885,7 @@ svn_editor_set_props(svn_editor_t *edito
>  * with checksum @a checksum.
>  * ### TODO @todo Does this send the *complete* content, always?
>  * ### TODO @todo What is REVISION for?
> + * ### TODO @todo Who is responsible for closing the stream?

The receiver does. I'll clarify this in the docco after my API
changes. The receiver has three choices:

1) close the stream without reading
2) close the stream after reading *some* of the content
3) close the stream after reading *all* the content

Drivers should generally use a lazy-load technique in case of (1).
They may not require disk access, resource consumption, etc, if the
receiver choose to immediately close the stream.

Example: the checksum already exists in the pristine store, so the
receiver can immediately close the stream [rather than spooling the
contents into the store].

Cheers,
-g
Permalink | Reply | 1 comment |
headers
Hyrum K Wright | 1 Feb 06:03
Favicon

Re: Ev2, RA, Commit Process (was: Editor v2 - suggestions and queries)

On Tue, Jan 31, 2012 at 10:55 PM, Greg Stein <gstein <at> gmail.com> wrote:
> On Tue, Jan 31, 2012 at 23:27, Hyrum K Wright <hyrum.wright <at> wandisco.com> wrote:
>>...
>> One of the purposes the "commit plan" could accomplish is establishing
>> a priori what content the server has, and what content we need to
>> send, identified via sha1.  Thus, if a commit is something like a
>> merge, where most of the content is already on the server, the client
>> doesn't have to transmit that information again.  (Though maybe this
>> is already served by the lazy-load semantics of the Ev2 contents
>> streams.)
>
> Ah. Right. Forgot about that part.
>
> Yes, the server can say "don't send C1, C2, C3 to me". Typically, that
> list will be empty, but sure: sometimes we can save delivery of
> content. Also note that the list of checksums [to not send] needs to
> be preserved from Step 1 to Step 2 of the commit process. Some sort of
> stateful commit process object needs to be carried between the two RA
> steps.
>
>> Another (complimentary?) approach might be to totally divorce content
>> from structure.  Just remove contents streams from Ev2 completely.
>
> I'm not sure that I like this approach. That goes back to the "save
> state across two API invocations" that we had before. It also means
> every Ev2 receiver will need to have some out-of-band way to receive
> the content or manage an interaction with a "checksum to content"
> mapping subsystem.
>
> Sounds yucky, or I'm not understanding your suggestion.
(Continue reading)

Permalink | Reply | 1 comment |
headers
Hyrum K Wright | 1 Feb 06:05
Favicon

Re: svn commit: r1238931 - /subversion/trunk/subversion/include/svn_editor.h

On Tue, Jan 31, 2012 at 11:01 PM, Greg Stein <gstein <at> gmail.com> wrote:
> On Tue, Jan 31, 2012 at 22:50,  <hwright <at> apache.org> wrote:
>>...
>> +++ subversion/trunk/subversion/include/svn_editor.h Wed Feb  1 03:50:40 2012
>> @@ -885,6 +885,7 @@ svn_editor_set_props(svn_editor_t *edito
>>  * with checksum @a checksum.
>>  * ### TODO @todo Does this send the *complete* content, always?
>>  * ### TODO @todo What is REVISION for?
>> + * ### TODO @todo Who is responsible for closing the stream?
>
> The receiver does. I'll clarify this in the docco after my API
> changes.

Yeah, I figured that's the answer, I just wasn't sure (and rather than
guess, I figured I'd record my uncertainty to ensure it gets clarified
at some point).

> The receiver has three choices:
>
> 1) close the stream without reading
> 2) close the stream after reading *some* of the content
> 3) close the stream after reading *all* the content
>
> Drivers should generally use a lazy-load technique in case of (1).
> They may not require disk access, resource consumption, etc, if the
> receiver choose to immediately close the stream.
>
> Example: the checksum already exists in the pristine store, so the
> receiver can immediately close the stream [rather than spooling the
> contents into the store].
(Continue reading)

Permalink | Reply | 0 comments |
headers
Greg Stein | 1 Feb 06:07
Picon

Re: svn commit: r1238932 - /subversion/trunk/subversion/libsvn_delta/compat.c

On Tue, Jan 31, 2012 at 23:35,  <hwright <at> apache.org> wrote:
>...
> +++ subversion/trunk/subversion/libsvn_delta/compat.c Wed Feb  1 04:35:54 2012
> @@ -937,6 +937,7 @@ struct operation {
>   svn_kind_t kind;  /* to copy, mkdir, put or set revprops */
>   svn_revnum_t base_revision;       /* When committing, the base revision */
>   svn_revnum_t copyfrom_revision;      /* to copy, valid for add and replace */
> +  svn_checksum_t *new_checksum;   /* An MD5 hash of the new contents, if any */
>   const char *copyfrom_url;       /* to copy, valid for add and replace */
>   const char *src_file;  /* for put, the source file for contents */
>   apr_hash_t *children;  /* const char *path -> struct operation * */
> @@ -1024,6 +1025,7 @@ build(struct editor_baton *eb,
>       svn_revnum_t rev,
>       apr_hash_t *props,
>       const char *src_file,
> +      svn_checksum_t *checksum,

Why are these not const? It will save you casts later on, and is Better anyways.

>...
> @@ -1205,6 +1208,14 @@ add_file_cb(void *baton,
>   struct editor_baton *eb = baton;
>   const char *tmp_filename;
>   svn_stream_t *tmp_stream;
> +  svn_checksum_t *md5_checksum;
> +
> +  /* We may need to re-checksum these contents */
> +  if (!(checksum && checksum->kind == svn_checksum_md5))
> +    contents = svn_stream_checksummed2(contents, &md5_checksum, NULL,
> +                                       svn_checksum_md5, TRUE, scratch_pool);
(Continue reading)

Permalink | Reply | 0 comments |
headers
Greg Stein | 1 Feb 06:13
Picon

Re: Ev2, RA, Commit Process (was: Editor v2 - suggestions and queries)

On Wed, Feb 1, 2012 at 00:03, Hyrum K Wright <hyrum.wright <at> wandisco.com> wrote:
>...
> It does require some state to be maintained, but not as much as it
> appears at first blush.
>
> In the current delta editor, we have to keep all these file batons
> around, since we need to tie a particular apply_txdelta call to a
> given file.  If we deal in the currency of hashes, all that state goes
> away.  The add_file/alter_file callbacks have told us the content they
> expect, and we trust that at some future date that content will get to
> the destination.
>
> Because the content has essentially a UUID, we don't need to maintain
> per-file state while we wait for it to arrive, we just need a simple
> list of things we're expecting, and a way to ensure they get sent.
> "Here's the tree; here's the content; I'll let you sort out what
> matches to what."  Or even "Here's the tree"  "Thanks, I need the
> following content"  "Okay, here's the content you want."
>
> Does that make sense?

Yup. But it still seems a bit more complicated.

I guess this approach means that we could keep a delayed content
delivery and not mess around with the commit process, but I do like
using the Plan to obviate delivery of certain content streams. Was
that your thinking here? (minimal impact to commit) Or is there
something a bit more? (and does that benefit outweight the extra
checksum/content matching step)
(Continue reading)

Permalink | Reply | 0 comments |
headers
Daniel Shahaf | 1 Feb 08:28
Picon

Re: Ev2, RA, Commit Process (was: Editor v2 - suggestions and queries)

Greg Stein wrote on Tue, Jan 31, 2012 at 23:08:17 -0500:
> The "plan" is some new XML report-like document, posted to the "me"
> resource

We've defined HTTPv2 such that the body of POSTs is skels, not XML.
(dav_svn__method_post())

Also, you didn't talk about ra_svn at all.  Presumably it's trivial
there (we can just extend the protocol to mirror the new APIs), but
explicitness won't hurt.

> Thoughts?
> 
> Cheers,
> -g
Permalink | Reply | 8 comments |

Gmane