Julian Foad | 29 Oct 13:22 2014

Don't reject arguments of the form ". <at> PEG"

An 'svn' command-line argument of the form ". <at> PEG" raises the error

  svn: E125001: ' <at> PEG' is just a peg revision. Maybe try ' <at> PEG <at> ' instead?

That error is annoying to me, and unnecessary.

This happens for pretty much any 'svn' command, with any path that is converted to "" on canonicalization
and conversion to "internal style". It is inconsistent with how we interpret other arguments containing
a peg specifier, including arguments such as "foo/.. <at> PEG" which also refer to "this directory" at a
semantic level but which are not collapsed to " <at> PEG" at a syntactic level.

I think the reason that error was introduced (by stsp in r878062) was to prevent people being caught out when
we changed the interpretation of a filename that starts with " <at> " such as " <at> file". Before r878062 that had
been interpreted as a filename, and afterwards as a peg revision.

It seems to me that check should only reject command-line arguments of the form " <at> PEG", and not of the form
". <at> PEG". However, the check was in a low-level function () that is used both after conversion to internal
style as well as before conversion, so arguments of the form ". <at> PEG" were also being rejected.

I intend to correct this, to make arguments of the form ". <at> PEG" syntactically acceptable.

- Julian

Picon

[l10n] Translation status report for trunk r1634775

Translation status report for trunk <at> r1634775

  lang   trans untrans   fuzzy     obs
--------------------------------------
    de    2759     107     279     483  ++++++++++++++++++++++++++U~~~oooo
    es    2261     605     823     527  ++++++++++++++++++UUUUU~~~~~~~oooo
    fr    2565     301     511     111  ++++++++++++++++++++++UUU~~~~~
    it    2121     745     947     340  ++++++++++++++++UUUUUU~~~~~~~~oo
    ja    2251     615     872     763  ++++++++++++++++++UUUUU~~~~~~~oooooo
    ko    2397     469     644     217  ++++++++++++++++++++UUUU~~~~~~o
    nb    2311     555     774     500  +++++++++++++++++++UUUU~~~~~~~oooo
    pl    2336     530     741     297  +++++++++++++++++++UUUU~~~~~~~oo
 pt_BR    2097     769     960     321  ++++++++++++++++UUUUUU~~~~~~~~oo
    sv    2725     141     305      78  +++++++++++++++++++++++++UU~~~
 zh_CN    2620     246     432      19  +++++++++++++++++++++++UUU~~~~
 zh_TW    2038     828     997     377  +++++++++++++++UUUUUUU~~~~~~~~oo

Sean Leonard | 25 Oct 20:53 2014

svn:mime-type arbitrary parameters

Greetings:

What the best way for Subversion to record parameters of a MIME type 
(Internet media type) in a repository?

I require svn:mime-type to be filled out in all of the repositories that 
I manage. This is also useful for serving the right content over media 
type-aware protocols (HTTP, but there are others).

Some time ago there was a dev discussion about storing the character set 
of text in svn:charset (simple textual content) vs. svn:mime-type (using 
the header format of RFC 2045).

http://svn.haxx.se/dev/archive-2008-06/0941.shtml
http://svn.haxx.se/dev/archive-2008-07/0138.shtml

It appears that the matter was not fully resolved. svn:charset seems to 
enjoy de-facto use. Using RFC 2045 format in svn:mime-type as an 
appendage (delimited by ";") would be "correct", but as the poster 
notes, is extremely unwieldy. It is not possible to store arbitrary 
UTF-8 data, for example, unless you use the syntax of RFC 2231, which 
looks like:

    Content-Type: application/x-stuff;
     title*0*=us-ascii'en'This%20is%20even%20more%20
     title*1*=%2A%2A%2Afun%2A%2A%2A%20
     title*2="isn't it!"

Thank you,

(Continue reading)

Mark Phippard | 24 Oct 15:06 2014
Picon

Subversion canonical URI

When we use the Subversion API via JavaHL it is up to the caller to provide repository URI in proper encoded format, such as %20 for spaces.  I have never found the perfect Java method for doing this.    In Subclipse, we are currently using this method:


It does a good job handling UTF8 characters, but as an example it does not convert spaces to %20 so we have do that ourselves.  There are other characters like [, ], #, % it does not convert.  So it is kind of a constant battle to workaround the differences.

I have a couple questions:

1) Could JavaHL itself expose a Subversion library method that could do this for us?  I assume Subversion must have some method that it uses to canonicalize a URL.  Maybe this could just be exposed to JavaHL.  Pass it a String and get back the canonical version?

2) Is the Subversion URI format governed by a specific RFC?  Mainly asking that question for an alternative to #1.  If we are searching for a better Java method to use and they mention an RFC it would help to find a better fit.

I am not sure if I am using "canonical" correctly here.  I assume you get the gist of what I am asking though.

--
Thanks

Mark Phippard
http://markphip.blogspot.com/
Stefan Fuhrmann | 23 Oct 14:51 2014

Deferring FSFS revprop caching to 1.10

Hi all,

I ran a few tests to determine the impact of the missing
revprop caching on trunk. For checkouts, the overhead
is ~2 extra CPU cores per saturated 10Gb network. Even
that will be not be severe issue to users and the changes
mentioned below bring it down to .5 cores.

'svn log -v' is a factor of 2 slower for the http: client and
4x for the svn: client. Server load is approx 3x and 8x
as high, respectively, if revprop caching is not available.

Most of the overhead is spent parsing packed revprop
manifest info. I wrote a series of patches that reduces
the overhead such that the http: client will see no real
difference while the svn: client takes only 50% longer
than it would with a revprop caching server.

These are only ballpark numbers run on a system with
high bandwidths, low latencies and low fopen() overhead.
However, I think we can tune the revprop file granularity
and parsing now and defer the FSFS revprop caching to
1.10 without a massive impact on users.

-- Stefan^2.
Picon

[l10n] Translation status report for trunk r1633270

Translation status report for trunk <at> r1633270

  lang   trans untrans   fuzzy     obs
--------------------------------------
    de    2759     107     279     483  ++++++++++++++++++++++++++U~~~oooo
    es    2261     605     823     527  ++++++++++++++++++UUUUU~~~~~~~oooo
    fr    2565     301     511     111  ++++++++++++++++++++++UUU~~~~~
    it    2121     745     947     340  ++++++++++++++++UUUUUU~~~~~~~~oo
    ja    2251     615     872     763  ++++++++++++++++++UUUUU~~~~~~~oooooo
    ko    2397     469     644     217  ++++++++++++++++++++UUUU~~~~~~o
    nb    2311     555     774     500  +++++++++++++++++++UUUU~~~~~~~oooo
    pl    2336     530     741     297  +++++++++++++++++++UUUU~~~~~~~oo
 pt_BR    2097     769     960     321  ++++++++++++++++UUUUUU~~~~~~~~oo
    sv    2725     141     305      78  +++++++++++++++++++++++++UU~~~
 zh_CN    2620     246     432      19  +++++++++++++++++++++++UUU~~~~
 zh_TW    2038     828     997     377  +++++++++++++++UUUUUUU~~~~~~~~oo

Stefan Fuhrmann | 20 Oct 15:16 2014

FS API functions missing FS API tests

Hi there,

I've been going through svn_fs.h and found that the following
functions aren't being called directly in any of our tests. They
may still be tested indirectly through higher level functions but
I think most of them should get a low-level test as well:

svn_fs_access_get_username
svn_fs_access_add_lock_token2
svn_fs_purge_txn
svn_fs_txn_base_revision
svn_fs_is_txn_root
svn_fs_is_revision_root
svn_fs_txn_root_name
svn_fs_path_change2_create
svn_fs_check_path
svn_fs_is_file
svn_fs_node_relation
svn_fs_node_created_path
svn_fs_props_different
svn_fs_props_changed
svn_fs_get_mergeinfo2
svn_fs_merge
svn_fs_dir_optimal_order
svn_fs_file_length
svn_fs_file_checksum(sha1)
svn_fs_try_process_file_contents
svn_fs_contents_different
svn_fs_contents_changed
svn_fs_info_config_files
svn_fs_get_file_delta_stream
svn_fs_lock_target_set_token
svn_fs_get_locks2
svn_fs_print_modules

[svn_fs_node_history2, svn_fs_history_prev2 are opaque and will
 be tested through 'svn log']

I'll be working through that list by the end of this week and add
tests where they are simple enough and then post an updated list
here.

-- Stefan^2.
Philip Martin | 15 Oct 12:15 2014

Some x509 branch review points

1)

In x509parse.c:x509_get_version:

  err = asn1_get_tag(p, end, &len,
                     ASN1_CONTEXT_SPECIFIC | ASN1_CONSTRUCTED | 0);

Why the "| 0"?  It doesn't do anything.

2)

In x509parse.c:x509_get_name:

  cur->next = apr_palloc(result_pool, sizeof(x509_name));

  if (cur->next == NULL)
    return SVN_NO_ERROR;

We generally assume apr_palloc will abort rather than return NULL,
that's certainly the assumption in other places in this file.  If we
were to detect an allocation failure then returning no error is not
correct.

3)

In x509parse.c:x509_get_name:

  oid = &cur->oid;
  oid->tag = **p;

  err = asn1_get_tag(p, end, &oid->len, ASN1_OID);
  if (err)
    return svn_error_create(SVN_ERR_X509_CERT_INVALID_NAME, err, NULL);

The asn1_get_tag() call verifies both that *p has not reached end and
that the tag is ASN1_OID.  This happens after assigning **p to oid->tag,
so if asn1_get_tag were to detect that *p had reached end it would
happen after we had dereferenced **p.  This bug occurs in other places:
x509_get_sig, x509_get_alg, etc.

Fix by assigning ASN1_OID explicitly or by moving the assignment after
asn1_get_tag.

--

-- 
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*

Picon

[l10n] Translation status report for trunk r1631608

Translation status report for trunk <at> r1631608

  lang   trans untrans   fuzzy     obs
--------------------------------------
    de    2765     108     279     477  ++++++++++++++++++++++++++U~~~oooo
    es    2263     610     825     526  ++++++++++++++++++UUUUU~~~~~~~oooo
    fr    2571     302     513     108  ++++++++++++++++++++++UUU~~~~~
    it    2123     750     949     340  ++++++++++++++++UUUUUU~~~~~~~~oo
    ja    2253     620     874     763  ++++++++++++++++++UUUUU~~~~~~~oooooo
    ko    2399     474     646     217  ++++++++++++++++++++UUUU~~~~~~o
    nb    2313     560     776     500  +++++++++++++++++++UUUU~~~~~~~oooo
    pl    2338     535     743     297  +++++++++++++++++++UUUU~~~~~~~oo
 pt_BR    2099     774     962     321  ++++++++++++++++UUUUUU~~~~~~~~oo
    sv    2731     142     305      72  +++++++++++++++++++++++++UU~~~
 zh_CN    2626     247     434      13  +++++++++++++++++++++++UUU~~~~
 zh_TW    2040     833     999     377  +++++++++++++++UUUUUUU~~~~~~~~oo

Lieven Govaerts | 10 Oct 17:41 2014
Picon

Re: ra_serf bug(s)

On Tue, Oct 7, 2014 at 2:57 PM, Conrad Rad <cse.cem <at> gmail.com> wrote:
> Hi all,
>
> I'm working on a thing that links libsvn_ra_serf (default RA in Fedora
> Linux) as well as other pieces of the SVN libraries.
>
> I am using the 'replay' feature to dump edits from a remote repository.
>
> Inadvertently, I requested r0 as my starting_revision. This is my
> mistake, however: nothing in the generic replay or ra_serf-specific
> replay code produced a warning or error. This could be improved if r0
> is always invalid (first bug). (I am less confident about this one
> than the next one.)
>
> ra_serf makes a series of valid requests (200 OK) before finally
> asking the server for REPLAY of r0. In this case the server is
> googlecode.com. In response to the invalid REPLAY it gave a 500 Server
> Error response. I will attach a pcap.

Googlecode returns the 500 Internal Error, svn.apache.org will happily
return the content of r0.
IMO googlecode is doing the wrong thing here, there's nothing special
about r0 (it's empty, but still a valid revision).

> The second bug is: ra_serf ignores the 500 response and then waits
> forever for a valid response that isn't coming. So at this point the
> program is just stuck in svn_ra_replay_range ->
> svn_ra_serf__replay_range -> svn_ra_serf__context_run_wait -> ... ->
> epoll(2) forever, when it should have aborted and returned an error on
> the 500.
>
> I'm guessing that a fix might involve toggling
> no_fail_on_http_failure_status or setting up a response_error()
> handler on the svn_ra_serf__handler_t, but I'm not an expert on SVN
> internals.
>
> I'm using subversion-1.8.10, libserf-1.3.7, and apr-1.5.1 from Fedora 20.

It looks like this issue has been solved on trunk. When I use a
modified svnsync with start_revision set to 0, and sync from a
googlecode repo, I get following error from the 500 error response:

subversion/svnsync/svnsync.c:1558,
subversion/svnsync/svnsync.c:1504,
subversion/libsvn_ra/ra_loader.c:1358,
subversion/libsvn_ra_serf/replay.c:788,
subversion/libsvn_ra_serf/util.c:930,
subversion/libsvn_ra_serf/util.c:895,
subversion/libsvn_ra_serf/multistatus.c:560:
(apr_err=SVN_ERR_FS_NO_SUCH_REVISION)
svnsync: E160006: No such revision -1

Maybe you can try your program with svn trunk libraries to see if it
works for you too?

Lieven

> Thanks,
> Conrad

Conrad Rad | 10 Oct 16:22 2014
Picon

Re: ra_serf bug(s)

On Tue, Oct 7, 2014 at 8:57 AM, Conrad Rad <cse.cem <at> gmail.com> wrote:
> ...
>
> The second bug is: ra_serf ignores the 500 response and then waits
> forever for a valid response that isn't coming. So at this point the
> program is just stuck in svn_ra_replay_range ->
> svn_ra_serf__replay_range -> svn_ra_serf__context_run_wait -> ... ->
> epoll(2) forever, when it should have aborted and returned an error on
> the 500.
>
> I'm guessing that a fix might involve toggling
> no_fail_on_http_failure_status or setting up a response_error()
> handler on the svn_ra_serf__handler_t, but I'm not an expert on SVN
> internals.
>
> I'm using subversion-1.8.10, libserf-1.3.7, and apr-1.5.1 from Fedora 20.

Due to overwhelming consensus / lack of objection (*birds chirping*),
I've filed an issue for this in the tracker:

http://subversion.tigris.org/issues/show_bug.cgi?id=4523

Thanks,
Conrad


Gmane