Tony Stevenson | 29 Jul 20:27 2014

Issue with incomplete rev

Hi folks,   

I am hoping someone will be willing to help us (ASF Infra) bottom out this issue and suggest the most sane way
to resolve it.   

We are using the ‘SVNMasterURI’ directive to enable the EU ‘mirror’. I am not aware of any special
circumstances that led to the situation we have.  If you look at [1] and [2] you will see the US master has
the revision, intact (as best we can tell) including log message and payload (not sure of the correct term
here, but essentially I mean the changed paths).  Where as the EU mirror only has a revision with the log
message no payload.   

Is there anyway to replay the single revision? Or are we going to have to dump the US master, and load it on the
EU mirror and svnsync it ?   

EU Mirror:  svn, version 1.8.1 (r1503906)  
US Master:  svn, version 1.8.1 (r1503906)  

[1] - http://svn.us.apache.org/viewvc?view=revision&revision=r1362927  
[2] - http://svn.eu.apache.org/viewvc?view=revision&revision=r1362927  

--  

Cheers,  
Tony  

On behalf of the Apache Infrastructure Team

-------------------------  
Tony Stevenson  

(Continue reading)

Julian Foad | 29 Jul 12:27 2014

The new svn_client_mtcc API

I just looked at svn_client_mtcc.h, scheduled for 1.9 release

I noticed it's basically an svn_delta_editor commit editor at the libsvn_client level, without the
'open' and 'close' methods.

The header file doesn't say anything about the semantics and rules for the provided functions. I'm
guessing it's basically the same as svn_delta_editor. For example:

  * You are making incremental edits to a 'current state'. The order of operations therefore matters.

  * Modify, delete, move(source) requires an existing node at the given path in the current state.
Add_file, mkdir, copy, move(dest) requires there is no node at the given path in the current state.(To
replace a node you must first delete it, then add at the same path.)

  * A sequence of changes need not be minimal. For example, after adding something you may delete it, and
then that path is again free to have something added there.

  * 'Delete' is recursive.

  * After 'add_file' you MAY but SHOULD NOT (?) then use 'update_file' on it.

  * ... and so on.

These rules are intuitive to someone familiar with svn_delta_editor_t and expecting it to work like that,
but otherwise are not intuitive. We need to document them.

It's not clear whether the copy source relpath is relative to the edit anchor URL of the edit or to the
repository root. Does this mean you need authorization to open a commit session to the root directory of
all paths, including copy source paths? If you want to make a tag of /trunk, do you need write access to the
repo root?
(Continue reading)

Picon

[l10n] Translation status report for trunk r1614245

Translation status report for trunk <at> r1614245

  lang   trans untrans   fuzzy     obs
--------------------------------------
    de    2725      59     230     471  +++++++++++++++++++++++++++~~~oooo
    es    2232     552     793     528  ++++++++++++++++++UUUUU~~~~~~~oooo
    fr    2536     248     474     109  +++++++++++++++++++++++UU~~~~~o
    it    2098     686     924     340  ++++++++++++++++UUUUUU~~~~~~~~oo
    ja    2225     559     846     763  ++++++++++++++++++UUUUU~~~~~~~oooooo
    ko    2367     417     613     219  ++++++++++++++++++++UUUU~~~~~~o
    nb    2282     502     744     501  +++++++++++++++++++UUUU~~~~~~~oooo
    pl    2307     477     711     298  +++++++++++++++++++UUUU~~~~~~~oo
 pt_BR    2074     710     938     321  ++++++++++++++++UUUUUU~~~~~~~~oo
    sv    2696      88     261      67  ++++++++++++++++++++++++++U~~~
 zh_CN    2585     199     386       7  ++++++++++++++++++++++++UU~~~~
 zh_TW    2015     769     976     377  ++++++++++++++++UUUUUU~~~~~~~~ooo

Lior | 22 Jul 18:58 2014
Picon

Subversion FAQ broken link

Julian Foad | 22 Jul 13:40 2014

Mergeinfo is not per node

For those interested in merging etc., a note on a recent line of thought.

For some time now I've had this idea going round my head that mergeinfo theoretically belongs to each node
separately, and that we "elide" subtree mergeinfo only for convenience, compactness, and to make it less
obtrusive and more easily understandable to the user.

It seemed a nice idea, but it's wrong. Mergeinfo is not inherently "per node".

WHY?

The content of two branches is usually *different* -- that's the point of branches.

In the per-file model of branching used by CVS, for example, each file is branched, and the content of each
branch of that file can differ. This means for each file in the source tree there is one obviously
corresponding file in the target tree.

In Subversion the intention is to version trees rather than just separate files, and so two branches can
differ in tree structure as well as in file content. The changes to one file on branch B1 can correspond to
changes in two files on branch B2, or in no particular file on branch B2, and so on. A merge cannot assume
there is a 1-to-1 mapping of nodes.

Imagine the change on branch B1 at revision 100 consists of renaming a function, and updating all calls to
it. The change affects files foo.c and foo.h and bar.c. When we merge this change to the target branch B2, we
have to adjust the result, manually and/or automatically,  to fit the target branch. Perhaps foo and bar
have been combined into a single file foobar.c on branch B2, and so the change affects only foobar.c. This
does not mean foobar.c alone has received that change, as that would imply all other nodes are still
eligible to receive that change. Rather, the information we need to track is that the target branch as a
whole has received the change as a whole.

- The merge source changes may be a selection of changes from just one subtree (or more generally a subset of
the nodes) in the source branch;

- but the target is not inherently "the corresponding subtree", it's the whole tree;

- and other target nodes/subtrees are *not* still eligible to receive this change.

NESTED BRANCHING

With nested branching, on the other hand, mergeinfo *does* belong to a subtree of the outer branch. The
intent is to track that a change was merged into a subtree B2/D1, but there may be another subtree B2/D2
where the same change is still eligible to be merged.

- The merge source is a selected subtree;

- the target is a "corresponding" subtree;

- other target subtrees are still eligible to receive this change.

THEREFORE

Mergeinfo belongs to the target branch as a whole, in the (common) case of a selective merge of a part of the
changes in the branch.

Mergeinfo belongs to the target subtree (as a whole) when the intent is nested branching.

SO WHAT?

In designing a revised repository model, we should not think of mergeinfo as an attribute that appears in
the model on every node and needs to be elided/normalized for storage efficiency.

On the client side, we should in future keep mergeinfo only on the branch root in most cases, more so than we do
today. We need to *distinguish* the two cases: whether the user intends to merge only a subset of the
changes in the whole branch, or to make a nested branch. To do so, we may consider heuristics (for example,
assume a subset merge is intended if there is no mergeinfo on the specified target but there is on a parent)
as well as explicit UI.

What I have been calling "mergeinfo" here is only part of the information we need for merging. We also need a
way to map nodes in the source branch to nodes in the target branch, in order to apply most of the individual
per-node changes in the source branch to the "right places" on the target branch before falling back to
conflicts and user input where this automatic attempt fails. I am starting to see this mapping as an almost
completely separate problem with its own metadata rather than something that the mergeinfo should give
us for free.

- Julian

Picon

[l10n] Translation status report for trunk r1612454

Translation status report for trunk <at> r1612454

  lang   trans untrans   fuzzy     obs
--------------------------------------
    de    2723      59     228     471  +++++++++++++++++++++++++++~~~oooo
    es    2230     552     791     528  ++++++++++++++++++UUUUU~~~~~~~oooo
    fr    2534     248     472     109  +++++++++++++++++++++++UU~~~~~o
    it    2096     686     922     340  ++++++++++++++++UUUUUU~~~~~~~~oo
    ja    2223     559     844     763  ++++++++++++++++++UUUUU~~~~~~~oooooo
    ko    2365     417     611     219  ++++++++++++++++++++UUUU~~~~~~o
    nb    2280     502     742     501  +++++++++++++++++++UUUU~~~~~~~oooo
    pl    2305     477     709     298  +++++++++++++++++++UUUU~~~~~~~oo
 pt_BR    2072     710     936     321  ++++++++++++++++UUUUUU~~~~~~~~oo
    sv    2694      88     259      67  ++++++++++++++++++++++++++U~~~
 zh_CN    2583     199     384       7  ++++++++++++++++++++++++UU~~~~
 zh_TW    2013     769     974     377  ++++++++++++++++UUUUUU~~~~~~~~ooo

Jacky wong | 21 Jul 14:44 2014
Picon

(unknown)

Julian Foad | 17 Jul 17:38 2014

Three little diff bugs

I just noticed some 'svn diff' bugs, using svn trunk <at> 1611327 (with a few local mods related to mergeinfo
parsing, that should not affect diffs).

Bug 1.

We print a diff header in front of an empty diff, for a copied-but-not-modified file, where we didn't previously.

[[[
$ svn cp ^/trunk/alpha <at> 1 alpha2
A         alpha2
$ svn diff
Index: alpha2
===================================================================
$ svn18 diff   # svn18 is Subversion 1.8.9
]]]

Bug 2.

We show the left-hand-side revision as 'nonexistent' for a copied-and-modified file diff, where the real
left-hand side of the diff is the copy-from revision (revision 1 in this example).

[[[
$ echo foo >> alpha2 
$ svn diff  # diffing against the copy source
Index: alpha2
===================================================================
--- alpha2    (nonexistent)
+++ alpha2    (working copy)
 <at>  <at>  -1 +1,2  <at>  <at> 
 alpha
+foo
$ svn info alpha2
[...]
Relative URL: ^/trunk/alpha2
Revision: 1
Schedule: add
Copied From URL: file:///.../trunk/alpha
Copied From Rev: 1
Last Changed Rev: 1
]]]

Bug 3.

A WC-WC diff shows a replace-with-copy-from-self as a delete and an add, when the 'notice-ancestry'
option is given. (If we commit the change, then a repos-repos diff of the change with '--notice-ancestry'
shows a single diff, as it should.)

I suppose the WC-WC diff always assumes that any copied replacement node is unrelated to the replaced node.

In the (very) special case where the copy source path and revision are exactly equal to the replaced node's
base path and revision, this would be easy to detect as related, but other cases would, in general, require
contacting the repository.

[[[
$ svn rm alpha 
D         alpha
$ svn cp ^/trunk/new/alpha <at> 3 alpha
A         alpha
$ echo bar >> alpha 

$ svn diff --notice-ancestry 
Index: alpha
===================================================================
--- alpha    (revision 6)
+++ alpha    (nonexistent)
 <at>  <at>  -1,2 +0,0  <at>  <at> 
-alpha
-hello
Index: alpha
===================================================================
--- alpha    (nonexistent)
+++ alpha    (working copy)
 <at>  <at>  -1 +1,2  <at>  <at> 
 alpha
+bar
]]]

- Julian

Matthias Gerstner | 17 Jul 13:22 2014

[PATCH] optionally disable normalization of working copy files in diff invocations

Hello,

attached is a patch against the current subversion trunk state (revision
1611327) that fixes a problem when using external diff programs that's
also been described years ago here:

http://svn.haxx.se/users/archive-2008-10/0664.shtml

----
When passing the new diff command line option --no-normalization then
svn diff won't create and pass on a temporary, normalized version of a
file for local working copy files.

This normalization is the default behaviour when svn diff encounters
files that have the svn:keywords or svn:eol-style properties set, such
that the base version and the working copy version of the file have the
same format.

This makes it impossible to edit diffed files when using an external
--diff-cmd that supports editing, because the file passed to the
external tool is a temporary file that will be deleted afterwards,
instead of the original working copy file.

When passing --no-normalization the original file is passed to the diff
tool instead. External tools that can ignore whitespace differences
(present due to svn:eol-style) can still display decent diffs and the
benefit of editing the diffed files in place is helpful.
----

Best regards,

Matthias Gerstner

--

-- 
Matthias Gerstner, Dipl.-Wirtsch.-Inf. (FH)
Entwicklung

NCP engineering GmbH
Dombühler Straße 2, D-90449, Nürnberg
Geschäftsführer Peter Söll, HRB-Nr: 77 86 Nürnberg

Telefon: +49 911 9968-153, Fax: +49 911 9968-229
E-Mail: Matthias.Gerstner <at> ncp-e.com
Internet: http://www.ncp-e.com
Attachment (normalize.patch): text/x-diff, 16 KiB
Picon

[l10n] Translation status report for trunk r1610571

Translation status report for trunk <at> r1610571

  lang   trans untrans   fuzzy     obs
--------------------------------------
    de    2722      59     226     471  +++++++++++++++++++++++++++~~~oooo
    es    2230     551     791     528  ++++++++++++++++++UUUUU~~~~~~~oooo
    fr    2533     248     470     109  +++++++++++++++++++++++UU~~~~~o
    it    2095     686     921     340  ++++++++++++++++UUUUUU~~~~~~~~oo
    ja    2222     559     843     763  ++++++++++++++++++UUUUU~~~~~~~oooooo
    ko    2364     417     610     219  ++++++++++++++++++++UUUU~~~~~~o
    nb    2279     502     741     501  +++++++++++++++++++UUUU~~~~~~~oooo
    pl    2304     477     708     298  +++++++++++++++++++UUUU~~~~~~~oo
 pt_BR    2071     710     935     321  ++++++++++++++++UUUUUU~~~~~~~~oo
    sv    2693      88     257      67  ++++++++++++++++++++++++++U~~~
 zh_CN    2582     199     383       7  ++++++++++++++++++++++++UU~~~~
 zh_TW    2013     768     974     377  ++++++++++++++++UUUUUU~~~~~~~~ooo

Stefan Fuhrmann | 14 Jul 01:54 2014

Performance Results on Windows

After almost a week of continuous measurements and 4320
individual data point, the results are in now. Summary:

* on cold disks, 'svn log -v' is 2x..3x as fast with packed f7 than with
  packed f6 in default config and up to 8x with advanced options
* on cold disks, 'svn export' with packed f7 is 40%.50% faster in
  default config and 2x as fast with advanced options
* ra_serf has an anomaly making "fast" config up to 100x slower
  than ra_svn - independent of repo format

Details to how I set up the repositories and measurements plus
the rationale behind it can be found in our wiki:

https://wiki.apache.org/subversion/MeasuringRepositoryPerformance

Tools and test script have been committed in r1610264. System
setup is close to what Ivan used in his tests, except for having
4GB to accommodate the much larger BSD repo as well.

Original measurement log and detailed results can be found in the
.zip file attached. The spreadsheet has been upgraded and now
displays f6/f7 because "+400%" is easier to understand than "-80%".
It now also highlights data cells with particularly high variation.

I'm currently waiting for the 'svnadmin dump' results to come in.
Those will take about 3 days but the expectation is that packed f7
may be slightly slower here due to the "unusual" access pattern.
After that, I'll try to isolate the trigger for the ra_serf anomaly.

-- Stefan^2.
Attachment (fsfsPerfWin.pdf): application/pdf, 159 KiB
Attachment (WinPerf.zip): application/zip, 655 KiB

Gmane