Julian Foad | 1 Oct 17:35 2014

FSFS changed-paths handling - review

I took a look through the FSFS code that deals with the changed-path lists, with a special interest in how it
reports no-op changes. In the course of this code inspection I found a number of issues that want
clarification and a few possible bugs. I have not tried to produce actual buggy behaviour from these observations.

Some of this code is new or changed for format 7, and some is old.

Comments, concerns and suggested code updates together in the form of a diff:

[[[
Index: subversion/include/svn_fs.h
===================================================================
--- subversion/include/svn_fs.h    (revision 1628514)
+++ subversion/include/svn_fs.h    (working copy)
 <at>  <at>  -1427,38 +1427,61  <at>  <at>  typedef enum svn_fs_path_change_kind_t
 /** Change descriptor.
  *
  *  <at> note Fields may be added to the end of this structure in future
  * versions.  Therefore, to preserve binary compatibility, users
  * should not directly allocate structures of this type.
  *
+ *  <at> note The  <at> c text_mod,  <at> c prop_mod and  <at> c mergeinfo_mod flags mean the
+ * text, properties and mergeinfo property (respectively) were "touched"
+ * by the commit API; this does not mean the new value is different from
+ * the old value.
+ *
  *  <at> since New in 1.6. */
 typedef struct svn_fs_path_change2_t
 {
   /** node revision id of changed path */
   const svn_fs_id_t *node_rev_id;
(Continue reading)

Julian Foad | 1 Oct 11:52 2014

Official way to create an empty revision

Daniel Shahaf wrote in the thread "No no-op changes":
> Should we provide an "official" way to create an empty revision?  That
> is, a revision whose changed-paths list is empty?
> 
> Use-cases:
> 
> 1. Suppose last backup is r100 and revisions r101:r105 were lost; then
> after restoring the backup, the admin would create 5 empty revisions.
> 
> 2. Force an empty revision for whatever reason, such as to make the
> revnums sync to something:
> 2.1. See r3 of the regression test merge_tests.py#125 svnmucc_abuse_1().
> 2.2. W hen loading our repository to the ASF repository, if Joe had
> created 26 empty revisions, then The Offset would have been 840100
> rather than 840074, which would make our mental math easier.

Hi Daniel. It seems a reasonable tool to have in the svn admin's tool kit. Perhaps not often, but people do
sometimes want this. I found two web pages where people discussed this. One wrote a script that spits out
the appropriate few lines of dump file text to represent an empty rev, N times [1]; the other is worse,
committing N changes to a temporary repo, dumping it and filtering everything out [2].

I'm assuming this proposal is restricted to the admin side. Your use cases 1. and 2.2 are both admin use
cases. Your use case 2.1 is a test which uses a client-side commit to make an uninteresting revision, in
order to make the subsequent revision numbers match (modulo 10) those in the original use case. While
people no doubt do this sort of thing sometimes in real life, I can't think of a general behaviour that would
make sense from the client side. In a shared repository, you never know what revision number your next
commit will have.

For a UI, I can envisage two useful ways to expose this functionality: commit N empty revisions as a
stand-alone operation, and commit N empty revisions before the first revision loaded from a dump stream.
(Continue reading)

Picon

[l10n] Translation status report for trunk r1628357

Translation status report for trunk <at> r1628357

  lang   trans untrans   fuzzy     obs
--------------------------------------
    de    2755      96     266     478  ++++++++++++++++++++++++++U~~~oooo
    es    2256     595     816     526  ++++++++++++++++++UUUUU~~~~~~~oooo
    fr    2566     285     505     108  ++++++++++++++++++++++UUU~~~~~
    it    2120     731     945     340  ++++++++++++++++UUUUUU~~~~~~~~oo
    ja    2247     604     867     763  ++++++++++++++++++UUUUU~~~~~~~oooooo
    ko    2392     459     637     217  ++++++++++++++++++++UUUU~~~~~~o
    nb    2307     544     768     499  +++++++++++++++++++UUUU~~~~~~~oooo
    pl    2332     519     735     296  +++++++++++++++++++UUUU~~~~~~~oo
 pt_BR    2096     755     958     321  ++++++++++++++++UUUUUU~~~~~~~~oo
    sv    2724     127     295      72  +++++++++++++++++++++++++UU~~~
 zh_CN    2618     233     423      13  +++++++++++++++++++++++UUU~~~~
 zh_TW    2037     814     996     377  +++++++++++++++UUUUUUU~~~~~~~~oo

Michael Uray | 27 Sep 21:30 2014
Picon

Request a new feature: Point with "svn:externals" to another "svn:externals" attribute

Hi guys,

 

I would like to request a new feature for SVN.

 

I found out, that it is not possible to point with an “svn:external” attribute directly to another “svn:external” attribute.

 

You can find on the following link an example, how I would like to setup it, but it does not work with the current SVN version:

https://dl.dropboxusercontent.com/u/16833149/SVN_test_rep.7z

 

If you like please find my forum post about this idea on the following page:

http://www.wandisco.com/svnforum/threads/69097-Link-quot-svn-externals-quot-to-another-quot-svn-externals-quot-Link

 

Best regards

Michael

Ivan Zhakov | 26 Sep 16:38 2014

FSFS7 repository corruption during commit with background upgrade

I was looking how fsfs upgrade code works and found particular fsfs7
log-addressing repository corruption bug:
0. Repository has 999 revisions.
1. Client begins committing some data to fsfs v6 repository through Apache
2. Apache web server opens svn_fs_t, reads format file. At this point
   repository has format 6 and uses physical (classic) addressing.
3. Client changes txn content
4. Before committing change, admin upgrades this repository to fsfs7. FSFS
   upgrade code marks that log-addressing will be available from next shard,
   i.e. from revision 1000.
5. Apache web server starting committing txn: obtaining write-lock and writes
   protorev for r1000. Since svn_fs_t instance was cached for connection so
   it didn't know that revision 1000 should be log addressing and writes
   physical addressing revision without any error (!)

Commit succeeded, but repository is unreadable.

I'm attaching patch with test reproducing this issue. The commit may
fail in maintainer mode because txn will be verified before commit.

-- 
Ivan Zhakov
Index: subversion/tests/libsvn_fs/fs-test.c
===================================================================
--- subversion/tests/libsvn_fs/fs-test.c	(revision 1627774)
+++ subversion/tests/libsvn_fs/fs-test.c	(working copy)
 <at>  <at>  -5403,7 +5403,57  <at>  <at> 
 #endif
 }

+static svn_error_t *
+upgrade_while_committing(const svn_test_opts_t *opts,
+                         apr_pool_t *pool)
+{
+  svn_fs_t *fs;
+  svn_revnum_t head_rev = 0;
+  svn_fs_root_t *root;
+  svn_fs_txn_t *txn;
+  const char *fs_path;
+  apr_hash_t *fs_config = apr_hash_make(pool);

+  /* Bail (with success) on known-untestable scenarios */
+  if (strcmp(opts->fs_type, "fsfs") != 0)
+    return svn_error_create(SVN_ERR_TEST_SKIPPED, NULL,
+                            "this will test FSFS repositories only");
+
+  if (opts->server_minor_version && (opts->server_minor_version < 6))
+    return svn_error_create(SVN_ERR_TEST_SKIPPED, NULL,
+                            "pre-1.6 SVN doesn't support FSFS packing");
+
+  /* Create test repository with greek tree. */
+  fs_path = "test-upgrade-while-committing";
+
+  svn_hash_sets(fs_config, SVN_FS_CONFIG_COMPATIBLE_VERSION, "1.7");
+  svn_hash_sets(fs_config, SVN_FS_CONFIG_FSFS_SHARD_SIZE, "2");
+  SVN_ERR(svn_test__create_fs2(&fs, fs_path, opts, fs_config, pool));
+
+  SVN_ERR(svn_fs_open(&fs, fs_path, NULL, pool));
+  SVN_ERR(svn_fs_begin_txn(&txn, fs, head_rev, pool));
+  SVN_ERR(svn_fs_txn_root(&root, txn, pool));
+  SVN_ERR(svn_test__create_greek_tree(root, pool));
+  SVN_ERR(test_commit_txn(&head_rev, txn, NULL, pool));
+
+  /* Upgrade filesystem, but keep existing svn_fs_t object. */
+  SVN_ERR(svn_fs_upgrade(fs_path, pool));
+
+  /* Create txn with changes. */
+  SVN_ERR(svn_fs_begin_txn(&txn, fs, head_rev, pool));
+  SVN_ERR(svn_fs_txn_root(&root, txn, pool));
+  SVN_ERR(svn_fs_make_dir(root, "/foo", pool));
+
+  /* Commit txn. */
+  SVN_ERR(test_commit_txn(&head_rev, txn, NULL, pool));
+
+  /* Verify filesystem content. */
+  SVN_ERR(svn_fs_verify(fs_path, NULL, 0, SVN_INVALID_REVNUM, NULL, NULL,
+                        NULL, NULL, pool));
+
+  return SVN_NO_ERROR;
+}
+
 /* ------------------------------------------------------------------------ */
 
 /* The test table.  */
 <at>  <at>  -5501,6 +5551,8  <at>  <at> 
     SVN_TEST_OPTS_WIMP(reopen_modify,
                        "test reopen and modify txn",
                        "txn_dir_cache fail in FSFS"),
+    SVN_TEST_OPTS_PASS(upgrade_while_committing,
+                       "upgrade while committing"),
     SVN_TEST_NULL
   };

Mohsin Abbas | 26 Sep 00:25 2014
Picon

SVN Commit Failed For Data larger Than 2 GB [How To Resole]

Good Day Team,

I am using Subversion 1.8.9 server on linux OS and tortoise SVN client
at windows. When I try to commit data larger then 2+ GB my commit
failed. I have put question on users <at> subversion.apache.org group but
no fruit full solution I got. I tried to google on different websites
they provided below solution :

1 : Set LimitRequestBody to 0 in the server side httpd.conf file.

But Apache Web server ( Apache 2.2.24 ) allows 2 GB only max limit for
data in HTTP request. Now tell me how can I commit files which having
size more than 2 GB ? How can I increase limit in apache webserver
more than 2 GB ? OR I assume that we can not commit data more than 2
GB because svn not allows us for this ?

Regards
Mohsin Abbas

James McCoy | 24 Sep 04:45 2014
Picon

[PATCH] Search for libtoolize, not libtool

Hi,

build/buildcheck.sh performs a search for a variant of libtool in order
to verify an acceptable version is available.  However, the similar
code in autogen.sh looks for and uses a variant of libtoolize.

Since libtoolize is the tool actually performing useful work between the
two, and buildcheck.sh fails the build if libtool isn't found, updating
buildcheck.sh to look for libtoolize seems appropriate.  This specific
build failure does occur[0] when trying to cross-build subversion with
Ubuntu's (and soon Debian's) Multi-Archified libtool packages[1].

The attached patch, proposed by Helmut Grohne[2], makes said change to
buildcheck.sh.

[0]: https://people.debian.org/~doko/logs/20140912/failed-libtool/subversion_1.8.10-1_unstable_jdk-libtool.log
[1]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=682045
[2]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=761789#12
--

-- 
James
GPG Key: 4096R/331BA3DB 2011-12-05 James McCoy <jamessan <at> debian.org>
Attachment (libtoolize.diff): text/x-diff, 869 bytes
Picon

[l10n] Translation status report for trunk r1626934

Translation status report for trunk <at> r1626934

  lang   trans untrans   fuzzy     obs
--------------------------------------
    de    2754      94     265     478  ++++++++++++++++++++++++++U~~~oooo
    es    2255     593     815     526  ++++++++++++++++++UUUUU~~~~~~~oooo
    fr    2565     283     504     108  ++++++++++++++++++++++UUU~~~~~
    it    2119     729     944     340  ++++++++++++++++UUUUUU~~~~~~~~oo
    ja    2246     602     866     763  ++++++++++++++++++UUUUU~~~~~~~oooooo
    ko    2391     457     636     217  ++++++++++++++++++++UUUU~~~~~~o
    nb    2306     542     767     499  +++++++++++++++++++UUUU~~~~~~~oooo
    pl    2331     517     734     296  +++++++++++++++++++UUUU~~~~~~~oo
 pt_BR    2095     753     957     321  ++++++++++++++++UUUUUU~~~~~~~~oo
    sv    2723     125     294      72  +++++++++++++++++++++++++UU~~~
 zh_CN    2617     231     422      13  ++++++++++++++++++++++++UU~~~~
 zh_TW    2036     812     995     377  +++++++++++++++UUUUUUU~~~~~~~~oo

Julian Foad | 19 Sep 16:49 2014

No no-op changes

Summary:
  * A "no-op change" is not a change.
  * Subversion should not report a "no-op change" as a "change".
  * We should bear this in mind when designing and reviewing.
  * Fixes are needed in a few places.

I noticed recently that we handle "no-op changes" inconsistently. By a "no-op change" I mean, for example,
a request to set property 'p' to value 'v' when it already had value 'v'.

[[[
$ svn propget p $REPO/trunk -r5
v

$ svnmucc -U file://$PWD/repo -m "" propset p v trunk
r6 committed ...

$ svn diff --summarize -c6 $REPO 
[no output]

$ svn log -vq --xml -r6 $REPO 
<...>
<path
   kind="dir"
   action="M"
   text-mods="false"
   prop-mods="true">/trunk</path>
<...>
]]]

This output says that there was a property "modification" in r6 ... and yet shows no changes.

Why is this wrong? Fundamentally, Subversion versions tree snapshots. A change is defined as the
difference between two tree snapshots. In this sense there is no such thing as a no-op change: it just means
no change.

Many of our APIs, including for example a "commit editor", allow specifying a new state which may be the same
as the old state. That is fine. But anywhere we receive a new state and blindly report it as a "change"
regardless of whether it differs from the previous state, that is wrong. The fact that we do sometimes
record such an event -- and such events are stored in repositories already -- does not make it a meaningful
event; it is an implementation artefact that happens to leak out at one or two places.

I talked about this with Stefan Fuhrmann. We think the main place where a fix is needed is in the
repository-side commit processing. It could elide no-op changes from a transaction before committing
it, or maybe just remove the change entry from the "changes" list so that new no-op changes don't show up in
'log'. We also need to filter out no-op "change" records when reading old repository data.

The ability to commit a new revision with no changes is fine -- a new revision number in Subversion does not
imply a change.

Can someone please confirm this all makes sense. I'll then identify specific bugs to fix.

Thanks.

- Julian

Picon

[l10n] Translation status report for trunk r1625212

Translation status report for trunk <at> r1625212

  lang   trans untrans   fuzzy     obs
--------------------------------------
    de    2753      94     264     478  ++++++++++++++++++++++++++U~~~oooo
    es    2254     593     814     526  ++++++++++++++++++UUUUU~~~~~~~oooo
    fr    2564     283     503     108  ++++++++++++++++++++++UUU~~~~~
    it    2118     729     943     340  ++++++++++++++++UUUUUU~~~~~~~~oo
    ja    2245     602     865     763  ++++++++++++++++++UUUUU~~~~~~~oooooo
    ko    2390     457     635     217  ++++++++++++++++++++UUUU~~~~~~o
    nb    2305     542     766     499  +++++++++++++++++++UUUU~~~~~~~oooo
    pl    2330     517     733     296  +++++++++++++++++++UUUU~~~~~~~oo
 pt_BR    2094     753     956     321  ++++++++++++++++UUUUUU~~~~~~~~oo
    sv    2722     125     293      72  ++++++++++++++++++++++++++U~~~
 zh_CN    2616     231     421      13  ++++++++++++++++++++++++UU~~~~
 zh_TW    2035     812     994     377  +++++++++++++++UUUUUUU~~~~~~~~oo

Stefan Sperling | 10 Sep 19:06 2014
Picon

[VOTE] merge the log-message-templates branch to trunk

I've been working on the log-message-templates branch recently.
This branch was started by cmpilato a long time ago but for some
reason sat around untouched ever since. It's now been overhauled
by me with much help from Bert.

I believe the functionality is useful and complete. One user I have
in mind who would probably make use of this is the FreeBSD project.
They patch their Subversion clients to define a log message template
with the following content:
[[[
PR:
Submitted by:
Reviewed by:
Approved by:
Obtained from:
MFC after:
MFH:
Relnotes:
Security:
Sponsored by:
]]]
The patch is here:
http://svnweb.freebsd.org/ports/head/devel/subversion/files/extra-patch-fbsd-template?revision=361003&view=markup
With the changes on the log-message-templates branch, they can drop their
custom patch and add their log message template to an svn:log-template
property instead.

I believe many other users might find this feature useful as well.

The current diff between the trunk and branch is this:
svn diff ^/subversion/trunk <at> 1623990 ^/subversion/branches/log-message-templates <at> r1624055

Because there was quite a bit of back-and-forth on this branch
here's a quick overview of what the branch implements right now:

[[[
Introduce a new svn:log-template property. This is an inherited property
which can be used to define static log message templates to be inserted
into the default log message by the client.

This branch adds the following public API elements:

  SVN_PROP_INHERITABLE_LOG_TEMPLATE: New property name.
  svn_client_get_log_message_template(),
  svn_client_get_log_message_templates_for_commit_items(): New functions
   which provide easy access to log templates defined for a given path
   or URL, or a set of commit items. 

For a given path, the svn:log-template property is always inherited
from the nearest ancestor which has the property set on itself.
For a given commit, however, the set of commit target paths is arbitrary,
so multiple templates may apply, each contributed by one or more paths
in the commit target list.

For example, with:

  $ svn ci iota.txt epsilon/gamma.txt

we might see the following default log message:

  --Log message template from '/'--
  The root log message template.
  This always applies since it is defined at the root of the repository.
  --Log message template from 'trunk/epsilon'--
  This is a template set on 'trunk/epsilon', which in this example
  was contributed by the commit target epsilon/gamma.txt.
  --This line, and those below, will be ignored--

The "--Log message template from --" line does not appear if only
a single template applies to the commit.

A rough overview of the files changed (see branch commit log for details):

* subversion/tests/libsvn_client/client-test.c
  (test_log_message_template): New test.

* subversion/svn/cl.h
  (svn_cl__make_log_msg_baton): Change 'config' hash parameter to a
   client context 'ctx'. 'config' is still available as 'ctx->config'.

* subversion/svn/move-cmd.c, subversion/svn/copy-cmd.c,
  subversion/svn/commit-cmd.c, subversion/svn/delete-cmd.c,
  subversion/svn/mkdir-cmd.c, subversion/svn/import-cmd.c,
  subversion/svn/propedit-cmd.c: Adjust callers of svn_cl__make_log_msg_baton().

* subversion/svn/util.c
  (log_msg_baton): Store entire client context instead of just the config hash.
  (svn_cl__make_log_msg_baton): Adjust accordingly.
  (svn_cl__get_log_message): Call the new API function
   svn_client_get_log_message_templates_for_commit_items() to obtain
   log templates and add the templates to the default log message.

* subversion/include/svn_props.h
  (SVN_PROP_INHERITABLE_LOG_TEMPLATE): New property.

* subversion/include/svn_client.h
  (svn_client_get_log_message_template,
   svn_client_get_log_message_templates_for_commit_items): New API functions.

* subversion/libsvn_client/commit_util.c
  (svn_client_get_log_message_template,
   svn_client_get_log_message_templates_for_commit_items): Provide convenient
   functionality for clients wishing to support log message templates.
]]]


Gmane