Re: patch implementing restart, other features

Hello,

 >Wow! Thanks for all the great patches. Unfortunately (as you probably
 >noticed), I have split much of the code up since 0.3.3 and hadn't
 >planned on changing it (the 0.3 branch) anymore, but there is too much
 >good stuff here to not use it. I'll probably throw this into a 0.3.4
 >release and backport (foreport?) what I can into trunk.

Yes, I saw that you split 0.3.3 into modules.  I didn't know what state 
trunk was in, because a couple of things looked newer in the old 
codebase (DST comes to mind).  I was also unsure how to deal with the 
modules, too.

 >> Idiosyncratic features you may not want:
 >> -- the ability to substitute a SS regex pattern that denotes a single
 >> file. I needed this because I had such files in my SS (!) and had no
 >> idea how to get them out (never mind how they were allowed in)!
 >
 >I'm sorry, I've read this three times and can't grok it. Are you saying
 >you allow a regex to specify files which should not be migrated? (I
 >haven't looked at the code in-depth yet, maybe it will be clear after I
 >look into it a bit more).

Ok.  Here is my situation:  I have files that have been checked into VSS 
that have as the first character dot (.).  For example: "$/foo/.foo" .  
I have no idea how these files made it into the project, but they are 
there.  I chose to try and migrate these files.  Now when I try and get 
history for "$/foo/.foo" ss.exe errors and the script dies. Fortunately 
for me, these files are named uniquely enough that a regex for 
"$/foo/.foo" (for example "$/foo/?foo") will uniquely identify 

Re: patch implementing restart, other features

Leo Davis wrote:

> Ok.  Here is my situation:  I have files that have been checked into 
> VSS that have as the first character dot (.).  For example: 
> "$/foo/.foo" .  I have no idea how these files made it into the 
> project, but they are there.  I chose to try and migrate these files.  
> Now when I try and get history for "$/foo/.foo" ss.exe errors and the 
> script dies. Fortunately for me, these files are named uniquely enough 
> that a regex for "$/foo/.foo" (for example "$/foo/?foo") will uniquely 
> identify "$/foo/.foo" and no other in that project.  When I get 
> history (or get) "$/foo/?foo" everything seems to work ok.
>
> Has anyone else seen this problem with VSS?

No, I've never encountered such a problem, but I'm glad there's a 
workaround!

Store only latest version?

Re: Store only latest version?

Labels

RE: patch implementing restart, other features

By using this patch I was able to get a lot further to getting out
source control moved over. I no longer have issues with labels. But I am
seeing one strange thing.

After importing our 3 year old VSS project into a clean svn repository
the revision number is 13143. The top of the source tree is the same as
VSS but I only see change information for last 2 weeks.

When I do a recursive log from the root the first log message is at
revision 10363 with a date of about 2 weeks ago.

Any idea why it would skip/miss all of the changes over 2 weeks old?

-allen

-----Original Message-----
From: Leo Davis [mailto:ldavis <at> fonix.com] 
Sent: Friday, February 25, 2005 5:09 PM
To: users <at> vss2svn.tigris.org
Subject: patch implementing restart, other features

Hello,

I have a patch created against v0.3.3 with probably too many features, 
but here it is.

Features:
  -- restartable during history gathering and committing to subversion
  -- tries to give a better authentication message, depending on the URL

schema used to access the repository
  -- uses only one user to authenticate to the repository 
(vss_migration, by default)
  -- by default assumes that you will set revision properties for date 
and author, but this selectable from a command line switch.
  -- turns off subversion authentication caching
  -- associates label text with the correct revision

Idiosyncratic features you may not want:
  -- the ability to substitute a SS regex pattern that denotes a single 
file.  I needed this because I had such files in my SS (!) and had no 
idea how to get them out (never mind how they were allowed in)!

Other changes:
I also had to make the database autocommit because it kept trying to 
commit while I was trying to commit.
Subroutine calls were modified to use prepared statements instead of do 
statements.
Scheduling commits and committing are now separate.

Re: Labels

These are all good suggestions. The label situation is very broken, I agree.

I can't think of any way to reliably fetch from VSS while a previous SVN 
commit is occurring. There would be a race condition allowing the 
possibility that VSS would overwrite a previous version with a newer one 
before SVN had a chance to commit it.

Roger Binns wrote:

> I am having a problem with labels.  Ideally I would like to just turn 
> them off.
>  
> vss2svn is creating huge log messages by inserting all the label 
> information, and in many cases is completely wrong.  For example a 
> file added two weeks ago now has labels going back 4 years listed in 
> the log, and that sure is a lot of them.
>  
> I'd also suggest updating the doc to suggest copying the VSS 
> repository to a local disk before running vss2svn.  By my estimates it 
> would have taken 3 weeks with the repository remote and it took around 
> 24 hours with it local.  Disk speed seems to be one of the crucial 
> factors.
>  
> There would also be a performance increase if new revisions are 
> fetched from VSS while the svn commit from the previous changeset is 
> still running.
>  
> Roger

Re: patch implementing restart, other features

Allen Unueco wrote:

>After importing our 3 year old VSS project into a clean svn repository
>the revision number is 13143. The top of the source tree is the same as
>VSS but I only see change information for last 2 weeks.
>
>When I do a recursive log from the root the first log message is at
>revision 10363 with a date of about 2 weeks ago.
>
>Any idea why it would skip/miss all of the changes over 2 weeks old?
>  
>
That is rather odd. Try looking at the logfile in the _vss2svn directory 
where you ran the conversion to see if anyting weird is going on. Also, 
run "svn status" in the WC that vss2svn created to see if any commits 
got missed somehow.

Re: patch implementing restart, other features

Allen Unueco wrote:

>By using this patch I was able to get a lot further to getting out
>source control moved over. I no longer have issues with labels. But I am
>seeing one strange thing.
>
>After importing our 3 year old VSS project into a clean svn repository
>the revision number is 13143. The top of the source tree is the same as
>VSS but I only see change information for last 2 weeks.
>
>When I do a recursive log from the root the first log message is at
>revision 10363 with a date of about 2 weeks ago.
>
>Any idea why it would skip/miss all of the changes over 2 weeks old?
>  
>
It might be a bug. I found that when I was getting history form ss.exe 
(at least in the original 0.3.3 code) I was missing history for some 
files. I changed the code in that part so at least the instance of 
history loss that I was seeing got all the history instead of a part. 
IIRC, it would skip history if there was no comment. That should be 
fixed in the patch.

Toby had never seen my problem with his ss.exe. Maybe your ss.exe is 
outputting history in yet another variation.

Is it possible to post the history for one of the files that is missing 
changes? The debug output is in _vss2svn\logfile.txt. If history is to 
blame, it should be pretty easy to see from this.

Leo

Re: Labels

On Thu, 2005-03-10 at 00:28 -0500, Toby Johnson wrote:
> These are all good suggestions. The label situation is very broken, I agree.

I think the label thing can be relatively easily fixed.  (By fixed I
mean not including them as parts of the commit comments for the next
revision.)  In the states you have parsing the ss history output, while
in state 4 (comments), have a flag that if you see a line of
'**********' then don't append any further lines to that comment.  Reset
the flag on the next state change.  (Or call it state 5 

> I can't think of any way to reliably fetch from VSS while a previous SVN 
> commit is occurring. There would be a race condition allowing the 
> possibility that VSS would overwrite a previous version with a newer one 
> before SVN had a chance to commit it.

I don't know the details of how SS works, but maybe you can have a set
of numbered directories where the changed files for each changeset are
checked out to (a ring buffer).  Another part can then swallow them into
the working area for svn as it does each transaction.

Also using Process Explorer from sysinternals.com, I see that a CMD.EXE
is started to spawn each svn.exe.  Things would be just that little bit
faster directly invoking svn.exe rather than indirecting through a
command interpreter.

You may also be able to reduce the amount of disk thrashing every time
svn.exe is run when committing transactions.  As far as I can tell, svn
commit is run each time from the top of the working copy.  It would be
better to run it from as deep into the working copy as possible that
encompasses all changed files.  Even just distinguishing the case when
one file has changed vs more than one would help since a significant
proportion of changesets only involve one file.

The reason why speed matters is simply that repository conversion needs
to take place in less than a weekend to allow a clean changeover!

There is also another buglet.  If someone changes the same file twice
with the same comment with no other commits inbetween then it is
regarded as one changeset.  The earlier short lived change is not seen
or noted in the svn history.

It would be really nice if there was a 0.3.4 with 3 simple changes:

  - ignore labels seen in history
  - invoke svn.exe rather than going via command interpreter
  - being as specific as possible in the svn commit command

Roger