Unison 2.40.16 problem?

AW: Unison 2.40.16 problem?

Re: AW: Unison 2.40.16 problem?

Another thing to check: It's possible that OCaml is running out of heap space.  Might be worth seeing if
there's a heap space limit that needs to be raised in your OS.

    - B

On Oct 1, 2010, at 1:41 PM, Harald Heigl wrote:

> 
> 
> Hi!
> Just some short thoughts:
> 1.       How fast is your internet connection between the two offices?
> 2.       Is that an initial sync or does it also break at further syncs with only small changes, btw. how much of the
20GB must be synced daily?
> 3.       You could start a try copying with rsync and look  if that syncs the 20GB data. (or if rsync works you could
try rsync always for large file transfer – option copyprog in unison, for the rest of smaller file
changes you use unison itself to copy)
> 4.       If you have direct (console) access to your readyNAS you could try compiling unison:
> a.       cd /tmp
> b.      download with wget http://www.seas.upenn.edu/~bcpierce/unison//download/releases/beta/unison-2.40.16.tar.gz
> c.       tar –xvf unison-2.40.16.tar.gz
> d.      cd unison-2.40.16
> e.      I think readyNas has apt to install packages, you need an ocaml compiler. I think this should do it, but
I’m not sure, not having ReadyNAS: apt-get install ocaml-nox ocaml-native-compilers
> f.        And now you should be able to compile unison: something like make or make install. Don’t know exactly
anymore. After that you should have a ready to use unison binary. If not you may write back …
>  
> Regards,
> Harald
>  
> Von: unison-users <at> yahoogroups.com [mailto:unison-users <at> yahoogroups.com] Im Auftrag von Matthew Colls
> Gesendet: Freitag, 01. Oktober 2010 14:54
> An: unison-users <at> groups.yahoo.com
> Betreff: [unison-users] Unison 2.40.16 problem?
>  
>  
> Hi All,
>  
> I’ve been trying to use Unison to synchronize files between two offices.  We have about 20GB of data in a
mishmash of files ranging from the very small to quite large (1GB ish), that we both edit at each end. 
Perfect for Unison over night I thought.
>  
> The problem is that it never manages to complete without crashing.
>  
> It displays a range of different crashes.  Some just freeze (CTL-C won’t exit it).  The most recent looks
like example 1 below.  I’ve tried everything I can think of with setup and configuration, reading debug
output etc.
>  
> I’d really appreciate any comments anyone has to offer.  My current setup is in Exhibit 2 and profile in
Exhibit 3 below.
>  
> It syncs quite a bit (I’ve managed about 1GB out of a total of 20GB), but tends to fail fairly randomly, but
possibly on large/deep directory trees.
>  
> Is this a known problem?  Is there anything I can do to improve reliability?  How can I find the cause of the crashes?
>  
>  
> Many thanks in advance and best regards,
>  
> Matthew.
>  
>  
>  
>  
>  
>  
>  
>  
> Exhibit 1
>   0%  184:41 ETA<>$<>$<>$<>$<>Starting new major GC cycle
>
!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>$<>$<Growing
heap to 2808k bytes
> >$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$
> <>$<>$<>Starting new major GC cycle
>
!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!Growing
heap to 3304k bytes
> <>!<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>
> $<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>Starting new major GC cycle
>
!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!Growing
heap to 3800k bytes
> <>!<>!<>!<>!<>!<>!<>!<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$
>  
> Exhibit 2
> Unison 2.40.16 running in socket mode on one machine, and client on the other, both running Windows 7.
> Data is in mapped drives connected from a Netgear ReadyNAS, mapped to c:\MOUNT at each end.  (since no
Unison binaries for readynas nv+ that I’ve found)
>  
> SSH isn’t required since it’s all running through a vpn anyway.
>  
> Exhibit 3
> root = c:/MOUNT
> root = socket://server-pc:10001/c:/MOUNT
> path = data_archive/Jobs/
>  
> mountpoint = data_archive
>  
> batch = true
>  
> log = true
>  
> debug=all
>  
> logfile = c:/unison_sync.log
>  
> fat = true
> halfduplex = true
>  
> ignorecase = true
>  
> times = true
>  
> fastcheck = false
>  
>  
>  
> Matthew Colls
> +44 7970 179785
> matthew.colls <at> prevailinganalysis.com
> www.prevailinganalysis.com
>  
> 
> 
> 

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

RE: AW: Unison 2.40.16 problem?

Ben, Harold,

Thanks ever so much for your prompt replies.

I think I've cracked it (at least mostly - still a couple of failed transfers but I'm retrying now).

In order:

1: connectivity:  internet connection is fast enough I think.  Asymmetric but 10 up and 2 down at each end. 
Never been a problem with anything else.  Direct copies work fine.

2: it's the initial sync.  We may be able to pare down the synced data to 10GB but not much lower.  I'm really keen
to get this working for all cases.

3: I don't believe it is a file size related problem.  It has been failing on a 300kB file, after transferring
larger ones.  Directory tree complexity perhaps?  They're quite deep trees of information.

4: Compilation for NAS: Just attempted but failing.  Apt is now installed, as is ocaml using apt-get, used
wget, but make fails as below.  Is this familiar to you?  I can't find any reference to a sh command "as", but
the errors look like it might be trying to use sh to run a .ml file somewhere in make, which doesn't sound right.

5: heap space:  I've maxed out the 3rd value as per (http://support.microsoft.com/kb/947246) but I'm way
outside my technical comfort zone at this stage.

Summary:

I think it has been (5) that has addressed the problem, so good advice for any other Windows 7 users would be to
follow the above link to fix this.  Our machine at the client end is 64 bit, so things may be a little different
for other computers.  No change required at the server end (yet).

A unison compile on the ReadyNAS would be a "nice to have".  From other fora, it looks like this has been on the
wish list for a few years.  That said I couldn't manage to get it to work.  Any ideas on (4) above or the make
output below?

Best regards,

Matthew.

Make output on ReadyNAS:

bristol-nas:/tmp/unison-2.40.16# make
UISTYLE = text
Building for Unix
NATIVE = true
THREADS = false
STATIC = false
OSTYPE =
OSARCH =
ocamlopt: ubase/rx.ml ---> ubase/rx.cmx
ocamlopt -I lwt -I ubase -I system -I system/generic -I lwt/generic -c /tmp/unison-2.40.16/ubase/rx.ml
sh: line 1: as: command not found
Assembler error, input left in file /tmp/camlasmd9ff82.s
make: *** [ubase/rx.cmx] Error 2

--
Matthew Colls
+44 7970 179785
matthew.colls <at> prevailinganalysis.com
www.prevailinganalysis.com

-----Original Message-----
From: unison-users <at> yahoogroups.com [mailto:unison-users <at> yahoogroups.com] On Behalf Of Benjamin C. Pierce
Sent: 01 October 2010 19:13
To: Harald Heigl
Cc: unison-users <at> yahoogroups.com
Subject: Re: AW: [unison-users] Unison 2.40.16 problem?

Another thing to check: It's possible that OCaml is running out of heap space.  Might be worth seeing if
there's a heap space limit that needs to be raised in your OS.

    - B

On Oct 1, 2010, at 1:41 PM, Harald Heigl wrote:

>
>
> Hi!
> Just some short thoughts:
> 1.       How fast is your internet connection between the two offices?
> 2.       Is that an initial sync or does it also break at further syncs with only small changes, btw. how much of the
20GB must be synced daily?
> 3.       You could start a try copying with rsync and look  if that syncs the 20GB data. (or if rsync works you could
try rsync always for large file transfer - option copyprog in unison, for the rest of smaller file changes
you use unison itself to copy)
> 4.       If you have direct (console) access to your readyNAS you could try compiling unison:
> a.       cd /tmp
> b.      download with wget http://www.seas.upenn.edu/~bcpierce/unison//download/releases/beta/unison-2.40.16.tar.gz
> c.       tar -xvf unison-2.40.16.tar.gz
> d.      cd unison-2.40.16
> e.      I think readyNas has apt to install packages, you need an ocaml compiler. I think this should do it, but
I'm not sure, not having ReadyNAS: apt-get install ocaml-nox ocaml-native-compilers
> f.        And now you should be able to compile unison: something like make or make install. Don't know exactly
anymore. After that you should have a ready to use unison binary. If not you may write back ...
>
> Regards,
> Harald
>
> Von: unison-users <at> yahoogroups.com [mailto:unison-users <at> yahoogroups.com] Im Auftrag von Matthew Colls
> Gesendet: Freitag, 01. Oktober 2010 14:54
> An: unison-users <at> groups.yahoo.com
> Betreff: [unison-users] Unison 2.40.16 problem?
>
>
> Hi All,
>
> I've been trying to use Unison to synchronize files between two offices.  We have about 20GB of data in a
mishmash of files ranging from the very small to quite large (1GB ish), that we both edit at each end. 
Perfect for Unison over night I thought.
>
> The problem is that it never manages to complete without crashing.
>
> It displays a range of different crashes.  Some just freeze (CTL-C won't exit it).  The most recent looks
like example 1 below.  I've tried everything I can think of with setup and configuration, reading debug
output etc.
>
> I'd really appreciate any comments anyone has to offer.  My current setup is in Exhibit 2 and profile in
Exhibit 3 below.
>
> It syncs quite a bit (I've managed about 1GB out of a total of 20GB), but tends to fail fairly randomly, but
possibly on large/deep directory trees.
>
> Is this a known problem?  Is there anything I can do to improve reliability?  How can I find the cause of the crashes?
>
>
> Many thanks in advance and best regards,
>
> Matthew.
>
>
>
>
>
>
>
>
> Exhibit 1
>   0%  184:41 ETA<>$<>$<>$<>$<>Starting new major GC cycle
>
!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>$<>$<Growing
heap to 2808k bytes
> >$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$
> <>$<>$<>Starting new major GC cycle
>
!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!Growing
heap to 3304k bytes
> <>!<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>
> $<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>Starting new major GC cycle
>
!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!<>!Growing
heap to 3800k bytes
> <>!<>!<>!<>!<>!<>!<>!<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$<>$
>
> Exhibit 2
> Unison 2.40.16 running in socket mode on one machine, and client on the other, both running Windows 7.
> Data is in mapped drives connected from a Netgear ReadyNAS, mapped to c:\MOUNT at each end.  (since no
Unison binaries for readynas nv+ that I've found)
>
> SSH isn't required since it's all running through a vpn anyway.
>
> Exhibit 3
> root = c:/MOUNT
> root = socket://server-pc:10001/c:/MOUNT
> path = data_archive/Jobs/
>
> mountpoint = data_archive
>
> batch = true
>
> log = true
>
> debug=all
>
> logfile = c:/unison_sync.log
>
> fat = true
> halfduplex = true
>
> ignorecase = true
>
> times = true
>
> fastcheck = false
>
>
>
> Matthew Colls
> +44 7970 179785
> matthew.colls <at> prevailinganalysis.com
> www.prevailinganalysis.com
>
>
>
>

------------------------------------

Yahoo! Groups Links

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

Some questions about Unison

I read now a couple of times the Unison manual but need some confirmation of my understanding resp. have
still some questions:

1.) Lets say I want to sync 10 local directory trees with 10 remote directory trees. Can I put these 10 sync
operations into ONE .unison profile? I could imagine that I can concatenate them in one profile similar to:

root = D:\localdir1
root = \\192.168.0.34\remdir1

ignore=....

root = D:\data\localdir2
root = \\192.168.0.22\remdir2

ignore = ....

...

Is this possible? Or do I have to setup 10 different profile files?

2.) Do I have to use forward slashes in a profile file even if I sync windows directories?

3.) If I want to run (cmdline based) Unison from batch file I want auto-acceptance for non-conflicting
actions BUT WANT a prompt for conflicts.
The "-auto" flag is suitable for this?

4.) Can I let (cmdline based) Unison show all conflicts at once rather than prompting me each conflict
individually ?

5.) How can I tell (cmdline based) Unison to only perform a preview run (and do not real operations)?

6.) How can I tell (cmdline based) Unison to use login/passwd e.g. myuser1/mypass23 as login to the remote
computer? ...before doing the actual syncs?

Thank you for the answers
Ben

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

Re: Some questions about Unison

Re: Some questions about Unison

> 5.) How can I tell (cmdline based) Unison to only perform a preview run (and do not real operations)?
> 
> Sorry, can't help with this one.  I only use the command line in batch mode (no interaction).  The GUI does it
this way by default.

   unison ...flags... < /dev/null

will work as long as the -batch and -auto preferences are not set.  By default, unison does absolutely
nothing to the filesystems without instructions.

    - Benjamin

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

remote unison server cannot get out of sleep

When doing a large sync, I noticed that if the remote unison server
first finished scanning all files and put to a sleep waiting for the
local unison machine to finish its scan, the remote server cannot be
waked up.

It ends up in a dead lock that both machines are waiting.

I am running unison version 2.32.52 on local XP and a remote SusE 11.3, sync via ssh
Any idea ?

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

No virus found in this outgoing message.
Checked by AVG - www.avg.com 
Version: 9.0.862 / Virus Database: 271.1.1/3181 - Release Date: 10/07/10 02:34:00

remote unison server cannot get out of sleep

One more thing to add on the above.

The problem happens only when I connect from my office XP to home linux
box. The home network connects to internet thro a router which I direct
port 22 (the ssh port) to the linux box. The everlasting sleep also
happens when I use a socket connection.

It work perfectly in the intranet. I presume it is the router that is
coursing the problem.

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

No virus found in this outgoing message.
Checked by AVG - www.avg.com 
Version: 9.0.862 / Virus Database: 271.1.1/3183 - Release Date: 10/08/10 02:34:00

Re: remote unison server cannot get out of sleep

On Thu, Oct 07, 2010 at 02:03:48PM +0800, kkksiu2000 wrote:
> When doing a large sync, I noticed that if the remote unison server
> first finished scanning all files and put to a sleep waiting for the
> local unison machine to finish its scan, the remote server cannot be
> waked up.
> 
> It ends up in a dead lock that both machines are waiting.
> 
> I am running unison version 2.32.52 on local XP and a remote SusE
> 11.3, sync via ssh
>
> Any idea ?

You should use either the ServerAliveInterval (on the client) or
the ClientAliveInterval (on the server) option of ssh to keep the TCP
connection alive.

-- Jerome

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/