evencoil evencoil | 1 Apr 16:19 2012
Picon

Large replicas

I am trying to sync a directory and its subdirectories that is both large in file size and has many files.

There is already a copy of this directory on both machines. I want to use Unison to sync future changes, which will be small in both file size and number of files.

My problem is that when I try to do the initial scan in Unison (2.27.57) I invariably get

"Write failed: Broken pipe
Fatal error: Lost connection with the server"

after a couple of hours. Is this an rsync problem? If so, could someone recommend additional commands that would keep the rsync connection alive? I have no issues with this procedure taking a long time, I just want it to actually finish.

Thanks!



__._,_.___

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___
Denis Howe | 1 Apr 20:25 2012
Picon

Re: Large replicas

On 2012-04-01 15:19, Even Coil <evencoil <at> gmail.com> wrote:

when I try to do the initial scan in Unison (2.27.57) I invariably get "Write failed: Broken pipe  Fatal error: Lost connection with the server" 

We had a similar problem with VPN that would time out and drop the connection.  If you just leave a remote shell (e.g. ssh or PuTTY) open, does the connection drop?

Googling the error suggests it's an ssh thing.   http://ocaoimh.ie/2008/12/10/how-to-fix-ssh-timeout-problems/  describes the server and client config changes for ssh time-outs.


__._,_.___

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___
lukshuntim | 3 Apr 15:40 2012
Picon

Re: Large replicas

On 04/02/2012 02:25 AM, Denis Howe wrote:
>  
> 
> On 2012-04-01 15:19, Even Coil <evencoil <at> gmail.com
> <mailto:evencoil <at> gmail.com>> wrote:
> 
>     when I try to do the initial scan in Unison (2.27.57) I invariably
>     get "Write failed: Broken pipe  Fatal error: Lost connection with
>     the server" 

The current version is 2.40. Is there any special reason why you need
version 2.27? It was already a few years old. The newer versions are in
general faster and more robust.

Regards,
ST
-- 

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

Luca Borrione | 3 Apr 18:28 2012
Picon

How to tell unison to first delete?


Hello,

I use unison to synchronize 2 usb drives.
As unison first adds new files and then performs deletion of removed files,
this can cause me some trouble if I don't have enough room to accept new files
without removing some first.
So is there a way to tell unison to first delete removed files and only after that
go on with adding new files or copying changed ones?

Thank you in advance
-- 
Luca Borrione <write <at> tiscali.it>

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

Pierce Benjamin C. | 5 Apr 23:07 2012

Unison 2.45 beta-release now available

[This release is mainly small fixes and GUI improvements.  Users that are happy with their current Unison
installation probably don't need to bother updating.]

Download address:
   http://www.cis.upenn.edu/~bcpierce/unison/download.html

Changes since 2.40.63:
  * New preference fastercheckUNSAFE, which can be used (with care!)
    to achieve much faster update detection when all the common files
    in the two replicas are known to be identical. See the manual for
    more information.
    This feature should still be considered experimental, but it's
    ready for other people to try out.
  * Added option clientHostName. If specified, it will be used to as
    the client host name, overriding UNISONLOCALHOSTNAME and the
    actual host name.
  * OS X GUI:
       + fix crash under Lion, because of problems with the toolbar,
         using the fix suggested in
         http://blitzbasic.com/Community/posts.php?topic=95778.
       + uimacnew09 is now the standard graphical interface on OSX
       + A small improvement to the uimacnew09 interface from Alan
         Schmitt and Steve Kalkwarf: when Unison is run with the
         -batch flag, the interface will now automatically propagate
         changes and terminate, without waiting for user interaction.
       + Show a modal warning window if there is no archive for the
         hosts. The user can then choose to exit or proceed (proceed
         is the default). The window is not shown if the batch
         preference is true.
       + file details panel selectable
  * GTK GUI:
       + New version of uigtk2.ml from Matt Zagrabelny that
         reorganizes the icons in a slightly more intuitive way.
  * Minor fixes:
       + Setting the prefer preference to older or newer now
         propagates deletions when there is no conflict.
       + Correctly quote the path when running merge commands.
       + Add quotes to paths when calling external file watcher
         utility.
       + Incorporate a patch to fsmonitor.py (the external filewatcher
         utility) from Tomasz Zernicki to make it work better under
         Windows.
       + Incorporated new version of fsmonitor.py from Christophe
         Gohle
       + Fixed incompatibility with OpenSSH 5.6.
       + Fixed fingerprint cache: do not cache file properties
       + Some spelling corrections in documentation and comments from
         Stephane Glondu
       + Fixed O_APPEND mode for open under Windows
       + Fixed String.sub invalid argument error when an AppleDouble
         file does not contain a finder information field
       + Trim duplicate paths when using "-repeat watch"
       + Unison now passes path arguments and -follow directives to
         fsmonitor.py. This seems to work except for one small issue
         with how fsmonitor.py treats -follow directives for
         directories that don't exist (or maybe this is an issue with
         how it treats any kind of monitoring when the thing being
         monitored doesn't exist?). If we create a symlink to a
         nonexistant directory, give Unison (hence fsmonitor.py) a
         'follow' directive for the symlink, start unison, and then
         create the directory, fsmonitor.py misses the change.
       + Lines added in profile files by unison always start at a new
         line

Changes since 2.40.1:
  * Added "BelowPath" patterns, that match a path as well as all paths
    below (convenient to use with nodeletion,update,creationpartial
    preferences)
  * Added a "fat" preference that makes Unison use the right options
    when one of the replica is on a FAT filesystem.
  * Allow "prefer/force=newer" even when not synchronizing
    modification times. (The reconciler will not be aware of the
    modification time of unchanged files, so the synchronization
    choices of Unison can be different from when "times=true", but the
    behavior remains sane: changed files with the most recent
    modification time will be propagated.)
  * Minor fixes and improvements:
       + Compare filenames up to decomposition in case sensitive mode
         when one host is running MacOSX and the unicode preference is
         set to true.
       + Rsync: somewhat faster compressor
       + Make Unicode the default on all architectures (it was only
         the default when a Mac OS X or Windows machine was involved).

Changes since 2.32:
  * Major enhancement: Unicode support.
       + Unison should now handle unicode filenames correctly on all
         platforms.
       + This functionality is controlled by a new preference unicode.
       + Unicode mode is now the default when one of the hosts is
         under Windows or MacOS. This may make upgrades a bit more
         painful (the archives cannot be reused), but this is a much
         saner default.
  * Partial transfer of directories. If an error occurs while
    transferring a directory, the part transferred so far is copied
    into place (and the archives are updated accordingly). The
    "maxerrors" preference controls how many transfer error Unison
    will accept before stopping the transfer of a directory (by
    default, only one). This makes it possible to transfer most of a
    directory even if there are some errors. Currently, only the first
    error is reported by the GUIs.
    Also, allow partial transfer of a directory when there was an
    error deep inside this directory during update detection. At the
    moment, this is only activated with the text and GTK UIs, which
    have been modified so that they show that the transfer is going to
    be partial and so that they can display all errors.
  * Improvement to the code for resuming directory transfers:
       + if a file was not correctly transferred (or the source has
         been modified since, with unchanged size), Unison performs a
         new transfer rather than failing
       + spurious files are deleted (this can happen if a file is
         deleted on the source replica before resuming the transfer;
         not deleting the file would result in it reappearing on the
         target replica)
  * Experimental streaming protocol for transferring file contents
    (can be disabled by setting the directive "stream" to false): file
    contents is transfered asynchronously (without waiting for a
    response from the destination after each chunk sent) rather than
    using the synchronous RPC mechanism. As a consequence:
       + Unison now transfers the contents of a single file at a time
         (Unison used to transfer several contents simultaneously in
         order to hide the connection latency.)
       + the transfer of large files uses the full available bandwidth
         and is not slowed done due to the connection latency anymore
       + we get performance improvement for small files as well by
         scheduling many files simultaneously (as scheduling a file
         for transfer consume little ressource: it does not mean
         allocating a large buffer anymore)
  * Changes to the internal implementation of the rsync algorithm:
       + use longer blocks for large files (the size of a block is the
         square root of the size of the file for large files);
       + transmit less checksum information per block (we still have
         less than one chance in a hundred million of transferring a
         file incorrectly, and Unison will catch any transfer error
         when fingerprinting the whole file)
       + avoid transfer overhead (which was 4 bytes per block)
    For a 1G file, the first optimization saves a factor 50 on the
    amount of data transferred from the target to the source (blocks
    are 32768 bytes rather than just 700 bytes). The two other
    optimizations save another factor of 2 (from 24 bytes per block
    down to 10).
  * Implemented an on-disk file fingerprint cache to speed-up update
    detection after a crash: this way, Unison does not have do
    recompute all the file fingerprints from scratch.
       + When Unison detects that the archive case-sensitivity mode
         does not match the current settings, it populates the
         fingerprint cache using the archive contents. This way,
         changing the case-sensitivity mode should be reasonably fast.
  * New preferences "noupdate=root", "nodeletion=root",
    "nocreation=root" that prevent Unison from performing files
    updates, deletions or creations on the given root. Also 'partial'
    versions of 'noupdate', 'nodeletion' and 'nocreation'
  * Limit the number of simultaneous external copy program ("copymax"
    preference)
  * New "links" preference. When set to false, Unison will report an
    error on symlinks during update detection. (This is the default
    when one host is running Windows but not Cygwin.) This is better
    than failing during propagation.
  * Added a preference "halfduplex" to force half-duplex communication
    with the server. This may be useful on unreliable links (as a more
    efficient alternative to "maxthreads = 1").
  * Renamed preference "pretendwin" to "ignoreinodenumbers" (an alias
    is kept for backwards compatibility).
  * Ignore one-second differences when synchronizing modification
    time. (Technically, this is an incompatible archive format change,
    but it is backward compatible. To trigger a problem, a user would
    have to synchronize modification times on a filesystem with a
    two-second granularity and then downgrade to a previous version of
    Unison, which does not work well in such a case. Thus, it does not
    seem worthwhile to increment the archive format number, which
    would impact all users.)
  * Do not keep many files simultaneously opened anymore when the
    rsync algorithm is in use.
  * Add "ignorearchives" preference to ignore existing archives (to
    avoid forcing users to delete them manually, in situations where
    one archive has gotten deleted or corrupted).
  * Mac OS
       + fixed rsync bug which could result in an "index out of
         bounds" error when transferring resource forks.
       + Fixed bug which made Unison ignore finder information and
         resource fork when compiled to 64bit on Mac OSX.
       + should now be 64 bit clean (the Growl framework is not up to
         date, though)
       + Made the bridge between Objective C and Ocaml code GC
         friendly (it was allocating ML values and putting them in an
         array which was not registered with the GC)
       + use darker grey arrows (patch contributed by Eric Y. Kow)
  * GTK user interface
       + assistant for creating profiles
       + profile editor
       + pop up a summary window when the replicas are not fully
         synchronized after transport
       + display estimated remaining time and transfer rate on the
         progress bar
       + allow simultaneous selection of several items
       + Do not reload the preference file before a new update
         detection if it is unchanged
       + disabled scrolling to the first unfinished item during
         transport. It goes way too fast when lot of small files are
         synchronized, and it makes it impossible to browse the file
         list during transport.
       + take into account the "height" preference again
       + the internal list of selected reconciler item was not always
         in sync with what was displayed (GTK bug?); workaround
         implemented
       + Do not display "Looking for change" messages during
         propagation (when checking the targe is unchanged) but only
         during update detection
       + Apply patch to fix some crashes in the OSX GUI, thanks to
         Onne Gorter.
  * Text UI
       + During update detection, display status by updating a single
         line rather than generating a new line of output every so
         often. Should be less confusing.
  * Windows
       + Fastcheck is now the default under Windows. People mostly use
         NTFS nowadays and the Unicode API provides an equivalent to
         inode numbers for this filesystem.
       + Only use long UNC path for accessing replicas (as '..' is not
         handled with this format of paths, but can be useful)
       + Windows text UI: now put the console into UTF-8 output mode.
         This is the right thing to do when in Unicode mode, and is no
         worse than what we had previously otherwise (the console use
         some esoteric encoding by default). This only works when
         using a Unicode font instead of the default raster font.
       + Don't get the home directory from environment variable HOME
         under Windows (except for Cygwin binaries): we don't want the
         behavior of Unison to depends on whether it is run from a
         Cygwin shell (where HOME is set) or in any other way (where
         HOME is usually not set).
  * Miscellaneous fixes and improvements
       + Made a server waiting on a socket more resilient to
         unexpected lost connections from the client.
       + Small patch to property setting code suggested by Ulrich
         Gernkow.
       + Several fixes to the change transfer functions (both the
         internal ones and external transfers using rsync). In
         particular, limit the number of simultaneous transfer using
         an rsync (as the rsync algorithm can use a large amount of
         memory when processing huge files)
       + Keep track of which file contents are being transferred, and
         delay the transfer of a file when another file with the same
         contents is currently being transferred. This way, the second
         transfer can be skipped and replaced by a local copy.
       + Experimental update detection optimization: do not read the
         contents of unchanged directories
       + When a file transfer fails, turn off fastcheck for this file
         on the next sync.
       + Fixed bug with case insensitive mode on a case sensitive
         filesystem:
            o if file "a/a" is created on one replica and directory
              "A" is created on the other, the file failed to be
              synchronized the first time Unison is run afterwards, as
              Unison uses the wrong path "a/a" (if Unison is run
              again, the directories are in the archive, so the right
              path is used);
            o if file "a" appears on one replica and file "A" appears
              on the other with different contents, Unison was unable
              to synchronize them.
       + Improved error reporting when the destination is updated
         during synchronization: Unison now tells which file has been
         updated, and how.
       + Limit the length of temporary file names
       + Case sensitivity information put in the archive (in a
         backward compatible way) and checked when the archive is
         loaded
       + Got rid of the 16mb marshalling limit by marshalling to a
         bigarray.
       + Resume copy of partially transferred files.

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-announce/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-announce/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-announce-digest <at> yahoogroups.com 
    unison-announce-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-announce-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

Eduard Braun | 9 Apr 14:05 2012
Picon
Picon

"Change Profile..." doesn't work

Hi all,

I've got a problem when changing profiles with "Change Profile..." 
button in Unison 2.40.63 GTK+ version for Windows.

I have two profiles using a socket connection to the same server 
(running Windows, too). When changing between these two profiles unison 
hangs (status shows "Contacting server...").
Changing from or to a local profile works without a problem as well as 
changing between local profiles works. Also choosing the same server 
profile again works without a problem.
Choosing one of the server profiles, switching to a local profile and 
then choosing the other server profile results in a hang, too.

So it seems as soon as two different server profiles are involved unison 
hangs. I assume it's a server bug then? Is the problem known? Are there 
any steps I can follow to debug the problem?

In the meantime I'm closing the Unison client (not the server!) and 
restart it as a workaround. I can then choose the other profile without 
problems. But that's probably not "as designed". ;-)

Hope you can help me solve my problem!

Regards
Eduard

P.S. my profiles lock like this:

     root = C:\folder1
     root = socket://192.168.178.20:123123/folder1

     times = true
     fastcheck = true
     rsync = true

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

jorduna | 12 Apr 14:51 2012
Picon

Windows environment variables in Profiles / Unison


This might well be a very basic question.. or even one that has already been responded several times...

I am using an external hard drive to synchronize several folders in two Windows XP machines.

I am using the cl unison, not the gui one

Ideally I woudl want to use a bat file that calls unison with a profile. This profile includes the roots,
paths etc.

My problem is that I would nee to use the %USERPROFILE% when stteing the roots, as users in both machines are different.

Can I use environment variables in PROFILES? I haven't succeded so far... Is there any other way of doing this?

Thx in advance for your help!

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

Dave Warren | 14 Apr 00:08 2012

Re: Windows environment variables in Profiles / Unison

On 4/12/2012 5:51 AM, jorduna wrote:
 


This might well be a very basic question.. or even one that has already been responded several times...

I am using an external hard drive to synchronize several folders in two Windows XP machines.

I am using the cl unison, not the gui one

Ideally I woudl want to use a bat file that calls unison with a profile. This profile includes the roots, paths etc.

My problem is that I would nee to use the %USERPROFILE% when stteing the roots, as users in both machines are different.

Can I use environment variables in PROFILES? I haven't succeded so far... Is there any other way of doing this?

Thx in advance for your help!



You mention that you're calling unison from a batch file, could you set those particular parameters from the command line (which does support variables) rather than from within the profile itself?

Or if you're not using a batchfile and calling unison directly from a shortcut, you can add a "%systemroot%\system32\cmd.exe /c " in front of the command line, cmd.exe will expand any variables on the command line, bypassing the need for a 1-line batch file.

Another, more complicated-but-possibly-more-flexible, option would be to create a batchfile that uses ECHO statements in the batch file to construct %USERNAME%.prf (by using ECHO, cmd.exe would take care of variables for you), using the INCLUDE directive to include in the other profile which contains the primary settings (or vise-versa, include in the constructed .prf from the main one, depending on what fits)

Both approaches are a bit roundabout, but will get the job done and will let you tweak on a per-machine or per-environment basis.
-- Dave Warren http://www.hireahit.com/ http://ca.linkedin.com/in/davejwarren

__._,_.___

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___
wbyeats74 | 17 Apr 08:40 2012

Synching two different USB drives not connected simultaneously

I'm trying to find a software that can handle my requirement. I have a notebook that I use at work and Home. I
have an external usb drive here at work and another one at home. I would like to find a software that
synchronize those two drives via my notebook.

This thing should be done in different ways but what I expect should be a behavior like this one:

I get to work and use the external HD. Meanwhile in the notebook (or remotely via a NAS or FTP server) it tracks
down every file I add or modify or delete. When I get home with my laptop and connect to the other drive the
software should recognize the drive and sync it with the info the notebook collected. Then it starts
collecting infos on the change I make on the Home HD that will be synched to the Work HD whenever I reconnect
to it. And so on

Is there any chance to configure Unison to work this way? How should I do this?

Thanks a lot in advance for your reply.

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

Alan Schmitt | 17 Apr 09:36 2012

Re: Synching two different USB drives not connected simultaneously

On 17 avr. 2012, at 08:40, wbyeats74 wrote:

> I'm trying to find a software that can handle my requirement. I have a notebook that I use at work and Home. I
have an external usb drive here at work and another one at home. I would like to find a software that
synchronize those two drives via my notebook.
> 
> This thing should be done in different ways but what I expect should be a behavior like this one:
> 
> I get to work and use the external HD. Meanwhile in the notebook (or remotely via a NAS or FTP server) it
tracks down every file I add or modify or delete. When I get home with my laptop and connect to the other drive
the software should recognize the drive and sync it with the info the notebook collected. Then it starts
collecting infos on the change I make on the Home HD that will be synched to the Work HD whenever I reconnect
to it. And so on
> 
> Is there any chance to configure Unison to work this way? How should I do this?

If I read between the lines, you don't want the files on the external HD to be duplicated in the notebook, you
only want the changed files to be there, so that they can be carried to the other HD. Is this correct?

If so, I'm afraid it goes beyond what Unison can currently do.

However, if you are ready to also have a copy of the files on the notebook, then it's simply a matter of setting
up two profiles: one to synchronize between notebook and external HD 1, and another between notebook and
external HD 2.

Alan

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/


Gmane