Discussions forum for users of the Unison file synchronization tool

Dave Warren | 1 Mar 2012 02:33

Automatically clean up .unison.*.tmp files

Is there any way to have Unison automatically clean up it's own .tmp files?

I've got a ton of them getting left behind when a synchronization is 
interrupted, and they fulfill their purpose nicely. However, I'd love to 
get rid of them once a subsequent synchronization completes successfully.

Is there anything in unison that can help me or do I need to script a 
solution outside of unison?

-- 
Dave Warren, CEO
Hire A Hit Consulting Services
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:

(Continue reading)

headers

Benjamin C. Pierce | 1 Mar 2012 02:54

Re: Automatically clean up .unison.*.tmp files

Unison should clean up .tmp files after 30 days.

   - B

On Feb 29, 2012, at 8:33 PM, Dave Warren wrote:

> Is there any way to have Unison automatically clean up it's own .tmp files?
> 
> I've got a ton of them getting left behind when a synchronization is 
> interrupted, and they fulfill their purpose nicely. However, I'd love to 
> get rid of them once a subsequent synchronization completes successfully.
> 
> Is there anything in unison that can help me or do I need to script a 
> solution outside of unison?
> 
> -- 
> Dave Warren, CEO
> Hire A Hit Consulting Services
> http://www.hireahit.com/
> http://ca.linkedin.com/in/davejwarren
> 
> 
> 
> 
> ------------------------------------
> 
> Yahoo! Groups Links
> 
> 
>

(Continue reading)

headers

Dave Warren | 1 Mar 2012 03:11

Re: Automatically clean up .unison.*.tmp files

Is that configurable or is the 30 day window hardcoded?

In case there's a better approach, my underlying problem is that I have a nightly backup of our file server to an offsite location and I'm billed per MB (transferred and stored). We had a set of files that totaled 14GB (of 16GB) that a user accidentally synchronized to a shared folder on the file server, the user noticed and killed unison hoping to save us some backup charges. The user then deleted the files at the source (unaware of the .tmp files on the other end) and synchronized again, leaving 14GB worth of junk on the backup directory.

Ironically had the user not attempted to kill unison, they could have cleaned up after themselves, but they meant well.

I realize the utility of being able to resume from incomplete files, but could unison safely assume that once a subsequent sync completes successfully, the temp files can be removed?

Unfortunately I can't exclude these tmp files from the backup job in a uniform way due to limits in the backup software. I'm independently working on addressing that problem, but it's not looking like a quick fix will be forthcoming.

On 2/29/2012 5:54 PM, Benjamin C. Pierce wrote:

Unison should clean up .tmp files after 30 days.

- B

On Feb 29, 2012, at 8:33 PM, Dave Warren wrote:

> Is there any way to have Unison automatically clean up it's own .tmp files?
>
> I've got a ton of them getting left behind when a synchronization is
> interrupted, and they fulfill their purpose nicely. However, I'd love to
> get rid of them once a subsequent synchronization completes successfully.
>
> Is there anything in unison that can help me or do I need to script a
> solution outside of unison?

__._,_.___

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___

headers

Benjamin C. Pierce | 1 Mar 2012 04:48

Re: Automatically clean up .unison.*.tmp files

Is that configurable or is the 30 day window hardcoded?

It's hardcoded, but it wouldn't be hard to make it an option. Or to compile up a custom version for yourself with a smaller window. The relevant code is in the function childrenOf in os.ml.

In case there's a better approach, my underlying problem is that I have a nightly backup of our file server to an offsite location and I'm billed per MB (transferred and stored). We had a set of files that totaled 14GB (of 16GB) that a user accidentally synchronized to a shared folder on the file server, the user noticed and killed unison hoping to save us some backup charges. The user then deleted the files at the source (unaware of the .tmp files on the other end) and synchronized again, leaving 14GB worth of junk on the backup directory.

Ironically had the user not attempted to kill unison, they could have cleaned up after themselves, but they meant well.

I realize the utility of being able to resume from incomplete files, but could unison safely assume that once a subsequent sync completes successfully, the temp files can be removed?

I'm not 100% certain this would work -- we'd need to double-check exactly how the temp files are handled -- but it seems plausible. Would take a tiny bit of work, because at the moment the deletion of the temp files is happening at a very low level, disconnected from the synchronization logic, but no rocket science.

Not sure if this should be the default behavior, though -- there are lots of imaginable situations where deleting the temp files early would be a shame (e.g., I successfully synchronize just one path in a large directory tree, and suddenly all the temp files elsewhere in the tree get deleted).

Another simple thing to do would be to a dd a "-purgetempfiles" switch. Then whoever was running Unison could notice whether the previous run succeeded and run again with this switch.

- Benjamin

Unfortunately I can't exclude these tmp files from the backup job in a uniform way due to limits in the backup software. I'm independently working on addressing that problem, but it's not looking like a quick fix will be forthcoming.

On 2/29/2012 5:54 PM, Benjamin C. Pierce wrote:

Unison should clean up .tmp files after 30 days.

- B

On Feb 29, 2012, at 8:33 PM, Dave Warren wrote:

> Is there any way to have Unison automatically clean up it's own .tmp files?
>
> I've got a ton of them getting left behind when a synchronization is
> interrupted, and they fulfill their purpose nicely. However, I'd love to
> get rid of them once a subsequent synchronization completes successfully.
>
> Is there anything in unison that can help me or do I need to script a
> solution outside of unison?

__._,_.___

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___

headers

David J. Haines | 1 Mar 2012 21:39

~/unison.log created despite options to the contrary

Hi all,

I'm using unison 2.40.63 on an Arch Linux "client" and a Mac OS X 10.6
"server" via the following command line in a crontab:

/usr/bin/unison -batch -silent -log=false -logfile /home/dhaines/.unison/logfile -servercmd
/opt/local/bin/unison $HOME/Sync ssh://<OSX Host>/Sync

Oftentimes while syncing, I get the "Fatal Error Lost connection with the server"
error, which I understand is related to ssh timeouts. Though that is a
problem (and I'd gladly accept any help people are willing to give on
it), my biggest issue is that said Fatal Error keeps appearing in
"unison.log" in my $HOME, despite my use of the -silent, -log=false, AND
-logfile /home/dhaines/.unison/logfile options.

Really, I'm willing to put up with the Fatal Error business, but not
being able to direct the logfile elsewhere is driving me mad. Does
anyone have any suggestions?

Thanks,

-- 
David J. Haines
djhaines <at> gmx.com

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

headers

Data Control Systems - Mike Elkevizth | 1 Mar 2012 22:06

Re: ~/unison.log created despite options to the contrary

Sorry I can't answer your main question about the log file (actually I have a cron script that runs unison and then moves the log file (and renames it) after unison runs, if that is a suitable workaround), but for the ssh timeout, I use the following option that keeps that from happening: -sshargs '-o ServerAliveInterval=60'

Hope that helps.

Mike E.

On Thu, Mar 1, 2012 at 3:39 PM, David J. Haines <djhaines <at> gmx.com> wrote:

Hi all,

I'm using unison 2.40.63 on an Arch Linux "client" and a Mac OS X 10.6
"server" via the following command line in a crontab:

/usr/bin/unison -batch -silent -log=false -logfile /home/dhaines/.unison/logfile -servercmd /opt/local/bin/unison $HOME/Sync ssh://<OSX Host>/Sync

Oftentimes while syncing, I get the "Fatal Error Lost connection with the server"
error, which I understand is related to ssh timeouts. Though that is a
problem (and I'd gladly accept any help people are willing to give on
it), my biggest issue is that said Fatal Error keeps appearing in
"unison.log" in my $HOME, despite my use of the -silent, -log=false, AND
-logfile /home/dhaines/.unison/logfile options.

Really, I'm willing to put up with the Fatal Error business, but not
being able to direct the logfile elsewhere is driving me mad. Does
anyone have any suggestions?

Thanks,

--
David J. Haines
djhaines <at> gmx.com

__._,_.___

Your email settings: Individual Email|Traditional
Change settings via the Web (Yahoo! ID required)
Change settings via email: Switch delivery to Daily Digest | Switch to Fully Featured
Visit Your Group | Yahoo! Groups Terms of Use | Unsubscribe

__,_._,___

headers

Adrian Klaver | 1 Mar 2012 22:16

Re: ~/unison.log created despite options to the contrary

On Thursday, March 01, 2012 12:39:45 pm David J. Haines wrote:
> Hi all,
> 
> I'm using unison 2.40.63 on an Arch Linux "client" and a Mac OS X 10.6
> "server" via the following command line in a crontab:
> 
> /usr/bin/unison -batch -silent -log=false -logfile
> /home/dhaines/.unison/logfile -servercmd /opt/local/bin/unison $HOME/Sync
> ssh://<OSX Host>/Sync
> 
> Oftentimes while syncing, I get the "Fatal Error Lost connection with the
> server" error, which I understand is related to ssh timeouts. Though that
> is a problem (and I'd gladly accept any help people are willing to give on
> it), my biggest issue is that said Fatal Error keeps appearing in
> "unison.log" in my $HOME, despite my use of the -silent, -log=false, AND
> -logfile /home/dhaines/.unison/logfile options.
> 
> Really, I'm willing to put up with the Fatal Error business, but not
> being able to direct the logfile elsewhere is driving me mad. Does
> anyone have any suggestions?

Eliminate the -log=false . The -logfile switch should be enough to redirect your 
messages:

http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-
manual.html#prefs
"
logfile xxx
By default, logging messages will be appended to the file unison.log in your HOME 
directory. Set this preference if you prefer another file.
"

> 
> Thanks,

-- 
Adrian Klaver
adrian.klaver <at> gmail.com

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

headers

Alan Schmitt | 2 Mar 2012 10:03

Re: ~/unison.log created despite options to the contrary

On 1 mars 2012, at 22:16, Adrian Klaver wrote:

> Eliminate the -log=false . The -logfile switch should be enough to redirect your 
> messages:
> 
> http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-
> manual.html#prefs
> "
> logfile xxx
> By default, logging messages will be appended to the file unison.log in your HOME 
> directory. Set this preference if you prefer another file.
> "

You could also try: "-logfile /dev/null".

Alan

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

headers

Benjamin C. Pierce | 3 Mar 2012 16:40

Re: ~/unison.log created despite options to the contrary

I'm afraid I don't see what's wrong here.  It's true that the -log and -logfile preferences are set by default
to values that would cause this behavior *if* Unison crashed immediately after startup, but once the
options have been parsed they should be set to the values you asked for.

    - B

On Mar 2, 2012, at 4:03 AM, Alan Schmitt wrote:

> On 1 mars 2012, at 22:16, Adrian Klaver wrote:
> 
>> Eliminate the -log=false . The -logfile switch should be enough to redirect your 
>> messages:
>> 
>> http://www.cis.upenn.edu/~bcpierce/unison/download/releases/stable/unison-
>> manual.html#prefs
>> "
>> logfile xxx
>> By default, logging messages will be appended to the file unison.log in your HOME 
>> directory. Set this preference if you prefer another file.
>> "
> 
> You could also try: "-logfile /dev/null".
> 
> Alan
> 
> ------------------------------------
> 
> Yahoo! Groups Links
> 
> 
> 

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

headers

Benjamin C. Pierce | 3 Mar 2012 19:26

Re: ~/unison.log created despite options to the contrary

Nope.  The -log setting controls whether anything happens at all; the -logfile setting controls where it
happens, if so.

On Mar 3, 2012, at 1:10 PM, Adrian Klaver wrote:

> On Saturday, March 03, 2012 7:40:21 am Benjamin C. Pierce wrote:
>> I'm afraid I don't see what's wrong here.  It's true that the -log and
>> -logfile preferences are set by default to values that would cause this
>> behavior *if* Unison crashed immediately after startup, but once the
>> options have been parsed they should be set to the values you asked for.
> 
> So -log=false with  -logfile /home/dhaines/.unison/logfile would not be sending 
> mixed signals?
> 
>> 
>>    - B
> 
> 
> -- 
> Adrian Klaver
> adrian.klaver <at> gmail.com

------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/unison-users/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/unison-users/join
    (Yahoo! ID required)

<*> To change settings via email:
    unison-users-digest <at> yahoogroups.com 
    unison-users-fullfeatured <at> yahoogroups.com

<*> To unsubscribe from this group, send an email to:
    unison-users-unsubscribe <at> yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/

Mon	Tue	Wed	Thu	Fri	Sat	Sun

			1	2	3	4
5	6	7	8	9	10	11
12	13	14	15	16	17	18
19	20	21	22	23	24	25
26	27	28	29	30	31

7	May 2013
15	April 2013
33	March 2013
17	February 2013
28	January 2013
48	December 2012
42	November 2012
30	October 2012
124	September 2012
29	August 2012
28	July 2012
43	June 2012
89	May 2012
16	April 2012
38	March 2012
49	February 2012
102	January 2012
41	December 2011
30	November 2011
56	October 2011
44	September 2011
19	August 2011
57	July 2011
34	June 2011
85	May 2011
41	April 2011
43	March 2011
43	February 2011
68	January 2011
26	December 2010
54	November 2010
51	October 2010
47	September 2010
37	August 2010
47	July 2010
71	June 2010
105	May 2010
104	April 2010
74	March 2010
64	February 2010
135	January 2010
62	December 2009
46	November 2009
60	October 2009
39	September 2009
54	August 2009
79	July 2009
104	June 2009
126	May 2009
99	April 2009
179	March 2009
36	February 2009
62	January 2009
50	December 2008
71	November 2008
73	October 2008
62	September 2008
56	August 2008
91	July 2008
64	June 2008
118	May 2008
77	April 2008
103	March 2008
110	February 2008
79	January 2008
49	December 2007
58	November 2007
79	October 2007
86	September 2007
57	August 2007
42	July 2007
33	June 2007
49	May 2007
110	April 2007
121	March 2007
97	February 2007
130	January 2007
84	December 2006
81	November 2006
109	October 2006
100	September 2006
124	August 2006
84	July 2006
104	June 2006
136	May 2006
84	April 2006
97	March 2006
59	February 2006
97	January 2006
76	December 2005
76	November 2005
78	October 2005
89	September 2005
129	August 2005
119	July 2005
150	June 2005
246	May 2005
114	April 2005
124	March 2005
80	February 2005
80	January 2005
63	December 2004
158	November 2004
107	October 2004
201	September 2004
139	August 2004
70	July 2004
60	June 2004
77	May 2004
98	April 2004
94	March 2004
100	February 2004
96	January 2004
70	December 2003
73	November 2003
62	October 2003
65	September 2003
57	August 2003
90	July 2003
53	June 2003
58	May 2003
62	April 2003
89	March 2003
83	February 2003
58	January 2003
104	December 2002
83	November 2002
49	October 2002
40	September 2002
82	August 2002
48	July 2002
40	June 2002
11	May 2002
30	April 2002