headers
Dmitry Bely | 2 Jul 2002 15:02
Picon

Re: Unison 2.9.1/Windows/GTK: memory leak?

"Benjamin C. Pierce" <bcpierce <at> saul.cis.upenn.edu> writes:

>> When syncronizing the large amount of data, unison consumes more and more
>> memory and finally dies in swapping. I had seen 600+ Mbytes of memory
>> commited by unison when syncronizing two replicas containing several
>> thousands files.
>> 
>> Probably, the problem is not directly Unison-related (GTK problems?) but
>> nonetheless it makes using Unison problematic for me. Have anybody else
>> seen the same problem? Is it known to Unison developers?
>
> There are situations where Unison can use a lot of memory (e.g. when
> transferring very large files).  A workaround is to synchronize in
> smaller pieces the first time.

No, that is not the case. I have no large files (more than several
megabytes), but Unison constantly consumes more and more memory when
syncronizing the large amount of files. Moreover, the memory is taken even
when only the properties are syncronized (100Kb per file or so). Just open
Windows Task Manager, and you will see. It is interesting that the text
version don't seem to suffer from that problem.

Hope to hear from you soon,
Dmitry

To unsubscribe from this group, send an email to:
unison-users-unsubscribe <at> onelist.com

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
(Continue reading)

Permalink | Reply | 2 comments |
headers
Benjamin C. Pierce | 2 Jul 2002 15:59
Favicon

Re: Unison 2.9.1/Windows/GTK: memory leak?

> > There are situations where Unison can use a lot of memory (e.g. when
> > transferring very large files).  A workaround is to synchronize in
> > smaller pieces the first time.
> 
> No, that is not the case. I have no large files (more than several
> megabytes), but Unison constantly consumes more and more memory when
> syncronizing the large amount of files. Moreover, the memory is taken even
> when only the properties are syncronized (100Kb per file or so). Just open
> Windows Task Manager, and you will see. It is interesting that the text
> version don't seem to suffer from that problem.

100Kb per file sounds *way* too much.  Did you get this by running Unison
on various numbers of files and fitting a line to the results?  Has
anyone else observed memory behavior this bad?

     B

To unsubscribe from this group, send an email to:
unison-users-unsubscribe <at> onelist.com

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
Permalink | Reply | 1 comment |
headers
Dmitry Bely | 2 Jul 2002 17:48
Picon

Re: Unison 2.9.1/Windows/GTK: memory leak?

"Benjamin C. Pierce" <bcpierce <at> cis.upenn.edu> writes:

>> > There are situations where Unison can use a lot of memory (e.g. when
>> > transferring very large files).  A workaround is to synchronize in
>> > smaller pieces the first time.
>> 
>> No, that is not the case. I have no large files (more than several
>> megabytes), but Unison constantly consumes more and more memory when
>> syncronizing the large amount of files. Moreover, the memory is taken even
>> when only the properties are syncronized (100Kb per file or so). Just open
>> Windows Task Manager, and you will see. It is interesting that the text
>> version don't seem to suffer from that problem.
>
> 100Kb per file sounds *way* too much.  Did you get this by running Unison
> on various numbers of files and fitting a line to the results?

It was just a very rough estimation. OK, let's get the exact figures:

1. Populate two local directories with 5000 empty files each using
the following script:

[populate.pl]

foreach my $f (1..5000)
{
  open F, ">$f" or die;
  print $f, "\r";
  close F;
}
(Continue reading)

Permalink | Reply | 0 comments |
headers
Benjamin C. Pierce | 2 Jul 2002 19:19
Favicon

Re: Unison 2.9.1/Windows/GTK: memory leak?

Since you've got the infrastructure in place, could you repeat the test
with just a single file instead of 5000?  I'd like to see what the fixed
overhead is.

Another interesting thing to check would be how much of that memory is
actually uncollected garbage.  If you start unison with the flag
'-expert', you'll get a menu in the UI containing a command "Show
memory/GC stats."  If you do this at the end of your run, it will show
the current memory usage (should be close to agreeing with the task
manager), then do a GC, then show you again.

     B

To unsubscribe from this group, send an email to:
unison-users-unsubscribe <at> onelist.com

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
Permalink | Reply | 5 comments |
headers
will | 3 Jul 2002 19:39
Favicon
Gravatar

unison on win32


I'm running on Windows 2000 with SSH from Cygwin.  I've been able to sync
from one Linux machine to another without any problems, but when I sync
from my Windows 2000 machine to my Linux server, I get the following
message:

"Received unexpected header from server: expected "Unison 2.9.1\n" but 
received "Unison 2.9.1\013".  This is probably because you have different 
versions of Unison installed on the client and server machines."

Cygwin allows you to set your file types (Unix or Dos) which changes the 
end of line sequence.  I thought that changing this might fix the problem, 
but that didn't seem to affect things at all.  I'm running the latest SSH 
from Cygwin (OpenSSH_3.4p1, SSH protocols 1.5/2.0 OpenSSL 0x0090604f).

I get the same error with the textui that I get with the gtkui and I'm
running the current stable version of Unison (2.9.1).

Any ideas as to what I could do to fix this?
/will

--

-- 
whatever it is, you can find it at http://www.bluesock.org/~willg/
except Will--you can only see him in real life.

To unsubscribe from this group, send an email to:
unison-users-unsubscribe <at> onelist.com

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
(Continue reading)

Permalink | Reply | 0 comments |
headers
Brian Johnson | 3 Jul 2002 17:42
Picon

Merging files

I'm looking for a Windows and Linux syncronization utility to sync files to
and from my laptop (currently dual boot but looking to switch fully to
Linux) and this looks like a real possibility but I have three questions:

1. does it just check what file is newest and copy it over the older one or
does it delve into the data in the files and allow an update to both files
if they have both changed
2. is there a list of types of data files it will support, or does it just
handle text files (how about OpenOffice data files)
3. will it work with an online database like MySQL or PostgreSQL (I need
some way to get read/write access on my laptop to contact info normally
stored in a databse on the LAN - currently through MS Briefcase and MS
Access but MS Access is getting too slow)

To unsubscribe from this group, send an email to:
unison-users-unsubscribe <at> onelist.com

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
Permalink | Reply | 1 comment |
headers
dnb | 3 Jul 2002 23:49
Favicon

Re: unison on win32

Howdy-

  I can confirm there is a problem, but my experiments show it is not
  a Unison problem. As far as I can tell, the latest cygwin or ssh
  cygwin port (I haven't determined which) now does end-of-line
  translation between win32 and unix (when launched from a command
  prompt) where it did not do this before before.

  Here's how to demonstrate this phenomena:

  ssh user <at> someunixhost ls | od -c |tail -2 

  on a UNIX box gives this result: 

  0001340   g   z  \n

  the same command on the newest cygwin/ssh port to the same unixhost
  (when run from a command window):

  0001440   g   z  \r  \n

  And in fact if you perform the ls without the od and tail, you'll
  see that you get back a response with a spurious linefeed. There's
  an EOL translation going on here that shouldn't be (as far as I can
  tell).

  Now, here's where it gets interesting, running the same command from
  a cygwin bash window:

  0001340   g   z  \n
(Continue reading)

Permalink | Reply | 0 comments |
headers
paired2000 | 4 Jul 2002 11:36
Picon
Favicon

problem with unison when using it via RAS

I want to use a socket unison via a RAS (Remote Access Service).

On a PC computer (NT4 SP6) used as server, i starts a unison socket 
(cmd "unison -socket xxxx"). The message "server started" appears. 
This computer has a modem connected and a RAS configured as server.

On an other computer (NT4 SP6) used as client, i use the cmd "unison 
directory_1 socket://server_name:xxxx/directory_2 -force 
socket://server_name:xxxx/directory_2 -batch". This computer has a 
modem connected and a RAS configured as client.

When i executes the command here above, the client modem connect the 
server one and the exchange begin.

with the unison 2.6.11 :
           - On the client PC computer, the Error Message appears 
twice:
                "Fatal Error : Lost Connection to remote server"
           - On the server one, The Error Message appears :
                "Fatal Error : Error in dumping : Invalid argument
                               [write()]

with unison 2.9.1 :
           - On the client, no Error message appears.
           - On the server, Error Message appears :
               "Fatal Error: Error in waiting on port :
                 The specified network name is no longer available
                 [read()]"

I try to do the same thing with two PC on the same network with the
(Continue reading)

Permalink | Reply | 0 comments |
headers
James Leifer | 4 Jul 2002 14:21
Picon
Picon
Favicon

suggestions for 2.9.1

Dear friends of Unison,

I just upgraded from 2.7.7 to 2.9.1.  I'm very happy about the upgrade:
the new version (with the multithreaded transfer system) is *fast*!
Well done!

I had a couple of pedantic comments/requests, which I enclose. These
result from playing with

      unison.linux-static-gtkui-2.9.1

Warmest regards,

James Leifer
INRIA Rocquencourt

=====================================================================

* [graphic ui, bug] If one of the files "has unknown type" (i.e. is a
  system file), then pressing "f" (i.e. "Retry on unsynchronised items")
  results in an error window and unison quiting.  To me "Retry" implies
  less drastic behaviour.  It should just report errors as normal.
  Minor problem: there's a missing space before "has unknown type" in
  the info bar at the bottom and in the error box.

* [graphic ui, wishlist] The documentation topics aren't searchable.  As
  a result "unison -doc running | less" is still indispensable if one
  wants to find anything.  I suggest adding a box

     "search in this topic: ---"
(Continue reading)

Permalink | Reply | 0 comments |
headers
Benjamin C. Pierce | 4 Jul 2002 14:47
Favicon

Re: Merging files

> I'm looking for a Windows and Linux syncronization utility to sync files to
> and from my laptop (currently dual boot but looking to switch fully to
> Linux) and this looks like a real possibility but I have three questions:
> 
> 1. does it just check what file is newest and copy it over the older one or
> does it delve into the data in the files and allow an update to both files
> if they have both changed
> 2. is there a list of types of data files it will support, or does it just
> handle text files (how about OpenOffice data files)
> 3. will it work with an online database like MySQL or PostgreSQL (I need
> some way to get read/write access on my laptop to contact info normally
> stored in a databse on the LAN - currently through MS Briefcase and MS
> Access but MS Access is getting too slow)

Unison treats individual files atomically -- it just copies the whole
thing.  

We're hard at work now on a new tool -- dubbed Harmony -- that will
understand and synchronize the internal structure of files

    http://www.cis.upenn.edu/~bcpierce/harmony

but don't hold your breath: it's going to take a while to get something
working. 

     Benjamin
Permalink | Reply | 0 comments |

Gmane