headers
Matt McCutchen | 1 Dec 2006 04:55
Picon

Re: File fragmentation

On 9/5/06, Matt McCutchen <hashproduct+rsync <at> gmail.com> wrote:
> There are, however, two things about the preallocate patch that I mean
> to revisit:
> (1) Rsync should preallocate locally copied files (e.g., due to
> --copy-dest) as well as transferred ones.
> (2) posix_fallocate actually extends the file with logical zeros.  If
> for some reason the file ends up being shorter than rsync expected,
> rsync needs to truncate the file to the new size.  I know this
> situation could arise if a file shrinks while it is being locally
> copied; I don't know whether it can arise if a source file shrinks
> while the sender is transferring it.

I have improved the preallocation patch to handle #1 and #2.  The
improved version is attached.  Please test and comment!  Wayne, please
consider adding the patch to patches/ of the rsync source code.

On 8/26/06, Wayne Davison <wayned <at> samba.org> wrote:
> I'm wondering if this will slow down rsync.  Since the function changes
> the size of the file before rsync writes out the data, does this result
> in extra disk I/O, especially for large files?  We'd probably need to
> test both Windows systems and Linux/unix systems separately and possibly
> conditionalize the code (it it does not slow things down somewhere) or
> make it a configure (command-line?) option (if someone wants to pay the
> price for reduced fragmentation).

I did a simple test with a 100MB file on Linux, and preallocation
indeed seemed to slow rsync down.  I looked at the strace, and
posix_fallocate appears to be implemented by writing a zero byte into
each needed disk block using pwrite, forcing it to be allocated.
Yuck!  I guess it works though.
(Continue reading)

Permalink | Reply | 7 comments |
headers
Paul Slootman | 1 Dec 2006 16:20
Picon
Favicon

Re: Bug#401189: rsync: using rltvz options spews sys_acl_get_file messages

Hi,
I got the following bug report on the Debian rsync which has the ACL
patch applied:

On Fri 01 Dec 2006, Bryan Gartner wrote:
> 
> When using the rltvz options to mirror files from a remote server, these
> kinds of issues are reported:
> 
>     default_perms_for_dir: sys_acl_get_file(wallpaper, SMB_ACL_TYPE_DEFAULT):
>     Function not implemented, falling back on umask
> 
> This is true for both 2.6.9-1 and 2.6.9-2 versions on this (hppa) platform,
> but doesn't seem to exist on i386 versions.
> 
> A workaround is simply to use "avz" cmd-line options, but this issue is
> new to these versions of the packages (as 2.6.4-6 on sarge-hppa worked fine).
> 
> Some mention of this issue resides on the rsync mailing lists.

I'm wondering whether hppa behaves differently in the face of no ACL
support, so that it gives errno == ENOSYS instead of errno == ENOTSUP.
I've come up with the following quick patch, but have no idea whether
that is the right thing to do in these circumstances:

--- acls.c.orig	2006-12-01 16:16:07.000000000 +0100
+++ acls.c	2006-12-01 16:16:43.000000000 +0100
 <at>  <at>  -515,7 +515,7  <at>  <at> 
 				free_acl(sxp);
 				return -1;
(Continue reading)

Permalink | Reply | 1 comment |
headers
Wayne Davison | 1 Dec 2006 19:04
Picon
Favicon

Re: rsync error: error starting client-server protocol

On Tue, Nov 28, 2006 at 03:00:03PM +0100, rsync.20.maillinglist <at> spamgourmet.com wrote:
> 2006/11/28 14:52:36 [26584] rsync denied on module home from localhost
> (127.0.0.1)

This means that the host was not authorized (2.6.6 would have displayed
an auth error if the authentication had failed).  I'd suggest editing
your rsyncd.conf file to temporarily remove the hosts allow line and all
other access restrictions.  If that doesn't allow you to connect then
you are probably editing the wrong config file for the daemon.  If that
does let you in, add back various auth rules to see what stops you from
connecting.

..wayne..
--

-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 1 comment |
headers
Wayne Davison | 1 Dec 2006 19:40
Picon
Favicon

Re: File fragmentation

On Thu, Nov 30, 2006 at 10:55:24PM -0500, Matt McCutchen wrote:
> Wayne, please consider adding the patch to patches/ of the rsync
> source code.

Sure -- looks good!  I've added it to CVS.

..wayne..
--

-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 1 comment |
headers
Wayne Davison | 1 Dec 2006 19:43
Picon
Favicon

Re: Bug#401189: rsync: using rltvz options spews sys_acl_get_file messages

On Fri, Dec 01, 2006 at 04:20:36PM +0100, Paul Slootman wrote:
> I'm wondering whether hppa behaves differently in the face of no ACL
> support, so that it gives errno == ENOSYS instead of errno == ENOTSUP.
> I've come up with the following quick patch

I don't have an hppa system to test on, but your changes look quite
reasonable to me.

..wayne..
--

-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 0 comments |
headers
Carles Pina i Estany | 1 Dec 2006 18:14
Gravatar

Re: Rsync and DTrace


Hi,

On Nov/30/2006, Matt McCutchen wrote:
> On 11/30/06, diversos <diversos <at> posix.brte.com.br> wrote:
> > The performance was terrible. After some tests with raid0, raid1, 
> > non-SATA discs, and using DTrace, i have realized that the problem was 
> > the "gnome-terminal". Basically the "-v" switch.
> 
> I've noticed something similar on my Linux machine.  If I run "rsync
> -v" at a virtual terminal (not through X) to back up my hard disk to a
> locally attached external disk, rsync runs considerably faster when I
> select a virtual terminal other than the one to which it is printing
> its messages.  I don't know what is going on here but I agree with
> Carles that it probably isn't specific to rsync.

for example:
#include <stdio.h>

int main () {
        int i;

        for (i=0;i<30000;i++) {
                printf("This is a test. How it takes to print 30000 times a line?\n");
        }
        return 0;
}

gcc test.c
time ./a.out: (in a xterm)
(Continue reading)

Permalink | Reply | 0 comments |
headers
Linda Walsh | 3 Dec 2006 01:38

File Fragmentation issue...especially relating to NTFS...

I'm very sorry for being late to the party on this note, but I'm
not sure what the original trigger for this fragmentation problem
was... I "thought" (imagining?) that one issue was related to
performance under cygwin.

I see the solution is going towards a pre-allocation switch, but
the question I have is, why?  rsync should know how big the file
it is going to write will be, under most circumstances, since it
has already compared all the files on the source and dest, where
I thought file-size differing would force a transfer. 

This thread seems(?) to go back to this email:
===============
Rob Bosch, on Sun, 06 Aug 2006 09:04:30, wrote:
>  I've been running some tests on files created by rsync and noticing
>  fragmentation issues. ...
>  The test I conducted was straighforward:
...
>  3. Resulting file had almost 12000 fragments.
>
>  It doesn't really surprise me since rsync grabs blocks as it needs them
>  for the new.  I was wondering why rsync doesn't preallocate the space
>  like copy or other utilites do under Windows.
...
>  I am running the cygwin version of rsync ...
--------------

In doing some investigation, I noticed the problem isn't
specific to rsync but is common to many "gnunix" utils running on
cygwin on Windows.
(Continue reading)

Permalink | Reply | 4 comments |
headers
david | 4 Dec 2006 04:53
Picon

make errors, rsync 2.6.9 with acls patch

	Hi. I'm attempting to compile rsync 2.6.9 with both the acls & xattr  
patches on an MacBook pro (ie: Intel processor), with the latest  
Apple Developer Tools. While this is OS X client, I've enabled acls  
via "sudo /usr/bin/fsaclctl -p / -e" and rebooted, prior to  
attempting to compile rsync. Is there something else inherent in OS X  
Server perhaps, that I'm still missing in OS X client and is needed  
for this to work ?

If I leave out the acls patch, I see a few warnings but no errors  
when running "make".

However, when following the info in the xattrs.diff patch:
patch -p1 <patches/acls.diff
patch -p1 <patches/xattrs.diff
./prepare-source
./configure --enable-acl-support --enable-xattr-support
make

the following occurs:

Patching appears to go fine, no failure messages of any kind.
$ ./prepare-source
make: Nothing to be done for `gen'.

$ ./configure --enable-acl-support --enable-xattr-support

#end of output:

checking sys/acl.h usability... yes
checking sys/acl.h presence... yes
(Continue reading)

Permalink | Reply | 5 comments |
headers
Dmitry Melekhov | 4 Dec 2006 08:37

2.6.9 , problem with acl patch

Hello!

Without acl patch it works OK,
but if I run
/usr/bin/rsync -A --delete -n -p -r -o -g -t 
192.168.72.8:/home/samba/docs/ /BACKUP/BACKUP/LUD/docs -vvv

received 14889 names
done
recv_file_list done
get_local_name count=14889 /BACKUP/BACKUP/LUD/docs
recv_files(14889) starting
send_file_list done
send_files starting
generator starting pid=27689 count=14889
deleting in .
delete_in_dir(.)
[generator] make_file(atc-35,*,2)
Internal error: wrong write used in receiver.
rsync error: protocol incompatibility (code 2) at io.c(1204) 
[generator=2.6.9]
_exit_cleanup(code=2, file=io.c, line=1204): about to call exit(2)
rsync error: received SIGUSR1 (code 19) at main.c(1182) [receiver=2.6.9]
_exit_cleanup(code=19, file=main.c, line=1182): about to call exit(19)

If I run it without -A all is OK.
Any ideas about what is wrong in acl code?

--

-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
(Continue reading)

Permalink | Reply | 13 comments |
headers
Dmitry Melekhov | 4 Dec 2006 10:16

Re: 2.6.9 , problem with acl patch

Dmitry Melekhov пишет:
> Hello!
>
> Without acl patch it works OK,
> but if I run
> /usr/bin/rsync -A --delete -n -p -r -o -g -t 
> 192.168.72.8:/home/samba/docs/ /BACKUP/BACKUP/LUD/docs -vvv
>
btw, 2.6.8 works OK
--

-- 
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 0 comments |

Gmane