jerrycwwong | 23 Jul 03:34 2014
Picon

Fw: rsync permission error

Hi All

I found that the rsync cannot fully sync permission although I have specified the -A option. The ACL is ok but the permission itself is not.

rsync command:
rsync -rtgoDAh --stats --files-from=flist.txt --log-file=$DetailLogPath $OldPath $NewPath &

Old Server (rsync version 3.0.4):  
-rw-r-----+  1 43810537 imopfs    246784 Jul 22 10:17 miki2.sas7bdat

New Server (rsync version 3.0.6):
-rw-rw----+ 1 43810537 imopfs     361472 Jul  1 14:09 miki2.sas7bdat

Thanks and regards
Jerry Wong

************************************************************
HSBC Global Asset Management (Hong Kong) Limited
whose registered address is HSBC Main Building, 1 Queen's Road Central, Hong Kong
************************************************************
----------------------------------------- ******************************************************************* This e-mail is confidential. It may also be legally privileged. If you are not the addressee you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return e-mail. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. ******************************************************************* "SAVE PAPER - THINK BEFORE YOU PRINT!"
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
mr queue | 20 Jul 02:43 2014
Picon

Adjusting transfer rate on the fly

Hello,

Does anyone know if it is possible to adjust the value of --bwlimt= on
the fly using gdb or the like? I've adjusted mysql max_connections in
the past with gdb and was curious if I would be able to make
adjustments to rsync in a similar fashion.

I am very well aware I can stop it and start it again with a new
value. That is not what I'm after here.

Regards,
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 18 Jul 05:20 2014
Picon

[Bug 10724] New: rsync 3.1.1 incorrectly creates extra dirs inside --backup-dir on Mac OSX

https://bugzilla.samba.org/show_bug.cgi?id=10724

           Summary: rsync 3.1.1 incorrectly creates extra dirs inside
                    --backup-dir on Mac OSX
           Product: rsync
           Version: 3.1.1
          Platform: x86
        OS/Version: Mac OS X
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned <at> samba.org
        ReportedBy: chdiza <at> gmail.com
         QAContact: rsync-qa <at> samba.org

On MacOSX 10.9.4, new to rsync 3.1.1, I observe the incorrect creation of empty
dirs inside a --backup-dir.

To reproduce:

1. mkdir foo
2. mkdir bar
3. cd foo; touch fun
4. cd ..
5. rsync -trlvhp foo bar

Everything is as it should be: inside bar there is a clone of foo.  But now
continue (from the same working dir as from setp 5):

6. mkdir foo/newdir
7. cd foo; mv fun newdir; cd ..
8. rsync -trlvhpb --delete --backup-dir=mybackups --suffix=~ foo bar

Expected result: the directory bar should look like this:

bar/
bar/foo/
bar/foo/newdir/fun
bar/mybackups/
bar/mybackups/foo/
bar/mybackups/foo/fun~

Actual result: rsync 3.1.1 has created an empty dir "newdir" inside
mybackups.  That is, this is what bar looks like:

bar/
bar/foo/
bar/foo/newdir/fun
bar/mybackups/
bar/mybackups/foo/
bar/mybackups/foo/fun~
bar/mybackups/foo/newdir/

As you can see there is magically a "newdir" there.  It shouldn't be
there, because there never was a "newdir" inside bar/foo prior to step 8.
Therefore, there was no "newdir" there to be "backed up" into mybackups.

This behavior is new to 3.1.1 as best as I can tell.  I'm positive it never
occurred
on any prior version of rsync that I've ever used (2.6-ish?).

The above was just a toy example designed to illustrate the issue.  In
practice, rsync 3.1.1 left dozens of such "ghost" directories inside my
--backup-dir.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Don Cohen | 17 Jul 03:40 2014

question about output of files copied/deleted


It seems to me that this output would be more useful if it
were possible to uniquely translate a line of output back into
a file path.
Right now that's not possible due to the control character encoding.
An output line like
asd\#002\#003zxc
could either mean a file of that name or
asd^B\#003zxc or asd^B^Czxc or asd\#002^Czxc

I was hoping to start with rsync -n, figure out what was
going to be replaced or deleted, archive those files, and
then use the list to do the rsync without the -n but not getting
files that had been updated since the original -n -- but this
ambiguity prevents that from working.

Is there some solution I'm missing?

Otherwise, what's wrong with not encoding the control characters on
output?  I see only one problem, which is that newline characters
would then make it ambiguous whether
asd
zxc
was actually one file or two.
Since there is one character that cannot appear in such a line of
output (for ext4 at least, which is ascii 0), you could replace
newlines with that.  Admittedly not portable...
Perhaps there could be an argument saying how to map file names?

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 16 Jul 16:02 2014
Picon

[Bug 10719] New: Error with cached effective process gid

https://bugzilla.samba.org/show_bug.cgi?id=10719

           Summary: Error with cached effective process gid
           Product: rsync
           Version: 3.1.1
          Platform: x64
        OS/Version: FreeBSD
            Status: NEW
          Severity: major
          Priority: P5
         Component: core
        AssignedTo: wayned <at> samba.org
        ReportedBy: oleg <at> mamontov.net
         QAContact: rsync-qa <at> samba.org

Tere was an error in previous git commit
3b83a22057b71e7df2b960d3997fb4474910f30e:
----------------------------------------
[lonerr <at> neon ~/src/rsync]% git show 3b83a22057b71e7df2b960d3997fb4474910f30e
commit 3b83a22057b71e7df2b960d3997fb4474910f30e
Author: Wayne Davison <wayned <at> samba.org>
Date:   Wed Sep 2 08:56:34 2009 -0700

    Define and use "our_gid" variable.
----------------------------------------

In this commit effective process gid determined once (in main.c), but after
client connect and daemon forked effective gid may be different (when you use
the 'gid' option in config file).
Now this commit was merged in 3.1.x thus broken some use cases.

Simple test case:

1. Prepare destination catalog:
rm -rf /tmp/dst && mkdir /tmp/dst && chmod 755 /tmp/dst && chown nobody:nobody
/tmp/dst

2. Prepare minimal rsyncd.conf:
--------------------------
pid file = /var/run/rsyncd.pid
uid = nobody
gid = nobody
read only = no
use chroot = no
[dst]
    path = /tmp/dst
--------------------------

3. Start rsyncd as standalone daemon with standard rc script.

4. Try to rsync empty file belongs to group wheel (gid==0).
touch /tmp/src && chown 0:0 /tmp/src && rsync -av /tmp/src localhost::dst/
--------------------------------
sending incremental file list
src
rsync: chgrp ".src.tIDLzz" (in dst) failed: Operation not permitted (1)

sent 79 bytes  received 111 bytes  380.00 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code
23) at main.c(1165) [sender=3.1.1]
--------------------------------

With this commit rollbacked - operation is restored.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Philip Hall | 16 Jul 09:53 2014
Picon

rsyncd.conf gid parameter no longer supports spaces in gid names?‏

<!-- .hmmessage P { margin:0px; padding:0px } body.hmmessage { font-size: 12pt; font-family:Calibri } -->
I have been using an rsyncd.conf file with a gid value that contains spaces. This worked fine until rsync got updated to v3.1.0 and I started getting invalid gid messages specifying the first word of the gid name as the gid that could not be found.

Having done some reading on the new functionality of the gid parameter and the ability to now specify a gid list I assume this list is space separated.

I have tried quoting it with both single and double quotes with no success.

I have had to resort to using the numeric gid.

Is there still a way to specify a gid name containing spaces?

Regards,

Phil.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
samba-bugs | 14 Jul 10:51 2014
Picon

[Bug 10715] New: IPv6 configure test fails on musl c-library

https://bugzilla.samba.org/show_bug.cgi?id=10715

           Summary: IPv6 configure test fails on musl c-library
           Product: rsync
           Version: 3.1.1
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned <at> samba.org
        ReportedBy: timo.teras <at> iki.fi
         QAContact: rsync-qa <at> samba.org

The autoconfigure tests for IPv6 relies on C-library internals, and it
currently fails for musl c-library based linux. For Alpine Linux bug report see
http://bugs.alpinelinux.org/issues/3158.

A better test (and non-OS dependent) would be to try creating AF_INET6 socket,
if it succeeds, then IPv6 is supported.

See e.g.:

http://ftp.netbsd.org/pub/NetBSD/NetBSD-release-6/src/crypto/dist/ipsec-tools/configure.ac
(and search for --disable-ipv6)

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 9 Jul 07:59 2014
Picon

[Bug 10705] New: usermap/groupmap do not escape wildcards passed to the remote

https://bugzilla.samba.org/show_bug.cgi?id=10705

           Summary: usermap/groupmap do not escape wildcards passed to the
                    remote
           Product: rsync
           Version: 3.1.1
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned <at> samba.org
        ReportedBy: darius <at> dons.net.au
         QAContact: rsync-qa <at> samba.org

Hi,
I was trying to map all GID/UID to 0:0 with..
rsync --chown 0:0 foo user <at> remote:/tmp

but it bombs out with..
rsync: No match.
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(226)
[sender=3.1.1]

I am guessing that --chown 0:0 maps internally to --usermap *:0 --groupmap *:0
but they aren't escaped properly.

As a work around I am using..
rsync --usermap \\\*:0 --groupmap \\\*:0 foo user <at> remote:/tmp

and it works.
(i.e. the *'s are escaped enough the remote invocation doesn't fail).

I imagine this would also work if the remote shell did not error on
non-matching globs, however I think the existing behaviour is still broken as
if the glob did match the behaviour could be very odd.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Adam Edgar | 7 Jul 22:18 2014
Picon

increasing the write block size for high latency

I am trying to transfer a group of files in whole using rsync using the —files-from option across a network
with high bandwidth but relatively high latency. When I log into the remote machine I see an rsync command
running like thus:

rsync --server --sender -Re.sf -B16384 --files-from=- --from0 . /

Note that I have used the -B option to increase block size but that seems to just apply to the blocks used for
checksumming instead of actual network I/O. I came to this conclusion by running strace on the PIDs on both
remote and local hosts. On the server side I got a stream of 4K writes on fd 1 followed by a select on that same
fd :

write(1,
"\275\214\357\357\22y4\237\215Z\226\331\355Z!\340\373\227\340V~\20\35C\3\337_\27\257\236\321\204"...,
4092) = 4092
select(2, NULL, [1], [1], {60, 0})      = 1 (out [1], left {59, 999996})

I’d like to see if I can increase the write block size to cut down on the delay of each select to this unnamed
pipe to sshd.  I’m not sure if that would help speed up the transfer as the issue may be below at the sshd
level (which I can’t strace as a non-root user).  If anyone has experience with slowness at the ssh level
please advice on which flags would allow me to speed up there. 

Any help will be much appreciated.

Adam S Edgar
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Carlos Carvalho | 6 Jul 01:42 2014
Picon

duplicated --itemize-changes hangs since 3.1.1pre1

One of our rsync updates has been getting stuck at the download point.
It gets the incremental list, deletes stuff but when it tries to
download new/updated files it gets stuck and aborts after the timeout;
it doesn't even create the temporary. Experimentation shows that it
happens because the --itemize-changes option is repeated (on purpose);
putting it only once works normally. Apparently it's been happening
since 3.1.1pre1, and continues with 3.1.1 downloaded from git today
(July 5).

It only happens with one of the many upstreams we use(!); protocol
version is 30. Bizarre...
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 1 Jul 05:17 2014
Picon

[Bug 10681] New: Add an option to delete non-existent files in --files-from

https://bugzilla.samba.org/show_bug.cgi?id=10681

           Summary: Add an option to delete non-existent files in
                    --files-from
           Product: rsync
           Version: 3.0.9
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: core
        AssignedTo: wayned <at> samba.org
        ReportedBy: darius <at> dons.net.au
         QAContact: rsync-qa <at> samba.org

I have a script which monitors file system events and then calls rsync using
--files-from, however that does not handle the case where those files do not
exist so I currently have a separate job which culls files have been deleted.

It would be nicer if rsync read a file explicitly mentioned in --files-from
that does not exist as one which needs deletion on the destination.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Gmane