samba-bugs | 18 Nov 02:48 2014
Picon

[Bug 10941] New: include should supersede max-size

https://bugzilla.samba.org/show_bug.cgi?id=10941

            Bug ID: 10941
           Summary: include should supersede max-size
           Product: rsync
           Version: 3.0.9
          Hardware: All
                OS: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
          Assignee: wayned <at> samba.org
          Reporter: dgermansbz <at> Real-world-systems.com
        QA Contact: rsync-qa <at> samba.org

Most of the very large files I have are distributions from elsewhere. 
I have used --max-size=10MB to skip them.
I do have some specific files that I want transferred which are greater than
10MB so I entered them in the --include-from=includes.
I expected rsync to transfer the files listed in includes but it didn't.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

(Continue reading)

yhu2 | 17 Nov 07:44 2014

Bug-report:rsync may hung if time jumps backwards


Hello eveyone!

According to below reproduce steps,you could observe a rsync hang:

1:configure and startup rsync service and

mkdir /root/a
mkdir /root/b
dd if=/dev/zero of=/root/b/1 bs=1M count=1
dd if=/dev/zero of=/root/b/2 bs=1M count=1
dd if=/dev/zero of=/root/b/3 bs=1M count=1
dd if=/dev/zero of=/root/b/4 bs=1M count=1
dd if=/dev/zero of=/root/b/5 bs=1M count=1

2: start testcase

./change-time-loop.sh > /dev/null &
./rsync-loop.sh

After applying this patch ,this issue went away.

--- rsync-2.6.9/BUILD/rsync-2.6.9/util.c    2014-11-11 
13:02:11.495609639 +0800
+++ rsync-2.6.9/BUILD/rsync-2.6.9/util.c    2014-11-11 
13:01:37.606569696 +0800
 <at>  <at>  -1174,8 +1174,11  <at>  <at> 
   * Always returns TRUE.  (In the future it might return FALSE if
   * interrupted.)
   **/
(Continue reading)

Joe | 16 Nov 22:53 2014
Picon

Using --fuzzy

I have a lot of files (and directories) (up to a few hundred at a time)
that I get from various sources. Some time after I get them (after they
are already backed up), I often have to move them around and normalize
their names.

When I do this, rsync sees them as unrelated to the copies of these
files which are already on the backup destination. When I can't use the
--delete option for various reasons, this causes multiple copies of the
files to be saved in the backup destination.

I see that there is a --fuzzy option which, specified twice, may address
this issue.

Is there a tutorial or howto that addresses this so I can better
understand it before I start experimenting? I don't want to have to
reinvent strategies which, doubtless, already exist.

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

eight ∞ | 15 Nov 11:26 2014

rsync stalls at .sparsebundle directories on OS X‏‏

Hi. I try to make backups from one disk to another. I have .sparsebundle[1] directories on the source drive
(/Volumes/data/). When rsync reaches the first .sparsebundle it stalls and stays there until i ctrl+c.
I'm using OS X Yosemite and rsync version 3.1.1 protocol version 31 installed via homebrew. However, I had
this problem on OS X Lion too using rsync 3.0.9. Actually, the reason I updated to Yosemite was partly
because I thought I might get rsync working again. 

Here is the full line:
sudo rsync -vaxAXES --log-file="/Volumes/backup/__rsync/$(date +"%Y-%m-%d_%H%M%S").log"
--exclude="__rsync/" --exclude=".Trashes/" --exclude=".Spotlight-V100"
--exclude=".fseventsd/" --exclude=".DocumentRevisions-V100/" --exclude=".TemporaryItems/"
/Volumes/data/ /Volumes/backup/

I'm also seeing 32 lines of of "rsync: unpack_smb_acl: sys_acl_get_info(): Undefined error: 0 (0)"
errors directly after "sending incremental file list".

Any clues would be very welcome.

[1] https://en.wikipedia.org/wiki/Sparse_image#Sparse_bundle_disk_images
 		 	   		  
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 14 Nov 07:51 2014
Picon

[Bug 10936] New: Rsync path hijacking attack vulnerability

https://bugzilla.samba.org/show_bug.cgi?id=10936

            Bug ID: 10936
           Summary: Rsync  path  hijacking  attack  vulnerability
           Product: rsync
           Version: 3.1.1
          Hardware: All
                OS: All
            Status: NEW
          Severity: critical
          Priority: P5
         Component: core
          Assignee: wayned <at> samba.org
          Reporter: gaojianfeng <at> baidu.com
        QA Contact: rsync-qa <at> samba.org

Created attachment 10433
  --> https://bugzilla.samba.org/attachment.cgi?id=10433&action=edit
Rsync  path  hijacking  attack  vulnerability.pdf (Detailed documentation)

Hi all:
    In  newest version rsync,Baidu Security Team found a vulnerability which is
similar to wget ftp CVE-2014-4877
(http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-4877).When a
clientuses parameter -a to synchronize files of the server-side(default), for
example:
rsync -avzP  127.0.0.1::share   /tmp/share
Rsync  recursive synchronous all files,An attacker can hijack the file path  by
modifying the code of the server-side,allows remote  servers to write to
arbitrary files, and consequently execute arbitrary code .
(Continue reading)

charith | 13 Nov 06:26 2014
Picon

rsync prevent destination only new folders but need new files

Good day to all!

I'm doing git file replication task and need to sync source destination, using below sync command.

rsync -atnvv --existing --exclude '.git' --progress source/ destination/;

Here I,

1) need  only sync directories that exist on destination ,no new folders should copy from source - [used  --existing did the job but it not copying new files also.]

2) need to copy new files on source  directory to destination.

Thanks in advance
Charith.



--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
samba-bugs | 7 Nov 02:38 2014
Picon

[Bug 10925] New: non-atomic xattr replacement in btrfs => rsync --read-batch random errors

https://bugzilla.samba.org/show_bug.cgi?id=10925

            Bug ID: 10925
           Summary: non-atomic xattr replacement in btrfs => rsync
                    --read-batch random errors
           Product: rsync
           Version: 3.1.0
          Hardware: All
               URL: http://article.gmane.org/gmane.comp.file-systems.btrfs
                    /40013
                OS: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
          Assignee: wayned <at> samba.org
          Reporter: oliva <at> gnu.org
        QA Contact: rsync-qa <at> samba.org

I tried to cross-post this by email to btrfs and rsync lists, but the latter
bounced because I'm not subscribed, so here's a copy of the post there, with
additional info at the end, that I forgot to put in the initial message, but
that I posted to the former as a followup:

A few days ago, I started using rsync batches to archive old copies of ceph OSD
snapshots for certain kinds of disaster recovery.  This seems to exercise an
unexpected race condition in rsync, which happens to expose what appears to be
a race condition in btrfs, causing random scary but harmless errors when
replaying the rsync batches.

strace has revealed that the two rsync processes running concurrently to apply
the batch both attempt to access xattrs of the same directory concurrently.  I
understand rsync is supposed to avoid this, but something's going wrong with
that.  Here's the smoking gun, snipped from strace -p 27251 -p 27253 -o
smoking.gun, where both processes are started from a single rsync
--read-batch=- -aHAX --del ... run:

0: 27251 stat("osd/0.6ed_head/DIR_D/DIR_E/DIR_6",  <unfinished ...>
1: 27253 stat("osd/0.6ed_head/DIR_D/DIR_E/DIR_6", {st_mode=S_IFDIR|0755,
st_size=5470, ...}) = 0
2: 27251 <... stat resumed> {st_mode=S_IFDIR|0755, st_size=5470, ...}) = 0
3: 27253 llistxattr("osd/0.6ed_head/DIR_D/DIR_E/DIR_6",
"user.cephos.phash.contents\0", 1024) = 27
4: 27251 llistxattr("osd/0.6ed_head/DIR_D/DIR_E/DIR_6",  <unfinished ...>
5: 27253 lsetxattr("osd/0.6ed_head/DIR_D/DIR_E/DIR_6",
"user.cephos.phash.contents",
"\x01F\x00\x00\x00\x00\x00\x00\x00\x0f\x00\x00\x00\x03\x00\x00", 17, 0
<unfinished ...>
6: 27251 <... llistxattr resumed> "user.cephos.phash.contents\0", 1024) = 27
7: 27251 lgetxattr("osd/0.6ed_head/DIR_D/DIR_E/DIR_6",
"user.cephos.phash.contents", 0x0, 0) = -1 ENODATA (No data available)
8: 27253 <... lsetxattr resumed> )         = 0
9: 27253 utimensat(AT_FDCWD, "osd/0.6ed_head/DIR_D/DIR_E/DIR_6", {UTIME_NOW,
{1407992261, 0}}, AT_SYMLINK_NOFOLLOW) = 0
a: 27251 write(2, "rsync: get_xattr_data: lgetxattr"..., 181) = 181

lines 0-2, 3-6 and 5-8, show concurrent access of both rsync processes to the
same directory.  This wouldn't be a problem, not even for replaying batches,
for the lsetxattr would put the intended xattr value in there regardless of
whether the scanner saw the xattr value before or after that.

What makes the problem visible is that btrfs appears to have a race in its
handling of xattr replacement, leaving a window between the removal of the old
value and the insertion of the new one, as shown by lines 5-8.  line 3 show the
attribute existed before, and lines 5-8 show it disappears in line 7, while
lsetxattr still runs to replace it.

If rsync tries hard enough to hit this window, the lgetxattr concurrent to the
lsetxattr eventually hits, and then rsync reports an error:

rsync: get_xattr_data:
lgetxattr(""/media/px/snapshots/cluster/20141102-to-20140816/osd/0.6ed_head/DIR_D/DIR_E/DIR_6"","user.cephos.phash.contents",0)
failed: No data available (61)

At the end, it exits with a nonzero status, even though nothing really wrong
went on and the tree ended up looking just as it was supposed to.

Now, I'm a bit concerned because the btrfs race condition, if exercised on
security-related xattrs or ACLs, could cause data to become visible that
shouldn't, which could turn this into a locally exploitable security issue. 
Sure enough nobody goes nuts repeatedly changing the ACLs of a dir or file
containing information that should be guarded by it, so as to increase the
likelihood that an attacker succeeds in accessing the data, but still...  I
don't think the temporary removal of the xattr for subsequent insertion should
be visible at all.

I'm sorry for reporting a potential security issue like that, but by the time
it occurred to me that it might have potential security implications, I'd
already mentioned the problem on #btrfs at FreeNode, so the horse was out of
the barn already :-(

The bugs described above occurred with rsync-3.1.0-5.fc20.x86_64 and
kernel-libre-3.16.7-200.fc20.gnu.x86_64.  The btrfs code in kernel-libre is
unchanged from the corresponding Fedora kernel.  The distro is BLAG
200k/x86_64, under development.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Touretsky, Gregory | 2 Nov 14:27 2014
Picon

Rsync hangs on "No space left on device" on the server side

Hi,

 

  We’re replicating data over WAN using rsync over hpn-ssh.

In case of destination disk full, rsync on the server side exits with

                rsync: close failed on <path> : No space left on device (28)

rsync error: error in file IO (code 11) at receiver.c(731)

However, parent sshd process doesn’t recognize it and remains running – keeping rsync on the source side waiting until ssh session timeout expires.

Looks like this issue was reported few times in the past (link1, link2)

 

Is there any known resolution?

We’re using rsync 3.0.6

 

Regards,

   Gregory

 

               

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
samba-bugs | 31 Oct 16:25 2014
Picon

[Bug 10910] New: build: FTBFS paralel build start before proto.h generation (patch)

https://bugzilla.samba.org/show_bug.cgi?id=10910

            Bug ID: 10910
           Summary: build: FTBFS paralel build start before proto.h
                    generation (patch)
           Product: rsync
           Version: 3.1.1
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: minor
          Priority: P5
         Component: core
          Assignee: wayned <at> samba.org
          Reporter: philippe.coval <at> open.eurogiciel.org
        QA Contact: rsync-qa <at> samba.org

build: FTBFS paralel build start before proto.h generation (patch)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Allen Supynuk | 29 Oct 21:46 2014
Picon

Re: rsync behavior on copy-on-write filesystems


Sent from my iPad
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 23 Oct 19:40 2014
Picon

[Bug 10893] New: Allow option to sync symlinks last (or delayed)

https://bugzilla.samba.org/show_bug.cgi?id=10893

            Bug ID: 10893
           Summary: Allow option to sync symlinks last (or delayed)
           Product: rsync
           Version: 3.1.1
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: core
          Assignee: wayned <at> samba.org
          Reporter: moffatjd <at> gmail.com
        QA Contact: rsync-qa <at> samba.org

Presently rsync will create symlinks first (on the receiver side) and then it
will sync the content that is potentially being linked. This can cause race
conditions on the receiver side.

For example, given the small hierarchy already existing on the receiver side:

/foo -> foo_1.0
/foo_1.0

And then assume we have an updated hierarchy on the sender side:

/foo -> foo_2.0
/foo_1.0
/foo_2.0

When rsync updates the receiver side with the new foo_2.0, it first does the
symlink (pointing foo to foo_2.0) but the contents of foo_2.0 have not been
synced yet. During this time frame (when link is sycned, but content is not)
the symlink points to a non-existent directory. If a program on the receiver
side should reference foo during the rsync, it will fail.

The option --delay-updates does not fix this behavior, and is the closest
option available to obtaining this 'atomic' like behavior.

Please consider an option --link-after (similar to --delete-after) that simply
waits and does all symlinks after all updates.

OR

Please consider creating a --delay-links option that would not sync a link
until the content of the link (including subdirs) is complete. 

Thank you.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


Gmane