Franke | 31 Aug 15:47 2014
Picon

rsync suggest / add option --exec_prefile/postfile

Hi and hello!

Since the Harddiscs will bigger and bigger the datavolumes are too. So
we become a problem with the data integrity.

IMHO the best was to test the data integrity are the checksum like md5,
shaX,...

My Idea: Extent rsync with optional parameters to do this while sync.

So we can create/test the checksums and wrote them to DB or Harddisk
etc. like the follow:

# DRAFT
 --exec_checksum_cmd contains the checksum programm to use
 --exec_prefile execute a command or script BEFORE copy/move a file
 --exec_postfile execute a command or script AFTER copy/move a file
 These Options can be used eg. for creating/testing extra checksums or
wrote a database like rsync .... --exec_checksum_cmd '/bin/sha1sum-b'
--exec_postfile
'make_test_and_wrote_sha1_to_db.sh'
 The command become follow parameters back from rsync:
from_<server>/file, to_<server>/file, from_hash, to_hash
 Additional works
 --exec_remote_prefile execute a command or script BEFORE copy/move a
file on the Remote side
 --exec_remote_postfile execute a command or script AFTER copy/move a
file on the Remote side
 but the command MUST exists on the remote side

(Continue reading)

Franke | 31 Aug 15:32 2014
Picon

rsync suggest / add new options --exec_postfile_execute

Hi and Hello!

Since Hard discs will bigger and bigger the datavolumes seems too. So we
become a problem with the integrity of data.

IMHO the best way for testing the data integrity is the additional
checksumhash (md5,sha,...).

And so my idea: to create/test these while doing a backup with rsync.

What we need is a extension of rsync with optional parameters like the
follow:

# DRAFT
 --exec_hash_cmd contains the checksum programm to use
 --exec_prefile execute a command or script BEFORE copy/move a file
 --exec_postfile execute a command or script AFTER copy/move a file
 These Options can be used eg. for creating/testing extra checksums or
wrote a database like: rsync .... --exec_hash_cmd '/bin/sha1sum -b'
--exec_postfile
'make_test_and_wrote_sha1_to_db.sh'
 The command become follow parameters from rsync: from_<server>/file
to_<server>/file from_file_hash to_file_hash
 Additional works
 --exec_remote_prefile execute a command or script BEFORE copy/move a
file on the Remote side
 --exec_remote_postfile execute a command or script AFTER copy/move a
file on the Remote side
 but the command MUST exists on the remote side

(Continue reading)

Dave Gradwell | 30 Aug 12:36 2014
Picon

Cleanest way to find out if it was a --dry-run

Hi, 

As far as I can see, the only way you can tell it's been a dry-run at the receiver-side, is to parse RSYNC_ARGs
in the pre-xfer script.
I think the only way is to crawl through the RSYNC_ARGs to find a value which:
- starts with a single dash, and;
- contains 'n', and;
- comes prior to the "." value, signifying the start of the 'path args' (which seem inherently variable,
ergo scary).

I can't see any other way to determine that it's been a dry run.  On the sender-side, the log contains a handy
(DRY RUN) indicator next to the stats:
2014/08/29 18:54:15 [62154] total size is 327,680,000  speedup is 68,739.25 (DRY RUN)
but I don't think you can output stats at the receiver-side, and I don't think there's any other mention of
the dry run in the receiver-side log.

In my pre-xfer script, I see this:
RSYNC_ARG1=--server
RSYNC_ARG2=-nltrRze.iLsfx
RSYNC_ARG3=--log-format=X
RSYNC_ARG4=--bwlimit=256
RSYNC_ARG5=--delete-during
RSYNC_ARG6=--delete-excluded
RSYNC_ARG7=--force
RSYNC_ARG8=--partial-dir
RSYNC_ARG9=/in-progress/partial
RSYNC_ARG10=--link-dest
RSYNC_ARG11=/latest/data
RSYNC_ARG12=--info=STATS2
RSYNC_ARG13=--debug=EXIT
(Continue reading)

samba-bugs | 26 Aug 19:01 2014
Picon

[Bug 10785] New: [PATCH] Add a flag to use numeric sort

https://bugzilla.samba.org/show_bug.cgi?id=10785

           Summary: [PATCH] Add a flag to use numeric sort
           Product: rsync
           Version: 3.1.1
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned <at> samba.org
        ReportedBy: rom <at> rom1v.com
         QAContact: rsync-qa <at> samba.org

Created attachment 10229
  --> https://bugzilla.samba.org/attachment.cgi?id=10229
the patch

I need to sync a lot of files with non-zero-padded numbered filenames.

I would like them to be transferred in order, to be able to use "partial"
transfers (it takes a lot of time to transfer the whole data).

My concrete need is to rsync duplicity backups. Let's say I have 21 files. For
now, they are transferred in that order :

duplicity-full.20140531T230029Z.vol1.difftar.gpg
duplicity-full.20140531T230029Z.vol10.difftar.gpg
duplicity-full.20140531T230029Z.vol11.difftar.gpg
(Continue reading)

wangtuma2 | 26 Aug 17:47 2014

how to get the rsync remote error code at client side

Hi,
 
I'm trying to use rsync to sync files between my desktop and a file server, using my desktop as the client, and the file server as the remote side (or the server side, of course), and I'm wondering how I can get the error code of the remote/server side at the client side. For example, if the file server is out of space, or the server couldn't write to file due to permission problems, or any other kinds of errors, how can I (or the client) know?
I read the man page; it lists some error code which doesn't seem relative or specific to the remote side. Is there a mechanism that allows the server to report an error to the client?

If there is not, how can I add such a mechanism to rsync?


Thanks!

--tuma

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Geoffrey Leach | 22 Aug 21:37 2014
Picon

problem with exclude

I've tried all of the permutations of exclude I can think of, so presumably I have some fundamental
misunderstanding. The goal here is to exclude the file tv11.html in the ftp directory.

rsync -avvvv -n --verbose --one-file-system -exclude='ftp/tv1.html' /usr/local/src
/Media/sata/usr/local > /tmp/rsync
(Server) Protocol versions: remote=30, negotiated=30

cmd=xclude=ftp/tv1.html machine=<NULL> user=<NULL> path=/Media/sata/usr/local
[sender] make_file(src/ftp/tv1.html,*,2)
[sender] i=1046 /usr/local src/ftp/tv1.html mode=0100644 len=130204 uid=1000 gid=1000 flags=0
[generator] i=1046 3 src/ftp/tv1.html mode=0100644 len=130204 uid=1000 gid=1000 flags=0
recv_generator(src/ftp/tv1.html,1046)
send_files(1046, /usr/local/src/ftp/tv1.html)
src/ftp/tv1.html
recv_files(src/ftp/tv1.html)

file /usr/local/src/ftp/tv1.html
/usr/local/src/ftp/tv1.html: HTML document, UTF-8 Unicode text, with very long lines

One additional piece of data. If I attempt to exclude "*swp"after the ftp exclude, that fails as well.
Before the exclude of ftp, it works as intended.

BTW, this is the smallest command that reproduces the problem. What I _really_ want to do is to exclude ftp/tv*

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 22 Aug 11:11 2014
Picon

[Bug 10776] New: SIGSEGV in utf8_internal_loop()

https://bugzilla.samba.org/show_bug.cgi?id=10776

           Summary: SIGSEGV in utf8_internal_loop()
           Product: rsync
           Version: 3.1.0
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned <at> samba.org
        ReportedBy: mluscon <at> redhat.com
         QAContact: rsync-qa <at> samba.org

It looks like the fix from #9798 was not sufficient and rsync can still crash
after improper iconv call. Any suggestions what might have gone wrong in this
case?

Version of rsync:
rsync-3.1.0

Truncated backtrace:
Thread no. 1 (10 frames)
 #0 utf8_internal_loop at ../iconv/loop.c:331
 #1 __gconv_transform_utf8_internal at ../iconv/skeleton.c:611
 #2 __gconv at gconv.c:79
 #3 iconv at iconv.c:52
 #4 iconvbufs at rsync.c:209
 #5 rwrite at log.c:374
 #6 rprintf at log.c:433
 #7 whine_about_eof at io.c:222
 #8 read_buf at io.c:1825
 #9 read_int at io.c:1720

Full backtrace is available at
https://bugzilla.redhat.com/attachment.cgi?id=913207 .

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

samba-bugs | 20 Aug 02:51 2014
Picon

[Bug 10772] New: Bad time logging as daemon

https://bugzilla.samba.org/show_bug.cgi?id=10772

           Summary: Bad time logging as daemon
           Product: rsync
           Version: 3.1.1
          Platform: x64
        OS/Version: Mac OS X
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned <at> samba.org
        ReportedBy: davegradwell <at> yahoo.co.uk
         QAContact: rsync-qa <at> samba.org

When the rsync daemon logs, it fails to determine the correct time:
2014/08/18 18:47:24 [1002] connect from localhost (::1)
2014/08/18 17:47:24 [1003] rsync to dave/in-progress/data/ from dave <at> localhost
(::1)

When I specify use chroot = false in the rsyncd.conf, it logs the time
correctly:
2014/08/18 19:21:32 [1138] connect from localhost (::1)
2014/08/18 19:21:32 [1139] rsync to dave/in-progress/data/ from dave <at> localhost
(::1)

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Dave Gradwell | 18 Aug 20:31 2014
Picon

Bad time logging - rsync 3.1.1 as daemon on Mac OS 10.9

Hi,

I'm on Mac OS 10.9, using rsync 3.1.1.

When the rsync daemon logs, it fails to determine the correct time:
2014/08/18 18:47:24 [1002] connect from localhost (::1)
2014/08/18 17:47:24 [1003] rsync to dave/in-progress/data/ from dave <at> localhost (::1)

When I specify use chroot = false in the rsyncd.conf, it logs the time correctly:
2014/08/18 19:21:32 [1138] connect from localhost (::1)
2014/08/18 19:21:32 [1139] rsync to dave/in-progress/data/ from dave <at> localhost (::1)

I have identified this bit in log.c:
	/* This looks pointless, but it is needed in order for the
	 * C library on some systems to fetch the timezone info
	 * before the chroot. */
	timestring(time(NULL));

...but I'm unsure how to go about fixing this.
Can anyone offer any advice?

With thanks,

Dave.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Robert DuToit | 17 Aug 03:10 2014
Picon
Picon

delete not working with multiple source directories

Hello All,

I have lately heard from a  couple of people that delete option was not deleting extraneous files on
destination. This only happens when there are several source directories and at destination some show
delete working and others not. I can not reproduce the issue so it is hard to pinpoint. 

one example:

This is on OSX Mavericks running rsync 3.0.9 with the usual osx patches and options

rsync -aHAXN --fileflags --force-change --protect-decmpfs --exclude-from=/tmp/BLexclusions.txt
--stats —delete —progress -vvv /Applications /Library /Users /Volumes/Backup

So  with /Applications /Library /Users  being source, only /Applications shows files being deleted.
Removing /Applications and the two other directories work with delete fine. Even changing the order has
same result. The exclude file shows only one directory in /Applications. 

I asked user to try different (new) external drive - same results. 

I’m stumped as there are no rsync errors. Rsync progress shows everything fine and mentions files
deleted in /Applications and nowhere else.

I am not very good with rsync source but perhaps there are areas I could look where --delete might be disabled
for whatever reasons? 

There are usually ACL's on both /Applications and /Library in OSX

0: group:everyone deny delete

but this is overridden by rsync as root any ways…  usual regular permissions on all directories as well. 

Any thoughts appreciated

Thanks,  Rob

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

jerrycwwong | 10 Aug 13:08 2014
Picon

rsync error: unexplained error (code 255) at rsync.c(541) [sender=3.0.4]

Does anyone know what this message is about? I keep getting this message once in a while after my rsync command is run for some time.

rsync error: unexplained error (code 255) at rsync.c(541) [sender=3.0.4]

Regards

Jerry

************************************************************
HSBC Global Asset Management (Hong Kong) Limited
whose registered address is HSBC Main Building, 1 Queen's Road Central, Hong Kong
************************************************************
----------------------------------------- ******************************************************************* This e-mail is confidential. It may also be legally privileged. If you are not the addressee you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return e-mail. Internet communications cannot be guaranteed to be timely, secure, error or virus-free. The sender does not accept liability for any errors or omissions. ******************************************************************* "SAVE PAPER - THINK BEFORE YOU PRINT!"
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Gmane