headers
Andreas Schildbach | 1 Jan 2010 15:26
Picon
Gravatar

temp name for destination directory, move in place after successfull complete?

Hello everyone,

In --link-dest/--copy-dest/--compare-dest modes, I'd like to distinguish
between destination directories that have been completely rsynced and
the ones that are only partially synced because of a network failure.

In short, I'd like to mimic the behaviour rync has on individual files
(building them as .filename-random and then renaming them to filename)
for the whole destination directory.

Of course, I could instruct rsync to create .destination-random rather
than just destination. However, I think I cannot rename the temporary
directory after successfull completion via the rsync protocol (this
would need shell access or a mounted filesystem, both of which I want to
avoid).

Is there any way to do this magic with rsync, or maybe is there any
option that removes the whole destination again if something went wrong?

Best regards,

Andreas

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 1 comment |
headers
Warren Oates | 1 Jan 2010 16:04
Picon

Re: rsync email notification on success and failure + Log

Hi:

Let cron(8) run your script for you. Cron will happily email you the
verbose output from rsync after its task has run.
man cron
man crontab
man 5 crontab

On Tue, Dec 29, 2009 at 7:12 PM, Sébastien Arpin <sebarpin <at> hotmail.com> wrote:
> Hi,
>
> I have a bash script for rsync that should tranfer all my filer from one
> drive to the other.
>
> I would like to know how I can make the script sending me an email after the
> script is run and be able to know if it was a success or not and also if
> possible to include the log file.
>
> This is my script:
>
> !/bin/bash
>
> rsync -av --delete --log-file=/home/duffed/RSyncLog/$(date
> +%Y%m%d)_rsync_Multimedia.log /media/Multimedia/ /media/MultimediaBackUp
>
>
> Thanks a lot!
>
> Sébastien
>
(Continue reading)

Permalink | Reply | 0 comments |
headers
Giampaolo Tomassoni | 1 Jan 2010 16:46

Occasional but consistent trouble with --filter

I'm running rsync version 3.0.6 in order to migrate a set of objects from an
old server to a new one.

The set of object to migrate is basically composed of mail dirs which follow
a well-defined name pattern, such that I believe it should be easily handled
by the --filter option.

I'm infact attempting to use the following rsync command in the new server:

rsync -auEAXDSvz --delete-after --numeric-ids --compress-level=9 --stats
--super --progress -e 'ssh -oCompression=no' '--filter=-!r
/srv/vhome/*/users/*/.mail/' user <at> oldserver:/srv/vhome /srv/

The command works *mostly* well, but occasionally (and consistently) it
attempts migrating or deleting stuff whose pathname seems not match the
filter spec above.

In example, the command always tries migrating files from
/vhome/xxx/home/www/admin/stats/, which I think can't fit in the
/srv/vhome/*/users/*/.mail/ filter.

Are double-wildcard filters like the one I'm using allowed in rsync? Is
there any way to circumvent this problem?

Please note that I can't simply merge the whole /srv/vhome/ dir, since I'm
attempting to complete the second phase of this server migration: all the
non mail related stuff had already been transferred and now is probably more
recent on the new server than on the old one...

Also note that the following command:
(Continue reading)

Permalink | Reply | 3 comments |
headers
Al Grant | 2 Jan 2010 00:21
Picon

Rsync, CP and Cron job workaround?


Hi,

I have a Linux based NAS, which unfortunately appears to be a bit
underpowered, heres why I say that:

5 PC's backup to the NAS on a weekly full with hourly incrementals 7 days a
week between 7am and 6pm (2 week retention policy). There is no problem with
that.

Where there is a problem is getting that data off the NAS to the USB
attached storage for off site (2 disks swapped weekly). The built in RSYNC
via the web gui takes about 3 days to sync to USB after the weekly full (Sun
night)- 100% CPU. The fulls are about 20-40Gb each.

I was thinking about one of two ways around this:

1. using cron and cp copy the weekly fulls to the USB storage, then use
nightly rsyncs to keep everything else up to date. I am assuming here that
rsync wont recopy the stuff copied with cp?

The logic of the cp cron job would need to be something like:
*delete stuff created over 8 days ago
*copy all files in backup dir recursively that dont already exist (should
only be full backups)
*run a rsync for cleanup

2. use rsync exclusively, but schedule it to only run it outside of business
hours, ie let it run from 7pm -5am and kill it at 5am. Over the course of
the week it should catch up?
(Continue reading)

Permalink | Reply | 1 comment |
headers
Wayne Davison | 2 Jan 2010 00:30
Picon
Favicon

Re: Occasional but consistent trouble with --filter

On Fri, Jan 1, 2010 at 7:46 AM, Giampaolo Tomassoni <Giampaolo <at> tomassoni.biz> wrote:
'--filter=-!r /srv/vhome/*/users/*/.mail/' user <at> oldserver:/srv/vhome /srv/

See the manpage discussion of anchoring at the top-level of the transfer for why your filter rule should not have a "/srv" prefix.  The 'r' modifier makes the rule only apply on the receiving side, which means you're specifying a protect rule to prevent removal of files, but does not stop the files from being copied.  The '!' modifier negates the match, which makes it protect everything from deletion except .mail dirs (though, since its anchored wrong, it would seem to protect everything from deletion).

Perhaps you wanted to specify --filter='- /vhome/*/users/*/.mail/'  (for both excluding and deleting of that dir)?  If not -- if you're really trying to copy just hte .mail dirs, you have a lot more things to change.

..wayne..
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 2 comments |
headers
Al Grant | 2 Jan 2010 03:08
Picon

Re: Rsync, CP and Cron job workaround?


rsync version is 3.0.6 protocol version 30.

-Al

-- 
View this message in context: http://old.nabble.com/Rsync%2C-CP-and-Cron-job-workaround--tp26989197p26990034.html
Sent from the Samba - rsync mailing list archive at Nabble.com.

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 0 comments |
headers
Giampaolo Tomassoni | 2 Jan 2010 12:23

RE: Occasional but consistent trouble with --filter

> On Fri, Jan 1, 2010 at 7:46 AM, Giampaolo Tomassoni <Giampaolo <at> tomassoni.biz> wrote:
> '--filter=-!r /srv/vhome/*/users/*/.mail/' user <at> oldserver:/srv/vhome /srv/
> 
> See the manpage discussion of anchoring at the top-level of the transfer for why 
> your filter rule should not have a "/srv" prefix.  The 'r' modifier makes the rule 
> only apply on the receiving side, which means you're specifying a protect rule to 
> prevent removal of files, but does not stop the files from being copied.  
> The '!' modifier negates the match, which makes it protect everything from 
> deletion except .mail dirs (though, since its anchored wrong, it would seem to 
> protect everything from deletion).
> 
> Perhaps you wanted to specify --filter='- /vhome/*/users/*/.mail/'  (for both 
> excluding and deleting of that dir)?  If not -- if you're really trying to copy 
> just hte .mail dirs, you have a lot more things to change.
> 
> ..wayne..

Thank you Wayne,

I'm actually trying to copy the .mail dirs.

Based on your hints, I also tried the followings in the new server:

rsync -auEAXDSvz --delete-during --numeric-ids --compress-level=9 --stats
	--super --bwlimit=15 --progress -e 'ssh -oCompression=no'
	'--filter=+ /*/ + /*/users/ + /*/users/*/ + /*/users/*/.mail/ + /*/users/*/.mail/* - *'
	--dry-run user <at> oldserver:/srv/vhome/ /srv/vhome/

rsync -auEAXDSvz --delete-during --numeric-ids --compress-level=9 --stats
	--super --bwlimit=15 --progress -e 'ssh -oCompression=no'
	'--filter=+ /*/ + /*/users/ + /*/users/*/ + /*/users/*/.mail/ + /*/users/*/.mail/* - /**'
	--dry-run user <at> oldserver:/srv/vhome/ /srv/vhome/

this command (which perhaps follows your "you have a lot more things to change" hint) yields exactly the
same results of the previous (with '--filter=-!r /srv/vhome/*/users/*/.mail/'). You may see that I'm
not using the 'r' modifier anymore.

I'm suspecting there is something wrong in the way the rsync pattern filtering code works: it works most of
the time, but occasionally it doesn't. In example it keeps replicating the content in a /srv/vhome/ dir
(XXXXX/home/www/components/com_lightgallery/views/list/tmpl/index.html) which can't resemble
any of the '+' patterns given in --filter.

In example, adding some -vvv to a --list-only version of the above commands, I see this:

	[sender] flist start=1, used=82, low=0, high=81
	[sender] i=1 /srv/vhome ./ mode=040755 len=2640 uid=0 gid=0 flags=5
	[sender] i=2 /srv/vhome .DS_Store mode=0100640 len=21508 uid=0 gid=0 flags=0
	[sender] i=3 /srv/vhome XXXXXX/ mode=040755 len=96 uid=0 gid=81 flags=4

Why the file /srv/vhome/.DS_Store is selected for the transfer? Which filtering rule selects it?

I'm stuck.

Giampaolo

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 1 comment |
headers
Giampaolo Tomassoni | 2 Jan 2010 18:09

RE: Occasional but consistent trouble with --filter

> -----Original Message-----
>
> ...omissis...
> 
> I'm stuck.
> 
> Giampaolo

Ok, I've found why. It seems that one can't put all the filtering patterns
in a single '--filter=...' option, but instead each filtering pattern has to
be specified by its own option.

This seems to work to me:

rsync -auEAXDSz --delete-after --numeric-ids --compress-level=9 --stats
--super --bwlimit=15 --progress -e 'ssh -oCompression=no' '--filter=+ /*/'
'--filter=+ /*/users/' '--filter=+ /*/users/*/' '--filter=+
/*/users/*/.mail/' '--filter=+ /*/users/*/.mail/**' '--filter=- /**'
user <at> oldserver:/srv/vhome/ /srv/vhome/

It seems to me this is not well explained in the rsync manpage, but maybe I
didn't pay enough attention to it.

Thanks everybody,

	Giampaolo

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 0 comments |
headers
samba-bugs | 3 Jan 2010 01:08
Picon
Favicon

DO NOT REPLY [Bug 7015] Build problems on HP-UX, Tru64 -- alloca, -Wno-unused-parameter

https://bugzilla.samba.org/show_bug.cgi?id=7015

------- Comment #7 from sms <at> antinode.info  2010-01-02 18:08 CST -------
rsync-HEAD-20100102-1909GMT looks better on Tru64.  (Should be similarly
better on HP-UX.)  Thanks.

There are still a few warnings with GCC on Tru64.  I haven't looked
closely to see where to assign blame.  Let me know if a closer look is
justified.

urtx# gcc -v
Using built-in specs.
Target: alphaev6-dec-osf5.1b
Configured with: ../gcc-4.3.2/configure
--enable-languages=c,c++,objc,obj-c++,tr
eelang
Thread model: posix
gcc version 4.3.2 (GCC)

[...]
gcc -std=gnu99 -I. -I. -g -O2 -DHAVE_CONFIG_H -Wall -W -I./popt
-Wno-unused-para
meter  -c uidlist.c -o uidlist.o
uidlist.c: In function 'recv_add_id':
uidlist.c:210: warning: signed and unsigned type in conditional expression
uidlist.c:213: warning: signed and unsigned type in conditional expression
uidlist.c: In function 'match_uid':
uidlist.c:242: warning: comparison between signed and unsigned
uidlist.c: In function 'match_gid':
uidlist.c:257: warning: comparison between signed and unsigned
uidlist.c:261: warning: comparison between signed and unsigned
uidlist.c: In function 'add_uid':
uidlist.c:284: warning: comparison between signed and unsigned
uidlist.c: In function 'add_gid':
uidlist.c:302: warning: comparison between signed and unsigned
[...]

/usr/include/sys/types.h says:

[...]
#ifdef _XOPEN_SOURCE_EXTENDED
typedef int             uid_t;          /* user ID */
#else
typedef uint_t          uid_t;          /* user ID */
#endif
[...]
typedef uint_t          gid_t;          /* group ID */
[...]

So, I'd expect both to be uint_t.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.
--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 0 comments |
headers
doug | 4 Jan 2010 21:33
Favicon

link-dest question

The command below was run on FreeBSD 7.2 with rsync-3.0.3.

The command works but copies all files, i.e. no hard links created.

   rsync -vaxH --numeric-ids --delete --exclude=/home/apache/logs \
    --link-dest=/mnt/200912/newharbor/home -e ssh newharbor.safeport.com:/home \
    201001/newharbor/home/

The command was run without the continuation breaks that I added for 
readability. The command ended with:

sent 3883375 bytes  received 36841205930 bytes  1001130.04 bytes/sec
total size is 36840981254  speedup is 1.00
rsync warning: some files vanished before they could be transferred (code 24) at 
main.c(1505) [6]

I assume this is my error but can not find anything wrong. At most 300-500MB 
should have been transfered.

Thank you for any thoughts

DougD

_____
Douglas Denault
http://www.safeport.com
doug <at> safeport.com
Voice: 301-217-9220
   Fax: 301-217-9277

--

-- 
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Permalink | Reply | 5 comments |

Gmane