Harry Putnam | 11 Jan 22:48 2015

about running rsync on both ends of an rsnapshot script.

I've been told a few times that running rsync on both ends of an
rsnapshot pull, is most efficient.  And was told that both methods of
doing that are explained in the rsync manual.

I guess it is, but I definitely did not come away from the manual
knowning how to do it.

And it is not even attempted in the rsnapshot manual.

Can anyone post some simple examples of how an rsnapshot.conf looks
that does both kinds of engaging rsync on the remote.

1) rsync to running rsync daemon
and
2) rsync thru ssh terminal on the remote.

I'd like to see how that is done inside rsnapshot.conf.

PS - example on the rsync cmdline would be a big plus too.

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
Harry Putnam | 7 Jan 03:57 2015

rsnapshot push from linux lchost to Solaris Remote backup server

If I wanted to run rsnapshot on a local linux host but send the
backups to a Solaris backup server with zfs file system what do I need
to do with rsnapshot_root

Just trying local lan address there with path to destination fails
immediately

snapshot_root   2x.local.lan:/rrsnap/gv

Must need some trickier line... but googling a while and all I'm
finding is lots of folks who only want to talk about working with
rsnapshot_root on localhost.

Whats the trick to it?

To try to make it a little clearer.

The host I want to backup is an Gentoo linux host.

I'm rsnaping lchost:/var lchost:/etc [...]

I was doing the runs on the local linux host and parking snapshot_root
there.

Then having a cron job on the solaris zfs server pulling that
rsnapshot_root over to the solaris host... each day, just using rsync.
But still keeping the solaris directory synced right up with
rsnapshot_root.

That seems like more work than I need to be doing.
(Continue reading)

Scott Hess | 6 Jan 23:36 2015

Re: Inclusively adding excludes.

On Mon, Jan 5, 2015 at 4:00 PM, Scott Hess <scott <at> doubleu.com> wrote:
I've been reading around in the code, but I haven't found an obvious thing I'm missing.  AFAICT you can't just + any old option successfully.  Currently I've just duplicated the global excludes into each backup, which is obviously not the way to go in the long term.  AFAICT, if I use a global exclude_file (/etc/rsnapshot.excludes) with per-backup exclude= rules, everything works as I'd like it to, so that's probably where I'll take things.

Oops - I wasn't reading things correctly, using per-backup exclude= also drops global exclude_file.  Fortunately, using an exclude_file is easier to mix with per-backup settings, because I can use exclude=local,exclude=local,exclude_file=/etc/rsnapshot.exclude

-scott

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Chris Tebb | 6 Jan 15:08 2015
Picon

New server - carrying on where you left off

Hi all,
So I’ve got a new server and have rsnapshot ready to go with the same configuration as the old server.

My plan is to simply create JUST /snapshots/daily.0 on the new server and rsync that over from the old server using:

rsync -azvvh --progress --delete --numeric-ids old.ip.old.ip:/snapshots/daily.0/ /snapshots/daily.0/

This might take a few passes and won’t be run during a backup so it’ll just keep this new daily.0 up to date with the current daily.0 on the old server until I’m happy everything has copied over.

The rsync command is setup I think to ignore hard links and just copy over every single file from daily.0 to the new server.
Then I’ll just install my crontab and let it take the first backup, effectively carrying on from where the old server left off.

Does this seem like the right solution and can anyone see any obvious gotchas with this approach?

Many thanks,
Chris.

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Andrew McGlashan | 6 Jan 01:09 2015
Picon

Re: Inclusively adding excludes.

Hi,

On 6/01/2015 11:00 AM, Scott Hess wrote:
> I recently added a line like:
>     exclude nobackup
> to rsnapshot.conf, because I was experimenting with storing an
> encryption key on a disk (*).  A few days later it occurred to me to
> double-check, and the nobackup directory was being backed up!

I use the rsync -F option (with other options), this allows me to use
.rsync-filter files at the top of each file system to exclude files
and/or directories.

Cheers
A.

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
Scott Hess | 6 Jan 01:00 2015

Inclusively adding excludes.

I recently added a line like:
    exclude nobackup
to rsnapshot.conf, because I was experimenting with storing an encryption key on a disk (*).  A few days later it occurred to me to double-check, and the nobackup directory was being backed up!

What I tracked it down to was that I was doing things like:

   exclude nobackup
   ...
   backup rsnapshot <at> hostname:/home/ hostname/home exclude=.cache,exclude=.ccache,etc

The backup line's excludes replace the global excludes, which was not what I expected (I expected to exclude the union).  I consulted the man page and found an example about using "+rsync_long_args=--exclude=/var/spool/" to add per-backup excludes, which implies that this is a known issue.

I've been reading around in the code, but I haven't found an obvious thing I'm missing.  AFAICT you can't just + any old option successfully.  Currently I've just duplicated the global excludes into each backup, which is obviously not the way to go in the long term.  AFAICT, if I use a global exclude_file (/etc/rsnapshot.excludes) with per-backup exclude= rules, everything works as I'd like it to, so that's probably where I'll take things.

Does all that make sense?  Did I miss a trick which is going to horribly mutilate my backups?

-scott

(*) My external mirror, which I rotate off-site.  So I wanted to make it easy to use when on-site, but secure off-site.  Admittedly, having the encryption key stored in an encrypted disk is merely stupid, rather than fatal, just trying to be appropriately paranoid.
------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Nico Kadel-Garcia | 1 Jan 00:57 2015

Testing updated rsnapshot "chef" cookbook

This is my work address: I'm being very cautious to segregate any work I do that is personal from work, and
this is work related.

I've updated a chef cookbook that handles rsnapshot configurations, and would seriously welcome any chef
users to test it. It's at:

                     https://github.com/nkadel-skyhook/nkadel-chef-rsnapshot/tree/v0.1.2

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel <at> skyhookwireless.com

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
Harry Putnam | 30 Dec 16:13 2014

rsnapshot script runs part way through and hangs <eternally>

Setup: Gentoo linux running as vbox guest on win7 64bit

I'm having an odd problem with rsnapshot that I don't recall ever
running into before.

my rsnapshot.conf (posted at the end) runs 2 backup.... It starts
running the first but part way thru.. not at the same spot far as I
can tell.

If I leave it run, it never finishes.  If I kill it with
interupsts... it takes two.  So both runs are hitting some spot in the
process and hanging.

If I check processes I see (wrapped for mail):

ps wwaux|grep rsync

  root 7033 0.0 0.0 13516 2956 pts/4 S+ 09:32 0:00 /usr/bin/rsync
  -avvlRv --stats --exclude-from=/etc/rsnap/ex/rsnap_Misc_exclude
  /usr/local/common/merc/ /bk/rsnap/misc/hourly.0/gv/

  root 7034 0.0 0.0 7788 2604 pts/4 S+ 09:32 0:00 /usr/bin/rsync -avvlRv
  --stats --exclude-from=/etc/rsnap/ex/rsnap_Misc_exclude
  /usr/local/common/merc/ /bk/rsnap/misc/hourly.0/gv/

  root 7035 0.0 0.0 13720 2676 pts/4 S+ 09:32 0:00 /usr/bin/rsync
  -avvlRv --stats --exclude-from=/etc/rsnap/ex/rsnap_Misc_exclude
  /usr/local/common/merc/ /bk/rsnap/misc/hourly.0/gv/

By the directory level I can see they are all running in the same
phase of the backup

I've used nearly the defaults for flags with the exception of adding
adding -vv (-l) --links, promoting `-relative' (-R), to short_args and
adding `--stats' to long_args.

I've got rsnapshots verbosity and log level set to 4.

However, tailing the designated log, there isn't enough yet releases
when the hang happens to tell me anything very usefull:

Then after ^c more is unbuffered but having to do whith the unexpected
interrupt.

In neither case it helping my understand what is happening.
(log output below)

So what can anyone suggest to solve the problem?

-------       -------       ---=---       -------       -------
 (output of tailing rsnapshot log: wrapped for mail)

  [30/Dec/2014:10:02:05] /usr/bin/rsnapshot -c
  /etc/rsnap/rc/rsnap_Misc.conf hourly: started

  [30/Dec/2014:10:02:05] Setting locale to POSIX "C"

  [30/Dec/2014:10:02:05] echo 7123 > /var/run/rsnapshot.pid

  [30/Dec/2014:10:02:05] mv /bk/rsnap/misc/hourly.3/
  /bk/rsnap/misc/hourly.4/
  
  [30/Dec/2014:10:02:05] mv /bk/rsnap/misc/hourly.2/
  /bk/rsnap/misc/hourly.3/
  
  [30/Dec/2014:10:02:05] mv /bk/rsnap/misc/hourly.1/
  /bk/rsnap/misc/hourly.2/
  
  [30/Dec/2014:10:02:05] /bin/cp -al /bk/rsnap/misc/hourly.0
  /bk/rsnap/misc/hourly.1

  [30/Dec/2014:10:02:05] /usr/bin/rsync -avvlRv --stats
  --exclude-from=/etc/rsnap/ex/rsnap_Misc_exclude /merb/
  /bk/rsnap/misc/hourly.0/gv/
  
  [30/Dec/2014:10:02:05] /usr/bin/rsync -avvlRv --stats
  --exclude-from=/etc/rsnap/ex/rsnap_Misc_exclude
  /usr/local/common/merc/ /bk/rsnap/misc/hourly.0/gv/
-------       -------       ---=---       -------       -------

After interrupt:

  [...]
  [30/Dec/2014:10:06:06] /usr/bin/rsnapshot -c
  /etc/rsnap/rc/rsnap_Misc.conf hourly: ERROR: /usr/bin/rsync returned
  20 while processing /usr/local/common/merc

  [30/Dec/2014:10:06:06] /usr/bin/logger -i -p user.err -t rsnapshot
  /usr/bin/rsnapshot -c /etc/rsnap/rc/rsnap_Misc.conf hourly: ERROR:
  /usr/bin/rsync returned 20 while processing /usr/local/common/merc

  [30/Dec/2014:10:06:06] touch /bk/rsnap/misc/hourly.0/
  
  [30/Dec/2014:10:06:06] rm -f /var/run/rsnapshot.pid

  [30/Dec/2014:10:06:06] /usr/bin/logger -i -p user.err -t rsnapshot
  /usr/bin/rsnapshot -c /etc/rsnap/rc/rsnap_Misc.conf hourly: ERROR:
  /usr/bin/rsnapshot -c /etc/rsnap/rc/rsnap_Misc.conf hourly: completed,
  but with some errors

  [30/Dec/2014:10:06:06] /usr/bin/rsnapshot -c
  /etc/rsnap/rc/rsnap_Misc.conf hourly: ERROR: /usr/bin/rsnapshot -c
  /etc/rsnap/rc/rsnap_Misc.conf hourly: completed, but with some errors

-------       -------       ---=---       -------       -------

Content of rsnap_Misc.conf

root # grep '^ *[^#]' /etc/rsnap/rc/rsnap_Misc.conf

config_version  1.2
snapshot_root   /bk/rsnap/misc
cmd_cp          /bin/cp
cmd_rm          /bin/rm
cmd_rsync       /usr/bin/rsync
cmd_logger      /usr/bin/logger
cmd_du          /usr/bin/du
cmd_rsnapshot_diff      /usr/bin/rsnapshot-diff
interval        hourly  6
interval        daily   7
interval        weekly  4
interval        monthly 3
verbose         4
loglevel        4
logfile /var/log/rsnap_Misc.log
lockfile        /var/run/rsnapshot.pid
rsync_short_args        -avvlR
rsync_long_args --stats --delete        --numeric-ids   --delete-excluded
exclude_file    /etc/rsnap/ex/rsnap_Misc_exclude
backup          /merb/                  gv/
backup          /usr/local/common/merc  gv/

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
Chris Tebb | 10 Dec 11:23 2014
Picon

Moving an rsnapshot root to a new server

Hi rsnapshot wizards!

I’m moving our backups to a new server with more space, I’ve got about 5TB of data to move so it’ll take
some time to migrate though I’m hoping to do it within 24 hours so it will not be copying during a backup.

From what I have read, the two options are a simple:

rsync -az -H --delete --numeric-ids /snapshots/ new.ser.ver.ip:/snapshots/

So using compression, archive, —delete (in case it needs to be re-run after a backup, which I’m not
planning) and critically -H for the hard links.

Or there is a utility called rsnapshot-copy which looks like it’s supposed to do the job I’m attempting
but I can’d find a lot of documentation around it and even fewer reported uses of it.

Is snapshot-copy simply designed to continue a copy when it’s been interrupted by a backup taking place
(therefore the cycling of the directories will cause a normal rsync to be confused?)

I’m thinking that the safest bet would be to simply disable cron on the old server as soon as it’s
completed a backup and then start the rsync, enabling cron on the new server when the copy is complete.

So my questions are:

1. Provided I can get the transfer done in-between backups, it that simple rsync command the correct one to use?
2. Is there any other reason to use the snapshot-copy utility?

Many thanks!

Chris.

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Aurelien Martin | 5 Dec 18:17 2014
Picon

rsnapshot marked as broken with perl 5.18 on FreeBSD

Hello,

I centralize my backup on a FreeBSD 10.0 machine, and I'm very happy 
with it !

But few days ago they have updated the perl version, and now the package 
is marked as broken on perl 5.18 and newer
http://www.freshports.org/sysutils/rsnapshot/
It cannot be build anymore

Can someone have a look ? I put the maintainer in copy

Cheers,Aurelien

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
Chris Tebb | 4 Dec 17:52 2014
Picon

Deleting no longer required backup intervals

Hi, we had out monthly retain set to 24 but we’ve decided to go back to 12, can I simply:

/bin/rm -rf /snapshots/monthly.13/

For all months outside of 12 without affecting the rest of the backups?

Thanks!

Chris.
--
chrispyfur.net

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Gmane