Chris Tebb | 6 Jan 15:08 2015
Picon

New server - carrying on where you left off

Hi all,
So I’ve got a new server and have rsnapshot ready to go with the same configuration as the old server.

My plan is to simply create JUST /snapshots/daily.0 on the new server and rsync that over from the old server using:

rsync -azvvh --progress --delete --numeric-ids old.ip.old.ip:/snapshots/daily.0/ /snapshots/daily.0/

This might take a few passes and won’t be run during a backup so it’ll just keep this new daily.0 up to date with the current daily.0 on the old server until I’m happy everything has copied over.

The rsync command is setup I think to ignore hard links and just copy over every single file from daily.0 to the new server.
Then I’ll just install my crontab and let it take the first backup, effectively carrying on from where the old server left off.

Does this seem like the right solution and can anyone see any obvious gotchas with this approach?

Many thanks,
Chris.

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Andrew McGlashan | 6 Jan 01:09 2015
Picon

Re: Inclusively adding excludes.

Hi,

On 6/01/2015 11:00 AM, Scott Hess wrote:
> I recently added a line like:
>     exclude nobackup
> to rsnapshot.conf, because I was experimenting with storing an
> encryption key on a disk (*).  A few days later it occurred to me to
> double-check, and the nobackup directory was being backed up!

I use the rsync -F option (with other options), this allows me to use
.rsync-filter files at the top of each file system to exclude files
and/or directories.

Cheers
A.

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
Scott Hess | 6 Jan 01:00 2015

Inclusively adding excludes.

I recently added a line like:
    exclude nobackup
to rsnapshot.conf, because I was experimenting with storing an encryption key on a disk (*).  A few days later it occurred to me to double-check, and the nobackup directory was being backed up!

What I tracked it down to was that I was doing things like:

   exclude nobackup
   ...
   backup rsnapshot <at> hostname:/home/ hostname/home exclude=.cache,exclude=.ccache,etc

The backup line's excludes replace the global excludes, which was not what I expected (I expected to exclude the union).  I consulted the man page and found an example about using "+rsync_long_args=--exclude=/var/spool/" to add per-backup excludes, which implies that this is a known issue.

I've been reading around in the code, but I haven't found an obvious thing I'm missing.  AFAICT you can't just + any old option successfully.  Currently I've just duplicated the global excludes into each backup, which is obviously not the way to go in the long term.  AFAICT, if I use a global exclude_file (/etc/rsnapshot.excludes) with per-backup exclude= rules, everything works as I'd like it to, so that's probably where I'll take things.

Does all that make sense?  Did I miss a trick which is going to horribly mutilate my backups?

-scott

(*) My external mirror, which I rotate off-site.  So I wanted to make it easy to use when on-site, but secure off-site.  Admittedly, having the encryption key stored in an encrypted disk is merely stupid, rather than fatal, just trying to be appropriately paranoid.
------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Nico Kadel-Garcia | 1 Jan 00:57 2015

Testing updated rsnapshot "chef" cookbook

This is my work address: I'm being very cautious to segregate any work I do that is personal from work, and
this is work related.

I've updated a chef cookbook that handles rsnapshot configurations, and would seriously welcome any chef
users to test it. It's at:

                     https://github.com/nkadel-skyhook/nkadel-chef-rsnapshot/tree/v0.1.2

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel <at> skyhookwireless.com

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
Harry Putnam | 30 Dec 16:13 2014

rsnapshot script runs part way through and hangs <eternally>

Setup: Gentoo linux running as vbox guest on win7 64bit

I'm having an odd problem with rsnapshot that I don't recall ever
running into before.

my rsnapshot.conf (posted at the end) runs 2 backup.... It starts
running the first but part way thru.. not at the same spot far as I
can tell.

If I leave it run, it never finishes.  If I kill it with
interupsts... it takes two.  So both runs are hitting some spot in the
process and hanging.

If I check processes I see (wrapped for mail):

ps wwaux|grep rsync

  root 7033 0.0 0.0 13516 2956 pts/4 S+ 09:32 0:00 /usr/bin/rsync
  -avvlRv --stats --exclude-from=/etc/rsnap/ex/rsnap_Misc_exclude
  /usr/local/common/merc/ /bk/rsnap/misc/hourly.0/gv/

  root 7034 0.0 0.0 7788 2604 pts/4 S+ 09:32 0:00 /usr/bin/rsync -avvlRv
  --stats --exclude-from=/etc/rsnap/ex/rsnap_Misc_exclude
  /usr/local/common/merc/ /bk/rsnap/misc/hourly.0/gv/

  root 7035 0.0 0.0 13720 2676 pts/4 S+ 09:32 0:00 /usr/bin/rsync
  -avvlRv --stats --exclude-from=/etc/rsnap/ex/rsnap_Misc_exclude
  /usr/local/common/merc/ /bk/rsnap/misc/hourly.0/gv/

By the directory level I can see they are all running in the same
phase of the backup

I've used nearly the defaults for flags with the exception of adding
adding -vv (-l) --links, promoting `-relative' (-R), to short_args and
adding `--stats' to long_args.

I've got rsnapshots verbosity and log level set to 4.

However, tailing the designated log, there isn't enough yet releases
when the hang happens to tell me anything very usefull:

Then after ^c more is unbuffered but having to do whith the unexpected
interrupt.

In neither case it helping my understand what is happening.
(log output below)

So what can anyone suggest to solve the problem?

-------       -------       ---=---       -------       -------
 (output of tailing rsnapshot log: wrapped for mail)

  [30/Dec/2014:10:02:05] /usr/bin/rsnapshot -c
  /etc/rsnap/rc/rsnap_Misc.conf hourly: started

  [30/Dec/2014:10:02:05] Setting locale to POSIX "C"

  [30/Dec/2014:10:02:05] echo 7123 > /var/run/rsnapshot.pid

  [30/Dec/2014:10:02:05] mv /bk/rsnap/misc/hourly.3/
  /bk/rsnap/misc/hourly.4/
  
  [30/Dec/2014:10:02:05] mv /bk/rsnap/misc/hourly.2/
  /bk/rsnap/misc/hourly.3/
  
  [30/Dec/2014:10:02:05] mv /bk/rsnap/misc/hourly.1/
  /bk/rsnap/misc/hourly.2/
  
  [30/Dec/2014:10:02:05] /bin/cp -al /bk/rsnap/misc/hourly.0
  /bk/rsnap/misc/hourly.1

  [30/Dec/2014:10:02:05] /usr/bin/rsync -avvlRv --stats
  --exclude-from=/etc/rsnap/ex/rsnap_Misc_exclude /merb/
  /bk/rsnap/misc/hourly.0/gv/
  
  [30/Dec/2014:10:02:05] /usr/bin/rsync -avvlRv --stats
  --exclude-from=/etc/rsnap/ex/rsnap_Misc_exclude
  /usr/local/common/merc/ /bk/rsnap/misc/hourly.0/gv/
-------       -------       ---=---       -------       -------

After interrupt:

  [...]
  [30/Dec/2014:10:06:06] /usr/bin/rsnapshot -c
  /etc/rsnap/rc/rsnap_Misc.conf hourly: ERROR: /usr/bin/rsync returned
  20 while processing /usr/local/common/merc

  [30/Dec/2014:10:06:06] /usr/bin/logger -i -p user.err -t rsnapshot
  /usr/bin/rsnapshot -c /etc/rsnap/rc/rsnap_Misc.conf hourly: ERROR:
  /usr/bin/rsync returned 20 while processing /usr/local/common/merc

  [30/Dec/2014:10:06:06] touch /bk/rsnap/misc/hourly.0/
  
  [30/Dec/2014:10:06:06] rm -f /var/run/rsnapshot.pid

  [30/Dec/2014:10:06:06] /usr/bin/logger -i -p user.err -t rsnapshot
  /usr/bin/rsnapshot -c /etc/rsnap/rc/rsnap_Misc.conf hourly: ERROR:
  /usr/bin/rsnapshot -c /etc/rsnap/rc/rsnap_Misc.conf hourly: completed,
  but with some errors

  [30/Dec/2014:10:06:06] /usr/bin/rsnapshot -c
  /etc/rsnap/rc/rsnap_Misc.conf hourly: ERROR: /usr/bin/rsnapshot -c
  /etc/rsnap/rc/rsnap_Misc.conf hourly: completed, but with some errors

-------       -------       ---=---       -------       -------

Content of rsnap_Misc.conf

root # grep '^ *[^#]' /etc/rsnap/rc/rsnap_Misc.conf

config_version  1.2
snapshot_root   /bk/rsnap/misc
cmd_cp          /bin/cp
cmd_rm          /bin/rm
cmd_rsync       /usr/bin/rsync
cmd_logger      /usr/bin/logger
cmd_du          /usr/bin/du
cmd_rsnapshot_diff      /usr/bin/rsnapshot-diff
interval        hourly  6
interval        daily   7
interval        weekly  4
interval        monthly 3
verbose         4
loglevel        4
logfile /var/log/rsnap_Misc.log
lockfile        /var/run/rsnapshot.pid
rsync_short_args        -avvlR
rsync_long_args --stats --delete        --numeric-ids   --delete-excluded
exclude_file    /etc/rsnap/ex/rsnap_Misc_exclude
backup          /merb/                  gv/
backup          /usr/local/common/merc  gv/

------------------------------------------------------------------------------
Dive into the World of Parallel Programming! The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net
Chris Tebb | 10 Dec 11:23 2014
Picon

Moving an rsnapshot root to a new server

Hi rsnapshot wizards!

I’m moving our backups to a new server with more space, I’ve got about 5TB of data to move so it’ll take
some time to migrate though I’m hoping to do it within 24 hours so it will not be copying during a backup.

From what I have read, the two options are a simple:

rsync -az -H --delete --numeric-ids /snapshots/ new.ser.ver.ip:/snapshots/

So using compression, archive, —delete (in case it needs to be re-run after a backup, which I’m not
planning) and critically -H for the hard links.

Or there is a utility called rsnapshot-copy which looks like it’s supposed to do the job I’m attempting
but I can’d find a lot of documentation around it and even fewer reported uses of it.

Is snapshot-copy simply designed to continue a copy when it’s been interrupted by a backup taking place
(therefore the cycling of the directories will cause a normal rsync to be confused?)

I’m thinking that the safest bet would be to simply disable cron on the old server as soon as it’s
completed a backup and then start the rsync, enabling cron on the new server when the copy is complete.

So my questions are:

1. Provided I can get the transfer done in-between backups, it that simple rsync command the correct one to use?
2. Is there any other reason to use the snapshot-copy utility?

Many thanks!

Chris.

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Aurelien Martin | 5 Dec 18:17 2014
Picon

rsnapshot marked as broken with perl 5.18 on FreeBSD

Hello,

I centralize my backup on a FreeBSD 10.0 machine, and I'm very happy 
with it !

But few days ago they have updated the perl version, and now the package 
is marked as broken on perl 5.18 and newer
http://www.freshports.org/sysutils/rsnapshot/
It cannot be build anymore

Can someone have a look ? I put the maintainer in copy

Cheers,Aurelien

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
Chris Tebb | 4 Dec 17:52 2014
Picon

Deleting no longer required backup intervals

Hi, we had out monthly retain set to 24 but we’ve decided to go back to 12, can I simply:

/bin/rm -rf /snapshots/monthly.13/

For all months outside of 12 without affecting the rest of the backups?

Thanks!

Chris.
--
chrispyfur.net

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Chris Tebb | 25 Nov 14:03 2014
Picon

Deleting a folder from all backup sets

Hi all!

I”m trying to figure out the best way to remove a particular folder from all backup sets.

So I have a backup of a server that contains the folder /home/something/massivespacehog that is no longer
required but it’s present in 30 daily and 14 monthly backup sets.

I have deleted the folder from the server that is being backed up but it’s obviously still on the backup
server and I don’t want to change my rsnapshot configuration.

Is there a “best practice” way of pruning this folder from all my backup sets?

Thanks,
		Chris.

--
chrispyfur.net

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss
Gordon Messmer | 22 Nov 19:10 2014
Picon

Snapshots (lvm and more)

Last year, I wrote a "snapshot" utility for backups.  There are a bunch 
of how-tos on the internet that describe how to make snapshots of your 
filesystems so that you don't need to dump server data (like SQL 
databases) to text files for backup. There isn't, however, a standard 
interface for doing so.

I want to change that.

snapshot provides a standard interface to making your data consistent, 
taking snapshots, making them available for backup, and cleaning up the 
snapshots on completion.

I've written integration for rsnapshot, since that's what I use in a lot 
of places, and I'm going to work on Bacula next.

I'd like to get feedback, testing, feature requests, etc from people 
interested in getting good, usable backups of their data.

https://bitbucket.org/gordonmessmer/dragonsdawn-snapshot
https://github.com/DrHyde/rsnapshot/pull/44

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
Timmy O'Mahony | 13 Nov 15:41 2014

sync_first and sync command is backing up every file every time (not just changed files)

I am performing SSH backups from my laptop to my home server. Everything is configured and working.
 
I've enabled sync_first=1 which I understand means that when I want to perform an hourly/daily/weekly backup, I first need to run rsnapshot sync to actually perform the backup (copying remote files to a .sync folder). Only then do I run the rsnapshot weekly/hourly/daily command which simply updates the symlinks to the various hourly.0 etc. folders
 
My problem is that every time I run sync  my entire laptop is backed up every time. This means that every backup takes a huge amount of time. I would have assumed that the sync command took into consideration what has changed since the last sync and only grabbed those particular files. Is this the correct behaviour or am I misusing this command?
 
Thanks,
 
Timmy
 
 
 
 
------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk
_______________________________________________
rsnapshot-discuss mailing list
rsnapshot-discuss <at> lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rsnapshot-discuss

Gmane