[rrd-users] Incremental backup rrd file

[Ryan Kubica]

I'll second all of this about rsync: it's very efficient and 'safe' for rrd data copies.

I don't have backups per-se, I run active mirrored rrd servers with millions of rrd datafiles per server and if one crashes where I need to rebuild one or install a new one for hardware upgrade like I'm doing today, then I use rsync to get a copy from another mirror ... actively.  The replacement-mirror writes behind in the rrd update queue so it's updating older intervals than the rest of the cluster and then I copy from another mirror.  I'm currently copying 1TB (one terabyte) and it works beautifully.

rsync would take a long time to do backups nightly of that many files (which is why it's not done); but on a few thousand'ish it can(should!) be used.

If you use rsync over ssh, at least do something like this: rsync -ave 'ssh -c blowfish' src dst

I've yet to bother with rsync daemon with no ssh, though that'd be more efficient as well.

-Ryan


________________________________
 From: Simon Hobson <linux at thehobsons.co.uk>
To: rrd-users at lists.oetiker.ch 
Sent: Tuesday, July 17, 2012 4:47 AM
Subject: Re: [rrd-users] Incremental backup rrd file
 
Darren Murphy wrote:
>Just to add a little to this, the --stats & --human-readable options
>provide useful insight as to the efficiency of rsync
<snip>
>So 3121 files totaling 4.3GB in size, and at least 90% of those files
>would change between successive sync runs, yet only a very small
>amount of data needs to be transferred.

That tallies with my experience. Obviously it varies considerably 
with the type of data, but I've yet to find something where it 
doesn't show a reduction in data transferred.
In general, RRD files should 'compress' quite well (unless you use 
very small consolidations).

>I'd also add that in my experience rsync is incredibly robust and reliable.
>I've been running an hourly rsync from my main MRTG server to 3
>separate "slaves" for almost 2 years now, and never once had a problem
>with data integrity.

I'll second that. And of course, even if the process dies part way 
through, you can just run it again and it will catch up.

-- 
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users at lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20120717/fe51666b/attachment.htm