[rrd-users] Incremental backup rrd file

[Steven Hartland]

If your running on ZFS another alternative is snapshots with send and recv which should be even more efficient than rsync as it doesn't need to calculate the changes it just "knows".

    Regards
    Steve
  ----- Original Message ----- 
  From: Ryan Kubica 
  To: Simon Hobson ; rrd-users at lists.oetiker.ch 
  Sent: Tuesday, July 17, 2012 5:05 PM
  Subject: Re: [rrd-users] Incremental backup rrd file




  I'll second all of this about rsync: it's very efficient and 'safe' for rrd data copies.


  I don't have backups per-se, I run active mirrored rrd servers with millions of rrd datafiles per server and if one crashes where I need to rebuild one or install a new one for hardware upgrade like I'm doing today, then I use rsync to get a copy from another mirror ... actively.  The replacement-mirror writes behind in the rrd update queue so it's updating older intervals than the rest of the cluster and then I copy from another mirror.  I'm currently copying 1TB (one terabyte) and it works beautifully.


  rsync would take a long time to do backups nightly of that many files (which is why it's not done); but on a few thousand'ish it can(should!) be used.


  If you use rsync over ssh, at least do something like this: rsync -ave 'ssh -c blowfish' src dst


  I've yet to bother with rsync daemon with no ssh, though that'd be more efficient as well.


  -Ryan



------------------------------------------------------------------------------
  From: Simon Hobson <linux at thehobsons.co.uk>
  To: rrd-users at lists.oetiker.ch 
  Sent: Tuesday, July 17, 2012 4:47 AM
  Subject: Re: [rrd-users] Incremental backup rrd file


  Darren Murphy wrote:
  >Just to add a little to this, the --stats & --human-readable options
  >provide useful insight as to the efficiency of rsync
  <snip>
  >So 3121 files totaling 4.3GB in size, and at least 90% of those files
  >would change between successive sync runs, yet only a very small
  >amount of data needs to be transferred.

  That tallies with my experience. Obviously it varies considerably 
  with the type of data, but I've yet to find something where it 
  doesn't show a reduction in data transferred.
  In general, RRD files should 'compress' quite well (unless you use 
  very small consolidations).

  >I'd also add that in my experience rsync is incredibly robust and reliable.
  >I've been running an hourly rsync from my main MRTG server to 3
  >separate "slaves" for almost 2 years now, and never once had a problem
  >with data integrity.

  I'll second that. And of course, even if the process dies part way 
  through, you can just run it again and it will catch up.

  -- 
  Simon Hobson

  Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
  author Gladys Hobson. Novels - poetry - short stories - ideal as
  Christmas stocking fillers. Some available as e-books.

  _______________________________________________
  rrd-users mailing list
  rrd-users at lists.oetiker.ch
  https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users





------------------------------------------------------------------------------


  _______________________________________________
  rrd-users mailing list
  rrd-users at lists.oetiker.ch
  https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users


================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. 

In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster at multiplay.co.uk.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20120718/4ae97d8a/attachment-0001.htm