[rrd-users] Incremental backup rrd file

[ccddtt]

Hi,

Thanks!
Very sorry, I did not carefully read the RSYNC document.I have just tested, RSYNC is really only to send the updated file block.
very grateful to you for help, and this solves the problem I wrote the code.(this is a happy thing)

By the way, RSYNC command parameters I use the rsync -azh ,
the -z is 
-z, --compress              compress file data during the transfer

I think use this good.

and I not use ssh over rsync. ssh encrypted data packet, it will increase my traffic.

Thanks!




2012-07-18

 



发件人： Ryan Kubica
发送时间： 2012-07-18 00:05:47
收件人： Simon Hobson; rrd-users at lists.oetiker.ch
抄送： 
主题： Re: [rrd-users] Incremental backup rrd file



I'll second all of this about rsync: it's very efficient and 'safe' for rrd data copies.


I don't have backups per-se, I run active mirrored rrd servers with millions of rrd datafiles per server and if one crashes where I need to rebuild one or install a new one for hardware upgrade like I'm doing today, then I use rsync to get a copy from another mirror ... actively.  The replacement-mirror writes behind in the rrd update queue so it's updating older intervals than the rest of the cluster and then I copy from another mirror.  I'm currently copying 1TB (one terabyte) and it works beautifully.


rsync would take a long time to do backups nightly of that many files (which is why it's not done); but on a few thousand'ish it can(should!) be used.


If you use rsync over ssh, at least do something like this: rsync -ave 'ssh -c blowfish' src dst


I've yet to bother with rsync daemon with no ssh, though that'd be more efficient as well.


-Ryan




From: Simon Hobson <linux at thehobsons.co.uk>
To: rrd-users at lists.oetiker.ch 
Sent: Tuesday, July 17, 2012 4:47 AM
Subject: Re: [rrd-users] Incremental backup rrd file


Darren Murphy wrote:
>Just to add a little to this, the --stats & --human-readable options
>provide useful insight as to the efficiency of rsync
<snip>
>So 3121 files totaling 4.3GB in size, and at least 90% of those files
>would change between successive sync runs, yet only a very small
>amount of data needs to be transferred.

That tallies with my experience. Obviously it varies considerably 
with the type of data, but I've yet to find something where it 
doesn't show a reduction in data transferred.
In general, RRD files should 'compress' quite well (unless you use 
very small consolidations).

>I'd also add that in my experience rsync is incredibly robust and reliable.
>I've been running an hourly rsync from my main MRTG server to 3
>separate "slaves" for almost 2 years now, and never once had a problem
>with data integrity.

I'll second that. And of course, even if the process dies part way 
through, you can just run it again and it will catch up.

-- 
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
rrd-users mailing list
rrd-users at lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20120718/7aeea7ef/attachment-0001.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/gif
Size: 1662 bytes
Desc: not available
Url : http://lists.oetiker.ch/pipermail/rrd-users/attachments/20120718/7aeea7ef/attachment-0001.gif