[rrd-users] Locking *read* (copy/backup) of RRD files?

[Jochen Bern]

On 10/25/2010 11:51 PM, Steve Shipway wrote:
>> I'd like to be able to make backups of RRD files while Nagios is
>> happily writing data to them.
> 
> This could be a good application of rrd cache chaining, if I can get
> it working well.  This would allow you to set up your rrdcached to
> chain to another local rrdcached which holds the backup; everything
> writes via the first rrdcached.  If you want to backup, you kill the
> secondary, backup the files, and restart it.
> 
> The trouble with this is that the rrdcached chaining code (1) gets
> upset if the remote rrdcached disappears, and (2) is not yet mature
> enough to go into trunk.

I'm afraid that I see a (3): In the constellation you describe, the RRD
files that get backed up would accumulate "holes" for every timeframe
where the backup was run, holes which are absent from the primary set of
RRD files. And the fact that you're blocking *all* RRD files for
whatever time the *entire* backup run takes sort of maximizes the
problem ...

> It might be better to add an 'rrdtool copy' function that locks the RRD
> and copies the file for backup?

As a matter of fact, I'm a bit surprised to see that rrdresize seems to
be the *only* part of rrdtool which sets a(ny) lock while *reading* an
RRD file. I would imagine that every reader which makes use of the most
dynamic parts of the file - the ones corresponding to live_head_t and
pdp_prep_t at a minimum, maybe cdp_prep_t and rra_ptr_t as well - should
try to prevent inconsistencies introduced by concurrent writes, and
rrddump is one such reader ...

Kind regards,
								J. Bern
-- 
Jochen Bern, Systemingenieur --- LINworks GmbH <http://www.LINworks.de/>
Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt
PGP (1024D/4096g) FP = D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C27
Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202
Unternehmenssitz Weiterstadt, Geschäftsführer Metin Dogan, Oliver Michel