[rrd-users] Disabling Last Update
linux at thehobsons.co.uk
Thu Nov 8 09:13:53 CET 2012
Wesley Wyche wrote:
>Is there a way to disable the last update value (or at least override it
>somehow)? I need to insert/update data in a NON-sequential manner based
>upon MY time value, not a step from last update.
No, but updates can be done on sub-second time values.
>The scenario is this:
>I have data coming into a landing area in batches. Those batches are data
>updates for thousands of rrds and they come in several per minute. However,
>there may be times where there is a backlog of batches that need to be
>processed offline perhaps due to a server outage or processing requirements
>Even if I'm processing data that is "current", there are still multiple
>batch files for that single minute of time. We could be processing the data
>out of order within that single minute because the sample rate is every 20
>seconds and I could update the rrd with the wrong order (due simply by
>processing that minutes batches in alphanumeric order by filename).
>It would be awesome if there was a way I didn't have to simply chunk hours
>of data batches away if I have a processing server go down and i can't keep
>up with the volume.
RRD does not make you throw the data away, your faulty
processing/feed system is doing that.
The nature of the RRD file is such that it really is not possible to
do historical updates - at least not like you propose (it would
always be possible to go back and edit prior consolidated data, but
RRC tools do not provide anything to do that).
The obvious solution is for you to fix your file naming convention to
include sub-second timing elements that you can sort on - I assume at
the moment you have something along the lines of yyyymmddhhmm in your
filename, while it sounds like you need yyyymmddhhmmt where t is a
tenth of a second, or even yyyymmddhhmmttt where ttt is thousandths
of a second (milliseconds).
A bit more on why youy can't do non-sequential updates ...
Well RRD does not store any of the data you enter, never, not at all.
As you feed in values, it accumulates them in a single register until
a step in time is complete. AT that point, it performs whatever
consolidations are required, stores the consolidated value in a
consolidated data series, and resets the accumulator (which will be
retaining some of the previous value). There's no "store a bunch of
values and then process them when told to, every time you do an
update, everything is processed - even if that process is merely to
add the current update to the accumulator.
It is possible to make these operations null (update only on step
boundaries, and on every step boundary), but the process is still
done, it's just a case of x/1=x as in divide by 1 is a null operation.
Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.
More information about the rrd-users