[rrd-users] Dynamic Steps

[Alex van den Bogaerdt]

----- Original Message ----- 
From: "Alan Finlay" <alan.websites2 at chankly.com>
To: "'Alex van den Bogaerdt'" <alex at vandenbogaerdt.nl>; 
<rrd-users at lists.oetiker.ch>
Sent: Friday, April 19, 2013 1:26 AM
Subject: RE: [rrd-users] Dynamic Steps


> Hi Alex,
>
> The rounding errors are due to my not using the timestamp to update, I was
> using N, with the values in the trace coming from Unix in a debug 
> statement.
>
> Clearly you understand something I don't.  The value 0.366 is 22/60 I take
> it?  But why should counts per second be stored?  The rrd db defs are for 
> a
> step size of 60 seconds, so I am expecting it to store counts per minute.

Why? Because that is how it works. Always.

Three clearly different steps to process your input: 
http://rrdtool.vandenbogaerdt.nl/process.php

Maybe it would have been better to call these steps "phases" so to not 
confuse step and step, but I'm not going to change it anymore now.

First phase: compute a rate from the input, using a set of rules such as 
"GAUGE", "COUNTER" and so on. The value of "step" is not used here.

Result: a rate. {something} per second. Yes, even when using RRDtool to 
store something like temperature, internally it is still handled as if it 
was a rate. We get away with this because we lie about the input: using 
GAUGE we are telling RRDtool the input is already {something per second} so 
don't touch. As a result, this "rate" goes to the next phase.

Sorry for sidestepping. I just want to show: no matter what parameters you 
use, it is ALWAYS going to be a rate of {something} per second.

We have a rate now, valid between the previous update and the current 
update. RRDtool does not work (by design) with arbitrary time intervals so 
this rate needs to be normalized into standard buckets of {step} seconds. 
And this is where, and for what purpose, {step} is used.

If you need higher precision, sample your input a lot (I mean: outside 
RRDtool), update a lot, and have a small {step} value (e.g. one second). See 
the web page I directed you to, the paragraph starting with "If you think it 
isn't ....", the part about the red rate could be twice as high as sampled. 
If it is important to know the "real rate" (really only a better 
approximation), for instance in the case of a MAX consolidation function, 
then step should be really low *and* the input needs to be sampled often.

Another sidestep: when updating multiple times per second, it could be 
useful to have {step} be smaller than one second. I don't know, and can 
currently not find out for myself, if this is possible. Please fill me in.

The third and final phase is to build CDPs from PDPs, and store these CDPs 
into an RRA. Even if you use 3600 PDPs of one second each, the resulting CDP 
will still have a rate in {something} per second. This rate is just valid 
during a whole hour.

This said, if you want to see "per minute" (or per hour, per 86400 seconds 
for a day, per 604800 seconds for a week) then just multiply your input, or 
the resulting rates, by the appropriate amount of seconds per unit you 
choose. After all, 0.3666666666666.... {something}s per second multiplied by 
60 seconds, results in 22 {something}s.

Although this is not a rule of some sorts, I prefer to keep the input 'as 
is', and do such multiplications in a CDEF:   CDEF:perminute=persecond,60,*

But in the case of Wh, I recommend to multiply 'ticks' by 3600 at input 
time, because each tick is worth 3600 Joules, and this results in a clean 
database storing Joules per second aka Watt. Although this could also have 
been done in a CDEF, it just does not "feel right".

To prevent another problem: no, after multiplying by 60 (either at graph 
time (CDEF) or at input time), the X-axis does not magically change. If the 
amount of PDPs per CDP is low enough (e.g. 1), and the step (duration of one 
PDP) is low enough (e.g. 10 seconds), and the amount of time shown on the 
graph is low enough (e.g. 12 minutes), and the graph width is large enough 
(e.g. 360 pixels wide), you can still see "something per minute" change a 
couple of times during a minute. Using the example numbers I just gave, the 
resulting graph will show 72 CDPs of 10 seconds each, using 5 pixels per 
CDP. This results in a somewhat staircase looking graph.

Similarly you can tweak your settings so that each of the stairs shows no 
less than 60 seconds, which would probably be a good idea if you use a "per 
minute" view. Just make sure each CDP contains n*60 seconds worth of data. 
With {step} being equal to 1, this means you need 60 PDPs per CDP for a 
minute, or 3600 PDPs per CDP for an hour, and so on.