[rrd-users] RRD Graphs with bad averaging of missing data?

[Alex van den Bogaerdt]

----- Original Message ----- 
From: "Matt Elias" <eliaschief at gmail.com>
To: <rrd-users at lists.oetiker.ch>
Sent: Thursday, April 10, 2014 6:39 PM
Subject: [rrd-users] RRD Graphs with bad averaging of missing data?


>I no longer have the rrd for the original sample graph I posted but here
> is a short exert from another anomaly.

<snip: data either 218 or NaN>

> <!-- 2014-04-10 09:44:36 CDT / 1397141076 -->
> <row><v>2.1449311575e+02</v></row>
> <!-- 2014-04-10 09:44:48 CDT / 1397141088 -->
> <row><v>2.1800000000e+02</v></row>
> <!-- 2014-04-10 09:45:00 CDT / 1397141100 -->
> <row><v>2.1800000000e+02</v></row>

Here we see one time 214, followed by 218 again.  Find the corresponding 
entries in your text file, just to make sure there was only 218 and not 
something else.
Please share the lines around this gap. The last update happening before the 
gap, and 2 after the gap.  Let's see if we can compute what RRDtool did to 
your data.

> As I have suspected, it is an issue of averaging. I don't know how to
> setup my RRDs properly. These are not rates, they are temperatures.

Rates or temperatures, it does not really matter.

If your step size equals 1, and if you feed timestamps which are a whole 
number of seconds (so not 'N' for now, and not timestamps with a decimal 
fraction) then there is nothing to normalize.  That may already help to get 
your input 'as is'.

But then there's still consolidation happening.

If one slot has temperature 218, and the next has 220, and you ask to plot 
both temperatures onto one pixel column, something has to happen.  Your 
choice to use last, min, max, or average.  This consolidation can happen at 
graph time or you can choose to have additional RRAs and have the data 
prepared for faster displaying.

> RRDs::create("$rrd/bin$t-$n.temps.rrd",
> "-s", "1",
> "DS:Temp:GAUGE:5:U:U",

heartbeat of 5. If there is no new data for more than 5 seconds, the next 
update will be treated as invalid.  If you are going to update every 5 
seconds, then you will have to increase the heartbeat a bit. It does not 
have to be double the frequency but it is a common choice made.  I usually 
double it plus add a bit to accomodate for delays in the update process. Or, 
if I am also very interested in outages, I set heartbeat to the update 
frequency plus a bit. 6 or 7 would be a good choice, as are 11 or 12. An 
update which is 1 or 2 seconds late is still valid, 0 or 1 missing updates 
are allowed.

> "RRA:AVERAGE:0:1:900",  # track 900 1sec samples for 15min graph

This is allowed but not necessary.  Keep it if you want to be able to really 
zoom in.  If not, then don't.
What I am saying here is that you do not HAVE TO have an RRA with steps == 1 
step.  It's a choice, just like any other RRA.

> "RRA:AVERAGE:0:5:900",  # 4500 1sec samples, 1hr graph

5 rates are going to be consolidated into one. If you don't want averages, 
then average is not what you should specify here. See above.

There will be no unknown data allowed (XFF==0) so if any of the 5 source 
PDPs was unknown, the entire CDP will be unknown.
Over time your gaps will become bigger than what really happened.  I'm not 
judging this, that's up to you.

<snip:  and so on for the other RRAs>