[rrd-users] A plague of NaNs

[Alex van den Bogaerdt]

----- Original Message ----- 
From: "Jeff Gilmore" <jeff at thegilmores.net>
To: <rrd-users at lists.oetiker.ch>
Sent: Wednesday, November 02, 2011 2:31 AM
Subject: [rrd-users] A plague of NaNs


> Hi all,
> I have an application where I read data from electric meters, each into its own RRD file storing only watts into a GUAGE.  The RRD files are all of identical format, with a 60 second heartbeat.

> Most of the meters are read approximately once a minute, but that may vary by a few seconds (generally slightly longer than 60 seconds).  These meters produce RRD files where 90+ percent of the values stored  (as shown in the "database" section of RRDdump) are NaN.  


Generally slightly longer than 60 seconds.

 
> One of the meters is set to upload data on 30 second intervals (but still feeds an RRD file with a 60 second heartbeat).  The RRD for that meter has almost all of the values stored successfully.

And this one has no problems, 30 seconds being generously lower than 60 seconds.

> Can you help me understand why this might be the case?  

As per the documentation: 
heartbeat defines the maximum number of seconds that may pass between two updates of this data source before the value of the data source is assumed to be *UNKNOWN*.

Unknown, aka NaN.

You told rrdtool that updates further than 60 seconds apart are unreliable and should be assumed to be unknown.

> 
> I assumed that RRDtool's "normalization" process would adjust for the slight variations in timestamp between the actual data and the heartbeat interval of the RRD, fitting the update into the closest timeslot and adjusting the data value accordingly.  It sounds like I don't understand it sufficiently, despite many hours searching the web on this topic.  

It's not a sample rate, it's a heartbeat. If there is no heartbeat within {$heartbeat} seconds, the subject is dead. 


> 
> In any case, I have a data source that produces data that is not perfectly aligned with the RRD heartbeat.  Can you suggest how my application can deal with this and still take advantage of the power of RRDTool?

You confuse the step setting with the heartbeat setting. Read the tutorials, see my site ( http://rrdtool.vandenbogaerdt.nl ) and play/experiment a bit with settings.

good luck.
HTH
Alex