[rrd-users] Missing data overnight from daylight saving changes

[A Darren Dunham]

On Mon, Oct 26, 2009 at 03:53:57PM +0000, Richard Curtis wrote:
> Hi,
>   I have a simple rrd to store filesystem utilisation on an hourly basis.
> The data is gathered on other hosts, then processed into rrd via a
> script run at 4am.
> 
> The rrd was created as:
> rrdtool create myrouter.rrd    DS:used:GAUGE:3600:0:100
> RRA:LAST:0.5:1:8760   --start 1256403600 --step 3600

So you've told RRD that unless a new value arrives within 3600 seconds,
it should assume the data between updates is unknown.

> And the following values entered:
> rrdtool update myrouter.rrd 1256410800:39
> rrdtool update myrouter.rrd 1256414400:39
> rrdtool update myrouter.rrd 1256418000:39
> rrdtool update myrouter.rrd 1256421600:39
> rrdtool update myrouter.rrd 1256425200:39
> rrdtool update myrouter.rrd 1256432400:39

And the gap between these two updates is 7200, which is greater than
3600.  So the data between is marked unknown.

> rrdtool update myrouter.rrd 1256436000:39
> rrdtool update myrouter.rrd 1256439600:39
> rrdtool update myrouter.rrd 1256443200:39
> 
> but I have a gap in data between midnight of the 25-Oct (BST) and 2am
> (GMT) on 26-Oct.

Yup.
>                         <!-- 2009-10-24 22:00:00 BST / 1256418000 -->
> <row><v> 3.9000000000e+01 </v></row>
>                         <!-- 2009-10-24 23:00:00 BST / 1256421600 -->
> <row><v> 3.9000000000e+01 </v></row>
>                         <!-- 2009-10-25 00:00:00 BST / 1256425200 -->
> <row><v> 3.9000000000e+01 </v></row>
>                         <!-- 2009-10-25 01:00:00 BST / 1256428800 -->
> <row><v> NaN </v></row>
>                         <!-- 2009-10-25 01:00:00 GMT / 1256432400 -->
> <row><v> NaN </v></row>
>                         <!-- 2009-10-25 02:00:00 GMT / 1256436000 -->
> <row><v> 3.9000000000e+01 </v></row>
>                         <!-- 2009-10-25 03:00:00 GMT / 1256439600 -->
> <row><v> 3.9000000000e+01 </v></row>
>                         <!-- 2009-10-25 04:00:00 GMT / 1256443200 -->
> <row><v> 3.9000000000e+01 </v></row>
> 
> Am I missing something here?  I did insert a value for 256432400, but
> it looks like rrdtool is expecting a value for both 01:00 BST and
> 01:00 GMT.

It needs a value for 1256428800 to keep the sequence intact.  

It doesn't care that one is 01:00 this or 01:00 that.  It just cares
that there's a gap of 7200 seconds, which exceeds the heartbeat limit of
3600.

> Do I need to do some magic when I insert the values and double input
> when the clocks go back?

Did you actaully collect a value for that point in time?  (One hour
after midnight BST).  It's difficult to tell if you're collecting one
point every hour and throwing it away in your post-processing script, or
if you're only collecting data on local clock hours and are missing the
time point.

Either way, RRD appears to be behaving correctly.  While the "fetch" is
translating the timestamps into local timezones, it really doesn't care
about the tiemzone.  It just needs the update every 3600 seconds and you
haven't provided that here.

-- 
Darren