[rrd-users] Problem on update

[Chris Raymond]

Hi,

I think I have found a bug with RRDtool, in the update area.  I used RRDtool 
as an armature to build a network performance monitoring application on. 
Once per hour, a number of source/destination pairs report on the status of
the links between them.  However, I do not control the input, so in some
cases, there are gaps--missing data for that pair or pairs.

In case of such a gap, RRDtool inserts extra copies of the data point.

Here's an example.  The RRDtool DB was built by the command below:

rrdtool create UNEP-Peking.rrd -b 957222000 -s  3600 \
DS:hop:GAUGE:18000:U:U \
DS:pktloss:GAUGE:18000:U:U \
DS:rtt:GAUGE:18000:U:U \
DS:thruput:GAUGE:18000:U:U \
RRA:LAST:0:1:8760 \
RRA:AVERAGE:0.5:24:1826 \
RRA:MIN:0.5:24:1826 \
RRA:MAX:0.5:24:1826   

Focusing on the year's worth of data saved in the LAST RRA,  since it's
one point per, if there's a gap, I should get one or more rows of "NaN".
Here is what does happen:

Before

rrdtool fetch UNEP-Peking.rrd LAST -s "20000504 12:00" -e "s+8h" 
                      hop       pktloss           rtt       thruput

 957456000: 2.7000000000e+01 0.0000000000e+00 1.0030000000e+03 4.7000000000e+01
 957459600: 2.7000000000e+01 0.0000000000e+00 9.0100000000e+02 6.1000000000e+01
 957463200: 2.7000000000e+01 0.0000000000e+00 9.2200000000e+02 6.6000000000e+01
 957466800: NaN NaN NaN NaN
 957470400: NaN NaN NaN NaN
 957474000: NaN NaN NaN NaN
 957477600: NaN NaN NaN NaN
 957481200: NaN NaN NaN NaN
 957484800: NaN NaN NaN NaN

Here's what the "example" script looks like:

cat example
#!/bin/sh
rrdtool update UNEP-Peking.rrd 957481200:27:0:890:81
corn.eos.nasa.gov% example 

Now after running it, RRDtool DB looks like:
rrdtool fetch UNEP-Peking.rrd LAST -s "20000504 12:00" -e "s+8h"
                      hop       pktloss           rtt       thruput

 957456000: 2.7000000000e+01 0.0000000000e+00 1.0030000000e+03 4.7000000000e+01
 957459600: 2.7000000000e+01 0.0000000000e+00 9.0100000000e+02 6.1000000000e+01
 957463200: 2.7000000000e+01 0.0000000000e+00 9.2200000000e+02 6.6000000000e+01
 957466800: 2.7000000000e+01 0.0000000000e+00 8.9000000000e+02 8.1000000000e+01
 957470400: 2.7000000000e+01 0.0000000000e+00 8.9000000000e+02 8.1000000000e+01
 957474000: 2.7000000000e+01 0.0000000000e+00 8.9000000000e+02 8.1000000000e+01
 957477600: 2.7000000000e+01 0.0000000000e+00 8.9000000000e+02 8.1000000000e+01
 957481200: 2.7000000000e+01 0.0000000000e+00 8.9000000000e+02 8.1000000000e+01
 957484800: NaN NaN NaN NaN

Notice that RRDtool has inserted 4 copies of the point for 19:00 into the slots
for the 15:00 -- 18:00 hours.  If I take the heartbeat down to just over the
stepsize (3600 seconds), RRDtool does as advertised, and throws away the data
that comes in, setting the values for the 15:00 through 19:00 hours to "NaN".
Switching the RRA to "AVERAGE" instead of "LAST" doesn't do anything because
there's no difference between the two when it's one input per cdp. I've tried
some games with the "xff" setting;  that doesn't do it either.  Pushing the 
heartbeat to a high value opens me up to possibly many (false) identical 
points inserted into the RRDtool DB.

If I apply the following diff to rrd_update.c (version 1.0.39), it fixes MY 
problem.  What happens then is that any data inserted within the heartbeat
span goes in with no extraneous copies created, the hours with no data get
put in as "NaN".  Can anybody see what it might break in other applications
of RRDtool?


560c560,561
< 				rrd.cdp_prep[iii].scratch[CDP_val].u_val = pdp_temp[ii];
---
> 				rrd.cdp_prep[iii].scratch[CDP_val].u_val = 
>                                    ( current_time > pdp_st ) ? DNAN : pdp_temp[ii];

Thanks,
Chris R.

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi