[rrd-developers] Data Loss on non-contiguous updates

[Lee Thompson]

I'm not sure if this issue has been covered before in
this forum.

When you write a program that updates rrd files at
adhoc intervals, non-contigous updates are dropped. 
What I mean by non-contiguous is a sample that is
several samples in the future to the last sample
written to rrd.  I spent some time tracking this
problem down.

I modified the "minmax.pl" example script supplied
with rrdtool to update the rrd on 20 minute
boundaries, instead of the expected 5 minutes
interval.  When you run "minmax_skip.pl" attatched to
the email, you see only one sample recorded even
though "rrdtool update" was called 100's of times.

I figured a work around to get rrdtool to record the
non-contigous samples.  You write an undefined sample
just prior to the sample you are trying to update. 
You can see and example of this by uncommenting this
line in minmax_skip.pl

      # RRDs::update $rrd, ($t-300).":U";

After re-executing minmax_skip.pl you will see
hundreds of samples in the rrd.

The following patch gets the value into rrdtool
without the manual backfill call, but also backfills
the update value backwards in history which isn't what
I want.


diff -Nru rrdtool-1.2.11.orig/src/rrd_update.c
rrdtool-1.2.11/src/rrd_update.c
--- rrdtool-1.2.11.orig/src/rrd_update.c       
2005-08-13 18:30:23.000000000 -0500
+++ rrdtool-1.2.11/src/rrd_update.c     2005-08-13
18:31:16.000000000 -0500
@@ -612,8 +612,7 @@
              * this case, an extra check is required.
*/

            if((updvals[i+1][0] != 'U') &&
-                  (dst_idx != DST_CDEF) &&
-              rrd.ds_def[i].par[DS_mrhb_cnt].u_cnt >=
interval) {
+                  (dst_idx != DST_CDEF)) {
               double rate = DNAN;
               /* the data source type defines how to
process the data */
                /* pdp_new contains rate * time ... eg
the bytes




I would prefer that rrdtool would detect this "fill
in" situation and automatically null out the prior
data and preserve the new update value.  This would
make client rrdtool programming easier.

I look forward to your thoughts,
Lee Thompson

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

--
Unsubscribe mailto:rrd-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-developers-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-developers
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi