[rrd-developers] rrd_fetch seek issue

[Tobias Oetiker]

Hi Daniel,

the patch is actually very simple ...

--- branches/1.3/program/src/rrd_fetch.c        (revision 1684)
+++ branches/1.3/program/src/rrd_fetch.c        (working copy)
@@ -359,6 +359,8 @@

     /* fill the gap at the start if needs be */

+    if (*start <= rra_end_time && *end >= rra_start_time - *step){
+
     if (start_offset <= 0)
         rra_pointer = rrd.rra_ptr[chosen_rra].cur_row + 1;
     else
@@ -374,6 +376,8 @@
     fprintf(stderr, "First Seek: rra_base %lu rra_pointer %lu\n",
             rra_base, rra_pointer);
 #endif
+    }
+
     /* step trough the array */

     for (i = start_offset;

the problem has been present in rrdtool forever it seems ... I have applied the fix in 1.2,1.3 and trunk

thanks for finding this
tobi


Thursday Daniel.Pocock at barclayscapital.com wrote:

>
>
> > -----Original Message-----
> > From: Tobias Oetiker [mailto:tobi at oetiker.ch]
> > Sent: 27 November 2008 10:57
> > To: Pocock, Daniel: IT (LDN)
> > Cc: rrd-developers at lists.oetiker.ch
> > Subject: Re: [rrd-developers] rrd_fetch seek issue
> >
> > Hi Daniel,
> >
> > (I have not yet looked at the code).
> >
> > I just tried to reproduce the problem here and it did not work.
> >
> > I guess you are hitting some sort of a corner case ... can
> > you create a little demo script ...
>
>
> I encountered the problem in my patched version of trunk, so I decided
> to explore it with an unpatched version of 1.2.15, as that is an
> unpatched version I already have on a box.
>
> My patched code is quite unforgiving - there is an assert() for bad
> seeks.  That's how I discovered the problem.
>
> Testing with 1.2.15, there is also a seek beyond the end of the file,
> and NaN data is returned by rrd_fetch.
>
> NOW=`date +%s`
> START_T=`expr $NOW - 3600`
> TS2=`expr $START_T + 10`
> TS3=`expr $START_T + 20`
>
> rrdtool create /tmp/test1.rrd \
>     --start ${START_T} \
>     --step 10 \
>     DS:ds0:COUNTER:900:U:U \
>     RRA:AVERAGE:0.8:1:180
>
> $ ls -l /tmp/test1.rrd
> -rw-r--r--  1 daniel daniel 1984 Nov 27 16:28 /tmp/test1.rrd
>
> # notice the file size is 1984 bytes, consistent with 180 rows in the
> RRA
> # put in a couple of updates with old time stamps:
>
> rrdtool update /tmp/test1.rrd ${TS2}:1000
> rrdtool update /tmp/test1.rrd ${TS3}:2000
>
>
> # Notice that last_update is over 1 hour ago
> # There can only be data for 30 minutes (180 rows)
> $ rrdtool info /tmp/test1.rrd |grep last_upd
> last_update = 1227799751
> $ date +%s
> 1227804688
>
> # Now I inspect the fetch code, notice how the seek is for
> # 544 + (569 * 1 * 8) = 5096
> # which is beyond the end of the file
>
> $ gdb rrdtool
>
> set args fetch /tmp/test1.rrd AVERAGE -r 10 -s -60s
>
> Breakpoint 3, rrd_fetch_fn (
>     filename=0xfffffe76 <Address 0xfffffe76 out of bounds>,
> cf_idx=CF_AVERAGE,
>     start=0xbffb8868, end=0xbffb886c, step=0xbffb8870,
> ds_cnt=0xbffb8874,
>     ds_namv=0xbffb8878, data=0xbffb887c) at rrd_fetch.c:346
> 346         if (start_offset <= 0)
> (gdb) next
> 338         end_offset = (long)(rra_end_time - *end ) / (long)*step;
> (gdb) next
> 346         if (start_offset <= 0)
> (gdb)
> 349             rra_pointer =
> rrd.rra_ptr[chosen_rra].cur_row+1+start_offset;
> (gdb)
> 351         if(fseek(in_file,(rra_base
> (gdb) print rra_base
> $1 = 544
> (gdb) print rra_pointer
> $2 = 569
> (gdb) print *ds_cnt
> $3 = 1
> (gdb) next
> 372         for (i=start_offset;
> (gdb) print ftell(in_file)
> $4 = 5096
> _______________________________________________
>
> This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
> _______________________________________________
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900