[rrd-users] Re: suspicious interpolation

Sun Oct 12 12:29:57 MEST 2003

On Sat, Oct 11, 2003 at 04:50:30PM +0200, Philippe Charnier wrote:

> This makes 1000000020:3.1 really different from the reality 1000000020:4 and
> 1000000010:3.1 not so close to 1000000009:2.
> 
> Any comment?

I think you are right.  When updating (nearly) every step time, this
problem does not show itself.

Here's a small bash script, followed by the rest of my mail.
Long lines not wrapped by me, sorry for that.

>---cut_here---<
RRDTOOL=~alex/bin/rrdtool
RRDTOOL=~alex/tmp_20030609/rrdtool-1.0.40/src/rrdtool
RRDTOOL=~alex/c/rrdtool-1.0.45/src/rrdtool

${RRDTOOL} create /tmp/a.rrd --start 999999999 --step 10 DS:gauge:GAUGE:300:U:U RRA:AVERAGE:0.5:1:20
${RRDTOOL} update /tmp/a.rrd 1000000000:U
${RRDTOOL} update /tmp/a.rrd 1000000019:10
${RRDTOOL} update /tmp/a.rrd 1000000100:0
${RRDTOOL} fetch /tmp/a.rrd AVERAGE --start 1000000000 --end 1000000100

echo '-----------------------------------'

${RRDTOOL} create /tmp/b.rrd --start 999999999 --step 10 DS:counter:COUNTER:300:U:U RRA:AVERAGE:0.5:1:20
${RRDTOOL} update /tmp/b.rrd 1000000000:0
${RRDTOOL} update /tmp/b.rrd 1000000019:190
${RRDTOOL} info /tmp/b.rrd 
${RRDTOOL} update /tmp/b.rrd 1000000020:190
${RRDTOOL} info /tmp/b.rrd 
${RRDTOOL} update /tmp/b.rrd 1000000100:190
${RRDTOOL} fetch /tmp/b.rrd AVERAGE --start 1000000000 --end 1000000100
exit 0
>---cut_here---<

There definitively seems to be a problem somewhere around calculating or using pdp_prep.

Starting from a known counter value of zero at time 1000000000.
Two updates, happening at time 1000000019 and at time 1000000100.

Defined intervals and counter values:

a 1000000000 .. 1000000010   0, not yet known
b 1000000010 .. 1000000020   not yet known,190 (@time 1000000019)
c 1000000020 .. 1000000030   190,not yet known
d 1000000030 .. 1000000040   190,not yet known
e 1000000040 .. 1000000050   190,not yet known
f 1000000050 .. 1000000060   190,not yet known
g 1000000060 .. 1000000070   190,not yet known
h 1000000070 .. 1000000080   190,not yet known
i 1000000080 .. 1000000090   190,not yet known
j 1000000090 .. 1000000100   190,190 (@time 1000000100)

No counter increment occurs during intervals c,d,e,f,g,h,i and j.
All incrementing occurs during intervals a and b.
Computations happen at time 1000000019 and time 1000000100
At time 1000000019, one PDP can be filled.
At time 1000000100, the other PDPs can be filled.

Expected behaviour:  either one of:

#1
constant rate between time 1000000000 to 1000000019; rate is (190-0)/(1000000019-1000000000) = 190 / 19 = 10
constant rate between time 1000000019 to 1000000100; rate is (190-190)/(1000000100-1000000019) = 0 / 81 =  0
rates:
  a: 10
  b: 90% of 10 and 10% of 0, result is 9
  c: 0
  d: 0
  and so on

#2
constant rate between time 1000000000 to 1000000020; rate is (190-0)/(1000000020-1000000000) = 190 / 20 = 9.5
constant rate between time 1000000020 to 1000000100; rate is (190-190)/(1000000100-1000000020) = 0 / 80 =   0
rates:
  a: 9.5
  b: 9.5
  c: 0
  d: 0
  and so on

IMHO #1 is the one that should occur.  I wouldn't care too much if #2 were the case

Observed behaviour:

Constant rate between time 1000000000 and time 1000000010; rate is (190-0)/(1000000019-1000000000) = 190 / 19 = 10
Constant rate between time 1000000010 and time 1000000100; rate is calculated from partially filled interval b

rates:
  a: 10   (this is OK)
  b: 1    (this is way too small)
  c: 1    (this is incorrect)
  d: 1    (  "        "  )
  and so on

So, the increase _before_ time 1000000019 is computed to have occured _after_ time 1000000019.

Each update describes the past (increase happened _before_ the current snapshot of the counter).
The increase that happened until time 1000000019 should therefore be used to fill interval a and b.

Somewhere in interval b, the counter was sampled and found to be at value 190.  After that, no increase happened.
RRDtool can know there is no increase between time 1000000019 and time 1000000100.

The normalized rate for interval b may be difficult.  The normalized rate for interval c and further is plain and simple; no increase gives rate zero.

CONCLUSION

At time 1000000019 one PDP can be filled.  The remainder is the larger portion of the next PDP.
At time 1000000100 the other PDPs can be filled.  There are two rates to take into account.  One rate is stored in pdp_prep, the other one is calculated.

After the update at time 1000000019:
   ds[counter].last_ds = "190"
   ds[counter].value = 9.0000000000e+01
   ds[counter].unknown_sec = 0

The "ds[counter].value = 9.0000000000e+01" line correctly describes 90% of rate 10; as is calculated in expected behaviour #1.

So far so good.

After the next update (at time 1000000100), this value 90 should be used to update _one_ PDP.  In reality, RRDtool seems to
combine this value with the _next_ increase (being zero in this example), thus shifting it in time from one update to the next.

I guess the problem is in rrd_update.c, line 444 and further.

cheers,
Alex

-- 
http://www.googlism.com/index.htm?ism=alex+van+den+bogaerdt&type=1

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi