[rrd-users] Re: RRD grouping, averaging, and storing data

Thu Jun 12 21:49:23 MEST 2003

On Thu, Jun 12, 2003 at 04:30:19AM -0700, Steil, Taylor wrote:
> Hello,
> 
> I am experimenting with RRD and I cannot figure out why it is doing some of
> the things it is doing.

Did you work your way through the tutorial?

> Take this list of commands for example:
> 
> rrdtool create test.rrd       \
>   --start 920804400           \
>   --step 120		      \
>   DS:speed:ABSOLUTE:120:U:U   \
>   RRA:AVERAGE:0.1:1:100       \

Create a database with rows each 2 minutes, where time modulo 120 is zero.
You require an update at least every 120 seconds.  Store 100 of such rows.

> rrdtool update test.rrd 920804550:5
> rrdtool update test.rrd 920804650:6
> rrdtool update test.rrd 920804750:7
> rrdtool update test.rrd 920804850:8
> rrdtool update test.rrd 920804950:9

Updating not every 120 seconds (which is not necessarily a bad thing)
and not on exact boundaries (which is not necessarily a bad thing).

You must understand the normalizing process (see docs, and faq.mrtg.org)
to understand what's happening in this case.

> rrdtool graph speed2.gif                          \
>   --start 920804400 --end 920805660               \

ask for 1260 seconds (21 minutes) worth of data...
>   --vertical-label m/s                            \
>   --step 240				              \

...with four minutes per step.  Huh?  Try dividing 21 by 4.

>   DEF:myspeed=test.rrd:speed:AVERAGE              \
>   "CDEF:realspeed=myspeed,100,*"                  \
>   LINE2:realspeed#FF0000
> 
> rrdtool fetch test.rrd AVERAGE --start 920804400 --end 920805660 -r 120

now you are comparing the graph, in 240 second resolution, with data in
120 second resolution... not bad, if you do it on purpose.

[copied the updates from above again]

> rrdtool update test.rrd 920804550:5

know nothing about the start, know the current counter is at 5.
Difference between unknown and 5 is unknown.  Rate is unknown.

> rrdtool update test.rrd 920804650:6

Know the counter increased from 0 to 6 (absolute counter) in 100 seconds.
The rate is 6/100 == 0.06 per second.

> rrdtool update test.rrd 920804750:7

Know the counter increased from 0 to 7 in 100 seconds, rate 0.07

> rrdtool update test.rrd 920804850:8

rate 0.08

> rrdtool update test.rrd 920804950:9

rate 0.09

When are these rates valid:

unknown rate:  before  920804550
rate 0.06:     between 920804550 and 920804650
rate 0.07:     between 920804650 and 920804750
rate 0.08:     between 920804750 and 920804850
rate 0.09:     between 920804850 and 920804950

> And the output from the fetch with my updates added:
>                     speed
>  920804400: nan
>  920804520: nan
> 	## 920804550:5
>  920804640: 6.0000000000e-02
> 	## 920804650:6
> 	## 920804750:7
>  920804760: 7.0000000000e-02
> 	## 920804850:8
>  920804880: 8.2500000000e-02
> 	## 920804950:9
>  920805000: nan

Normalizing process kicks in:

time slot 920804280 to 920804400: rate unknown
time slot 920804400 to 920804520: rate unknown
time slot 920804520 to 920804640: partially known to be 0.06, partially unknown
time stot 920804640 to 920804760: partially 0.06, partially 0.07, partially 0.08
time slot 920804760 to 920804880: partially 0.08, partially 0.09
time slot 920804880 to 920805000: not yet updated so unknown

4th time slot:

920804640 to 920804650 at rate 0.06;  10 out of 120 seconds
920804650 to 920804750 at rate 0.07; 100 out of 120 seconds
920804750 to 920804760 at rate 0.08;  10 out of 120 seconds

Weighted averaging of these: 10/120*0.06 + 100/120*0.07 + 10/120*0.08 = 0.07

> I have 4 questions:
> 1) Why is the data being stored as e-02 numbers (very very small)? I put the
> numbers in as whole integers, why is RRD dividing them by 100?

Because you asked it to do so.  The numbers are per second.
0.07 events per second times 120 seconds is 8.4 events in two minutes.

> 2) Why are the first and last data values I am sending it not being
> displayed?

Rates, not numbers.  And the rates are unknown in this case because you
didn't tell RRDtool what they are.

> 3) Why is the value of 920804760 equal to only 7 and not 6 + 7?

Because of what I described in detail above

> 4) Why is the value of 920804880 equal to 8.25?

Do the math and it will work out.

> Here is what I am trying to do:
> I have a server that does a lot of events, and I would like to be able to
> use RRD to store these events as they happen, then go back later and be able
> to graph the frequency. I can send the RRD database data every X number of
> minutes, and I want it to COUNT this data until the step, then start over.
> 
> So if the step is every 120 seconds, any data I send it during that 120
> seconds I want it to add together. Then, after those 120 seconds are over I
> want it to store that in the RRD database and start back over at 0.

RRDtool does not work this way.  I'm sure you can find a way to make it
work but you'll have to find a way to convert your numbers into rates.

If you update every two minutes (that is 120 seconds, not 100), you are
storing events per 120 seconds.  When multiplied with 120 (a CDEF) you
will get your number of events again.  The exact numbers will only be
inserted in the database if you enter them at exactly the right time,
which is the unix epoch time + n*(your step size).  Even then, using
an absolute counter will result in a calculation.  This is a good thing.
Do not get rid of it, use it!

HTH
Alex
-- 
Much of what looks like rudeness in hacker circles is not intended to give
offence. Rather, it's the product of the direct, cut-through-the-bullshit
communications style that is natural to people who are more concerned about
solving problems than making others feel warm and fuzzy.

http://www.tuxedo.org/~esr/faqs/smart-questions.html

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi