[rrd-users] Questions about effective database parameters

Wed Dec 22 17:47:34 CET 2010

Hi again.
I can't make it work. A rrd is setup as:
rrdtool create -s 1 \
DS:DS1:GAUGE:2:0:3600 \
DS:DS1:GAUGE:2:0:3600 \
RRA:AVERAGE:0:1:86400 \
RRA:AVERAGE:0.02:48:3600 \
RRA:AVERAGE:0.002:336:3600 \
RRA:AVERAGE:0.001:720:3600 \
RRA:MAX:0.02:48:3600 \
RRA:MAX:0.002:336:3600 \
RRA:MAX:0.001:720:3600

Nothing gets stored in the rra's except in the 86400 one. And when generating an image, only time spans around 3.5 hours produces
graphs with data but there is data anywhere during a day. Is this because RRDtool tries to plot data from the "wider" rra? 

Plot image width is 500. I'm using rrdcached with my own timestamps, and the journals are full of updates and writes.
I have rrd with step 60 and they work fine. But all the 1s step are not working as intended.

Here's a info from a file. 
filename = "INT-7055.rrd"
rrd_version = "0003"
step = 1
last_update = 1293034861
header_size = 2632
ds[DS1].index = 0
ds[DS1].type = "GAUGE"
ds[DS1].minimal_heartbeat = 2
ds[DS1].min = 0.0000000000e+00
ds[DS1].max = 3.6000000000e+03
ds[DS1].last_ds = "U"
ds[DS1].value = NaN
ds[DS1].unknown_sec = 0
ds[DS2].index = 1
ds[DS2].type = "GAUGE"
ds[DS2].minimal_heartbeat = 2
ds[DS2].min = 0.0000000000e+00
ds[DS2].max = 3.6000000000e+03
ds[DS2].last_ds = "0.0441428571428571"
ds[DS2].value = 0.0000000000e+00
ds[DS2].unknown_sec = 0
rra[0].cf = "AVERAGE"
rra[0].rows = 86400
rra[0].cur_row = 28857
rra[0].pdp_per_row = 1
rra[0].xff = 0.0000000000e+00
rra[0].cdp_prep[0].value = NaN
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[1].value = NaN
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[1].cf = "AVERAGE"
rra[1].rows = 3600
rra[1].cur_row = 2285
rra[1].pdp_per_row = 48
rra[1].xff = 2.0000000000e-02
rra[1].cdp_prep[0].value = 0.0000000000e+00
rra[1].cdp_prep[0].unknown_datapoints = 13
rra[1].cdp_prep[1].value = 8.8142857143e-02
rra[1].cdp_prep[1].unknown_datapoints = 11
rra[2].cf = "AVERAGE"
rra[2].rows = 3600
rra[2].cur_row = 1438
rra[2].pdp_per_row = 336
rra[2].xff = 2.0000000000e-03
rra[2].cdp_prep[0].value = 0.0000000000e+00
rra[2].cdp_prep[0].unknown_datapoints = 13
rra[2].cdp_prep[1].value = 8.8142857143e-02
rra[2].cdp_prep[1].unknown_datapoints = 11
rra[3].cf = "AVERAGE"
rra[3].rows = 3600
rra[3].cur_row = 1343
rra[3].pdp_per_row = 720
rra[3].xff = 1.0000000000e-03
rra[3].cdp_prep[0].value = 0.0000000000e+00
rra[3].cdp_prep[0].unknown_datapoints = 541
rra[3].cdp_prep[1].value = 2.8852595238e+00
rra[3].cdp_prep[1].unknown_datapoints = 474
rra[4].cf = "MAX"
rra[4].rows = 3600
rra[4].cur_row = 1784
rra[4].pdp_per_row = 48
rra[4].xff = 2.0000000000e-02
rra[4].cdp_prep[0].value = -inf
rra[4].cdp_prep[0].unknown_datapoints = 13
rra[4].cdp_prep[1].value = 4.4142857143e-02
rra[4].cdp_prep[1].unknown_datapoints = 11
rra[5].cf = "MAX"
rra[5].rows = 3600
rra[5].cur_row = 2409
rra[5].pdp_per_row = 336
rra[5].xff = 2.0000000000e-03
rra[5].cdp_prep[0].value = -inf
rra[5].cdp_prep[0].unknown_datapoints = 13
rra[5].cdp_prep[1].value = 4.4142857143e-02
rra[5].cdp_prep[1].unknown_datapoints = 11
rra[6].cf = "MAX"
rra[6].rows = 3600
rra[6].cur_row = 2751
rra[6].pdp_per_row = 720
rra[6].xff = 1.0000000000e-03
rra[6].cdp_prep[0].value = -inf
rra[6].cdp_prep[0].unknown_datapoints = 541
rra[6].cdp_prep[1].value = 5.7166666667e-02
rra[6].cdp_prep[1].unknown_datapoints = 474

 I've been staring at all this for too long and can't come up with a 
solution.
Merry Christmas
Henrik
From: henrikhegardt at hotmail.com
To: s.shipway at auckland.ac.nz; rrd-users at lists.oetiker.ch
Date: Wed, 15 Dec 2010 16:10:15 +0100
Subject: Re: [rrd-users] Questions about effective database parameters

Thank you for your reply!
For some reason I was thinking of the xff the other way around. I'm using your ideas, and it looks promising.
Regards Henrik
From: s.shipway at auckland.ac.nz
To: henrikhegardt at hotmail.com; rrd-users at lists.oetiker.ch
Subject: RE: [rrd-users] Questions about effective database parameters
Date: Wed, 15 Dec 2010 01:27:16 +0000

RRA:AVERAGE:0.5:30:720 

RRA:AVERAGE:0.5:60:720  

You need to set the XFF (the fraction of the data which must be known for an RRA to store a value) to be much lower; in fact, you might even want to make it as near to zero as you can, maybe 1/(#dp)

So, for the above, use:

RRA:AVERAGE:0.03:30:720 

RRA:AVERAGE:0.01:60:720 

and even make it smaller for other RRAs.

This is a rather clever way to get around the whole data normalisation thing; by making the step size 1s you avoid normalisation, but use a tiny XFF to make sure your 1-min RRA still stores the data as posted (and averages if you get multiple samples).

Steve

Steve Shipway
University of Auckland ITS
UNIX Systems Design Lead
s.shipway at auckland.ac.nz
Ph: +64 9 373 7599 ext 86487

From: rrd-users-bounces+s.shipway=auckland.ac.nz at lists.oetiker.ch [rrd-users-bounces+s.shipway=auckland.ac.nz at lists.oetiker.ch] on behalf of Henrik Hegardt [henrikhegardt at hotmail.com]

Sent: Tuesday, 14 December 2010 10:59 p.m.

To: rrd-users at lists.oetiker.ch

Subject: [rrd-users] Questions about effective database parameters

Hello.

I'm writing a program to monitor a server's activities. Its about response times for services on the server. There are quite many services and most of the written data are unknowns. The response times is quite crucial and would want to have as high resolution
 as possible even though there are unknowns. If there's a response time I write it to the DB with 1 second interval though since there are several other services per rrd a update operation could look like 'update database.rrd 1292313983:U:U:U:U:U:U:U:0.012:U:U:U:U:U:U'.
 I would like to store it for atleast 6 months and thus view it over that time. I'm not interested of if there was no value, but the value itself.

I can't tell if there will be a lot of updates during a time interval or none at all which should result in some of the entries won't even get a 'U' value (most probably).

I have worked with following

rrdtool create -s 1 \

DS:ds1:GAUGE:2:0:3600 \

...

RRA:AVERAGE:0.5:30:720 

RRA:AVERAGE:0.5:60:720 

RRA:AVERAGE:0.5:420:720

RRA:AVERAGE:0.5:1800:720 

RRA:MAX:0.5:30:720 

RRA:MAX:0.5:60:720 

RRA:MAX:0.5:420:720 

RRA:MAX:0.5:1800:720

This didn't work, since I believe there a lot more 'U' values than 0.5 and the resolution isn't good enough and really doesn't make sense according to what I'm trying to sample.

RRA:AVERAGE:0.5:1:86400 worked but then having a rra over 6 months (6*30*86400) will result in a big db and slow rendering of images. Any ideas?

Forgive my English.

_______________________________________________
rrd-users mailing list
rrd-users at lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20101222/a5349e98/attachment-0001.htm 