[rrd-users] Re: larger rrdfiles (hugish even) and systemperformance

[Alex van den Bogaerdt]

On Thu, Nov 16, 2006 at 01:57:40PM +0100, Boer, Martin wrote:

> In our current production situation we have approx 15000 rrdfiles created using the following values;
>          $rrd_cmd .= "RRA:$rrd{$check}{cons}:0.5:1:300 ";
>          $rrd_cmd .= "RRA:$rrd{$check}{cons}:0.5:3:3500 ";
>          $rrd_cmd .= "RRA:$rrd{$check}{cons}:0.5:288:365 ";
> 
> What the endusers want is the following (very detailed information for a couple of months)
>          $rrd_cmd .= "RRA:$rrd{$check}{cons}:0.5:1:30000 ";
>          $rrd_cmd .= "RRA:$rrd{$check}{cons}:0.5:576:732 ";


Are you saying here that you did not implement what your users need?
Or are you saying that you did implement those last two RRAs as well?

> Using vmstat it seemed that the IO-bi value increased from a peacefully 50 to a staggering 25000.
> This cripples the system somewhat to say the very least.

Find out why this is.  A guess: you are using a high-resolution RRA
to create graphs at a much lower resolution.  For instance, you are
using an RRA with steps==1_step to show steps==12_step (e.g. 300 seconds
per row, times 12, to display one hour per pixel).

If RRDtool can use an RRA at low resolution (12 steps per row) it needs
to fetch 400 rates.  If RRDtool needs to build the data from a high resolution
RRA (1 step per row) it needs to fetch 4800 rates.  More cpu required to
average those 12 rates into 1 (400 times), but also more IO due to more
data to fetch.

- Make sure you understand normalization and consolidation (see my site).
- Know what is going to be displayed, and make sure an appropriate RRA
  is available in your RRD.
- Set start and end times to a whole multiple of the amount of seconds
  per pixel.  Not: "end=now" (either explicitly or implicitly), but
  "end=${endtime}" where variable endtime is computed by your front end
  and is a whole multiple of 3600 (or 86400, or 300, or 604800 etc.)
  whatever your step*steps is.

HTH
-- 
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-users
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi