[rrd-users] Scaling rrd tables for best performance

[Boer, Martin]

All

I agree with Simon that having one rrd per check means you need to do a lot of housekeeping,
but on the other hand it gives you the possibility to add and delete checks per server very easily.
What we've done is the following;

We've made a monitoring tool where each operator can create his/hers own checks using a webinterface. Each configured check goes into a MySQL database and a script runs every 5 minutes to add data to the rrdfiles if they excist or create them if they don't.
When a check is deleted we keep the rrdfiles because they don't take up a lot of space and
since no new data will be put in the database for them the won't eat any performance.
In another table we have definitions on which parameters to use for each type of check/rrdfile. 
It was quite some work to create all this, but when it works it looks very sweet. :)
We now have over 19000 configured checks on 1500 or so hosts. 
Our endusers, the operators, can also select which timeperiod they prefer to graph, giving them a lot of freedom to zoom in on a day to see exactly when something went wrong.
Of course this means we have to keep quite detailed rrd's (15 minute steps over the last 35 days). 

Regards,
Martin


-----Original Message-----
From: rrd-users-bounces at lists.oetiker.ch
[mailto:rrd-users-bounces at lists.oetiker.ch]On Behalf Of Simon Hobson
Sent: donderdag 6 december 2007 16:18
To: rrd-users at lists.oetiker.ch
Subject: Re: [rrd-users] Scaling rrd tables for best performance


Eduardo M. Bragatto wrote:

>I can have each server information store on a "server.rrd" file, like:
>
>server1.rrd, server2.rrd, etc...
>
>Or have it split among several rrd files for the same server, like:
>
>server1_cpu.rrd, server1_mem.rrd, server1_network.rrd,
>server1_generic.rrd, server2_cpu.rrd, etc...

 From a simple practicality perspective, the latter would be quite 
hard to maintain. Each time you add or delete a server you will have 
to redefine your rrd and change the collection routines to suit. 
Also, you cannot update DSs independently in one rrd.

So as a minimum, you would probably be best keeping one rrd per 
server. Also, since the number of data volumes/partitions/filesystems 
is likely to vary between servers, I'd personally be inclined to have 
a separate rrd for each so that the rrds are of a standard format.


If you are graphing multiple systems on one graph then I suspect 
there isn't too much difference in memory requirements between 
opening one big rrd and multiple smaller rrds with the same 
information. However, if you want to graph an individual system, then 
I find it hard to imagine opening/processing a big rrd (and using 
only part of it's data) to be more efficient than opening/processing 
a smaller one.

_______________________________________________
rrd-users mailing list
rrd-users at lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: disclaimer.txt
Url: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20071207/7b1277eb/attachment.txt