[rrd-developers] patch/tuning for very large RRD systems (was "Re: SQL Backend request")

[Henrik Stoerner]

On Thu, May 31, 2007 at 12:04:48AM +0200, Henrik Stoerner wrote:
> The system used for testing has:
> * 40.000 RRD files with 100.000 datasets. Each has 4 RRA with a
>   5 minute/30 minute/4 hours/24 hours averaging function. This
>   is currently a very stable set of files, new files haven't
>   been added during the tests.
> * Update frequency is once per 5 minutes (as with MRTG)

This is of course wrong - data is collected every 5 minutes, but this
happens on another system which then feeds the data to the RRD box
over TCP. Updates to the RRD files are done for every 6 updates (i.e.
once every 30 minutes), except that to spread out the load over time
every 6th update of any file will go to disk, bringing with it any
other pending updates for that same file. (Without this, I'd have
25 minutes of pure silence, then 5 minutes with hectic disk activity).

Also, when reading the iostat data keep in mind that the first reading
always covers the full system uptime, and therefore should be ignored.


Regards,
Henrik