[rrd-developers] patch/tuning for very large RRD systems (was "Re: SQL Backend request")

[Henrik Stoerner]

On Thu, May 24, 2007 at 01:43:56PM -0500, Dave Plonka wrote:
> Note however, that such a configuration couldn't even do 100K
> RRD files without the patch to fadvise RANDOM to suppress readahead.
> I believe any post-2.6.5 Linux has the posix_fadvise behavior that
> the patch leverages.
> 
> Also, Tobi has integrated the patch in the code he's testing.

I "only" have about 40K RRD's with 5-minute update intervals, but the
machine had problems keeping up with this so I decided to try out your
patches.

Since we're on RRDtool 1.0.49, though, I did a back-port of your 
fadvise code to that version - very easy, but if anyone else
wants it drop me a note and I'll put the diff up for download.

The result was that the memory utilisation for the buffer cache
dropped dramatically, as I'd expected. However, the general I/O
activity did not go down - in fact, it increased by about 15-20%,
as measured by vmstat.

This is a Linux 2.6.18 (Debian 4.0) system; the RRD files are on a
ReiserFS filesystem.

Re-coding the application that handles the RRD updates to cache them in
memory and bundle updates together turned out to be a lot more effective 
way of tackling the problem. We're running Hobbit, not MRTG - I have no
idea if this would even be possible with MRTG.


Regards,
Henrik