[rrd-developers] Re: How to get the most performance when using lots of RRD files

[Ole Bjørn Hessen]

Hans Jørgen Jakobsen <hjj at wheel.dk> writes:
> At the company where I work we have a system grown out of
> MRTG at 1997. From 1K to +500K interfaces today. The interfaces
> are sampled in 1, 3, 5, 10 or 20 min intervals. At the moment
> we are collecting 3.4M sets of data each hour.

...

> That lead to a design where the poller process(es) spools
> the results to files. This is every 5 min spread out to a
> file for each router and put in a dir. There are a pool of
> rrd-update-processes. A rrd-update-process takes the oldest
> datafile and all other files to the same router and sorts
> input so all updates to one interface is done at once.
> This meens efficiency grows by queue size.

A usual setting is that customers want graphs to be updated to the
lates sample when they have trouble with the network or whatever.  The
above approach could be combined with a special "flushing" operation
initiated from the webserver on the set of graphs that is to be viewed
before graphing them. It depends on the ratio of how many graphs are
generated versus how many are updated. In our situation we update > 20k
files every 5 min, but only a "handful" graphs are generated in the
same period.

Ole Bjørn Hessen,
NMS-IP, PF-Nett, Telenor Networks

--
Unsubscribe mailto:rrd-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-developers-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-developers
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi