[rrd-developers] Accelerator Daemon

[John P. Rouillard]

In message <alpine.DEB.1.10.0807020552180.19506 at sebohet.brgvxre.pu>,
Tobias Oetiker writes:
>Yesterday Scott Brumbaugh wrote:
>> On Mon, Jun 30, 2008 at 10:52:23AM +0200, Florian Forster wrote:
>> > On Sun, Jun 29, 2008 at 02:21:10PM -0700, Scott Brumbaugh wrote:
>> > > When a large number of rrd updates occur at the same time the
>> > > filesystem tries to write all updates to disk file at the same time.
>> > > This slows the system.  I may be missing something but it looks like
>> > > that will happen with this patch if a client is both writing and
>> > > reading large numbers of rrd files simultaneously.
>> >
>> > Yes, that's probably right. I didn't expect anyone with such a huge
>> > number of RRD files to generate graphs periodically/statically instead
>> > of on demand/dynamically.. Is anyone with a big number of files doing
>> > that? If so: Why?
>> Graphing isn't the only use for the data in the rrd files.  For large
>> networks visually watching graphs for anomalies don't scale well.  At
>> some point there will just be too many graphs to watch.
>>
>> Rrdtool allows for automated network monitoring through the rrd_xport
>> function.  You can readout the current rate estimates and use them to
>> update filters to do automated monitoring.  For this type of
>> application it is good to be able to read out the last few rate
>> estimates quickly after an update.
>
>for monitoring, it is not very efficient to pull data back out of
>rrd files ... what you should do, is to use updatev (I wonder how
>the accelerator daemon deals with that one) and then take the
>output todo your monitoring on the fly ... this is MUCH more
>efficient.

That assumes two things:

  1) the system doing the monitoring is the system that does the data
     collection. E.G. If you use nagios to centralize all the
     escalations/notifications alert suppression etc, but use cacti to
     collect data, your choices are to have cacti submit the data to
     nagios on every collection or have nagios go out and poll the rrd
     files for info.

  2) that a single data point is all that is needed for problem
     determination. I look at rrdtool as a control chart. So I often
     need 7 or so data points to see if I have a trend that needs to
     generate an alert.

Now you could write the code that gathers 7 data points, and when a
new one is submitted it somehow gets that new data point, rolls the
old value off the cache and pushes the new one on ....  Or you could
just poll the rrd file on a rgular basis and grab the last 7 values
(stored unmodified) in the rra and analyze them. One is very simple to
code (and provides for an analysis frequencey different from data
collection frequency if that matters), the other less so.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.