[mrtg-developers] Idea for alternative MRTG scheduler

[Tobias Oetiker]

Hi Steve,

Yesterday Steve Shipway wrote:

> Here's an idea.
>
> Currently, MRTG will process all Targets until there are none
> left, using up all the available threads, and then sleep until
> the next polling cycle.
>
> This can be problematic if you configure more targets, or there
> is an outage, and suddenly you do not have enough threads to
> process everything in the 5min window.  Also, it results in a
> large burst of activity at the start of the window followed by
> silence.
>
> So, how about this - MRTG already knows how many targets there
> are, and the interval.  It calculates x=(interval/#targets)x0.9
> (the 0.9 is to allow time for the final checks to complete) and
> then kicks off a new Target to process every x seconds, starting
> a new thread if required (possibly up to a specified upper
> limit).  This would possibly end up with each thread processing a
> single target and then exiting, with the master starting a new
> thread per Target.
>
> I think this may be how the Nagios check scheduler works?  It
> would certainly solve the problems of (a) uneven CPU usage and
> (b) running out of window time when you add more targets but not
> more threads.  The drawback is that, of course, you need to have
> sufficient CPU/memory to handle the potentially large number of
> threads that could result.

I agree scheduling could be improved. Your aproach asumes an even
delay with all targets I guess ... so if there is a slow target at
a late stage of the spread out polling activity it would not have
enough time to complete unless every target is run acynchronously
causing a large horde of processes or threads.

The motivation for your suggesting seems to be to not overwhelm
devices or networks with intense polling, so I guess one aproach
would be to implement some sort of polling rate limit which makes
sure mrtg does not 'kill' anyone by being to hasty.

> Since Tobi is currently in a coding mood, I thought it best to
> get the suggestions in quick :)

I only processe the mrtg backling ... no clever new stuff from my
end ... and I am intending todo the same for rrdtool now ... and
then publish 1.4.5.

a bugtracker with a backlog is such a sad thing ...

cheers
tobi



>
> Steve
>
> ________________________________
> Steve Shipway
> ITS Unix Services Design Lead
> University of Auckland, New Zealand
> Floor 1, 58 Symonds Street, Auckland
> Phone: +64 (0)9 3737599 ext 86487
> DDI: +64 (0)9 924 6487
> Mobile: +64 (0)21 753 189
> Email: s.shipway at auckland.ac.nz<mailto:s.shipway at auckland.ac.nz>
> P Please consider the environment before printing this e-mail
>
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900