[mrtg-developers] Re: proposed mrtg performance improvements?

[Dave Plonka]

On Fri, Nov 21, 2003 at 07:43:59AM +0100, Tobias Oetiker wrote:
<snip>
> 
> Yes, but the snmpgets only happen in readtargets, getcurrent is
> just data mangling ... there is no latency as fahr as I remember my
> code ...
<snip>
> the snmp stuff happens in readtargets ...

Got it.  Thanks for clarifying that...

I had mistakenly thought that getcurrent was responsible for getting
the current values for the targets because, well, its called
"getcurrent" and does contain a call to snmpget.

On closer reading of getcurrent though, I see that the snmpget is
conditional and is actually getting something completely different:

    # Get the device name if $nloc (community at host) has been specified
    # one way or the other
    ( $name ) = snmpget( $nloc, $noid ) if $nloc;
 
> > > Have you used the perl profiler on the code, this might reveal
> > > where the hotspots are in the code ...
> >
> > No, I might try that.
> 
> when I did this last time I discovered a very bad performance
> killer in the cfg reading code ... It cut the startup time by a
> huge factor witch grew progressively with the size of the config
> file ...
> 
> > Anotehr thing I'm wondering about is trying to avoid the often
> > unnecessary RRDs::tune and "threscheck" calls.  Perhaps the mrtg
> > process could remember the previous tune that it performed so as
> > not to do it repeated in RunAsDaemon mode.
> 
> this might be a a sensible thing to do yes ... actually it does
> not have to remember it at all, because the config will not change
> after startup, so a single tune run is perfectly sufficient ...

OK, the path I've chosen to pursue is:

1) I took your advice to run multiple mrtg instances, each on a subset
   of the 30K targets.

   Currently I'm running 4 (each with about 10,000 targets) and have
   Forks set to 16.  It's working much better than when I had each
   instance handling 3,000 targets previously.

2) I'll investigate speeding up the target loop by reducing the number
   of system calls on the RRD files.  For instance, it currently seems
   to be open(2)ing each target's RRD file 4 times each round
   (presumably because of the tune and threshcheck).

Thanks for your help - and also thanks to the others that followed-up
in the list and privately!

Dave

-- 
plonka at doit.wisc.edu  http://net.doit.wisc.edu/~plonka  ARS:N9HZF  Madison, WI

--
Unsubscribe mailto:mrtg-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:mrtg-developers-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/mrtg-developers