[mrtg] Re: linux performance

[Steve Shipway]

Daniel McDonald wrote: 
> On Tue, 2006-10-24 at 07:45 -0500, Perry, Duane wrote:
> > I may be trying to do more than MRTG was ever designed to 
> do but right
> > now I am polling about 25,000 interfaces with a Dell 2850 
> running Red
> > Hat Linux Enterprise rel 4. 
> 
> Ok, I'm doing about 10,000, Dell 2650, Mandriva Linux.  
...
> > Does anyone have any suggestions on ways to speed up the 
> processing?  

(Our system: Linux RedHat EL4, 2Gb mem, 2x3GHz Xeon CPU, Adaptec SCSI;
9200 MRTG targets at 5min interval; Normal load: CPU 60%, LoadAvg 3.0,
180 Processes)

We're doing 9200 checks currently, about 30% of which are more complex
host checks involving extra perl processes to retrieve from remote
agents.

We found that the main bottlenecks were initially memory - we now have
2Gb on the server which is easily enough - and Disk throughput.  We
installed a nice fast Adaptec SCSI controller and fast access time
disks, and this improved things considerably.

Also, we used an in-house written script to smear the checks out over
the 5-min window, rather than kicking everything off at once.  Since it
is a dual-CPU 3GHz hyperthreading Xeon box, we can runs several threads
at once but by staggering the collection over the 5-min window it lowers
the peak CPU and memory requirements.  This was another big win.

The final thing we have is to split some checks off onto a separate
server.  As we use the routers2 frontend, this has the ability to
seamlessly merge two separate systems into one nice big virtual
frontend.  The users don't know that one particular graph comes from
MRTG host A and the other from MRTG Host B.  It also lets us devolve
some of the MRTG .cfg file admin to other people.  This is done by
creating stub .cfg files on each system which contain the
'routers.cgi*Redirect:' directive to make routers2 send the request to
the other host.  Both hosts have the same routers2.conf and you're
sweet.

Steve

--
Steve Shipway
ITSS, University of Auckland 
(09) 3737 599 x 86487
s.shipway at auckland.ac.nz

--
Unsubscribe mailto:mrtg-request at list.ee.ethz.ch?subject=unsubscribe
Archive     http://lists.ee.ethz.ch/mrtg
FAQ         http://faq.mrtg.org    Homepage     http://www.mrtg.org
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi