[mrtg] Re: linux performance

[Daniel J McDonald]

On Tue, 2006-10-24 at 07:45 -0500, Perry, Duane wrote:
> I may be trying to do more than MRTG was ever designed to do but right
> now I am polling about 25,000 interfaces with a Dell 2850 running Red
> Hat Linux Enterprise rel 4. 

Ok, I'm doing about 10,000, Dell 2650, Mandriva Linux.  But most of mine
are snmp v3, which has more cpu load.

>  It is actually working but barely finishes
> processing the rrd files in the 5 minute polling cycle.  We ordered a
> new Dell 2950 with Dual Core cpu and I expected much faster processing.
> The reality is that it does not complete much faster than the old box.
> I had consolidated all of my Targets into one large config file for
> polling and keep separate config files for display.  I have been using a
> "fork" setting of 16.

Make certain you aren't running out of ram - monitor it with top and see
if you are swapping.  You will get better performance with fewer forks
if you can keep from swapping.

>   I issue "/usr/bin/mrtg /usr/local/cfg/targ1.cfg"
> from a crontab.  The polling part seems to go quite quickly (~30
> seconds) but writing the results to the rrd files is taking as long as 3
> or 4 minutes.  

The other thing to do is ensure that snmp times out quickly.  I
use ::2:1:1:3 as my snmp options.  If I have a core switch missing, it
can take a long time and mrtg never really recovers.  I suppose I should
hack something to verify the box is up with a ping before I fire off the
snmp.


> I tried splitting the config up into multiple parts and
> running them concurrently in the background but it does not seem to
> help.  I suspect that some module called by the mrtg script is single
> threading the process and more processes just leads to more swapping and
> is actually LESS efficient.  
>  
> 
> Does anyone have any suggestions on ways to speed up the processing?  Is
> anyone out there monitoring an enterprise this large?  Gathering
> IfInOctets and IfOutOctets for every port on campus is 90% of the load.
> I also monitor firewall connections, wireless connections, modems and a
> few other miscellaneous stats.  I would like to continue adding other
> graphs as we identify MIBS that will help us monitor network health.  I
> thought throwing massive processing power at the problem would give me
> some room for growth but right now I don't feel like I have much of a
> cushion.

Hate to say it, but multiple pollers is probably the way to scale.  Then
rsync all of the rrdfiles to a central display box.


-- 
Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX
Austin Energy
http://www.austinenergy.com

--
Unsubscribe mailto:mrtg-request at list.ee.ethz.ch?subject=unsubscribe
Archive     http://lists.ee.ethz.ch/mrtg
FAQ         http://faq.mrtg.org    Homepage     http://www.mrtg.org
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi