From s.shipway at auckland.ac.nz Thu Dec 16 23:36:38 2010 From: s.shipway at auckland.ac.nz (Steve Shipway) Date: Thu, 16 Dec 2010 22:36:38 +0000 Subject: [mrtg-developers] Idea for alternative MRTG scheduler Message-ID: <28E447343A85354483BCF7C3E9D5EAA5149F23C9@uxcn10-1.UoA.auckland.ac.nz> Here's an idea. Currently, MRTG will process all Targets until there are none left, using up all the available threads, and then sleep until the next polling cycle. This can be problematic if you configure more targets, or there is an outage, and suddenly you do not have enough threads to process everything in the 5min window. Also, it results in a large burst of activity at the start of the window followed by silence. So, how about this - MRTG already knows how many targets there are, and the interval. It calculates x=(interval/#targets)x0.9 (the 0.9 is to allow time for the final checks to complete) and then kicks off a new Target to process every x seconds, starting a new thread if required (possibly up to a specified upper limit). This would possibly end up with each thread processing a single target and then exiting, with the master starting a new thread per Target. I think this may be how the Nagios check scheduler works? It would certainly solve the problems of (a) uneven CPU usage and (b) running out of window time when you add more targets but not more threads. The drawback is that, of course, you need to have sufficient CPU/memory to handle the potentially large number of threads that could result. Since Tobi is currently in a coding mood, I thought it best to get the suggestions in quick :) Steve ________________________________ Steve Shipway ITS Unix Services Design Lead University of Auckland, New Zealand Floor 1, 58 Symonds Street, Auckland Phone: +64 (0)9 3737599 ext 86487 DDI: +64 (0)9 924 6487 Mobile: +64 (0)21 753 189 Email: s.shipway at auckland.ac.nz P Please consider the environment before printing this e-mail -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.oetiker.ch/pipermail/mrtg-developers/attachments/20101216/845c1bb5/attachment.htm From tobi at oetiker.ch Fri Dec 17 08:23:12 2010 From: tobi at oetiker.ch (Tobias Oetiker) Date: Fri, 17 Dec 2010 08:23:12 +0100 (CET) Subject: [mrtg-developers] Idea for alternative MRTG scheduler In-Reply-To: <28E447343A85354483BCF7C3E9D5EAA5149F23C9@uxcn10-1.UoA.auckland.ac.nz> References: <28E447343A85354483BCF7C3E9D5EAA5149F23C9@uxcn10-1.UoA.auckland.ac.nz> Message-ID: Hi Steve, Yesterday Steve Shipway wrote: > Here's an idea. > > Currently, MRTG will process all Targets until there are none > left, using up all the available threads, and then sleep until > the next polling cycle. > > This can be problematic if you configure more targets, or there > is an outage, and suddenly you do not have enough threads to > process everything in the 5min window. Also, it results in a > large burst of activity at the start of the window followed by > silence. > > So, how about this - MRTG already knows how many targets there > are, and the interval. It calculates x=(interval/#targets)x0.9 > (the 0.9 is to allow time for the final checks to complete) and > then kicks off a new Target to process every x seconds, starting > a new thread if required (possibly up to a specified upper > limit). This would possibly end up with each thread processing a > single target and then exiting, with the master starting a new > thread per Target. > > I think this may be how the Nagios check scheduler works? It > would certainly solve the problems of (a) uneven CPU usage and > (b) running out of window time when you add more targets but not > more threads. The drawback is that, of course, you need to have > sufficient CPU/memory to handle the potentially large number of > threads that could result. I agree scheduling could be improved. Your aproach asumes an even delay with all targets I guess ... so if there is a slow target at a late stage of the spread out polling activity it would not have enough time to complete unless every target is run acynchronously causing a large horde of processes or threads. The motivation for your suggesting seems to be to not overwhelm devices or networks with intense polling, so I guess one aproach would be to implement some sort of polling rate limit which makes sure mrtg does not 'kill' anyone by being to hasty. > Since Tobi is currently in a coding mood, I thought it best to > get the suggestions in quick :) I only processe the mrtg backling ... no clever new stuff from my end ... and I am intending todo the same for rrdtool now ... and then publish 1.4.5. a bugtracker with a backlog is such a sad thing ... cheers tobi > > Steve > > ________________________________ > Steve Shipway > ITS Unix Services Design Lead > University of Auckland, New Zealand > Floor 1, 58 Symonds Street, Auckland > Phone: +64 (0)9 3737599 ext 86487 > DDI: +64 (0)9 924 6487 > Mobile: +64 (0)21 753 189 > Email: s.shipway at auckland.ac.nz > P Please consider the environment before printing this e-mail > > > -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900 From s.shipway at auckland.ac.nz Fri Dec 17 08:43:10 2010 From: s.shipway at auckland.ac.nz (S Shipway) Date: Thu, 16 Dec 2010 23:43:10 -0800 (PST) Subject: [mrtg-developers] Idea for alternative MRTG scheduler In-Reply-To: References: <28E447343A85354483BCF7C3E9D5EAA5149F23C9@uxcn10-1.UoA.auckland.ac.nz> Message-ID: <28E447343A85354483BCF7C3E9D5EAA5149F5519@uxcn10-1.UoA.auckland.ac.nz> The reason I thought the scheduling interval should be multiplied by 0.9 was to allow 10% of the interval free at the end for the last targets to complete. Obviously this assumes that no target will take >10% of the time, which would be 30s in a normal 5min interval. The other possibility is to make the window completely sliding; so it doesnt matter if the poll overruns the interval, it just gets stored into the next interval. This would probably not work so well with the existing scheduler, though. You could think of the existing Forks: directive as an indicator of the maximum polling rate; the problem is if this is not high enough to handle the number of defined targets, the system won't automatically increase it (but maybe it could? If the polling cycle doesnt complete within the interval, automatically increase Forks: by one until it reaches some other defined limit?) I did write an alternative scheduler for MRTG using shellscript that used this algorithm; the disadvantage was that it respawned MRTG for each cfg file individually, so it was not efficient with resources, and still had the problem of individual targets within a single cfg file having to schedule together. Steve Steve Shipway University of Auckland ITS UNIX Systems Design Lead s.shipway at auckland.ac.nz Ph: +64 9 373 7599 ext 86487 -- View this message in context: http://mrtg-mailinglists.795376.n2.nabble.com/Idea-for-alternative-MRTG-scheduler-tp5843836p5844683.html Sent from the MRTG Developers Mailinglist mailing list archive at Nabble.com. From nickmbugus at gmail.com Mon Dec 20 09:50:38 2010 From: nickmbugus at gmail.com (Nick Wambugu) Date: Mon, 20 Dec 2010 11:50:38 +0300 Subject: [mrtg-developers] MRTG Aggregate Message-ID: Have issues with mrtg aggregate which have been running ok but broke down last week showing a straight line and seems it's not running at all even when i re-run the indexmaker command, any idea why this behavior -- "STRENGTH DON COME FROM PHYSICAL CAPACITY. COMES FROM AN INDOMITABLE WILL" -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.oetiker.ch/pipermail/mrtg-developers/attachments/20101220/ce2ee675/attachment-0001.htm From tobi at oetiker.ch Mon Dec 20 09:58:34 2010 From: tobi at oetiker.ch (Tobias Oetiker) Date: Mon, 20 Dec 2010 09:58:34 +0100 (CET) Subject: [mrtg-developers] MRTG Aggregate In-Reply-To: References: Message-ID: Hi Nick, I guess your mrtg is not getting data anymore ... try running it by hand to see what it complains about. cheers tobi Today Nick Wambugu wrote: > Have issues with mrtg aggregate which have been running ok but broke down > last week showing a straight line and seems it's not running at all even > when i re-run the indexmaker command, any idea why this behavior > -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900 From s.shipway at auckland.ac.nz Wed Dec 22 22:01:50 2010 From: s.shipway at auckland.ac.nz (S Shipway) Date: Wed, 22 Dec 2010 13:01:50 -0800 (PST) Subject: [mrtg-developers] OID for getting CPU usage in Win XP In-Reply-To: <1293022486011-5859062.post@n2.nabble.com> References: <1293022486011-5859062.post@n2.nabble.com> Message-ID: <28E447343A85354483BCF7C3E9D5EAA514A10758@uxcn10-1.UoA.auckland.ac.nz> There is a cfgmaker template available at http://www.steveshipway.org/cfgmaker that probes the host MIB and builds an appropriate cfg file. Steve ________________________________ Steve Shipway ITS Unix Services Design Lead University of Auckland, New Zealand Floor 1, 58 Symonds Street, Auckland Phone: +64 (0)9 3737599 ext 86487 DDI: +64 (0)9 924 6487 Mobile: +64 (0)21 753 189 Email: s.shipway at auckland.ac.nz P Please consider the environment before printing this e-mail From: prakashm [via MRTG Mailinglists] [mailto:ml-node+5859062-354394647-56796 at n2.nabble.com] Sent: Thursday, 23 December 2010 1:55 a.m. To: Steve Shipway Subject: OID for getting CPU usage in Win XP Hi, I am writing a small program to get the "CPU usage" in Windows XP. I have installed SNMP Service in the target Win XP system. I assume that MRTG gets this value through SNMP. I am new to SNMP and if someone can point me to the appropriate MIB/OID, for doing the same, it would be very helpful. Thanks in advance, Prakash ________________________________ View message @ http://mrtg-mailinglists.795376.n2.nabble.com/OID-for-getting-CPU-usage-in-Win-XP-tp5859062p5859062.html To start a new topic under MRTG Developers Mailinglist, email ml-node+795384-758908351-56796 at n2.nabble.com To unsubscribe from MRTG Developers Mailinglist, click here. -- View this message in context: http://mrtg-mailinglists.795376.n2.nabble.com/OID-for-getting-CPU-usage-in-Win-XP-tp5859062p5860600.html Sent from the MRTG Developers Mailinglist mailing list archive at Nabble.com. From prakashm at hcl.com Wed Dec 22 13:54:46 2010 From: prakashm at hcl.com (prakashm) Date: Wed, 22 Dec 2010 04:54:46 -0800 (PST) Subject: [mrtg-developers] OID for getting CPU usage in Win XP Message-ID: <1293022486011-5859062.post@n2.nabble.com> Hi, I am writing a small program to get the "CPU usage" in Windows XP. I have installed SNMP Service in the target Win XP system. I assume that MRTG gets this value through SNMP. I am new to SNMP and if someone can point me to the appropriate MIB/OID, for doing the same, it would be very helpful. Thanks in advance, Prakash -- View this message in context: http://mrtg-mailinglists.795376.n2.nabble.com/OID-for-getting-CPU-usage-in-Win-XP-tp5859062p5859062.html Sent from the MRTG Developers Mailinglist mailing list archive at Nabble.com.