[rrd-users] MRTG/14all ideas! Parser module and "Perl Code Targets" (Was "Re: [rrd-users] Re: new log2rrd beta - handles MAX values, etc.")

[Jakob Ilves]

Hello!

(My apologies for cross posting to two lists.  Before you answer,
consider if the reply is relevant for both list and if not, remove the
inappropriate.)

Rainer Bawidamann wrote:

[ Quite some lines deleted ]

> I think it wouldn't take much time (for me). I would take the parser out
> of mrtg and add some calls to log2rrd. Maybe I'll do it because it's
> good for 14all ;-)

It would be VERY good for the MRTG and 14all community, especially if
you isolate the MRTG-config parser into a independent Perl module.  That
way, you could let MRTG, 14all, log2rrd and other utilities use the SAME
code for interpreting the config file.  Once you've created the Perl
module, submit it to Tobi so he can review it.  If it's good enough he
will most likely use that Perl module for future MRTG releases.

Ideally, the code should accept a file name and return a reference to a
(big) data structure with the contents of the MRTG config file.  This
way you could actually parse several MRTG files and have several data
structures as a result.

Of course, the structure of the data must be well documented (volunteers
could read your modules code and then write the docs).  Such
documentation could make it possible to implement an enhancement for
MRTG:  "Perl Code Targets"!  That is, when MRTG "polls" the "Perl Code
Target", a snippet of Perl code is run in a strictly defined run time
environment with all the MRTG data structures available.

Consider these two (imaginary) MRTG targets in a cfg file:

-----8<------------8<------------8<-------

Target[my-router.response]: {
	my($maxtime,$mintime,$sentpackets,$receivedpackets);
	($maxtime,$mintime,$sentpackets,$receivedpackets)=
		split(`hacked-mrtg-ping-probe -k 5 my-router.mycompany.com`);

	$MRTG::invalue=$maxtime;
	$MRTG::outvalue=$mintime;

	$MRTG::thisfile{my-router.availability}{invalue}=
		(100*$receivedpackets/$sentpackets);
	$MRTG::thisfile{my-router.availability}{outvalue}=0;

}

Target[my-router.availability]: {}

-----8<------------8<------------8<-------

So, what does this "Pseudo-Perl" code do?

The first target (my-router.response) runs a snippet of Perl code within
curly braces ("{}").  The code first defines four temporary variables
for max and min response times and how many packets were sent and
received.  Then it runs a modified mrtg-ping-probe which also provides
number of sent and received packets and assigns the right values to the
right temporary variables (and here the code is really PSEUDO Perl ;-). 
Then the code stores the $maxtime and $mintime into the current target's
in and out values (current target is my-router.response).  After that it
does a neat trick, it calculates how many percent of the packets
actually made it and then use that to set the in and out of ANOTHER
target (my-router.availability).  Then the code exit.

The second target (my-router.availability) does NOTHING!  This because
it's values were set when the first target was polled!

(Yes, I just made up those
"$MRTG::thisfile{my-router.availability}..."-things but you get the
idea, I hope.  After all, it's PSEUDO Perl)

I believe these Perl Code Targets to be very powerful!  Imagine you run
a script to get a bunch of metrics, then you can feed them all into
different MRTG targets!  You can also do a lot of other things as well! 
Of course, these "Perl Code Targets" must be read by the MRTG parser and
then stored as code which then is evaluated by the Perl script.

Also, consider using Perl code snippets as the action for thresholds,
that could also be a blast!  Hmm, there might be other places as well
where cute pieces of Perl could do wonders.  Create a global title with
Title[$] but include some perl code which alters the code slightly to
reflect which target we are dealing with.

(Also consider how easy it will be to create amazingly complex, ugly
and/or uncomprehensive MRTG configuration files.  A potential candidate
for "Application config maintenance in Hell"! ;-).

The case I use as an example above solves the problem that even if you
get more than two metrics from a poll, you can only collect two metrix
in a single MRTG target.  This is of course possible to work around by
doing multiple polls but it's better to just do one poll and get every
metric, especially as a poll means sending 10 ping packets.

> Bye ... Rainer

Best regards

/IlvJa


PS.  The reason I stress the need for parsing multiple independent MRTG
files is that I've written a prototype of an app which reads several rrd
files from collections defined in MULTIPLE MRTG config files and then
presents the results as a HTML page and I'm certain that others will
value the ability to let a script read a bunch of MRTG files.

What my app does?  Well, it provides you with a screen containing of a
large number of "strips", two per MRTG collection.  The strips are
actually 10 pixel high RRD graphs which are green under ideal conditions
but shows red parts whenever the MRTG collection violates a specified
threshold in any direction.  This way you get an index of your MRTG
collections and the index is very high density.

The app required a hack to rrd_tool (namely to output pictures
containing nothing but the graph itself or pictures containing only the
X-axis etc.) but I'm currently not terribly satisfied how I implemented
that hack so I'm considering to rewrite it.  (No, It's not published in
it's current version)
/DS
-- 
                (Jakob Ilves) <jakob.ilves at oracle.com> 
             {Oracle Global IT, Network Management Group}
[Office as well as mobile phone: +46/8/477 3666 | Fax: +46/8/477 3572]

-- Attached file removed by Listar and put at URL below --
-- Type: text/x-vcard
-- Desc: Card for Jakob Ilves
-- Size: 424 bytes
-- URL : http://www.ee.ethz.ch/~slist/pantomime/02-jakob.ilves.vcf