[mrtg] Re: Whither the 95th percentile parser?

[Pete Templin]

On Wed, 15 Dec 1999, Norm Mackey wrote:

> Pete Templin wrote:
> 
> Thanks!  Could you tell me, though, over what time period this is
> doing the averaging and discarding?  Am I assuming wrongly that the
> 95th percentile discards are being calculated only over the 48 hour
> archiving period you mentioned?

Yes, you have an incorrect assumption (probably due to my lack of
comments).  :)

Just to be safe, each night the archive script archives about 48 hours of
data by appending it to a file.  The 95 script (which is configurable by
$threshold to another threshold) stuffs the values into a hash (to discard
duplicate archive entries of the same sample; most entries will actually
show up twice in the logfile), and then calculates the usage over the
whole period of the archive.
 
> If so, I'll try to contribute a modified version for a one-month
> period of 5-minute samples, what I need according to the contracts.
> And again thanks for the example!

Actually, I've just updated both scripts to provide a monthly summary of
all of the data contained in the archive (you may or may not wish to
truncate the archives - site policy, etc.).  They both read from a config
file now, to make it easier (no arguments are necessary if you have a
config file).  So my T1 at my apartment is using under 6kbps.  :)

Here's the output:

tcsh> mrtg95
Client 1, Street Address usage for Dec 1999 is 22176
Pete Templin, Camino Villa usage for Dec 1999 is 5192


mrtg_archive.conf:

cisco/cisco3.2.log:Client 1, Street Address
cisco/cisco2.2.log:Pete Templin, Camino Villa


mrtg95:

#!/usr/bin/perl
#

$archive_dir = "/usr/local/mrtg/archives";
$threshold = "95";
$conf = "/usr/local/scripts/mrtg_archive.conf";
@months = ("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec");

if (-e $conf) {
    open(CONF,"$conf");
    while(<CONF>) {
	chomp;
	push(@elements,$_);
    }
}

@elements = @ARGV if (@ARGV);

die "Need a list of parameters to monitor!\n" unless (@elements);

for ($a=0; $a<=$#elements; $a++) {	# Each logfile as an argument.
    ($path,$comment) = split(/:/,$elements[$a]);
    open(LOG,"$archive_dir/$path");
    while (<LOG>) {
	(@data) = split(/ /);
	@date_info = localtime($data[0]); # Get the date info
	$date_info[5] = $date_info[5] + 1900; # Calendar Year
	if ($date_info[4] < 10) {
	    $date_info[4] = "0" . $date_info[4];
	}
	$date_info[4] = "00" unless ($date_info[4]);
	$datestamp = "$date_info[5]$date_info[4]";
	$record{$datestamp} = 1;	# Know which months our data covers
	$data{$datestamp}{$data[0]} = "$data[1]:$data[2]";
    }

    @dates = keys ( %record );
    @sorted_dates = sort {$a <=> $b} @dates;

    for ($b=0; $b<=$#sorted_dates; $b++) { # Loop through each month
	@in = undef;
	@out = undef;

	$year = substr($sorted_dates[$b],0,4);
	$month = substr($sorted_dates[$b],4,2);
	$month_name = $months[$month];

	@values = values %{ $data{$sorted_dates[$b]}};

	$entries = scalar (@values);
	$index = -1 + int($entries * $threshold / 100);

	for ($c=0; $c<=$#values; $c++) {
	    ($in,$out) = split(/:/,$values[$c]);
	    push(@in,$in);
	    push(@out,$out);
	}
	
	@sort_in = sort {$a <=> $b} @in;
	@sort_out = sort {$a <=> $b} @out;

	$high_in = $sort_in[$index];
	$high_out = $sort_out[$index];

	$level = $high_in;
	$level = $high_out if ($high_out > $level);

	$level = $level * 8;

	print "$comment usage for $month_name $ year is $level\n";
    }
}


mrtg_archive:

#!/usr/bin/perl

$mrtg_dir = "/usr/local/mrtg";
$archive_dir = "/usr/local/mrtg/archives";
$conf = "/usr/local/scripts/mrtg_archives.conf";

if (-e $conf) {
    open(CONF,"$conf");
    while(<CONF>) {
	chomp;
	push(@elements,$_);
    }
}

@elements = @ARGV if (@ARGV);

die "Need a list of parameters to monitor!\n" unless (@elements);

@time_data = localtime(time);
$time_data[4]++;
$time_data[5] = $time_data[5] + 1900;

for ($a=0; $a<=$#elements; $a++) {
    ($path,$comment) = split(/:/,$elements[$a]);
    (@path) = split(/\//,$path);
    unless (-d "$archive_dir/$path[0]") {
	system("mkdir $archive_dir/$path[0]");
    }

    unless (-f "$archive_dir/$ARGV[$a]") {
	system("touch $archive_dir/$path[0]");
    }

    open(LOG,"$mrtg_dir/$ARGV[$a]");
    open(COPY,">>$archive_dir/$ARGV[$a]");
    while (<LOG>) {
	if ($count >= 3) {
	    print COPY;
	}
	if ($count >= 598) {
	    last;
	}
	$count++;
    }
}



Pete

--
Peter J. Templin, Jr., CCNA
Systems and Networks Administrator

On-Line Internet Services - URDirect.net
A division of Global On-Line Computers
2414 Babcock Rd. Suite 106		templin at urdirect.net
San Antonio, TX 78229			(210)692-9911

--
Unsubscribe mailto:mrtg-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:mrtg-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/mrtg