[rrd-users] FW: 95th %ile

[David Gabler]

Posted back in Feb...

-----Original Message-----
From: David Gabler [mailto:dgabler at TRUELINK.com] 
Sent: Tuesday, February 04, 2003 11:58 AM
To: rrd-users at list.ee.ethz.ch
Subject: [rrd-users] 95th %ile



All,

Inline is some code to get the 95th %ile from a rrd.  There are a few
options and defaults. 
the options are:
--version     version
--help        help
--file        rrd file to read in
-b            display data in bits ( result * 8 )
-s            display standard deviation and mean as well
-q            display the query used to fetch data from rrdtool
-d            display the 95th data

They are fairly self explanatory.  The only required option is --file(-f).
To get the 95th %ile from a rrd I use  ./95th.pl -f my.rrd -s.  

So.  there are a few known issues.
1. The output when using -s most likely will be unsorted.
2. you will need to change $rrdtoolLocation to be correct for your
installation.

The code in here is NOT clean, there are comments missing and much better
ways to do things and most likely there are some bugs.

This program has not been tested on windows, and was never intended to run
on a windows platform.

DISCLAIMER:
I David Gabler nor my Employer will be held liable for ANY problems arising
from this script.  This includes but is not limited to; billing issues,
machine performance degradation, loss of your job, bodily harm, you over all
sanity.  There is no warranty implied or stated.

Best of luck,

David Gabler




Now all that mumbo jumbo is over here you go:

95th.pl:
#!/usr/bin/perl 
############################################################################
###
# 95th.pl - parses rrd files and gives a 95th %ile as well as stats
#
# Copyright (C) 2002 David Gabler
#
#
#
# This program is free software; you can redistribute it and/or
#
# modify it under the terms of the GNU General Public License
#
# as published by the Free Software Foundation; either version 2
#
# of the License, or (at your option) any later version.
#
#
#
# This program is distributed in the hope that it will be useful,
#
# but WITHOUT ANY WARRANTY; without even the implied warranty of
#
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#
# GNU General Public License for more details.
#
#
#
# You should have received a copy of the GNU General Public License
#
# along with this program; if not, write to the Free Software
#
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA  02111-1307,
USA. #
############################################################################
###
use Getopt::Long;

#define where rrdtool is located.
$rrdtoolLocation = '/usr/bin/rrdtool';

#here we assume a 30 day month
$secPerMonth = 2592000;
$start = time - $secPerMonth;

$version = 0.02;

&GetOptions("version", "h|?|help", "f|file=s" => \$file, "b", "s", "q", "d")
and $file or usageDie();

if ($opt_version) {
   print "version $version\n";
   exit;
}
if ($opt_help) {
   usageDie();
}
if (!($file)) {
   die "Specify file to pull data from";
}

#get the data
@data = `$rrdtoolLocation fetch -s $start $file AVERAGE`;

if ($opt_q) {
   print "$rrdtoolLocation fetch -s $start $file AVERAGE\n";
}

#start at 1 to get rid of the header row
($sp, @dss) = split/\s+/, $data[0];
for ($x=1;$x<=$#data;$x++) {
   ($time, @sdata) = split/\s+/, $data[$x];
   for ($y=0;$y<=$#sdata;$y++) {
      $tt = 1;
      $tt *= $sdata[$y]; # Convert the "string" in scientific notation to
non sfn.
      push @{$darr[$y]}, $tt;
      undef $tt;
   }
}

if ($opt_b) {
   print "Results are in BITS\n";
} else {
   print "Results are in BYTES\n";
} 
$y=0;
for $each (@darr) {

   # sort the stack, need to cmp as may have nan and spaceship only croaks 
   @sorted = sort { ($a <=> $b) || ($a cmp $b) } @{$each};

   $num = $#sorted + 1;
   # "dynamic" as to not limmit to 5 min inputs
   $top = $num * .05;

   for ($x=1;$x<=$top;$x++) {
      pop @sorted;
   }
   $last = $sorted[$#sorted];
   if ($opt_b) {
      $last *= 8;
   }
   if ($opt_s) {
      $avgs{$dss[$y]} = $last;
      pop @{$each};
      @{$dss[$y].'sorted'} = @{$each};
   } else {
      print "$dss[$y]:", $last, "\n";
   }
   $y++;
}
if ($opt_s) {
   for $each (keys %avgs) {
         print "$each 95th    : ", $avgs{$each}, "\n";
      if ($opt_b) {
         print "$each std dev : ", stddev(@{$each.'sorted'}) * 8, "\n";
         print "$each mean    : ", mean(@{$each.'sorted'}) * 8, "\n";
      } else   {
         print "$each std dev : ", stddev(@{$each.'sorted'}), "\n";
         print "$each mean    : ", mean(@{$each.'sorted'}), "\n";
      }
      print "\n";
   }
}

if ($opt_d) {
   for $each (keys %avgs) {
      print "$each\n";
      print join("\n", @{$each.'sorted'});
      print "\n";
   }
}

sub usageDie
{
   print STDERR <<EOD;
Usage:
   $0 [-h|-?|--help] [--version] \\
      -f|--file <rrdFile> [-b] [-s] [-q] [-d]
  
      Description
      --version     version
      --help        help
      --file        rrd file to read in
      -b            display data in bits ( result * 8 )
      -s            display standard deviation and mean as well
      -q            display the query used to fetch data from rrdtool
      -d            display the 95th data
EOD
   exit 1;
}


###
# Following lifted from the lite statistics module.
# http://cpan.valueclick.com/authors/id/B/BR/BRIANL/
###

sub sum
{
  return unless @_;
  return $_[0] unless @_ > 1;
  my $sum;
  foreach(@_) { $sum+= $_; }
  return $sum;
}

sub mean
{
  return unless @_;
  return $_[0] unless @_ > 1;
  return sum(@_)/scalar(@_);
}

sub variance
{
  return unless @_;
  return 0 unless @_ > 1;
  my $mean= mean @_;
  return mean map { ($_ - $mean)**2 } @_;
}

sub stddev
{
  return unless @_;
  return 0 unless @_ > 1;
  return sqrt variance @_;
}

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi