[rrd-users] A script to collect information from IPFilter firewalls

[Richard Rose]

Hello all,

After a bit of tinkering, I wrote a script to gather byte counts from
IPFilter firewalls, and because I knew I wouldn't remember how to create
the databses the script expects, it will create those for you to
(destructively or not).

WHAT IT IS

I've called it "rrdgather". It's relatively simple to use. It expects a
*WELL FORMED* /etc/ipf.rules. By wel formed, I mean that if you specify
an IP address, you ought to put a "/32" on the end, and if you make a
rule like "in on xl0" then you have to add "from any to any", and that
you work with IP addresses and port numbers only, no names.

Why these silly restrictions? Well, in this first version, I couldn't be
bothered puttingin the code to lookup by name, so that the output was
the same as ipfstat's. If it's not the same, then you'll get "use of
uninitialized value at line XX CHUNK YY".

HOW TO RUN IT

When called as rrdtool --debug, it will read your /etc/ipf.rules and
print what it's understood of your rules from it, then quit.

When called as rrdtool --create, it will call rrdtool to create the
databases it needs and has understood from your /etc/ipf.rules. Note
this version is *destructive*. It will overwrite existing databases.
This would be called from cron when your database wants to be replaced.

Each RRDTool database created has 2 DS's. One called "out", one called
"in". This means making your logging rules in pairs is not just a good
idea, it's What Rik Does (tm). This doesn't mean it's the right thing to
do, it's just what I wanted to do. If you want to do something else, I'm
accepting patches.

When called as rrdtool --new, it will create only the new databases it
needs. This is for when you add firewall conting rules.

When called as rrdtool --run, it will take a single snapshot of the
counters, and log them. This is designed to run from cron every 5
minutes. You will need to modify hot the databases are created (in this
version) if you want to change how often you are going to sample.

HOW IT WORKS

It's basically a wrapper for ipfstat -haio, which parses the output and
calls rrdtool with the relevant numbers. Note that if you want to store
the average packet sizes too, you can do. I'm simply throwing them away,
but they're the first number repotred by ipfstat -haio

HOW TO MAKE IT DO STUFF

Put a comment in /etc/ipf.rules like this:
# RRD_DIR: /var/log/

Where "/var/log/" is replaced by the directory you keep your RRDTool
databases, WITH A TRAILING SLASH. Yes, I'm that lazy I haven't even
added that check yet. This turns on the parsing of rules.

To specify which database to log to (and create, etc), put a comment in
to your /etc/ipf.rules file like this:
# LOG: ssh

Any rules following the LOG will be logged to the ssh.rrd database.

To stop looking for logs and rules, put a comment like this into
/etc/ipf.rules:
# END-OF-LOG

Now, all you need to know is what kind of rules count the amount of
bytes that go through. This only took e a few days to find, because
they're only mentioned in all the ipfilter docs that I could find about
*once*. This is not good. The rules have an action of "count" rather
than "pass" or "block", for example:
count in on xl0 from any to any

would count all inbound traffic on xl0.

EXAMPLE /etc/ipf.rules

# This is an ipf.rules comment which will be ignored
# So will this
# RRD-DIR: /var/log/
# This is not recognised as a command to rrdgather, so is ignored too
# LOG: ssh
# All machines allow ssh
count in on xl0 proto tcp from any to any port = 22
count out on xl0 proto tcp from any port = 22 to any

# LOG: smtp
# My mail server is at 10.0.0.25
count in on xl0 proto tcp from any to 10.0.0.25 port 25
count out on xl0 proto tcp from 10.0.0.025 port 25 to any

# END-OF-LOG

# Normal firewall rules go here.
block in on xl0 from any to any
pass out on xl0 from any to any keep state

FUTURE IMPROVEMENTS

Only look for count rules in /etc/ipf.rules
Put in name to number conversions for ports and IP addresses.
Anything other people suggest/send me patches for.

THE CODE

At last, the code...

Enjoy,

rik

---8<---
#!/usr/bin/perl -w

sub usage {
    print "usage: rrdgather --debug   -  Show that we can parse the lists\n";
    print "       rrdgather --create  -  Create the relevant databases\n";
    print "       rrdgather --new     -  Create new databases, but leave old in place\n";
    print "       rrdgather --run     -  Gather data every 30 seconds\n";
    print "Handy hint. Run rrdgather --run from cron every 5 minutes\n";
}

if ($ARGV[0] eq ""){
    usage();
    exit 0;
}

# Parse rules
open RULES, "/etc/ipf.rules" or die "Could not open rules for parsing\n";
while (<RULES>){
    chomp;
    s/^\s+//;
    s/\s+$//;
    last if /^# END-OF-LOG/;
    if (/^# RRD-DIR: (\/.*$)/){ $dbdir = $1 unless defined $dbdir; }
    if (/^# LOG: (.*)/ ){ $currlog = $1; push @logs, $currlog; }
    if (defined $dbdir and defined $currlog){
        next if /^#/ or /^$/;
        last if /^# END-OF-LOG/;
        push @$currlog, $_;
    }
}
close RULES;

die "No valid rules found\n" unless defined $dbdir and defined $currlog;

# ... and print
if ($ARGV[0] eq "--debug"){
    print "DB Directory is: $dbdir\n";
    foreach $log (@logs){
        print "Logging to $log\n";
        foreach $rule (@$log){
            print " - " . $rule . "\n";
        }
    }

    exit 0;
}

# ... and create new databases
if ($ARGV[0] eq "--create" || $ARGV[0] eq "--new" ){
    foreach $log (@logs){
        $command = "/usr/local/bin/rrdtool create $dbdir/$log.rrd "; 
        $command .= "DS:in:COUNTER:600:U:U "; # in counter
        $command .= "DS:out:COUNTER:600:U:U "; # out counter
        $command .= "RRA:AVERAGE:0.5:1:288 "; # 5 minute average for 1 day
        $command .= "RRA:AVERAGE:0.5:288:31"; # hour average for 1 month
        if ($ARGV[0] eq "--new"){
            system( $command ) unless -f "$dbdir/$log.rrd";
        } else {
            system( $command );
        }
    }

    exit 0;
}

# ... and run
if ($ARGV[0] eq "--run"){
    open DATA, "/sbin/ipfstat -haio |";
    while (<DATA>){
        chomp;
        @data = split / /;
        $byteCount = $data[1];
        $rule = join ' ', at data[2 ... $#data];
        #print "$rule\n";

        $bytes{$rule} = $byteCount;
    }

    foreach $log (@logs){
        foreach $rule (@$log){
            #print "rule: $rule\n";
            if ($rule =~ /^count in /){ $inBytes = $bytes{$rule}; }
            if ($rule =~ /^count out /){ $outBytes = $bytes{$rule}; }

            $inBytes = $bytes{$rule} if $rule =~ /^count in/;
            $outBytes = $bytes{$rule} if $rule =~ /^count out/;
        }
        $command = "/usr/local/bin/rrdtool update $dbdir/$log.rrd N:$inBytes:$outBytes";
        $command .= " > /dev/null"; # make it a little quieter
        system( $command );
        #print $command . "\n";
    }
}

---8<---


--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi