[rrd-users] Re: Big rrd files (too big !)

[Vial, Sylvain]

Hello,
I need to create reports for tcp_source_ip_addresses,
udp_source_ip_addresses, tcp_destination_ports and
udp_destination_ports.
For the moment the best way I found is to generated 4 lists (4 top ten).
But I want 4hours period reports and 1 daily reports for each list. So I
obtain 28 reports that I have to sort to eliminate duplicated values.
Then I will obtain 4 lists of the frequent used dest/src ports/ip-addr
for tcp/udp protocols.
But I have difficulties to understand how a round robin database works.
For me, I understood that I have to create one rrd file per dest/src
port/ip-addr (tcp/udp) I have in my 4 lists.
For each rrd files, I create 2 data sources in order to display graphs
in bytes or packets, but then I don't know how many RRA I have to
create.

I don't know what is the aim to create RRA, for example, for a weekly
archive, monthly archive, etc if it's, for example, display graph for
one month of captured flows.

I think this have a big influence on the size of the rrd files.

Thanks for your help.


-----Original Message-----
From: rrd-users-bounce at list.ee.ethz.ch
[mailto:rrd-users-bounce at list.ee.ethz.ch] On Behalf Of paul
Sent: Thursday, May 19, 2005 4:55 PM
To: rrd-users at list.ee.ethz.ch
Subject: [rrd-users] Re: Big rrd files (too big !)

I have done similar, however used a different approach.

using flowcat, flowfilter etc generated the output I needed. From the
analysis of this data, I created a list of most frequent used ports
(ssh, telnet, ftp, rdp etc)

make sure you have a totaldata variable somewhere, to have a reference
on the data found.

create a job to update these specific ports. add rrd db for tcp, udp and
icmp

create a graph where the data of all ports is stacked using different
colors. On top, the line with totaldata and color the difference.

In the same graph, multiplied by -1, create a graph for tcp, udp and
icmp

For me, this has given a great insight in traffic patterns and analysis
if things were different than normal.

Additionally, I created a small job to display the top50 from netflow in
a table on a webserver - combined with the rrd graph, the tooling to
defend.

hth

paul

On Thu, 19 May 2005 15:41:54 -0500
"Vial, Sylvain" <Sylvain.Vial-1 at ou.edu> wrote:

> Hello,
>  
> I'm actually working on a perl script to provide the top ten for
source
> ip addresses and destination ports for the tcp/udp protocols.
> I use the netflow tools (flow-cat, flow-report) to generate top ten
and
> I've created rrd files to generate graph as you can find on the
honeynet
> brazilian project 
> (http://www.honeypots-alliance.org.br/stats/flows/tcp-udp/).
> My problem is that I generate rrd for each ip address and each port I
> detect in my ft files (flow capture).
> Each file takes 3MB of hard disk space, so it takes finally a huge
space
> on my pc.
> I'm like a rookie with rrdtool and perl, so if someone could explain
me
> why it takes so much space and if it exits a better way to realize the
> same thing as brazilian.br, it will be great.
> Thanks for your help.
>  
> Sylvain VIAL
> --
> Unsubscribe
mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
> Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
> Archive     http://lists.ee.ethz.ch/rrd-users
> WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi
> 
> 

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-users
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-users
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi