[rrd-users] Conception problem on a rrd database

[Carsten Aulbert]

Hi, I'm new here so please bear with me.

Gwenael Lahay wrote:
>  
> For the moment, i create rrd file to store data for one net flow :
> - a total file which stores the flow for a destination of the subnet
> - a service file which store the flow for a service and for a 
> destination of the subnet
> - a total file which stores the flow from a source (in direction of the 
> subnet)
> - a service file which store the flow for a service and for a source (in 
> direction of the subnet)
> - a protocol file which store the flow for a protocol and for a source 
> (in direction of the subnet)
>  
> As i create a file for each sources and for each destinations (for each 
> services and for each protocols), the size of rrd database is enormous.

What's the typical size for each file and how long do you want to store 
the data for? You might want to average over some time to save data if 
that is possible.

How large is the subnet, i.e. how many machines are you expecting? Even 
with a rrd size of 2MB and 250 possible hosts on a net you would 
generate "only" 250*249*2 ~ 120 GB of archives.

Remember: rrd is constant size, i..e once created they won't change 
their size.

>  
> My questions :
> Do you think that it exists a better way to store all this data ?
> Should i replace RRD database with another type of database (MySQL, ...) ?
>  

I don't know, depends on what you need. However, I think that with any 
SQL you need to make sure of a way to purge/summarize data. Otherwise 
the tables will become too large and *SQL too slow.

Just an estimate:
Storing 32bit counters for each possible connection with 250 hosts, with 
a datapoint every 1s yields already: ~ 250kB/s

You have to decide how much data you want to keep - and all this does 
not yet include the service traffic.

So I think rrds are still good enough for you, but you need to decide 
how much data you need to keep and visualize.

HTH

Carsten