[rrd-users] Conception problem on a rrd database

[Gwenael Lahay]

Thank you for your answer.

I need to store the data during 1 month.
I store the data every 2 minutes.
One rrd file has a size of 150 ko. I create 5 files for a connection (source
<-> destination for a protocol and for a service)
The subnet size is not too big but there is a lot of different source ip
(thousands).
My hard drive could no manipulate so much files at the same time.
Consequently, i have performance problems to write and read data.


2008/2/18, Carsten Aulbert <carsten at welcomes-you.com>:
>
> Hi, I'm new here so please bear with me.
>
> Gwenael Lahay wrote:
> >
> > For the moment, i create rrd file to store data for one net flow :
> > - a total file which stores the flow for a destination of the subnet
> > - a service file which store the flow for a service and for a
> > destination of the subnet
> > - a total file which stores the flow from a source (in direction of the
> > subnet)
> > - a service file which store the flow for a service and for a source (in
> > direction of the subnet)
> > - a protocol file which store the flow for a protocol and for a source
> > (in direction of the subnet)
> >
> > As i create a file for each sources and for each destinations (for each
> > services and for each protocols), the size of rrd database is enormous.
>
> What's the typical size for each file and how long do you want to store
> the data for? You might want to average over some time to save data if
> that is possible.
>
> How large is the subnet, i.e. how many machines are you expecting? Even
> with a rrd size of 2MB and 250 possible hosts on a net you would
> generate "only" 250*249*2 ~ 120 GB of archives.
>
> Remember: rrd is constant size, i..e once created they won't change
> their size.
>
> >
> > My questions :
> > Do you think that it exists a better way to store all this data ?
> > Should i replace RRD database with another type of database (MySQL, ...)
> ?
> >
>
> I don't know, depends on what you need. However, I think that with any
> SQL you need to make sure of a way to purge/summarize data. Otherwise
> the tables will become too large and *SQL too slow.
>
> Just an estimate:
> Storing 32bit counters for each possible connection with 250 hosts, with
> a datapoint every 1s yields already: ~ 250kB/s
>
> You have to decide how much data you want to keep - and all this does
> not yet include the service traffic.
>
> So I think rrds are still good enough for you, but you need to decide
> how much data you need to keep and visualize.
>
> HTH
>
> Carsten
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20080218/f2558e48/attachment.html