[rrd-users] Disk space usage when using RRDTool

[Eldor Rødseth (SystemSoft AS)]

Dear all,

I am looking into efficient ways of storing large amounts of data
for post-processing.

My application shall collect periodically reported data (historical)
from a number of "Units" in my system, and store such data to file(s).
There may be (n*100) such units in my system (max number of units
will probably be <= 1000). I have to assume that each unit will
store its data to one-, or more files.

Reported data items are typically 32-bit number(s). Each reported
data item group may consist of one-, or more 32-bit numbers.
Reporting intervals are typically 5-15 seconds.

Post-processing of the data will be done to enable presentation
by means of graphs and numerical output. The presentation by means
of graph(s) must be done by Java applets, hence I believe that
"rrdtool graph ..." will not be used by us.

I have done a simple test using "rrdtool create ...." and
"rrdtool update ..." and was surprised regarding the file size
produced by RRDTool. I experienced a file size of 450MB within
a very short period after a statement like this:

#ifdef RRDTOOL_TEST
  sprintf( str, "rrdtool create varyparam.rrd --start %lu \
                 --step 5 DS:VaryparamLong:GAUGE:60:U:U \
                 RRA:AVERAGE:0.5:1:12614400", curr_time );
  if (result = system( str ))
    fprintf( stderr, "exit code = %d, errno = %d\n", result, errno );
#endif

I am using a very large number of entries - actually one entry
per 5 second for the next 2 years!
However, this could be the number of records I need to store for
each unit mentioned above.

I guess my bottom-line question is: Would you, as experienced RRDTool users,
recommend the usage of RRDTool for such a scenario?
(An alternative is of course to use ssimple ASCII-files...)

Also - can someone explain to me the reason for the rapidly increasing
file size in my test above? Can this be avoided?

Thanks.

Regards, Eldor R.

------------------------
Eldor Rødseth (MD)
SystemSoft AS
http://www.systemsoft.no
Cellular: +47 950 86888




--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi