[rrd-developers] rrd::graph memory usage patch

[Holger Jahn]

Tobi & Gents,
Some days ago I bumped into an issue with rrd::graph that encouraged me
to reactivate my C skills..... ;-)

The issue:

We are using RRD files which are 1.8GB each (800 RRAs per file with 5
mins resolution for one year), which get updated every minute. When I
generate a graph out of the RRAs it works file with time periods "Now =>
2 Weeks back". Every attempt to generate graphs that span a larger time
period, such as half a year, ends up pushing our server into memory
swapping.

Splitting the RRD files into many smaller ones is not an option, since
this would push the I/O subsystem on our server against the wall
(currently we use a total of 36 GB data stored in RRD databases,
reflecting 144,000 SNMP variables).

After having had a deep look into the sources I found that rrd_fetch()
basically reads the entire file into memory (in case of a 1 year
graphing period), even if only one RRA is required to be graphed.

The solution:

I modified some files in the source code of 1.2.10 to only read the CDPs
that are required to be graphed, effectively skipping the RRAs we don't
need.

The code is a crude hack, but working. Basically I have copied all lines
I have changed and commented out the original sections (using C++ //
comment markers to make it easy to identify the original parts I have
changed).

Since I have modified the rrd_fetch(), data_fetch() and rrd_graph()
functions, it might be
highly likely that I have introduced many side-effects into other parts
of RRDtool. Thus it would be great if someone of the RRD Gods would
revise my patch.

Cheers,

Holger


-- Attached file removed by Ecartis and put at URL below --
-- Type: application/octet-stream
-- Desc: rrdtool-1.2.11.patch
-- Size: 18k (18522 bytes)
-- URL : http://lists.ee.ethz.ch/p/02-rrdtool-1.2.11.patch


--
Unsubscribe mailto:rrd-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-developers-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-developers
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi