[rrd-users] Re: Memory usage on large RRAs

[Alex van den Bogaerdt]

On Sun, Dec 04, 2005 at 11:31:07PM +0100, Fabien Wernli wrote:

> For each node/rrd:
> 	for each CPU: user, nice, sys, irq, sirq, io and idle
> 
> Then I do a CDEF to sum up all CPUs
>      Then a CDEF to sum up all nodes

If everything is working as intended, the first DEF collects
data from the database, each subsequent DEF points to the
existing data.  This is, of course, per RRD file you use.

Then each CDEF will generate a new array of data. If memory
serves me well it will be a one dimensional array.  So if you
are storing four CPUs, each CDEF will need about 1/28 of the
amount of memory a DEF needs.

> I just tried to sum up only one CPU per node. I can do a few more nodes, but it seems
> difficult to graph more than 100 cpus on a 2G RAM machine without getting
> the annoying "ERROR while updating mydemo.rrd: malloc fetch data area"
> message...

What I ment is: if you are only adding "user" for each cpu, you may
be better off with one database for "user", another for "nice", and
so on.  This way each DEF will only consume memory for that one DS,
multiplied by the amount of time intervals.  This won't help (even
make things slightly worse) when you are using all DSes.  Think of
it like this: the first DEF consumes memory.  Each other DEF using
the same file (and, in case of v1.2, the same time) will reuse the
already fetched copy so is "free" in memory usage terms.  If you
only have "user" in this database, each DEF uses only 1/7 of what
you would currently use.  If you use "user" and "sys", you only
need 2/7 the amount of memory, and so on.  However, because each
RRD fetch results in a bit of overhead/house keeping, there is a
slight disadvantage.  You may not even notice it if you do use all
seven DSes.  The overhead is per file, not per DS or per row.

-- 
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-users
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi