[rrd-users] Re: Stacked area's limit

[Alex van den Bogaerdt]

On Sun, Sep 24, 2006 at 10:59:30AM +0100, Simon Hobson wrote:

> >The result being I loose all negative data ( the second
> >~100 set of stacks ) and 95% totals go nan.
> 
> I have an application that graphs a stack of 254 positive plus a 
> stack of 254 negative values (traffic by IP address). As long as it 
> works then it works no differently to the same graphs using a dozen 
> values - it's slow as it needs nearly 2G of memory and sometimes the 
> system runs out of swap space. I didn't realise I'd need so much when 
> I build the system and allowed 1G swap on a system with 256M ram - 
> since upgraded to 1G ram.
> 
> Interestingly, I've noticed that the same graph needs increasing 
> amount of memory as you go back in time - ie using coarser 
> consolidations.


Careful planning of your RRD may save memory.

If you only have RRAs spanning 5 minutes per row, and if you are looking
at a year's worth of data, that's 105120 rows.

If on the other hand you are able to select the RRA with one day's worth
of data in each row, that's only 365 rows.

Each memory location is 6 or 8 bytes.

Now, suppose you have 510 DEFs.  The difference would be (105,120 - 365)*6 =
628,530 bytes per DEF, or 320,550,300 bytes in total.  And this is for
one DS per RRA.  Often one has two or even more DSes.

Then you start using CDEFs... increasing the memory usage still more.

It also matters how you process your data:
"
CDEF:a=b,c,+
CDEF:d=a,d,+
"

uses more memory than

"
CDEF:d=a,b,c,+,+
"

This mail does not describe all of the memory used.  Also, the information
is simplified (and thus slightly wrong).  However, the biggest memory
consumer is the data in DEF and CDEF.  The biggest improvement you can
make is to have the input data (RRAs, fetched by DEF) in the resolution
you are going to need.

Make sure you fetch the right data.  There's a huge difference between
"--start end-24h --end now" run at midnight and "--start end-24h --end 00:00".
The difference is "--end now" not being equal to "--end 00:00" unless
you happen to run the commend _exactly_ at midnight.

If you have 5-minute RRA and 1-hour RRA:
Selecting "--start end-7d --end now" just after the whole hour results in
fetching data from the 5-minute RRA.  7*24 hours plus one more interval (to
cover the difference between 'now' and the whole hour), or 2017 rows.

Selecting "--start end-7d --end 00:00" may result in fetching data from
the 1-hour RRA, 168 rows.  This depends on more than just start and end
times but if you're displaying 168 pixel columns (so:  --width 168) then
this RRA would be selected.


Also, you may not want midnight to midnight.  You probably want UTC 00:00
to UTC 00:00 to view days.
To view weeks, you do not want sunday to sunday, you want thursday 00:00 UTC
to thursday 00:00 UTC.  

In general: make sure start and end times match your RRA, and the amount
of pixels matches the amount of rows.

HTH
-- 
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-users
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi