[rrd-users] Re: Memory consumption (Was: Stacked area's limit)

[Simon Hobson]

Alex van den Bogaerdt wrote:

>  > Interestingly, I've noticed that the same graph needs increasing
>>  amount of memory as you go back in time - ie using coarser
>>  consolidations.
>
>
>Careful planning of your RRD may save memory.
>
>If you only have RRAs spanning 5 minutes per row, and if you are looking
>at a year's worth of data, that's 105120 rows.

Got four consolidations :
5 minutes (1 sample) for 48hr
1/2 hour for 14 days
2 hour for 62 days
12 hour for 720 days

And I plot graphs for last 24hour, 7days, 30 days, and 365 days.

I wrote down the VM reached for each graph, and found that as the 
time period went up, so did the memory requirements - with the years 
graph using something like 50% more memory than the days graph, with 
the others fairly evenly spaced in between.

>This mail does not describe all of the memory used.  Also, the information
>is simplified (and thus slightly wrong).  However, the biggest memory
>consumer is the data in DEF and CDEF.  The biggest improvement you can
>make is to have the input data (RRAs, fetched by DEF) in the resolution
>you are going to need.
>
>Make sure you fetch the right data.  There's a huge difference between
>"--start end-24h --end now" run at midnight and "--start end-24h --end 00:00".
>The difference is "--end now" not being equal to "--end 00:00" unless
>you happen to run the commend _exactly_ at midnight.
>
>If you have 5-minute RRA and 1-hour RRA:
>Selecting "--start end-7d --end now" just after the whole hour results in
>fetching data from the 5-minute RRA.  7*24 hours plus one more interval (to
>cover the difference between 'now' and the whole hour), or 2017 rows.
>
>Selecting "--start end-7d --end 00:00" may result in fetching data from
>the 1-hour RRA, 168 rows.  This depends on more than just start and end
>times but if you're displaying 168 pixel columns (so:  --width 168) then
>this RRA would be selected.

Now that is interesting.

All the graphs are done as end=now and start=end-<24H|7D|31D|365D>

I had assumed that all the graphs would use the right consolidation 
because the higher resolution ones will not cover the requested 
period. What I think you are saying is that if I kept the 5 minute 
samples for 8 days, then it would use the 5 minute samples, not the 
1/2 hour consolidation so as to best fill the graph.

Am I right in thinking it will never use two separate consolidations 
(eg the 2 hour one for the bulk of the months graph plus the latest 5 
minute samples to fill in the gap at the end) ?

>
>Also, you may not want midnight to midnight.  You probably want UTC 00:00
>to UTC 00:00 to view days.
>To view weeks, you do not want sunday to sunday, you want thursday 00:00 UTC
>to thursday 00:00 UTC.

At least for the daily and weekly graphs, we pretty well need graphs 
ending about now - and tbh I can't be bothered working out what the 
last period end time should be for any particular graph, the odd 
blank pixel doesn't bother us.

>In general: make sure start and end times match your RRA,

>and the amount
>of pixels matches the amount of rows.

That's one where I probably could improve things.

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-users
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi