[rrd-users] to many "data agregation" transactions at the same time
kristoff.bonne at skypro.be
Thu Dec 15 21:00:27 CET 2011
On 15-12-11 17:58, Simon Hobson wrote:
>> We have an annoying problem.
>> However, as this runs on crontab, we have now end up with a situation
>> where a lot of .rrd files (thousands) where created at the same time of
>> the day. The result is that the "data aggregation" process of all these
>> .rrd files start at roughly the same time; and this causes disk IO
> I doubt it. RRD tool does not aggregate data based on arbitrary times
> (eg file creation). Aggregation (consolidation/normalisation) is done
> when data is entered, and all consolidated time periods are an
> integer multiple of the period size since unix epoch (midnight, 1st
> Jan 1970).
> Basically, every time you do an update, rrd tool will update all
> consolidated time periods which have ended since the last update.
> Thus if you run a script once a day at 1am, it will close the 24 hour
> consolidation for the previous day regardless of what time of day the
> file was created. It would also update and close 12 2 hour
> consolidation periods for the previous day.
OK, I get it. I did notice all .rrd files had a timestamp for the 24h
stats at 2 AM CEST/1AM CET (midnight UTC).
Hmm. As most script run either somewhere between every 5 to 15 minutes;
all rrd files (over 10000) do the "24h consolidation" at the same time.
There is no solution for that.
Cheerio! Kr. Bonne.
More information about the rrd-users