[rrd-users] to many "data agregation" transactions at the same time

[Simon Hobson]

Kristoff Bonne wrote:

>We have an annoying problem.
>
>We have a tool that runs daily that does "network discovery" of net
>devices and creates .rrd automatically based on this.
>
>
>However, as this runs on crontab, we have now end up with a situation
>where a lot of .rrd files (thousands) where created at the same time of
>the day. The result is that the "data aggregation" process of all these
>.rrd files start at roughly the same time; and this causes disk IO
>congestion.

I doubt it. RRD tool does not aggregate data based on arbitrary times 
(eg file creation). Aggregation (consolidation/normalisation) is done 
when data is entered, and all consolidated time periods are an 
integer multiple of the period size since unix epoch (midnight, 1st 
Jan 1970).

>Are the timestamps in the "every 24 hours" stats related to the
>timestamps of (say) the "every 2 hours" stats?

As above, all 24 hour consolidations will use periods that start/end 
at midnight UTC. 2 hours consolidations will have periods that 
start/end at midnight, 2am, 4am, ... 10pm, midnight - again all in 
UTC. Thus every 12th 2 hour consolidation will end at the same time 
(midnight UTC) as a 24 hour consolidation.

Basically, every time you do an update, rrd tool will update all 
consolidated time periods which have ended since the last update. 
Thus if you run a script once a day at 1am, it will close the 24 hour 
consolidation for the previous day regardless of what time of day the 
file was created. It would also update and close 12 2 hour 
consolidation periods for the previous day.

-- 
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.