[rrd-users] buffers flush every midnight

[Maxim Kuleshov]

Hi!

In self-developed monitoring system I use rrd to manage counters data.
There are tens of thousands of device ports and thus - the same number
of .rrd files. Most of them are updated every 5 minutes.

For the performance optimization I had to tune some of vm.dirty_*
parameters, so with reasonable amount of RAM (used for cache) all runs
smoothly.

But every GMT midnight (local time - 04:00 MSD or 03:00 MSK) system
runs very slowly for about 15 minutes. Analysis of collected performance
data shows that 'Buffers' value greatly decreases during that period
(and then again - increase). So, I guess for unknown reason kernel
flushes all buffered data, therefore large amount of disk I/O occurs.

I have a suspicion that the problem relates to rrd update mechanism.
May be once a day during rrdupdate additional work is performed,
extra file system read/write occurs or something like this?