[rrd-developers] Locking concept for RRD files, concurrent write/read operations

[Roger J. Meier]

Hi all,

There is a LockRRD() function in rrd_update.c which is called only from
rrd_update() and rrd_resize() functions.  The LockRRD() allocates an exclusive
write lock (F_WRLCK) on the whole RRD file and fails if it can't lock the
file.

Apparently, the concept is to lock the file for each write access, but not for
reading (no shared read locks).

In our environment it seems that -- on a heavy loaded system -- the reading
of RRD data fails from time to time (Linux).

Now the "usual suspect":

What if some process writes an RRD file and at the same time another process
reads it to fetch data or generate a graph? The writing of an RRD file is not
atomic, and the read operations of the reading process may get a mixture of
old and new data.  This is possible since the LockRRD() only prevents
concurrent writings, but allows one process writing a file which may be read
at the same time by other processes.  There is no read lock acquired.

Shouldn't I lock RRD files before reading them (acquire a shared lock) and
wait until the exclusive lock is freed by the writing process?

Did anybody encounter problems fetching data from files on a heavy loaded
system, as we do?

Any feedback concerning this behaviour and the locking concept is appreciated.

-- Roger

-- 
---------------------------------------------------------------
addr://Kasinostrasse 30, CH-5001 Aarau   fon://++41 62 823 9355
http://www.terreactive.com               fax://++41 62 823 9356
---------------------------------------------------------------
Wir sichern Ihren Erfolg.                        terreActive AG
---------------------------------------------------------------