[rrd-users] Performance over NFS

[Simon Hobson]

Unfortunately, it's such a common term that I can't find anything 
particularly useful via searches.

I'm collecting data on one box (a gateway router), and graphing on 
another box - with the directory holding the rrd files exported by 
NFS from the gateway. Unfortunately, while it seemed to work when I 
first set it up, it's now giving very poor performance - system 
upgrades have changed the setup from NFSv3 over UDP to V4 over TCP 
(if I remember the details correctly).

The box doing the graphing is running Debian Etch and rrd version 
1.2.15-0.3. /etc/fstab contains :
w.x.y.z:/var/rrd  /gate  nfs  ro,soft  0  0

The gateway doing the data collection is running Debian Lenny and rrd 
version 1.3.1-4. /etc/exports contains :
/var/rrd  a.b.c.d(ro,subtree_check,all_squash,anonuid=1000,anongid=1000)

The main rrd file that's giving problems is 22M byte in size and 
contains 510 data sources, with eight aggregations (Average and Max 
for four time periods/resolutions). Some graphs only use a couple of 
data sources, one frequently used one uses nearly 200 (in and out 
traffic for just under 100 addresses in a stacked graph), one 
(infrequently used for good reason) graphs 508 sources (in and out 
traffic for 254 addresses).


The rrdtool processes end up spending significant amounts of time 
with ps showing them in state "rpc_wa" or "sync_p" (and I've just 
seen on in state "get_re"). It's not too bad if only one person is 
generating graphs (and waiting for them), but once or twice I've seen 
people being impatient and the box gets bogged down (not to mention 
swapping heavily) with a dozen of more rrdtool processes running. 
Even just one patient user can get the load average up into the teens.


Can anyone give me some hints or pointers on this ?

-- 
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.