[rrd-users] mrtg + rrdcached

[Mathieu Gagné]

On 2/1/10 6:59 PM, Steven Hartland wrote:
> I tried getting mrtg to work with rrdcached but it uses updatev
> in the rrdtool v1.2+ code path so it doesn't work. Looking through
> the code it seemed like there was no reason why if we forced the
> old 1.1 path it wouldn't work just fine.
>
> Thought it was worth a shot so tried it, and it works great!
>
> In fact it works better than great, it transformed out stats machine
> from a totally unusable box, due to excessive io, back
> to what it was before we started running mrtg on it :)
>
> Given this I thought I'd share this experience, so others can
> benefit.
>
> The patch attached is what we changed, its pretty rough with the
> cache server hard coded but it would be easy to fix that and works ;-)
>
> Hopefully future version of rrd will add updatev support to cached
> so this isn't needed but in the mean time we're just really happy
> to see the massive improvement that this new version brings, so
> thanks to everyone who worked on it!

The idea is great (using MRTG with rrdcached). I was planning on trying 
to install multiple MRTG servers updating a central remote "rrdcached" 
server.

However several issues occurred on my side:

- MRTG checks if the .rrd exists already locally before creating a new 
one. But since the .rrd is located on a remote server, MRTG can't find 
it and doesn't try to create it. An rrdupdate obviously fails since 
rrdcached can't find the .rrd on its side.

- MRTG uses rrdupdatev to retrieve additional information for the 
threshold check although I'm not using it and don't plan too. This leads 
to the following problem: MRTG tries to "fix" the lack of information by 
doing a rrdfetch and therefore forcing rrdcached to flush data to the 
disk. This isn't a great thing because I installed rrdcached to avoid 
sync writes to the disk for each rrdupdate.

- MRTG tries to (see if it can) convert .log to .rrd each time 
writegraphics() is called. Replacing "! -e $dotrrd" by 
"RRDs::info($dotrrd)" wouldn't be great since it would generate a new 
connection to the rrdcached daemon and forcing a read of the RRD's 
headers when in facts it would only be to check if the file exists.


Part of the solution would be to mount via NFS the folder containing my 
RRDs (on the rrdcached server) on my MRTG server so it can find locally 
the RRD files and check theirs existence. (it isn't awesome but if it 
can avoid several rrdinfo calls per seconds...)

The second part would be to skip the "forced" rrdfetch() needed for the 
threshold logic (I'm not using) so I can avoid the forced flush to the 
disk on the rrdcached server and don't kill the performance.


On a side note: Why should MRTG requires GD if I'm not planning on using 
rateup? :(
I guess I should only copy ./bin/mrtg and ./lib to /usr/local/mrtg and 
be done with the install.

-- 
Mathieu