[rrd-developers] How to get the most performance when using lots of RRD files

[Henrik Stoerner]

I am using a network/systems monitoring tool - Hobbit - which uses 
lots of RRD files for tracking all sorts of data. This works really
well - kudos to Tobi.

However, my main system for this currently has about 20.000 RRD files,
all of which are updated every 5 minutes. So that's about 70 updates
per second, and I can see that the amount of disk I/O happening on
this server will become a performance problem soon, as more systems are
added and hence more RRD files need updating.

This is really not a problem with RRDtool itself, but more a question
of how to run a system that uses RRDtool extensively. So I am
considering a couple of  possibilities and would like some comments 
on them.

One option is to avoid some of the overhead associated with looking up, 
opening and closing all of the files. It might be beneficial to implement
a way of storing multiple RRD files in one physical file, to avoid
this overhead. This is aimed at getting more performance from a
single server.

Another possibility is to implement a kind of RPC mechanism on top
of the rrdtool API. I've only worked with the basic rrdtool functions
- rrd_create(), rrd_update(), rrd_graph() - so I don't know if this
would work for all of the API calls, but my idea is that since the
current API is more or less text based, it would be pretty simple
to implement a network service to make this API usable over a network
connection. Obviously, some lookup mechanism would be needed to figure 
out which server has a specific RRD file.  That would let me have multiple 
servers handling the load of updating the RRD files. 

I tend to prefer the latter solution, because I think it will have 
the least impact on the current API - if any at all - and it is
probably also the solution that scales best in the end. But I'm 
obviously eager to hear if there is some obvious solution that I
have overlooked.

I am willing to implement this, and if there is interest then it could
be part of the RRDtool kit.


Regards,
Henrik

--
Unsubscribe mailto:rrd-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-developers-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-developers
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi