[rrd-developers] Patch: rrdtool+libdbi-Database backend - patch for rrdtool 1.3
tobi at oetiker.ch
Fri Nov 7 15:09:53 CET 2008
since there seems to be no opposition on the ML and I quite like
your patch, only touching little existing code, I have included it
Monday Martin Sperl wrote:
> As some of you may know that I have created a patch for rrdtool 1.2 a few
> years ago, so that a database could be queried for values for graphing.
> I have created an updated patch for rrdtool version 1.3 (against SVN version
> The patch has been mostly rewritten and the following changes have been made:
> * high dependency on mysql has been reduced by avoiding the
> temporary tables (which was bad for mysql replication)
> * The number of executed SQL-Statements for one CDEF has been
> reduced to 1 compared to 11 SQLs (including CREATE TEMPORARY
> TABLE) - for patch against version 1.2
> * All consolidation is done in rrdtool itself (MIN,MAX,AVERAGE)
> * Additional consolidation functions are COUNT and SIGMA, which give
> information on statistics on a per "time-bin" basis.
> * All these consolidation values are always returned as separate
> columns, that are returned by RRD and the consolidation function
> given as Argument is ignored.
> Main reason is that this way there is only one call to rrd_fetcht
> and thus the database even if we need to fetch for example min,
> avg and max. Compare this to 3 calls in case of different
> consolidation functions - and if you want to get SIGMA and COUNT
> as well it is still only one call to the backend and the database.
> * Some previous existing features have been taken out at the moment
> to allow for this reduced set of SQL queries.
> o prediction using the values from the last X days at the same
> o the corresponding sigma calculation
> * The idea is to create generic CDEF's that will do the same thing,
> but that is also available when using RRD-files (similar to TREND,
> but with another scope)
> This will get posted as a separate patch.
> * Overall performance should be much better and the patch as a whole
> * The patch also includes modifications to the configuration
> infrastructure, to make libdbi support optional.
> As requested by Tobi in a private email-thread, the patch should be highly
> localized (and configurable via configure), to allow a possible merge with
> minimal shared code touched.
> Please review the code and comment...
> P.s: I know that there is the "new" approach of rrdcached to improve
> write-performance with massive updates on rrd. This patch was an initial
> attempt to get a similar problem solved.
> Here for comparison: our production system (running with the 1.2 patch for
> graphing) is updating on average 150k different values every 300 seconds or
> approximately 500 values/s on a HP-DL360G3 with Dual Xenon 3.0GHz, 4GB RAM
> connected via ISCSI (over GBit network) to our storage system. (data-gathering
> Application and the mysql DB is done on the same host)
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900
More information about the rrd-developers