<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi!<br>
<br>
As some of you may know that I have created a patch for rrdtool 1.2 a
few years ago, so that a database could be queried for values for
graphing.<br>
<br>
I have created an updated patch for rrdtool version 1.3 (against SVN
version 1644).<br>
<br>
The patch has been mostly rewritten and the following changes have been
made:<br>
<ul>
<li>high dependency on mysql has been reduced by avoiding the
temporary tables (which was bad for mysql replication)</li>
<li>The number of executed SQL-Statements for one CDEF has been
reduced to 1 compared to 11 SQLs (including CREATE TEMPORARY TABLE) -
for patch against version 1.2<br>
</li>
<li>All consolidation is done in rrdtool itself (MIN,MAX,AVERAGE)<br>
</li>
<li>Additional consolidation functions are COUNT and SIGMA, which
give information on statistics on a per "time-bin" basis.</li>
<li>All these consolidation values are always returned as separate
columns, that are returned by RRD and the consolidation function given
as Argument is ignored.<br>
Main reason is that this way there is only one call to rrd_fetcht and
thus the database even if we need to fetch for example min, avg and
max. Compare this to 3 calls in case of different consolidation
functions -
and if you want to get SIGMA and COUNT as well it is still only one
call to the backend and the database.<br>
</li>
<li>Some previous existing features have been taken out at the moment
to allow for this reduced set of SQL queries.</li>
<ul>
<li>prediction using the values from the last X days at the same
time<br>
</li>
<li>the corresponding sigma calculation</li>
</ul>
<li>The idea is to create generic CDEF's that will do the same thing,
but that is also available when using RRD-files (similar to TREND, but
with another scope)<br>
This will get posted as a separate patch.<br>
</li>
<li>Overall performance should be much better and the patch as a
whole simpler.</li>
<li>The patch also includes modifications to the configuration
infrastructure, to make libdbi support optional.</li>
</ul>
As requested by Tobi in a private email-thread, the patch should be
highly localized (and configurable via configure), to allow a possible
merge with minimal shared code touched.<br>
<br>
Please review the code and comment...<br>
<br>
Thanks,<br>
Martin<br>
<br>
P.s: I know that there is the "new" approach of rrdcached to improve
write-performance with massive updates on rrd. This patch was an
initial
attempt to get a similar problem solved. <br>
Here for comparison: our
production system (running with the 1.2 patch for graphing) is updating
on average 150k different values every 300 seconds or approximately 500
values/s on a HP-DL360G3 with Dual Xenon 3.0GHz, 4GB RAM connected via
ISCSI (over GBit network) to our storage system. (data-gathering
Application and the mysql DB is done on the same host)
</body>
</html>