[rrd-developers] Re: SUM Consolidation Function in RRDTool

[Mark Lawrence]

On Mon, 4 Apr 2005, Stanislav Sinyagin wrote:

> Average value over a period is already a sum of values, divided by the
> period.

Unfortunately the consolidation feature of RRDs plus the possibility for
UNKN values means that the conversion from the average back to the sum is
not always the same as the forward case.

Take the following data as input for a single DS RRD (from a hypothetical
time 0):

  0:UNKN 300:10 600:UNKN 900:UNKN

and an RRA that consolidate 900 seconds into one data point. The result of
fetch from that RRA will result in UNKN, which will probably be converted
to zero by the caller so that CDEFs based on this value don't also return
UNKN values. The SUM function would nicely return 10.

Also the following data:

  0:UNKN 300:10 600:10 900:10

consolidated and then mulitplied by the period will produce the same value
as if all entries were 10, and not with one UNKN (given the cf is 0.5). A
SUM function would nicely return 30.


So for example if I am wanting to accurately calculate the number of mails
delivered to my domain I am forced to:

  * ensure the RRD begins on the largest RRA boundary
  * fill the RRD with zeros from the previous RRA boundary to the first
    measurement epoch
  * make sure that whenever no mails were delivered I update with 'zero'
  * If my monitoring box or mailserver go down for some reason, I then
    also have to 'fill' the un-measured time with zeros as well.

The processing required with multiple data sources over hundreds of mail
domains is not trivial, and can lead to delays when doing updates.

This is why a SUM function would be nice.


Sebastian, I once started looking at implementing this - I've attached
some diffs to give you an idea of where to start.

Regards,
Mark.
-- 
Mark Lawrence



diff -u -r /tmp/rrdtool-1.0.48/src/rrd_format.c src/rrd_format.c
--- /tmp/rrdtool-1.0.48/src/rrd_format.c	2004-04-06 23:38:13.000000000 +0200
+++ src/rrd_format.c	2004-10-27 09:25:16.000000000 +0200
@@ -44,6 +44,7 @@
     converter(MIN,CF_MINIMUM)
     converter(MAX,CF_MAXIMUM)
     converter(LAST,CF_LAST)
+    converter(SUM,CF_SUM)
     rrd_set_error("unknown consolidation function '%s'",string);
     return(-1);
 }
diff -u -r /tmp/rrdtool-1.0.48/src/rrd_format.h src/rrd_format.h
--- /tmp/rrdtool-1.0.48/src/rrd_format.h	2004-04-06 23:38:13.000000000 +0200
+++ src/rrd_format.h	2004-10-27 08:55:50.000000000 +0200
@@ -168,7 +168,8 @@
 enum cf_en           { CF_AVERAGE=0,     /* data consolidation functions */
                        CF_MINIMUM,
                        CF_MAXIMUM,
-                       CF_LAST};
+                       CF_LAST,
+                       CF_SUM};

 enum rra_par_en {   RRA_cdp_xff_val=0};   /* what part of the consolidated
 					    datapoint may be unknown, while
diff -u -r /tmp/rrdtool-1.0.48/src/rrd_graph.c src/rrd_graph.c
--- /tmp/rrdtool-1.0.48/src/rrd_graph.c	2004-04-06 23:38:14.000000000 +0200
+++ src/rrd_graph.c	2004-10-27 09:06:59.000000000 +0200
@@ -590,6 +579,9 @@
 			case CF_LAST:
 			    newval = srcptr[i*(*ds_cnt)+col];
 			    break;
+			case CF_SUM:
+			    newval += srcptr[i*(*ds_cnt)+col];
+			    break;
 		    }
 		}
 	    }
@@ -601,6 +593,7 @@
 		    case CF_MINIMUM:
 		    case CF_MAXIMUM:
 		    case CF_LAST:
+		    case CF_SUM:
 			break;
 		}
 	    }
@@ -1624,6 +1617,10 @@
 		    break;
 		case CF_LAST:
 		    printval = im->gdes[vidx].data[ii];
+		    break;
+		case CF_SUM:
+		    printval += im->gdes[vidx].data[ii];
+		    break;
 		}
 	    }
 	    if (im->gdes[i].cf ==  CF_AVERAGE) {
diff -u -r /tmp/rrdtool-1.0.48/src/rrd_update.c src/rrd_update.c
--- /tmp/rrdtool-1.0.48/src/rrd_update.c	2004-04-06 23:38:15.000000000 +0200
+++ src/rrd_update.c	2004-10-27 09:02:52.000000000 +0200
@@ -629,6 +629,13 @@
 						rrd.cdp_prep[iii].scratch[CDP_val].u_val);
 #endif
 					break;
+				    case CF_SUM:
+					rrd.cdp_prep[iii].scratch[CDP_val].u_val+=pdp_temp[ii];
+#ifdef DEBUG
+					fprintf(stderr,"  ** SUM %e\n",
+						rrd.cdp_prep[iii].scratch[CDP_val].u_val);
+#endif
+					break;
 				    default:
 					rrd_set_error("Unknown cf %s",
 						      rrd.rra_def[i].cf_nam);

--
Unsubscribe mailto:rrd-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-developers-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-developers
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi