[rrd-users] Calculating AVG, treating UNKNOWNs as zero

[Alex van den Bogaerdt]

----- Original Message ----- 
From: "Derek Haynes" <derek.haynes at gmail.com>
To: <rrd-users at lists.oetiker.ch>
Sent: Monday, July 30, 2012 8:54 PM
Subject: [rrd-users] Calculating AVG, treating UNKNOWNs as zero


> The AVG consolidation function excludes UNKNOWN values. Instead of
> excluding them, I'd like to convert each UNKNOWN value to zero and
> then generate the average.

In your example, you deliberately update with "Unknown", and then you want 
to get rid of it.
Why?

> Example use case: recording visits to a URL, updating once-per minute
> (but only if any visits), and then calculating the throughput over a
> given period.

So what you know is: the amount of visits in a minute, which may be zero.
Then store that, and I think your problem is gone.

> Example ----
>
> 1-minute Average RRA. 1 second step:
>
> rrdtool create test.rrd \
> --start 1325376000 \
> --step 1 \
> DS:x:GAUGE:60:U:U \
> RRA:AVERAGE:0.99:60:10080
>
> Updates - 1 at Jan 1 00:01, another at Jan 1 00:30:
>
> rrdtool update test.rrd \
> 1325376060:100 \
> 1325377799:U \
> 1325377800:25
>
> ...this means the values for the CDPs from 00:02 - 00:29 will be
> UNKNOWNs. The 2nd update above forces the 3rd to be recorded.

The problem you have: that 25 is not going in the database because it is 
ignored because it is too late.
You tried to solve this problem by updating with unknown, a second before 
the actual update you're going to perform.
Additionally, you want to have those 25 be valid from 
1325377740..1325377800, just like the first update should be valid from 
1325376000..1325376060 (correct?)

In this example you know that between 1325376060 and 1325377740 there were 
no visits, so I would solve this by increasing the heartbeat to a large 
number (could be as large as a week even) and updating at time 1325377740, 
not 1325377799, with zero, not with unknown.

This would result in uniform buckets of 60 seconds, with the rate you set or 
with a rate of zero.

Details may vary, depending on the information you left out, but IMHO this 
is the path to your solution