[rrd-users] How to calculate desired value?

[Alex van den Bogaerdt]

> I am not sure, if I should be more confused than before.
>
> First of all, thanks for the correct formula. If I use the
> CDEF:heattime2=heat_flag,24,*
> VDEF:totalheattime=heattime2,AVERAGE
> I get the desired value - most of the time.

So focus on those other times and see what's different there.

> Then I played around with some scenarios:
> My timeframe is:
> -s 1408917600 -e 1409004000
> Mon Aug 25 00:00:00 CEST 2014
> Tue Aug 26 00:00:00 CEST 2014
>
> I found out, that the very first value (1408917600)
> will NOT be taken into account for the calculation.
> This is probably by design.

It is. The intervals have a timestamp which denotes the end of that 
interval.
Consider one interval of 15 minutes.  It starts at 00:00 and it ends at 
00:15.
If RRDtool would also include the interval stamped "00:00", you would get 
two intervals, for a total of 30 minutes, from 23:45 to 00:15.

> The very last value is taken into account (1409004000),
> this is also by design.

Yup.  I asked because sometimes bugs fly into the memory and then there are 
off-by-one errors.

> My mistake was, that I did not always use a complete 24h timeframe for the
> graph creation.
> The other mistake was, that the calculation does not work, if you have
> missing values within
> the timeframe.

And, if for some unknown reason, you get values which are not exactly 0 or 
1, you will also see problems.

> For that reason, I substituted missing values with "0" using:
> CDEF:heattimeclean=heat_flag,UN,0,heat_flag,IF \
> and then used the cleaned time series for the calculation.
> CDEF:heattime2=heattimeclean,24,* \
> VDEF:totalheattimeclean=heattime2,AVERAGE \

> Question (1):
>
> Unfortunately this introduces (rounding?) errors.
> In my artificial testdata I have 17 x "1" values for heat_flag; which
> corresponds to 4.25h of heattime.
> The totalheattime is calculated as "4.25" - perfect!
> But the totalheattimeclean is calculated as 4.21 - is this a rounding 
> error?

Right now I have no clue what causes this. Your script is a bit big, and my 
time is a little limited. Sorry.

> I do not understand this, because the heat_flag,UN,0,heat_flag,IF should 
> not
> touch the original values, only unknowns.

I agree.

>
> How do I avoid the rounding?

Make sure you are working with exactly 96 intervals: 24 hours times 15 
minutes per hour. Don't assume everything is working as designed, test and 
verify. The problem may be in your script (i don't think so at a first 
glance) or in RRDtool (it does sometimes happen, try another version), or 
there is still an error in your logic which you and I don't spotted yet.


> Question (2)
> My other problem is:
> On my original "production" database, I have always the above described
> "rounding" error - but for both values!
> If I re-create the very same day in my artificial database, I get the
> rounding errors only for the cleaned values.

Which sounds very unlikely. My guess is that "very same" is not true.


> We have 8.25h of heating time
> "Production" database:
> For this day (Aug. 21) with 8.25h heattime I get:
> totalheattimeclean = totalheattime = 8.16h

33 periods "1", 63 others are 0.
Suppose you are looking at 97, not 96, intervals:
33/97 * 24 = 8,164948453608247422680412371134 which "%2.2lf" prints as 8.16

A bug may have creeped in somewhere.


> "Artificial" database:
> For this day (Aug. 21) with 8.25h heattime I get:
> totalheattimeclean = 8.16; totalheattime = 8.25h
>
>
> I thought, well, I am comparing apples with oranges.
> I verified:
> - parameters are the identical (copy & paste any and all parameter into
> testdata script)
> - used same environment (e.g. export LANG=de_DE.UTF8)
> - used the same data
>
> I took the data from my "production" rrd database and wrote it with 
> enclosed
> script into a new rrd database.
> I then did a fetch with
> rrdtool fetch /testtemp/temp_pool_debug.rrd -s 1408572000 -e 1408658400
> AVERAGE
> on both databases and compared the resulting file - no difference except 
> for
> the timestamp
> 1408659300, which is outside of the timeframe. Or is this the key?

You ask for "--end 1408658400" yet RRDtool gives you the interval beyond 
that.

If memory serves me right, this has come up a couple of years ago, for some 
unknown reason this bug was decided not to be fixed, and the workaround was 
to "--end 1408658399".

HTH
Alex