[rrd-developers] Unexpected behaviour of PREDICT and PREDICTSIGMA

[Martin Sperl]

> On 27.04.2014, at 10:03, Steve Shipway <s.shipway at auckland.ac.nz> wrote:
> 
> Percentile calculations -- which we already have via the PERCENT VDEF operation -- are progressively more inaccurate the larger the granularity of the underlying RRA.  Explaining this ended up on the Routers2 FAQ it was asked so often.  Although you can use step= in the DEF to pull out a better resolution RRA this is only an option if one actually exists for the graph time window.  Else you get the same problems.  However, even with this caveat, people do love having them... so maybe a version of PERCENT to work on CDEFs with a PREDICT-style window would be useful for some people.  Maybe syntax like s1,s2...sn,n,x,p,PREDICTPCT would work?  You'd only get bad performance hits if people had VERY long high-resolution RRAs...

So,  I have now created a pull-request for predictperc with the same logic as predict, but with an additional parameter to define the percentile value requested.
There is an option for (linear) interpolation when the exact value is not available and the "nearest" value can get used (if a negative percentile value is used).
The implementation reuses most of the same code that predict already uses.

Comparing the percentile prediction (5th,50th and 95th) to normal prediction+-4sigma shows that the percentile prediction is more sensitive than the average/sigma, which can be good or bad depending on what your other parameters and how noisy your data is...

Still it may be of interest to have this tool...

For an example graph showing both predictions see the github pull request:
https://github.com/oetiker/rrdtool-1.x/pull/465

Ciao, Martin

P.s: this also implements the extra field in rpnp_t (mentioned in another email) to avoid hundreds of allocate/frees...
During that time I found that the structures are not automatically zeroed when resizing/reallocating, which made the clearing a bit of a bug-hunt, where I hope I found all necessary places. Maybe we should look into adding the zeroing, as this current behavior may have produced a few bugs already, that required longer debug sessions...

Sent from my iPad