[rrd-users] RRD PERCENT question (95 percentile)
Alex van den Bogaerdt
alex at vandenbogaerdt.nl
Wed May 12 09:08:33 CEST 2010
> Hello List,
>
> I'm a happy RRD user, but there's something I need help with.
> We're currently moving to RRDtool for accounting & billing.
> For this I've created RRD files that keep 105120 5minute samples (1 year).
>
> Now I'm comparing the 95% numbers generated by RRDtool with the 95%
> number generated by our old script. The problem is that these numbers
> are significantly different.
Note that "the" 95th percentile does not exist. There are many different
methods of computing this value, and although they are similar and will more
or less provide the same result, they do differ.
I don't recall exactly how I started, but I think originally I used
data[n*steps/100]. Then, after some discussion on the mailing list, round()
was introduced.
> I hope someone can help me understand why these numbers are so different.
Because if the array index changes by only one, the returned value may be
quite different.
> This is how I determine the 95 percentile number using RRDtool
>
[snipped some]
> VDEF:95thin=inbits,95,PERCENT \
> VDEF:95thout=outbits,95,PERCENT \
Looking fine.
> My manual test was done like this:
> 1) fetch rawdata:
> /usr/local/rrdtool-1.2.19/bin/rrdtool fetch \
> --start '1271894400' --end '1272240000' \
> "deviceid11_XXX_Transit.rrd" AVERAGE > OUT_RAW;
>
> 2) read this data with a perl script than sort values and show 95% number.
>
> In this case the data set contains 1153 samples (no NaN in sample).
> so after sorting the 95% percentile should be the value (times 8 for
> bits) on position 1096.
> The problem is that this number is quite different (lower) from what is
> returned using PERCENT above.
Please see if the number on position 1095 or 1097 equals that of what
rrdtool finds.
> Note that this sample does not contain any NaN values.
> I also tried this with the latest version of RRDtool, same result.
>
> Can anyone explain why this is different? Is this expected?
> How exactly does RRD this internally?
Create an array, fill it with the data, use qsort and then find the correct
spot:
qsort(array, step, sizeof(double), vdef_percent_compar);
field = round((dst->vf.param * (double)(steps - 1)) / 100.0);
dst->vf.val = array[field];
In here, vdef_percent_compar is a function that sorts NAN < -INF < numbers <
+INF
Your calculation: 1153 samples, 95% = 1095,35 so you take 1096.
RRDtool: round(95*1152/100)=1094, based on an array with first member is 0,
so 1094 is the 1095th position.
If I recall correctly, the original version did use truncation instead of
rounding, which makes no difference in this case.
Anyway, unless I made a mistake here, rrdtool takes data[1094] and you take
data[1096]. Your returned value should be higher than what RRDtool reports.
> I would like to use RRDtool for this, but need to be sure that the
> numbers are correct, i.e understand why the numbers are different than
> when calculated manually.
I would also worry why it is opposite to what I reasoned above.
More information about the rrd-users
mailing list