[mrtg-developers] Calculating 95th percentile

[Alex van den Bogaerdt]

On Thu, Oct 11, 2007 at 11:59:48AM -0400, Westlake, Simon wrote:

> As a short introduction, the code you provided works great for drawing a
> line at 95% of the max value. If I can find a way to do what I need this
> simply, it will make my life a lot easier.

It does not, at least not when I wrote that code.

> However, what I need to calculate is 95th percentile in billing terms
> (see http://en.wikipedia.org/wiki/Burstable_billing). The problem with
> the vdef 95 percent method is that if there is any time a user bursts to
> 100 megs, it will calculate the 95th percentile as 95 megs. The
> universally accepted method (from a billing perspective) of calculating
> 95th percentile is to take all your samples (typically 1 every 5
> minutes), sort them from smallest to largest, discard the top 5% and
> then bill at the largest value. By this method, if a user had burst to
> 100 megs 4.9% of the month and had maxed at 12 megs the rest of the
> month, the 95th percentile would be calculated as 12 megs. Quite a big
> difference..

Using somewhat old code, which may have changed since then:

- sort:   NaN < -INF < finite_values < INF

This gives you sorted rates:
                qsort(array,step,sizeof(double),vdef_percent_compar);

This computes the Xth percentile position:
                field = (steps-1)*dst->vf.param/100;
This takes the rate from that position:
                dst->vf.val  = array[field];

I don't know why you think the percentile function is just shorthand
for "*95/100". It really is not.

> Before I go too far in figuring out the problem with my fetch statement,
> is there some reasonable way of doing this simply with RRDTool that I
> should investigate, or for a calculation of this complexity, will I need
> to do the calculation via some other means? Looking at the RRDTool
> documentation, I don't really see an easy way to do this.

Investigate what happens in the scenario you talk about. Update with
all zeros, except one update which should be 100M. Then look what value
is reported back.

Alex
P.S.  What is this doing on mrtg-developers, while it should clearly
be on rrdtool-users ?