[rrd-users] VDEF, 95th percentile and graph width

[Marius Karthaus]

Hello list,

Thank you Martin for your answer. The step=1 is indeed something I tried 
last, but it did not have any effect. I've put it back to 300.

The issue remains still that on every place in the internet I can find 
about this, including the docs of RRDtool this is the 'way to do it' and 
it is very inaccurate. Even our datacenter provider appears to be using 
this calculation as grounds for billing and as such their traffic 
calculation for our type of use is twice as high (38Mbit  vs. 17Mbit) as 
the actual 5 minute measurements on our router.

There is no reason why the VDEF calculation for percentile should be 
influenced by the pixel width of the graph. I invite people that are 
actually using RRD for 95th percentile printouts to vary their 
graph-width and see if their measurements are influenced by their design 
choice instead of their actual usage.

Regards,
Marius Karthaus


On 08-09-14 11:39, Martin Sperl wrote:
> Please try using :step=300 on the DEF to avoid the selection of a
> different resolution with consolidated data.
>
> Your page shows that you are using :step=1 on the DEF, which seems to be
> using the "best" one it can find (which may be consolidated already) if
> there is no exact match...
>
> The work-arround approach is to make the same graph "twice":
> * first run it with a very high width but only use:
>    VDEF:pct=allbitspct,95,PERCENT
>    PRINT:pct:percentile95=%lf
>    --width $((END-START+1)/300)
> * take that value and re-render the graph with your preferred width
>    hardcoding this value into the output.
>
> Ciao,
> 	Martin
>
> P.s: unfortunately the COMPUTE datasource definition does not allow to
> calc averages/MIN/MAX  from the last X samples, otherwise you could do
> the running percentile computation automatically on every insert:
>
> DS:percentile95:VCOMPUTE:value,95,PERCENT:start=-1day:end=now:step=300
>
> But that is not feasible right now.
>
> You can only do it yourself on every insert in some manner like this:
>
> #
> # CURRENT=value to insert and calculate the percentile for
> #
> # get the Value of the last percentile (including CURRENT)
> # assumes step=300 so maxrows=86400/300=288
> PERCENTILE=`rrdtool xport --start=-1day --end=now --maxrows=288 \
>    DEF:test=/tmp/vmstat.rrd:cpu_usr:AVERAGE XPORT:test \
> | awk -F\> 'BEGIN{V[LAST]='$CURRENT';}
>    /NaN/{next;}
>    /<row>/{V[NR]=0+$3;}
>    END{C=asort(V);PINDEX=int(C*0.95);print V[PINDEX];}' `
> # now update the file with the current value and the percentile
> rrdtool update /tmp/vmstat.rrd N:$CURRENT,$PERCENTILE
>
>
>
> On 06.09.2014 11:28, Marius Karthaus wrote:
>> Hello Steve,
>>
>> The version I am using was 1.4.7 on ubuntu precise. I've now compiled
>> the latest stable (1.4.8) but there are no differences between them.
>> Also I've played around with the order of the commands (putting all the
>> graph rendering in front/behind etc), but that also does not make a
>> difference.
>>
>> Regards,
>> Marius Karthaus
>>
>>
>>
>>
>> On 06-09-14 01:54, Steve Shipway wrote:
>>> Looking at the two graphs, they cover the same data width but the
>>> second is 2x the pixel width, and so has less on-the-fly consolidation
>>> before graphing.  From this, as I would expect, the peaks in the first
>>> graph are averaged out more, and so show up as lower (around 50M).The
>>> second graph shows the peaks narrower but higher (around 70M) due to
>>> their having less averaging applied.  So far, so good and as expected.
>>>
>>> However, the percentile calculation (as I understand RRDTool's data
>>> handling) should not be affected by the graph-time consolidation, and
>>> should apply to the extracted data set, which we have forced to the
>>> same resolution in both graphs.  However, the lower value printed
>>> would seem to indicate that this is not the case, and the graph
>>> consolidation is taking place /before/ VDEF calculation.  This only
>>> shows up because the data are extremely spiky in pattern and have a
>>> fair amount of consolidation being applied.  If this is the case it
>>> would seem to be contrary to the behaviour implied by the
>>> documentation (that sounds like a long way of saying "i think its a
>>> bug' :)
>>>
>>> I don't believe that you have told us which version of RRDtool you are
>>> using?  If you're not on a recent version, it might be worth trying an
>>> upgrade., just in case this is related to a recently-fixed bug.
>>>
>>> Steve
>>>
>>> *Steve Shipway*
>>> University of Auckland ITS
>>> /UNIX Systems Design Lead/
>>> s.shipway at auckland.ac.nz <mailto:s.shipway at auckland.ac.nz>
>>> Ph: +64 9 373 7599 ext 86487
>>> //
>>
>>
>>
>> _______________________________________________
>> rrd-users mailing list
>> rrd-users at lists.oetiker.ch
>> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>>
> _______________________________________________
> rrd-users mailing list
> rrd-users at lists.oetiker.ch
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users