[rrd-users] How to find first valid dp in rrd - repost - where arethe experts?

[Karl Fischer]

Alex van den Bogaerdt wrote:
> ----- Original Message ----- 
> From: "Karl Fischer" <rrd-users at ficos.de>
> To: <rrd-users at lists.oetiker.ch>
> Sent: Tuesday, November 18, 2008 3:17 PM
> Subject: [rrd-users] How to find first valid dp in rrd - repost - where 
> arethe experts?
> 
> 
>> Hello,
>>
>> since there was no answer to my last post, I'm trying again.
>> Where are the experts?
>>
>> Is there a good way to find the first valid (not UNKN) datapoint in rrd?
>>
>> rrdtools first doesn't quite do what I expected.
>> Instead it gives you "rrdtools last" - rrasize.
> 
> Have you looked at, and tried, VDEF first ?
> 
> In http://oss.oetiker.ch/rrdtool/doc/rrdgraph_rpn.en.html search for 
> Example: VDEF:first=mydata,FIRST

Thanx Alex,

yes, I've tried that, but no matter which way I'm doing it, it eventually
ends up having to read the entire database ...
Quite expensive for a database that size, and you have to do multiple
passes to narrow down to get to the values in the RRA with the lowest
step size ...

And, even worse, if I'm using any graph/xport functions, I'm only looking
at one of the values in a multiple DS database unless I add additional
CDEF/VDEF/RPN to look at all the values ... that means I need a different
function for differently defined rrds :-(


The bash function I'm currently using (to initially point my user to the
entire existing data range in my graphical user interface) is this:
(might be useful for others, too)



# find the first and last entry in a rrd file and leave the
# timestamps in $rrdFirst and $rrdLast
FirstLast () {
  # maximum size to expect for the DB (= 5 years)
  local -i MAXSIZE=$(( 5 * 365 * 24 * 60 * 60 ))

  # maximum expected step size to expect in the DB (= 1 hour)
  local -i MAXSTEP=3600

  local CF
  declare -i rrdFirst
  declare -i rrdLast
  declare -i NEW

  # last is easy
  rrdLast=$(rrdtool last $1)

  # first is a bit more tricky
  CF=( $( rrdtool info $1 | grep -F .cf | tail -1 ))
  eval CF=${CF[2]}
  rrdFirst=$(( $rrdLast - $MAXSIZE ))
  NEW=$rrdLast
  while true; do
    NEW=$( rrdtool fetch $1 $CF -s $(($rrdFirst - $MAXSTEP)) -e $NEW \
             | pcregrep -v "^\d+:( nan)+\s*$" \
             | sed -n "/^[0-9]\+:/{s/:.*//p;q}" )
    # echo $NEW   # enable this fore debugging
    [ $NEW -eq $rrdFirst ] && break
    rrdFirst=$NEW
  done
}





It iterates multiple times until the output value ($NEW) doesn't get any
more precise ...
If $MAXSTEP (the step size of the RRA with the largest step size) is larger
than the size of the RRA with the lowest step size, the iteration function
would have to be changed - other than that it seems to work good so far.


- Karl