[rrd-users] zeroing out nans in a big CDEF rpn, or something

[Ryan Goldberg]

Hi all-

I've got about 3500 rrds, each one keeping track of 1 years worth of 5 
minute netflow data.  Each rrd tracks one IP and is updated every 5 
minutes using the output of "flow-stat" from the flow-tools collection.

I want to be able to generate reports from this data for arbitrary lists 
of IPs.  For example, I want to say: give me a report for IP1,IP2,IP3 
including the graph of the 5 minute PDPs and the total in/total out and 
the 95th percentile.

Things are working great, except when the data contains nans.  I probably 
need to give an exmaple of one of my rrdgraph commands:

This example would give me the "sum" graph and 95th percentile for x.y.z.0/30

rrdtool graph -t "Report for (jimmy)" -v "bit/s" -E -w 800 -h 300 
reports/report-jimmy-s200802191422-e200802261422.png --start -7d \
DEF:BytesOut0=/netflow/awesome-data/x.y.z.0/data.rrd:BytesOut:AVERAGE \
DEF:BytesIn0=/netflow/awesome-data/x.y.z.0/data.rrd:BytesIn:AVERAGE \
CDEF:BitsOut0=BytesOut0,8,* \
CDEF:BitsIn0=BytesIn0,8,* \
DEF:BytesOut1=/netflow/awesome-data/x.y.z.1/data.rrd:BytesOut:AVERAGE \
DEF:BytesIn1=/netflow/awesome-data/x.y.z.1/data.rrd:BytesIn:AVERAGE \
CDEF:BitsOut1=BytesOut1,8,* \
CDEF:BitsIn1=BytesIn1,8,* \
DEF:BytesOut2=/netflow/awesome-data/x.y.z.2/data.rrd:BytesOut:AVERAGE \
DEF:BytesIn2=/netflow/awesome-data/x.y.z.2/data.rrd:BytesIn:AVERAGE \
CDEF:BitsOut2=BytesOut2,8,* \
CDEF:BitsIn2=BytesIn2,8,* \
DEF:BytesOut3=/netflow/awesome-data/x.y.z.3/data.rrd:BytesOut:AVERAGE \
DEF:BytesIn3=/netflow/awesome-data/x.y.z.3/data.rrd:BytesIn:AVERAGE \
CDEF:BitsOut3=BytesOut3,8,* \
CDEF:BitsIn3=BytesIn3,8,* \
CDEF:SumBitsOut=BitsOut0,BitsOut1,+,BitsOut2,+,BitsOut3,+ \
CDEF:SumBitsIn=BitsIn0,BitsIn1,+,BitsIn2,+,BitsIn3,+ \
VDEF:95pBitsOut=SumBitsOut,95,PERCENT \
VDEF:95pBitsIn=SumBitsIn,95,PERCENT \
LINE1:SumBitsOut#ff0000:"out" \
LINE1:95pBitsOut#ffff00:"95th out:" \
GPRINT:95pBitsOut:'%.2lf%Sb\n ' \
LINE1:SumBitsIn#0000ff:"in" \
LINE1:95pBitsIn#00ff00:"95th in:" \
GPRINT:95pBitsIn:'%.2lf%Sb' \
-c CANVAS#000000

What would happen in this case is I'd get a bunch blank spots in my graph 
because most of the data for x.y.z.0 is nan, and I assume the addition 
function for the CDEFs doesn't allow adding nans to numbers.

The problem is due to how I collect the data: the rrds only get updated 
when the netflow data actually has some data for an IP, thus x.y.z.0 
rarely gets updated, and will have long periods of time filled with nans.

Now, I can change the way I collect data, and ensure that in each 5 minute 
time period every rrd gets updated, but I was hoping I could leave my 
collection methods alone and convince rrd graph to treats nans as 0s, or 
something along those lines that would achieve the same effect.


Thanks much for your time-
Ryan