[rrd-users] Bug in rrdtool xport?

Sat Sep 27 13:49:07 CEST 2008

Hi Peter,

indeed rrdxport was/is totally broken for exports where the
different datasources do not use the same step size ...

http://oss.oetiker.ch/rrdtool-trac/changeset/1529

fixes this for 1.2.x, 1.3.x and trunk.

thanks for the test example, and also for reminding me that your
report did go unannswered.

Interesting that such broken code could survive for so long, I
guess not to many people use xport ...

cheers
tobi

Today Tobias Oetiker wrote:

> Peter,
>
> indeed, there seems something odd going, I guess I have to finally
> take a look at that xport code :-(
>
> cheers
> tobi
> Yesterday "Peter Valdemar Mørch (Lists)" wrote:
>
> > Hi all,
> >
> > On Sept 9 I posted what I think is a bug here, and have gotten no
> > responses so far. Was there something wrong with my report?
> >
> > http://thread.gmane.org/gmane.comp.db.rrdtool.user/14609
> >
> > I provided a simple test case where "rrdtool xport" cannot correctly
> > export data from two different rrd files that have different step
> > values. "rrdtool graph" works fine with the same rrd files and shows
> > correct data. More and newer observations on the same issue below.
> >
> > Hoping for a response,
> >
> > Peter
> >
> > ----
> >
> > In the meantime, I've discovered that I don't even have to have a CDEF
> > composed of DS'es from two different rrd files for the output to be
> > wrong. Just using the DEF terms for ns3 and ns4 in the rrd xport line
> > below also produces incorrect output:
> > rrdtool xport --start 1220625000 --end 1220629800 \
> > DEF:ns3=snmp2-0.rrd:ns3:AVERAGE XPORT:ns3:ns3 \
> > DEF:ns4=snmp3-0.rrd:ns4:AVERAGE XPORT:ns4:ns4
> >
> > ns4's value in the output varies every 5 minute row, but snmp3-0.rrd
> > containing ns4 has a step of 900 ( == 15 min ), so that is clearly
> > wrong, isn't it? Here are the first rows, with <row></row> taken out so
> > it doesn't wrap in the mail:
> >
> > <t>1220625300</t><v>6.1567155556e+02</v><v>2.6935130000e+03</v>
> > <t>1220625600</t><v>6.1546555556e+02</v><v>2.6935130000e+03</v>
> > <t>1220625900</t><v>6.7551154122e+02</v><v>NaN</v>
> > <t>1220626200</t><v>3.9705685194e+02</v><v>1.2931680988e+03</v>
> > <t>1220626500</t><v>3.2386227350e+02</v><v>3.6926825679e+02</v>
> > <t>1220626800</t><v>2.0573415556e+02</v><v>NaN</v>
> > <t>1220627100</t><v>1.0843768889e+02</v><v>0.0000000000e+00</v>
> >
> > Take out the DEF and XPORT terms for ns3, and ns4 is shown with exactly
> > the same values, but now 15 minutes apart. Seems rrdtool xport is taking
> > one value pr. <row> regardless of step, which is wrong:
> >
> > <t>1220625900</t><v>2.6935130000e+03</v>
> > <t>1220626800</t><v>2.6935130000e+03</v>
> > <t>1220627700</t><v>NaN</v>
> > <t>1220628600</t><v>1.2931680988e+03</v>
> > <t>1220629500</t><v>3.6926825679e+02</v>
> > <t>1220630400</t><v>NaN</v>
> >
> > Also, if I leave in the DEF for ns3, but remove the XPORT, I get the
> > incorrect output. So:
> >
> > Correct output:
> > rrdtool xport --start 1220625000 --end 1220629800 \
> > DEF:ns4=snmp3-0.rrd:ns4:AVERAGE \
> > XPORT:ns4:ns4
> >
> > Incorrect output:
> > rrdtool xport --start 1220625000 --end 1220629800 \
> > DEF:ns3=snmp2-0.rrd:ns3:AVERAGE \
> > DEF:ns4=snmp3-0.rrd:ns4:AVERAGE \
> > XPORT:ns4:ns4
> >
> > Also strange, is that these two give different output:
> >
> > rrdtool xport --start 1220625000 --end 1220629800 \
> > DEF:ns3=snmp2-0.rrd:ns3:AVERAGE \
> > DEF:ns4=snmp3-0.rrd:ns4:AVERAGE \
> > XPORT:ns3:ns3 \
> > XPORT:ns4:ns4
> >
> > rrdtool xport --start 1220625000 --end 1220629800 \
> > DEF:ns4=snmp3-0.rrd:ns4:AVERAGE \
> > DEF:ns3=snmp2-0.rrd:ns3:AVERAGE \
> > XPORT:ns3:ns3 \
> > XPORT:ns4:ns4
> >
> > (I swapped the sequence of the DEFs)
> >
> > Looks like the step is calculated from the first DEF, whatever that
> > happens to be, and each row uses one value, regardless of step. "rrdtool
> > graph" seems smarter than that.
> >
> > Hope someone else can see that this is wrong too!
> >
> > Peter
> >
> > Peter Valdemar Mørch (Lists) 4ux6as402-at-sneakemail.com |Lists| wrote:
> > > I'm have two rrd files with different steps. I try to "see" three
> > > values: a DEF from each of these files, and a CDEF which is an average
> > > of each of them. It seems rrdtool graph does it "right", but rrdtool
> > > xport does it "wrong". Do you agree?
> > >
> > > In my reproducing script
> > > http://www.morch.com/misc/rrdxport_bug/REPRODUCE.sh.txt
> > > there are similar lines for graph and xport. Here graph:
> > >
> > > rrdtool graph --start 1220625000 --end 1220629800 ok.png \
> > > DEF:ns3=snmp2-0.rrd:ns3:AVERAGE 'LINE1:ns3#0000ff:ns3' \
> > > DEF:ns4=snmp3-0.rrd:ns4:AVERAGE 'LINE1:ns4#ff0000:ns4' \
> > > CDEF:avg=ns3,ns4,+,2,/          'LINE2:avg#00ff00:avg'
> > >
> > > ( Both the required RRD files + samle output are in:
> > > http://www.morch.com/misc/rrdxport_bug/ )
> > >
> > > rrdtool graph shows these values "correctly":
> > > http://www.morch.com/misc/rrdxport_bug/ok.png
> > >
> > > But as far as I can tell, rrdtool xport does not:
> > > http://www.morch.com/misc/rrdxport_bug/wrong.xml
> > >
> > > E.g. I would expect that exactly if either of the components (either of
> > > the two DEFs) are NaN, so would the average, but that seems not to be
> > > the case. Here are two example rows from wrong.xml:
> > >
> > > <row><t>1220627400</t><v>1.3481197778e+02</v><v>3.6073928446e-313</v><v>NaN</v></row>
> > > <row><t>1220626800</t><v>2.0573415556e+02</v><v>NaN</v><v>1.4496235778e+03</v></row>
> > >
> > > Also, the value of ns4 in the xport output is wrong when not NaN. I
> > > would have expected, that
> > > http://www.morch.com/misc/rrdxport_bug/wrong.xml
> > > only contain values of ns4 that are also present in
> > > http://www.morch.com/misc/rrdxport_bug/good.xml which agrees with the
> > > values in ok.png above.
> > >
> > > It sure looks to me like ok.png and wrong.xml disagree on what to show.
> > > And that ok.png/good.xml are "right". Do you agree?
> > >
> > > If this is a bug, should I file it in trac? I seem not to be able to
> > > create new bugs or register, only query existing bugs... Please let me
> > > know how if you want me to file a bug report.
> > >
> > > Peter
> >
> >
> >
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900