[rrd-users] rrdtool resize corrupts timestamps in both 1.3.9 and 1.4.7

[Joe Loiacono]

Hmm. I used resize recently without any *apparent* problems. I will do a
review. I'm pretty sure I would have seen something along the lines of what
you've shown though.

I have a set of databases that mimics MRTG. I expanded the 'Last 3 Years'
rra (i.e., a derived one, not the base that you updated.)

v1.4.7.

Joe Loiacono




From:	Jo Rhett <jrhett at netconsonance.com>
To:	rrd-users at lists.oetiker.ch
Date:	07/12/2012 06:54 PM
Subject:	[rrd-users] rrdtool resize corrupts timestamps in both 1.3.9
            and	1.4.7
Sent by:	rrd-users-bounces+jloiacon=csc.com at lists.oetiker.ch



I found that doing a resize on an rrd file caused a total loss of data with
rrdtool 1.3.9 which is bundled with zenoss 3.2.1 on CentOS 5.
Investigating with the tools found that the data was there, but the
timestamps were corrupt.

So I grabbed rrdtool 1.4.7 and tried to use it and found that it has the
same problem on CentOS 5.

$ rrdtool dump testfile.rrd | head -40
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
<!-- Round Robin Database Dump --><rrd> <version> 0003 </version>
<step> 15 </step> <!-- Seconds -->
<lastupdate> 1342133281 </lastupdate> <!-- 2012-07-12 15:48:01 PDT -->

<ds>
<name> ds0 </name>
<type> COUNTER </type>
<minimal_heartbeat> 45 </minimal_heartbeat>
<min> NaN </min>
<max> 1.0000000000e+04 </max>

<!-- PDP Status -->
<last_ds> 5585260819 </last_ds>
<value> 5.2688224977e+02 </value>
<unknown_sec> 0 </unknown_sec>
</ds>

<!-- Round Robin Archives --> <rra>
<cf> AVERAGE </cf>
<pdp_per_row> 1 </pdp_per_row> <!-- 15 seconds -->

<params>
<xff> 5.0000000000e-01 </xff>
</params>
<cdp_prep>
<ds>
<primary_value> 5.0500416920e+02 </primary_value>
<secondary_value> 5.0571394032e+02 </secondary_value>
<value> NaN </value>
<unknown_datapoints> 0 </unknown_datapoints>
</ds>
</cdp_prep>
<database>
<!-- 2012-07-12 13:18:15 PDT / 1342124295 --> <row><v> 5.2835238201e+02
</v></row>
<!-- 2012-07-12 13:18:30 PDT / 1342124310 --> <row><v> 5.2950058206e+02
</v></row>
<!-- 2012-07-12 13:18:45 PDT / 1342124325 --> <row><v> 5.3706333105e+02
</v></row>
<!-- 2012-07-12 13:19:00 PDT / 1342124340 --> <row><v> 5.2656011537e+02
</v></row>
<!-- 2012-07-12 13:19:15 PDT / 1342124355 --> <row><v> 5.0348379147e+02
</v></row>

$ rrdtool resize testfile.rrd 0 GROW 1416

$ rrdtool dump resize.rrd | head -40
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
<!-- Round Robin Database Dump --><rrd> <version> 0003 </version>
<step> 15 </step> <!-- Seconds -->
<lastupdate> 4048795654275216693 </lastupdate> <!-- -547790155-01-11136
05:31:33 PST -->

<ds>
<name> ds0 </name>
<type> COUNTER </type>
<minimal_heartbeat> 45 </minimal_heartbeat>
<min> NaN </min>
<max> 1.0000000000e+04 </max>

<!-- PDP Status -->
<last_ds>  </last_ds>
<value> 0.0000000000e+00 </value>
<unknown_sec> 0 </unknown_sec>
</ds>

<!-- Round Robin Archives --> <rra>
<cf> AVERAGE </cf>
<pdp_per_row> 1 </pdp_per_row> <!-- 15 seconds -->

<params>
<xff> 5.0000000000e-01 </xff>
</params>
<cdp_prep>
<ds>
<primary_value> 0.0000000000e+00 </primary_value>
<secondary_value> 0.0000000000e+00 </secondary_value>
<value> 0.0000000000e+00 </value>
<unknown_datapoints> 0 </unknown_datapoints>
</ds>
</cdp_prep>
<database>
<!-- -547790155-01-11136 21:07:45 PST / 892845857 --> <row><v> NaN
</v></row>
<!-- -547790155-01-11136 21:08:00 PST / 892845872 --> <row><v> NaN
</v></row>
<!-- -547790155-01-11136 21:08:15 PST / 892845887 --> <row><v> NaN
</v></row>
<!-- -547790155-01-11136 21:08:30 PST / 892845902 --> <row><v> NaN
</v></row>
<!-- -547790155-01-11136 21:08:45 PST / 892845917 --> <row><v> NaN
</v></row>

Let's find the first value that was in the old file.

$ rrdtool dump tmp.0.resize.rrd | grep 5.28352
<!-- -547790155-01-11098 03:01:15 PST / 892867067 --> <row><v>
5.2835238201e+02 </v></row>

Hm. That's a way bit different timestamp than this record
<!-- 2012-07-12 13:18:15 PDT / 1342124295 --> <row><v> 5.2835238201e+02
</v></row>

--
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet
projects.


_______________________________________________
rrd-users mailing list
rrd-users at lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users