[rrd-users] rrdtool resize corrupts timestamps in both 1.3.9 and 1.4.7

[Jo Rhett]

It doesn't appear to matter which RRA I update -- I thought of that too. Whichever RRA I update the timestamps get corrupted.

Oddly, the base RRA goes centuries farther forward while the remaining RRAs all go back to 1969 while keeping the correct day. Machine timezone is UTC if that matters. I tried changing the TZ variable a few times with no apparent effect.

On Jul 12, 2012, at 7:38 PM, Joe Loiacono wrote:
> Hmm. I used resize recently without any *apparent* problems. I will do a
> review. I'm pretty sure I would have seen something along the lines of what
> you've shown though.
> 
> I have a set of databases that mimics MRTG. I expanded the 'Last 3 Years'
> rra (i.e., a derived one, not the base that you updated.)
> 
> v1.4.7.
> 
> Joe Loiacono
> 
> 
> 
> 
> From:	Jo Rhett <jrhett at netconsonance.com>
> To:	rrd-users at lists.oetiker.ch
> Date:	07/12/2012 06:54 PM
> Subject:	[rrd-users] rrdtool resize corrupts timestamps in both 1.3.9
>            and	1.4.7
> Sent by:	rrd-users-bounces+jloiacon=csc.com at lists.oetiker.ch
> 
> 
> 
> I found that doing a resize on an rrd file caused a total loss of data with
> rrdtool 1.3.9 which is bundled with zenoss 3.2.1 on CentOS 5.
> Investigating with the tools found that the data was there, but the
> timestamps were corrupt.
> 
> So I grabbed rrdtool 1.4.7 and tried to use it and found that it has the
> same problem on CentOS 5.
> 
> $ rrdtool dump testfile.rrd | head -40
> <?xml version="1.0" encoding="utf-8"?>
> <!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
> <!-- Round Robin Database Dump --><rrd> <version> 0003 </version>
> <step> 15 </step> <!-- Seconds -->
> <lastupdate> 1342133281 </lastupdate> <!-- 2012-07-12 15:48:01 PDT -->
> 
> <ds>
> <name> ds0 </name>
> <type> COUNTER </type>
> <minimal_heartbeat> 45 </minimal_heartbeat>
> <min> NaN </min>
> <max> 1.0000000000e+04 </max>
> 
> <!-- PDP Status -->
> <last_ds> 5585260819 </last_ds>
> <value> 5.2688224977e+02 </value>
> <unknown_sec> 0 </unknown_sec>
> </ds>
> 
> <!-- Round Robin Archives --> <rra>
> <cf> AVERAGE </cf>
> <pdp_per_row> 1 </pdp_per_row> <!-- 15 seconds -->
> 
> <params>
> <xff> 5.0000000000e-01 </xff>
> </params>
> <cdp_prep>
> <ds>
> <primary_value> 5.0500416920e+02 </primary_value>
> <secondary_value> 5.0571394032e+02 </secondary_value>
> <value> NaN </value>
> <unknown_datapoints> 0 </unknown_datapoints>
> </ds>
> </cdp_prep>
> <database>
> <!-- 2012-07-12 13:18:15 PDT / 1342124295 --> <row><v> 5.2835238201e+02
> </v></row>
> <!-- 2012-07-12 13:18:30 PDT / 1342124310 --> <row><v> 5.2950058206e+02
> </v></row>
> <!-- 2012-07-12 13:18:45 PDT / 1342124325 --> <row><v> 5.3706333105e+02
> </v></row>
> <!-- 2012-07-12 13:19:00 PDT / 1342124340 --> <row><v> 5.2656011537e+02
> </v></row>
> <!-- 2012-07-12 13:19:15 PDT / 1342124355 --> <row><v> 5.0348379147e+02
> </v></row>
> 
> $ rrdtool resize testfile.rrd 0 GROW 1416
> 
> $ rrdtool dump resize.rrd | head -40
> <?xml version="1.0" encoding="utf-8"?>
> <!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
> <!-- Round Robin Database Dump --><rrd> <version> 0003 </version>
> <step> 15 </step> <!-- Seconds -->
> <lastupdate> 4048795654275216693 </lastupdate> <!-- -547790155-01-11136
> 05:31:33 PST -->
> 
> <ds>
> <name> ds0 </name>
> <type> COUNTER </type>
> <minimal_heartbeat> 45 </minimal_heartbeat>
> <min> NaN </min>
> <max> 1.0000000000e+04 </max>
> 
> <!-- PDP Status -->
> <last_ds>  </last_ds>
> <value> 0.0000000000e+00 </value>
> <unknown_sec> 0 </unknown_sec>
> </ds>
> 
> <!-- Round Robin Archives --> <rra>
> <cf> AVERAGE </cf>
> <pdp_per_row> 1 </pdp_per_row> <!-- 15 seconds -->
> 
> <params>
> <xff> 5.0000000000e-01 </xff>
> </params>
> <cdp_prep>
> <ds>
> <primary_value> 0.0000000000e+00 </primary_value>
> <secondary_value> 0.0000000000e+00 </secondary_value>
> <value> 0.0000000000e+00 </value>
> <unknown_datapoints> 0 </unknown_datapoints>
> </ds>
> </cdp_prep>
> <database>
> <!-- -547790155-01-11136 21:07:45 PST / 892845857 --> <row><v> NaN
> </v></row>
> <!-- -547790155-01-11136 21:08:00 PST / 892845872 --> <row><v> NaN
> </v></row>
> <!-- -547790155-01-11136 21:08:15 PST / 892845887 --> <row><v> NaN
> </v></row>
> <!-- -547790155-01-11136 21:08:30 PST / 892845902 --> <row><v> NaN
> </v></row>
> <!-- -547790155-01-11136 21:08:45 PST / 892845917 --> <row><v> NaN
> </v></row>
> 
> Let's find the first value that was in the old file.
> 
> $ rrdtool dump tmp.0.resize.rrd | grep 5.28352
> <!-- -547790155-01-11098 03:01:15 PST / 892867067 --> <row><v>
> 5.2835238201e+02 </v></row>
> 
> Hm. That's a way bit different timestamp than this record
> <!-- 2012-07-12 13:18:15 PDT / 1342124295 --> <row><v> 5.2835238201e+02
> </v></row>
> 
> --
> Jo Rhett
> Net Consonance : net philanthropy to improve open source and internet
> projects.
> 
> 
> _______________________________________________
> rrd-users mailing list
> rrd-users at lists.oetiker.ch
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
> 
> 

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20120713/767b0c20/attachment.htm