[rrd-users] rrdtool resize corrupts timestamps in both 1.3.9 and 1.4.7

Fri Jul 13 20:45:13 CEST 2012

Just FYI -- not sure how the RPMs are compiled, but recompiling 1.4.7 by hand on the target system with the following options appears to have solved the problem.

./configure --prefix=/home/jrhett/rrdtool --disable-mmap --disable-rrdcgi --disable-rrd_graph

I used this rrdtool only to do the resize, and it worked properly. So there's still bugs around mmap and resize on x86_64.

On Jul 13, 2012, at 9:22 AM, Jo Rhett wrote:
> It doesn't appear to matter which RRA I update -- I thought of that too. Whichever RRA I update the timestamps get corrupted.
> 
> Oddly, the base RRA goes centuries farther forward while the remaining RRAs all go back to 1969 while keeping the correct day. Machine timezone is UTC if that matters. I tried changing the TZ variable a few times with no apparent effect.
> 
> On Jul 12, 2012, at 7:38 PM, Joe Loiacono wrote:
>> Hmm. I used resize recently without any *apparent* problems. I will do a
>> review. I'm pretty sure I would have seen something along the lines of what
>> you've shown though.
>> 
>> I have a set of databases that mimics MRTG. I expanded the 'Last 3 Years'
>> rra (i.e., a derived one, not the base that you updated.)
>> 
>> v1.4.7.
>> 
>> Joe Loiacono
>> 
>> 
>> 
>> 
>> From:	Jo Rhett <jrhett at netconsonance.com>
>> To:	rrd-users at lists.oetiker.ch
>> Date:	07/12/2012 06:54 PM
>> Subject:	[rrd-users] rrdtool resize corrupts timestamps in both 1.3.9
>>            and	1.4.7
>> Sent by:	rrd-users-bounces+jloiacon=csc.com at lists.oetiker.ch
>> 
>> 
>> 
>> I found that doing a resize on an rrd file caused a total loss of data with
>> rrdtool 1.3.9 which is bundled with zenoss 3.2.1 on CentOS 5.
>> Investigating with the tools found that the data was there, but the
>> timestamps were corrupt.
>> 
>> So I grabbed rrdtool 1.4.7 and tried to use it and found that it has the
>> same problem on CentOS 5.
>> 
>> $ rrdtool dump testfile.rrd | head -40
>> <?xml version="1.0" encoding="utf-8"?>
>> <!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
>> <!-- Round Robin Database Dump --><rrd> <version> 0003 </version>
>> <step> 15 </step> <!-- Seconds -->
>> <lastupdate> 1342133281 </lastupdate> <!-- 2012-07-12 15:48:01 PDT -->
>> 
>> <ds>
>> <name> ds0 </name>
>> <type> COUNTER </type>
>> <minimal_heartbeat> 45 </minimal_heartbeat>
>> <min> NaN </min>
>> <max> 1.0000000000e+04 </max>
>> 
>> <!-- PDP Status -->
>> <last_ds> 5585260819 </last_ds>
>> <value> 5.2688224977e+02 </value>
>> <unknown_sec> 0 </unknown_sec>
>> </ds>
>> 
>> <!-- Round Robin Archives --> <rra>
>> <cf> AVERAGE </cf>
>> <pdp_per_row> 1 </pdp_per_row> <!-- 15 seconds -->
>> 
>> <params>
>> <xff> 5.0000000000e-01 </xff>
>> </params>
>> <cdp_prep>
>> <ds>
>> <primary_value> 5.0500416920e+02 </primary_value>
>> <secondary_value> 5.0571394032e+02 </secondary_value>
>> <value> NaN </value>
>> <unknown_datapoints> 0 </unknown_datapoints>
>> </ds>
>> </cdp_prep>
>> <database>
>> <!-- 2012-07-12 13:18:15 PDT / 1342124295 --> <row><v> 5.2835238201e+02
>> </v></row>
>> <!-- 2012-07-12 13:18:30 PDT / 1342124310 --> <row><v> 5.2950058206e+02
>> </v></row>
>> <!-- 2012-07-12 13:18:45 PDT / 1342124325 --> <row><v> 5.3706333105e+02
>> </v></row>
>> <!-- 2012-07-12 13:19:00 PDT / 1342124340 --> <row><v> 5.2656011537e+02
>> </v></row>
>> <!-- 2012-07-12 13:19:15 PDT / 1342124355 --> <row><v> 5.0348379147e+02
>> </v></row>
>> 
>> $ rrdtool resize testfile.rrd 0 GROW 1416
>> 
>> $ rrdtool dump resize.rrd | head -40
>> <?xml version="1.0" encoding="utf-8"?>
>> <!DOCTYPE rrd SYSTEM "http://oss.oetiker.ch/rrdtool/rrdtool.dtd">
>> <!-- Round Robin Database Dump --><rrd> <version> 0003 </version>
>> <step> 15 </step> <!-- Seconds -->
>> <lastupdate> 4048795654275216693 </lastupdate> <!-- -547790155-01-11136
>> 05:31:33 PST -->
>> 
>> <ds>
>> <name> ds0 </name>
>> <type> COUNTER </type>
>> <minimal_heartbeat> 45 </minimal_heartbeat>
>> <min> NaN </min>
>> <max> 1.0000000000e+04 </max>
>> 
>> <!-- PDP Status -->
>> <last_ds>  </last_ds>
>> <value> 0.0000000000e+00 </value>
>> <unknown_sec> 0 </unknown_sec>
>> </ds>
>> 
>> <!-- Round Robin Archives --> <rra>
>> <cf> AVERAGE </cf>
>> <pdp_per_row> 1 </pdp_per_row> <!-- 15 seconds -->
>> 
>> <params>
>> <xff> 5.0000000000e-01 </xff>
>> </params>
>> <cdp_prep>
>> <ds>
>> <primary_value> 0.0000000000e+00 </primary_value>
>> <secondary_value> 0.0000000000e+00 </secondary_value>
>> <value> 0.0000000000e+00 </value>
>> <unknown_datapoints> 0 </unknown_datapoints>
>> </ds>
>> </cdp_prep>
>> <database>
>> <!-- -547790155-01-11136 21:07:45 PST / 892845857 --> <row><v> NaN
>> </v></row>
>> <!-- -547790155-01-11136 21:08:00 PST / 892845872 --> <row><v> NaN
>> </v></row>
>> <!-- -547790155-01-11136 21:08:15 PST / 892845887 --> <row><v> NaN
>> </v></row>
>> <!-- -547790155-01-11136 21:08:30 PST / 892845902 --> <row><v> NaN
>> </v></row>
>> <!-- -547790155-01-11136 21:08:45 PST / 892845917 --> <row><v> NaN
>> </v></row>
>> 
>> Let's find the first value that was in the old file.
>> 
>> $ rrdtool dump tmp.0.resize.rrd | grep 5.28352
>> <!-- -547790155-01-11098 03:01:15 PST / 892867067 --> <row><v>
>> 5.2835238201e+02 </v></row>
>> 
>> Hm. That's a way bit different timestamp than this record
>> <!-- 2012-07-12 13:18:15 PDT / 1342124295 --> <row><v> 5.2835238201e+02
>> </v></row>
>> 
>> --
>> Jo Rhett
>> Net Consonance : net philanthropy to improve open source and internet
>> projects.
>> 
>> 
>> _______________________________________________
>> rrd-users mailing list
>> rrd-users at lists.oetiker.ch
>> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>> 
>> 
> 
> -- 
> Jo Rhett
> Net Consonance : net philanthropy to improve open source and internet projects.
> 
> 
> 
> _______________________________________________
> rrd-users mailing list
> rrd-users at lists.oetiker.ch
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

-- 
Jo Rhett
Net Consonance : net philanthropy to improve open source and internet projects.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20120713/cb0c2523/attachment-0001.htm 