<html><body><div style="color:#000; background-color:#fff; font-family:arial, helvetica, sans-serif;font-size:10pt"><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><span><br></span></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><span>My 'but' list was just as a point of reference that my situation is different than yours ... my concern came from you finding something that seemed like corruption, so I wanted to know for myself what it was too and thus check myself if it was reproducible. Your right, not everyone recompiles.</span></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><span><br></span></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><span>To your 'this system is too critical to pause' ... rrdtool is very fast ... and this is my apps writer:</span></div><div style="font-family: arial, helvetica, sans-serif; font-size:
10pt; "><span><br></span></div><div><span><div><font size="2">Tue Jul 17 20:58:16:525 2012 - pid:614 - <span style="font-weight: bold;">1031706</span>/0/0/0 updates/errors/saved/creates completed in 6.514 seconds at a rate of <span style="font-weight: bold;">158376.232</span> a second.</font></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><br></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; ">In that write-cycle, 1 million updates (<span style="font-weight: bold;">no</span> rrdcached) at 158k updates per second; which is a bit slower than prod runs since this output is from a load test rrd server.</div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><br></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; ">a resize takes milliseconds (16 milliseconds for my rrd test I gave you); so an inline task in a writer can do these with no problem;
unless you want to resize many thousands all at once which may(probably) incur IO penalty/latency ... to that, you just reduce the rate of resizes per minute (which my app does too.)</div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><br></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; ">All apps are different; it's just food for thought but likely easiest/safest to resize when it's not being written to, and once the resize is done a rename is atomic.</div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><br></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; ">I'm running OEL6 (RHEL6.1) 2.6.32 on an Intel based 12-core X5650 @ 2.67GHz, 72GB memory, 16 1TB 7400RPM SATA RAID-10; it's a nice server ... stores 1.5 million rrds w/step 60 every minute.</div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt;
"><br></div><div><div><span style="font-size: 13px;">[~]$ /apps/epic/rrdtool/bin/rrdtool -v</span></div><div><span style="font-size: 13px;">RRDtool 1.4.7 Copyright 1997-2012 by Tobias Oetiker <tobi@oetiker.ch></span></div><div><span style="font-size: 13px;"> Compiled Jan 24 2012 21:56:18</span></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><br></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><br></div></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; ">-Ryan</div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><br></div></span></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><br></div> <div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "> <div style="font-family: 'times new roman', 'new york', times, serif; font-size:
12pt; "> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Jo Rhett <jrhett@netconsonance.com><br> <b><span style="font-weight: bold;">To:</span></b> Ryan Kubica <kubicaryan@yahoo.com> <br><b><span style="font-weight: bold;">Cc:</span></b> Steve Shipway <s.shipway@auckland.ac.nz>; "rrd-users@lists.oetiker.ch" <rrd-users@lists.oetiker.ch> <br> <b><span style="font-weight: bold;">Sent:</span></b> Tuesday, July 17, 2012 10:27 AM<br> <b><span style="font-weight: bold;">Subject:</span></b> Re: [rrd-users] rrdtool resize corrupts timestamps in both 1.3.9 and 1.4.7<br> </font> </div> <br>
<div id="yiv1895015146"><div><div><div>On Jul 16, 2012, at 10:28 PM, Ryan Kubica wrote:</div><blockquote type="cite"><div><div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: arial, helvetica, sans-serif; font-size: 10pt; "><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; ">I've never run
into this issue before (and store a <span style="font-weight:bold;">lot</span> of data) and on 64bit hosts, but:</div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><br></div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><span class="yiv1895015146Apple-tab-span" style="white-space:pre;"> </span>a) I always compile rrdtool not use stock OS rpm</div></div></div></blockquote><div><br></div>This isn't a common situation -- most people use RPMs, and that means that this is broken for most people. Also we are using a package that bundles a binary image of rrdtool, another situation where many people won't or can't change it out.</div><div><br><blockquote type="cite"><div><div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: arial, helvetica, sans-serif; font-size: 10pt; "><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><span
class="yiv1895015146Apple-tab-span" style="white-space:pre;"> </span>b) It is compiled with mmap()</div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><span class="yiv1895015146Apple-tab-span" style="white-space:pre;"> </span>c) I only resize an rrdtool file when it's not being written to (my apps write daemon does the resize in place as a task.)</div></div></div></blockquote><div><br></div>That's not a choice I get. This system is too critical to stop updates long enough for that to happen. (although I have been digging around in the code to see if there is any locking mechanism that rrdtool will honor)</div><div><br><blockquote type="cite"><div><div style="color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: arial, helvetica, sans-serif; font-size: 10pt; "><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; ">and it works for resizes just fine (just tested manually), I grew RRA 0 by
2x.</div><div style="font-family: arial, helvetica, sans-serif; font-size: 10pt; "><br></div><div><div><span style="font-size:13px;">[tmp]$ grep 1340865780 *xml</span></div><div><span style="font-size:13px;">Epic_io_90.xml:<span class="yiv1895015146Apple-tab-span" style="white-space:pre;"> </span><!-- 2012-06-28 06:43:00 UTC / 1340865780 --> <row><v>9.7033939036e+04</v></row></span></div><div><span style="font-size:13px;">resize.xml:<span class="yiv1895015146Apple-tab-span" style="white-space:pre;"> </span><!-- 2012-06-28 06:43:00 UTC / 1340865780 --> <row><v>9.7033939036e+04</v></row></span></div><div><span style="font-size:13px;"><br></span></div><div><span style="font-size:13px;">[tmp]$ grep 1340865840 *xml</span></div><div><span style="font-size:13px;">Epic_io_90.xml:<span class="yiv1895015146Apple-tab-span" style="white-space:pre;"> </span><!-- 2012-06-28 06:44:00 UTC / 1340865840
--> <row><v>9.7633476991e+04</v></row></span></div><div><span style="font-size:13px;">resize.xml:<span class="yiv1895015146Apple-tab-span" style="white-space:pre;"> </span><!-- 2012-06-28 06:44:00 UTC / 1340865840 --> <row><v>9.7633476991e+04</v></row></span></div></div></div></div></blockquote><div><br></div>Curious, what OS / kernel are you using?</div><div><br></div><div>
<span class="yiv1895015146Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; font-size: medium; "><span class="yiv1895015146Apple-style-span" style="font-size:12px;"><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;"><font size="3" style="font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-family: Helvetica; ">-- </font></div><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;"><font size="3" style="font-style: normal; font-variant: normal; font-weight: normal; font-size: 12px; line-height: normal; font-family: Helvetica; ">Jo Rhett</font></div></span><span
class="yiv1895015146Apple-style-span" style="font-size:12px;">Net Consonance : </span><span class="yiv1895015146Apple-style-span" style="font-size:12px;">net philanthropy to improve open source and internet projects.</span><br><span class="yiv1895015146Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; "><div style="word-wrap:break-word;"><div><div><span class="yiv1895015146Apple-style-span" style="font-size:12px;"><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;"><br></div></span></div></div></div></span></span><br class="yiv1895015146Apple-interchange-newline">
</div>
<br></div></div><br><br> </div> </div> </div></body></html>