<html><body><div style="color:#000; background-color:#fff; font-family:arial, helvetica, sans-serif;font-size:10pt"><div><span><br></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span>The short answer is No.</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span><br></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span>RRD will not take out of order data; nor should it. CF RRAs are calculated dependent on RRA[0]; and RRA[0] CFs are based on insert(s) for that interval and immutable once the next interval is written to.</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif;
background-color: transparent; font-style: normal;"><span><br></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span>commit order should always be governed by an SLA; if you have 20 second interval batches and wish to commit that data as soon as possible you can 'back-off' in your queue a specific period of time (the SLA) which would permit some latent data to come in out of order to be sorted before commits. Generally, out-of-order issues don't occur even at high-rate commits (1s) as long as the producer of the data remains the same (ie: it is submitting serially so it can't itself submit out of order.)</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span><br></span></div><div style="color: rgb(0, 0, 0); font-size: 13px;
font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span>Your backlog during a failure? Well... either:</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span><br></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span><span class="Apple-tab-span" style="white-space:pre"> </span>a) design around that failure scenario with a persistent fault-tolerant queue</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span><span class="Apple-tab-span" style="white-space:pre"> </span>b) design the queue to only go through a single server for a series of data (ie: so if it goes down then all data
destined for it is backlogged, thus no out-of-order issue)</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span><span class="Apple-tab-span" style="white-space:pre"> </span>c) accept the loss; after all you had an outage</span></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>d) only store one datasource per rrd file; 2+ ds per file runs the risk of coordination and severely hinders (a) and (b)</div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span style="font-weight: bold;"><span class="Apple-tab-span" style="white-space:pre"> </span>e) all of the above</span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><br></div><div
style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span class="Apple-tab-span" style="white-space:pre"> </span>f) if you really must have a database (datastore) accepting of out of order commits; don't use rrdtool</div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><span class="Apple-tab-span" style="white-space:pre"> </span>g) you can rewrite the rrd, xport the data, calculate what it should be, then rebuild a new rrd ( I don't suggest this, but it's doable )</div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style:
normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;">Good luck,</div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;">-Ryan</div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: arial, helvetica, sans-serif; background-color: transparent; font-style: normal;"><br></div> <div style="font-family: arial, helvetica, sans-serif; font-size: 10pt;"> <div style="font-family: 'times new roman', 'new york', times, serif; font-size: 12pt;"> <div dir="ltr"> <font size="2" face="Arial"> <hr size="1"> <b><span style="font-weight:bold;">From:</span></b> Wesley Wyche <wesley.wyche@verizon.com><br> <b><span style="font-weight: bold;">To:</span></b> rrd-users@lists.oetiker.ch <br> <b><span style="font-weight: bold;">Sent:</span></b> Wednesday,
November 7, 2012 9:39 PM<br> <b><span style="font-weight: bold;">Subject:</span></b> [rrd-users] Disabling Last Update<br> </font> </div> <br>
Is there a way to disable the last update value (or at least override it<br>somehow)? I need to insert/update data in a NON-sequential manner based<br>upon MY time value, not a step from last update.<br><br>The scenario is this:<br><br>I have data coming into a landing area in batches. Those batches are data<br>updates for thousands of rrds and they come in several per minute. However,<br>there may be times where there is a backlog of batches that need to be<br>processed offline perhaps due to a server outage or processing requirements<br>elsewhere. <br><br>Even if I'm processing data that is "current", there are still multiple<br>batch files for that single minute of time. We could be processing the data<br>out of order within that single minute because the sample rate is every 20<br>seconds and I could update the rrd with the wrong order (due simply by<br>processing that minutes batches in alphanumeric order by
filename).<br><br>It would be awesome if there was a way I didn't have to simply chunk hours<br>of data batches away if I have a processing server go down and i can't keep<br>up with the volume. At some point, the server problem would be fixed<br>eventually and then it would start processing the backlog of data and<br>inserting it into the RRDs in the PAST where it should have been inserted<br>and belongs instead of the NOW.<br><br><br><br>--<br>View this message in context: http://rrd-mailinglists.937164.n2.nabble.com/Disabling-Last-Update-tp7580590.html<br>Sent from the RRDtool Users Mailinglist mailing list archive at <a target="_blank" href="http://nabble.com/">Nabble.com</a>.<br><br>_______________________________________________<br>rrd-users mailing list<br><a ymailto="mailto:rrd-users@lists.oetiker.ch" href="mailto:rrd-users@lists.oetiker.ch">rrd-users@lists.oetiker.ch</a><br><a href="https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users"
target="_blank">https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users</a><br><br><br> </div> </div> </div></body></html>