<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:'times new roman', 'new york', times, serif;font-size:12pt"><div><br></div><div>Oh, it doesn't 'break' ... it lags back and queues all incoming data until the rrds exist.</div><div><br></div><div>The entire data model is a store-forward-queue, so all the data is in a queue already. Just that when the datapoint files don't exist yet, they need to be created, so the queue will back up while that is occurring.</div><div><br></div><div>This is all about how quickly the data can be written out from the queue and how latent the data in graphs and exports are due to issues resulting from new datapoints being created.</div><div><br></div><div>Currently, one host has 913 thousand active datapoints being updated every minute with a write rate of ~68k rrd updates per second. RRD file create rate is 70/s. I can make creates quicker by parallel
creating but that will further stomp on the disks.</div><div><br></div><div>I'm looking for a better solution than consuming 'peak IO' during high create periods. Since all of the data written is reported on every minute through aggregation and then alerted on, it's important to try and not have latency built into the system during surges.</div><div><br></div><div>I'll pick through the rrd code to see if I can manage a rebase option.</div><div><br></div><div>Thanks,</div><div><br></div><div>-Ryan</div><div><br></div><div><br></div><div><br></div><div style="font-family:times new roman, new york, times, serif;font-size:12pt"><div style="font-family:arial, helvetica, sans-serif;font-size:13px">
Hi Ryan,<br><br>Today Ryan Kubica wrote:<br><br>><br>><br>> RRD Developers,<br>><br>> I have the need to pre-create rrd datapoints before they are used; such that<br>> there are already existing datafiles on the filesystem which are in a free pool<br>> and used when required.<br>><br>> The problem is that without a free pool the system is creating the datapoints on<br>> demand when newly monitored data is being inserted. That for the most part is<br>> okay if the new datapoint count is low thousands. However the issue arises when<br>> >50 thousand datapoints are required quickly. Creating them causes noticable<br>> latency in storing all new datapoint data.<br>><br>> So the answer seems to be pre-create.<br>><br>> The concern I have is in the past (5 years ago) I tested creating rrd data files<br>> with an RRA of 1 year of data and the create time set to unix epoch. When
doing<br>> an update to that file it seemed like the entire rrd was rewritten since none of<br>> it's 'intervals' were valid for the update time of 'now'.<br>><br>> Does the latest 1.4 version do the same thing?<br>><br>> Or is it essentially a no-cost-difference in IO to update an rrd with a start<br>> time (and thus round-robin end time) with a time interval in the future of it?<br>><br>> I don't want this feature to have a negative impact on IO in nearly the same<br>> manner as creating them would be.<br>><br>> I could try and test it, but RRD developers will know for certain what the total<br>> cost is, which is what I'm most interested in.<br><br>there is no smartness about updating 'old' rrd files in rrdtool<br>today ... if you have to go this route you may want to add a<br>'rebase' option to rrdtool tune which lets you modify the last<br>update property in an existing rrd file.<br><br>I wonder though if you
could not design your system in a way that<br>it does NOT break when it gets a request for 50'000 rrds by makeing<br>sure the process who tries to submit information does not have to<br>way for it to be actually stored back ...<br><br>cheers<br>tobi<br>><br>><br>> Thanks,<br>><br>> -Ryan<br>><br>><br>><br><br>-- <br>Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland<br><span><a target="_blank" href="http://it.oetiker.ch">http://it.oetiker.ch</a> </span><a ymailto="mailto:tobi@oetiker.ch" href="mailto:tobi@oetiker.ch">tobi@oetiker.ch</a> ++41 62 775 9902 / sb: -9900<br></div></div><div style="position:fixed"></div>
</div><br>
</body></html>