Hi , <br><br><div class="gmail_quote">On Sun, Mar 15, 2009 at 6:08 AM, Florian Forster <span dir="ltr"><<a href="mailto:rrdtool@nospam.verplant.org">rrdtool@nospam.verplant.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi,<br>
<div class="im"><br>
On Sat, Mar 14, 2009 at 12:01:42PM +0800, lily wrote:<br>
>> Recently I use the rrdtool to create about 12 thousand rrd-file, and<br>
>> update them in 30s .<br>
<br>
</div>> the important metric here is updates per second. Updating 12k RRD files<br>
> every 30 seconds qeuals 400 updates per second. In my experience, you<br>
> can expect about 120-150 updates per second from decent disks. If you<br>
> can stay in memory, you can do significantly more updated of course..</blockquote><div><br> How can I keep the data in memory? Use the mmap ? Is upgrade the version to 1.3.6 wil be OK or I have to something in my code ? <br>
My rrdtool is the 1.2.27, I don't know if I upgrade my version to 1.3.6, Is there some <strong>compatibility </strong>problem ?</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<br>
>> [...], and the FS is ext2<br>
<br>
>> [...], and do the vm Optimizations. But it seems not work. The IO<br>
<div class="im">>> await list is sometime very high.<br>
<br>
</div>> You did mount with `noatime' and `nodiratime', right? While this doesn't<br>
> perform miracles, it'll certainly increase performance some percent..</blockquote><div><br> Yes , I did not do that . But if I do that , I don't have the access time , rigth ?<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<div class="im"><br>
>> Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util<br>
>> cciss/c0d0 0.00 0.00 0.00 112.44 0.00 907.77 0.00 453.89 8.07 112.99 1001.39 9.23 103.73<br>
<br>
</div>> 450 kBytes/s seems a bit low for a RAID-10 (or RAID-0+1? I keep<br>
> forgetting which one is the one with the better write performance ;)<br>
> setup. With 15k disks you should be able to get as high as<br>
> 1.5 - 2.0 MBytes/s. Can you tell us some more about your system? How<br>> much RAM do you have? What kind of disks? What are the precise options<br>
> you used when mounting the file system? This kind of stuff..</blockquote><div><br> The await list is very high(1001.39), so the wkB/s is very low (458.89). If the await is very low ( < 3), In my system , a HP 380G5, the wkB/s can be the speed between 30M ~ 40MB.<br>
My system is HP 380G5, with 4G RAM . I mount the DISK with no options, use the default. <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<div class="im"><br>
>> The second is sometime when I update the files , there is an error :<br>
>> illegal attempt to update using time 1236857494 when last update<br>
>> time is 1236857494 (minimum one second step)<br>
>> but I use the timestamp is 1236886011, not the 1236857494. And I use<br>
>> the rrdtool last to see the last update time of the file , I find the last<br>
>> update time is 1236857468, also not 1236857494. I have debug a lot of time ,<br>
>> but have no idea about it .<br>
<br>
</div>> I haven't observed this problem yet, but due to misconfiguration by<br>
> users I see this message a lot (when it is legitimate). I see other<br>> error messages on two machines that are under heavy load though. Such<br>> messages include:<br>
<br>
> - closing rrd<br>
> - reading the cookie off $FILE faild [sic]<br>
> - is not an RRD file<br>
<br>
> When things go really bad, I get "Unknown CF ''" followed by a SIGFPE.<br>
<br>
> I suspect that the high load on the machines leads to some incomplete<br>
> reads/write which cause these messages/problems. Unfortunately I don't<br>> have debugging symbols on these machines and no way of getting some onto<br>
> there, so haven't been able to track these problems down yet :/</blockquote><div><br> Now I reduce the number of the file from 12K to 8K , I have not encountered the problem yet. So I think the reason is the high load . <br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>
<br>
Regards,<br>
-octo<br>
<font color="#888888">--<br>
Florian octo Forster<br>
Hacker in training<br>
GnuPG: 0x91523C3D<br>
<a href="http://verplant.org/" target="_blank">http://verplant.org/</a><br>
</font><br>-----BEGIN PGP SIGNATURE-----<br>
Version: GnuPG v1.4.6 (GNU/Linux)<br>
<br>
iD4DBQFJvCrfHdggu3Q05IYRAmd3AJiZOH4ACXcE2S9KKM6V/vjSvjEwAJ9PW/0x<br>
9c2MNqwQGo8Oaxx3YexeVQ==<br>
=8dAd<br>
-----END PGP SIGNATURE-----<br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Yours Sincerely,<br>lleelm -at- gmail -dot- com<br>