[rrd-developers] two problem about RRD UPDATE

[lily]

Hi ,

On Sun, Mar 15, 2009 at 6:08 AM, Florian Forster <
rrdtool at nospam.verplant.org> wrote:

> Hi,
>
> On Sat, Mar 14, 2009 at 12:01:42PM +0800, lily wrote:
> >>     Recently I use the rrdtool to create about 12 thousand rrd-file, and
> >> update them in 30s .
>
> > the important metric here is updates per second. Updating 12k RRD files
> > every 30 seconds qeuals 400 updates per second. In my experience, you
> > can expect about 120-150 updates per second from decent disks. If you
> > can stay in memory, you can do significantly more updated of course..


       How can I keep the data in memory? Use the mmap ? Is upgrade the
version to 1.3.6 wil be OK  or I have to something in my code ?
       My rrdtool is the 1.2.27, I don't know if I upgrade my version to
1.3.6, Is there some *compatibility *problem ?

>
>
> >> [...], and the FS is ext2
>
> >> [...], and do the vm Optimizations. But it seems not work. The IO
> >> await list is sometime very high.
>
> > You did mount with `noatime' and `nodiratime', right? While this doesn't
> > perform miracles, it'll certainly increase performance some percent..


      Yes , I did not do that . But if I do that , I don't have the access
time , rigth ?

>
>
> >>        Device:    rrqm/s wrqm/s   r/s   w/s  rsec/s  wsec/s    rkB/s
> wkB/s avgrq-sz avgqu-sz   await  svctm  %util
> >>       cciss/c0d0   0.00   0.00  0.00 112.44    0.00  907.77     0.00
>  453.89     8.07   112.99 1001.39   9.23 103.73
>
> > 450 kBytes/s seems a bit low for a RAID-10 (or RAID-0+1? I keep
> > forgetting which one is the one with the better write performance ;)
> > setup. With 15k disks you should be able to get as high as
> > 1.5 - 2.0 MBytes/s. Can you tell us some more about your system? How
> > much RAM do you have? What kind of disks? What are the precise options
> > you used when mounting the file system? This kind of stuff..


       The await list is very high(1001.39), so the wkB/s is very low
(458.89). If the await is very low ( < 3), In my system , a HP 380G5, the
wkB/s can be the speed between 30M ~ 40MB.
        My system is HP 380G5, with 4G RAM . I mount the DISK with no
options, use the default.

>
>
> >>     The second is sometime when I update the files , there is an error :
> >>          illegal attempt to update using time 1236857494 when last
> update
> >> time is 1236857494 (minimum one second step)
> >>      but I use the timestamp is 1236886011, not the 1236857494. And I
> use
> >> the rrdtool last to see the last update time of the file , I find the
> last
> >> update time is 1236857468, also not 1236857494. I have debug a lot of
> time ,
> >> but have no idea about it .
>
> > I haven't observed this problem yet, but due to misconfiguration by
> > users I see this message a lot (when it is legitimate). I see other
> > error messages on two machines that are under heavy load though. Such
> > messages include:
>
> >  - closing rrd
> >  - reading the cookie off $FILE faild [sic]
> >  - is not an RRD file
>
> > When things go really bad, I get "Unknown CF ''" followed by a SIGFPE.
>
> > I suspect that the high load on the machines leads to some incomplete
> > reads/write which cause these messages/problems. Unfortunately I don't
> > have debugging symbols on these machines and no way of getting some onto
> > there, so haven't been able to track these problems down yet :/


      Now I reduce the number of the file from 12K to 8K , I have not
encountered the problem yet. So I think the reason is the high load .

>
>
> Regards,
> -octo
> --
> Florian octo Forster
> Hacker in training
> GnuPG: 0x91523C3D
> http://verplant.org/
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
>
> iD4DBQFJvCrfHdggu3Q05IYRAmd3AJiZOH4ACXcE2S9KKM6V/vjSvjEwAJ9PW/0x
> 9c2MNqwQGo8Oaxx3YexeVQ==
> =8dAd
> -----END PGP SIGNATURE-----
>
>


-- 
Yours Sincerely,
lleelm -at- gmail -dot- com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-developers/attachments/20090315/9cd17c98/attachment-0001.html