[rrd-users] rrdtool resize corrupts timestamps in both 1.3.9 and 1.4.7

[Ryan Kubica]

NFS is fine for read-only access to rrd data; in fact it's very fast.

NFS is not fine for mmap'ed writes to rrds from multiple sources (even from the same server.)  Though building rrdtool with mmap disabled gets around that and with a proper flock'ing nfs client/server, it'll be fine ... with the inherent issue that write use of NFS like this on many files is likely going to be slow.

An architecture that works well is a rrd data node exports it's volume via NFS to a rrd ui server, so the data node only serves bytes out to the ui generator doing the computation/graph/exports/whatever.


Jo,

I'm curious, you say compiling your own resolved the issue, did you try compiling with mmap support?  or only without?

I've never run into this issue before (and store a lot of data) and on 64bit hosts, but:

a) I always compile rrdtool not use stock OS rpm
b) It is compiled with mmap()
c) I only resize an rrdtool file when it's not being written to (my apps write daemon does the resize in place as a task.)

and it works for resizes just fine (just tested manually), I grew RRA 0 by 2x.

[tmp]$ grep 1340865780 *xml
Epic_io_90.xml:<!-- 2012-06-28 06:43:00 UTC / 1340865780 --> <row><v>9.7033939036e+04</v></row>
resize.xml:<!-- 2012-06-28 06:43:00 UTC / 1340865780 --> <row><v>9.7033939036e+04</v></row>

[tmp]$ grep 1340865840 *xml
Epic_io_90.xml:<!-- 2012-06-28 06:44:00 UTC / 1340865840 --> <row><v>9.7633476991e+04</v></row>
resize.xml:<!-- 2012-06-28 06:44:00 UTC / 1340865840 --> <row><v>9.7633476991e+04</v></row>


-Ryan


________________________________
 From: Steve Shipway <s.shipway at auckland.ac.nz>
To: Jo Rhett <jrhett at netconsonance.com> 
Cc: "rrd-users at lists.oetiker.ch" <rrd-users at lists.oetiker.ch> 
Sent: Monday, July 16, 2012 7:28 PM
Subject: Re: [rrd-users] rrdtool resize corrupts timestamps in both 1.3.9 and 1.4.7
 

>Nope, they are on local storage.  Anyway, what do you find "unlikely" ?  
>This is 100% reproducible across many machines, centos 5.4 all the way 
>through 5.8. Some with stock installed kernel, some with the very latest 
>from RedHat.  I haven't tested yet on CentOS 6.x.
 
It is the use of NFS to hold the RRD files that I find unlikely, not the existence of a bug in RRDTool.  As we all know, bugs are always likely J
 
My guess is that this is somehow connected to MMAP and 64bit kernels but I'm not familiar with that part of the RRDTool development.  Best to see if Tobi can come up with anything – or anyone else on this list know?
 
Steve
 

________________________________

Steve Shipway
ITS Unix Services Design Lead
University of Auckland, New Zealand
Floor 1, 58 Symonds Street, Auckland
Phone: +64 (0)9 3737599 ext 86487
DDI: +64 (0)9 924 6487
Mobile: +64 (0)21 753 189
Email: s.shipway at auckland.ac.nz
PPlease consider the environment before printing this e-mail
 
 
_______________________________________________
rrd-users mailing list
rrd-users at lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20120716/2c0c2cc4/attachment.htm