[rrd-users] Trouble getting RRDtool restore to work

Tue Oct 21 02:37:41 MEST 2003

Hi All,

I have 3000+ MRTG targets in .log format on a Win2K server, and I would like
to migrate them to use RRDtool, but I am having trouble with the conversion.
When I set LogFormat to  "RRDtool" the resulting RRD files are corrupted,
usually part way through the 30 minute average sections (See below). Newly
created RRD files are fine but I don't wish to lose the history in the
existing .log files.

I have isolated the problem to the RRDtool restore function. The XML text
passed by MRTG to RRDtool looks OK but the RRD output file is always
corrupted. I wonder if the problem has something to do with the old chestnut
of  the new line being \x0D\x0A on NT and \x0A on *nix, as the "restore"-ed
.rrd files contain several \x0D\x0A byte sequence where as the small sample
of good RRD files I looked at do not have any. Is this a coincidence?

I know very (very ...) little about C and only enough Perl to be dangerous,
but I wonder if line 283 of rrd_restore.c
      int fd = open(file_name,O_RDWR|O_CREAT|O_EXCL,0666);
should be something like 
      int fd = open(file_name,O_RDWR|O_CREAT|O_EXCL|O_BINARY,0666);
I'm guessing that it's too late for fdopen to force binary mode when open
already set the file handle's parameters.  I have no way to test this as I
don't have a C compiler installed on my PC.

>From a little Google searching I can see a few very similar messages so I
think I am not alone in this problem:
	http://www.ee.ethz.ch/~slist/rrd-users/msg07120.html
<http://www.ee.ethz.ch/~slist/rrd-users/msg07120.html>   
	http://www.ee.ethz.ch/~slist/mrtg/msg25531.html
<http://www.ee.ethz.ch/~slist/mrtg/msg25531.html> 
	http://groups.yahoo.com/group/mrtg/message/26502
<http://groups.yahoo.com/group/mrtg/message/26502> 
This last one looked very promising but the email archive just stops. It is
suggested in this thread and I have just confirm that 1.0.33 works (but,
alas, not with ActivePerl 5.8). Looking at the source code of 1.0.33 the
same line 282 uses fopen rather than the open/fdopen combination. [I have no
idea what the difference is or why the change was made]
      if ((rrd_file = fopen(file_name,"wb")) == NULL ) {

I would gratefully appreciate some confirmation that RRDtool restore does
work on Win2K (etc), and if so did are you using the pre-compiled binaries
provided on http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/pub/
<http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/pub/>   ? 

Cheers,

Alistair

PS Further diagnostics that may be of interest follows:
#Here are my version numbers:
	RRDtool	rrdtool-1.0.40.x86distr.zip-5.8
	MRTG	mrtg-2.9.29
	Perl 	Activestate 5.8.0 build 806
	OS	Win2K 

#Here's a sample of the corruption in XML format:
<snip>
	<!-- 2003-10-17 09:30:00 GMT Daylight Time / 1066379400 --> <row><v>
7.3100000000e+002 </v><v> 4.5200000000e+002 </v></row>
	<!-- 2003-10-17 10:00:00 GMT Daylight Time / 1066381200 --> <row><v>
7.6600000000e+002 </v><v> 5.2700000000e+002 </v></row>
	<!-- 2003-10-17 10:30:00 GMT Daylight Time / 1066383000 --> <row><v>
-5.1876611110e-058 </v><v> -1.5208732530e-210 </v></row>
	<!-- 2003-10-17 11:00:00 GMT Daylight Time / 1066384800 --> <row><v>
-7.8498582224e+136 </v><v> -3.9103185453e-148 </v></row>
	<!-- 2003-10-17 11:30:00 GMT Daylight Time / 1066386600 --> <row><v>
-2.4273924526e+160 </v><v> -1.2712440113e-074 </v></row>
<snip>

# Here's some typical output from a test showing the problem.
S:\>RRDtool dump test.rrd | RRDtool restore -  test_out.rrd

S:\>RRDtool dump test.rrd | find "1066417200"
	<!-- 2003-10-17 20:00:00 GMT Daylight Time / 1066417200 --> <row><v>
1.0000000000e+000 </v><v> 1.0000000000e+000 </v></row>
	<!-- 2003-10-17 20:00:00 GMT Daylight Time / 1066417200 --> <row><v>
1.0000000000e+000 </v><v> 1.0000000000e+000 </v></row>

S:\>RRDtool dump test_out.rrd | find "1066417200"
	<!-- 2003-10-17 20:00:00 GMT Daylight Time / 1066417200 --> <row><v>
5.2998088236e-315 </v><v> 5.2998088236e-315 </v></row>
	<!-- 2003-10-17 20:00:00 GMT Daylight Time / 1066417200 --> <row><v>
8.0868664911e-320 </v><v> 8.0868664911e-320 </v></row>
S:\>

-----------------------------------------------------------------------

Registered Office:
Marks & Spencer p.l.c
Michael House, Baker Street,
London, W1U 8EP
Registered No. 214436 in England and Wales.

Telephone (020) 7935 4422 
Facsimile (020) 7487 2670

www.marksandspencer.com

Please note that electronic mail may be monitored.

This e-mail is confidential. If you received it by mistake, please let us know and then delete it from your system; you should not copy, disclose, or distribute its contents to anyone nor act in reliance on this e-mail, as this is prohibited and may be unlawful.

The registered office of Marks and Spencer Financial Services PLC, Marks and Spencer Unit Trust Management Limited, Marks and Spencer Life Assurance Limited and Marks and Spencer Savings and Investments Limited is Kings Meadow, Chester, CH99 9FB. These firms are authorised and regulated by the Financial Services Authority. 

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi