[rrd-users] Problem with different architectures

[Andreas Maus]

On Thu, Aug 23, 2007 at 03:10:37PM +0200, Fredrik W?nglund wrote:
> there is actually two problems, CPU and amount of data;
O.K.
I didn't thought about the amount of data because I assumed
the rrdfiles were stored on an external USB disk (the Asus
has external USB 2.0 ports).

> I have rrd-files with 1 to ~8 ds and two rra's. It takes about 50 
> seconds to dump a file with one ds and 70 seconds for a file with 3 
> ds'es. I plan to have about 30 rrd-files, so it would take about 30 
> minutes just to convert then.
I see. That is really a problem.

> Another problem is that the ASUS is located in Singapore, and the 
> central server in Sweden. The communication between them is via an quite 
> slow VPN-link.
Ouch! Now _that_ is a long distance !;)

> An rrd-file with one ds is 730kB and the corresponding xml-file is 
> 6.6MB, so its almost ten times more data to transfer between the hosts!
Hmmm ... How about compressing the XML data before you transfer it using
gzip or bzip2 ? OTOH this will require some CPU additional cycles.

> What is the difference between the rrd-files?
> The ASUS is MIPS-based and the central server is an PC. Both platforms 
> are 32-bit, but the PC is little-endian and MIPS is big.
> If this is the only difference, it should not be so hard for an 
> C-programmer to write routine that converts the rrd-files between 
> different architectures...
Ahhh no. Unfortunately it is not that trivial.

Although IEEE specify the format of floating numbers
the represantation of floats also depends on the architecture.

There are different representations of some "magic" floats
like -NaN, +NaN, Inf,-1,+1. Even the alignment may vary!

For some examples you can take a look at http://oss.oetiker.ch/rrdtool-trac/wiki/PortableRrdFormat
(And yes a portable way representing rrd date is planned ;)

I don't know your infrastructure (line speed, etc.) but my first
try would be do rrdtool dump | gzip -9 -c | nc <remote_server> <port>
and set up an listener on the remote server using nc -l <IP> -p <port> | rrdtool restore ...

But thats just my wild speculation. YMMV.

So long,

Andreas Maus.

-- 
Dipl.-Ing. Andreas Maus             science+computing ag
System Administration               Hagellocher Weg 73
tel.: +49 7071 9457 671             72070 Tuebingen, Germany
fax: +49 7071 9457 411              www.science-computing.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.oetiker.ch/pipermail/rrd-users/attachments/20070823/95794714/attachment.bin