[rrd-developers] a portable rrd format

[Tobias Oetiker]

Hi Alex,

> On Wed, Jun 20, 2007 at 10:26:31PM +0200, Tobias Oetiker wrote:
>
> > * Rolling our own should not be all that difficult.
> >
> > I have created a little writeup on
> >
> > http://oss.oetiker.ch/rrdtool-trac/wiki/PortableRrdFormat
>
> A couple of random thoughts:
>
> Is there really a need to convert NaNs to the local architecture or
> will processors deal with different but correct types?
> I'm not talking endianness here; I'm talking about 0x7ff400...
> vs. 0x7ff800... vs 0x7ff00...1 and so on.

I don't know, but I would rather not bet on this ...

> Should there be a flag (or similar) to decide wether to convert
> a database to/from the independant format, and otherwise work with
> the local format (optimize for speed)?
> -a- creating a database: in which format
> -b- updating a database in a wrong format: write back
>     -b1- in native format (whole database, obviously)
>     -b2- in independant format (whole database, obviously)
>     -b3- in the same format (just the relevant part, of all?)
>
> Should there be some compromise (another flag): use the system's
> endianness but be compatible in every other fashion; meaning always
> use the same size of data?

the conversion does not have any measurable performance impact ...

        for (ii=0;ii<10e6;ii++)
            target.l = x862sparc(source.l);

For 10 million NAN conversions it takes         u:56.0040ms s:0.0000ms r:57.6040ms
For 10 million normal conversions I get         u:60.0040ms s:0.0000ms r:58.1790ms

since rrd_update runs at 20k updates per second if it all goes to
memory, the format change will have no measurable performance
impact.

> Is using floating point really the best option?  You are aware
> that 10 times 0.1 is not 1, right?

I don't think I will be able to come up with a data format that is
better suited to random numerical data than the brains who did
IEEE 754 :-) ... the only challenge is to not do 'bad' math where
we loose precision ...

cheers
tobi

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
http://it.oetiker.ch tobi at oetiker.ch ++41 62 213 9902