[rrd-developers] implementing portable format

[Bernhard Reutner-Fischer]

On Tue, Apr 07, 2009 at 09:08:02PM +0200, Tobias Oetiker wrote:
>Kevin,
>
>Today kevin brintnall wrote:
>
>> On Tue, Apr 07, 2009 at 08:12:26PM +0200, Bernhard Reutner-Fischer wrote:
>> > On Sat, Nov 01, 2008 at 10:08:06AM +0100, Tobias Oetiker wrote:
>> >
>> > >This sounds like quite a bit of added complexity. It would
>> > >require us to be able to convert from all platforms to all other
>> > >platforms ... I have the feeling that adding one new format with
>> > >the ability to read the old one already puts quite something on our
>> > >plate.
>> >
>> > Why didn't we use htonl(), ntohl() etc, btw?
>> >
>> > It's not that converting between "host" and "network" byte order is
>> > something new, ISTR..
>> >
>> > This means that we just store the data in network byte order on disk
>> > and read it back with ntohl(), ntohs(). This should result in
>> > much reduced pain, i would say.
>>
>> Given the various machine architectures (now and future), and the various
>> packing strategies employed by all the compilers out there, it makes sense
>> to use 64-bit values.  Is there a standard ntohl() equivalent for 64-bits?

quoting man 3 endian:
SYNOPSIS
       #define _BSD_SOURCE
       #include <endian.h>

       uint16_t htobe16(uint16_t host_16bits);
       uint16_t htole16(uint16_t host_16bits);
       uint16_t be16toh(uint16_t big_endian_16bits);
       uint16_t le16toh(uint16_t little_endian_16bits);

       uint32_t htobe32(uint32_t host_32bits);
       uint32_t htole32(uint32_t host_32bits);
       uint32_t be32toh(uint32_t big_endian_32bits);
       uint32_t le32toh(uint32_t little_endian_32bits);

       uint64_t htobe64(uint64_t host_64bits);
       uint64_t htole64(uint64_t host_64bits);
       uint64_t be64toh(uint64_t big_endian_64bits);
       uint64_t le64toh(uint64_t little_endian_64bits);

so yes, this shouldn't be a real problem.

>> Most of the work is in ensuring that all accesses to the file use the
>> proper xtoy() and ytox()...  not actually picking the function.
>
>my vision is that we create accessor functions for the individual
>parts of the rrd file format, the number of input formats supported
>is basically just a 'fleissarbeit' as we say in german, given the
>structure if the accessors is chosen such that they can be easily
>enhanced for different architectures ...

This is really not concerned with the architecture in any way.

You would just read any data back with the appropriate per data accessor
from libc. like e.g.:
typedef struct stat_head_t /* can drop float_cookie now! :) */ {
	unsigned long ds_cnt;
}
inline void get_stat_head_ds_cnt(unsigned long ul) {}
inline void set_stat_head_ds_cnt(unsigned long ul) {}
You get the idea..