[rrd-developers] striping - initial observations

[Daniel.Pocock at barclayscapital.com]

> Try removing the "-z" argument.  It causes RRD write times to 
> be randomly spread over the "-z" time.  This may reduce the 
> degree to which your writes are coalesced.

I will try that and report back
> 
> I'd be interested to see what your approach is for managing 
> the striped files once we need to rm individual RRDs.  In 
> many cases, the set of RRDs is a moving target.  It would 
> also be nice if the "slots" in the stripe files could be 
> reused once a file was rm'ed..  Or re-packed when there are a 
> lot of holes.

They can be deleted by clearing the filename from the index (a very
simple array of structs, also mmaped).  RRD_CREAT overwrites the file in
place.

> 
> Also, the striped RRDs must all have the same RRA/DS 
> definitions right?

Yes

> Then, it would make sense to set the stripe size to some 
> multiple of a full row..  For some K:
> 
> 	strip_size = K * sizeof(rrd_value_t) * ds_cnt


That makes sense - Ganglia has one ds per RRD, and 32 is a multiple of
sizeof(rrd_value_t), so I believe this is fine.  I intend to make stripe
size configurable rather than having a system for guessing it, and it
will probably need to be a power of 2 to make the math efficient for
finding the correct position in the file.

Regards,

Daniel

_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________