[rrd-developers] striping - initial observations

[Daniel.Pocock at barclayscapital.com]

> > > Try removing the "-z" argument...

IO is now down to 160 IO/s - see attached.  Thanks for pointing this
out.

> > > Then, it would make sense to set the stripe size to some 
> multiple of 
> > > a full row..  For some K:
> > > 
> > > 	strip_size = K * sizeof(rrd_value_t) * ds_cnt
> > 
> > 
> > That makes sense - Ganglia has one ds per RRD, and 32 is a 
> multiple of 
> > sizeof(rrd_value_t), so I believe this is fine.  I intend to make 
> > stripe size configurable rather than having a system for 
> guessing it, 
> > and it will probably need to be a power of 2 to make the math 
> > efficient for finding the correct position in the file.
> 
> Lots of RRDs have more than one DS.  I don't think it's 
> "guessing", since the RRD geometry is known and fixed when 
> it's installed into the stripe file.
>
I probably should have used the work `calculating' instead of
`guessing'.  I didn't mean to suggest this wouldn't be useful.

> I can't think of any case where it would make sense to split 
> a single RRD row over more than one stripe.  The logic to 
> make the stripe_size = K*row_size should not be difficult.  
> Let's allow RRDTool to do the right thing regardless of the 
> RRD geometry...

Everyone who needs this will (hopefully) know enough about the subject
to calculate a suitable value manually.  However, it is something that
can be done automatically, based on a user specified K.

The essential issue for this type of storage, however, is the API for
accessing these `virtual' files, and that is the next area I am going to
focus on.  Without that, applications like gmetad can't see which RRDs
exist in the striped file.

_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________
-------------- next part --------------
A non-text attachment was scrubbed...
Name: io.png
Type: image/png
Size: 61531 bytes
Desc: io.png
Url : http://lists.oetiker.ch/pipermail/rrd-developers/attachments/20081022/0a6ac343/attachment-0001.png