[rrd-developers] madvise (MADV_WILLNEED) read-ahead behaviour

[Daniel.Pocock at barclayscapital.com]

I suspect that the memory growth observed with rrdcached under high load
results from small I/O sizes.  The kernel is only getting those pages
that are being written, and so the queue is growing.

I've made some improvement by using madvise(MADV_WILLNEED) followed by
mlock() on the area of my stripe file that covers the RRD headers.  This
keeps about 150MB of data locked in RAM (the machine has 8GB).

However, the data section (RRA storage) is more complicated.  The
simplest idea that comes to mind is calling madvise() each time someone
calls rrd_read() or rrd_write().  This increases the I/O request sizes
(by pre-loading whole rows of stripes into RAM for future use), which is
good, but I quickly find that there are a lot of overlapping I/O
requests sent to the hardware - that seems a bit odd.  Shouldn't the
read-ahead algorithm only read pages that are not already in RAM?

Next I tried using mincore() to determine if the region used for
rrd_read() or rrd_write() is already resident in RAM.  If mincore()
returns true, then don't call madvise().  However, this isn't very
effective either, I suspect that it is because I/O requests have already
been queued but not yet completed.

Can anyone suggest a way to ensure that overlapping calls to
madvise(MADV_WILLNEED) don't trigger overlapping I/O read requests?

Regards,

Daniel Pocock

-------------------------------------------
Barclays Capital
40 Bank St, London, E14 5EN, United Kingdom
Phone: +44 (0) 20 3134 7982

_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________