[rrd-developers] rrdcached contention when flushing

[Daniel.Pocock at barclayscapital.com]

> > The memory leak could be the cause of the performance issue 
> - it grows 
> > to several gigabytes and there is swapping, that might be 
> reducing the 
> > amount of RAM available for caching the mmap() pages.  Can you make 
> > any suggestions for using valgrind or another tool in this scenario?
> 
> The leak may be causing other problems..  I'd try running a 
> separate instance with smaller files and see if you can get 
> valgrind to cooperate.
> I haven't found a better tool for tracking down leaks.
> 

I've shrunk it down to monitor just one host, that successfully avoids
the mmap() issue with valgrind.  The first thing that stands out is
this:

==9115== 15,872 bytes in 32 blocks are possibly lost in loss record 10
of 13
==9115==    at 0x4A04A1D: memalign (vg_replace_malloc.c:332)
==9115==    by 0x4A04A76: posix_memalign (vg_replace_malloc.c:425)
==9115==    by 0x3314041F50: (within /lib64/libglib-2.0.so.0.1200.3)
==9115==    by 0x331404326F: g_slice_alloc (in
/lib64/libglib-2.0.so.0.1200.3)
==9115==    by 0x331404BFFD: (within /lib64/libglib-2.0.so.0.1200.3)
==9115==    by 0x331404C221: (within /lib64/libglib-2.0.so.0.1200.3)
==9115==    by 0x404945: handle_request_update (rrd_daemon.c:1456)
==9115==    by 0x404F2E: handle_request (rrd_daemon.c:1633)
==9115==    by 0x405A3A: connection_thread_main (rrd_daemon.c:1987)
==9115==    by 0x33134062F6: start_thread (in /lib64/libpthread-2.5.so)
==9115==    by 0x33120CE85C: clone (in /lib64/libc-2.5.so)


In rrd_daemon.c, I found the tree initialised like this:

  cache_tree = g_tree_nw((GCompareFunc)strcmp);

There is an alternate form of g_tree_new() that allows you to specify
functions for freeing the memory used by keys - do you think this could
be an issue?  Are the keys meant to be re-used as new values come in, or
do they need to be completely freed and then re-created when new values
arrive for a given file?

_______________________________________________

This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing.  Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP.  This email may relate to or be sent from other members of the Barclays Group.
_______________________________________________