[rrd-developers] [Ganglia-developers] Integrating rrdcached with Ganglia

[Bernard Li]

Hi Daniel:

On Mon, Sep 29, 2008 at 8:34 AM,  <Daniel.Pocock at barclayscapital.com> wrote:

>> Thanks for this - I decided to try it:
>>
>> - RHEL5
>> - 64 bit
>> - APR 1.2.7
>> - rrdtool 1.3.2 + rrdcached patch from here:
>>     http://dist.rufus.net/~kbrint/rrdcached/rrdcache.patch
>> - Ganglia 3.1.0
>>
>> gmetad stores the data and IO levels are definitely better.
>>
>> However, when I display the web page with graphs for every
>> farm, rrdcached crashes, SEGV:
>>
>> (gdb) bt
>> #0  0x0000003312077b60 in memcpy () from /lib64/libc.so.6
>> #1  0x00002aaaaaae1633 in rrd_write () from /usr/lib64/librrd.so.4
>> #2  0x00002aaaaaae5e10 in rrd_lock () from /usr/lib64/librrd.so.4
>> #3  0x00002aaaaaae69e2 in rrd_lock () from /usr/lib64/librrd.so.4
>> #4  0x00002aaaaaae7cfb in rrd_lock () from /usr/lib64/librrd.so.4
>> #5  0x0000000000405113 in ?? ()
>> #6  0x00000033134062f7 in start_thread () from /lib64/libpthread.so.0
>> #7  0x00000033120ce85d in clone () from /lib64/libc.so.6
>>
>> The problem is reproducable every time I refresh
>> http://my-server/ganglia/
>
>
>
>
> I've upgraded to 1.3.3 (the patch applies with 1 reject, which is only
> the copyright message) and installed the debug symbols and found that
> each time this error occurs, rrd_update is trying to update an RRD file
> that is smaller than the expected size (i.e. smaller than the other RRD
> files in the same directory)
>
> I haven't yet established whether the RRD files were created earlier
> with the wrong size, or if they have somehow been truncated during the
> crash.
>
> The whole directory tree was empty before I started doing these tests,
> so the RRD files have definitely been created with rrdtool 1.3.

Instead of patching RRDTool manually, why don't you just check out the
code from trunk (which includes the rrdcached)?  I am not sure how
up-to-date the patch you pulled from Kevin is and whether he keeps
that up to date -- relative to the code that is being checked into
RRDTool trunk.

http://oss.oetiker.ch/rrdtool/download.en.html

BTW, are you subscribed to rrdtool-developers?

Thanks for testing.

Cheers,

Bernard