[rrd-developers] MRTG , Segmentation Fault , After re-installing perl .
Mr. James W. Laferriere
babydr at baby-dragons.com
Thu Mar 12 19:43:16 CET 2009
Hello Kevin ,
On Wed, 18 Feb 2009, kevin brintnall wrote:
> On Wed, Feb 18, 2009 at 03:13:18PM -0900, Mr. James W. Laferriere wrote:
>> Starting program: /usr/local/bin/perl /usr/local/mrtg-2/bin/mrtg-ORIG
>> /var/www/htdocs/router2cgi/cfg/to-rtr01.cfg
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread -1210280256 (LWP 20444)]
>> 0xb7e3822b in strlen () from /lib/libc.so.6
>> (gdb) bt
>> #0 0xb7e3822b in strlen () from /lib/libc.so.6
>> #1 0xb7e0a281 in vfprintf () from /lib/libc.so.6
>> #2 0xb7e28d91 in vsnprintf () from /lib/libc.so.6
>> #3 0xb7d552ac in sprintf_alloc (fmt=0xb7d615c2 "[%lli]RRA[%s][%lu]DS[%s]") at rrd_info.c:34
>> #4 0xb7d5a193 in write_to_rras (rrd=0xbffd7a24, rrd_file=0x89c2c88, rra_step_cnt=<value optimized out>, rra_begin=2760,
>> current_time=1234999805, skip_update=0x89ce7c8, pcdp_summary=0xbffd7a80) at rrd_update.c:1944
>> #5 0xb7d5ad1f in process_arg (step_start=0x89ccfa0 "1234999805", rrd=0xbffd7a24, rrd_file=0x89c2c88, rra_begin=81864,
>> current_time=0xbffd7a58, current_time_usec=0xbffd7a54, pdp_temp=0x81520c0, pdp_new=0x89ce7f0, rra_step_cnt=0x89ce6f0,
>> updvals=0x87fcce0, tmpl_idx=0x88748d8, tmpl_cnt=3, pcdp_summary=0xbffd7a80, version=3, skip_update=0x89ce7c8,
>> schedule_smooth=0xbffd7a50) at rrd_update.c:819
>> #6 0xb7d5be8a in _rrd_update (filename=0x89ce730 "/var/www/htdocs/router2cgi/graphs/to-rtr01/to-rtr01_1.rrd", tmplt=0x0, argc=1,
>> argv=0x8873d80, pcdp_summary=0x887c9d0) at rrd_update.c:497
>> #7 0xb7d5c4a1 in rrd_update_v (argc=2, argv=0x8873d78) at rrd_update.c:362
>
> This dump validates your statement that the error only occurs during
> updatev... This is where sprintf_alloc() is being called. updatev
> implies (*pcdp_summary != null).
>
> -----------------------------------
> if (*pcdp_summary != NULL) {
> *pcdp_summary = rrd_info_push(*pcdp_summary,
> sprintf_alloc
> ("[%lli]RRA[%s][%lu]DS[%s]",
> rra_time,
> rrd->rra_def[rra_idx].cf_nam,
> rrd->rra_def[rra_idx].pdp_cnt,
> rrd->ds_def[ds_idx].ds_nam),
> RD_I_VAL, iv);
> }
> -----------------------------------
>
> However, that doesn't match up exactly with the call trace you posted (it
> should be in the function 'write_RRA_row'). It's probably because the
> compiler is optimizing away some of the function calls. You may want to
> re-compile with debugging enabled and no optimizations (i.e. CFLAGS=-g).
CFLAGS=-g ./configure ; CFLAGS=-g make ; CFLAGS=-g make install
root at dsb-mntr01:~# cat /root/rrdtool-segfault-20090312.log
GNU gdb 6.6
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i486-slackware-linux"...
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) set args /usr/local/mrtg-2/bin/mrtg-ORIG
/var/www/htdocs/router2cgi/cfg/to-rtr01.cfg
(gdb) run
Starting program: /usr/local/bin/perl /usr/local/mrtg-2/bin/mrtg-ORIG
/var/www/htdocs/router2cgi/cfg/to-rtr01.cfg
[Thread debugging using libthread_db enabled]
[New Thread -1210722624 (LWP 26013)]
Error while reading shared library symbols:
Cannot find new threads: generic error
Program exited normally.
(gdb) bt
No stack.
(gdb) run
Starting program: /usr/local/bin/perl /usr/local/mrtg-2/bin/mrtg-ORIG
/var/www/htdocs/router2cgi/cfg/to-rtr01.cfg
[Thread debugging using libthread_db enabled]
[New Thread -1210239296 (LWP 26077)]
Error while reading shared library symbols:
Cannot find new threads: generic error
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1210239296 (LWP 26077)]
0xb7e4222b in strlen () from /lib/libc.so.6
(gdb) bt
#0 0xb7e4222b in strlen () from /lib/libc.so.6
#1 0xb7e14281 in vfprintf () from /lib/libc.so.6
#2 0xb7e32d91 in vsnprintf () from /lib/libc.so.6
#3 0xb7d5f2ac in sprintf_alloc (fmt=0xb7d6b5c2 "[%lli]RRA[%s][%lu]DS[%s]") at rrd_info.c:34
#4 0xb7d64193 in write_to_rras (rrd=0xbfd57f94, rrd_file=0x89cd7d8, rra_step_cnt=<value optimized out>, rra_begin=2760,
current_time=1236882919, skip_update=0x89cd838, pcdp_summary=0xbfd57ff0) at rrd_update.c:1944
#5 0xb7d64d1f in process_arg (step_start=0x89cdd10 "1236882919", rrd=0xbfd57f94, rrd_file=0x89cd7d8, rra_begin=81864,
current_time=0xbfd57fc8, current_time_usec=0xbfd57fc4, pdp_temp=0x88795d0, pdp_new=0x89cdcf8, rra_step_cnt=0x89cd888,
updvals=0x881d4d8, tmpl_idx=0x89a5408, tmpl_cnt=3, pcdp_summary=0xbfd57ff0, version=3, skip_update=0x89cd838,
schedule_smooth=0xbfd57fc0) at rrd_update.c:819
#6 0xb7d65e8a in _rrd_update (filename=0x89cdd60 "/var/www/htdocs/router2cgi/graphs/to-rtr01/to-rtr01_1.rrd", tmplt=0x0, argc=1,
argv=0x89a3c60, pcdp_summary=0x89cda08) at rrd_update.c:497
#7 0xb7d664a1 in rrd_update_v (argc=2, argv=0x89a3c58) at rrd_update.c:362
#8 0xb7d730cf in XS_RRDs_updatev () from /usr/local/lib/perl5/site_perl/5.10.0/i686-linux//auto/RRDs/RRDs.so
#9 0x0809ed60 in Perl_pp_entersub ()
#10 0x0809d638 in Perl_runops_standard ()
#11 0x08099fa4 in perl_run ()
#12 0x0805ed7f in main ()
(gdb) exit
Undefined command: "exit". Try "help".
(gdb) quit
The program is running. Exit anyway? (y or n) root at dsb-mntr01:~#
root at dsb-mntr01:~#
root at dsb-mntr01:~#
root at dsb-mntr01:~# cat /root/rrdtool-segfault-20090312.log | grep -i write_rra
The above shows that either I have not configured & made rrdtool with
debugging or there is no write_RRA_row() available . I suspect the former .
> We need to see inside the args being passed to sprintf_alloc(). Can you
> reproduce the problem and then do the following:
>
> (gdb) bt
> -- note which frame is the call to write_RRA_row(), use in the next command
>
> (gdb) select <FRAME>
> (gdb) p *rrd
> (gdb) p rra_idx
> (gdb) p rrd->rra_def[rrd_idx]
> (gdb) p rrd->rra_def[rrd_idx].cf_nam
> (gdb) p rrd->rra_def[rra_idx].pdp_cnt
> (gdb) p rrd->ds_def[ds_idx].ds_nam
>
> Also, I have a few other questions:
>
> - is it always the same RRD file that causes the crash?
> - does that RRD cause the crash every time?
>
> - do you have any similar files (i.e. same RRA and DS definitions) that
> do not cause crash?
>
>
--
+------------------------------------------------------------------+
| James W. Laferriere | System Techniques | Give me VMS |
| Network&System Engineer | 2133 McCullam Ave | Give me Linux |
| babydr at baby-dragons.com | Fairbanks, AK. 99701 | only on AXP |
+------------------------------------------------------------------+
More information about the rrd-developers
mailing list