[rrd-users] rrdtool graph lockup

[Sean Harding]

Suddenly a couple of my 'rrdtool graph' commands have stopped working. They
just hang forever, spinning on the CPU:

   PID USERNAME THR PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
 25672 sharding   1   0    0 1552K 1472K run      1:36 90.16% rrdtool


I haven't played with it in a debugger, but truss shows this:

open("/usr/local/www/rrddata/mail.rrd", O_RDONLY) = 3
fstat64(3, 0xFFBD60A8)                          = 0
brk(0x00074120)                                 = 0
brk(0x00076120)                                 = 0
ioctl(3, TCGETA, 0xFFBD6034)                    Err#25 ENOTTY
read(3, " R R D\0 0 0 0 1\0\0\0\0".., 8192)     = 8192
brk(0x00076120)                                 = 0
brk(0x00078120)                                 = 0
llseek(3, 0, SEEK_CUR)                          = 8192
lseek(3, 20724, SEEK_SET)                       = 20724
lseek(3, 11124, SEEK_SET)                       = 11124
read(3, "\0\0\0\0\0\0\0\0\0\0\0\0".., 8192)     = 8192
llseek(3, 0xFFFFFFFFFFFFE910, SEEK_CUR)         = 13444
close(3)                                        = 0
brk(0x00078120)                                 = 0
brk(0x0007A120)                                 = 0
brk(0x0007A120)                                 = 0
brk(0x0007C120)                                 = 0
brk(0x0007C120)                                 = 0
<continues the same until I kill the process>


This is with rrdtool-1.0.42, just upgraded from 1.0.34 (after the problem
started, in hopes that it would be fixed in a new version) on Solaris 8. 

I am creating 6 graphs from the same rrd file, and 4 of them work. The other
2 hang like this. One of the ones that hangs is this command:

/usr/local/rrdtool/bin/rrdtool graph ${imagedir}/mail-day.gif --units-exponent 0 --start -86400 --vertical-label "Messages" DEF:inoctets=${datadir}/mail.rrd:messages:AVERAGE DEF:outoctets=${datadir}/mail.rrd:spam:AVERAGE AREA:inoctets#00FF00:"Total messages received" LINE1:outoctets#0000FF:"Spam messages received"


One that works is this command:

/usr/local/rrdtool/bin/rrdtool graph ${imagedir}/mail-15.gif --units-exponent 0 --start -900 --vertical-label "Messages" DEF:inoctets=${datadir}/mail.rrd:messages:AVERAGE DEF:outoctets=${datadir}/mail.rrd:spam:AVERAGE AREA:inoctets#00FF00:" Total messages received" LINE1:outoctets#0000FF:"Spam messages received"


(Yeah, yeah. So I just copied the commands from a config I use for router
graphing, which I pretty much copied from the tutorial. And I was too lazy
to rename the vnames from "inoctets" and "outoctets.")

The database was created with this command (also copied from another config,
copied from the tutorial):

/usr/local/rrdtool/bin/rrdtool create ${name}.rrd \
        --start 1052197321 --step 60 \
        DS:messages:COUNTER:120:U:U \
        DS:spam:COUNTER:120:U:U \
        RRA:AVERAGE:0:1:240 \
        RRA:AVERAGE:0:10:600 \
        RRA:AVERAGE:0:60:700 \
        RRA:AVERAGE:0:240:775 \
        RRA:AVERAGE:0:2880:797 \
        RRA:MAX:0:1:240 \
        RRA:MAX:0:10:600 \
        RRA:MAX:0:60:700 \
        RRA:MAX:0:240:775 \
        RRA:MAX:0:2880:797

The ones that work are the 15 minute, 1 week, 1 month and 1 year graphs. The
ones that fail are the 1 hour and 1 day graphs.

There is one thing that may have caused this problem to start (they happened
around the same time). The data in the rrd file is built from counts of
matching rows in a database. In normal operation, this number will
constantly increase. But due to an error, I had to make a change that caused
one of the two numbers being tracked to decrease by 1. I thought that this
might create bogus data (as COUNTER expects increasing numbers only, except
in case of overflow, right?), but I didn't think it would hose things badly
enough to make 'rrdtool graph' stop working completely. And I'm still
thinking it may have just been a coincidence. I dunno.

Is this just a matter of corrupted data in my data file? If so, is there any
way to clean it up without starting over?

Thanks.

sean

-- 
Sean Harding -- sharding at dogcow.org -- http://dogcow.org/s/ -- KD7UAY
Seattle, WA, USA

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi