[rrd-users] performance problems on AIX 4.3.2

[REIBENSCHUH Alfred]

Hi !

first my configuration

System: IBM-RS6000 S7A (4x232MHz RISC, 8Gb Mem, 8Gb Swap, 36Gb Raid-5)
OS: AIX 4.3.2 with Maintainance Level 2 Patches
C-Compilers: IBM xlc 4.0.3 (Visual Age C/C++), GCC 2.95.2
RRD: rrdtool 1.0.10

the rrd is created with the following command:

----------------------------------------------------------
# snmpcolrrd_createrrd.sh
/opt/gnu/bin/rrdtool create $1 -b ` mktime -F "%t" -D 1990-01-01 ` -s 300 \
	DS:data:GAUGE:1200:U:U \
		RRA:AVERAGE:0.9999:1:144 \
		RRA:MAX:0.9999:1:144 \
		RRA:MIN:0.9999:1:144 \
		RRA:AVERAGE:0.9999:6:288 \
		RRA:MAX:0.9999:6:288 \
		RRA:MIN:0.9999:6:288 \
		RRA:AVERAGE:0.9999:36:124 \
		RRA:MAX:0.9999:36:124 \
		RRA:MIN:0.9999:36:124 
----------------------------------------------------------


i'm using rrdtool to update rrds im blocks of 120 values which
requires about 29 seconds, compiled rrdtool with both xlc and
gcc plus the -O3 flag and i'm wondering which such an update
takes this long. under normal circumstances i wouldn't mind but
i have either the choice to update all my 20000 rrds every 5 minutes
or 1 rrd on demand as it is displayed on the web which requires
much more datasets to update (~2000)

is it simply the task or should i upgrade to more CPUs

here is the script i use to update the rrds:
(it uses fork to speed up the update-process)
----------------------------------------------------------
#!/usr/local/bin/perl
# snmpcolrrdupdate.pl

if(! -e "/var/rrd/rrd.ini") {
	` cat /usr/OV/conf/snmpCol.conf | /opt/planet/bin/parsesnmpcolconf |
sort >/var/rrd/rrd.ini`;
}

@lines=` cat /var/rrd/rrd.ini `;
$fi=0;
foreach $line (@lines) {
	chomp($line);
	($h,$v)=split(/\s+=\s+/,$line);
	@hosts=split(/\s+/,$h);
	@vars=split(/\s+/,$v);
	while($fi>50){wait();$fi--;} # waiting for number of childs to drop
below 50
	$f=fork();$fi++; #spawning another child
	if($f){next;}
	foreach $host (@hosts) {
		if(!$host){next;}
		@v=`host $host | awk '{print \$3;}'`;
		$dir=$host;
		$dir=~s/[\/_]/-/cgi;
		$dir=uc($dir);
		chomp(@v);
		$ip=@v[0];
		$ip=~s/,//g;
		foreach $var (@vars) {
			if(!$var){next;}
			@v=`/opt/planet/bin/snmpcolis
/usr/OV/databases/snmpCollect/$var $ip`;
			if(!($v[0]=~/yes/i)){next;}
			print STDOUT "$host($ip) -- $var\n";
			if($var=~/octets/i) {
				$multi=8;
			} elsif($var=~/cell/i) {
				$multi=416;
			} else {
				$multi=1;
			}
			if(! -d "/var/rrd/out/$dir") {
				mkdir("/var/rrd/out/$dir",0777);
			}
			if(! -e "out/$dir/$var.rrd") {
				`/opt/planet/bin/snmpcolrrd_createrrd.sh
/var/rrd/out/$dir/$var.rrd`;
			}
			@last=`/opt/gnu/bin/rrdtool last
/var/rrd/out/$dir/$var.rrd`;
			$last=shift(@last); 
			chomp($last);
			$last++;
		#	$last=time()-7200;	#
			@data=`/opt/planet/bin/snmpcol2rrds
/usr/OV/databases/snmpCollect/$var $last $ip $multi`;
			chomp(@data);
			@data=sort(@data);
			while(@d=splice(@data,0,120)) {
                                $d=join(' ', at d);
				`/opt/gnu/bin/rrdtool update
/var/rrd/out/$dir/$var.rrd -t data $d `;
			}
		}
	}
	if(!$f){last;}
}
while(wait()>0){;}
----------------------------------------------------------

this requires hours to complete, which is not much to my liking.
will the perl-module of RRD speed up the process significantly (x20 or
better) ?


thanks ?


mfG. Alfred Reibenschuh
INFORMATIONSTECHNOLOGIE AUSTRIA GMBH
TELEKOMMUNIKATION
Networkmanagement
Intranet Info-koordinator
A-1020 Wien, Lassallestrasse 5
T: ++43-1-21717-58947
F: ++43-1-21717-58979
E: alfred.reibenschuh at it-austria.com

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users