[rrd-users] Re: Adding many data samples at once

[Marc Bilodeau]

You're right about the order of variables. It was a typo!.

I tried your recommendation and still I get the error message:

ERROR while updating 21.rrd: expected 2 data source readings (got 576)

The issue is with LOTS of RRDs the performance lags when you create an RRD
and try to fit a months worth of 5 minutes samples into several rrds at once
(converting some raw 5 minutes samples into rrd files). Right now, to do 10
rrd files at 8928 (number of 5 minutes samples in 31 days) takes 10 to 11
seconds. I'm trying to reduce this into a single RRD update perl file in
order to reduce the overall time to convert.

For those who are interested, here is my complete example:

use RRDs;

# 288 = day
# 2016 = 7 days
# 8928 = 31 days

if (! $ARGV[0]) { 
	print "\n\nNo Arg specified defaulting to 8928 (31 days)\n";
	$ARGV[0] = 8928;
}

print "Total Datapoints: $ARGV[0]\n";

foreach (1..10) {
	unlink "2${_}.rrd";
	RRDs::create ("2${_}.rrd",
		"--start","1",
		"DS:input:GAUGE:600:U:U",
		"DS:output:GAUGE:600:U:U",
		"RRA:AVERAGE:0.5:1:600",
		"RRA:AVERAGE:0.5:6:700",
		"RRA:AVERAGE:0.5:24:775",
		"RRA:AVERAGE:0.5:288:797",
		"RRA:MAX:0.5:1:600",
		"RRA:MAX:0.5:6:700",
		"RRA:MAX:0.5:24:775",
		"RRA:MAX:0.5:288:797"
	);
}

$StartTime = time();

foreach $c (1 .. 10) {
	@DataPoints = undef;
	foreach $d (1 .. $ARGV[0]) {
		$TimeStamp = time() + ($d * 300);
		$in = int(rand(1000000));
		$out = int(rand(1000000));

		push (@DataPoints, "${TimeStamp}:${in}:${out}");
	}
	
	#RRDs::update ("2${c}.rrd", @DataPoints);
	RRDs::update ("2${c}.rrd", join (" ", at DataPoints));

	my $ERR=RRDs::error;
	die "ERROR while updating 2${c}.rrd: $ERR\n" if $ERR;
}


$totalTime = time() - $StartTime;

print "total time : $totalTime\n";


I appreciate the help!

Sincerely,
Marc

-----Original Message-----
From: Serge Maandag [mailto:serge.maandag at staff.zeelandnet.nl] 
Sent: Thursday, April 14, 2005 10:46 AM
To: Marc Bilodeau; rrd-users at list.ee.ethz.ch
Subject: RE: [rrd-users] Re: Adding many data samples at once

> foreach (1..5000) {
> 	$NewTimeStamp = $baseTime + 300;
> 	Push (@DataPoints, "ValIn:ValOut:$NewTimeStamp");
> } 

shouldn't that be "$NewTimeStamp:ValIn:ValOut" ?

> RRDs::update ("test.rrd", @DataPoints);
> 
> The problem is that this will produce an error because it 
> tells me that it
> was looking for 2 data points but got 5000 (in this example). 
> How can I
> perform this update in 1 rrd update instead of (much slower) 
> 5000 updates?

Uhm, perhaps this works?

RRDs::update ("test.rrd", join (" ", at DataPoints));

Serge.

-------------
Op de inhoud van dit e-mailbericht en de daaraan gehechte bijlagen is de
inhoud van de volgende disclaimer van toepassing:
http://www.zeelandnet.nl/disclaimer.php

-------------
For the content of this e-mail message and its attachment(s) the following
disclaimer applies: http://www.zeelandnet.nl/disclaimer.php

---
[This E-mail scanned for viruses by Declude Virus]


---
[This E-mail scanned for viruses by Declude Virus]

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-users
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi