[rrd-users] Re: adding many rows with rrdtool resize causing problems.... HELP!

[Kempf, Reed]

well, I didn't seem to have the same kind of problems that I was reading
about on the list.  I used a bash and sed script to dump out the rrd into
xml format and then used sed to add another ds.  That worked smoothly, then
I used rrdtool resize command to add 78,000+ rows to the rrd which was
adding 9 months of data into my rrd.  I left my graphing function alone
which graphed on a daily, weekly, monthly and yearly basis basically using
the start -s <epoch_time> and end -e <epoch_time> syntax.

I designed the program to only insert on 5 minute intervals which is how my
rrd's are set up.  I collect data for 5 minutes and determine the current
interval epoch time, then do an insert into the rrd with this value so I am
not inserting values constantly throughout the interval.  This might have
something to do with it.

If anybody wants to see the specifics of my bash, sed or graphing scripts, I
would be happy to share them.  I thankfully didn't encounter any problems
with this method.

BTW - this program parses an apache_web_log file and calcutes bytes_sent,
load_times and error_codes and graphs trend data on these data sources.

Regards,

ReedK

-----Original Message-----
From: Alex van den Bogaerdt [mailto:alex at ergens.op.het.net]
Sent: Thursday, December 26, 2002 3:20 PM
To: rrd-users at list.ee.ethz.ch
Subject: [rrd-users] Re: adding many rows with rrdtool resize causing
problems.... HELP!



On Thu, Dec 26, 2002 at 01:35:51PM -0700, Kempf, Reed wrote:

> The issue is that I am trying to resize about 2000 rrdtool databases
because
> my requirements changed from keeping 3 months of data to keeping a years
> worth of data and I would really like to do this maintenance once instead
of
> once per week.  The databases are continuously being written to and it
would
> increase the possibilities of data loss if I had to do this every week not
> to mention the maintenance headache of adding rows to 2000 databases every
> week.
> 
> If anyone can enlighten me on why the "yearly" graph will look odd if I
add
> too many rows would be appreciated.

If you add rows to the 5-minute RRA these rows will contain NaN values.
If you add enough rows to satisfy a request for a complete year, you
will fetch the 5-minute rows and thus see unknown data.  You expect to
see known data thus RRDtool doesn't seem to match your expectations.

This can be circumvented:  If you ask for "--end hh:mm" "--start
end-365days"
where "hh:mm" is midnight UTC represented in your local time, RRDtool
will use the original 1-day RRA and thus will not use the freshly added
5-minute rows.

In general: RRDtool will try to match your request as close as possible.

For MRTG-compatible RRDs this means RRDtool won't use the "daily" RRA
to graph a full year.  It cannot use the "weekly" or "monthly" RRAs
either so the only choice is to use the "yearly" RRA.

If you alter the amount of rows in the "daily" RRA to, for instance,
120000 rows (more than a year) RRDtool has to choose between the
"daily" RRA and the "yearly" RRA as both contain enough rows to
satisfy your request.  RRDtool will then look at the start and end
time of the data to fetch.  The "yearly" RRA contains rows that start
and end on 00:00 UTC.  The "daily" RRA contains rows that start and
end on multiples of 5 minutes.

If you request "now" it is highly likely the "daily" RRA will be
selected.  If you request n*86400 (for example: 1040947200) as the
end time and "--start end-400days" for the start time, RRDtool will
select the "yearly" RRA.

-- 
Much of what looks like rudeness in hacker circles is not intended to give
offence. Rather, it's the product of the direct, cut-through-the-bullshit
communications style that is natural to people who are more concerned about
solving problems than making others feel warm and fuzzy.

http://www.tuxedo.org/~esr/faqs/smart-questions.html

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi


--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi