[rrd-users] Difficulty getting rrdtool to store "daily" information

[Bill Moran]

In response to Alex van den Bogaerdt <alex at ergens.op.het.net>:

> On Fri, Feb 16, 2007 at 01:34:29PM -0500, Bill Moran wrote:
> 
> > However, the data isn't going in to rrdtool the way I expect, which means
> > there's something I don't understand.
> 
> > The theory is that this will give me discreet values for each "day" (where
> > day is a period between noon and noon of the following day)
> 
> Times are UTC and are a whole multiple of the step size.
> 
> Unless you are living in a time zone 12 hours from Greenwich,
> "day" is most definately not noon to noon.

You lost me here ... I'm not having problems with this portion, I was just
describing the scenario.

> > Then I can get max values to graph monthly and yearly so we have history.
> > 
> > I seem to be hitting a problem getting the values _in_ to the rrd.  Take
> > this chunk of sample data:
> > rrdtool update /var/www/bacula/jobs.rrd 1171083600:68440880988
> 
> 1171083600 is not a whole multiple of 86400.  See my site and
> look for normalization.

You lost me again.  And, unfortunately (for me), your site doesn't appear
to be in english.

Does it make a tremendous difference if the sample is done at an exact
multiple of 86400?  I can't imagine that it does, otherwise a good
portion of our MRTG graphs would be broken as most of them aren't
updated on exact multiples of 300s.

> > Now those e+04, e+03 values are way lower than what I put in.  I'm guessing that
> > somewhere rrdtool is still averaging that data over some period or something,
> > but I'm at a loss as to what I can do to get it in there the way I want it.
> > I thought the MAX declarations would take care of it, but I must still be
> > missing something.
> 
> You're guessing too much.

You're correct.  I realized that and figured it was a good reason to ask
for help.

> Also, I think you need a graphing program,
> not RRDtool.  RRDtool is about rates, bytes per second, not backed up
> data per run.

Well, I am interested in rates.  First off, data per run, _is_ a rate, just
not measured against time.  Additionally, I'm graphing bytes per day, which
is a rate measured against time ...

> Suggestions to think about (not necessarily to implement!):
> * update hourly, not every 24 hours

I'm not sure how this would work.  We need to see total bytes per day
in order to have useful graphs.  If I graph hourly, we then have to do
a bunch of work to either determine which backup jobs were running at
a particular time, or add up all the individual amounts.

The first one is silly ... if I want to see individual jobs, I'll just
read Bacula's logs.  The second one defeats the purpose of doing this.

> * combine start time of backup, end time of backup, amount of
>   data backed up into two updates.  Using the ABSOLUTE counter type
>   and a suitable high heartbeat:
>   - at the start time of the backup, update with 0
>   - at the end time of the backup, update amount of bytes
>   The result is amount of bytes_per_second.  If you use GAUGE
>   instead of ABSOLUTE, then you would fool RRDtool in thinking
>   that 7604606807 bytes per second were backed up.  It will
>   display this number, unless further data manipulation is needed
>   (think normalization and consolidation).  Eventually you will
>   find out that you have fooled yourself, not RRDtool.

Uhh ... on the one hand, changing ABSOLUTE to GAUGE gives me the
results I'm looking for.

On the other hand, I don't see the necessity to update with 0 ever.
Also, that comment about fooling myself sounds rather Zen ...

> * Don't bother using correct times.  If you're only interested in
>   per-day values over a long period of time, then who cares about
>   the few hours difference?  Just update at midnight UTC.  You
>   still need to think about the difference between a rate and a
>   value.  Car analogy: you want 480mi, but you get 60mph during 8 hours.

Hmm ... but if I don't care about the time of the update, why is it
a problem to update at noon?  I'm afraid I'm missing the point of
this comment, and the graphs I get from rrdtool after changing to
GAUGE are the result I'm looking for.

> * Don't use RRDtool, use a drawing program, or openoffice calc, or ...

Yeah.  I don't know how this is going to come across, but if I start
using calc (or any other spreadsheet) I'll have to commit suicide.  One
of the reasons I learned programming is to avoid ever having to use
a spreadsheet again.

Honestly, once I switched ABSOLUTE to GAUGE, rrdtool seems to fit my
needs exactly.  I'm unsure why you're recommending I go elsewhere.
I'm chalking this up to me not fully understanding how ABSOLUTE works.

Thanks for the input.

-- 
Bill Moran
Collaborative Fusion Inc.