[rrd-users] Difficulty getting rrdtool to store "daily" information

[Bill Moran]

I've got most of rrdtool figured out, but I'm missing something.  The
problem I'm trying to solve is we want to monitor how much data is backed
up via our Bacula system at each job run.  Since our jobs run overnight,
any system out there that graphs data on a "daily" basis splits the job
and gives us inaccurate results.

I thought, this is why rrdtool exists ...

So I formulated some SQL queries that give me the total bytes backed up
between noon today and noon yesterday -- this will fit into a shell script
nicely.

However, the data isn't going in to rrdtool the way I expect, which means
there's something I don't understand.

I'm creating my rrd with the following command:
rrdtool create /var/www/bacula/jobs.rrd \
 --start 1151515500 --step 86400 \
 DS:bytes:ABSOLUTE:100000:0:100000000000000 \
 RRA:MAX:0.5:1:20 \
 RRA:MAX:0.5:7:8 \
 RRA:MAX:0.5:4:18

The theory is that this will give me discreet values for each "day" (where
day is a period between noon and noon of the following day)

Then I can get max values to graph monthly and yearly so we have history.

I seem to be hitting a problem getting the values _in_ to the rrd.  Take
this chunk of sample data:
rrdtool update /var/www/bacula/jobs.rrd 1171083600:68440880988
rrdtool update /var/www/bacula/jobs.rrd 1171170000:384966685
rrdtool update /var/www/bacula/jobs.rrd 1171256400:7604606807
rrdtool update /var/www/bacula/jobs.rrd 1171342800:5511113163
rrdtool update /var/www/bacula/jobs.rrd 1171429200:7160839755

Now, an XML dump of the rrd shows me _very_ different data than what I would
expect: (trimmed for clarity, let me know if I need to provide the whole
thing)

                <cf> MAX </cf>
                <pdp_per_row> 1 </pdp_per_row> <!-- 86400 seconds -->

                <params>
                <xff> 5.0000000000e-01 </xff>
                </params>
                <cdp_prep>
                        <ds>
                        <primary_value> 7.8902161053e+04 </primary_value>
                        <secondary_value> NaN </secondary_value>
                        <value> NaN </value>
                        <unknown_datapoints> 0 </unknown_datapoints>
                        </ds>
                </cdp_prep>
                <database>
                        <!-- 2007-01-25 19:00:00 EST / 1169769600 --> <row><v> NaN </v></row>
                        <!-- 2007-01-26 19:00:00 EST / 1169856000 --> <row><v> NaN </v></row>
                        <!-- 2007-01-27 19:00:00 EST / 1169942400 --> <row><v> NaN </v></row>
                        <!-- 2007-01-28 19:00:00 EST / 1170028800 --> <row><v> NaN </v></row>
                        <!-- 2007-01-29 19:00:00 EST / 1170115200 --> <row><v> NaN </v></row>
                        <!-- 2007-01-30 19:00:00 EST / 1170201600 --> <row><v> NaN </v></row>
                        <!-- 2007-01-31 19:00:00 EST / 1170288000 --> <row><v> NaN </v></row>
                        <!-- 2007-02-01 19:00:00 EST / 1170374400 --> <row><v> NaN </v></row>
                        <!-- 2007-02-02 19:00:00 EST / 1170460800 --> <row><v> NaN </v></row>
                        <!-- 2007-02-03 19:00:00 EST / 1170547200 --> <row><v> NaN </v></row>
                        <!-- 2007-02-04 19:00:00 EST / 1170633600 --> <row><v> NaN </v></row>
                        <!-- 2007-02-05 19:00:00 EST / 1170720000 --> <row><v> NaN </v></row>
                        <!-- 2007-02-06 19:00:00 EST / 1170806400 --> <row><v> NaN </v></row>
                        <!-- 2007-02-07 19:00:00 EST / 1170892800 --> <row><v> NaN </v></row>
                        <!-- 2007-02-08 19:00:00 EST / 1170979200 --> <row><v> NaN </v></row>
                        <!-- 2007-02-09 19:00:00 EST / 1171065600 --> <row><v> NaN </v></row>
                        <!-- 2007-02-10 19:00:00 EST / 1171152000 --> <row><v> 4.4556329282e+03 </v></row>
                        <!-- 2007-02-11 19:00:00 EST / 1171238400 --> <row><v> 7.0607813830e+04 </v></row>
                        <!-- 2007-02-12 19:00:00 EST / 1171324800 --> <row><v> 6.8834000835e+04 </v></row>
                        <!-- 2007-02-13 19:00:00 EST / 1171411200 --> <row><v> 7.8902161053e+04 </v></row>
                </database>


Now those e+04, e+03 values are way lower than what I put in.  I'm guessing that
somewhere rrdtool is still averaging that data over some period or something,
but I'm at a loss as to what I can do to get it in there the way I want it.
I thought the MAX declarations would take care of it, but I must still be
missing something.

Anyway, help is greatly appreciated.

-- 
Bill Moran
Collaborative Fusion Inc.