[rrd-users] Separating data by time of day

[Alex van den Bogaerdt]

On Sat, Nov 24, 2007 at 10:32:40PM +0000, Matthew Booth wrote:
> I collect in and out volume data from my router continuously at 15
> minute intervals. I use this to graph when I use bandwidth, and how
> much. I currently graph daily, weekly and monthly totals, as well as
> rate graphs.

OK.

> My ISP has the concept of a 'peak' period between 8am and 6pm. I would
> like to separate my daily, weekly and monthly totals into bandwidth used
> in the peak period, and bandwidth used outside that time. The following
> RPN zeroes all data points outside of that time period:
> 
> LTIME,86400,%,DUP,28800,GT,EXC,64800,LE,+,2,EQ,in,0,IF

(I had: "Probably so (didn't check).". But I changed my mind!)

> LTIME,86400,%,DUP,28800,GT,EXC,64800,LE,+,2,EQ,in,0,IF

local datetime, only time part, duplicate
stack: time,time
check the copy, see if it's > 28800 (which is 8am)
result: boolean (0 or 1)
stack: time,result
swap
stack: result,time
check the time, see if it's <= 64800 (which 6pm)
result: boolean (0 or 1)
stack: result,result
add the two booleans
result: 0 or 1 or 2
compare to two
if true, return "in" else return zero.

This is fine. It works for any day, even for more days in the same DEF
or CDEF.

Now you have data during a 10-hour period. An average computed from
this however is still a division by the total amount of time on this
graph.

You may want to check what happens if you push UN instead of zero.

bash script to copy or for inspiration:

---cut here---
RRD=/path/to/your/rrdtool

ST=1195945200

if [ ! -f test.rrd ]
then
        $RRD create test.rrd --step 60 --start $((ST-60)) DS:x:GAUGE:120:U:U RRA:AVERAGE:0:1:100

        i=0
        while [ $i -lt 100 ]
        do
                $RRD update test.rrd $((ST+i*60)):$i
                i=$((i+1))
        done
        fi

$RRD graph test.png --start $ST --end start+6000 --width 400 \
        DEF:x=test.rrd:x:AVERAGE \
        CDEF:x1=LTIME,86400,%,DUP,600,GT,EXC,1800,LE,+,2,EQ,x,0,IF \
        CDEF:x2=LTIME,86400,%,DUP,600,GT,EXC,1800,LE,+,2,EQ,x,UNKN,IF \
        LINE1:x1#FF0000 \
        GPRINT:x1:AVERAGE:%6.2lf \
        GPRINT:x2:AVERAGE:%6.2lf

---cut here---



> My problem is that I can't find a way to use this to create totals over
> a wider period than a single step. If I want to create a daily total,
> then I can't just use a step size of 1 day and multiply by 86400,
> because the time of that coalesced data point will be midnight, and I'll
> get a zero for the whole day. If I use a smaller step size, I can't find
> a way to aggregate multiple CDEF data points to create a daily total.

I think what you are saying here is that you can't do what you want with
larger intervals, such as a year worth of data. I do not see why you would
get zero at midnight, unless you use LAST as your CF. Are you in a time
zone which does not differ from UTC? If not, you also should be prepared
for the fact that RRDtool uses midnight UTC, not midnight your timezone.

But indeed, for graphs which show more than 2 hours per pixel column,
you won't be able to separate the data anymore.

Why not do literally what you want:
"I would like to separate my daily, weekly and monthly totals into
 bandwidth used in the peak period, and bandwidth used outside that time.
"

Two RRDs, update both RRDs at 08:00 and at 18:00, update only one of
the two on other times. Set heartbeat no larger than 35999 and let
RRDtool do its stuff.

Upto and including 08:00 is updated in the first RRD. At 18:00 this
RRD is updated again, doing:
1: filling 08:00 upto 18:00 with "Unkn"
2: setting the counter start value at 18:00

between 18:00 and 08:00 next day, only the 2nd RRD is updated.

At 08:00 when you update both RRDs, filling 18:00 upto 08:00 with Unkn
and setting the counter start value for the first RRD.


Ofcourse you are keeping your current RRD during this experiment, and
perhaps it is also wise to keep a copy of your input.

HTH
-- 
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/