[rrd-users] If input is already in text format and I craft a perl script to parse the text format and update rrd database, what should the step and heartbeat be?

[Steve Shipway]

Since your first RRA has a count of 1, then 1cdp = 1pdp = 15min.  There are 96 rows, and so (as you say) this RRA has a length of 96x15min = 1 day.

 

Your second RRA has a count of 4, so 1cdp = 4pdp = 1hour.  There are 24 rows, so it also has a length of 1 day (in which case it is likely not necessary).

 

Probably, you want to make these RRA much longer; 2880 rows in the first RRA will be about 30 days long.

 

If your metrics are all coming in the same file, for the same point in time, and all being pushed into the RRD at the same time, then it makes sense to have a single RRD to hold them as in your example.  You would usually use a separate RRD if the data came separately, potentially for different times.  Then separate RRD would make sense as you may get one sample but not another, or they were sampled at differing times.

 

However, in your example, your TapeRate and DASDRate variables have been given a valid range of 0 – 100.  This makes sense if they are a percentage, but not if they are data rates which could be larger than 100B/s.  You may want to make the upper bound on these DS higher or undefined.  Also, if you can extract a data counter rather than a rate, you will get more accurate results overall by using this with a DS type of ‘counter’ and allowing RRDTool to calculate the rate.

 

Steve

 

 

Steve Shipway

s.shipway at auckland.ac.nz

 

From: stevensim at principal-systems.com [mailto:stevensim at principal-systems.com] On Behalf Of Steven Sim
Sent: Tuesday, 10 June 2014 12:57 p.m.
To: Steve Shipway
Cc: rrd-users at lists.oetiker.ch
Subject: Re: [rrd-users] If input is already in text format and I craft a perl script to parse the text format and update rrd database, what should the step and heartbeat be?

 

Steve;

 

Firstly, thanks deeply for replying!

 

I do have the sample times in my file and this can be easily converted to epoch time using perl for each data point.

 

Two more questions;

 

The data measures mainframe CPU Busy and other metrics, much like Solaris prstat or Linux top command with each data point 15 minutes apart.

 

I've created a database like so ...

 

rrdtool create ${RRDBfile} \

                --step 900 \

                --start ${STARTIME} \

                DS:CPUBusy:GAUGE:1800:0:100 \

                RRA:AVERAGE:0.5:1:96 \

                RRA:AVERAGE:0.5:4:24 \

 

The ${STARTIME} variable is calculated as the first sample time minus 900 seconds.

 

The first RRA archive is 1 PDP with 96 CDP or 24 hours. Is my understanding correct?

 

(Since 1 PDP = 15 minutes, 96 CDP will be 15 minutes * 96 or 24 hours)

 

So if I wish to store for a month, then i do

 

RRA:AVERAGE:0.5:1:2880 ??

 

As for the other metrics, like Tape Rate, DASD Rate, can I storage them in a single RRD database or would you recommend I create a database each for each metric?

 

For example, my understanding for a multi metric database creation would be ..

 

rrdtool create ${RRDBfile} \

                --step 900 \

                --start ${STARTIME} \

                DS:CPUBusy:GAUGE:1800:0:100 \

                DS:DASDRate:GAUGE:1800:0:100 \

                DS:TAPERate:GAUGE:1800:0:100 \

                RRA:AVERAGE:0.5:1:96 \

                RRA:AVERAGE:0.5:4:24 \

 

Would the above understanding be correct?

 

Deepest Regards

Steven Sim

On Tue, Jun 10, 2014 at 6:22 AM, Steve Shipway <s.shipway at auckland.ac.nz> wrote:

If the readings are taken at 15min intervals, then your step should be 15min (900s), because this is the step between samples.

 

Generally speaking, unless you have a special case, you should set your heartbeat to be twice your step – in this case, 1800s

 

When you update your RRDtool database, you will give RRDTool a point in time and the data to store at that point in time.  DO NOT use ‘N’ (‘now’) for your point in time as you should be specifying the point in time when the data were sampled, not when you store them.  The step and heartbeat apply to the data point in time, not when you happen to store them.

 

Obviously, you need to have the sample times in your text file.  If you do not have them, you may be able to deduce the time by knowing that they are 15min apart and extrapolating?

 

Steve

 

Steve Shipway

s.shipway at auckland.ac.nz

 

From: rrd-users-bounces+s.shipway=auckland.ac.nz at lists.oetiker.ch [mailto:rrd-users-bounces+s.shipway <mailto:rrd-users-bounces%2Bs.shipway> =auckland.ac.nz at lists.oetiker.ch] On Behalf Of Steven Sim
Sent: Tuesday, 10 June 2014 3:12 a.m.
To: rrd-users at lists.oetiker.ch
Subject: [rrd-users] If input is already in text format and I craft a perl script to parse the text format and update rrd database, what should the step and heartbeat be?

 

Hello;

 

my sensor tech staff has collated the readings onto several text files, each reading 15 minutes apart.

 

I have crafted a Perl script to parse the text file and execute rrdtool update for each reading in the text file.

 

In this case, what should be my step and heartbeat?

 

The actual readings are taken 15 minutes apart, but the actual feeding into the rrdtool database are actually loops through the perl script, each update at most a second or less apart

 

Do i still retain step=15 minutes and heartbeat=15 minutes x 2 ?

 

Deepest Regards
Steven Sim

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20140610/9fc5f0ce/attachment-0001.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5909 bytes
Desc: not available
Url : http://lists.oetiker.ch/pipermail/rrd-users/attachments/20140610/9fc5f0ce/attachment-0001.bin