[rrd-users] If input is already in text format and I craft a perl script to parse the text format and update rrd database, what should the step and heartbeat be?

[Steven Sim]

Steve;

Thanks for your reply!

Just to clarify, a metric like CPU Busy, Number of JOBS etc would be better
off as a COUNTER?

Deepest Regards
Steven Sim

On Tue, Jun 10, 2014 at 9:09 AM, Steve Shipway <s.shipway at auckland.ac.nz>
wrote:

> Since your first RRA has a count of 1, then 1cdp = 1pdp = 15min.  There
> are 96 rows, and so (as you say) this RRA has a length of 96x15min = 1 day.
>
>
>
> Your second RRA has a count of 4, so 1cdp = 4pdp = 1hour.  There are 24
> rows, so it also has a length of 1 day (in which case it is likely not
> necessary).
>
>
>
> Probably, you want to make these RRA much longer; 2880 rows in the first
> RRA will be about 30 days long.
>
>
>
> If your metrics are all coming in the same file, for the same point in
> time, and all being pushed into the RRD at the same time, then it makes
> sense to have a single RRD to hold them as in your example.  You would
> usually use a separate RRD if the data came separately, potentially for
> different times.  Then separate RRD would make sense as you may get one
> sample but not another, or they were sampled at differing times.
>
>
>
> However, in your example, your TapeRate and DASDRate variables have been
> given a valid range of 0 – 100.  This makes sense if they are a percentage,
> but not if they are data rates which could be larger than 100B/s.  You may
> want to make the upper bound on these DS higher or undefined.  Also, if you
> can extract a data counter rather than a rate, you will get more accurate
> results overall by using this with a DS type of ‘counter’ and allowing
> RRDTool to calculate the rate.
>
>
>
> Steve
>
>
>
>
>
> *Steve Shipway*
>
> s.shipway at auckland.ac.nz
>
>
>
> *From:* stevensim at principal-systems.com [mailto:
> stevensim at principal-systems.com] *On Behalf Of *Steven Sim
> *Sent:* Tuesday, 10 June 2014 12:57 p.m.
> *To:* Steve Shipway
> *Cc:* rrd-users at lists.oetiker.ch
> *Subject:* Re: [rrd-users] If input is already in text format and I craft
> a perl script to parse the text format and update rrd database, what should
> the step and heartbeat be?
>
>
>
> Steve;
>
>
>
> Firstly, thanks deeply for replying!
>
>
>
> I do have the sample times in my file and this can be easily converted to
> epoch time using perl for each data point.
>
>
>
> Two more questions;
>
>
>
> The data measures mainframe CPU Busy and other metrics, much like Solaris
> prstat or Linux top command with each data point 15 minutes apart.
>
>
>
> I've created a database like so ...
>
>
>
> rrdtool create ${RRDBfile} \
>
>                 --step 900 \
>
>                 --start ${STARTIME} \
>
>                 DS:CPUBusy:GAUGE:1800:0:100 \
>
>                 RRA:AVERAGE:0.5:1:96 \
>
>                 RRA:AVERAGE:0.5:4:24 \
>
>
>
> The ${STARTIME} variable is calculated as the first sample time minus 900
> seconds.
>
>
>
> The first RRA archive is 1 PDP with 96 CDP or 24 hours. Is my
> understanding correct?
>
>
>
> (Since 1 PDP = 15 minutes, 96 CDP will be 15 minutes * 96 or 24 hours)
>
>
>
> So if I wish to store for a month, then i do
>
>
>
> RRA:AVERAGE:0.5:1:2880 ??
>
>
>
> As for the other metrics, like Tape Rate, DASD Rate, can I storage them in
> a single RRD database or would you recommend I create a database each for
> each metric?
>
>
>
> For example, my understanding for a multi metric database creation would
> be ..
>
>
>
> rrdtool create ${RRDBfile} \
>
>                 --step 900 \
>
>                 --start ${STARTIME} \
>
>                 DS:CPUBusy:GAUGE:1800:0:100 \
>
>                 DS:DASDRate:GAUGE:1800:0:100 \
>
>                 DS:TAPERate:GAUGE:1800:0:100 \
>
>                 RRA:AVERAGE:0.5:1:96 \
>
>                 RRA:AVERAGE:0.5:4:24 \
>
>
>
> Would the above understanding be correct?
>
>
>
> Deepest Regards
>
> Steven Sim
>
> On Tue, Jun 10, 2014 at 6:22 AM, Steve Shipway <s.shipway at auckland.ac.nz>
> wrote:
>
> If the readings are taken at 15min intervals, then your step should be
> 15min (900s), because this is the step between samples.
>
>
>
> Generally speaking, unless you have a special case, you should set your
> heartbeat to be twice your step – in this case, 1800s
>
>
>
> When you update your RRDtool database, you will give RRDTool a point in
> time and the data to store at that point in time.  DO NOT use ‘N’ (‘now’)
> for your point in time as you should be specifying the point in time when
> the data were sampled, not when you store them.  The step and heartbeat
> apply to the data point in time, not when you happen to store them.
>
>
>
> Obviously, you need to have the sample times in your text file.  If you do
> not have them, you may be able to deduce the time by knowing that they are
> 15min apart and extrapolating?
>
>
>
> Steve
>
>
>
> *Steve Shipway*
>
> s.shipway at auckland.ac.nz
>
>
>
> *From:* rrd-users-bounces+s.shipway=auckland.ac.nz at lists.oetiker.ch
> [mailto:rrd-users-bounces+s.shipway=auckland.ac.nz at lists.oetiker.ch] *On
> Behalf Of *Steven Sim
> *Sent:* Tuesday, 10 June 2014 3:12 a.m.
> *To:* rrd-users at lists.oetiker.ch
> *Subject:* [rrd-users] If input is already in text format and I craft a
> perl script to parse the text format and update rrd database, what should
> the step and heartbeat be?
>
>
>
> Hello;
>
>
>
> my sensor tech staff has collated the readings onto several text files,
> each reading 15 minutes apart.
>
>
>
> I have crafted a Perl script to parse the text file and execute rrdtool
> update for each reading in the text file.
>
>
>
> In this case, what should be my step and heartbeat?
>
>
>
> The actual readings are taken 15 minutes apart, but the actual feeding
> into the rrdtool database are actually loops through the perl script, each
> update at most a second or less apart
>
>
>
> Do i still retain step=15 minutes and heartbeat=15 minutes x 2 ?
>
>
>
> Deepest Regards
> Steven Sim
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20140610/df1a40b7/attachment.htm