[rrd-users] If input is already in text format and I craft a perl script to parse the text format and update rrd database, what should the step and heartbeat be?

[Steve Shipway]

I don’t really understand why you think that you’d need to have separate RRD files for the ‘daily’ graphs and the ‘monthly’ one.

 

Why not have a single RRD, with two RRAs – one with 1cdp=1pdp (that you use for your daily graph) and one with a higher granularity – maybe 1cdp=1hour=4pdp – that extends for 62 days (to allow a graph of the last and current calendar months).  When you generate your graph, simply use the start and end params to specify that you’re making a daily or monthly graph, and the correct RRA should be used.  This is how MRTG, Cacti and similar use RRDTool.

 

As you add new data to the RRD, RRDtool will take care of the summarisation and expiry of data.  If you want to archive daily data for longer, then simply extend the first RRA as far as required.

 

Steve

 

Steve Shipway

s.shipway at auckland.ac.nz

 

From: rrd-users-bounces+s.shipway=auckland.ac.nz at lists.oetiker.ch [mailto:rrd-users-bounces+s.shipway=auckland.ac.nz at lists.oetiker.ch] On Behalf Of Steven Sim
Sent: Wednesday, 11 June 2014 3:53 a.m.
To: Simon Hobson
Cc: rrd-users at lists.oetiker.ch
Subject: Re: [rrd-users] If input is already in text format and I craft a perl script to parse the text format and update rrd database, what should the step and heartbeat be?

 

Simon;

 

What if a single data file has data metrics for a single day ... I can easily generate the RRD and Graphs based on this single file single day. (much thanks to Steve Shipway valuable pointers)

 

But at the end of the month, my management would want to review the trend for the ENTIRE month.

 

What would be an elegant practice for the above?

 

to accumulate data from each day onto a single RRD file?

Warmest Regards
Steven Sim

 

On Tue, Jun 10, 2014 at 4:18 PM, Simon Hobson <linux at thehobsons.co.uk> wrote:

Steve Shipway <s.shipway at auckland.ac.nz> wrote:

> If your metrics are all coming in the same file, for the same point in time, and all being pushed into the RRD at the same time, then it makes sense to have a single RRD to hold them as in your example.  You would usually use a separate RRD if the data came separately, potentially for different times.  Then separate RRD would make sense as you may get one sample but not another, or they were sampled at differing times.

To add to that, also consider how things may change over time. Eg, suppose you are logging disk/filesystem utilisation - both in terms of data transferred, and space used/available.

It would make sense to collate all the quantities from one filesystem into a single RRD - so perhaps an RRD with bytes written, bytes read, % space used, $inodes used. But, a machine will almost certainly have multiple filesystems, and more importantly the number may change - so it would make sense to have one RRD/filesystem. Of course, there may well be more than one filesystem on a disk - so you might want to collect stats for the physical disk (probably just bytes read/written) into one RRD, and have a separate RRD for each physical disk (or array) since the number of physical disks/arrays may change (eg if you add a disk because you've run out of space).

There isn't really a right and wrong. It's perfectly OK to have lots of small RRDs with a single DS each. It's also perfectly OK to have fewer RRDs with many DSs each. It's a matter of balancing your requirements with the ability to manage the RRDs - and of course, as mentioned above, the requirement to update all DSs in a single RRD at the same time.

I tend to use a mixture.
At one extreme I have an RRD for our UPS stats with many parameters logged, and another with 508 DSs (data in and out for all 254 usable addresses in our /24 subnet) - in both cases, data is collected and graphed with custom scripts, and all the data is collected in a single operation.
At the other extreme, I have a whole bunch of RRDs with just 2 DSs (data in and out) - one per port for a bunch of switches (the data is collected and graphed with Cacti), and the data for each port is collected separately (it's the way Cacti works).

_______________________________________________
rrd-users mailing list
rrd-users at lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20140610/50009c7e/attachment-0001.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5909 bytes
Desc: not available
Url : http://lists.oetiker.ch/pipermail/rrd-users/attachments/20140610/50009c7e/attachment-0001.bin