[rrd-users] RE rrdupdate behaviour when data already exist
Cyrille Bollu
Cyrille.Bollu at fedasil.be
Wed Aug 20 15:22:16 CEST 2008
Hi Sam,
thank you for your answer.
Hereunder you'll find the output of rrdinfo.
Concerning your script, yes please send it to me; I'm planning to write a
script that'll automatically export *all* data from a RRD file into
several XML files, each of them having the best resolution possible within
the largest time window possible. So, maybe your tool will help me in that
goal.
Cyrille
[root at srv-fhq-mgt test-rrd]# rrdtool info 10.0.1.254_con.rrd | sort
ds[ds0].last_ds = "1495"
ds[ds0].max = 1.0000000000e+05
ds[ds0].min = 0.0000000000e+00
ds[ds0].minimal_heartbeat = 600
ds[ds0].type = "GAUGE"
ds[ds0].unknown_sec = 0
ds[ds0].value = 3.4833500000e+05
ds[ds1].last_ds = "1495"
ds[ds1].max = 1.0000000000e+05
ds[ds1].min = 0.0000000000e+00
ds[ds1].minimal_heartbeat = 600
ds[ds1].type = "GAUGE"
ds[ds1].unknown_sec = 0
ds[ds1].value = 3.4833500000e+05
filename = "10.0.1.254_con.rrd"
last_update = 1219224533
rra[0].cdp_prep[0].unknown_datapoints = 0
rra[0].cdp_prep[0].value = 1.6240000000e+03
rra[0].cdp_prep[1].unknown_datapoints = 0
rra[0].cdp_prep[1].value = 1.6240000000e+03
rra[0].cf = "AVERAGE"
rra[0].pdp_per_row = 1
rra[0].rows = 8656
rra[0].xff = 5.0000000000e-01
rra[1].cdp_prep[0].unknown_datapoints = 0
rra[1].cdp_prep[0].value = 1.6780000000e+03
rra[1].cdp_prep[1].unknown_datapoints = 0
rra[1].cdp_prep[1].value = 1.6780000000e+03
rra[1].cf = "AVERAGE"
rra[1].pdp_per_row = 3
rra[1].rows = 35077
rra[1].xff = 5.0000000000e-01
rra[2].cdp_prep[0].unknown_datapoints = 0
rra[2].cdp_prep[0].value = 1.6260000000e+03
rra[2].cdp_prep[1].unknown_datapoints = 0
rra[2].cdp_prep[1].value = 1.6260000000e+03
rra[2].cf = "AVERAGE"
rra[2].pdp_per_row = 12
rra[2].rows = 19113
rra[2].xff = 5.0000000000e-01
rra[3].cdp_prep[0].unknown_datapoints = 0
rra[3].cdp_prep[0].value = 1.6240000000e+03
rra[3].cdp_prep[1].unknown_datapoints = 0
rra[3].cdp_prep[1].value = 1.6240000000e+03
rra[3].cf = "MAX"
rra[3].pdp_per_row = 1
rra[3].rows = 8657
rra[3].xff = 5.0000000000e-01
rra[4].cdp_prep[0].unknown_datapoints = 0
rra[4].cdp_prep[0].value = 1.7870000000e+03
rra[4].cdp_prep[1].unknown_datapoints = 0
rra[4].cdp_prep[1].value = 1.7870000000e+03
rra[4].cf = "MAX"
rra[4].pdp_per_row = 3
rra[4].rows = 35077
rra[4].xff = 5.0000000000e-01
rra[5].cdp_prep[0].unknown_datapoints = 0
rra[5].cdp_prep[0].value = 1.7870000000e+03
rra[5].cdp_prep[1].unknown_datapoints = 0
rra[5].cdp_prep[1].value = 1.7870000000e+03
rra[5].cf = "MAX"
rra[5].pdp_per_row = 12
rra[5].rows = 19113
rra[5].xff = 5.0000000000e-01
rrd_version = "0003"
step = 300
Sam Stickland <sam_mailinglists at spacething.org>
20/08/2008 13:04
A
Cyrille Bollu <Cyrille.Bollu at fedasil.be>
cc
rrd-users at lists.oetiker.ch
Objet
Re: [rrd-users] RE rrdupdate behaviour when data already exist
Hi,
I have a working script that can replay MRTG style RRD files into
another RRD file (with differing RRA definations). It preserves the
average and max DS's by extrapolating the minimum required data for the
rrd updates.
I wrote it for converting MRTG RRD files to Torrus, but it's pretty
general, if a little rough and ready. It uses an RRD file as a template
to produce a new file containing the MRTG data:
# ./convert-to-torrus.pl mrtg.rrd template.rrd out.rrd
If template.rrd file contained data then you can use the merged-rrd.py
script (in contrib/) to combine it with 'out.rrd'.
Let me know if you want a copy now - when I have a little more time it
will be available from the Torrus Wiki.
In answer to your question about growing the RRA - it depends on how you
created the file. If you post the output of "rrdtool info my.rrd | sort"
I can have a look.
Sam
Cyrille Bollu wrote:
>
> Alex,
> Other rrdtool gurus,
>
> I gave a try to my modifications and it's working (almost :-) ) great.
>
> However I still have a concern:
>
> The RRA created don't contain the *exact* amount of rows specified;
> They have 8656 instead of 8640 (1 point per row), 35077 instead of
> 35040 (3 points per row), and, what's ennoying me, 19113 instead of
> 43800 (12 points per row). I understand that this is due to the lack
> of data in the original MRTG log files. But, in the future, I want to
> use the RRD files created to store data in a RRA with 43800 rows/12
> point per row.
>
> So my question is: Will subsequent calls to rrdupdate make the 12
> points/row RRA grow or is it definitivelly limited to 19113 rows?
>
> And, if it is limited, will rrdresize help me?
>
> Side question: Is it possible to add an RRA to a RRD file (using
> rrdtool-1.2)?
>
> Best regards,
>
> Cyrille
>
>
>
>
> *Cyrille Bollu/FHQ/USERS/FEDASIL*
>
> 09/07/2008 17:30
>
>
> A
> Cyrille Bollu <Cyrille.Bollu at fedasil.be>
> cc
> Alex van den Bogaerdt <alex at ergens.op.het.net>,
> rrd-users at lists.oetiker.ch, rrd-users-bounces at lists.oetiker.ch
> Objet
> rrdupdate behaviour when data already exist (was: Tips
and thoughts
> on how to optimally import data points from MRTG .log files into new
> RRD databases)Link
> <
Notes://FHQMAIL01/C1256FB30037352A/E8700555BBB69B0380256C3F003DA467/C65863980A382D55C1257481002F23CC
>
>
>
>
>
>
>
>
>
>
> Great Alex,
>
> It was almost as simple as I thought.
>
> For the record, I modified function "log2rrd" in file MRTG_lib.pm as
> follow:
>
> sub log2rrd ($$$) {
> ...
> my %next;
> my $rrd;
> # CBU 2008.07.09: Change the way MRTG l.log files are converted into
> RRD databases
> # my @steps = qw(300 1800 7200 86400);
> # my %sizes = ( 300 => 600, 1800 => 700, 7200 => 775, 86400 => 797);
> my @steps = qw(300 900 3600);
> my %sizes = ( 300 => 8640, 900 => 35040, 3600 => 43800);
> open R, "<$$cfg{logdir}$$rcfg{'directory'}{$router}$router.log" or
> die "ERROR: opening
> $$cfg{logdir}$$rcfg{'directory'}{$router}$router.log: $!";
> debug('rrd',"converting
> $$cfg{logdir}$$rcfg{'directory'}{$router}$router.log");
> ...
> </ds>
> RRD $first_step{300} = 0; # invalidate
> # CBU 2008.07.09: Change the way MRTG l.log files are converted into
> RRD databases
> addarch(1,'AVERAGE','in','out',\%store,\%first_step,\$rrd);
> addarch(3,'AVERAGE','in','out',\%store,\%first_step,\$rrd);
> addarch(12,'AVERAGE','in','out',\%store,\%first_step,\$rrd);
> addarch(1,'MAX','maxin','maxout',\%store,\%first_step,\$rrd);
> addarch(3,'MAX','maxin','maxout',\%store,\%first_step,\$rrd);
> addarch(12,'MAX','maxin','maxout',\%store,\%first_step,\$rrd);
> # addarch(1,'AVERAGE','in','out',\%store,\%first_step,\$rrd);
> # addarch(6,'AVERAGE','in','out',\%store,\%first_step,\$rrd);
> # addarch(24,'AVERAGE','in','out',\%store,\%first_step,\$rrd);
> # addarch(288,'AVERAGE','in','out',\%store,\%first_step,\$rrd);
> # addarch(6,'MAX','maxin','maxout',\%store,\%first_step,\$rrd);
> # addarch(24,'MAX','maxin','maxout',\%store,\%first_step,\$rrd);
> # addarch(288,'MAX','maxin','maxout',\%store,\%first_step,\$rrd);
> $rrd .= <<RRD;
> </rrd>
> RRD
>
> if ( $OS eq 'NT' ) {
> open (R, "|$$cfg{rrdtool} restore -
> $$cfg{logdir}$$rcfg{'directory'}{$router}$router.rrd");
> ...
> }
>
> Now, I have one additionnal question:
>
> For a few months, we are also collecting data from these sensors via
> another monitoring system using rrdtools.
>
> So, I would like to import data from this other monitoring system into
> the .RRD that I will generate from MRTG (ie: using "rrdtools update").
>
> But, since there will already be some data, I'm wondering what will be
> the behaviour of rrdtools in this case (ie: when one inserts values at
> times when there are already data).
>
> Can you enlighten me?
>
> Best regards,
>
> Cyrille
>
>
>
> *Cyrille Bollu <Cyrille.Bollu at fedasil.be>*
> Envoyé par : rrd-users-bounces at lists.oetiker.ch
>
> 09/07/2008 10:32
>
>
> A
> Alex van den Bogaerdt <alex at ergens.op.het.net>
> cc
> rrd-users at lists.oetiker.ch,
rrd-users-bounces at lists.oetiker.ch
> Objet
> Re: [rrd-users] Tips and thoughts on how to optimally
import data
> points from MRTG .log files into new RRD databases
>
>
>
>
>
>
>
>
>
>
> rrd-users-bounces at lists.oetiker.ch wrote on 09/07/2008 02:25:35:
>
> > > I know that I can convert my MRTG .log file to RRD databases by
> adding the
> > > "LogFormat: rrdtool" line in MRTG's configuration files. However,
> the RRA
> > > generated by doing this don't satisfy me.
> >
> > I don't remember when/where/what, but I'm almost 100% sure I've seen
> > a solution for this. Try google.
> >
>
> I'm pretty sure also but you already gave me a very good tip hereunder
> :-)
>
> >
> >
> > I suggest looking in MRTG or RateUp, find the part where it creates
the
> > new database and modify that part.
> >
>
> Great! I didn't realized MRTG was written in Perl.
>
> Actually, I think I'll have to modify function "log2rrd" in file
> MRTG_lib.pm:
>
> sub log2rrd ($$$) {
> ...
> my @steps = qw(300 1800 7200 86400);
> my %sizes = ( 300 => 600, 1800 => 700, 7200 => 775, 86400
> => 797);
> ...
>
> Am I dreaming? Could it be so simple? :-)
>
> Thanks Alex
>
> Cyrille_______________________________________________
> rrd-users mailing list
> rrd-users at lists.oetiker.ch
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> rrd-users mailing list
> rrd-users at lists.oetiker.ch
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20080820/a28973f8/attachment.html
More information about the rrd-users
mailing list