[rrd-users] convert rrd into new rrd with different RRAs

[Tobias Oetiker]

Today rrd-users at ficos.de wrote:

> Tobias Oetiker schrieb:
> > Hi Stefan,
> >
> > there is no such program to that does this as fahr as I can tell
> > ... the simples way to achieve this is to write an app that pulls
> > the data from the original rrd, sorting the updates by time picking
> > the higest resolution available for each instant and then feeding
> > it back to a newly setup rrd ... it will get trickier if you want
> > to support MAX RRAs as well, but you get the idea ?
> >
> > cheers
> > tobi
>
> I've tried this aproach before, but with large rrds this takes forever,
> because it always feeds the finest resolution first and then runs through
> all aggregation steps. I have rrds with values for years, so they're really
> large ...
>
> Now lets say I'm sorting the updates by time picking the higest resolution
> available, that highest resolution might be 10m (after 2 weeks in that given
> example), but I'm feeding those 10m values into the new rrd which has a 2s
> step size, so it has to go through all the aggregation steps for all DPs.

have you actually tried it ? it should not take all that long since
it will all happen in memory ...

> A smarter solution might be: create multiple new rrds, one for each wanted
> rra-size, and then feed each of the new rrds with the resolution suitable for
> it's rra-size, eg. not the highest, but the lowest that is still higher than
> the new wanted.
> After having done that, combine the multiple rrds into one rrd with multiple
> rras. That is much faster, but much more complicated :-(
>
> It get's even more trickier if you need to keep track on 'gaps' in the data,
> because the system was offline for a while. The gap might be large enough to
> trigger the heartbeat/xff for the first rra, but not for the others ...
> That makes it even more difficult ...
>
> I have postponed this project to some long dark cold winter nights ... :-/

yes ... in that case it might be best to resample from the
available data directly into the target RRAs maybe into the xml
format and then restore this ... I have written such code for mrtg
when it converts the old mrtg.log files to rrdtool.

cheers
tobi

>
> - Karl
>
>
>
>
>
>
>
> > Today Stefan M. Brandl wrote:
> >
> > > On Mon, Mar 23, 2009 at 07:32:07PM +0100, Karl Fischer wrote:
> > > > Hi,
> > > >
> > > Hello Karl,
> > >
> > > > I'm re-posting this since I didn't get a single reply - any ideas?
> > > >
> > > > I'm trying to convert many existing rrds into new rrds with different
> > > > RRAs.
> > > >
> > > > so for example I might have rrd's created with:
> > > > (RRAs: 1s, 10s, 60s, 600s)
> > > >
> > > >   --step=1
> > > >   DS:somevalue:GAUGE:120:0:U
> > > >   RRA:AVERAGE:0.5:1:3600    # 1s for 1h
> > > >   RRA:AVERAGE:0.5:10:8640   # 10s for 1d
> > > >   RRA:AVERAGE:0.5:60:10080  # 60s for 1w
> > > >   RRA:AVERAGE:0.5:600:4320  # 10m for 1m
> > > >
> > > > and I'd like to convert this into a new rdd that looks like this:
> > > > (RRAs: 2s, 10s, 60s, 900s)
> > > >
> > > >   --step=2
> > > >   DS:somevalue:GAUGE:60:0:U
> > > >   RRA:AVERAGE:0.5:1:5400    # 2s for 3h
> > > >   RRA:AVERAGE:0.5:10:8640   # 10s for 1d
> > > >   RRA:AVERAGE:0.5:30:20160  # 60s for 2w
> > > >   RRA:AVERAGE:0.5:300:12960 # 10m for 3m
> > > >
> > > > Of course that means that some data needs to be calculated from higher
> > > > RRAs, for example:
> > > >
> > > > the first half of the 2s RRA in the new database can be taken 1:1 from
> > > > the
> > > > 1s RRA of the exiting database by using the CF on two datapoints each
> > > > time.
> > > > The second half is more tricky since there is no data in 1s resolution
> > > > in
> > > > the first RRD, so the values need to be interpolated from the 10s RRA
> > > > ...
> > > >
> > > > I wonder if I have to re-invent the wheel and to write the required code
> > > > completely myself or if someone has been there already ... ?
> > > >
> > > > Any hints welcome ...
> > > >
> > > As I did not see any hints to the mailing list, I wanted to
> > > ask if there are any news on this topic?
> > >
> > >
> > > Stefan
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900