[rrd-users] Adding aberrant detection to existing RRD files

[Eduardo M. Bragatto]

Hi there..

I'm new to the list but I've been using rrdtool for sometime now and I'd 
like to say that I really looked all around before coming here with my 
question.

Recently I decided to start using the aberrant behavior detection on a 
few network graphs just as a test. Once I got familiar with the feature, 
I decided that it was time to implement it on all my current graphs.

As you guys know, to use aberrant behavior detection its necessary to 
have a few extra RRAs.

My problem is that I do not want to lose my past data by simply creating 
new rrd files and using them without my past data.

My current RRD files have RRAs with 5min step for 40days, so I can see 
past graph with daily resolution without having to be concerned about 
RRD consolidation. I'm only interested about that RRA, the others (30min 
average and daily average) don't matter and I'm not worried about not 
having them after migration. I just wanna get the past 40days with 5min 
step.

Since my graphs are pretty simple (only in/out counters from the routers 
nics), I created a script to parse "rrdtool fetch" taking "rrdtool 
lastupdate" in account. Basically, it takes the last counter value and 
subtracts past values so I can get the value of the counter when each 
past value was calculated.

Its output is like this:

1195485900:74335472553937:10931651301159
1195486200:74335473981897:10931651790798
1195486500:74335475467931:10931652437662
1195486800:74335477065359:10931652884268

Once it was done, I thought I could simply create my new rrd files with 
--start in the past and use rrdtool update to insert those values into 
the new files.

When I first saw the graphs from the new rrd files, I was very happy to 
see the very same graph from my live system (from where I fetched the 
data). But, after a closer look, I noticed that all values were way 
lower than on the original graph, although they both have the same 
perfect shape.

If I fetch the data from old rrd file, that's what I get:

1195485900: 1.4279600000e+06 4.8963927558e+05
1195486200: 1.4860344651e+06 6.4686484990e+05
1195486500: 1.5974283122e+06 4.4660653507e+05
1195486800: 1.4744286246e+06 6.8138033515e+05

But, If I fetch from the new one:

1195485900: nan nan
1195486200: 4.7598666667e+03 1.6321300000e+03
1195486500: 4.9534466667e+03 2.1562133333e+03
1195486800: 5.3247600000e+03 1.4886866667e+03

The first "nan" is expected, as I can only calculate the counter for the 
second time frame (by subtracting the counter from the from the first 
one). But as you can see, the values are way lower.. And if you compare 
to what I put on the graph:

1195485900:74335472553937:10931651301159
1195486200:74335473981897:10931651790798

74335473981897 - 74335472553937 = 1427960

1427960 is exactly what's on my original rrd file.
So, why the entry 1195486200 has 4759 instead?

The proportion is being kept, as the new graphs have the exact same 
shape as the old ones, but it seems that rrd is reducing all values by a 
certain number of times.

Does anyone knows why this kind of behavior is happening?

Or does anyone knows how to add the aberrant behavior detection RRAs 
without recreating the rrd files?

If I manage to get that working, I'd be glad to submit the script to the 
crontrib repository as I believe that would be useful to a lot of other 
people.

Thankfully,
Eduardo M. Bragatto.