[rrd-users] Tips and thoughts on how to optimally import data points from MRTG .log files into new RRD databases

[Cyrille Bollu]

Hi,

We are using MRTG for more than 2 years here and I would like to import 
all the data collected during these 2 years into newly created RRD 
databases.

I know that I can convert my MRTG .log file to RRD databases by adding the 
"LogFormat: rrdtool" line in MRTG's configuration files. However, the RRA 
generated by doing this don't satisfy me.

The RRA generated by adding the "LogFormat: rrdtool" line in MRTG's 
configuration files are as follow:

AVERAGE, time resolution 5 minutes, 559 rows (about 2 days)
AVERAGE, time resolution 30 minutes, 700 rows (about 15 days) 
AVERAGE, time resolution 2 hours, 775 rows (about 65 days)
AVERAGE, time resolution 1 day, 796 rows (about 2 years)
MAX, time resolution 30 minutes, 700 rows
MAX, time resolution 2 hours, 775 rows
MAX, time resolution 1 day, 796 rows

But I would like to have the following:

AVERAGE, time resolution 5 minutes, 8640 rows (1 month)
AVERAGE, time resolution 15 minutes, 35040 rows (1 year)
AVERAGE, time resolution 1 hours, 43800 rows (5 years)
MAX, time resolution 5 minutes, 8640 rows
MAX, time resolution 15 minutes, 35040 rows
MAX, time resolution 1 hours, 43800 rows
MIN, time resolution 5 minutes, 8640 rows
MIN, time resolution 15 minutes, 35040 rows
MIN, time resolution 1 hours, 43800 rows

Does anyone know how to do this?

I'm thinking about doing something as follow:

                 step 1: convert MRTG .log file to .rrd databases
                             ( the "LogFormat: rrdtool" line)
                                              |
                                             V
          step 2a: export .rrd databases generated using AVERAGE CF
                                    (rrdtool xport ...)
                                              |
                                             V
              step 2b: create new rrd databases with correct RRA
                                              |
                                             V
Step 3: update newly created rrd databases with data exported in step 2a
        (custom script to run "rrdtool update" with data from step 2a)

But is it really the best solution? 

The downsides I can see from this process are:

Several exports must be performed from a single MRTG .log file in order to 
have the best time resolution possible (eg: rrdtool xport 
DEF:ds0=inside.rrd:ds0:AVERAGE XPORT:ds0 will give me a time resolution of 
5 minutes wheras rrdtool xport --start "20080101 00:00" 
DEF:ds0=inside.rrd:ds0:AVERAGE XPORT:ds0 will give me a time resolution of 
one day).
Necessary to develop a custom script
loss of the original MAX CF
Possibly lots of UNKNOWN values will be created since time resolution from 
original .log files is much lower than time resolution in new .rrd 
databases

Did anyone already faced the same problems? How did you solved it?

Thanks in advance for your help.

Best regards,

Cyrille
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20080708/409b4714/attachment-0001.html