[rrd-users] Deleting old data from an archive

[Shack Toms]

I'd like to set all the saved values to undefined for a particular RRA
within an RRD.   What's going on is that the data associated with the DS
turns out to be invalid and I want to start supplying good data, and I don't
want the old bad data to appear, mixed with the good.   There are lots of
other DS's that have good data in them, and I don't want to lose that
either.

I thought that an easy way to do this would be to dump the database to xml,
edit the xml, and then restore the database.   Then all I would need to
write would be the xml filter to go in the middle, to change some values to
NaN.  This works well on small databases, but on any database over a few
tens of MB it seems to take a *very* long time to do the restore (even if I
don't edit the xml, so don't blame it on that).   The dump goes reasonably
fast.   A restore seems go go fairly fast for small databases, but slows
down for large ones.   It looks like the rrd_restore code operates by trying
to work on everything in memory and then writing out the file, so I'm
wondering if my problem is excessive paging.   On my machine restore starts
doing something like 25,000 page faults per second.  I notice that the
rrd_dump code seems to read and dump the values incrementally, so it ought
to (and seems to) have a lot better paging performance.

Maybe I would be better off doing some kind of a record by record update,
since updates seem to go relatively quickly.   The problem there is that I
think I would need to create an inverse for my consolodation function so
that I could provide data that would re-create the RRA's.   I would have to
trust a lot of cleverness to be sure I didn't corrupt the RRA's.

Alternatively, I could try to change the behavior of restore to work
incrementally through the data, the way dump does (and submit the results if
it improved things).   Is there a FAQ on how to build/submit changes, so
that I fit in with the way things are done?  Is there a reason why an
incremental restore of the rows won't work/is a bad idea?   Am I going about
this all wrong?

One more small point.   I wanted to make sure that the dump/restore
operation worked, so I did a dump/restore/dump and compared the xml from the
first dump with the xml from the second.  The only difference was that I
noticed that the parameter "unknown_sec" changes to 0 for all the DS's,
although looking at the code it is not immediately obvious to me why this
is.   I don't see a lot of documentation for "unknown_sec", and I wondered
how bad this would turn out to be and how best to keep the old, non-zero
value.

Thanks,
Shack Toms


-- Attached file removed by Listar and put at URL below --
-- Type: application/ms-tnef
-- Size: 2k (2844 bytes)
-- URL : http://www.ee.ethz.ch/~slist/pantomime/73-winmail.dat


--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi