[rrd-users] Error in database?

[Mats Gustafsson]

Karl Fischer wrote:
> Mats Gustafsson wrote:
>> Karl, I think you are right :-)
>> I have been investigating this problem for over one month now, so I can
>> tell you that your help is really appreciated :-)
>>
>> I just tried with three databases: one that is 30 years long, one that
>> is 20 rows long and one that is 50 years long. And in accordance with
>> your prediction the two first are ok while the third one fails.
>>
>> I guess that this explains why my databases all of a sudden stopped
>> working one month ago. It must have been when the new rrdtool version
>> was released in Gentoo and I updated to it.
>>
>> My issue now is that I have been updating four databases (50 years long
>> each) with information for almost two years now, and I'd rather not
>> throw away all that data.
>>
>> I see three possibilities:
>> 1. Re-arrange my old databases and reduce them to 30 years length.
>> 2. Wait for a new release of rrdtool that behaves in the same way as the
>> old version.
>> 3. Create new databases and throw away all my collected data.
>>
>> What would you recommend?
> 
> Mats,
> 
> I'd dump the the databases and filter out the invalid rows ...
> 
> I quickly tried something that seems to work:
> 
> First I just eliminated the negative timestamps (leaving everything since 1970),
> but my rrdtool complains when trying to access anything before 1980:
> 
> $ rrdtool fetch new.rrd AVERAGE --start 1/1/70
> ERROR: the first entry to fetch should be after 1980
> 
> So I decided to remove the 1970s as well which leaves you with ~29 years.
> (BTW: do you *really* believe you will run that system for 30+ years??)
> 
> rrdtool dump measure10.rrd \
> | pcregrep -v '<!-- 19[567]\d-\d+-\d+ \d+:\d+:\d+ \w+ / -?\d+ -->' > new
> rrdtool restore new new.rrd
> 
> did work for me - YMMV
> 
> hth
> 
> - Karl
> 
Hi Karl,

before I received your proposal I tried to shrink the databases using
rrdtool resize. This seems to work just fine (knock on wood).
Could you see any disadvantage with this method?
I resized them to 30 years of data. And, no I don't know if I will need
that amount of data. I am collecting weather information, and maybe I
will be able to keep them running for that long periond, but...

Best regards, Mats