[rrd-users] Timestamp problem

[Mats Gustafsson]

I have now dug a bit deeper into my problems and will try to explain it now.

I have three databases that I write to every second minute, calculate an
average out of every 5 values and have enough rows for 50 years. I have
been running those databases for almost two years and have scripts
reading the data from them with rrdfetch and I also use drraw to publish
diagrams on the web. This has been working fine until a couple of weeks
ago when I did something stupid (don't know what yet) to my server on
which the databases are stored.

The problems I see now...
At the moment drraw always publish the same diagrams, i.e. the data that
drraw reads from my databases are constant and never changes (at least
this is what I think). The same is also what I see when reading with
rrdfetch.

So, I now tried to create a new database that is a copy of one of my old
ones, but instead of having 50 years of data I only store 20 rows.
The databases are created like this (the 50 year one):

#!/bin/sh
# Creates databse with RRDTool for collecting weather data.
# Collects temp, humidity and dewpoint.

STORAGE=/home/mats/server1/mirror/weather_db/
RRDFILE=measure4.rrd

rrdtool create ${STORAGE}/${RRDFILE} \
	--step  120\
	DS:temp3:GAUGE:200:-55:125 \
	DS:humi3:GAUGE:200:0:100 \
	DS:dew3:GAUGE:200:-55:125 \
	DS:spare1:GAUGE:200:U:U \
	DS:spare2:GAUGE:200:U:U \
	DS:spare3:GAUGE:200:U:U \
	DS:spare4:GAUGE:200:U:U \
	RRA:AVERAGE:0.5:5:2628000 \
	RRA:MAX:0.5:30:24 \
	RRA:MIN:0.5:30:24

(The one with 20 rows):

#!/bin/sh
# Creates databse with RRDTool for collecting weather data.
# Collects temp, humidity and dewpoint.

STORAGE=/home/mats/server1/
RRDFILE=measure7.rrd

rrdtool create ${STORAGE}/${RRDFILE} \
	--step  120\
	DS:temp3:GAUGE:200:-55:125 \
	DS:humi3:GAUGE:200:0:100 \
	DS:dew3:GAUGE:200:-55:125 \
	DS:spare1:GAUGE:200:U:U \
	DS:spare2:GAUGE:200:U:U \
	DS:spare3:GAUGE:200:U:U \
	DS:spare4:GAUGE:200:U:U \
	RRA:AVERAGE:0.5:5:20 \
	RRA:MAX:0.5:30:24 \
	RRA:MIN:0.5:30:24


I then let the two databases be updated every second minute for a while
and then did some readings from them:

$ rrdtool fetch server1/mirror/weather_db/measure4.rrd AVERAGE -s -600
temp3               humi3                dew3              spare1
       spare2              spare3              spare4

1235716200: nan nan nan nan nan nan nan
1235716800: nan nan nan nan nan nan nan

$ rrdtool fetch server1/measure7.rrd AVERAGE -s -600
temp3               humi3                dew3              spare1
       spare2              spare3              spare4

1235716200: 2.0437500000e+01 2.8639299787e+01 9.4423393423e-01 nan nan
nan nan
1235716800: nan nan nan nan nan nan nan

The thing here that I don't understand is why the first reading just
gives nan, while my new short database gives reasonable values.

I then did a dump of the databases and looked into the AVERAGE sections.

First the last lines in measure7.rrd AVERAGE section:
<!-- 2009-02-27 07:20:00 CET / 1235715600 --> <row><v> 2.0437500000e+01
</v><v> 2.7959105415e+01 </v><v> 6.7728652180e-01 </v><v> NaN </v><v>NaN
</v><v> NaN </v><v> NaN </v></row>
<!-- 2009-02-27 07:30:00 CET / 1235716200 --> <row><v> 2.0437500000e+01
</v><v> 2.8639299787e+01 </v><v> 9.4423393423e-01 </v><v> NaN </v><v>
NaN </v><v> NaN </v><v> NaN </v></row>
<!-- 2009-02-27 07:40:00 CET / 1235716800 --> <row><v> 2.0499409274e+01
</v><v> 2.8510558860e+01 </v><v> 8.7472457879e-01 </v><v> NaN </v><v>
NaN </v><v> NaN </v><v> NaN </v></row>

Then the last lines in measure4.rrd AVERAGE section:
<!-- 2009-02-27 07:20:00 CET / 1235715600 --> <row><v> 2.0437500000e+01
</v><v> 2.7931009885e+01 </v><v> 6.0425614559e-01 </v><v> NaN </v><v>
NaN </v><v> NaN </v><v> NaN </v></row>
<!-- 2009-02-27 07:30:00 CET / 1235716200 --> <row><v> 2.0437500000e+01
</v><v> 2.8710888934e+01 </v><v> 9.7062691851e-01 </v><v> NaN </v><v>
NaN </v><v> NaN </v><v> NaN </v></row>
<!-- 2009-02-27 07:40:00 CET / 1235716800 --> <row><v> 2.0462100719e+01
</v><v> 2.8759267848e+01 </v><v> 1.0742378724e+00 </v><v> NaN </v><v>
NaN </v><v> NaN </v><v> NaN </v></row>

In principle they look the same. The timestamps etc are the same and the
values are roughly the same.

If I now again try to fetch data from them with specific start and end
times:
$ rrdtool fetch measure7.rrd AVERAGE -s 1235715600 -e 1235716200
                          temp3               humi3                dew3
             spare1              spare2              spare3
 spare4

1235716200: 2.0437500000e+01 2.8639299787e+01 9.4423393423e-01 nan nan
nan nan
1235716800: 2.0499409274e+01 2.8510558860e+01 8.7472457879e-01 nan nan
nan nan

$ rrdtool fetch measure4.rrd AVERAGE -s 1235715600 -e 1235716200
                          temp3               humi3                dew3
             spare1              spare2              spare3
 spare4

1235716200: nan nan nan nan nan nan nan
1235716800: nan nan nan nan nan nan nan

That is, I get the same problem again with just nan from the large database.

I have also looked into the "info", but can't see any major differences
between the two databases.

So, is there anyone who could hint me on why I from two more or less
similar databases get different readings, and from one of them always nan?

Thanks for you support.

/Mats