[rrd-users] Capturing a noisy count variable (Geiger counter)

Sun Apr 14 02:44:18 CEST 2013

Hi rrdtool users,

I have produced the graphs I wanted, (e.g. see
http://www.chankly.com/geiger/cpm.html) by providing the explicit timestamp
in the update, and exactly timing the probes.  I also changed the datasource
type back to GAUGE.  The interpolation is still a mystery to me, but at
least I can store exact values now:

alan at Titan:/var/apps/geiger$ rrdtool fetch /var/apps/geiger/geiger.rrd
AVERAGE | tail -20

1365898680: 2.1000000000e+01

1365898740: 2.1000000000e+01

1365898800: 1.9000000000e+01

1365898860: 1.8000000000e+01

1365898920: 1.9000000000e+01

1365898980: 3.8000000000e+01

1365899040: 1.9000000000e+01

1365899100: 1.9000000000e+01

1365899160: 2.6000000000e+01

1365899220: 2.9000000000e+01

1365899280: 1.7000000000e+01

1365899340: 2.4000000000e+01

1365899400: 2.7000000000e+01

1365899460: 3.0000000000e+01

1365899520: 2.2000000000e+01

1365899580: 1.6000000000e+01

1365899640: 2.5000000000e+01

1365899700: 2.3000000000e+01

1365899760: 2.7000000000e+01

1365899820: -nan

So, it's not a pressing issue for me now, but is there an explanation of the
interpolation anywhere?

                Thanks, Alan F.

From: Alan Finlay [mailto:alan.websites2 at chankly.com] 
Sent: Sunday, 14 April 2013 2:28 AM
To: 'rrd-users at lists.oetiker.ch'
Subject: RE: Capturing a noisy count variable (Geiger counter)

Hi rrdtool users,

I just sent this to the list, but it is waiting for the moderator to approve
as I incorrectly thought I was a member.  That was lucky!  I just noticed
that I was mistaken about the size of the results I was getting.  They are
too small, not too large.  I was so used to seen +e01, that I didn't notice
the change to -e01 in my latest experiments.  So it seems the five minute
counts are not being accumulated by rrdtool, but averaged as raw values.

However, my main question remains, what is the correct way to update a count
like this?  If I change to datasource type COUNTER, and accumulate the
counts myself, will it be ok to update every 5 seconds?  Is that
recommended, or is there a better way?

                Thanks, Alan F. 

Subject: Capturing a noisy count variable (Geiger counter)

Hi rrdtool users,

I am trying to generate some graphs of background radiation counts per
minute from a Geiger counter. I thought to use rrdtool and learn a bit about
the tool as the same time.

An issue I have is to understand how the interpolation works.  I originally
used a datasource of type GAUGE, and then I found the database was not
storing integral values.  That led to learning about how rrdtool captures
data.  I understood that some form of interpolation was being done, but I
found that even when I timed my captures close to the datasource time period
(i.e. within a few seconds), I was still getting significant averaging that
I could not understand (counts were occasionally well off integral values).
I then tried using a MAX RRA for the primary archive and that also did not
seem to help.  For an example of the effect, see
http://www.chankly.com/geiger_underconstruction/cpm.html

If my graphs contain non integral counts for one minute intervals, then I am
going to need to explain why.

I started reading some forums, I decided to try using and ABSOLUTE
datasource type since that seemed to be more appropriate.  However it did
not seem to change much. Then I considered that since I was actually getting
counts every second from my Geiger counter, and accumulating them for one
minute in my scripts, I could instead update the database more often than
the step interval.  That way, I may not expect to get integral counts, but
they should accurately reflect the "counts per minute" as a gauge.

So I have implemented an update every 5 seconds, and I am getting very
strange results, way too large values.  Is it reasonable to update more
often than the step interval?

If not, then what is a recommended way to accurately capture these counts?
What I really want to be sure of is that the integral of the displayed graph
should accurately reflect the count for any chosen interval, i.e. avoiding
any systemic error.  The longer I capture the more accurate my averages
should be.  The backround CPM is quite stable for long intervals, and I want
to look for trends related to sun activity, seasons etc.  So I need to be
confident the long timeframe averages reported by rrdtool are correct.

Here is an extract of the db, these values are much higher than they should
be:

$ rrdtool fetch /var/apps/geiger/geiger.rrd AVERAGE | tail -20

1365867360: 3.5410117577e-01

1365867420: 3.1358877058e-01

1365867480: 3.7223225156e-01

1365867540: 3.2656777052e-01

1365867600: 4.9940582148e-01

1365867660: 3.9069307467e-01

1365867720: 2.6239790085e-01

1365867780: 4.2388944703e-01

1365867840: 5.0398253216e-01

1365867900: 3.2639678663e-01

1365867960: 4.4485142178e-01

1365868020: 2.9939793410e-01

1365868080: 3.6032755674e-01

1365868140: 2.8273224979e-01

1365868200: 2.4941790093e-01

1365868260: 2.7300076071e-01

1365868320: 4.3303353491e-01

1365868380: 2.3303745712e-01

1365868440: 3.0171077034e-01

1365868500: -nan

Here is the perl code I am using to update the db, in a loop every 5
seconds:

            my $timestamp = `date +%s`;

            chomp $timestamp;

            print LOG "time=$timestamp, returned count = $count \n";

            my $result = `rrdtool updatev /var/apps/geiger/geiger.rrd
N:$count`;

            print LOG "updatev returned: $result";

And here is a section of the log, and it is clearly evident something is
wrong with the counts (the average CPM is around 21).

updatev returned: return_value = 0

time=1365866940, returned count = 1

updatev returned: return_value = 0

[1365866940]RRA[AVERAGE][1]DS[geiger] = 2.4979012130e-01

time=1365866945, returned count = 2

updatev returned: return_value = 0

time=1365866950, returned count = 3

updatev returned: return_value = 0

time=1365866955, returned count = 5

updatev returned: return_value = 0

time=1365866960, returned count = 0

updatev returned: return_value = 0

time=1365866965, returned count = 2

updatev returned: return_value = 0

time=1365866970, returned count = 1

updatev returned: return_value = 0

time=1365866975, returned count = 7

updatev returned: return_value = 0

time=1365866980, returned count = 6

updatev returned: return_value = 0

time=1365866985, returned count = 3

updatev returned: return_value = 0

time=1365866990, returned count = 1

updatev returned: return_value = 0

time=1365866995, returned count = 3

updatev returned: return_value = 0

time=1365867000, returned count = 1

updatev returned: return_value = 0

[1365867000]RRA[AVERAGE][1]DS[geiger] = 5.6636103674e-01

time=1365867005, returned count = 0

updatev returned: return_value = 0

time=1365867010, returned count = 0

updatev returned: return_value = 0

time=1365867015, returned count = 3

updatev returned: return_value = 0

time=1365867020, returned count = 2

updatev returned: return_value = 0

saving incomplete line <CPS, 1, >

time=1365867025, returned count = 1

updatev returned: return_value = 0

time=1365867030, returned count = 1

updatev returned: return_value = 0

time=1365867035, returned count = 2

updatev returned: return_value = 0

time=1365867040, returned count = 1

updatev returned: return_value = 0

time=1365867045, returned count = 5

updatev returned: return_value = 0

time=1365867050, returned count = 2

updatev returned: return_value = 0

time=1365867055, returned count = 2

updatev returned: return_value = 0

time=1365867060, returned count = 3

updatev returned: return_value = 0

[1365867060]RRA[AVERAGE][1]DS[geiger] = 3.6438774029e-01

time=1365867065, returned count = 1

updatev returned: return_value = 0

time=1365867070, returned count = 1

updatev returned: return_value = 0

time=1365867075, returned count = 4

updatev returned: return_value = 0

time=1365867080, returned count = 1

updatev returned: return_value = 0

time=1365867085, returned count = 1

updatev returned: return_value = 0

time=1365867090, returned count = 1

updatev returned: return_value = 0

time=1365867095, returned count = 0

updatev returned: return_value = 0

time=1365867100, returned count = 1

updatev returned: return_value = 0

time=1365867105, returned count = 1

updatev returned: return_value = 0

time=1365867110, returned count = 1

updatev returned: return_value = 0

time=1365867115, returned count = 1

updatev returned: return_value = 0

time=1365867120, returned count = 0

updatev returned: return_value = 0

[1365867120]RRA[AVERAGE][1]DS[geiger] = 2.1964405340e-01

The db definition is as follows:

rrdtool create geiger.rrd -s 60 DS:geiger:ABSOLUTE:100:0:U \

RRA:AVERAGE:0.5:1:50000 \

RRA:MIN:0.5:60:8800 \

RRA:MAX:0.5:60:8800 \

RRA:AVERAGE:0.5:60:8800 \

RRA:MIN:0.5:1440:7305 \

RRA:MAX:0.5:1440:7305 \

RRA:AVERAGE:0.5:1440:7305

                Alan F.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20130414/fcf433d7/attachment-0001.htm 