[rrd-users] Counter resetting causing huge spikes

[Alex van den Bogaerdt]

On Sun, Sep 09, 2007 at 09:32:25AM +0100, Piers Kittel wrote:

> I'm currently measuring my electricity usage using an counter - each  
> time the meter flashes 1/800kWh, the counter value is put in the  
> database.  As it's a 32 bit integer, I've set the known limits to be  
> 0 - 4,294,967,297 in the database.  Thing is, if the counter resets  
> to zero due to the sensor recording program restarting, the database  
> assumes I've gone all the way to 4,294,967,296 and then back again,  
> causing a huge spike showing I've used 228GW in 1 min.

Indeed, it does.  Expected and documented behaviour.
You did understand that part.  But you did not understand the limit
you put in: you told RRDtool that a rate (not a counter value) of
2^32 is OK.  That's 2^32 flashes a second...

>                                                         Now all of my  
> graphs all the way up to the yearly graph shows that spike, and it's  
> likely I'm not able to see anything until a year later - is there a  
> way to prevent this?  Should I not put in a known upper limit, but  
> rather, putting in an "U" - will this fix it?  Am I able to remove  
> the errant data out of the database, or will I need to re-create the  
> database?

In that one minute you had a huge rate. This rate is included in
all the RRAs because it was within the limits.  If you set an unknown
upper limit, the same will happen.

What you need is a limit which is much lower.  What is the most you
will ever encounter under the most exrtreme circumstances?  It could
be 64A and 230V, which means 14720 W, which means 14720 J/s.

14720 Joule / 3600000 Joule per kilowatthour = 0.0040888888 kWh.
Multiply by 800 to get flashes per second: 3.3 (rounded up).

This should be you maximum rate, provided that I did not make mistakes
in my calculations or reasoning.  Check it !!!


But all this is just a workaround.  A real solution is to let
rrdtool know you've started again.  RRDtool should forget about
history, because you don't know what happened in the past.
Update the database with rate U at the moment your script starts.
Update the database with rate zero immediately after. This way
you tell rrdtool that you don't know the counter value (this
counter value that used to count your flashes before restarting).
You also tell rrdtool that you do know the new current value,
which is zero.  And then you can continue counting again.

Another viable way is to let your script handle all the intelligent
stuff and come up with a rate.  Then monitor a GAUGE in rrdtool.


Spikes can be removed.

Set a new upper limit.                   man rrdtune
Dump the database                        man rrddump
Restore the database, using the -r flag  man rrdrestore


Tip: write your data not just to rrdtool but also to disk. If
needed, you can recreate your database including input data.
This is handy if you are going to setup an entire new way of
monitoring, or if you want to remove a mistake, and so on.
Just log the timestamp when these pulses arrive.  This enables
you to have a virtual pulse detector so to speak.


HTH
-- 
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/