[rrd-users] Re: Anomaly detection with RRD + Can RRDtool graph two datasources with independant auto-scaling?

[Cooper Nelson]

Abberant  behavior detection is implemented in the current development 
release of RRDtool.

I've been experimenting with it, after some considerable study I was 
able to get it to succesfully detect anomalies.  For example, I created 
a RRD of the UPS temp of a remote site that had some overheating 
problems in the last year.  RRDtool correctly marked the time period 
when the overheating began as a failure.  Interestingly it also marked 
when the system returned to a normal temp. as a failure (by then the 
Holt-Winter algo.  had adapted to the high temp state as being 
"normal").  Even more interesting was that it flagged the a few points 
preceding the actual overheating (which looked fine to my eyes) as being 
abnormal/failures.  After the system returned to normal no failures were 
reported at all.

Some problems I ran into:.

1.  Setting the sensitivity of the failure detection has to be done when 
the RRD is created.  This makes testing and tweaking a bit cumbersome.

2.  Even with the failure thresholds set to the highest settings (28 
straight confidence band violations before marking a failure) it was 
still too sensitive for some of the data sources I was working with. 
 This maybe be my fault though, as I still dont understand many of the 
variables of the H-W prediction model.

3.  I would love to be able to simultaneous graph data sources and the 
degree of their anomalous behavior, with red areas to mark failures.  So 
far I havent been able to do this, DEVPREDICT doesnt look right to me 
when I graph it, and since failures are marked as a zero or 1 they dont 
make much of an impression when graphed with other data sources.  One 
approach Im looking at this week is logging all confidence band 
violations to an RRD (1 or 0 for each 5 minute data point) and then 
computing the total failures per hour.  The idea being to create a sort 
of  "anomaly rating" for each hour of data collection.

My biggest question right now is, it is possible to graph two 
datasources simultaneously and have the y-axis auto-scale independantly 
for each source?  If anyone knows the answer I would greatly appreciate it.

Cheers,
Cooper

Stanley Hopcroft wrote:

>Dear Ladies and Gentlmen,
>
>About two years ago if I understand correctly, the Cricket/WebTV people
>modified rrdtool to support anomaly detection with a new RRA called
>HWPREDICT (Holt Winters time series prediction).
>
>Please would you let me know of
>
>. any experience with that version of rrdtool - does it actually
>usefully detect anomalies ?
>
>. if the modifcations are going to be incorporated into rrdtool ?
>
>Thank you,
>
>Yours sincerely.
>
>
>--
>------------------------------------------------------------------------
>Stanley Hopcroft
>------------------------------------------------------------------------
>
>'...No man is an island, entire of itself; every man is a piece of the
>continent, a part of the main. If a clod be washed away by the sea,
>Europe is the less, as well as if a promontory were, as well as if a
>manor of thy friend's or of thine own were. Any man's death diminishes
>me, because I am involved in mankind; and therefore never send to know
>for whom the bell tolls; it tolls for thee...'
>
>from Meditation 17, J Donne.
>
>--
>Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
>Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
>Archive     http://www.ee.ethz.ch/~slist/rrd-users
>WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi
>
>  
>

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi