[rrd-developers] RRDTool: baselining, where to start ?

[arne.vandamme at pandora.be]

First off, I'm new to the list so I'd like to give a big greetings to everyone.  I've only been using RRDTool a couple of months, but like many others I think it's a great app.

I'm in my final year of my studies computer engineering (Belgium) and as a 'thesis' (actually I can't come up with the best word in English at the moment).  I'm building a webbased network monitoring portal on the Java platform, making extensive use of rrdtool.  An extremely important aspect of the monitoring is the so-called baselining, basically this is just an average over a period of time.

Example: we have the traffic of a tuesday, we want to compare that traffic with the tuesday baseline.  This can be the average traffic of X tuesdays BEFORE this tuesday.  That way we can see how much we deviate from the baseline 'today', but the baseline is also quite handy in longterm monitoring, to see how traffic evolves (always more traffic, traffic level doesn't fluctuate that much etc).  In some cases a baseline can be 'day-bound' (traffic on a monday is de facto higher for example, we should only use a monday baseline then, as not to 'influence' the statistics too much), in other cases we just want for example a general day baseline, since every day is pretty much alike (take an online ftp server for example, apart from maybe the weekends, most days will show pretty much the same behaviour).

Few, a lot of text :-) Now to the point:
I can calculate a baseline without any real problems, I simply fire up an rrdtool process, do some fetches, calculate some averages, map them to the correct time and put them in a new rrd database.  I have perfectly graph-able and query-able data, in different rrd databases.  

The time mapping is the most important issue ofcourse, it always comes down to taking values of certain points in time and mapping the averages back on a specific time.  I could probably graph using overlay, but by mapping them on the 'now' i can easily work with all the data in a graph command.

Ofcourse, the way i do it now is quite 'inefficient'.  Maybe not for the calculation of a single baseline, but when it comes down to calculating lost of baselines the cost will be moderately high.  In comes the question if it is a thing that can be implemented by RRDTool itself.

Some quick thinking led me to believe it could be done, even without huge problems (again, i thought 'quickly' :p).  Do we want a command that allows you to calculate a specific baseline and map it to a certain time and then returns the new tuples (would be very handy), or do we want some kind of new 'datatype' that keeps a baseline of a certain format for a certain related DS, and updates the baseline when a new datapoint gets inserted (seems to me this would ALSO be very handy)... or do we want both?  The we is mostly me in this case, unless other people find this an interesting thing.

In comes the 'how'.  I'm not that bad a programmer myself, but I haven't even looked at the RRDTool code yet *g*, I'll download the CVS snapshot asap, unfortunately I'm unable to do that right now, so I don't know what information is available in it (so please excuse any stupid questions that have already been answered somewhere).

- I work on Windows 2000 mainly, I know RRDTool is crossplatform, but is it in any way possible or easy to compile the CVS on win ?  If so, any preferred way ?  I am a complete newbie when it comes to trying to compile Linux C++ on Windows :-)  If it really is a big bother, then I'll just switch to Linux for the programming I suppose, everything i've developed is cross platform, but I'm just not that big a fan of Linux desktops ;p
- Is there any good documentation available about what the code does, and how it does it ?  Or is this mainly a 'read the bloody code' thing ?
- Where's the information about new functionality added to the development code ?
- Is there any 'release' date for the latest development code, as to when it will make final.  Since my project is due may/june 2003, it might be important to know before I'd try to even use ANY of the new features :-)

Anyways, I hope I didn't bore you too much with all this.  Feel free to share ideas and give comments, and I hope I'll be able to contribute in some way to the core of RRDTool.  I also developed some handy apps & scripts for basic stats monitoring and will put them on Sourceforge soon, not that they're great or special, but as said, they might be handy for some people. 

regards,
Arne

--
Unsubscribe mailto:rrd-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-developers-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-developers
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi