[mrtg] TrafGrapher - interactive multiple MRTG data in one graph

[Ján ONDREJ]

Hello,

On Wed, Apr 29, 2015 at 09:49:33AM +0000, Steve Shipway wrote:
> So, if I understand you correctly, then TrafGrapher is a complete
> replacement for MRTG (replacing the collector, storage and web frontend),
> though it does have a built-in migration path (it will migrate an MRTG
> .log datafile).

Yes and no. TrafGrapher has ability to use multiple data sources.  It can
use native MRTG log files and get interfaces from indexmaker HTML files.

It also can use internal json index files with it's own log files or combine
them. For TrafGrapher does not matter, if log file is created with MRTG
or with it's own collector.

For SAN storage monitoring, it can use data from IBM StorWize (directly
their data, no need to convert them) or data from EMC CX4-120.

Other data sources can be added in future.

> So, since it uses its own data storage backend (text files, similar but
> not identical to Native MRTG files) it will not use RRDTool, and you've
> addressed the MRG Native-mode IO performance issues with your own
> collection/storage engine.  Data aggregation is only performed once per
> day, rather than on each update in a heirarchical mode as MRTG and RRDTool
> do, by summarising a file that is incrementally appended to.

Yes, but this is only one mode, how it can work. :-)

> My first query would be on how the graphing performance handles on the fly
> graph generation efficiently, since you'll have to do a lot more
> on-the-fly aggregation than MRTG/RRD do due to your aggregation being done
> only once per day.

Graphing is done on client side. Ne need to process these files on server
and client's PC/browser will process only required data and only when it
needs it, so no need to process large number of data every 5 minutes on
server. I tested TrafGrapher with my switches, I have aprox. 100 switches on
one stats server. Displaying all of them in one graph requires lot's of RAM
(because it's not possible to clear browser cache after processing of log
file), but displaying hundred interfaces is not a problem. It depends on
client momry and CPU, but 1000-2000 interfaces with well and are processed
in a few seconds.

> Also, although you are only appending a single log line, you could find
> the log file grows rather large by the end of the day as opposed to the
> static size of MRTG/RRD (which minimise their IO by memory-mapped IO and
> writing just the updated bytes, not the whole file).

I can add ability to aggregate log files in customizable intervals.
But I think, this is not a big problem.

> You will also have a larger file, as you're storing in a text format
> rather than binary; however, that would certainly make the data files more
> portable between architectures, which is one of the problems with RRD
> files.
> 
> If you want to add support for RRDTool, there are a large number of
> libraries available to let you access the data with simple function calls. 
> You can always extract the data locally and still send the data as text to
> the javascript client, of course.

The main idea of TrafGrapher is to do not require server side CGI scripts.
Data are collected by standard applications and published on web,
then processed in browsers javascript.

> Best of luck with it all; I would maybe suggest that you concentrate on
> the UI and collector, and maybe use RRDTool as the storage - since it
> would be hard to beat the efficiency of the RRD engine, but both MRTG and
> the currently available web interfaces to it are a bit dated.

Thank you. For now I have limited free time to enhance it. It's pushed
public as opensource and waiting for more developers.
I can add RRD support, but I don't use RRD, so need some example data,
but I will add it only if there will be more users requiring this feature.

							SAL