[rrd-users] Incorrect MAX total from consolidated cacti graph

[Simon Hobson]

jbhowe <jbhowe at gmail.com> wrote:

> I have a cacti graph which is a consolidation of 34 graphs of online user
> counts. A primary purpose of this graph is to identify peaks in user counts.
> My problem is that the SUM of the MAX values is incorrect unless I constrain
> the timeline of the graph to the minute containing the peak.

You can't add MAX values once data is consolidated - it will give you incorrect results. Eg, take the following sets of data :

100,0,0,0
0,100,0,0
0,0,100,0
0,0,0,100

If you add up the unconsolidated data, you'll get the correct answer that the max values are 100,100,100,100.

But if you consolidate them, you'll get the following max values :
100
100
100
100

and if you then add them up, you get 400.

OK, that's an extreme example, but it shows the problem.

The answer is that you need to do the adding up BEFORE consolidating the data. I wrote a script to do just this (for generating combined mail queue stats across several servers). It's harder than it looks at first glance, but basically I iterate round a loop, find the last update time of the combined stats file, and if all the source files have an update time which is later then I pull down the next set of numbers, add them up, and update the combined file - else I sleep for a while to wait for new data. That bit is fairly easy - but you also need to consider if one (or more) server is down and not updating it's stats - then you need to decide how long you'll wait (in case it's lost connectivity but is still collecting stats) before you ignore it's values and just combine what you do have. And if you get behind for whatever reason (eg the server doing the consolidation is down for maintenance), then you also need to restrict how far back you go so you can always work with unconsolidated data.

If you have a variable number of sources then it's a bit more fiddly still.

Summary: Create a "total users" data file, have a script that fetches new data as it's available (before consolidation) and totals it into this summary file. Then consolidate the data in that summary file.