[rrd-users] Consolidation Happening even through I don't want it to!!

[O'Neil, Donald A.]

>> I've been running Cacti for quite some time now, and used RRDTool
v1.2.15 all the way through 1.3.7, which is what we're currently using
with Cacti.

>This discussion, at least if it is cacti-dependant, can be continued at
the cacti forum if need be.

I opened up a message on the cacti forum, but haven't gotten any
resolution, which is why I posted here too.

>> When I setup up RRA files that have NO consolidation, I'm still
seeming to get consolidation in the graphs. 

>It may not actually be getting consolidated, but maybe it's the way the
graphing algorithms 'squish' 8900 rows into a 500 pixel wide graph...
>There's a good chance that your assumption is correct. This is what I
called "graphical consolidation" at http://docs.cacti.net.
>The "raw data" is still there but cannot be displayed. You may define a
consolidation function for this as well (but cacti currently accepts
only >the default = not yet implemented).
>When zooming into the graph, the required granularity will show up as
soon as you have "enough pixels" to display each data point.
>For a better answer, the current rrd file definitions should be listed
here.

Here's my RRD defs:

Hourly (5 Minute Average)  1  300  14400    
Daily (5 Minute Average)  1  600  86400    
Weekly (5 Minute Average)  1  2100  604800    
Monthly (5 Minute Average)  1  9000  2678400    
Yearly (5 Minute Average)  1  110000  33053184  

>> I'm thinking there could be 1 of 2 things going on... cacti isn't
calling RRDTool correctly, so the graphs are being consolidated, or
RRDTool is >>interpolating the data, and mis-calculating the actual
peak/average, as well as graphing it incorrectly, maybe due to a
simplistic approach to >>making the data 'fit'. I dig through all the
cacti code and it appears to be sending all the raw data to RRDtool,
otherwise I'd expect the export >>to not have all the data points.

>Find the call to rrdtool in lib/rrd.php to proove this.

Actually I dig through rrd.php and pretty much disproved that cacti
wasn't calling RRDtool correctly, it is indeed. I didn't see anything in
the function that could be causing this, so I posted here.

>Find the generated command when visiting "Graph Management" and switch
to DEBUG. You may run this command from cli. I'm quite sure this will
>exactly match your graph ;-)

It does... but is still doesn't match the real data.

>> If I plot the data in Excel, the graph represents the true data, with
all the peaks preserved.

> How do you force excel to draw this amount of data with 500 pixels
only?

Excel automatically graphs whatever size you want the graph to be, and
anti-aliases/scales things appropriately. I don't know how it does it,
just that if I graph the data in Excel it looks MUCH different than in
RRDtool...

The main point is that the MAX and MIN values in the raw data do NOT
match the calculated MAX and MIN values on the graph. It appears as
though RRDTool calc's these max/mins from its own 'squished' data.

I think the issue here is how RRDtool interprets so many data points to
create a graph that is only 500 pixels wide. I would think rather than
averaging the data it should take the absolute value of the max as the
point it plots, rather than taking an average of a number of points and
plotting it. I would bet this is how Excel does it.