[rrd-users] RRD graph/xport performance

[Ryan Kubica]

Tobi,

I'm curious what benchmarks/tracing you do during development to localize performance bottlenecks in rrdtool graphing.

The legend does slow graph generation (23ms for a standard graph no-legend for me, 34ms with the legend added) -- worse on long legend graphs; which is one reason I implemented legends in datatables (via JSON) for sizable legend lengths (50 +) instead of having rrdtool draw the legend ( plus of course datatables is a better format for that sort of thing, but I digress. )

What I'm really curious about is this during a graph/xport creation of 25,000 (25K) datasources rrdtool (perl RRDs) spends considerable time doing this:

mremap(0x7fe0454df000, 56258560, 56262656, MREMAP_MAYMOVE) = 0x7fe0454df000
mremap(0x7fe0454df000, 56262656, 56266752, MREMAP_MAYMOVE) = 0x7fe0454df000
mremap(0x7fe0454df000, 56266752, 56270848, MREMAP_MAYMOVE) = 0x7fe0454df000
mremap(0x7fe0454df000, 56270848, 56274944, MREMAP_MAYMOVE) = 0x7fe0454df000
mremap(0x7fe0454df000, 56274944, 56279040, MREMAP_MAYMOVE) = 0x7fe0454df000
mremap(0x7fe0454df000, 56279040, 56283136, MREMAP_MAYMOVE) = 0x7fe0454df000
mremap(0x7fe0454df000, 56283136, 56287232, MREMAP_MAYMOVE) = 0x7fe0454df000
mremap(0x7fe0454df000, 56287232, 56291328, MREMAP_MAYMOVE) = 0x7fe0454df000
mremap(0x7fe0454df000, 56291328, 56295424, MREMAP_MAYMOVE) = 0x7fe0454df000
mremap(0x7fe0454df000, 56295424, 56299520, MREMAP_MAYMOVE) = 0x7fe0454df000
mremap(0x7fe0454df000, 56299520, 56303616, MREMAP_MAYMOVE) = 0x7fe0454df000


... granted the performance of what it's doing is still very good (99 seconds for the graph on 25k datasources for 3 hours of 1 minute data == ~45k datapoints calculated per second), it certainly seems to have substantially decreasing performance (more than linear) as datasources increase for a request.  (note: I say very good in comparison to other numerical data export systems.)

Other tests (less datasources longer time frame) have datapoints per second in the millions:

5,000 datasources same time-frame/data takes 5.48 seconds == ~164k datapoints calculated per second.
100 datasources 90 day time-frame 5 minute data, 1.32 seconds == 1.9 million datapoints calculated per second.

Oddly enough the above mremap()'s happen before files are even opened and read.  I've strace'd httpd during a request like this and it spends all it's time doing mremap's then very quickly reads all datasources and produces the graph... more than a minute mremap'ing then seconds open/read/close'ing data files and producing the graph.

I mention it happens with xports because I have the dual purpose of requiring the above type of merge (it's sum() and rpn math for many datasources) to be quick.

It's as if the rrdtool syntax parser is very slow or rrdtool spends a lot of time up front creating a 'workspace' in memory to calculate with and constantly resizes memory?

Any tips on how to trace down why the above occurs so I can work-on/suggest a fix would be helpful.

Thanks,
-Ryan


________________________________
 From: Tobias Oetiker <tobi at oetiker.ch>
To: Christoph Anton Mitterer <calestyo at scientia.net> 
Cc: rrd-users at lists.oetiker.ch 
Sent: Sunday, August 12, 2012 12:07 AM
Subject: Re: [rrd-users] RRA/RRD tuning
 
... snip...

the biggest win for graphing time would be to integrate support for
setting the png compression level into rrdtool ... libcairo sets
this very high, and thus spends considerable time compressing the
resulting png file ... setting the compression level to 1 should
give a considerable performance gain ...

I will be glad to integrate your patch

cheers
tobi
>
> Thanks for your insights,
> Chris.
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900

_______________________________________________
rrd-users mailing list
rrd-users at lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-users/attachments/20120812/5d91aca0/attachment.htm