[rrd-developers] rrdtool 1.3 update slowness

[Bernhard Fischer]

On Tue, Aug 28, 2007 at 04:28:13PM -0700, Evan Miller wrote:
>Disabling MMAP in 1.3 makes things worse:
>
>Create     10 rrds      1 c/s (0.00427 sdv)   Update     10 rrds   22975 u/s (0.00013 sdv)
>Create     10 rrds      1 c/s (0.00428 sdv)   Update     20 rrds   19045 u/s (0.00087 sdv)
>Create     20 rrds      1 c/s (0.00826 sdv)   Update     40 rrds   12433 u/s (0.00205 sdv)
>Create     40 rrds      5 c/s (0.03187 sdv)   Update     80 rrds      15 u/s (0.06627 sdv)
>Create     80 rrds      1 c/s (0.03359 sdv)   Update    160 rrds      15 u/s (0.06745 sdv)
>Create    160 rrds      5 c/s (0.15173 sdv)   Update    320 rrds      13 u/s (0.11210 sdv)
>
>I know very little about IO, so I have no idea what's going on. Let me know if
>there are other tests I can run, or if anyone can reproduce the behavior.
>Perftest is here:
>
>http://oss.oetiker.ch/rrdtool/pub/contrib/perftest.zip

perftest went into the repository a long time ago and is in examples/perftest.pl

I cannot really reproduce your report. I see:
perftest, 20k RRDs (100 per dir):

rrdtool-1.2.r1195:
100000    104 Up/s (0.05685 sdv)
100000   3126 Up/s (0.00428 sdv)
100000  12629 Up/s (0.00503 sdv)
100000  10722 Up/s (0.00997 sdv)
100000  11445 Up/s (0.00611 sdv)
100000   8878 Up/s (0.01648 sdv)
100000  11101 Up/s (0.00326 sdv)
100000   3965 Up/s (0.06022 sdv)
100000   3326 Up/s (0.07576 sdv)
100000   3374 Up/s (0.05249 sdv)
100000   3066 Up/s (0.08344 sdv)

rrdtool-1.3.r1196:
100000  19937 Up/s (0.00001 sdv)
100000  21399 Up/s (0.00002 sdv)
100000  22701 Up/s (0.00001 sdv)
100000  22742 Up/s (0.00001 sdv)
100000     60 Up/s (0.10694 sdv)
100000  21996 Up/s (0.00001 sdv)
100000  22448 Up/s (0.00002 sdv)
100000  21023 Up/s (0.00001 sdv)
100000  21835 Up/s (0.00001 sdv)
100000  22515 Up/s (0.00001 sdv)
100000  18455 Up/s (0.00001 sdv)

Where a very heavy I/O job was running for some time during the "60
Up/s" timeframe.

PS: Tobi recently changed perftest on trunk, i used the version from
trunk revision 1115 for the numbers above. rrdtool itself were the
revisions indicated above.
>
>Thanks,
>
>Evan
>
>On Tue, Aug 28, 2007 at 04:17:47PM -0700, Evan Miller wrote:
>> I ran Tobi's perftest.pl against RRDtool 1.2 and the current subversion 
>> checkout. Updating a single RRD is noticeably faster under 1.3, but 
>> performance grinds to a halt on my machine once I get up to 100 or so 
>> RRDs. I consistently produce results similar to these:
>> 
>> 
>> RRDtool 1.2:
>> 
>> Create     10 rrds      1 c/s (0.00117 sdv)   Update     10 rrds   23288 
>> u/s (0.00002 sdv)
>> Create     10 rrds      1 c/s (0.00062 sdv)   Update     20 rrds   21125 
>> u/s (0.00065 sdv)
>> Create     20 rrds      1 c/s (0.00125 sdv)   Update     40 rrds   23385 
>> u/s (0.00002 sdv)
>> Create     40 rrds      1 c/s (0.00259 sdv)   Update     80 rrds   23150 
>> u/s (0.00005 sdv)
>> Create     80 rrds      3 c/s (0.41032 sdv)   Update    160 rrds   22688 
>> u/s (0.00006 sdv)
>> Create    160 rrds     10 c/s (0.02412 sdv)   Update    320 rrds   19940 
>> u/s (0.00124 sdv)
>> Create    320 rrds      5 c/s (0.02459 sdv)   Update    640 rrds   17601 
>> u/s (0.00142 sdv)
>> Create    640 rrds      3 c/s (0.04289 sdv)   Update   1280 rrds   10012 
>> u/s (0.00432 sdv)
>> Create   1280 rrds      1 c/s (0.35445 sdv)   Update   2560 rrds    4294 
>> u/s (0.00614 sdv)
>> Create    768 rrds      1 c/s (0.05230 sdv)   Update   3328 rrds    3626 
>> u/s (0.00785 sdv)
>> 
>> 
>> RRDtool 1.3:
>> 
>> Create     10 rrds      1 c/s (0.00771 sdv)   Update     10 rrds   31434 
>> u/s (0.00007 sdv)
>> Create     10 rrds      1 c/s (0.00421 sdv)   Update     20 rrds   29388 
>> u/s (0.00031 sdv)
>> Create     20 rrds      1 c/s (0.00937 sdv)   Update     40 rrds   24646 
>> u/s (0.00068 sdv)
>> Create     40 rrds      3 c/s (0.40489 sdv)   Update     80 rrds   16121 
>> u/s (0.00132 sdv)
>> Create     80 rrds      5 c/s (0.04052 sdv)   Update    160 rrds    2000 
>> u/s (0.00519 sdv)
>> Create    160 rrds      2 c/s (0.12109 sdv)   Update    320 rrds      33 
>> u/s (0.02963 sdv)
>> 
>> $ uname -a
>> Linux yossarian 2.6.20-16-generic #2 SMP Thu Jun 7 20:19:32 UTC 2007 
>> i686 GNU/Linux
>> 
>> I didn't configure with any special options. Ideas?
>> 
>> Evan
>> 
>> _______________________________________________
>> rrd-developers mailing list
>> rrd-developers at lists.oetiker.ch
>> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers
>
>_______________________________________________
>rrd-developers mailing list
>rrd-developers at lists.oetiker.ch
>https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers
>