[rrd-developers] [PATCH,RFC] optional mmap based file I/O

[Bernhard Fischer]

On Thu, May 24, 2007 at 10:34:45PM +0200, Tobias Oetiker wrote:

>> Therefor, i suggest to build the perl wrapper with the CC which we used
>> to build rrdtool. Opinions?
>
>the reason I use $CC from perl is, that perl has its own ideas as to
>what options should be applied when building ... these options may
>be incompatible with the $CC we use ... for example on solaris,
>this is a very common problem ...
>
>I find things work best when I use as much of the module building
>infrastructure the language in question provides ... see python and
>ruby bindings ...

Ok. In the patch mentioned below, we define _GNU_SOURCE generally also
for rrdtool itself, so this problem does not trigger anymore.
>
>> For the most part, the changes are now done¹) (checked FD-based vs. mmap
>> based vs. vanilla 1.2.23-mmap for 'last', 'fetch' and 'dump'), from my
>> POV. This means that the fine-tuning can start (finally, yay! :).
>
>cool ...
>
>regarding mincore ... I think we do not need it at all,
>since we can tell exactly which blocks of an rrd file are 'hot' or
>even better, will be hot when the next update comes in ... so by
>releasing all the non-hot blocks we can be even 'better' than we
>would be by keeping stuff around that had been incore before doing
>a fetch ...
>
>this could even be tuned, only blocks could be kept that will be
>updated within 1 hour or so. this would preserve memory for those
>who are short on cache ... maybe this should be configurable ...
>
>in any event, we must make sure that all hot blocks of the
>highest resolution RRA can stay in core for all rrds regularly
>updated ... only in this way a sensible performance can be reached
>...
>
>I will start a document on this in the wiki ... some of these days
>...

I'm attaching an update which does:

- flag rrd_resize's old file with RRD_COPY
- cleanup error-handling pathes in rrd_update and fix a few typos in
  comments
- rrd_close(): implement printing mincore results for the rrd if
  DEBUG=2 was defined
- rrd_open(): madvise start addresses need to be page-aligned; implement
  alternative path to the fine-grained (i.e. exact) madvise by flagging
  just the first two pages as needed (see TWO_PAGES). Implement
  alternative path that records the last madvise()ed area to avoid
  redundant calls to madvise() on identical areas (due to
  page-alignment constraints) -- see CHECK_MADVISE_OVERLAPS. Implement
  path for USE_DIRECT_IO.
- configure: add check for O_DIRECT flag to open(2). Add option
  --enable-direct-io. Add _GNU_SOURCE to CFLAGS to silence warnings
  about chroot which is marked LEGACY since SUSv2 and is a non POSIX
  extension. Make checks for posix_fadvise() dependant on
  --disable-mmap, since we do not need fadvise for the mmap case.

Please apply.

I did have a look at possible other typos (checking the wrong length of
rrd_read/rrd_write checks like in fetch, that i introduced and which is
already fixed on trunk) and did not find any, FYI.

I did a little performance comparison between the 1.2 branch and trunk
with abovementioned changes. The updatev was against an 1.7MB database,
2 DS, with some values already entered, updating 3 values.
The create was a simple
rrdtool create in.rrd -s 86400 -b 962496000 DS:ds0:GAUGE:86400:U:U RRA:AVERAGE:.5:1:10 RRA:HWPREDICT:10:.5:.1:2
Below are the averages of 4 runs, each consisting of 1000 invocations of
the respective command.
All runs were done on an old, slow 2.5" HDD. The vendor says the disc
has these specs:
Track to Track 2 ms
Average 12 ms
Full Stroke 22 ms
Average Latency 5.6 ms
Rotational Speed 5,400 rpm
The host itself is equipped with a slow 'Intel(R) Pentium(R) M processor'
and has a slow memory bus as well as slow RAM.

..........................updatev......create
1.2.23-svn-r1064..........11m:10s......0m:57s
1.3-r1086_mmap_aligned.....0m:23s.......0m:3s
1.3-r1086_mmap_2pages......0m:23s.......0m:3s
1.3-r1086_fd...............0m:23s.......0m:8s
1.3-r1086_fd-directio......7m:51s.......0m:8s

As you can see, using trunk with mmap()ed I/O certainly has some
advantage over using the 1.2 branch.

If somebody has a workload that is _slower_ on trunk, with mmap()ed I/O
than the current stable branch, i'd like to hear about it, including a
test-case for me to reproduce!

cheers,
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rrdtool-1.2.99-svn.20070529-0944.diff
Type: text/x-diff
Size: 34307 bytes
Desc: not available
Url : http://lists.oetiker.ch/pipermail/rrd-developers/attachments/20070529/d7392de1/attachment-0001.bin