[rrd-developers] mmap considerations

[Tobias Oetiker]

Hi Bernhard,

the time bit is trecherouse ... I HAVE todo other stuff ... alas
... but I sneek out every now and then ...

the reason rrd_create is 'critical' is not performance as such, but
cache pollution ... by creating just a few rrd files cache gets
thrown out of whak quite badly .. I have been running my
performance.pl script which creates a bunch of rrd files and then
updates them ... I found that the memory system takes quite a long
time to figure which data it should keep and which it should drop
... a single rrd_create can have quite a long lasting negative
effect ...

I assume the kernel prefers keeping a whole file in memory as to
just a few scattered blocks in many files ... (which makes sense in
general, just not for rrdtool)

the test case shows that the mmap code goes to about 20k updates
per second when stuff is in cache while 1.2.24dev goes to 12k
updates per second ...

I have not quite figured out the logic behind it all ...

if you care to play as well, I have attached my testing code ...

fincore to see which blocks are cached
fadont to drop all non dirty blocks
performance.pl my performance test

regarding the graphing part, the only bit where it accesses the
filesystem directly is when writing out the png/pdf/svn/... file
the rest of the interaction goes through rrd_fetch ...

not sure about dropping the png file ... this only makes sense for
people who create many pngs and not on demand ...

dropping rrd_fetch (except the hot blocks) does make sense in my opinnion
since it stops cache pollution ... should be configurable for cases
where the user knows that he is going to re-read the same rrd file
twice in a row ...

cheers
tobi


Today Bernhard Fischer wrote:

> On Tue, Jun 12, 2007 at 11:20:39PM +0200, Tobias Oetiker wrote:
> >Hi Bernhard,
> [I'm changing the subject a little bit; Please feel free to move this
> discussion to the list, if you prefer. Discussing this a little bit in
> private mode is of course fine with me. Nice that you seem to have a
> little bit time to work again on rrdtool :)]
> >
> >looking at the new code, I find that configure does not check for
> >posix_fadvise if mmap code is active ...
> >
> >the effect is that in rrd_create no fdatasync and dontneed happens,
> >which makes rrd_create faster but also evicts all previously cached
> >data quite effectively ...
> >
> >is there a reason to not check for posixs fadvise if mmaping is
> >active ?
>
> My way of thinking is that for mmap we generally do not want to fadvise
> but madvise, where possible. It is correct that rrd_create in it's
> current incarnation should use fadvise (since it doesn't [yet] use
> mmap).
>
> I suggest we do one of these:
> 1) rewrite rrd_create to not use filps (FILE*) but FD/mmap based I/O
>    Up until now i didn't implement this since in my POV rrd_create is
>    not really performance critical.
>    The advantage is that potentially this would use less memory than the
>    current implementation.
>    open the new file, with O_CREAT (see rrd_resize as an example how to
>    rrd_open() a new file in the new accessor impl).
>    This is IMHO the preferred way to go but a bit more work than the
>    alternative below.
> 2) leave rrd_create alone and check for fadvise also for the mmap case.
>    Change all HAVE_POSIX_FADVISE in sources which provide new
>    accessor-methods to not be called. The net effect is that
>    a) not updated functions (create,graph come to mind) use fadvise
>    b) updated functions (update, resize, etc) do *not* call fadvise but
>    use only madvise.
>
> thoughts?
> PS: while i think that rrd_create is not too performance critical,
> rrd_graph certainly is since it is potentially called very often (for
> obvious reasons, i.e. users :). So, while switching rrd_create to the
> nwe accessor functions would be nice to have, updating rrd_graph is
> overall more benefical, i assume.
>
> cheers,
> Bernhard
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
http://it.oetiker.ch tobi at oetiker.ch ++41 62 213 9902
-------------- next part --------------
A non-text attachment was scrubbed...
Name: faddont.c
Type: text/x-csrc
Size: 233 bytes
Desc: 
Url : http://lists.oetiker.ch/pipermail/rrd-developers/attachments/20070613/6e83c10e/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fincore.c
Type: text/x-csrc
Size: 953 bytes
Desc: 
Url : http://lists.oetiker.ch/pipermail/rrd-developers/attachments/20070613/6e83c10e/attachment-0001.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: perftest.pl
Type: text/x-perl
Size: 5127 bytes
Desc: 
Url : http://lists.oetiker.ch/pipermail/rrd-developers/attachments/20070613/6e83c10e/attachment-0002.bin