[rrd-developers] Use of madvise / msync kills performance for me

[Bernhard Fischer]

On Thu, Jun 26, 2008 at 02:28:11PM -0700, Marcus Reid wrote:
>On Thu, Jun 26, 2008 at 07:30:03AM +0200, Tobias Oetiker wrote:
>> Hi Marcus,
>> 
>> Have you tried compiling rrrdtool without mmapping ? Note that
>> removing msync is BAD. Have a look at the manual page.
>> 
>>        msync()  flushes  changes  made  to  the in-core copy of a
>>        file that was mapped into memory using mmap(2) back to disk.
>>        Without use of this call there is no guarantee that changes
>>        are written back before munmap(2) is called.
>
>I think things may be different in FreeBSD land.  From the msync
>man page:
>
>     The msync() system call is obsolete since BSD implements a coherent file
>     system buffer cache.  However, it may be used to associate dirty VM pages
>     with file system buffers and thus cause them to be flushed to physical
>     media sooner rather than later.
>
>> Obviously it will be faster without this call, but then again,
>> the price (potential file corruption) might be a bit high.
>> 
>> Why some of the madvise calls are are taking so long is unclear to
>> me. You might want to try only to drop the WILLNEED calls only and not
>> the RANDOM since it is crucial in preserving cache memory ...
>
>Yeah, that part seems odd to me as well.  I'll try asking about that on the
>right mailing list and find out what some kernel guys think about it.

That's the crucial information, yes.
I wouldn't be surprised if there are some loose ends in your kernel (we
also tripped a timestamp buglet on linux, fwiw :), but that's obviously
pure speculation for now.
>
>> My guess is that the WILLNEED calls actually cause data to be read
>> which saves you time later on, your trace does not give absolute
>> time, it would be interesting to see that too.
>
>I'll let you know if I find out anything interesting.

Just for reference (we shouldn't need to do this), let me point you to
some thoughts about alternative tweaks i had back then:
http://oss.oetiker.ch/rrdtool-trac/changeset/1166/trunk/program/src/rrd_open.c