[rrd-developers] [PATCH,RFC] optional mmap based file I/O
rep.dot.nop at gmail.com
Wed May 30 09:37:04 CEST 2007
On Wed, May 30, 2007 at 09:20:20AM +0200, Tobi Oetiker wrote:
>> Well, I was not sure about this. rrd_info does close the fd early on, so
>> we cannot blindly close the fd in rrd_close. If it is ok for you, we can
>> change rrd_close to close the fd (and adjust the callers accordingly).
>yes I think it this makes sense as rrd_cloes realy imples close ...
>or do you see a risk in this ?
No risk, Sounds fine for me. Will you do this, or should i add it to my
TODO (i'd prefer the former, ATM)?
>> >* why do you flag the first TWO pages ? rrd files with headers
>> > takeing up two pages should be pretty rare ...
>> Fair enough. My small test rrd's all had about 4192 bytes of headers
>> (IIRC), so i thought that using 2 pages should be a good starting point.
>> Easy enough to change, if you are confident that 1 page is enough for
>> the majority.
>hmmm well at the moment your strategy with madvise is not entirely
>transparent to me ... I think we need the following:
>a) tell the OS that we do RANDOM access to prevent readahead while
> accessing the header portion
>b) set sequential access for stuff like rrd_fetch, rrd_resize, rrd_dump
>c) set DONTNEED (after reading/wrting) for all blocks except the
> header and the 'hot' RRA blocks.
I think i have a note from you about the 'hot' RRA blocks. I'll dig it
out and see if i understand what you mean.
>> >* for figuring the pagesize you should use getpagesize is
>> > available.
>> So i vote for using sysconf, and adding checks for
>> _SC_PAGESIZE, _SC_PAGE_SIZE, PAGESIZE, PAGE_SIZE
>ok convinced ... sunos3 (or whatever os changed the page size)
>compatibility is probably not such an issue ...
>> >* if mmap is not used, then it would be cool if posix fadvise was
>> > still called (have not checked the code, just saw that you
>> > removed the check from configure)
>> fadvise is still called for the FD path. Wrapping the check for fadvise
>> in $enable_mmap just makes sure that we do not call fadvise for mmap,
>> but only check for it and eventually use it for the non-mmap path.
I'm just seeing that there is something wrong (performance-wise) with
rrdtool update v.rrd --template traffic_out:traffic_in 1180512000:2013364287020:558938738885
I'm looking at this now. Sounds like it was not a great idea to use
updatev for benchmarking and i should have concentrated more on update ;)
More information about the rrd-developers