[rrd-developers] Use of madvise / msync kills performance for me

[Marcus Reid]

On Mon, Jun 30, 2008 at 09:41:13AM +0200, Bernhard Fischer wrote:
> On Fri, Jun 27, 2008 at 01:41:59PM -0700, Marcus Reid wrote:
> >On Fri, Jun 27, 2008 at 01:13:17PM +0200, Bernhard Fischer wrote:
> >> On Fri, Jun 27, 2008 at 02:58:15AM -0700, Marcus Reid wrote:
> >> >
> >> >Matt Dillon provided some good information on this subject that I would
> >> >like to pass on.  First, a little background..  I discovered that the
> >> >long slow msync() calls only happen on files over a certain size.  The
> >> >file that's slowing things down is 1161mB long, and msync() calls to a
> >> >file that's 940mB long are fast.  That's probably a kernel problem that
> >> >could be worth looking into.
> >> 
> >> eh, that's quite big, an order of magnitudes bigger than my files, at
> >> least.
> >> 
> >> >
> >> >This is probably an edge case -- I'm updating an rrd file that's over
> >> >a gig in size and I don't know how common that is.
> >
> >Well, this is starting to feel like beating on a dead horse, but
> >I've been doing some more testing and msync() makes things really
> >slow for smaller files as well.
> >
> >I think the main difference between the platforms is that under
> >FreeBSD, msync() even with MS_ASYNC doesn't return until the data
> >is on the physical disk, whereas with Linux it's just scheduled to
> >make it there at some point.  I'm not sure that this behavior will
> >change due to the OpenGroup spec that Matt mentioned, but I will
> >bring the issue up again.
> 
> SUS requires that an MS_ASYNC and MS_SYNC is available. If your ASYNC
> impl is broken (i.e. behaves as if it was SYNC) then you should really
> fix your VM :)
> 
> Of course we (userspace) can avoid calling msync on __FreeBSD__, but
> that's not exactly the proper thing to do, imo.

Hi Again,

Just writing to say that I agree with you, and that I understand that
avoiding cluttering up your codebase with a bunch of compatibility
stuff is a good thing.

> >I'm just going to file a PR and try and get the msync() call
> >patched out in the ports collection.

The patch that I submitted that takes out the msync() call got
accepted, and things are good for the time being.  I haven't made
another issue of the root of the problem yet -- though it does seem
to me that a big difference in msync() behavior could impact a lot
of things.

As an aside, though I have been using FreeBSD everywhere I can since
2001 or so, I took offense when you referred to it as "your VM."  It's
easy to polarise peoples opinions when you define a common enemy, but
I think we're all working toward the same thing.

Thanks for your time,

Marcus