<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
kevin brintnall wrote:
<blockquote cite="mid:20091014021410.GA34947@rufus.net" type="cite">
<pre wrap="">On Mon, Oct 12, 2009 at 11:33:40AM -0700, Thorsten von Eicken wrote:
</pre>
<blockquote type="cite">
<pre wrap="">Thorsten von Eicken wrote:
- I'm wondering how we could overcome the RRD working set issue. Even
with rrdcached and long cache periods (e.g. I use 1 hour) it seems that
the system comes to a crawl if the RRD working set exceeds memory. One
idea that came to mind is to use the caching in rrdcached to convert
the random small writes that are typical for RRDs to more of a
sequential access pattern. If we could tweak the RRD creation and the
cache write-back algorithm such that RRDs are always accessed in the
same order, and we manage to get the RRDs allocated on disk in that
order, then we could use the cache to essentially do one sweep through
the disk per cache flush period (e.g. per hour in my case). Of course
on-demand flushes and other things would interrupt this sweep, but the
bulk of accesses could end up being more or less sequential. I believe
that doing the cache write-back in a specific order is not too
difficult, what I'm not sure of is how to make it such that the RRD
files get allocated on disk in the that order too. Any thoughts?
</pre>
</blockquote>
<pre wrap=""><!---->
Thorsten,
Read back about a year in the archives. Daniel Pocock
<a class="moz-txt-link-rfc2396E" href="mailto:Daniel.Pocock@barclayscapital.com"><Daniel.Pocock@barclayscapital.com></a> started some work on "striping" RRD
files together.. i.e. you would have one large disk file that contained
multiple RRD databases, where the rows were clustered on disk by time..
I'm not sure how far he got on it.
</pre>
</blockquote>
Yeah, that sounds like a really exciting approach if you have a lot of
time! I believe it's not too difficult if you don't implement RRDtools'
notion of aggregation. But if do... good luck... I was looking for
creative ways to get some disk sweeping without turning RRDtool upside
down. Ha, maybe I should just use a FAT filesystem, that would make it
easy to get a file -> block mapping :-).<br>
TvE<br>
</body>
</html>