<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:times new roman, new york, times, serif;font-size:12pt"><DIV></DIV>
<DIV>Hi Dave </DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
<DIV>
<P style="MARGIN-BOTTOM: 0in">Could you please highlight your RRD structure for 300,000 targets? I assume this is mainly ports activity and you have hundreds ports per switch or you referring to some other systems (more complicated). Also, how many metrics per RRD do you have? How many different templates are you using? What is physical size of your RRD archive? Are you split your archive between disks, or all files on same disks and in single location. I'm doing study on infrastructure with 500,000+ targets for at least four groups of target systems (UNIX, Linux, NT and Network). Because of infrastructure size I'm planning to use Z/Linux with SuSe 9.0. Any recommendations?</P>
<P style="MARGIN-BOTTOM: 0in"><BR></P>
<P style="MARGIN-BOTTOM: 0in">And, I have second group of questions. Because of large size of your infrastructure (300,000+ is a lot) are you doing any forecasting (aka capacity planning), or any calculations for performance/activity prediction. As example, based on data that was collected for last tree months I would like to produce capacity planning chart for next tree months. Any idea how I can implement linear/nonlinear regression analysis with ignoring all extremes and outliers. I can extend research that will be helpful for community, but at this time would like to hear a comments on my direction.</P>
<P style="MARGIN-BOTTOM: 0in"> </P>
<P style="MARGIN-BOTTOM: 0in">Thanks</P>
<P style="MARGIN-BOTTOM: 0in">Val Shiro</P></DIV>
<DIV><BR><BR> </DIV>
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">
<DIV style="FONT-SIZE: 12pt; FONT-FAMILY: times new roman, new york, times, serif">----- Original Message ----<BR>From: Dave Plonka <plonka@doit.wisc.edu><BR>To: rrd-developers@lists.oetiker.ch<BR>Cc: Dale Carder <dwcarder@doit.wisc.edu>; Archit Gupta <archit@cs.wisc.edu><BR>Sent: Wednesday, May 16, 2007 7:43:50 AM<BR>Subject: [rrd-developers] patch/tuning for very large RRD systems (was "Re: SQL Backend request")<BR><BR>
<DIV>Hi Tobi,<BR><BR>This is one of those serendipitous occasions -<BR>I think we can save you perhaps lots of time:<BR><BR>Archit Gupta, Dale Carder, and I have just comleted a research project<BR>on RRD performance and scalability. I beleive we may have the largest<BR>single system MRTG w/RRD - over 300,000 targets and RRD files updated<BR>every five minutes.<BR><BR>On Wed, May 16, 2007 at 10:00:12AM +0200, Tobias Oetiker wrote:<BR>> <BR>> I am still gathering data ... but it seems all to come down to<BR>> <BR>> * file system cache pollution through other processes<BR>> * and the time you give the system to deal with dirty bufffers<BR>> * the block queuse size may also play a role.<BR>> * I think we could gain quite a lot by using fadvise in rrdtool to<BR>> only keep the header portion of the file in cache.<BR>> will try this later today ...<BR><BR>Over the past month, we've done a posix_fadvise RANDOM patch
and<BR>completely evaluated the perforance impacts. It's really good<BR>(obviously). What it does is get both the readahead and page faults<BR>under control, in Linux at least.<BR><BR>For others observing performance issues, the pertinent things are:<BR><BR>* What operating system and version are you running?<BR> (to determine initial file readahead and availablity of fadvise syscall)<BR><BR>* What is your hardware, including physical memory?<BR> to determine CPU available and max buffer-cache size<BR><BR>* What is your update interval (rrd step) and RRD file definitions (RRAs)?<BR> (e.g. typical MRTG, or output of rrdtool info.)<BR><BR> This determines the page fault characteristics of RRD files<BR> when buffer-cache is scarce.<BR><BR>* What version of rrdtool?<BR> (for applying patch)<BR><BR>With that we can determine if it's possible to update a givent number<BR>of RRD files, or
how to properly (re)size it.<BR><BR>Dave<BR><BR>-- <BR>plonka@doit.wisc.edu <A href="http://net.doit.wisc.edu/~plonka/" target=_blank rel=nofollow>http://net.doit.wisc.edu/~plonka/</A> Madison, WI</DIV></DIV><BR></DIV></DIV><BR>
<HR SIZE=1>
Got a little couch potato? <BR>Check out fun <A href="http://us.rd.yahoo.com/evt=48248/*http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&cs=bz" target=_blank rel=nofollow>summer activities for kids.</A></DIV><BR></DIV></div><br>
<hr size=1>Yahoo! oneSearch: Finally, <a href="http://us.rd.yahoo.com/evt=48252/*http://mobile.yahoo.com/mobileweb/onesearch?refer=1ONXIC"> mobile search
that gives answers</a>, not web links.
</body></html>