<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Webdings;
panose-1:5 3 1 2 1 5 9 6 7 3;}
@font-face
{font-family:"Arial Narrow";
panose-1:2 11 6 6 2 2 2 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-NZ link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Looks like your collections are being done via MRTG, going by the structure.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>You can’t specify when consolidations are done, but on the whole it shouldn’t make such a difference. We don’t experience anything like this pattern on our MRTG/RRD servers.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>In order to spread things out over time, there are a number of things you can do. Using RRD 1.4.x (possibly the trunk version) allows you to use rrdcached which has a noticeable (~20%?) performance saving; also you should tune your use of the Forks: directive in MRTG to make sure you’re multithreading appropriately. Adding more memory to the server might also help, if you need to increase the threads (our machines tend to be memory-bound rather than CPU-bound, but we use many data-collection plugins)<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>If you don’t use MRTG in daemon mode then it is less efficient; RRDTool 1.3 and 1.4 can use memory-mapping and other nice things to improve performance, as well as MRTG caching the config files, but this works better when in daemon mode. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Steve<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div class=MsoNormal align=center style='text-align:center'><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><hr size=2 width="100%" align=center></span></div><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Steve Shipway<o:p></o:p></span></b></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>ITS Unix Services Design Lead<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>University of Auckland, New Zealand<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Floor 1, 58 Symonds Street, Auckland<o:p></o:p></span></p><p class=MsoNormal><i><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>Phone: +64 (0)9 3737599 ext 86487<o:p></o:p></span></i></p><p class=MsoNormal><i><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>DDI: +64 (0)9 924 6487<o:p></o:p></span></i></p><p class=MsoNormal><i><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>Mobile: +64 (0)21 753 189<o:p></o:p></span></i></p><p class=MsoNormal><i><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>Email: <a href="mailto:s.shipway@auckland.ac.nz"><span style='color:#595959'>s.shipway@auckland.ac.nz</span></a><o:p></o:p></span></i></p><p class=MsoNormal><span lang=EN-GB style='font-size:18.0pt;font-family:Webdings;color:green'>P</span><span lang=EN-GB style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:blue'> </span><span lang=EN-GB style='font-size:10.0pt;font-family:"Arial Narrow","sans-serif";color:green'>Please consider the environment before printing this e-mail</span><span lang=EN-GB style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:blue'> </span><span lang=EN-GB style='font-size:7.5pt;font-family:"Verdana","sans-serif";color:navy'><o:p></o:p></span></p><p class=MsoNormal><i><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></i></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> rrd-users-bounces+s.shipway=auckland.ac.nz@lists.oetiker.ch [mailto:rrd-users-bounces+s.shipway=auckland.ac.nz@lists.oetiker.ch] <b>On Behalf Of </b>Joshua Keroes<br><b>Sent:</b> Sunday, 1 May 2011 5:53 p.m.<br><b>To:</b> rrd-users@lists.oetiker.ch<br><b>Subject:</b> [rrd-users] Slow collection runtimes occurring regularly<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal style='margin-bottom:12.0pt'>Our collectors run long at regular intervals; in particular every two hours, and to lesser extents every hour and half hour. Here's a graph showing how long each collection cycle lasts on one of the collection machines: <a href="http://i.imgur.com/xaZJ5.png">http://i.imgur.com/xaZJ5.png</a> – note the regular spikes.<o:p></o:p></p><div><p class=MsoNormal>Most RRD's consolidate every 30 minutes, 2 hours, and 24 hours; see the bottom for a sample `rrd info`. Our current theory is that the RRD consolidations are causing these long runtimes. If that's the case, is there a way to evenly stagger the consolidations over time so we can better distribute RRD update load?<o:p></o:p></p></div><div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Thanks,<o:p></o:p></p></div><div><p class=MsoNormal>Joshua<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><div><p class=MsoNormal>filename = "/rrd/router/<a href="http://cr01.ptleorte.integra.net/tengigabitethernet134.rrd">cr01.ptleorte.integra.net/tengigabitethernet134.rrd</a>" <o:p></o:p></p></div><div><p class=MsoNormal>rrd_version = "0003" <o:p></o:p></p></div><div><p class=MsoNormal>step = 300 <o:p></o:p></p></div><div><p class=MsoNormal>last_update = 1304228713 <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds0].type = "COUNTER" <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds0].minimal_heartbeat = 600 <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds0].min = 0.0000000000e+00 <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds0].max = 1.2500000000e+09 <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds0].last_ds = "1596044569532963" <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds0].value = 4.0248335433e+08 <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds0].unknown_sec = 0 <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds1].type = "COUNTER" <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds1].minimal_heartbeat = 600 <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds1].min = 0.0000000000e+00 <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds1].max = 1.2500000000e+09 <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds1].last_ds = "3460406816844600" <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds1].value = 8.9596753966e+08 <o:p></o:p></p></div><div><p class=MsoNormal>ds[ds1].unknown_sec = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[0].cf = "AVERAGE" <o:p></o:p></p></div><div><p class=MsoNormal>rra[0].rows = 600 <o:p></o:p></p></div><div><p class=MsoNormal>rra[0].pdp_per_row = 1 <o:p></o:p></p></div><div><p class=MsoNormal>rra[0].xff = 5.0000000000e-01 <o:p></o:p></p></div><div><p class=MsoNormal>rra[0].cdp_prep[0].value = NaN <o:p></o:p></p></div><div><p class=MsoNormal>rra[0].cdp_prep[0].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[0].cdp_prep[1].value = NaN <o:p></o:p></p></div><div><p class=MsoNormal>rra[0].cdp_prep[1].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[1].cf = "AVERAGE" <o:p></o:p></p></div><div><p class=MsoNormal>rra[1].rows = 600 <o:p></o:p></p></div><div><p class=MsoNormal>rra[1].pdp_per_row = 6 <o:p></o:p></p></div><div><p class=MsoNormal>rra[1].xff = 5.0000000000e-01 <o:p></o:p></p></div><div><p class=MsoNormal>rra[1].cdp_prep[0].value = 9.4104250250e+07 <o:p></o:p></p></div><div><p class=MsoNormal>rra[1].cdp_prep[0].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[1].cdp_prep[1].value = 2.0174889583e+08 <o:p></o:p></p></div><div><p class=MsoNormal>rra[1].cdp_prep[1].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[2].cf = "AVERAGE" <o:p></o:p></p></div><div><p class=MsoNormal>rra[2].rows = 600 <o:p></o:p></p></div><div><p class=MsoNormal>rra[2].pdp_per_row = 24 <o:p></o:p></p></div><div><p class=MsoNormal>rra[2].xff = 5.0000000000e-01 <o:p></o:p></p></div><div><p class=MsoNormal>rra[2].cdp_prep[0].value = 6.5449761744e+08 <o:p></o:p></p></div><div><p class=MsoNormal>rra[2].cdp_prep[0].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[2].cdp_prep[1].value = 1.4734297081e+09 <o:p></o:p></p></div><div><p class=MsoNormal>rra[2].cdp_prep[1].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[3].cf = "AVERAGE" <o:p></o:p></p></div><div><p class=MsoNormal>rra[3].rows = 732 <o:p></o:p></p></div><div><p class=MsoNormal>rra[3].pdp_per_row = 288 <o:p></o:p></p></div><div><p class=MsoNormal>rra[3].xff = 5.0000000000e-01 <o:p></o:p></p></div><div><p class=MsoNormal>rra[3].cdp_prep[0].value = 2.2692529674e+09 <o:p></o:p></p></div><div><p class=MsoNormal>rra[3].cdp_prep[0].unknown_datapoints = 3 <o:p></o:p></p></div><div><p class=MsoNormal>rra[3].cdp_prep[1].value = 4.7002069004e+09 <o:p></o:p></p></div><div><p class=MsoNormal>rra[3].cdp_prep[1].unknown_datapoints = 3 <o:p></o:p></p></div><div><p class=MsoNormal>rra[4].cf = "MAX" <o:p></o:p></p></div><div><p class=MsoNormal>rra[4].rows = 600 <o:p></o:p></p></div><div><p class=MsoNormal>rra[4].pdp_per_row = 1 <o:p></o:p></p></div><div><p class=MsoNormal>rra[4].xff = 5.0000000000e-01 <o:p></o:p></p></div><div><p class=MsoNormal>rra[4].cdp_prep[0].value = NaN <o:p></o:p></p></div><div><p class=MsoNormal>rra[4].cdp_prep[0].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[4].cdp_prep[1].value = NaN <o:p></o:p></p></div><div><p class=MsoNormal>rra[4].cdp_prep[1].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[5].cf = "MAX" <o:p></o:p></p></div><div><p class=MsoNormal>rra[5].rows = 600 <o:p></o:p></p></div><div><p class=MsoNormal>rra[5].pdp_per_row = 6 <o:p></o:p></p></div><div><p class=MsoNormal>rra[5].xff = 5.0000000000e-01 <o:p></o:p></p></div><div><p class=MsoNormal>rra[5].cdp_prep[0].value = 3.2405792329e+07 <o:p></o:p></p></div><div><p class=MsoNormal>rra[5].cdp_prep[0].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[5].cdp_prep[1].value = 6.9813629778e+07 <o:p></o:p></p></div><div><p class=MsoNormal>rra[5].cdp_prep[1].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[6].cf = "MAX" <o:p></o:p></p></div><div><p class=MsoNormal>rra[6].rows = 600 <o:p></o:p></p></div><div><p class=MsoNormal>rra[6].pdp_per_row = 24 <o:p></o:p></p></div><div><p class=MsoNormal>rra[6].xff = 5.0000000000e-01 <o:p></o:p></p></div><div><p class=MsoNormal>rra[6].cdp_prep[0].value = 3.4089842030e+07 <o:p></o:p></p></div><div><p class=MsoNormal>rra[6].cdp_prep[0].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[6].cdp_prep[1].value = 7.6745619740e+07 <o:p></o:p></p></div><div><p class=MsoNormal>rra[6].cdp_prep[1].unknown_datapoints = 0 <o:p></o:p></p></div><div><p class=MsoNormal>rra[7].cf = "MAX" <o:p></o:p></p></div><div><p class=MsoNormal>rra[7].rows = 732 <o:p></o:p></p></div><div><p class=MsoNormal>rra[7].pdp_per_row = 288 <o:p></o:p></p></div><div><p class=MsoNormal>rra[7].xff = 5.0000000000e-01 <o:p></o:p></p></div><div><p class=MsoNormal>rra[7].cdp_prep[0].value = 4.4271024386e+07 <o:p></o:p></p></div><div><p class=MsoNormal>rra[7].cdp_prep[0].unknown_datapoints = 3 <o:p></o:p></p></div><div><p class=MsoNormal>rra[7].cdp_prep[1].value = 8.8648080465e+07 <o:p></o:p></p></div><div><p class=MsoNormal>rra[7].cdp_prep[1].unknown_datapoints = 3 <o:p></o:p></p></div></div><div><p class=MsoNormal><o:p> </o:p></p></div></div></div></div></body></html>