<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=gb2312">
<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta name=Generator content="Microsoft Word 12 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:PMingLiU;
panose-1:2 2 3 0 0 0 0 0 0 0;}
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:MingLiU;
panose-1:2 2 3 9 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Webdings;
panose-1:5 3 1 2 1 5 9 6 7 3;}
@font-face
{font-family:"\@MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"\@MingLiU";
panose-1:2 2 3 9 0 0 0 0 0 0;}
@font-face
{font-family:"\@PMingLiU";
panose-1:2 2 3 0 0 0 0 0 0 0;}
@font-face
{font-family:"Arial Narrow";
panose-1:2 11 6 6 2 2 2 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-NZ link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Generally, you should have a separate RRD file when the source data do not arrive together, or if the number of data are likely to change. However, updating a single RRD with many DS is cheaper (in IO terms) than updating many RRD with one DS each.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>EG1: <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Querying CPU usage on a single machine results in stats for User, System, Wait and Idle. These are all retrieved simultaneously, so can be held as separate DSs within a single RRD.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>EG2:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>An ISP is storing data usage by its clients. Clients may come and go, and the statistics for client data usage are obtained by separate queries from separate routing devices. In this case, we have a separate RRD for each client, each RRD holding just the DSs that relate to that one query (probably bytes in and bytes out).<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Of course every application is different and the optimal way to use RRDtool depends on your particular requirements. So think about where the data come from, if the number of DS will change, how often the data arrive, do you have control over when the data arrive, and so on.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>For example, our setup has 30,000+ RRD files, each about 1.5MB in size, usually holding 8 RRAs and 2 DSs. The number of RRD changes often, the data all arrive at different times. We use rrdcached to manage disk IO requirements.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Steve<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div class=MsoNormal align=center style='text-align:center'><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><hr size=2 width="100%" align=center></span></div><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Steve Shipway</span></b><b><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p></o:p></span></b></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>ITS Unix Services Design Lead<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>University of Auckland, New Zealand<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Floor 1, 58 Symonds Street, Auckland<o:p></o:p></span></p><p class=MsoNormal><i><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>Phone: +64 (0)9 3737599 ext 86487<o:p></o:p></span></i></p><p class=MsoNormal><i><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>DDI: +64 (0)9 923 6487<o:p></o:p></span></i></p><p class=MsoNormal><i><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>Mobile: +64 (0)21 753 189<o:p></o:p></span></i></p><p class=MsoNormal><i><span style='font-size:10.0pt;font-family:"Calibri","sans-serif";color:#595959'>Email: <a href="mailto:s.shipway@auckland.ac.nz"><span style='color:#595959'>s.shipway@auckland.ac.nz</span></a><o:p></o:p></span></i></p><p class=MsoNormal><span lang=EN-GB style='font-size:18.0pt;font-family:Webdings;color:green'>P</span><span lang=EN-GB style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:blue'> </span><span lang=EN-GB style='font-size:10.0pt;font-family:"Arial Narrow","sans-serif";color:green'>Please consider the environment before printing this e-mail</span><span lang=EN-GB style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:blue'> : </span><span lang=ZH-TW style='font-size:10.0pt;font-family:"MS Gothic";color:green'>打印本</span><span lang=ZH-TW style='font-size:10.0pt;font-family:MingLiU;color:green'>邮件，将减少一棵树存活的机会</span><span lang=EN-GB style='font-size:7.5pt;font-family:"Verdana","sans-serif";color:navy'><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> rrd-users-bounces+s.shipway=auckland.ac.nz@lists.oetiker.ch [mailto:rrd-users-bounces+s.shipway=auckland.ac.nz@lists.oetiker.ch] <b>On Behalf Of </b>S Ahmed<br><b>Sent:</b> Thursday, 24 October 2013 9:58 a.m.<br><b>To:</b> rrd-users<br><b>Subject:</b> Re: [rrd-users] large dataset considerations<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Hi,</span><o:p></o:p></p><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Is this tool used by any large scale usages?<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>What is considered a large database size?<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Just looking to get an idea on how people use this tool at scale.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p> </o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Scenerio: Say you want to store time series time informaiton in a Saas application.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>I'm guessing there is some sort of threshold where it makes sense to partition your data e.g. by a group of customers in order to scale out the usage<o:p></o:p></span></p></div></div></div></div></body></html>