<html><head><title>Re: [smokeping-users] Smokeping only keeping ~8 weeks of data</title>
<meta charset="utf-8" http-equiv="X-UA-Compatible" content="IE=9; IE=8; IE=7; IE=EDGE" />
</head>
<body>
<span style=" font-family:'Courier New'; font-size: 9pt;">It's your call on how much full res data you keep. [I keep two weeks of one minute full-res data.] You could keep as little as 144 minutes of full res data (provided I'm thinking correctly) - because that's the maximum "step" you currently have. i.e. It needs 144 samples to make your third tier of data, so it needs 144 full-res samples to "make" each row. That would be, IMO, pretty nuts, but it would work. But storage is way cheap, even SSD (unless you're keeping a massive amount of data on a really massive set of targets.)<br>
<br>
The main reason to keep more is you get better visibility for longer.<br>
<br>
For example.<br>
In two samples (back to back), packet loss is 100%. In the others, it's zero.<br>
<br>
But by the time you get around to looking at it, say the day after those samples were taken [24 hours, say] it's averaged those into 12m samples. And, if you're particularly unlucky, each of the 100% loss samples got averaged into two different rows. So, 11 rows of 0% loss and 1 @ 100% - so it's an average of ~8% loss - in two back-to-back rows of second tier data.<br>
<br>
So, if you try to figure out what went wrong, it's going to be really hard to find the "signal" where that 2m of 100% loss was. [Your graph will show two pixels with 5-10% loss in the middle of a sea of green. To wit: You'll never see it.]<br>
<br>
So, for me at least, keeping the full res data long enough to have diagnostic value is important. I may not realize something bad was going on for a while - days, perhaps even a week or two. If I start averaging that data out, the "signal" usually gets fainter - and I'm often obtuse enough to need all the help I can get. :)<br>
<br>
---<br>
A restart of SP will do the trick. But as noted, any existing data that's in the RRD from the past won't make any sense if you change the number of rows or steps and start capturing new data. [I believe it will restructure the RRD "automagically" but it will simply leave the data in the RRD as is. Though, don't gamble that I'm recalling that right. There's mention of how that's handled in the docs/list-postings - I'm too lazy to go dig it up.]<br>
<br>
HTH<br>
<br>
-Greg<br>
<br>
<a name="divRplyFwdMsg"></a>
</span><table style =" border-collapse: collapse;" cellpadding = 1 cellSpacing = 2>
<tr>
<td width=3 bgcolor= #0000ff><br>
</td>
<td ><span style=" font-family:'calibri'; font-size: 12pt;">Hi Greg,<br>
<br>
Thank you very much. This explains a lot. I don't intend to change the step time for now.<br>
<br>
I'll do the math as advised <br>
<br>
However, do I need to change the number of minutes for the full res data which is currently 1008?<br>
<br>
In addition, are there any additional actions I need to perform asides restarting the smokeping service for the changes to take effect?<br>
<br>
Many Thanks,<br>
Debo.<br>
<br>
<hr style="height: 2px; color : #000000; background-color : #000000; border-width : 0px;"><span style=" font-size: 11pt;"><b>From:</b> smokeping-users <smokeping-users-bounces+otubushin=hotmail.com@lists.oetiker.ch> on behalf of Gregory Sloop <gregs@sloop.net><br>
<b>Sent:</b> 23 April 2020 07:22<br>
<b>To:</b> smokeping-users@lists.oetiker.ch <smokeping-users@lists.oetiker.ch><br>
<b>Subject:</b> Re: [smokeping-users] Smokeping only keeping ~8 weeks of data<span style=" font-family:'courier new'; font-size: 9pt;"> <br>
<br>
*** Database ***<br>
step = 60<br>
pings = 60<br>
<br>
# consfn mrhb steps total<br>
AVERAGE 0.5 1 1008<br>
AVERAGE 0.5 12 4320<br>
MIN 0.5 12 4320<br>
MAX 0.5 12 4320<br>
AVERAGE 0.5 144 720<br>
MAX 0.5 144 720<br>
MIN 0.5 144 720<br>
<br>
---<br>
Your database section isn't configured to save data any older than 72 days.<br>
<br>
1008 minutes of full res data [Just under 17 hours - that's kind of an odd number, but whatever.]<br>
36 days of 12m data<br>
72 days of 144m data<br>
<br>
If you want to have more data, you're going to have to change the database section to keep more rows.<br>
<br>
Your step is 60s - so 1008 samples of 60s data.<br>
Then 4320 rows averaging 12m of data. (4320*12=Total minutes of data. That comes out to 36 days worth.]<br>
Then 720 rows of 144m averaged data. [720*144=Total minutes of data. That comes out to 72 days.]<br>
<br>
I'll leave it as an exercise to figure out how many rows you'll need to keep of each sample type to have history long enough to cover the time-frame you want.<br>
<br>
But, for example. the 144m samples - if you want 2 years of data [365*2] - you'd need<br>
365 days * 2 = 2 years or 730 days<br>
730 days * 24 hours = 17520 hours.<br>
17520h * 60minutes = 1,051,200 minutes in 2 years.<br>
Now divide those minutes by 144, and that's how many rows you'll need, if you want two years of 144m averaged data.<br>
(i.e. 7300)<br>
***That all assumes you continue to use 60s step times. If you change the step, you'll have to recalculate given the new step size.<br>
If you change your step, you'll either have to re-sample your data, or more likely, dump the old data and let it rebuild the RRD files again with the proper structure and lose the historical data.<br>
<br>
HTH<br>
<br>
-Greg<br>
<br>
<br>
<br>
</span></span></span><table style =" border-collapse: collapse;" cellpadding = 2 cellSpacing = 4>
<tr>
<td width=3 bgcolor= #0000ff><br>
</td>
<td ><span style=" font-family:'calibri'; font-size: 12pt;">Hi All,<br>
<br>
I am having an issue with smokeping not displaying any graphs older than ~8 weeks. I have it configured to display graphical data for 3hrs, 30hrs, 10days and 400days.<br>
So far, it does not show anything before the 7th week(in February 2020).<br>
<br>
Is there anything I have configured wrongly here?<br>
<br>
<br>
<br>
<span style=" font-family:'courier new'; font-size: 9pt;">*** General ***<br>
<br>
owner = ######################################<br>
contact = ######################################<br>
#mailhost = my.mail.host<br>
#sendmail = /sbin/sendmail<br>
# NOTE: do not put the Image Cache below cgi-bin<br>
# since all files under cgi-bin will be executed ... this is not<br>
# good for images.<br>
imgcache = /opt/smokeping/cache<br>
imgurl = cache<br>
datadir = /opt/smokeping/data<br>
piddir = /opt/smokeping/var<br>
cgiurl = http://10.20.1.41/smokeping.fcgi<br>
smokemail = /opt/smokeping/etc/smokemail.dist<br>
tmail = /opt/smokeping/etc/tmail.dist<br>
# specify this to get syslog logging<br>
syslogfacility = local0<br>
# each probe is now run in its own process<br>
# disable this to revert to the old behaviour<br>
# concurrentprobes = no<br>
<br>
*** Alerts ***<br>
to = ######################################<br>
from = ######################################<br>
+someloss<br>
type = loss<br>
# in percent<br>
pattern = >0%,*12*,>0%,*12*,>0%<br>
comment = loss 3 times in a row<br>
<br>
<br>
+bigloss<br>
type = loss<br>
# in percent<br>
pattern = ==0%,==0%,==0%,==0%,>0%,>0%,>0%<br>
comment = suddenly there is packet loss<br>
<br>
+startloss<br>
type = loss<br>
# in percent<br>
pattern = ==S,>0%,>0%,>0%<br>
comment = loss at startup<br>
<br>
+rttdetect<br>
type = rtt<br>
# in milli seconds<br>
pattern = <10,<10,<10,<10,<10,<100,>100,>100,>100<br>
comment = routing messed up again ?<br>
<br>
+hostdown<br>
type = loss<br>
# in percent<br>
pattern = ==0%,==0%,==0%, ==U<br>
comment = no reply<br>
<br>
+lossdetect<br>
type = loss<br>
# in percent<br>
pattern = ==0%,==0%,==0%,==0%,>20%,>20%,>20%<br>
comment = suddenly there is packet loss<br>
<br>
<br>
*** Database ***<br>
<br>
step = 60<br>
pings = 60<br>
<br>
# consfn mrhb steps total<br>
<br>
AVERAGE 0.5 1 1008<br>
AVERAGE 0.5 12 4320<br>
MIN 0.5 12 4320<br>
MAX 0.5 12 4320<br>
AVERAGE 0.5 144 720<br>
MAX 0.5 144 720<br>
MIN 0.5 144 720<br>
<br>
*** Presentation ***<br>
<br>
template = /opt/smokeping/etc/basepage.html.dist<br>
<br>
+ charts<br>
<br>
menu = Charts<br>
title = The most interesting destinations<br>
<br>
++ stddev<br>
sorter = StdDev(entries=>4)<br>
title = Top Standard Deviation<br>
menu = Std Deviation<br>
format = Standard Deviation %f<br>
<br>
++ max<br>
sorter = Max(entries=>5)<br>
title = Top Max Roundtrip Time<br>
menu = by Max<br>
format = Max Roundtrip Time %f seconds<br>
<br>
++ loss<br>
sorter = Loss(entries=>5)<br>
title = Top Packet Loss<br>
menu = Loss<br>
format = Packets Lost %f<br>
<br>
++ median<br>
sorter = Median(entries=>5)<br>
title = Top Median Roundtrip Time<br>
menu = by Median<br>
format = Median RTT %f seconds<br>
<br>
+ overview<br>
<br>
width = 600<br>
height = 50<br>
range = 10h<br>
<br>
+ detail<br>
<br>
width = 600<br>
height = 200<br>
unison_tolerance = 2<br>
<br>
"Last 3 Hours" 3h<br>
"Last 30 Hours" 30h<br>
"Last 10 Days" 10d<br>
"Last 400 Days" 400d<br>
<br>
<span style=" font-family:'calibri'; font-size: 12pt;">#+ hierarchies<br>
#++ owner<br>
#title = Host Owner<br>
#++ location<br>
#title = Location<br>
<br>
*** Probes ***<br>
<br>
+ FPing<br>
<br>
binary = /usr/sbin/fping<br>
<br>
*** Slaves ***<br>
secrets=/opt/smokeping/etc/smokeping_secrets.dist<br>
+boomer<br>
display_name=boomer<br>
color=0000ff<br>
<br>
+slave2<br>
display_name=another<br>
color=00ff00<br>
<br>
*** Targets ***<br>
<br>
probe = FPing<br>
<br>
Many Thanks,<br>
Debo</td>
</tr>
</table>
</td>
</tr>
</table>
</body></html>