[smokeping-users] Smokeping only keeping 4 weeks of data (Solved)

Ryan Harden hardenrm at uiuc.edu
Wed Sep 10 20:14:10 CEST 2008 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

After lots of poking around and some help from a colleague that is much
more familiar with RRDtool, I've been able to track this down.

The problem was in the "Database" section. I had changed the step from
300 to 30 because I wanted a much more granular dataset. I didn't
understand and the documentation didn't point out that I needed to
reconfigure how the database was created after changing the step. When
you change the steps, you need to recalculate the 'steps' and 'total'
columns in order to get the same amount of data.

After figuring this out, and doing the math, I came up with this:
#########################
*** Database ***

step     = 30
pings    = 20

# consfn mrhb steps total

AVERAGE 0.5   1  20160 #7-day worth 30-second avg
AVERAGE 0.5   10  8640  #30-day worth 5-minute avg
AVERAGE 0.5   30 38400 #400-day worth 15-minute avg
MIN 0.5 10 8640
MAX 0.5 10 8640
MIN 0.5 30 38400
MAX 0.5 30 38400
#########################

Math here:
step x #ofSteps x Y = length of database in seconds
(#ofSteps is a multiple of your step adding up to how granular you want
your data to be. In my case 30s=1, 2m=4, 5m=10, etc)

To calculate for 7 days worth of 30 second averages:
30 x 1 x Y = 7days x 24h/day x 60m/hour x 60s/min
(translated to)
30 x 1 x Y = 7 x 24 x 60 x 60
(30 x Y = 604800) or (Y = 604800/30) or 20160

When you the math backward starting with a step of 30 and the default
line: AVERAGE  0.5 144   720, you come up with 36 days of 72-minute
averages.

Thanks Tobi for taking a look at it. Keep up the good work.

/Ryan


Tobias Oetiker wrote:
> Hi Ryan,
> 
> the pdp_per_row value show how many steps there are per rra_row.
> 
> 
> but your settings seem to be exactly as expected (config default)
> so no reason going into this any further.
> 
> The only possible cause for the effect you see is that the rrd
> files got re-created on the date where the data starts up again.
> 
> But since neither you nor anyone else has touched the machines that
> day, I have no idea what could have happend there short of divine
> intervention.
> 
> mistified
> tobi
> 
> 
> Tobi,
> 
> I'm not sure how to interpret the output I'm getting.
> I ran the following on one of my rrds.
> 
> ctweb at nightwing:~/applications/smokeping/data/HOSTS> rrdtool info
> host.rrd | awk '/cf/||/rows/{print $0}'
> rra[0].cf = "AVERAGE"
> rra[0].rows = 1008
> rra[1].cf = "AVERAGE"
> rra[1].rows = 4320
> rra[2].cf = "MIN"
> rra[2].rows = 4320
> rra[3].cf = "MAX"
> rra[3].rows = 4320
> rra[4].cf = "AVERAGE"
> rra[4].rows = 720
> rra[5].cf = "MAX"
> rra[5].rows = 720
> rra[6].cf = "MIN"
> rra[6].rows = 720
> 
> I'm not sure how to tell which RRA is the 1 day resolution you specify.
> 
> I can verify that this seems to be a 'rolling window' as my data only
> goes back to June 26, instead of June 25 as it did yesterday.
> 
> I'm confused how this might have happened. Only a few weeks ago I had
> data going back several months. What might I have done on all three
> machines to cause this?
> 
> Is my only recourse to remove all the RRDs and start from scratch?
> 
> /Ryan
> 
> Tobias Oetiker wrote:
>>>> Hi Ryan,
>>>>
>>>> you should look at the number of rows in the RRA with the 1 day
>>>> resolution
>>>>
>>>> cheers
>>>> tobi
>>>>
>>>> Yesterday Ryan Harden wrote:
>>>>
>>>> What line would I look for to find whether or not it is configured for
>>>> 400days or not?
>>>>
>>>> /Ryan
>>>>
>>>>
>>>> Tobias Oetiker wrote:
>>>>>>> Ryan,
>>>>>>>
>>>>>>> have a look at the setp of yor rrd files, maybe they are actually
>>>>>>> configured to be short ...
>>>>>>>
>>>>>>> rrdtool info
>>>>>>>
>>>>>>> cheers
>>>>>>> tobi
>>>>>>>
>>>>>>> Today Ryan Harden wrote:
>>>>>>>
>>>>>>> I would understand if this were just one installation, but this is
>>>>>>> happening on three installations, on three difference machines. No
>>>>>>> changes were made to the config file at that time.
>>>>>>>
>>>>>>> Any Ideas?
>>>>>>>
>>>>>>> /Ryan
>>>>>>>
>>>>>>> Tobias Oetiker wrote:
>>>>>>>>>> Hi Ryan,
>>>>>>>>>>
>>>>>>>>>> This is odd indeed, it could be that for some reason new rrds were
>>>>>>>>>> created because you changed some nameing in the connfig file ?
>>>>>>>>>>
>>>>>>>>>> as you can see on the demo site, the phaenomenon you see does not
>>>>>>>>>> happen normally.
>>>>>>>>>>
>>>>>>>>>> cheers
>>>>>>>>>> tobi
>>>>>>>>>>
>>>>>>>>>> Today Ryan Harden wrote:
>>>>>>>>>>
>>>>>>>>>> To clarify all 400day graphs go back to June 25. So it's not exactly 4
>>>>>>>>>> weeks like I said previously.
>>>>>>>>>>
>>>>>>>>>> Still very odd.
>>>>>>>>>>
>>>>>>>>>> /Ryan
>>>>>>>>>>
>>>>>>>>>> Ryan Harden wrote:
>>>>>>>>>>>>> I'm having an issue where I have three installations of smokeping.
>>>>>>>>>>>>> (versions 2.3, 2.3.2, and 2.3.5)
>>>>>>>>>>>>>
>>>>>>>>>>>>> A user pointed out that on the 300 days graph, all probes across all
>>>>>>>>>>>>> three installations stop in the last week of June. One has been running
>>>>>>>>>>>>> since October 2007, another January, and another since February.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This section of the config file is pretty much identical on all three
>>>>>>>>>>>>> installations. Is there something I have configured incorrectly? Am I
>>>>>>>>>>>>> missing something on my linux boxes?
>>>>>>>>>>>>>
>>>>>>>>>>>>> *** General ***
>>>>>>>>>>>>>
>>>>>>>>>>>>> owner    = UIUC CITES Network Engineering
>>>>>>>>>>>>> contact  = xxxxxxxxxxxxxxxx
>>>>>>>>>>>>> mailhost = localhost
>>>>>>>>>>>>> sendmail = /usr/lib/sendmail
>>>>>>>>>>>>> imgcache = /services/ctweb/public_html/http/smokeping/cache
>>>>>>>>>>>>> imgurl   = cache
>>>>>>>>>>>>> datadir  = /services/ctweb/applications/smokeping/data
>>>>>>>>>>>>> piddir  = /services/ctweb/applications/smokeping/var
>>>>>>>>>>>>> cgiurl   = http://url/smokeping.cgi
>>>>>>>>>>>>> smokemail = /services/ctweb/applications/smokeping/etc/smokemail
>>>>>>>>>>>>> tmail = /services/ctweb/applications/smokeping/etc/tmail
>>>>>>>>>>>>> display_name = ctweb1
>>>>>>>>>>>>> # specify this to get syslog logging
>>>>>>>>>>>>> syslogfacility = local2
>>>>>>>>>>>>> # each probe is now run in its own process
>>>>>>>>>>>>> # disable this to revert to the old behaviour
>>>>>>>>>>>>> # concurrentprobes = no
>>>>>>>>>>>>>
>>>>>>>>>>>>> *** Alerts ***
>>>>>>>>>>>>> to = xxxxxxxxxxxxx
>>>>>>>>>>>>> from = xxxxx
>>>>>>>>>>>>>
>>>>>>>>>>>>> +someloss
>>>>>>>>>>>>> type = loss
>>>>>>>>>>>>> # in percent
>>>>>>>>>>>>> pattern = >0%,*12*,>0%,*12*,>0%
>>>>>>>>>>>>> comment = loss 3 times  in a row
>>>>>>>>>>>>>
>>>>>>>>>>>>> *** Database ***
>>>>>>>>>>>>>
>>>>>>>>>>>>> step     = 30
>>>>>>>>>>>>> pings    = 20
>>>>>>>>>>>>>
>>>>>>>>>>>>> # consfn mrhb steps total
>>>>>>>>>>>>>
>>>>>>>>>>>>> AVERAGE  0.5   1  1008
>>>>>>>>>>>>> AVERAGE  0.5  12  4320
>>>>>>>>>>>>>     MIN  0.5  12  4320
>>>>>>>>>>>>>     MAX  0.5  12  4320
>>>>>>>>>>>>> AVERAGE  0.5 144   720
>>>>>>>>>>>>>     MAX  0.5 144   720
>>>>>>>>>>>>>     MIN  0.5 144   720
>>>>>>>>>>>>>
>>>>>>>>>>>>> *** Presentation ***
>>>>>>>>>>>>>
>>>>>>>>>>>>> template = /services/ctweb/applications/smokeping/etc/basepage.html
>>>>>>>>>>>>>
>>>>>>>>>>>>> + charts
>>>>>>>>>>>>>
>>>>>>>>>>>>> menu = Charts
>>>>>>>>>>>>> title = The most interesting destinations
>>>>>>>>>>>>>
>>>>>>>>>>>>> ++ stddev
>>>>>>>>>>>>> sorter = StdDev(entries=>4)
>>>>>>>>>>>>> title = Top Standard Deviation
>>>>>>>>>>>>> menu = Std Deviation
>>>>>>>>>>>>> format = Stdandard Deviation %f
>>>>>>>>>>>>>
>>>>>>>>>>>>> ++ max
>>>>>>>>>>>>> sorter = Max(entries=>5)
>>>>>>>>>>>>> title = Top Max Roundtrip Time
>>>>>>>>>>>>> menu = by Max
>>>>>>>>>>>>> format = Max Roundtrip Time %f seconds
>>>>>>>>>>>>>
>>>>>>>>>>>>> ++ loss
>>>>>>>>>>>>> sorter = Loss(entries=>5)
>>>>>>>>>>>>> title = Top Packet Loss
>>>>>>>>>>>>> menu = Loss
>>>>>>>>>>>>> format = Packets Lost %f
>>>>>>>>>>>>>
>>>>>>>>>>>>> ++ median
>>>>>>>>>>>>> sorter = Median(entries=>5)
>>>>>>>>>>>>> title = Top Median Roundtrip Time
>>>>>>>>>>>>> menu = by Median
>>>>>>>>>>>>> format = Median RTT %f seconds
>>>>>>>>>>>>>
>>>>>>>>>>>>> + overview
>>>>>>>>>>>>>
>>>>>>>>>>>>> width = 600
>>>>>>>>>>>>> height = 200
>>>>>>>>>>>>> range = 10h
>>>>>>>>>>>>>
>>>>>>>>>>>>> + detail
>>>>>>>>>>>>>
>>>>>>>>>>>>> width = 600
>>>>>>>>>>>>> height = 200
>>>>>>>>>>>>> unison_tolerance = 2
>>>>>>>>>>>>> nodata_color = dddddd
>>>>>>>>>>>>> loss_background = yes
>>>>>>>>>>>>>
>>>>>>>>>>>>> "Last 3 Hours"    3h
>>>>>>>>>>>>> "Last 30 Hours"   30h
>>>>>>>>>>>>> "Last 10 Days"    10d
>>>>>>>>>>>>> "Last 400 Days"   400d
>>>>>>>>>>>>>
>>>>>>>>>>>>> # NOT CURRENTLY USING THIS
>>>>>>>>>>>>> #+ hierarchies
>>>>>>>>>>>>> #++ UIUCBuoys
>>>>>>>>>>>>> #title = UIUC Buoys
>>>>>>>>>>>>> #++ OTHERBuoys
>>>>>>>>>>>>> #title = Other Buoys
>>>>>>>>>>>>> #++ DNS
>>>>>>>>>>>>> #title = DNS Probes
>>>>>>>>>>>>> #++ HOSTS
>>>>>>>>>>>>> #title = Various Hosts
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> *** Probes ***
>>>>>>>>>>>>>
>>>>>>>>>>>>> +FPing
>>>>>>>>>>>>> # binary = /usr/sbin/fping
>>>>>>>>>>>>> binary = /services/ctweb/bin/fp-test
>>>>>>>>>>>>>
>>>>>>>>>>>>> +DNS
>>>>>>>>>>>>> forks = 5
>>>>>>>>>>>>> timeout = 5
>>>>>>>>>>>>> offset = random
>>>>>>>>>>>>> binary = /usr/bin/dig
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> smokeping-users mailing list
>>>>>>>>>> smokeping-users at lists.oetiker.ch
>>>>>>>>>> https://lists.oetiker.ch/cgi-bin/listinfo/smokeping-users
>>>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> smokeping-users mailing list
>>>>>>> smokeping-users at lists.oetiker.ch
>>>>>>> https://lists.oetiker.ch/cgi-bin/listinfo/smokeping-users
>>>>>
>>
_______________________________________________
smokeping-users mailing list
smokeping-users at lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/smokeping-users
>>
>>

- --
Ryan M. Harden, BS, KC9IHX		Office: 217-265-5192
CITES - Network Engineering		Cell:  	630-363-0365
2130 Digital Computer Lab		Fax:    217-244-7089
1304 W. Springfield	 		email:  hardenrm at illinois.edu
Urbana, IL  61801 			

	 University of Illinois at Urbana/Champaign
		University of Illinois - ICCN
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)

iD8DBQFIyA5ytuPckBBbXboRAv/EAJ9RlS0fYgW5u4zz2O3NjoyiUgdGNgCbB5Qc
rEAGsRe8kWlvbPH0bbE3kAw=
=Ovoo
-----END PGP SIGNATURE-----

More information about the smokeping-users mailing list