[smokeping-users] Storing high time resolution results over longer periods

[Luigi Corsello]

Chris,

Bad news, but thanks for the excellent explanation.

My memory is limited so I went for two+two months and switched to 
FastCGI to cope with the huge memory footprint of the plain smokeping.cgi

AVERAGE  0.5   1   87600
AVERAGE  0.5   5   87600


So leaving a notice for whoever wishes to set a lower polling interval 
in their smokeping setup, to my experience at least:
- in etc/config: change the step value
- in /etc/config: adapt the rrd parameters according to your needs*
- stop smokeping
- clear or move the RRDs under smokeping/data (yes, your old graphs are 
gone!)
- start smokeping

*a basic rule to keep the same resolution you were previously seeing 
could be:
multiply all "total" values for your increase factor, for example 
passing from 5 to 1 minutes polling intervals, multiply the totals by 5.



L.C.



On 12/4/12 2:19 PM, Chris Wilson wrote:
> Hi Luigi,
>
> On Tue, 4 Dec 2012, Luigi Corsello wrote:
>
>> I am using a step of 60 seconds in my smokeping setup, had all RRDs 
>> regenerated to get the graphs I need to see:
>>
>> However all graphs of my benchmarks are captured in batches one or 
>> more days later (up to 4-5 months in corner cases). But taking a 
>> graph just one days later all details are gone! (for good, I fear):
>>
>> One Hour fping graph - yesterday - ALL DETAIL GONE!
>
> Yes. That is the point of rrdtool, to make it easy to automatically 
> aggregate old results, which drops the resolution. The raw data is 
> discarded once it's been incorporated into the aggregates.
>
>> If the detail is really lost, this is a bit of a disaster forcing to 
>> repeat an all bunch of measurements that were already done and then 
>> immediately capture the graphs. Can anyone confirm most details of my 
>> history are gone?
>
> Yes, because:
>
>> # consfn mrhb steps total
>>
>> AVERAGE  0.5   1  1008
>
> You only store 1008 results at full detail (step 1). That's 1008 
> minutes if you use step = 60.
>
>> AVERAGE  0.5  12  4320
>
> And 4320 results are stored at one data point per 12 steps (12 minutes 
> in your config). That means that the last 4320*12 minutes are stored 
> averaged over 12 minutes (and the minimum and maximum over each 12 
> minute period).
>
> If you really want to store full resolution data for 6 months, you 
> need a line that has type average, step 1, and length 60*24*(365/2) = 
> 262800 minutes (365/2 days):
>
> AVERAGE 0.5 1 262800
>
> You may not find much use for the other detail levels in this case, so 
> you might want to delete them.
>
> Cheers, Chris.