[smokeping-users] Regarding Database size ......

[Peter Kristolaitis]

Hi Jimmy;

1)  Granularity is simply the level of detail at which data is recorded
in the RRD file -- do we record every data point (steps = 1), or
aggregate multiple data points into a single value (steps > 1)?  AFAIK,
there is no performance penalty either way; it's simply a matter of
balancing history, presentation and disk space.

2)  Smokeping (and all RRD-based applications) use ALL of them.  Data is
recorded at both steps=144 and steps=12.  The difference comes when you
draw graphs -- rrdtool will use the lowest granularity it can to draw
graphs, given time constraints (that is, if you tried to draw a graph
showing 20 days of history, you wouldn't have enough data at steps=12,
so it would use the steps=144 data instead).

3)  Keep in mind that step=300 would affect Smokeping's poll rate as
well, not just the database storage.  But yes, you can play with any (or
all) of step, steps, and rows to retain data as long as you want.

Typically what I do when deploying Smokeping is sit down and figure out
a few things:
    - How often do I need to poll (the step setting)
    - How much history do I need?
    - How wide (in pixels) will my graphs be?  (the 'width' settings in
*** Presentation ***)
    - What time ranges will I be graphing (e.g. 6 hours, 24 hours, 7
days, 30 days, 365 days, etc)

Using all that information, I'll do some quick math to figure out the
steps and rows settings I need to make the graphs look nice -- that is,
to use consistent and easy-to-see-sized bars (1 pixel wide bars are hard
to see -- 4-8 pixels are much easier!).

- Peter



Jimmyboy wrote:
>  
> Thanks Peter, for the detailed description. However, I have some doubts.
>  
> 1.       What does granularity mean here? How does a lower or a
> greater granularity affect or improve the performance of Smokeping’s
> graphing? In other words, how does that bother an administrator?
>
> 2.       With steps=144, granularity is 72 minutes lasting for 36
> days, with steps=12, granularity is 6 minutes lasting for 18 days.
> Which one does Smokeping use or default to?
> 3.       With this calculation, does it mean that *“step=300”* under
> my **** Database **** section in *config *file will always maintain a
> graph of *360 days*? If yes, can I simply change the value of *step*,
> *steps* or *rows* to retain data for as long as I wish to?
>  
> Regards,
> Jimmy
>  
> =-=-=-=
>
>
>  
> On Sat, Jan 2, 2010 at 12:27 AM, Peter Kristolaitis
> <alter3d at alter3d.ca <mailto:alter3d at alter3d.ca>> wrote:
>
>     First of all, happy new year to everyone on the list!  I hope that
>     2010 is a productive and fruitful year for you all.
>
>     To answer the easy question first -- yes, your RRA files will be
>     2.8MB forever.  An RRA 'database' contains a finite, static number
>     of records, and old records are discarded as new data is put in; 
>     rrdtool makes the files large enough to handle all possible
>     records at creation time, so your file will never grow.
>
>     With regard to how long Smokeping will continue graphing -- well,
>     forever.  Because old records are cycled out when new data is put
>     in, Smokeping will continue to operate indefinitely and will only
>     show 'current' data.   A related issue is how much *history* will
>     be retained, and this is strictly dependent on how you've
>     configured Smokeping (specifically, the 'step' value and the
>     'steps' and 'rows' data from the RRA definitions in the '***
>     Database ***' section of the configuration file).
>
>     Example:
>
>     *** Database ***
>
>     step = 30
>     pings = 20
>
>     # cons   xff steps rows
>     AVERAGE  0.5   1   1008
>     AVERAGE  0.5  12   4320
>         MIN  0.5  12   4320
>         MAX  0.5  12   4320
>     AVERAGE  0.5 144    720
>         MAX  0.5 144    720
>         MIN  0.5 144    720
>
>     The 'step = 30' part means that Smokeping will poll (and thus
>     generate a new RRA row) every 30 seconds.
>
>     The 'steps' part of the RRA definition are how many 'step's are
>     consolidated for a given row.  In the example, we have 1, 12, and
>     144;  this means that we're storing data at 3 different levels of
>     granularity:  1*30 = 30 seconds, 12*30 = 360 seconds, and 144*30 =
>     4320 seconds.
>
>     The 'rows' part of the RRA definition tells you how many records
>     that RRA definition will hold.
>
>     Using 'step', 'steps' and 'rows', you can calculate how much
>     'time' an RRA will cover:
>        time in seconds = step * steps * rows
>
>     So if we look at the line "AVERAGE 0.5 144 720", it will hold:
>         time = 30 * 144 * 720 = 3,110,400 seconds (36 days)
>
>     This means that you will be able to get at most 36 days of history
>     at a granularity of 72 minutes (30*144 seconds).   If you wanted
>     to see the data more granularly (6 minutes -- 30*12 seconds), then
>     you would have to set your graphs to show at most 30*12*4320 =
>     1,555,200 seconds (18 days).
>
>     - Peter
>
>
>
>     Jimmyboy wrote:
>>     Hi All,
>>      
>>     I am using Smokeping 2.3 on my Ubuntu desktop machine since more
>>     than 1 month. The size of all .rra files shows up as 2.8MB ever
>>     since they have been created.
>>      
>>     Can anyone tell me how do I know till what time would Smokeping
>>     continue graphing. Is it going to be a few months, 1 year or 2,
>>     etc...? Will the size remain 2.8MB forever?
>>      
>>      
>>     Thanks,
>>     Jimmy Jose
>>      
>>     =-=-=
>>     ------------------------------------------------------------------------
>>
>>     _______________________________________________
>>     smokeping-users mailing list
>>     smokeping-users at lists.oetiker.ch <mailto:smokeping-users at lists.oetiker.ch>
>>     https://lists.oetiker.ch/cgi-bin/listinfo/smokeping-users
>>       
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/smokeping-users/attachments/20100104/ee2f9473/attachment.htm