[rrd-users] trying to understand the relationship between source data, what's in rrd and what gets plotted

[Mark Seger]

more experiments and I'm getting closer...  I think the problem is the 
AVERAGE in my DEF statements of the graphing command.  The only problem 
is I couldn't find any clear description or examples of how this works.  
I did try using LAST (even though I have no idea what it does) and my 
plots got better, but I'm still missing data points and I want to see 
them all.  Again, I have a step size of 1 second so I'd think everything 
should just be there...
-mark

Mark Seger wrote:
> thanks for the detailed reply and the good news is it more or less 
> agrees with how I thought this works.  in fact, I did find a bug in my 
> table loading script which caused the table to get created with an 
> interval of 3 seconds vs 1 and so yes, things got normalized in 
> unexpected way.  once I got my step to 1 second which corresponds to 
> my data samples everything did load into the table as expected.  now 
> keep in mind I have multiple data samples > 50K.
>
> when I plot the data I only see max data points of 27K.  I tried 
> starting the plot 10 seconds later and finishing 10 seconds earlier to 
> make sure I'm within the range of the data file and I still don't see 
> those points, so I tried just plotting the first 100 elements and lo 
> and behold I can now see the points of interest.  Now I can understand 
> the beginning or ending data points possibly getting modified if the 
> plotting range is outside the data points, by why would internal 
> values get modified?  remember, my samples are 1 second apart and I 
> have a step of 1 second.
>
> I'm still not sure I fully appreciate the xff in the create but I'm 
> also guessing since I don't have any missing data it shouldn't make a 
> difference, should it?
>
> Any ideas about what I'm doing wrong?  I'm sure it's me... 8-(
>
> -mark
>
>
> Simon Hobson wrote:
>> Mark Seger wrote:
>>
>>  
>>> I'm sure it's me and not rrdtool, but I can't figure out what's
>>> happening!  Here's what did:
>>> - I have a data that has been generated at 1 second intervals
>>> - generated a plot with gnuplot as a known quantity
>>> - loaded the same data into rrdtool as absolute values and also with an
>>> interval of 1 sec
>>> - did a fetch of all the data
>>> - converted it to something I could load into excel and compare with 
>>> the
>>> original values to make sure correct.  they do NOT agree but are 
>>> close. perhaps I a still to new to rrdtool to appreciate how my data 
>>> can get
>>> adjusted, but I also would have thought using absolute and 1 second
>>> samples there wouldn't be anything to adjust.
>>>     
>>
>> Wrong.
>>
>>  
>>>   that's question 1.  is
>>> there a way to make it not do that?  I want to plot the values I've
>>> recorded and not some manipulated value.  that's question 2.
>>>     
>>
>> Yes, if you follow some basic rules.
>>
>> RRDs only store rates, not values - so even though you update with 
>> absolute values, this is converted internally into rates.
>>
>> You will get out what you put in (subject to internal rounding errors 
>> etc) only if you meet all these conditions :
>>
>> 1) You input an update with a timestamp which is exactly aligned with 
>> a multiple of step. Eg, if you use a step of 10s, then every update 
>> must be an exact multiple of 10s.
>>
>> 2) You have an RRA within the RRD which is ONE step/consolidated value.
>>
>> These two between them will eliminate normalisation (1) and 
>> aggregation (2).
>>
>> If you ignore 1 then your input data will be normalised, if you 
>> ignore 2 then your data will be consolidated.
>>
>>
>> 3) When plotting/extracting data, you must select start and end times 
>> which are also an exact multiple of your step AND the end time must 
>> not be after the last full step time AND the start time must not be 
>> before the length of your one step/consolidation RRA AND the number 
>> of pixels in the graph (or data points extracted) should match the 
>> number of samples between start and end.
>>
>> This will eliminate normalisation of the extracted data or the use of 
>> a lower resolution RRA.
>>
>> Note that you can only extract data from RRAs (ie data that has 
>> nominally been consolidated), but it's convenient that if you perform 
>> consolidation with one value then you don't change it !
>>
>> Alex has more on normalisation at his site 
>> http://www.vandenbogaerdt.nl/rrdtool/
>>
>> _______________________________________________
>> rrd-users mailing list
>> rrd-users at lists.oetiker.ch
>> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
>>