[rrd-users] RRDs with unpredictable update time

[Till Dörges]

Hello everybody,

thanks for all the answers. :-)

On 25.03.2009 09:27, Karl Fischer wrote:
> Till Dörges wrote:

>> I'm trying to collect and store data (e.g. how many times a TCP port is being
>> accessed in a certain interval) but with a rather unpredictable source. Ideally I get
>> a measurement every 300 seconds. But sometimes I don't get anything for hours or
>> days. I intend to store the data for 7 days in the original resolution (300s).
>>
>> For performance reasons, 'rrdtool update' should only be called, when there actually
>> is data. This means for the example below, it's called only 5 times.
[...]
> well, as Simon and Darren said already, rrdtool is not really designed for that.
> However, you can get around this by changing your heartbeat to a week and insert
> just one zero at N-300 if there hasn't been an update at N-300, so you need to
> remember the last update or to query the rrdb:

I was thinking into that direction, too. After meditating over the problem for a
while I tried inserting 1 additional values for every original value. Exactly one 1
second before the original:

--- snip ---
rrdtool create test.rrd --start=1237923899 DS:events:GAUGE:600:0:U RRA:AVERAGE:0.5:1:2016
rrdtool update test.rrd '1237923899:123'
rrdtool update test.rrd '1237923900:123'
rrdtool update test.rrd '1237924199:456'
rrdtool update test.rrd '1237924200:456'
rrdtool update test.rrd '1237924499:789'
rrdtool update test.rrd '1237924500:789'
rrdtool update test.rrd '1237931699:1111'
rrdtool update test.rrd '1237931700:1111'
rrdtool update test.rrd '1237938899:2222'
rrdtool update test.rrd '1237938900:2222'
--- snap ---

Which also seems to give me what I want:

--- snip ---
  <!-- 2009-03-24 20:45:00 CET / 1237923900 --> <row><v> 1.2300000000e+02 </v></row>
  <!-- 2009-03-24 20:50:00 CET / 1237924200 --> <row><v> 4.5489000000e+02 </v></row>
  <!-- 2009-03-24 20:55:00 CET / 1237924500 --> <row><v> 7.8789000000e+02 </v></row>
  <!-- 2009-03-24 22:55:00 CET / 1237931700 --> <row><v> 1.1110000000e+03 </v></row>
  <!-- 2009-03-25 00:55:00 CET / 1237938900 --> <row><v> 2.2220000000e+03 </v></row>
--- snap ---

Looking at the documentation this approach shouldn't have any side effects. I'm
interested in the primary data points and they seem to come out just right in the RRA.

Are there any problems with this approach?

The downside, however, is that I double the amounts of writes to an RRD. OTOH with
the other solutions suggested I'd have to do a query on the RRD. W/o having measured
anything my guess would be that there is not much difference.

Also this approach has another benefit for me. The RRDs are usually queried up to the
last update step. For the above example 1237938900. Without the update at
1237938900-1 I don't have an entry in the RRA at 1237938900.

Regards -- Till
-- 
Dipl.-Inform. Till Dörges                      td at pre-secure.de
Senior Researcher                 Phone: +49 (0)700 / PRESECURE

PRESECURE Consulting GmbH, Münster         AG Münster, HRB 6581
Geschäftsführer/Managing Director   Dr. Klaus-Peter Kossakowski

                            CarmentiS - Early Warning Expertise
                                       http://www.carmentis.org