[rrd-developers] Re: Microsecond measuremend precision

[Sasha Mikheev]

Hi,

This is detailed patch description. Please comment on it.

> Hi Sascha,
>
> the patch has to be for the 1.1.x tree (the rrd format can not
> change in the 1.0.x series (officially))
>
> and there breaking compatibility is fine as long as old rrd files
> continue to work. The important thing is to make the integration
> 'beautiful' ... so best is to send a detailed description of the
> intended change to the developers list to have some consensus on
> what should be done.
>


When we need it ?
======================

Minimal rrdtool time unit is one second. It means that measurement precision
is about 1/dT where dT is rrdtool measurement step. In reality the error can
be much worse because rrdool uses linear interpolation, there can be several
measurements during the rrdtool interval etc.

Anyway 1/dT gives us about 10% error once we measure anything with 10s
frequancy. The patch seeks to solve this problem.


Patch description
=======================

1. We need to store time in microseconds since last update in rrd file.
Later we will discuss how to store it without causing compatibility
problems.
Microsecond correction is always positive and stored us unsigned long value.

2. Time value can be specified  as <seconds>.<microseconds> or as <seconds>
   Following code is used to compute microsecond correction:
	 tmp = strtod(updvals[0], 0);
         current_time = floor(tmp);
         current_time_usec = (long)((tmp - current_time) * 1000000L);

  Instead of the current code:
	current_time = atol(updvals[0]);

3. If time is not specified then

	gettimeofday(&tmp_time, 0);
        normalize_time(&tmp_time); /* make sure tv_usec are positive */
        current_time = tmp_time.tv_sec;
        current_time_usec = tmp_time.tv_usec;

  Instead of
  	current_time = time(NULL);

4. Variables interval,post_int,pre_int are changed to double

The variables are adjusted in following way:

        interval     = current_time + ((double)current_time_usec -
(double)rrd.stat_head->last_up_usec)/1000000.0 - rrd.live_head->last_up;

        if (occu_pdp_st > proc_pdp_st) {
            /* OK we passed the pdp_st moment*/
            pre_int  = occu_pdp_st - rrd.live_head->last_up; /* how much of
the input data
                                                              * pdp_st
moment*/
            pre_int -= ((double)rrd.stat_head->last_up_usec)/1000000.0; /*
adjust msecs */
            post_int = occu_pdp_age; /* how much after it */
            post_int += ((double)current_time_usec)/1000000.0;  /* adjust
msecs */
        } else {
            pre_int  = interval;
            post_int = 0;
        }



4. At the end of update time is written to the rrd file.


Keeping backward compatibility
==============================

1. RRD file version is incremented.
2. Additional field last_up_usec is added to rrd_live_head_t
3. On rrd_open
	If we have old version we only read last_up field. last_up_usec initialized
to zero.
	If we have new version we read two fields.
4. On header write back we do the same.

Potential problems
==================================

1. microseconds are not counted in the  PDP_unkn_sec_cnt
2. portability of gettimeofday to Win32




--
Unsubscribe mailto:rrd-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-developers-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-developers
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi