[rrd-developers] rrdcached performance problem

[kevin brintnall]

> spends all of its time (more than 99??%) in ???realloc???. So in consequence
> 37.1??% of the time it spent in ???handle_request_update??? the daemon is
> actually waiting for ???realloc???. This is (to me) very unexpected and a
> schoolbook example of ???measure before you optimize???.

Florian et al,

I'd rather see the realloc in linear chunks vs. exponential.. i.e. when we
run out, allocate another X pointers.  We'll still reduce calls to
realloc(), but without risking exponential jumps in memory utilization.

i.e. we could add cache_item_t->values_alloc to keep track of how many
values have been allocated.  Then, we can realloc() a new chunk on demand.
This chunk size could even be configurable..  a clever admin could
pre-allocate all the pointers (i.e. "-w"/rrd_step).

P.S.: is_power_of_two = old_size && !(old_size & (old_size-1));

-- 
 kevin brintnall =~ /kbrint at rufus.net/

> +  if (old_size > 0)
> +  {
> +    is_power_of_two = old_size;
> +    while ((is_power_of_two & 0x01) == 0)
> +      is_power_of_two >>= 1;
> +  }
> +
> +  /* If the current size is a power of two (or zero), increase the array size.
> +   * Otherwise enough space has been allocated previously. */
> +  if (old_size == 0)
> +    new_size = 1;
> +  else if (is_power_of_two == 1)
> +    new_size = 2 * old_size;
> +  else
> +    new_size = old_size;
> +
> +  /* We need to allocate more space. */
> +  if (new_size != old_size)
> +  {
> +    char **tmp;
> +
> +    tmp = realloc (*dest, new_size * sizeof (char *));
> +    if (tmp == NULL)
> +      return (ENOMEM);
> +    *dest = tmp;
> +  }
> +
> +  /* Duplicate the string. On error, the array size is not decreased. This
> +   * means the callee may have to free `dest' even if this function failed. */
> +  (*dest)[old_size] = strdup (src);
> +  if ((*dest)[old_size] == NULL)
> +    return (ENOMEM);
> +
> +  *dest_size = old_size + 1;
> +  return (0);
> +} /* }}} int add_strdup_exp */
> +
>  static int count_lines(char *str) /* {{{ */
>  {
>    int lines = 0;
> @@ -1433,7 +1486,7 @@ static int handle_request_update (HANDLER_PROTO) /* {{{ */
>      else
>        ci->last_update_stamp = stamp;
>  
> -    if (!rrd_add_strdup(&ci->values, &ci->values_num, value))
> +    if (add_strdup_exp (&ci->values, &ci->values_num, value) != 0)
>      {
>        RRDD_LOG (LOG_ERR, "handle_request_update: rrd_add_strdup failed.");
>        continue;
> -- 
> 1.6.3.3
>