[rrd-developers] Re: Posible bug/unexpected behaviour on templated update

[javier at rediris.es]

Hello again,

although it was a little bit harder than expected, I got it.
The patch might be too long for the list, so it's available
for the curious at
http://ftp.rediris.es/pub/rediris/software/stats-collector/rrdtool-partial_upda
te.patch
The patch is constructed in a way that doesn't change the behaviour
of 'rrdtool update', and the patched parts only play when is called
as rrdupdate.

I'll explain again our motivation and how we get the patch.

We get a set of related values (a verbose output of netstat command) to store
into RRD's at a remote location. And the whole string is too long for our
message passing system. So, we split in smaller pieces. Instead of wait for
all pieces to arrive and then update RRD's, we favour feeding the values when
they arrive, and thus we collide with the fact that template update sets to
unknown the other datasources. As most of the values are counters, that's
a disaster, first solved splitting the RRD's into smaller pieces also. This
produces minor problems, so I decided to pursue template based update which
doesn't touches the not updated datasources.

The first trial stops when I realized that the consolidation of the
datasources was done for all of them at the same time (and that's the way
it must be), so I started another path.
First, I use a free element of pdp_prep.scratch array to store the values
that rrd_update.c stores in pdp_temp[]. This allow me to update only a
subset of the data sources without interfering the remaining ones. As
a side effect, I added a command line option to jump over consolidation
(--no-rra), and moved the calculation of consolidated points into a function
(consolidate_rra). In this way I use rrdupdate without consolidation
many times, and I issue a real consolidation when the last piece of values
has arrived.

To make it slightly more flexible, I've also incorporated my initial attemps,
allowing some kind of non-syncronous data feeding. To achieve this, I use
another pdp_prep.scratch element to store the time when that data source was
last updated (instead of the live_head.last_up). This kind of updating is
not very useful, as the updating can not cross a consolidation time, but most
of the work was done, and I incorporate it as a kind of exercise.

On the first steps, I attempted to use both elements of the unival type of the
scratch[] elements, but then stopped when realized that there is some strange
effect that produces interaction among the float and integer parts of the 
union.
After discovering, I thought it was some mis-programming I was done, but later
I realized that was not the case. A printf before and after the assignment
showed that the stored value changed. And inspection of other rrd files that
have never known of my patches showed things like
ds[Ipkts].minimal_heartbeat = 3600
ds[Ipkts].minimal_hrbt_valu = 1.0526763434e-314
where minimal_hrbt_valu is the float part of the corresponding unival.

In my opinion, that is not a problem of the rrdtools code, but It happens with
a variety of cpu/os/arch. I'm quite sure that I've tested with
solaris8 sparc gcc 2.95.3 and redhat 7.2 AMD-K7 gcc-2.96, and probably with
immunix 7.0 PIII egcs-2.91.66-StackGuard

And that's all.

								Javi


--
Unsubscribe mailto:rrd-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-developers-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-developers
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi