[rrd-users] Re: COUNTER to DERIVE

[David Koski]

Dave,

	Just to clarify...  I'm moving from COUNTER to DERVIE with rrd-min
set to 0 to remove any spikes that occur in the couter wrap code due to
huge counter wraps, router reloads, etc.  Since RRD tries to compensate
for the counter roll instead of just throwing out the data (AKA Concord
Network Health's way of dealing with a counter roll), I need some way to
just ignore any spike that could occur in the RRD counter wrap logic.
Since folks both here and on the cricket list seem to lean twards the only
"neat" way to get rid of such a spike occurance is to move to DERIVE with
an rrd-min of 0 to ignore any negative deltas on the graph (Without having
to worry about anyones buggy implementation of SNMP), seems about the best
way I can get around having to constantly explain weird spikes to
customers :)

Also to clarify my mixture of Gauge and Counter explanation.  Currently
I'm using 32 bit traffic counters for anything under 12 megs (Cisco
routers), and anything over 12 megs I'm using Cisco ifloc integers.
Works for now, but since the ifloc integers are really unsupported and
Cisco claims they are going to eventually disappear (?), so we will
eventually move everything possible to a 64-bit ifHC counter, however I'm
having some strange problems with HUGE spikes occuring on every sub
interface (ATM, FrameRelay and channelized interfaces) all at the same
time (AKA Peta bits per second).  I've not been able to blame anything as
of yet.  However, I'm not trying to compensate for the 64-bit problem at
this point only the once in a while occurance of a huge spike in a 32-bit
graph.  The unfortunate part is I have about a years worth of data stored
in RRDs at this point and would like to maintain that data that would not
shift to a 64-bit counter (Non-backbone customer router, I.E. Cisco 2501
running 11.x)

	David


------------------------------------------------------------------------
UMNet                         MichNet                         Merit RSNG
------------------------------------------------------------------------
dkoski at umich.edu                       			    Technologist
Unix Administration                                	  Technical Lead
Phone: (734)/647-8993                          Network Operations Center
	          	  U of M Information Technology Central Services

On Fri, 29 Dec 2000, Dave Plonka wrote:

>
> David,
>
> Let me preface this by saying that I might not have a deep
> understanding of DERIVE, since I don't use it.  I've just read the
> description in the "rrdcreate" man page.
>
> On Fri, Dec 29, 2000 at 09:08:10AM -0500, David Koski wrote:
> >
> > I'm about ready to take the plunge and re-do my RRDs from there intial set
> > to COUNTER to DERIVE as set about a year and a half ago.
>
> I'm curious what sort of data it is that you are collecting.
>
> It seems to me that you wouldn't want to convert something that had
> been stored as a COUNTER type (such as a contiuously increasing SNMP
> counter) to a DERIVE type cuz your going to get a huge anomaly in the
> graph every time the counter wraps around.  Presumably the value will
> drop dramatically at those times, and DERIVE will record a huge
> negative delta value, no?
>
> If you're saying that what you were collecting shouldn't have been a
> COUNTER in the first place, should it have been a GAUGE instead?  I can
> see, as Tobi describes in the "rrdcreate" man page, why one might wish
> to convert a GAUGE type to a DERIVE type.
>
> > Is there a
> > simple way to do this since COUNTER and DERIVE really are the same minus
> > the Counter roll logic?  So far the only suggested way I have is to unload
> > the data, recreate the RRD and reload the data, but for several thousand
> > RRD files, this would be a mess frankly.  Is there any other way suggested
> > way?  I noticed rrd tune has an option to alter the data-source type, but
> > don't know if this would be wise on an already populated RRD (Its still
> > messy as well but more manageable than unloading and reloading), the other
> > problem is writing a script to detect its a COUNTER and does the change,
> > since I have several hundred GAUGE RRDs as well...
>
> An alternative would be to use dump and restore, and perl or something
> to intelligently edit the files in between:
>
>    $ rrdtool dump file.rrd > file.xml
>    $ perl -pi -e 's/COUNTER/DERIVE/' file.xml # do some intelligent edit
>    $ mv file.rrd file.rrd_backup              # save the original!
>    $ rrdtool restore file.xml file.rrd
>
> About manipulating RRD data in general, using dump and restore is the
> key.  Between those operations, you can use an XML parser to manipulate
> the raw data.
>
> I like to work in perl and there is a reasonable XML::Parser for perl,
> and other modules layered atop, like XML::Simple.  I have hacked
> XML::Simple to work with the XML from "rrdtool dump".  You can find
> that modified module embedded within this "log2rrd" utility:
>
>    http://net.doit.wisc.edu/~plonka/log2rrd/
>
> which tries to convert MRTG log files to RRD files.
>
> Since I wrote log2rrd, Tobi developed other ideas about how best to
> convert from old MRTG log files, so my util is now defunct - perhaps
> now just an example of how one can manipulate RRD files using XML in
> perl.
>
> One last disclaimer - of course you can do major damage to RRD files by
> manipulating their raw values.  No warranties.  Test! Test! Test! after
> you make changes.
>
> Dave
>
> --
> plonka at doit.wisc.edu  http://net.doit.wisc.edu/~plonka  ARS:N9HZF  Madison, WI
>



--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi