[rrd-users] Re: Who has the most RRD files (or data sources)?

[Peter Valdemar Mørch]

Steve Friedl steve+rrd-users-at-unixwiz.net |Lists| wrote:
> On Mon, Mar 20, 2006 at 12:35:30PM -0500, Mark Plaksin wrote:
> 
>>And there are no sequential XML parsers?  And thus XML is only good for
>>small amounts of data?  I know little about XML but find this hard to
>>believe :)
> 
> 
> You'd be right: there are two broad categories of XML parsers:
> 
> DOM - load the whole thing in to a tree
> SAX - stream parser with callbacks
> 
> DOM is useful if you need to manipulate everything as a whole, but it
> sucks everything into memory at once. Not really great for huge trees.
> 
> SAX gives callbacks at the various points in the process, so if you
> need to just pick out one part of the tree, it can be done with no
> more memory than just the intersting parts.
> 

I didn't even bother considering a DOM parser for the huge dump files. I
tried running the dump files through perl's XML::Parser which behind the
scenes is James Clark's expat library.

Memory consumption was not high, but IIRC parsing one of our "typicial
rrd files" took 30 secs with XML::Parser, without doing *anything* with
the parsed data.

Parsing and extracting the structure only took 6 seconds with regexps -
so I dropped the XML parser. But yes, if Tobi decides to change the
output of rrdtool dump while still adhering to the same DTD, we'll be
vulnerable to that.

Peter

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-users
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi