[rrd-users] Re: VISIONARY: New tool to consider to create: a "DynamicData Set" tool...

[BAARDA, Don]

G'day,

Sounds like the biggest problem is you are going to have bucket-loads of
data...storage may be cheap, but searching through and analysing that data
is a headache. Are you really sure you want to record byte counts for every
single src-dest pair, on every port, for every 5 minutes/hour/day/year
whatever? And then you want to highlight the busy ones? Fun :-)

> -----Original Message-----
> From:	Chris Snell [SMTP:chris at bikeworld.com]
> Sent:	Thursday, November 23, 2000 1:05 AM
> To:	Jakob Ilves
> Cc:	rrd-users at list.ee.ethz.ch
> Subject:	[rrd-users] Re: VISIONARY: New tool to consider to create: a
> "DynamicData Set" tool...
> 
> 
> 
> On Wed, 22 Nov 2000, Jakob Ilves wrote:
> 
> > Well, if we extend the scope from not just destinations for traffic, I
> want the
> > tool to produce graphs with the same information as those produced by
> the
> > Netmetrix product, but better (of course ;-).  Netmetrix provides you
> with
> > statistics for a link such as graphs showing the distribution of:
> > 
> >    * protocol usage during the day
> >    * top talkers during the day
> >    * top listeners during the day
> >    * top conversation during the day.
> > 
> 
> I think I'd go with a mix of Tobi's suggestions and your suggestions.  I'd
> use a SQL database to store such things as total bytes in/out for a
> particular destination.  More specifically, I'd store these totals for
> each day.  I think it would be easier to determine "top talkers for
> today" by making a SQL call than it would be to query 10,000+ RRD
> files.  For the actual data measurements, I'd grab them every 30 seconds
> and store them in individual RRDs, as Tobi suggested.  Yes, it's a lot of
> files and disk usage but, hey, disks are cheap.
> 
	I was going to suggest the same thing, but why not whack the whole
RRD database into the SQL database? That way it's all contained in the one
place, and you don't have the filesystem overheads. You would add whatever
data you needed to search on (ie talkers for last 5 mins/hrs/days/whatever),
and then you could pull the RRD database out to create graphs or more
detailed analysis. This way you would get the SQL searching and storage
features for the huge number of DS's, and you would get RRD's efficient
storage/accumulation over time. If you find you need to do searches on
fields you didn't originally cater for, at least you have _all_ the data in
the RRDs, so you can do slow exhaustive searches. If the data you want to
search on can be incrementaly updated by your data collector, you can then
add those fields and initialise them from the existing RRD data for future
searches.

> I'd roll my own data collector that could store the collected data in the
> RRDs but would also update the SQL tables with the current "total" counts.
> 
	I'd add that it needs to pull the RRD's out of the database to
update them, then shove them back in.

	ABO

--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi