[rrd-users] Re: RRD-Performance Issue - Need Help

[Martin Sperl]

Hi!
We are currently updating 15000  rrd-files every 5 minutes, and we have 
not found any problems, except for required IO/memory usage.
But this also may have to do with that our scripts that fetch the data 
are written in perl and do not fork awk/rrdtool/... processes but do 
everything with perl and RRDs means.

So I would propose, that you are switching to perl or another language 
(pha,python) , that has a native rrd implementation, you can use.

In my experience the snmpwalk tool from the netsnmp toolset does not do 
bulk SNMP requests when it walks the tree, but only fetches 1 value at a 
time, which also includes higher Network and CPU overhead and latencies.

This definitely helps a lot, but we still have got a performent machine 
to do this. (enough CPU and Memory to keep most of the files in Cache)

Still we came accross the problem that we need ever more performent 
machines to fullfill our automatic graphing requirements.
One option that I have now also implemented as a patch to rrdtool, is to 
have the data not in separate rrd files but store everything in a mysql 
db and directly graph with rrdtool from the database. In there we 
already have 56000 different datasources and the IO-penalty is no longer 
when adding data, but  when graphing data... But you will not have a 
look at 56000 graphs every 5 minutes, so you gain there again ;)

Ciao,
          Martin

Fabien Wernli wrote:
> On Tue, Jun 20, 2006 at 12:38:20PM +0200, Alex van den Bogaerdt wrote:
>   
>>> | awk '/whatever/{print $2}' in.txt
>>>       
>> How is this different from my:
>>
>>     
>>>  and remove grep as well:
>>>
>>>  >>>   < in.txt awk '/whatever/{print $2}'
>>>       
>> Thank you very much for your contribution!
>>     
>
> For the record I:
>
> 1) missed the end of your post *sorry*
> 2) the difference is small. In one case the shell makes the redirection, in
>    the other awk takes care of opening the file. Performance is the same.
>
> --
> Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
> Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
> Archive     http://lists.ee.ethz.ch/rrd-users
> WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi
>
>   



--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-users
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi