[rrd-developers] Distributed rrd files centralized graphing

[Scott Brumbaugh]

Hi rrd developers,

This is a patch for rrdtool to provide for distributed rrd file
storage with graphing done centrally.  This patch is currently in use
with rrdtool 1.2 on centos5 with snmp pollers in remote data centers
writing to their own local rrd storage.  Graphs are produced centrally
including aggregates across all data centers with rrd data retrieved
from across the internet.

Transport between an rrd grapher and rrd store is layered on top of an
rpc protocol.  A grapher fetches rrd data by invoking an rpc which is
serviced by a remote rpc server running on an rrd data
storage/collecting server.

This patch extends the rrd_fetch syntax to allow rpc dispatching to be
specified.  The rrd filename argument to the fetch function can be
passed as rpc//server_alias/rrdfile.rrd, example,

  $> rrdtool xport \
  >    DEF:in1=rpc//server1/ifInUcastPkts.1.rrd:pps:MAX \
  >    DEF:in2=rpc//server2/ifInUcastPkts.1.rrd:pps:MAX \
  >    CDEF:sum=in1,in2,+ \
  >    XPORT:sum

Other than that file syntax change, rrd raph and rrd xport functions
operate the same.

On the graphing host different server_aliases are mapped to rrd
storage servers in a configuration file (rrdrpc.conf)

  Storage-Server.server1=RRDDataFetch:tcp -h remote1.com -p 12345
  Storage-Server.server2=RRDDataFetch:tcp -h remote2.com -p 12345

On the remote rrd storage/collection servers a similar configuration
file is used to setup the rrd rpc server daemon with its endpoint and
rrd file location (rrdrpcd.conf)

  RRD.Base=/var/rrds
  RRDFetchAdapter.Endpoints=tcp -p 12345

This patch is against the current rrdtool 1.2 svn branch and requires
the zeroc ice rpc framework,

  http://www.zeroc.com/download/Ice/3.2/Ice-3.2.1.tar.gz

It is known to build and run on centos5 x86_64 and i686.  This patch
touches autotools files so prior to the first build autoreconf should
be run. With the ice framework installed a confgure build sequence
would look similar to this

  $ autoreconf
  $ ./configure --enable-rrdrpc --with-zeroc-icelib=/ice_install_location
  $ make

After a successful build a test program can be run

  $ make check



Thanks,


Scott Brumbaugh




-------------- next part --------------
A non-text attachment was scrubbed...
Name: rrdtool-rpc-svn.patch
Type: text/x-diff
Size: 36340 bytes
Desc: not available
Url : http://lists.oetiker.ch/pipermail/rrd-developers/attachments/20080522/de4776e0/attachment-0001.bin