[rrd-developers] Anybody working on this? / Proposal

[Peter Stamfest]

Hello,

I'm currently thinking about the possiblity to extend the "reach" of 
rrdgraph. When dealing with many different sources of data for RRD 
files, there is the problems that they might be dispersed over a large 
number of systems.

When one wants to use data from different machines in a single graph, 
this becomes a problem:

  - it is possible to use rrdcached to do graphing for every set of RRDs 
on a single server
  - rrdtool itself can deal with graphing from ordinary files
  - using files from different rrdcached instances and local files 
cannot be expressed easily, IIUC.

One solution could be to use NFS to allow distributed access to RRD 
files. I'm not a big fan of NFS though, so I am thinking about something 
else.

I propose to allow for the specification of different styles of 
<rrdfile> in DEF arguments to rrdgraph.

Something like this already exists for libdbi (using sql// as a prefix).

So I would like to allow for

    rrdcached//tcp/<HOST>/<PORT>/<PATH>

and

   rrdcached//unix/<SOCKET>//<PATH>

or something better (also see below).


Remarks:

- This would require to extend the FETCH command in rrdcached.
- For better performance, a "packed/binary" format might be useful in 
the output of "FETCH" - the protocol already should allow for this
- It would be better to allow for full URL syntax, but the use of ":" as 
the delimiter in DEF makes that problematic
- To allow for full URLs, we could introduce "UDEF" with a different syntax:

   UDEF:variable=<URL>

where <URL> might be

     file://<rrdfile>?ds=<ds-name>&cf=<CF>&step=<step>&....
     socket://<HOST>:<PORT>/<rrdfile>?ds=....
     unix://<SOCKET-PATH>/<rrdfile>?ds=....
     sql://<I DUNNO>

- Unix sockets make it hard in both cases. When it comes to a unix:// 
url and the <rrdfile>, a path lookup for every leading path might be 
required, <rrdfile> becoming the "path info" in url-speech. This might 
be fixed by requiring a double "/" between the socket and the path.


Proposal:

  - extend the FETCH rrdcached command
  - implement UDEF (requires some URL handling stuff, but this could be 
done using regexps)
  - add support code in the local rrd_fetch code similar to that for 
libdbi specs

Comments?



peter