[rrd-developers] RRDCacheD - Client rewriting path

[Thorsten von Eicken]

Sebastian Harl wrote:
> *Personally*, I'm not sure about the benefits of being able to
> transparently hide the daemon in a local setup (i.e. keeping a
> consistent behavior in regard to file names when talking to the daemon
> or accessing the files directly). The reasons for that are two-fold:
>
>  a) RRDCacheD is (mostly) for large setups which (imho) will not benefit
>     from that transparency. So the benefits would be limited to a small
>     number of users only, while (again imho) bringing along some
>     negative aspects as well (see, e.g., (b)).
>
>  b) Introducing that transparency probably will encourage mixing direct
>     and "cached" access to RRD files. This is error prone (e.g. cached
>     updates will basically be lost after a direct access using a later
>     time stamp), which is especially bad when it happens accidentally
>     (which, in turn, is not unlikely given the many, transparent ways to
>     use or skip the daemon; e.g. think about the environment variables,
>     etc.). I could perfectly understand any user who's seriously annoyed
>     by that.
>
> Then again, I'm thinking of RRDCacheD as a network daemon and, thus, it
> should imho be optimized for that. It might not be perfect yet, but it's
> already very well suited for that purpose and I'm pretty sure that's
> where we're going to.
>
> There's a lot of "imho" in the above text, so it'd be nice to hear some
> other voices on that topic but from previous discussions I got the
> impressions that there are other people thinking similar to me.
>
> I hope this could clarify my point of view.
>   
I totally concur. We use collectd and rrdtool in production, monitoring
thousands of machines and I rather not know how many rrds. We have tens
of servers running to collect all the data and can't wait to deploy
rrdcached. We've tried but ran into a bug with flushing (it didn't do
it) which we still need to reproduce cleanly. We desperately want a full
networked daemon so we can aggregate across arbitrary monitored servers
whose rrds end up on different rrdcached servers. Security is a
low-priority issue because (a) the data is not all that sensitive to
start with and (b) all this is behind firewall, plus (c) the server ids
used by the data collection are not easy to map back to something
meaningful. The one feature we will need is the ability to fetch data
from rrdcached. This is necessary to aggregate across multiple rrd
stores and also for performance. We compute alerts from the data in rrds
so there is a constant fetch stream going on, not just when someone is
actively looking at graphs.

Please take the above as encouragement, not as "request". I responded to
Sebastian's request for input so I provided it. I don't have manpower at
the moment to contribute so I would normally refrain from stating what
I'd like to see.
Cheers and thanks for a great software!
Thorsten - CTO RightScale