[rrd-developers] RRDCacheD - Client rewriting path

[Tobias Oetiker]

Hi Thorsten,

Today Thorsten von Eicken wrote:

> Sebastian Harl wrote:
> > *Personally*, I'm not sure about the benefits of being able to
> > transparently hide the daemon in a local setup (i.e. keeping a
> > consistent behavior in regard to file names when talking to the daemon
> > or accessing the files directly). The reasons for that are two-fold:
> >
> >  a) RRDCacheD is (mostly) for large setups which (imho) will not benefit
> >     from that transparency. So the benefits would be limited to a small
> >     number of users only, while (again imho) bringing along some
> >     negative aspects as well (see, e.g., (b)).
> >
> >  b) Introducing that transparency probably will encourage mixing direct
> >     and "cached" access to RRD files. This is error prone (e.g. cached
> >     updates will basically be lost after a direct access using a later
> >     time stamp), which is especially bad when it happens accidentally
> >     (which, in turn, is not unlikely given the many, transparent ways to
> >     use or skip the daemon; e.g. think about the environment variables,
> >     etc.). I could perfectly understand any user who's seriously annoyed
> >     by that.
> >
> > Then again, I'm thinking of RRDCacheD as a network daemon and, thus, it
> > should imho be optimized for that. It might not be perfect yet, but it's
> > already very well suited for that purpose and I'm pretty sure that's
> > where we're going to.
> >
> > There's a lot of "imho" in the above text, so it'd be nice to hear some
> > other voices on that topic but from previous discussions I got the
> > impressions that there are other people thinking similar to me.
> >
> > I hope this could clarify my point of view.
> >
> I totally concur. We use collectd and rrdtool in production, monitoring
> thousands of machines and I rather not know how many rrds. We have tens
> of servers running to collect all the data and can't wait to deploy
> rrdcached. We've tried but ran into a bug with flushing (it didn't do
> it) which we still need to reproduce cleanly. We desperately want a full
> networked daemon so we can aggregate across arbitrary monitored servers
> whose rrds end up on different rrdcached servers. Security is a
> low-priority issue because (a) the data is not all that sensitive to
> start with and (b) all this is behind firewall, plus (c) the server ids
> used by the data collection are not easy to map back to something
> meaningful. The one feature we will need is the ability to fetch data
> from rrdcached. This is necessary to aggregate across multiple rrd
> stores and also for performance. We compute alerts from the data in rrds
> so there is a constant fetch stream going on, not just when someone is
> actively looking at graphs.
>
> Please take the above as encouragement, not as "request". I responded to
> Sebastian's request for input so I provided it. I don't have manpower at
> the moment to contribute so I would normally refrain from stating what
> I'd like to see.

:-) well 1.4 will probably only bring part of the sollution for
you. Also note that the fetching will cause the daemon to be
flushed which kills the performance gains ... the sollution here is for
the daemon to support updatev (there is a plan on how todo this but
it is also for 1.5) then you get the data values as they are shoved into the
RRAs right which allows you todo alerts without killing performance ...

regarding the flusing problem, could you test this again, as I
think there were some fixes in this area.

cheers
tobi


> Cheers and thanks for a great software!
> Thorsten - CTO RightScale
>
> _______________________________________________
> rrd-developers mailing list
> rrd-developers at lists.oetiker.ch
> https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers
>
>

-- 
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch tobi at oetiker.ch ++41 62 775 9902 / sb: -9900