[rrd-users] RRD files with more than one data source

[Alex van den Bogaerdt]

On Mon, Aug 27, 2007 at 12:37:09PM +0300, Yishai Hadas wrote:

> >From looking at the documentation and the code it seems that both update
> and graph (uses internally fetch) commands always use explicitly or
> implicitly all DS defined in a given RRD file.

Update: yes. Fetch: yes.

Graph: you don't have to use it, but internally rrdtool will still
fetch it.

> When using update it seems that even if <-template> option is used -
> still unknown value is set to any non mentioned DS. 

Indeed. If you don't tell RRDtool, then it's unknown.


> When fetch is used there is no way in the API to relate to a specific DS
> and all are fetched.

I believe this to be true. Use awk, sed, perl, or any other tool you like.

> If I'm correct:
> 
> It seems reasonable to put some DS in same RRD file just if those DS are
> really updated together and may be used together to build graphs.
> 
> Otherwise to use different RRD files - Each RRD file per DS.

Indeed, sounds reasonable.
 
> But on the other hand:
> 
> What about disk space when many files are used?

Each file will have a little bit of overhead, instead of one file
having this little bit of overhead.  Nevertheless, the majority of
your precious diskspace is used by data, no matter if it is in one
file or in many.

The more RRA rows you have, the less significant this overhead becomes.


> What about performance as each update requires open and closes the file
> - while bulk update of different DS in same RRD file can be done in one
> update call?

What about memory used when rrdtool fetches lots of data from one single
file, and you use only one or two DSes from it?

Same answer you get: it depends.

Is memory short or is diskspace?
Does it take much time to open each file or not?
Does the system have to use swapspace to fulfill your request or not?

> Am I correct in my understanding the APIs?

I think you do. At least a lot better than some newbies.
Thanks for RTFM.

> Is it recommended to define some DS on same RRD file? 

Depends

> If yes when it's recommended?

When, after careful consideration, it is better.

My rule of thumb: if data belongs together and is processed together,
it ends up in the same file.  Think ifInOctets and ifOutOctets.

Data sources which cannot be updated together do not belong in the
same file, because as you know this results in unknown intervals.

100 DSes which are updated together but aren't usually graphed together,
may also need to be in separate files.  This is a performance issue
and sometimes I just don't need to care, sometimes I do.


Look in this maillist's archives. Lots has been written about this
subject, including huge setups with many similar problems to solve.

HTH
-- 
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/