[rrd-developers] Proposing a new instruction for rrdraph: DSN

[Maciek Kolbusz]

Hello List,

I migrated my growing collection of RRD files to MySQL several months ago
and it really reduced the IO load on the data collecting host as well as
improved the speed of drawing graphs. A new problem I faced was that the
length of rrdgraph commands increased due to longer DEF statements - with
sql/ notation on average the length of a single DEF statement is about 250
characters. I have scripts that generate rrdgraph commands to draw complex
graphs with tens or sometimes hundreds of DEFs, CDEFs, GPRINTs for
formatted legends etc. and many times it happened that rrdgraph failed to
generate a picture, because the command was too long (the length limit on
my system is 130k characters). It would help to avoid this issue and
shorten the command length if it was possible to define a DSN (Data Source
Name) and reuse it in DEFs, ie.:

rrdtool graph ... \
DSN:dsn1=sql//mysql/host=
mysql1.example.com/port=3306/dbname=mydb/username=dbuser/password=dbpassword
\
... \
DEF:defA=dsn1/rrdminstepsize=60/TABLE_A/timestamp/value/RRDKeyID='"keyA'":avg:AVERAGE
\
DEF:defB=dsn1/rrdminstepsize=60/TABLE_B/timestamp/value/RRDKeyID='"keyB'":avg:AVERAGE
\
DEF:defC=dsn1/rrdminstepsize=60/TABLE_C/timestamp/value/RRDKeyID='"keyC'":avg:AVERAGE
\
...

It's debatable what elements should be included in DSN and what should be
left for DEF, however I would suggest that DSN only contained host, port,
dbname, dbuser and password, because one database can have multiple tables
with different step sizes, so the in DEFs it would be possible to use
different tables in the same DB referring to the same DSN, but giving
different step sizes and table names.

Another feature that I think would be very useful is reading DSN from
files: store DSN definition in a text file and then point to this file in
the command, ie.:

rrdtool graph ... \
DSN:dsn1=file:///path/to/file \
...

This way DB credentials would be better protected as there would be no risk
that they appear in the commands and later in application logs or elsewhere
in the system where other users could see them.

Please share your thoughts and comments on this feature.

--
Regards,
Maciek Kolbusz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.oetiker.ch/pipermail/rrd-developers/attachments/20140627/a4a52298/attachment.htm