[rrd-users] Re: Weird field name

[Mark Smith]

Everything everyone else said about restricted character set still 
applys here.

I had a similar problem (getting DS names which had invalid characters, 
but still needing to store them.)  I had the added complication of 
having variable names that were too long.  (The same documentation also 
states that a DS name can't be more than 19 characters.)

My solution was to hash the DS name and store the numeric value of the 
hash as the DS name.  The numeric value of a 64bit number (in HEX) is 16 
characters long and of the set [0-9A-F] which is easy for RRD to deal with.

NOTE:  This is _NOT_ the Perl concept of a Hash Table.  This is totally 
different.  This is more akin to a Cryptographic Hash, but we don't need 
many of the characteristics of a Cryptographic Hash.

This kinda assumes you have wrappers you can use to perform the hashing, 
but it works.

I used a 64bit CRC (from CPAN, String::CRC), but I'm not real confident 
that I don't have collisions (haven't checked yet.  :)  If you can deal 
with the performance hit, you could use something with a more even 
output distribution such has MD-5, but that's a bit overkill for what 
we're doing here..

WARNING:  Much Math/Crypto Geeking to follow:
When selecting your hash, make sure it's large enough;  if you assume a 
random output distribution (which is not safe with CRC, BTW), the 
Birthday Paradox tells us that you'll have a 1 in 2 chance of a 
collision with only the square root of the number of total possible 
outputs.  That is, with a 64bit hash (eg: CRC64), you have a total of 
2^64 possible outputs.   If you collect (2^64)^.5  (the square root of 
2^64), or 2^32, or about 4 billion random outputs, you'll have a 1 in 2 
chance of a collision.

1 in 2 is a really bad ratio.  You want something much smaller than that 
(for my purposes, I _REALLY_ don't want a collision, so a 1 in 65536 
chance was good enough.), so if you expect to be hashing 4 billing 
things, you'll want a much larger hash output.

END MUCH MATH/CRYPTO GEEKING.  :)

Hope this helps!

-Mark

Philip Molter wrote:

> I have an automated process running.  This process creates RRD files.
> One of the RRD files has a DS named 'Load-1'.  It has no problem
> creating and storing data into this DS.  However, when I go to fetch
> data from this file, I get:
> 
> can't parse DEF 'Load-1=file.rrd:Load-1:AVERAGE'
> 
> I can't seem to escape the hyphens (which I assume are the problem),
> and it's kind of important that the virtual name and the DS name be the
> same.  How do I work around this?
> 
> * Philip Molter
> * DataFoundry.net
> * http://www.datafoundry.net/
> * philip at datafoundry.net
> 
> --
> Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
> Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
> Archive     http://www.ee.ethz.ch/~slist/rrd-users
> WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi


--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi