[rrd-developers] RRD PostgreSQL extension

Wed Sep 12 12:06:23 CEST 2012

> Not sure how you are going to handle multiple datasources in a single
> RRD in your trigger, but I'm sure you have an idea.

Anthony Johnson <ansoni at gmail.com> wrote:

> Not sure how you are going to handle multiple datasources in a single
> RRD in your trigger, but I'm sure you have an idea.  Another
> table(datasource) could be used via a foreign key with your trigger
> located there.
Hm... I think this would require a column per datasource, as you have
to distinguish the values.

> 
> That said... I am also inspired:-)
> 
> I am going to create a RRD FUSE FS prototype that will be backed by
> Postgres and also give a unique way to dive into an RRD file:
> 
> # ls
> # _rw_rw_rw_ whatever.rrd
>    drwxrwxrwx whatever
> # cd whatever
> # ls
>    datasource1
>    datasource2
>    datasource3
> # cd datasource1
> now -> 1832488103813  # Symlink current timestamp
> # echo 12345 > now
Nice idea: "the linux everything-is-a-file mantra" for RRD. This could
make BASH scripting very easy.

> 
> Will be nice to see how this works(or fails).  I will base my Postgres
> table off your example so that we can be compatible.  The cool thing
> with this is that you can get the best of both worlds... traditional
> tools, update RRDs from an application using direct DB access... etc.
I never had some luck with my FUSE ideas. But I always failed in some
parts not really related to FUSE.

The example table is far from perfect. And I don't think there will be
a default table layout. The extension just provides the SQL functions.

cheers
fooker

> 
> Fun stuff,
> 
> Anthony
> 
> 
> On Tue, Sep 11, 2012 at 11:32 PM, Dustin Fisch <fooker at lab.sh> wrote:
> >
> > Hm. Also a nice idea.
> >
> > But the benefit I want to from the PostgreSQL extension is to use
> > psql as a network protocol and feeding the RRD database from data I
> > have in my PostgreSQL database.
> >
> > The idea is to have a table with the following columns:
> > id BIGSERIAL,
> > name VARCHAR,
> > value DOUBLE,
> > history RRD
> >
> > now I can use a trigger for the "value" column to update the
> > history on each update automatically.
> >
> >
> > And as almost all programming languages have a PostgreSQL driver,
> > they can would have a network-able RRD interface out of the box.
> >
> > cheers
> > fooker
> >
> >
> > Anthony Johnson <ansoni at gmail.com> wrote:
> >
> > > Just curious... couldn't you simply build or use one of the
> > > existing Database filesystems that work with Fuse?  Then you
> > > could mount a filesystem whose backend is Postgres.
> > >
> > > http://sourceforge.net/apps/mediawiki/fuse/index.php?title=DatabaseFileSystems
> > >
> > >
> > > The postgres FUSE project seems to be dead, but these seem to be
> > > pretty simple to create.  This would mean that all existing tools
> > > such as Cacti would not need any changes and migrating to such a
> > > setup would require only a cp /olddirectory/* /sqlbackeddirectory/
> > >
> > > I would hope that a database backed filesystem would also be able
> > > to take advantage of filesystem caches at the OS-level which
> > > might be a free performance gain.
> > >
> > > Good luck,
> > >
> > > Anthony
> > >
> > > On Tue, Sep 11, 2012 at 7:44 PM, Dustin Fisch <fooker at lab.sh>
> > > wrote:
> > >
> > > >
> > > >
> > > > Tobias Oetiker <tobi at oetiker.ch> wrote:
> > > >
> > > > > Hi Dustin,
> > > > >
> > > > > Yesterday Dustin Fisch wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I am currently thinking about building an RRD extension for
> > > > > > PostgreSQL.
> > > > > >
> > > > > > The extension will provide a RRD data type which represents
> > > > > > a complete database and functions for all the existing
> > > > > > functions of the RRD tool, like create, update, first and
> > > > > > graph...
> > > > > >
> > > > > > PostgreSQL has a nice API for LOB, which allows to store the
> > > > > > whole RRD database using the PostgreSQL file management.
> > > > > >
> > > > > > I already have taken a look to the RRD code. As far as I can
> > > > > > see, there are functions like rrd_create_r(...) which
> > > > > > provides the functionality I need for such an extension.
> > > > > >
> > > > > > The PostgreSQL LOB API currently provides a file descriptor
> > > > > > for the access to the data stored in the LOB.
> > > > > >
> > > > > > Unfortunately, it looks like the RRD API only allows the
> > > > > > usage of a filename.
> > > > > >
> > > > > > Is there any way to use a file descriptor instead of the
> > > > > > filename? Or is there any chance such a patch would be
> > > > > > accepted?
> > > > >
> > > > > if done nicely, I'll be glad to integrate such a patch ... I
> > > > > guess you would have to compile without mmap support ...
> > > > I'm not sure about mmap. I think I have to try it.
> > > >
> > > > The plan for the patch is to refactoring the rrd_*_r methods by
> > > > moving out the real work of the method into something called
> > > > rrd_*_x. The only thing remaining in the original method are the
> > > > calls to rrd_open and rrd_close and the call to the new method.
> > > >
> > > > The new rrd_*_x methods will mirror the interface of there
> > > > rrd_*_r counterparts expect the filename parameter, which will
> > > > be replaced by the already opened rrd_file_t.
> > > >
> > > > >
> > > > > how would this pg integration work, would the rrd data be
> > > > > stored in a blob ? wouldn't this get read from disk entirely
> > > > > on every update and make things rather slow ?
> > > > Right. The idea is to store the RRD data in a BLOB.
> > > >
> > > > But I don't see any performance problem with this as a
> > > > PostgreSQL extension is allowed to work on the file backing the
> > > > BLOB directly using a file descriptor.
> > > >
> > > > This makes no difference to the way RRDtool currently works.
> > > > Maybe this allows to to use the current mmap code, too.
> > > >
> > > > >
> > > > > cheers
> > > > > tobi
> > > > >
> > > > >
> > > >
> > > > cheers
> > > > fooker
> > > >
> > > > --
> > > > http://lab.sh
> > > >
> > > > _______________________________________________
> > > > rrd-developers mailing list
> > > > rrd-developers at lists.oetiker.ch
> > > > https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers
> > > >
> > > >
> >
> >
> > --
> > http://lab.sh

-- 
http://lab.sh
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
Url : http://lists.oetiker.ch/pipermail/rrd-developers/attachments/20120912/b09a68ea/attachment.pgp 