[rrd-users] rrdtool, python and threads

[Yiannis Vavouranakis]

> > Hello!
> > 
> >  
> > 
> > I'm completely new to rrdtool AND mailing lists, so please bear with me.
> > 
> >  
> > 
> > Since I don't know whether this has been answered before, and I don't
know
> > how (or if) I can search the mailing list archives, I'm posting a
question. 
> > 
> >  
> > 
> > Here's my situation:
> > 
> >  
> > 
> > I have a series of resources I'm monitoring (nothing to do with traffic
> > etc). I've already figured out what DS and RRA I need. I'm planning on
> > updating the relative RRDatabases (1 file per resource) via a python
script.
> > So far, I've created a bash script that calls the same python script
over
> > and over again with different command line arguments each time, so that
I
> > collect data from each resource, i.e. my bash script is like this:
> > 
> >  
> > 
> > #!/bin/bash
> > 
> > python get_the_resource.py resource1
> > 
> > python get_the_resource.py resource2
> > 
> > python get_the_resource.py resource3
> > 
> > python get_the_resource.py resource4
> > 
> > .
> > 
> >  
> > 
> > The python script itself is quite I/O intensive: The whole bash script
takes
> > about 15 seconds to run for 6 resources monitored. I've also tried one
> > single resource (i.e. running one of the above lines on its own) and saw
> > that it took consistently about 2 seconds, most of which was used up
merely
> > for fetching data from the web.
> > 
> >  
> > 
> > So I'm guessing that if I make a threaded python script that sends each
> > request in its own thread, I'll have a more effective program.
> I'd just use the main thread to update the rrd. Use worker threads to 
> fetch the data and pass it to your main thread via a queue. This way 
> only one thread has the rrd opened.
>
> hth
>   Paul
Hello.

Thanks for the answer Paul. 

Well, actually, since each resource has its own rrd file, abd I have one
thread per resource, then only one thread will have the rrd file open for
writing, and maybe another one will be reading from it to create a graph, so
I don't see the need for a queue here. Unless the python wrapper is entirely
thread unsafe...

Regards,
Yiannis Vavouranakis