[rrd-developers] Re: status of threaded RRDTool and Perl
peter at stamfest.at
Sat Nov 26 13:58:31 MET 2005
On Sat, 26 Nov 2005, Ole Bjørn Hessen wrote:
> Eric ZYLBERSTEJN <ezylby at yahoo.fr> writes:
>> Implementing the thread safe API is of course possible
>> in XS. It requires a little bit more coding because
>> the API is not as generic as before.
>> Since some parameters are now required and in a
>> specific order, it could break a lot of the present
>> Perl code. One way to circumvent that could be to keep
>> the low level MT implementation in XS and offer a
>> compatible higher layer in Perl which decodes the
>> variable arguments list and calls the lower layer.
> Yes, I came to the same conclusion. But there's a but here. Only a few
> of these functions are thread-safe yet.
> Perhaps the best way is to start on a new XS perl-binding exposing the
> low level function in a new module with different names so it can be
> extended one by one function until it is finished.
> I startet on a RRDts.pm and perl-shared-thread/RRDts.xs binding below
> defining rrd_error_r, rrd_update_r and rrd_create_r, but I'm not sure
> if I want to do more than this.
I just want to shed some light on the initial issues with RRD
thread-safety, maybe this points anybody going down that road in the
When I did the initial thread safety support for RRD, I did look at many
other RRD functions to check if I could make them thread safe as well, but
for the following reasons I couldn't and didn't:
* Most operations other than rrd_update, rrd_error (and rrd_create, to
support the software I needed the thread support for initially) are
operations that _normally_ do not get used that heavily/often, so I
envisioned that most people could live with the restriction to use
those functions in a non-threaded environment only. One very notable
exception to this is rrd_graph, which could not be made thread-safe due
to the next reason.
* Most of the functions in their initial state are/were inherently
non-thread safe, due to the use of third party libraries with unknown
thread-safety status and the use of known non thread-safe library
functions. For those cases where both apply, I did not try hard to take
care of solvable issues.
* I did think of RRD language bindings (I am using RRD with perl in one
project), but at that time perl wasn't multithreaded by itself, so there
was no immediate need for this.
One (seemingly) possible solution for perl would be to introduce a kind of
"Big RRD lock" (like ye olde Big-Kernel-Lock in linux) and protect all
currently non-thread safe operations with it. However, this does not solve
the problem when this code gets embedded in a multi-threaded environment,
because RRD code might then get in the way of other code using non-thread
safe library calls. True, there shouldn't be any such code in a
multi-threaded environment anyway, but think of using a second perl module
that does the same dirty trick....
The only real solution would be to re-evaluate all the RRD functions to
check for their current thread-safety status. Maybe the used 3rd party
libraries have become thread-safe in the meantime and thus some more RRD
functions can be _r'ed without much trouble.
Another possible solution that comes to mind would be to use a helper
process (not thread) that does all the dirty RRD work. It would be
single-threaded and non-thread-safe operations would simply be queued for
the helper, but this scenario seems rather complex (but doable). If
anybody wants to try this, it might make sense to make that helper generic
enough so that it can be used for multiple language bindings. This helper
would over time be uses less and less, until everything RRD has been made
Unsubscribe mailto:rrd-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help mailto:rrd-developers-request at list.ee.ethz.ch?subject=help
More information about the rrd-developers