[rrd-developers] Multi-Threaded JNI code

[Robert Halstead]

Hello all,
After a couple of weeks battling rrdtool and JNI, I finally come out on
top.  I'm fairly new to the java community and really new to rrdtool, but we
are developing a writer for cacti in java trying to do multi-threaded rrd
writes and thought I might share my experiences.

First off, I didn't choose java, I took the project from a co-worker of mine
and he is a java nut so I decided to keep with java.  Anyways, for the JNI,
I used Sas Markovic's RRDjTool original code for a basis on my rrdjtool and
really just updated his code to use the multi-threaded rrdtool functions.
While I left his duplicate(), getTokens(), and releaseTokens() alone,  I
modified the create and update to reflect the added parameters that update
and create need.  Now, this is my first release, so only update and create
have been done.  I plan on doing the others, just that my work only needs
these two and I don't have the free time to work on the others right now.
If anyone else wants to finish it go right ahead ^_^.

/**************************************
 * Function: Java_com_bresnan_cacti_archiver_RRD_nativeCreate
 * Arguments: JNIEnv*, jobject, jstring, jlong, jlong, jint, jobjectArray
 * Returns: jstring
 * Modified By: Robert Halstead < rhalstead at bresnan.com> (Jul 13, 2006)
 * Author: Sasa Markovic < sasam at dnseurope.co.uk> (Feb 17, 2003)
 * Summary: Creates an RRD File using the rrd library calls in librrd_th.
 */
JNIEXPORT jstring JNICALL Java_com_bresnan_cacti_archiver_RRD_nativeCreate
    (JNIEnv *env, jobject obj, jstring jFilename, jlong start, jlong
pdp_step, jint argc, jobjectArray jArgv)
{
    char **tokens = getTokens(env, jArgv, argc);
    char *output;
    char *rrd_error;

    const char *str = (*env)->GetStringUTFChars(env, jFilename, 0);
    char *filename = duplicate(str);
    (*env)->ReleaseStringUTFChars(env, jFilename, str);

    time_t last_up = start;
    rrd_clear_error();
    struct rrd_context *rrd_ctx = rrd_get_context();
    int status = rrd_create_r(filename, pdp_step, last_up, argc, tokens);

    // Not sure if rrd's error functions are working properly yet. A lot of
the code was commented out.
    /*if (rrd_test_error_r(rrd_ctx)) {
        rrd_error = rrd_get_error_r(rrd_ctx);
        output = duplicate(rrd_error);
        rrd_clear_error_r(rrd_ctx);
    } */
    if (status != 0) {
        output = duplicate("There was a problem creating the rrd file.");
    } else {
        output = duplicate("OK");
    }
    releaseTokens(tokens, argc);
    return (*env)->NewStringUTF(env, output);
}

One thing that i'm not sure of is if the error functions are multi-threaded
yet and are working properly.  I looked through rrdtool's rrd_error.c and
notice quite a number of commented lines in the code.  If they are, will
someone let me know please?  We sure could use em when things go wrong =)

Also, when passing in argc and argv, if no one knew this, all argv needs is
the DS and RRA definition lines in the create command line.  You don't need
the filename or --start or --step since those are directly passed.

Update is the same way:

/**************************************
 * Function: Java_com_bresnan_cacti_archiver_RRD_nativeUpdate
 * Arguments: JNIEnv*, jobject, jstring, jstring, jint, jobjectArray
 * Returns: jstring
 * Modified By: Robert Halstead <rhalstead at bresnan.com> (Jul 13, 2006)
 * Author: Sasa Markovic <sasam at dnseurope.co.uk> (Feb 17, 2003)
 * Summary: Updates an RRD File using the rrd library calls in librrd_th.
 */
JNIEXPORT jstring JNICALL Java_com_bresnan_cacti_archiver_RRD_nativeUpdate
    (JNIEnv *env, jobject obj, jstring jFilename, jstring jTemplates, jint
argc, jobjectArray jArgv) {

    char **tokens = getTokens(env, jArgv, argc);
    char *output;
    char *rrd_error;
    const char *str = (*env)->GetStringUTFChars(env, jFilename, 0);
    char *filename = duplicate(str);
    (*env)->ReleaseStringUTFChars(env, jFilename, str);
    str = (*env)->GetStringUTFChars(env, jTemplates, 0);
    char *tmplt = duplicate(str);
    (*env)->ReleaseStringUTFChars(env, jTemplates, str);

    rrd_clear_error();
    struct rrd_context *rrd_ctx = rrd_get_context();
    int status = rrd_update_r(filename, tmplt, argc, tokens);
    // Not sure if rrd's error functions are working properly yet. A lot of
the code was commented out.
/*    if(rrd_test_error_r(rrd_ctx)) {
        rrd_error = rrd_get_error_r(rrd_ctx);
        output = duplicate(rrd_error);
        rrd_clear_error_r(rrd_ctx);
    }*/
    if (status) {
        output = duplicate("There was a problem updating the rrd file.");
    } else {
        output = duplicate("OK");
    }
    releaseTokens(tokens, argc);
    return (*env)->NewStringUTF(env, output);
}

Again, I wasn't sure about the error functions.  Also, argv only contains
the "N|timestamp:val[:val...]" and tmplt contains "ds-name[:ds-name...]".
Since we are using --template for every call, I kind of suited this to our
needs.  Not very generic I know.

Ironically, despite our attempts to make this thing multi-threaded, we're
still going down to one thread somewhere because we're seeing no difference
in the amount of time between single thread and multi-threaded operations.
Which leaves me to ask, is RRDTool's multi-threaded functions really just
locking it down to one thread access at a time?  I've checked my java code
up and down to make sure I wasn't using any functions that lock threads, but
I'm not noticing any.

Can anyone comment on these functions?

I have no problems with adding these to the current rrdjtool and uploading
them to the listserv.  I don't know how to do that yet, and since it's not
complete, I figured that these patches would work for now.  If anyone wants
the full thing though, send me an email and I'll email the whole thing to
you =)

-- 
"A fool acts, regardless; knowing well that he is wrong. The ignoramus acts
on only what he knows, but all that he knows.
The ignoramus may be saved, but the fool knows that he is doomed."

Robert Halstead


--
Unsubscribe mailto:rrd-developers-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-developers-request at list.ee.ethz.ch?subject=help
Archive     http://lists.ee.ethz.ch/rrd-developers
WebAdmin    http://lists.ee.ethz.ch/lsg2.cgi