[rrd-developers] implementing portable format

[kevin brintnall]

On Sun, Nov 02, 2008 at 12:42:13PM +0100, Sebastian Harl wrote:
> Well, I don't really have any questions about the current implemen-
> tation. I was just going to bring up a few ideas about how to get a
> correct status even when using the BATCH mode but I did not know what
> has been discussed on that topic so far.

Response in-line.

>> * expose BATCH mode to API/bindings?
>>   (for my setup I wrote my own protocol speaker and talk directly to the
>>   daemon for high update rate)
>> 
>>   + enables higher update rates
>>   - client won't get return codes
> 
> I suppose, the goal is to be able to run a series of commands in a row
> with a high update rate but _not_ to be able to continuously run
> commands forever.

Correct, one must get results eventually.

> Just some random ideas how to implement this (as part of the API and
> presumably the daemon / underlying protocol as well):
> 
>   * For each command that should be supported in BATCH mode, add an
>     appropriate BATCH_* command.
> 
>   * Add a new command BATCH_FINISH (or something like that) that tells
>     the daemon that the current job is finished. Now, the client will
>     block until all commands are done and the return code / values are
>     available.

All commands are executed when received.  I don't see a reason to change
that since the command execution part is very fast.

BATCH mode implies success.  It only returns messages in case of error.
This reduces the amount of server->client communication that's required.

>     - BATCH_* returns a handle identifying the command. BATCH_FINISH
>       then returns the appropriate handle followed by the return values
>       of the command. This happens for one command at a time in random
>       order (presumably the order in which the commands have been
>       finished). The daemon needs to cache those return values until
>       they can be sent to the client. A special handle might be used to
>       indicate the end of return values.

You may as well just return the result code directly in this case, instead
of a handle.  Note the current RRDTool API (i.e in Perl):

  for ($x=0; $x<100; $x++)
  {
    RRDs::update("$x.rrd", "N:1", "N:2");
    my $error = RRDs::error;
  }

How do you propose to return a meaningful error string for "1.rrd" before
proceeding to "2.rrd"?  If the answer is "wait for the daemon to give it
to you", then that's no different from non-BATCH mode.  Otherwise, the
update for "2.rrd" must necessarily be sent before a result for "1.rrd" is
available.

I don't think a change to the public API is reasonable for 1.4.. so no
async callback with results.  How, then, does the client get the results
for "1.rrd" before sending the update string for "2.rrd"?

If the API isn't changing, I don't see a reason to change the protocol.
Any async changes on the client should be able to be handled by the
current protocol.

What do you think?

-- 
 kevin brintnall =~ /kbrint at rufus.net/