[rrd-users] RRD::update failing on first update every time?

[Simon Hobson]

Jo, please do read to the end. I realise that you may want to switch 
off by the end of the first paragraph, or just explode at my apparent 
arrogance, but do read to the end.


Jo Rhett wrote:
>Darren, I really appreciate that you are trying to help but 
>skepticism about assertions I make about the code are just a waste 
>of time.  You don't know me well enough to realize I don't write in 
>with stuff like this unless my facts are in order, but you shouldn't 
>start your conversation with me assuming I'm an idiot.

Well you haven't been around on this list long enough to realise how 
often it IS the basics that catch people out. You are correct, no-one 
does know you, what you do, or how you do it - so when faced with a 
post that is "a bit light" on details and appears to be suggesting 
there's a bug (which apparently isn't affecting others BTW) then it's 
natural to ask what you are doing and can you simplify it/provide a 
hard coded example.

Your first post doesn't explicitly say you are using Perl - that is 
only inferred by the subject line.

>On Aug 10, 2010, at 8:18 AM, A Darren Dunham wrote:
>>  Unfortunately, you don't show what is actually being run by the system.
>
>15k of source code from my application is going to help you?   And 
>seriously, we're not talking about complicated code.   It's easier 
>to type out an update() call than it was to write the question to me.

The comment is quite valid. From here we have your assertion that 
what is in those debug statements is what is actually being done - 
but until the bit you've grepped below, no actual code. As you've 
ably demonstrated, suggesting that you need to post 15k of code is 
just being facetious.

>  > My first guess is that RRD is correct, but there's a bug in your script
>>  and the first 'rrdupdate' is getting run twice.
>
>That's not possible, since the only place where update is being run 
>is preceded by the debug message you see.
>
>$ grep -B 5 -A 2 update collector.pl
>
>         $DEBUG = "\t\tinsert into $filename values 
>$data->{'timeRecorded_nomsec'}, " .
>                 "$data->{'Ingress_OctetsForwarded'}, " .
>                 "$data->{'Egress_OctetsForwarded'}\n";
>
>         RRDs::update( $filename, '--template', 'ds0:ds1',
>                 $data->{'timeRecorded_nomsec'} . ':' . 
>$data->{'Ingress_OctetsForwarded'} . ':' . 
>$data->{'Egress_OctetsForwarded'}
>         );
>
>>>  And no, I'm not doing an update at any other time, there's only one
>>>  place in the code where update() is called, and it outputs the debug
>>>  message you see here...
>>
>>  Maybe you could wrap the update() call to show the command line?
>
>That's exactly what you were looking at.   Same operands, same 
>order.   See code shown above.

I agree it does look that way. There have been previous occasions 
where people have had a problem, and it turns out the update wasn't 
actually for the value someone thought they were for. And of course, 
we only had your word for what the script was doing.

>The only question you asked which could move this forward was:
>>  Can you recreate this situation with a simple 'rrdcreate' and
>>  'rrdupdate' lines that are runnable by others?
>
>Yes, but building the replication script for this is most probably a 
>waste of time if I'm dealing with a well known 'feature' in 
>RRDtoool.   I've run into them before.
>
>If this really is a new problem (I doubt it) then I'll be happy to 
>produce a replication script.  But first let's start with the 
>obvious.

Lets assess obvious : RRD Tool, used by many people who don't have 
the problem. Unknown code, written by an unknown person, doing 
unknown stuff, and having a problem. Look at it like that, where 
would you say was logical to look first ?

The reason people are asking for a reproduceable script is that it 
makes life a lot easier. It may be that when you write a simple 
script, you find it actually works, or it may not - in either case we 
have a fixed point of reference and the problem has been narrowed 
down a bit.

I don't do Perl, but would this be something like the script to 
reproduce what you originally offered up ?

         RRDs::update( /rrd/vlan-0.0/queue-1.rrd, '--template', 'ds0:ds1',
                 '1281406473:3420012071625:3420012071625'
         );
and so on.

Ie, hard code what your script is supposed to be doing and see what 
happens. Also, go back to the previous update batch which you 
**didn't** include in your original posting. I assume it would be 
fairly trivial to add a line or two of code and have your program 
actually spit out the hardcoded version for you.

If it fails in the same way, then it might well be that you've found 
a bug - and the fixed data should make that bug easier to find.
If on the other hand you find the hardcoded script actually works, 
well then you can go away and see how it differs from what you 
thought you were doing.

In either case, it's then possible to halt at certain points and 
examine what's actually gone into the rrd file itself.


Now, having written all that, I can see a potential issue (but I 
don't know because there isn't enough information in your posts). You 
seem to assume that since the last timestamp is 1281406200 that an 
update for time 1281406473 **must** be acceptable. At this point, if 
you go to Alex's tutorials at http://www.vandenbogaerdt.nl/rrdtool/ 
and in particular read the one on normalisation then you may spot the 
error.

So go back and see what the last update time was for the previous 
update batch - and I suspect you may find it was 1281406473. If so 
then the problem is indeed in your 15k of code.

-- 
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.