[smokeping-users] error when slave connects to the master

[Fligor, Debbie]

We finally solved this problem, it was all in fast cgi.  Either a default value changed, or we had just been lucky. Here are the details in case anyone else happens to hit this problem.

This site: http://foaa.de/old-blog/2010/11/php-apache-and-fastcgi-a-comprehensive-overview/trackback/index.html  was a big help

we needed to use -idle-timeout parameter on "FastCgiExternalServer"   ours is now set to 300, and it seems to be working.

if the idle timeout is too short, it cuts out the end of the headers, and you hit this from the FastCGI FAQ ( http://www.fastcgi.com/docs/faq.html#incomplete_headers ):

FastCGI: incomplete headers (d bytes) received from server "s"

The FastCGI application, s, didn't terminate the headers properly. A total of d bytes were recieved.

Headers are terminated by an empty line, e.g.

    printf("Content-type: text/html\r\nStatus: 200 OK\r\n\r\n");

See the CGI specification for more information.


-debbie


On Sep 17, 2012, at 17:01, G.W. Haywood <ged at jubileegroup.co.uk> wrote:

> Hi there,
> 
> On Mon, 17 Sep 2012, Debbie Fligor wrote:
> 
>> A few weeks ago our sysadmins patched our RHEL6 box that our
>> smokeping master runs on.. Since then none of the slaves have been
>> able to report.
> 
> I hate it when that happens. :(
> 
>> I get this error in our logs:
>> [Mon Sep 17 12:54:31 2012] [error] [client xxx.xxx.xxx.xxx] FastCGI: comm with (dynamic) server "/services/netmetrics/public_html/http/smokeping/smokeping.cgi" aborted: (first read) idle timeout (30 sec)
>> [Mon Sep 17 12:54:31 2012] [error] [client xxx.xxx.xxx.xxx] FastCGI: incomplete headers (0 bytes) received from server "/services/netmetrics/public_html/http/smokeping/smokeping.cgi"
>> 
>> but when I run this on the command line, or web to it with a browser, it works fine with no errors:
>> 
>> /services/netmetrics/public_html/http/smokeping/smokeping.cgi
> 
> Sounds like the master can connect to itself but the slaves are having
> problems.  I'd sniff the network traffic with tcpdump to a file for a
> while, and then run Wireshark on the file.  Something like
> 
> tcpdump -i eth0 -s 0 -Uw - 'tcp port 80' > /home/debbie/tcpdump.out.`date +"%Y%m%d%H%M%S"` &
> 
> I'd run tcpdump on both the master and a slave.  I'd hope that you
> have your system clocks all in sync to within a fraction of a second
> using ntpd so that you can compare the logs. :)
> 
>> ... here's the packages that they upgraded in case anyone knows of
>> anything in one of those: ...
> 
> My guess is that something else has been changed too.
> 
> --
> 
> 73,
> Ged.

-- 
-debbie
Debbie Fligor, n9dn       Lead Network Engineer, CITES, Univ. of Il
email: fligor at illinois.edu          
"Every keystroke can be monitored. And the computers never forget."