[rrd-users] Re: program to monitor apache access log?

[Jan L. Peterson]

I have a system that does something similar to this...

Web logs are fed via a pipe to a batcher program.  The batcher, among 
other things, multicasts all of the log data out on a multicast port.  
I have a script that listens to that multicast traffic (we have several 
web servers) and counts things like pageviews, hits, etc.  It updates 
an rrd every five seconds.

I'll attach the metering script, but you'll have to come up with your 
own batching script (it's not releasable at the moment).  The script 
expects to receive log data that looks like this:

site access www2 165.230.46.168 - - [02/May/2002:14:17:03 -0600] "GET /js/city-list.fhtml?loc=NJ HTTP/1.0" 200 4245 "http://www.flipdog.com/js/loc.html?_requestid=731741" "Mozilla/4.77 [en] (Windows NT 5.0; U)" "www.flipdog.com"
site access www2 206.53.0.186 - - [02/May/2002:14:17:03 -0600] "GET /images/nav/flipdog_support.gif HTTP/1.1" 200 343 "http://www.flipdog.com/" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)" "www.flipdog.com"
site access www3 152.163.207.46 - - [02/May/2002:14:17:03 -0600] "GET /js/footer.fhtml?step=1 HTTP/1.0" 200 5916 "http://www.flipdog.com/js/loc.html;$sessionid$RKQUSZXLF20D3QFIDAVSFEQ?_requestid=731699" "Mozilla/4.0 (compatible; MSIE 5.5; CS 2000; Windows 98)" "www.flipdog.com"

where the first word on the line indicates that this is site traffic, 
the second word indicates that it is access log information (as opposed 
to error or ssl logs), and the third word tells which web server kicked 
out this log entry.  After those three keywords, it's basically an 
apache extended log format log line.

The script also interactively dumps it's counts for each web server 
every five seconds.  I run it in a screen session that I can reattach 
to if I need to look at how the site is performing at the moment.

	-jan-
-- 
Jan L. Peterson
<jlp at smarthome.net>



-- Attached file removed by Listar and put at URL below --
-- Type: text/plain 
-- Desc: meter_rrd
-- Size: 2k (2638 bytes)
-- URL : http://www.ee.ethz.ch/~slist/pantomime/meter_rrd


--
Unsubscribe mailto:rrd-users-request at list.ee.ethz.ch?subject=unsubscribe
Help        mailto:rrd-users-request at list.ee.ethz.ch?subject=help
Archive     http://www.ee.ethz.ch/~slist/rrd-users
WebAdmin    http://www.ee.ethz.ch/~slist/lsg2.cgi