No Hair Blog

Setting up awstats with virtual hosts on Snow Leopard Server (Repost from No Hair News)

I first tried the Wordpress.com Stats plugin but recent updates include quantserve.com javascript and has been associated with slow loading of sites, no to mention the privacy concerns. I then used Google Analytics which was cool, but did not allow me the see the ip addesses. In addition, only 50% or so of the visitors were tracked. The others are probably using NoScript, etc. I decide to go back to a local web server log analyzer, awstats. Istalling it on Snow Leopard Server is done manually for clarity.

1. Change logs to proper format:

The standard format for awstats is the NCSA combined/XLF/ELF format which can be enabled in Apache by either:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" combined
CustomLog "/yourlogpath/yourlogfile" combined

or

CustomLog "/yourlogpath/yourlogfile" "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""

However, if you use a front-end nginx proxy, %h just shows you the ip addess of your nginx proxy. You won't get the ip address of the client unless you add:

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

in your server{} block of the nginx proxy. Then you can get this ip address passed to the Apache back-end server and pick it up in the logs with %{X-forwarded-for}i.

So, my log format in the /etc/apache2/sites/000X_site1.conf is

CustomLog "/var/log/apache2/site1/access_log" "%{X-forwarded-for}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""

[%l is the identity of the client determined by identd on the client machine, per RFC 1413. This is unreliable but is part of the standard "combined" log format.

This variation on the combined log format exactly maps to the default awstats type "1" log as defined in awstats.site1.conf (see later).

# LogFormat = 1

or

# LogFormat = "%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"

Do this for every site for which you want to use awstats.

See the Apache docs, the awstats docs, and here (on 10.5 but you'll get the idea).

2. Set up apache 2 log file rotation.

If you enable "Archive Files" for your site using Server Admin, this enables Apaches 'rotatelogs' function. Unfortunately, as each file is archived, a new file is created named access_log.126000000 or something like that. There is a workaround, but I decided to use cronolog as it allows you to create a symlink which always point to the active log file, allowing regular updates of awstats with a simple script or on-the-fly updates to your awstats statistics from a browser.

So, fire up a terminal in you Snow leopard server and:

cd /usr/local/src/
wget http://cronolog.org/download/cronolog-1.6.2.tar.gz
sudo tar xzf cronolog-1.6.2.tar.gz
cd cronolog-1.6.2
sudo nano -w configure

Find the lines:

infodir='${prefix}/info'
mandir='${prefix}/man'

and change them to:

infodir='${prefix}/share/info'
mandir='${prefix}/share/man'

Now, ./configure, make, and make install per usual.

To configure your Apache virtual sites to use cronolog to rotate files, try this:

CustomLog "|/usr/local/sbin/cronolog --symlink=/var/log/apache2/site1/access_log /var/log/apache2/site1/access_log.%m.%Y" "%{X-forwarded-for}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" "

This rotates the logs every month into a log named "access_log.MO.YEAR" while a symlink to the current log is maintained as indicated. You don't need to specify '--period=\"1 month\"' because cronolog rotates the files when the filename changes. You can no longer read your logs in Server Admin but that's the reason for this anyway.

See the Apache wiki for details.

3. Install awstats 7.0.

Download the most recent version of awstats, then put the cgi files in /Library/Webserver/CGI-Executables, the config files in /etc/apache2/awstats/, the static files in /Library/Webserver/Sites/site1, create the folders for the awstats database, and make sure all the ownership and permissions are sorted out:

wget http://prdownloads.sourceforge.net/awstats/awstats-7.0.tar.gz
tar xzf awstats-7.0.tar.gz
sudo cp -a awstats-7.0/wwwroot/cgi-bin/* /Library/WebServer/CGI-Executables/
/* /Library/WebServer/CGI-Executables/ is linked to /cgi-bin/ for all sites */
sudo chown -R root:admin /Library/WebServer/CGI-Executables/*
sudo chmod -R 755 root:admin /Library/WebServer/CGI-Executables/*
sudo mkdir /etc/apache2/awstats
sudo chmod 755 /etc/apache2/awstats
sudo chown root:wheel /etc/apache2/awstats/
sudo mv /Library/WebServer/CGI-Executables/awstats/awstats.model.conf /etc/apache2/awstats/
sudo mkdir -p /var/awstats/site1
sudo mkdir /var/awstats/site2
sudo chmod -R 755 /var/awstats/
sudo chown -R root:wheel /var/awstats
sudo mkdir /usr/local/awstats
sudo mv awstats-7.0/tools/awstats_updateall.pl /usr/local/awstats

sudo cp -a sudo mv awstats-7.0/wwwroot/js /Library/Webserver/Sites/site1

and repeat that last step for /classes, /icon, and /css for each virtual host. Now edit your /etc/apache2/sites/000X_site1.conf.

<IfModule mod_alias.c>
	...
	Alias /awstatsclasses "/Library/Webserver/Sites/site1/classes/"
	Alias /awstatscss "/Library/Webserver/Sites/site1/css/"
	Alias /awstatsicons "/Library/Webserver/Sites/site1/icon/"
	ScriptAlias /awstats "/Library/Webserver/CGI-ExecutablesSites/"
	...
</IfModule>

Now open /etc/apache2/awstats/awstats.model.conf, save as awstats.site1.conf, and change these lines:

LogFile="/var/log/apache2/site1/access_log"
SiteDomain="site1.com"
HostAliases="www.site1.com"
DirData="/var/awstats/site1"
DirCgi="/cgi-bin"
DirIcons="/icon"

Next you need to initially build/update your log files with:

sudo perl /Library/Webserver/CGI-Executables/awstats.pl -config=/etc/apache2/awstats/awstats.site1.conf -update

Now restart apache and try the url "http://www.site1.com/cgi-bin/awstats.pl?config=/etc/apache2/awstats/awstats.site1.conf" and you should see the awstats page in its awesome beauty.

4. Set up a regular awstats update job in launchd.

Here I use Lingon to create a launchd job. Open Lingon and select a new lauchd job. Select it to run as root, name it "com.awstats.updateall" and enter the script:

perl /usr/local/awstats/awstats_updateall.pl -awstatsprog=/Library/Webserver/CGI-Executables/awstats.pl -configdir=/etc/apache2/awstats now > /var/log/awstats/update.log 2>&1

and set it to run daily. You can edit awstats_updateall.pl to include the paths easily enough.

5. Once it's working you can mess around with updating from a browser, securing awstats with passwords or by ip, etc. Good Luck.


Posted by Gordon, No Hair News, Jan 10, 2011

© nohair.net and the author

For comments, corrections, and addenda, email: gordon[AT]nohair.net

Blog | Entries | Tags | Home