A R U N

INSTALL AND CONFIGURE STATISTICS FOR LOAD BALANCED IIS SITES-

Ingredients-

* In this scenario I’m going to use the following things-

# 3 NLB nodes – 2K31 (IP- 192.168.1.129), 2K32 (IP- 192.168.1.130), 2K33 (IP- 192.168.1.131)

# A server for stats installation – 2K34 (IP- 192.168.1.132)

# Statistics – AWStats (download from http://awstats.sourceforge.net). I used AWStats 6.8 version.

# Website for testing – www.a-r-u-n.com (must be setup in all the above 3 NLB nodes)

# Stats website in 2K34. I’m going to use server’s IP itself for testing.

Preparation-

1. Install AWStats 6.8 in the server 2K34 in the following location – C:\AWStats.

2. During installation it prompts you to specify the location of apache installation. Specify ‘none’ since this one is for IIS-

—–> Running OS detected: Windows

—–> Check for web server install

awstats_configure did not find your Apache web main runtime.

Please, enter full directory path of your Apache web server or ‘none’ to skip this step if you don’t have local web server or don’t have permission to change its setup.

Example: c:\Program files\apache group\apache

Apache Web server path (‘none’ to skip):

> none

3. Next step is it will ask to create a new config file for website, specify ‘y’-

—–> Need to create a new config file ?

Do you want me to build a new AWStats config/profile

file (required if first install) [y/N] ? y

4. It then asks you to specify the name of the website for config file. Specify the website name-

—–> Define config file name to create

What is the name of your web site or profile analysis ?

Example: www.mysite.com

Example: demo

Your web site, virtual server or profile name:

> a-r-u-n.com

5. Press Enter to complete the installation of AWStats.

6. Create the data directory called ‘Data’ under the following location C:\AWStats\ and assign modify permission to the following objects – Network Service, web user(IUSR).

7. Now open the config file ‘awstats.a-r-u-n.com.conf’ located under ‘cgi-bin’ folder (in my case – C:\AWStats\wwwroot\cgi-bin).

8. Search for ‘LogFile’ section. You need to specify the path where log files are created for the website.

Note – Since this configuration is for websites hosted in Load Balanced servers the format for ‘LogFile’ differs from the usual log format. There are two options available to generate logs for websites-

a. Copy all the log files from log folders of a-r-u-n.com from all the 3 nodes to the location in local server. In this case, I copied from \\2K31\C$\Domains\a-r-u-n.com\logs\W3SVC100\*.log to C:\Logs\a-r-u-n.com\2K31\*.log for 1st node. \\2K32\C$\Domains\a-r-u-n.com\logs\W3SVC190\*.log to C:\Logs\a-r-u-n.com\2K32\*.log for 2nd node. And \\2K33\C$\Domains\a-r-u-n.com\logs\W3SVC150\*.log to C:\Logs\a-r-u-n.com\2K33\*.log for 3rd node. Now all the 3 nodes’ log files for a-r-u-n.com have been copied to the location in local server – C:\Logs\a-r-u-n.com\ in 2K34. Now we need to merge all the above 3 nodes’ log files into one file using a perl tool called logresolvemerge.pl which ships with AWStats installation. This can be found in C:\AWStats\tools\logresolvemerge.pl. To merge all the log files to a single file, perform these steps-

1. Goto command prompt and to C:\AWStats\tools\ and type the following command –

perl logresolvemerge.pl C:\Logs\a-r-u-n.com\2K31\ex*.log C:\Logs\a-r-u-n.com\2K32\ex*.log C:\Logs\a-r-u-n.com\2K33\ex*.log > C:\Logs\a-r-u-n.com\all.log

2. Now you will see all the nodes’ log files have been merged to a single log file at C:\Logs\a-r-u-n.com\all.log

3. Now specify this log file path ‘C:\Logs\a-r-u-n.com\all.log’ under the ‘LogFile’ section in ‘awstats.a-r-u-n.com.conf’. This will looks like-

LogFile=”C:/Logs/a-r-u-n.com/all.log”

b. Second option is to specify UNC share path in config file. Note down the UNC share path of all the nodes for a-r-u-n.com (\\2k31\C$\Domains, \\2k32\C$\Domains and \\2k33\C$\Domains). Create a map networked drive for the above UNC shares. Say ‘G’ drive mapped to \\2k31\C$\Domains and ‘H’ drive mapped to \\2k32\C$\Domains and ‘I’ drive mapped to \\2k33\C$\Domains respectively. Now you can access those log files directly by G:\a-r-u-n.com\logs\W3SVC100\*.log for 2K31 server and H:\a-r-u-n.com\.. for 2K32 server and so on. Now open the config file ‘awstats.a-r-u-n.com.conf’ and look for ‘LogFile’ section. Specify the following-

LogFile=”perl C:/AWStats/tools/logresolvemerge.pl G:/a-r-u-n.com/logs/W3SVC100/ex*.log H:/a-r-u-n.com/logs/W3SVC190/ex*.log I:/arun1.com/log/W3SVC150/ex*.log |”

By specifying the above format in config file of a-r-u-n.com AWStats generates stats automatically during update the process. This is most suited in the environment where the log files are too large and you need to save space. Also this is time saver.

o Note – First option is time consuming and requires human intervention for exporting each and every logs. Where as second option is easier and can be specified only in config file and stats generation is automated during rebuild process.

9. Now you need to specify the log format for your website. Simply copy the format to config file from the top of the web log file(exmmddyy.log) if you not sure about the exact LogFormat of your web sites. (Note – Websites in each nodes must have same ‘Logging options’ enabled in IIS, this can be confirmed by right clicking web site properties > Logging Properties > in General and Advanced tab make sure that particular website in all the nodes have same settings)-

#Software: Microsoft Internet Information Services 6.0

#Version: 1.0

#Date: 2010-08-31 14:39:06

#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken

Copy the fields so that it looks like this-

LogFormat= “date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken”

10. Please specify DirData path in ‘DirData’ section as C:\AWStats\Data which you created in step 6 so that AWStats will store the result of stats analysis in AWStats files.

11. Now you need to update the stats in order to view it as GUI from web browser. To do this goto command prompt to the location – C:\AWStats\wwwroot\cgi-bin and key in the following command-

perl awstats.pl -config=www.a-r-u-n.com –update

You will see the result like this-

C:\AWStats\wwwroot\cgi-bin>perl awstats.pl -config=www.a-r-u-n.com -update

Create/Update database for config “./awstats.www.a-r-u-n.com.conf” by AWStats version 6.8 (build 1.910)

From data in log file “perl C:/AWStats/tools/logresolvemerge.pl G:/a-r-u-n.com/logs/W3SVC100/ex*.log H:/a-r-u-n.com/logs/W3SVC190/ex*.log I:/arun1.com/log/W3SVC150/ex*.log |”…

Phase 1 : First bypass old records, searching new record…

Searching new records from beginning of log file…

Phase 2 : Now process new records (Flush history on disk after 20000 hosts)…

Jumped lines in file: 0

Parsed lines in file: 1560

Found 43 dropped records,

Found 0 corrupted records,

Found 197 old records,

Found 1320 new qualified records.

Make sure you see new qualified records at the end. If there’s any problem or error please verify you have followed previous steps correctly.

12. Stats have been generated for your website. Now we need URL to view the generated stats. Please create a website in IIS mapped to AWStats ‘cgi-bin’ directory (ie, C:\AWStats\wwwroot\cgi-bin).

13. Call the website from browser – http://192.168.1.132/awstats.pl?config=a-r-u-n.com

14. You are almost done. To generate stats automatically on daily or hourly basis you need to automate it by using Windows Scheduled Tasks. Create a batch file under cgi-bin folder of AWStats and specify the following command – perl awstats.pl -config=www.a-r-u-n.com –update

And finally create a scheduled task to schedule the batch file every 6 hrs or 12 hours.

15. Add sugar to taste… sorry add websites.