Measuring the effectiveness of a Web site is frequently an overlooked task.
There are several reasons for this. Usually so much effort has gone into the analysis and development of the Web site that there may be little energy or enthusiasm left once it is up and running to properly evaluate its effectiveness.
In addition, the evaluation could easily be postponed because of the argument that the Web site is quickly evolving, therefore there is little point in evaluating now. Of course, there is never an ideal point at which to evaluate the effective- ness of the system, but it should still be done.
The effectiveness of the Web site should be evaluated because it is an investment of organizational resources and should not be treated differently from any other resource. Evaluations should be completed periodically as the Web site develops and matures. Useful information can be obtained from evaluation that can be fed back into the development and evolution process.
Reviews can be conducted whenever the development team feels they are neces- sary. A good time to perform a review of the effectiveness of the Web site is at 3 and 6 months after it has gone online. This should be enough time to identify any immediate problems with the system. Further reviews can be conducted on a 6-month basis.
Senior management should be involved in the reviews to determine if the requirements that were detailed early on in the project are being met by the Web site. It is quite possible that the requirements may change over several months of development. These need to be discussed, and if viewed as critical they should be built into the system as early as it is feasible to do so.
McKeown and Watson [10] define some Web efficiency variables that can be used in determining the overall efficiency of the Web site:
• Number of people with Web access;
• Number of people aware of the site;
• Number of hits on the site (visitors);
• Number of active visitors (visitors who interact with the site);
• Number of purchases;
• Number of repurchases (people who purchase repeatedly).
If only a small number of users are accessing the site, then it may not be the Web site in particular that is at fault but rather how it is being advertised and promoted. This may be rectified by registering the Web site URL with all of the major search engines, by advertising on other sites, and by promoting the site by more traditional methods such as putting the URL on company station- ery and business cards.
Using basic statistics such as the number of Web hits is a very limited way of evaluating a Web site. What is required is a more business-centric approach rather than a technical one that relies on hits and views. For example, North- west Airlines in the United States has the goal of selling as many tickets online as possible [11]. The advantage of selling tickets online is that they are 50 to 70% cheaper per transaction to process than through traditional methods. So the most important measurement for them is the number of bookings made through the Web. A second important statistic is the number of inquiries han- dled through the Web site because every inquiry handled in this way is cheaper to deal with compared to telephone calls made to a call center. Northwest is primarily concerned with increasing revenue and reducing costs and so their metrics reflect this.
Another business metric is the percentage of visitors to a site that actually make a purchase. Just attracting visitors may not actually do a business much good in the long run. It means that the site has to be improved to handle large numbers of visitors and, of course, there is a cost associated with that task.
Unless there is a payback it may be a poor business investment. The site has to be designed to attract the right type of person, ideally one that is likely to make a purchase or interact with the site in some cost-saving way for the organization.
Some organizations that have nothing to sell directly via the Web can monitor how long people stay at the site. Overall, they can try to measure whether the person is actively engaged with the content and whether they return to the site. Employment recruitment sites can monitor the number of inquiries made through the Web site and the number of applications for jobs that are made. Companies can review the effectiveness of the Web site from the perspective of how much revenue is generated from advertising if that is part of their objectives.
Interviews and questionnaires can be used to obtain feedback from inter- nal and external users of the Web site. The results can provide information on the problems users face, what they like about it, and whether the design features are easy to use.
Some of the benefits of the Web site may be very difficult to quantify;
an example would be improved public relations. Surveys can be conducted to determine what effect the Web site is having in such areas. This may mean that external research companies have to be utilized if the expertise to conduct pro- fessional surveys does not exist within the organization. When the metrics are problematic, a company’s Web site can be compared with other companies’
Web sites for functionality, design effectiveness, and level of innovation. There- fore, considerably more work has to be put into obtaining information of these intangible benefits compared with finding out how many new orders have come in online.
References
[1] Arnold, K., and J. Gosling, The Java Programming Language, Reading, MA: Addison Wesley, 1996.
[2] Gosling, J., “The Feel of Java,” Computer, Vol. 30, No. 6, 1997, pp. 53–58.
[3] Flanagan, D., Java in a Nutshell, Sebastopol, CA: O’Reilly and Associates, 1996.
[4] Naughton, P., and H. Shildt, Java: The Complete Reference, Berkeley, CA:
Osborne/McGraw-Hill, 1997.
[5] Java Unleashed, Indianapolis, IN: Sams Net, 1997.
[6] Graham, I., Object Oriented Methods, Reading, MA: Addison Wesley, 1994.
[7] Dietal, H. M., and P. J. Dietal, Java: How To Program, Upper Saddle River, NJ: Prentice Hall, 1997.
[8] Srinivas, K., et al., “Java and Beyond: Executable Content,” Computer, Vol. 30, No. 6, 1997, pp. 49–52.
[9] “Java Commerce: A Business Perspective,” Sun Microsystems White Paper, 1998 (http://java.sun.com/marketing/collateral/bs_perspective).
[10] McKeown, P., and R. Watson, Metamorphosis—A Guide to the World Wide Web and Elec- tronic Commerce, New York: John Wiley & Sons, 1996.
[11] Kirsner, S. (1998). “Beyond the Log,” CIO Business Magazine, 1998 (http://www.cio.
com/archive/webbusiness/100198_main_content.html).