Exploratory research is carried out by businesses to find out their competitive position in the market and the general service needs of their customers. This knowledge is needed to understand how e-services fit within a customer's overall service experience, and needs, and how the Web might be used to meet those needs and create competitive advantage.
This type of research is the most general "testing" required. It sets the scene. It may involve pulling together general research already done by the business and user surveys done by research companies. A word of caution, however. Businesses may find that current research does not adequately factor in the Web as a service medium, and it may be difficult to create linkages between their own customer data and that given by general user surveys. Also bear in mind that current user surveys tend to provide quantitative, rather than qualitative, data on Web usage. Quantitative data may not be enough to give businesses an idea of their customers' attitudes and behaviors.
If businesses end up having to do exploratory research from scratch, they should remember it is just that—exploratory. Businesses want to get a broad view of the opportunities open to them. They want to know what makes their customers tick in e- space and what customers want them to provide there. Businesses should be gathering information on customers' propensities to use certain e-services alongside other service channels and their basic requirements in doing so.
Businesses and their research companies should not be afraid to test the boundaries of customer usage and adoption to find out how customers may, or may not, respond to new e-service and Web site concepts. But also remember that it can be difficult for customers to envision themselves using new technology and techniques, particularly where they would need to modify or develop new ways of doing things, and you may not be able to rely on their view of their intended behavior. However, customers will be able to tell you about the things they want to do as customers, and from there it's up to you to apply technology for their benefit.
Also, businesses should avoid engaging research companies that come back and tell them things people hate about the Web in general because, given the amount of writing and discussion that has been had about the weaknesses of the Web and the way people feel about them, this is stating the obvious. Businesses should engage research companies that can look at customers' unique reactions toward a business' particular e-service offering.
Understanding what people hate, and like, about the Web is only a background to the testing businesses and research companies need to do.
Remember that the information businesses gather will provide a useful steer for both them and the development company throughout the life of the project and beyond.
2.. Concept Testing
Concept testing can occur at different times within the life of a project. It is required early on to help a business define its e-service offering and later on when design concepts have been developed by the Web site designer.
E-service concept testing
A business will have an idea (possibly based on exploratory research) of what customers want and it will consider its business goals and capabilities alongside customers' to develop a concept for its e-service offering. This concept will support the business' e- service strategy. At this stage, the business can go to customers to test the viability, and appeal, of its proposed e-service offering.
It's a matter of bouncing ideas off customers to see if you're heading in the right direction with your e-service offering. At this stage you're not concerned with putting visual concepts in front of customers; it's too early for that. You are concerned with creating
"doing scenarios" that customers can relate to—they put themselves in the position of doing certain things and tell you what they think about being able to do it and how much they like doing it that way. And they'll tell you what you're missing.
Concept testing also gives you the opportunity to prioritize what you will deliver to customers. Of all the things the customer wants to be able to do on the Web site, what is most important? Understanding this will help you decide where to direct your energies and what to develop first.
Web site concept testing
When coming up with a proposed solution, the Web site designer will start with a concept for your Web site. For example, the Web site might be based on a particular "metaphor"
that helps customers relate their knowledge of another object or experience to what they can, and need to, do on your Web site. The shopping cart metaphor is an example of a common metaphor for a Web site where customers go about shopping online.
Metaphors can be double-edged swords, and it is a good idea to test them on customers before basing a whole system around them. The risks, and opportunities they present are discussed further in Chapter 6. You will quickly find out whether or not a metaphor conjures up the right associations with customers.
There are other fundamental ideas that your Web site will be based on: the system by which customers get around (the navigation system) and the navigation devices (such as icons) used to help them on their way, the type of content provided, and the tools
provided to support particular e-customer activities. These may be tested at a high level, but chances are they will also be subject to a prototyping process where customers get to interact with physical examples of different ideas.
3.. Prototyping
Let me preface this section by saying that prototyping is one of the most confusing terms commonly used; everyone seems to have a slightly different interpretation of what it means.
From what I have observed and read, there seem to be two basic approaches:
1. Prototyping components. Concepts such as navigation, content, and tools are prototyped as stand-alone components or objects.
2. Prototyping slices of the "system." This is where a formative part of the Web site is tested along a particular thread. Concepts such as navigation, content, and utility will form part of that thread.
Both of these approaches can be employed at a high level when development is only rudimentary or later when development is more advanced.
Prototyping components
This approach may be useful to the Web site designer as they seek to "sanity check" their ideas early on in development. Navigation is something that can be presented in a fairly abstract way and tested as a "stand-alone" system, for example. (Of course, navigation is not truly stand-alone and should also be tested in the context of the entire Web site system. However, a stand-alone test will provide feedback on whether the navigation system makes sense to customers and fits with the mental model they use when doing things.)
As an example, imagine you are testing the highest level of the basic navigation system used on the Amazon.com, reputedly one of the Web's most successful online stores. We want to present the "essence" of Amazon's navigation system, or its most simple way of working, to see if it makes sense to customers and can be used by them.
The navigation system would be abstracted and would be presented without other Web site components such as graphics or content. Using either simple interactive screens, or paper-based mock-ups, the test facilitator would walk through the flow of the navigation system with customers.
In testing the highest level of Amazon's navigation system, the customer would move between the home page and each of the next-level pages. Of course, your test could be broader than this and include more layers and dimensions of the navigation system.
Figure 3-4 shows the representation of the Amazon home page followed by the "Books"
page at the next level in the navigation system (received as a result of clicking on
"Books" on the top navigation bar on the home page).
Figure 3-4. Testing the highest level of Amazon's basic navigation system.
The test facilitator would be interested in finding out things like:
• How well the navigation system reflects what the customer expects to find on the Amazon site, and why.
• What customers expect to receive as a result of each interaction with the navigation system, and how this compares with what they do receive and their relative level of satisfaction.
• How well customers are oriented as they move around the navigation system: i.e., if they are aware of where they are, where they've been, and where they're going next.
• How comfortable customers are in using the navigation system: i.e., if they avoid making common errors or misjudgments and if they use the system intuitively without explanation.
Prototyping slices of the system
This approach may be useful to businesses that need to ensure the Web site system will deliver the required experience to their customers. By testing customers' reactions to a critical slice of the system, they get an idea of customers' reactions to the whole system.
In my experience, this approach works best when slices are interactive because customers can experience something naturally (as they would in real life), making their feedback that much more meaningful.
As an example, imagine you are testing the basic search functionality on the Amazon site.
Given that customers go to Amazon to browse for, and purchase, books, CDs etc., the search function is critical to customers who may know exactly, or roughly, what they are looking for.
This critical "slice" could include the search entry screens, customer entry of search criteria, and the search results generated from those criteria. You would be interested in finding out things like:
• How customers use the search functionality, how they expect it to behave, and their relative level of satisfaction.
• How well the search function allows customers to search against their search criteria (such as keywords they naturally expect to produce search results).
• The general usefulness of search results and how well they meet customer expectations.
You could give Amazon customers a list of items for which they can search without giving away the "rules" that you want them to intuitively work out for themselves. In addition, providing a list of search items will allow you to develop the test slice against a manageable dataset. Too often "Search" does not get tested until development is complete (if at all) because it is seen to be dependent on a complete database.
Figure 3-5 shows how Amazon's search function, as a critical slice of the Web site system, might be represented.
Figure 3-5. Testing Amazon's basic search system.
Fundamentals
Regardless of your approach to prototyping, the underlying premise is customer participation. This is the point in the project where you can put something reasonably tangible in front of customers and find out how well it fits with what they need to do as customers. To do this, we need to show enough of the system for customers to relate it to their intended actions. And this can be tricky in the early stages of development.
However, some people are happy to prototype earlier on and have developed techniques that work in that context. Some of these techniques, such as paper-based prototyping and
"walkthroughs," are covered later under "Testing Techniques."
Prototyping is also fundamentally different from usability testing. However, the two often get confused or combined. Given the rapid development of many Web sites, prototyping and usability testing often get rolled into one. Prototyping is about the fit with customer intentions and actions; usability is about whether the Web site is usable. While some usability testing includes exploration into customer behavior and expectations, it is important to realize, however, that usability doesn't necessarily equate to prototyping.
Whatever you want to call your testing, make sure you do test your Web site's customer behavior and its usability.
4.. Usability Testing
Having just discussed the difference between prototyping and usability testing, we will define usability testing as the task of measuring users' performance against set tasks to ensure that the tasks can be done with ease and efficiency.
Usability can come late in the development project when fine-tuning of the visual interface is required and earlier in the development process as part of the Web site designer's "sanity checking".
When testing usability late in development, the structure and organization of the Web site should be determined already. Such usability testing should not have to deal with last- minute structural problems. At this late stage in development you can't fix structural problems with band-aids, and designers should not be faced with this task. Structure can be fixed only with restructure. For this reason, concept testing or prototyping should ideally occur before final usability testing.
Ten usability "heuristics"
The previous icon intuitiveness test is an example of usability testing. In his Web site useit.com (see Netography) Jakob Nielsen gives ten heuristics that can be used as benchmarks in usability testing. These heuristics have become pretty well-known, so, rather than just regurgitate them, I have put my spin on them, as follows:
1. Visibly provide feedback on what the system is doing when it's doing it, or shortly thereafter.
2. The system should adopt real-world language and concepts to be natural and logical to the customer.
3. Give users the freedom to do and undo tasks.
4. The system should consistently follow its own standards.
5. Make the system error-proof.
6. Lead the customer with dialogue and don't expect customers to retain information on behalf of the system.
7. Cater for the experienced and novice user by providing shortcuts for frequently performed actions.
8. Provide only the information required by the user; "fluff" just gets in the way.
9. Help users recognize, diagnose, and recover from, errors.
10. Provide customer-centric help online.
Nielsen based these heuristics on a large volume of usability problems. In heuristic evaluation, you would rate the Web site's success against each of the above heuristics.
5.. Beta Testing
While testing occurs during development, beta testing is required in the live environment once development is complete and before the Web site goes public. This is the only time customers have to experience the fully functional Web site before its launch and a business' last chance to ensure that, overall, customers have a rewarding and quality experience.
Beta testing is the final, customer-driven, quality assurance for the Web site. Inevitably, this testing will retest the concept, interactive site components and usability. It also tests the interface with a business' internal systems (if any) and site performance in a live environment. It is important to keep testing focused on specifics so that changes can be meaningfully defined in terms of system modifications and process improvements.
Additionally, customer reactions will help businesses sanity check their ideas on positioning and promoting their Web sites.
By the time businesses do beta testing they should know they've got things roughly right.
After all, they don't want to redo things at the last minute. Chances are businesses will also need to prioritize changes and decide what must be changed before the site goes public, and what can occur later as part of ongoing development modifications and improvements.
Internal Testing
Businesses develop Web sites to meet business and customer objectives. Testing should ensure that these objectives can be met (and the reconciliation of the business' and customers' needs should have been taken into account in concept definition and testing).
Internal testing is the process by which a business can judge development against
business objectives. This can run parallel to customer testing, or a company may prefer to fully test internally before testing with customers (however, this can lead to protracted testing timeframes).
In addition, businesses and Web site developers should always avoid putting a
substandard product in front of customers; this aggravates them and gets in the way of meaningful testing. Irrespective of whether or not internal testing is carried out before external customer testing, some sort of quality testing is always required before Web sites, and their components, are seen by customers.
We will examine different types of testing that businesses will complete internally before going public with any online offering (either a completely new Web site or any changes to an existing Web site):
1. Acceptance testing 2. Quality testing
1. Interactivity testing 2. Integration testing 3. Compatibility testing 4. Security auditing 5. Performance testing Acceptance Testing
There must be formal process whereby the business signs off the Web site before it is accessible to the public. This sign off is formal acceptance of the product delivered by the Web site developers. Of course, the business sign off will be hugely influenced by
feedback from customers, but there also needs to be an internal process of testing and feedback.
To keep this process manageable, it is best to define acceptance testing in terms of meeting agreed Web site objectives. Criteria that realize these objectives need to be clearly established and benchmarked against by business stakeholders who are able to focus on evaluation against their area of responsibility and not become would-be Web designers. Of course, this is easier said than done. Guidelines and scripting can help focus internal testers, and this technique is covered later under "Testing techniques."
Quality Testing
At the same time, business sign off will be contingent on internal quality testing. The business must be sure that the Web site system functions the way it should, as a customer interface and in relation to internal systems and processes. To achieve this surety, the system must be subjected to:
1. Interactivity testing 2. Integration testing 3. Compatibility testing 4. Security auditing 5. Performance testing 1.. Interactivity testing
Interactivity is the means by which customers get around a Web site and receive the information they are looking for. To ensure a quality customer experience, all interactive objects, such as external and internal links and navigation devices must, on interaction,
return the expected response. The system should also return expected responses, such as appropriate error messages, when the customer does the "wrong" thing.
Also, some interactivity, such as calculators or decision support tools, provide customers with intelligent responses. It is a good idea to identify these intelligent components and test them as stand-alone pieces of functionality. The tester should be provided with the rules regarding inputs and associated outcomes. Testers must ensure that every input and action generates the right output. This can become difficult if the logic behind the
functionality has not been clearly defined, and this situation should be avoided. Any process of input-process-output must be defined according to some rules, and these can be captured in test scripts.
Too often, businesses try to define rules at the same time as testing functionality. This usually results in confusion over what is being tested and results in a poor-quality product.
If a business does not know outcomes relative to customer inputs, how are customers supposed to make any sense of the results they receive? If interaction promises an intelligent response, customers will expect it. It is better not to promise this type of intelligent interactivity than to provide something that is only half thought through.
2.. Integration testing
Some Web sites interface with a business's internal systems and processes. An online order process, for example, interfaces with internal systems receiving the order and the systems and processes used to process the order and provide the product.
The interface between a Web site and internal systems must deliver customers with a seamless, end-to-end service experience, whether it be ordering, making an inquiry, requesting information, or activating a service.
The flow of each process should be known and documented, and tests should be carried out to trace customer actions and requests through these processes to their logical conclusion. (The mapping of such processes is discussed in Chapter 5 under Process Improvement). If a business already tests parts of these processes, then only those parts affected by the Web site interface may need to be tested.
3.. Compatibility testing
Different development languages can be treated in different ways by different browsers.
Early on in the development process, the business and the Web site development company should agree on which browsers must support the Web site. Testing should determine that the Web site runs as expected on each appropriate browser, and browser version.
Different operating systems and different machines will also render Web pages
differently. Again, the necessary user configuration should have been agreed on early in