The objective of OPAC research is to gather information that enables more effec- tive OPAC systems to be designed and implemented in libraries (Large & Beheshti, 1997).
A national study of user behavior on requirements of existing online catalog sys- tems was conducted by OCLC and sponsored by the Council on Library Resources (CLR). Ten OPAC systems from 10 organizations were compared. After analyzing written documentation and accessing each OPAC system, researchers compared the systems’ capabilities and experiences with human communication. The success of the interaction between users and OPAC systems is determined both by the willingness of users to interact with systems as they normally do with humans and by OPACs, especially the interactive interfaces that facilitate the interaction. There is a need for improved dialogue design (Hildreth, 1982). The OCLC/CLR study agreed with Hayes, Ball, and Reddy (1981) that users of interactive systems are frustrated with their communication experience with these systems because the existing systems are not very good at communicating with their users.
Another national survey of OPAC use, sponsored by the Council on Library Resources (CLR), focused on the interaction between human users and the online catalogs. By analyzing the survey of users and nonusers of online catalogs in the United States, Matthews, Lawrence, and Ferguson (1983) identified six categories that affect the usage of online catalogs: the user, the task, the organizational interface (the library), the online system, the database, and the human-online interface. Subject searching
is a major aspect that affects users’ search satisfaction. The survey suggested that OPACs should implement keyword searching for subjects, and allow users to browse the subject index or thesaurus.
There are a series of review articles on OPAC research. Hildreth’s (1985) com- prehensive review of OPAC research presented an overview of OPAC design, development, and use. In particular, he reviewed OPAC research studies published before 1985. He pointed out that “online library catalogs are a class of interactive information retrieval systems, and user interaction with them is a form of human- computer interaction” (p. 272). In that sense, much of the research on information retrieval and human-computer interaction can be applied to the design and evaluation of OPACs. Hildreth identified problems of OPAC searching revealed by previous research: failed searches, nonrelevant results, high expectations from users, lack of using controlled vocabulary, and so forth. He also suggested further research and design directions. Lewis (1987) reviewed research on the use of OPACs in early online catalog studies, the CLR OPAC studies, and other studies of online catalogs.
He further discussed the implications of these studies for library practice. He argued that current subject retrieval capabilities are not adequate and need to be enhanced even though users like online catalogs.
Seymour (1991) reviewed research methodologies applied in OPAC user studies from March 1986 through November 1989. She concluded users could not understand OPAC structure and subject organization, and they rarely used online and off-line Help. She discussed the problems in applying a variety of methods to OPAC user studies, especially experimental statistical methods. Yee (1991) reviewed the current research on specific user interfaces to online catalogs. She pointed out that in-depth research on user needs of online catalogs is required in order to design effective interfaces for OPACs. Large and Beheshti (1997) provided an overview of OPAC studies since 1990. They focused on the crucial problems of OPAC research: users, library settings, search strategies, systems, and, in particular, relevance. They found that most of the OPAC studies had concentrated on subject searching, a challenge for OPAC users. OPAC systems and their enhancements and developments were also reviewed by O’Brien (1994). In conclusion, it is clear that OPAC studies cover a wide range of areas, from user studies to system evaluation and comparison.
Interaction.Studies
After reviewing the research on interactive IR in OPAC environments, the author identified the following types of interaction studies: 1) user goals and their impact, 2) strategies/behaviors and affecting factors, 3) effect of knowledge structure on
Interactve IR n OPAC Envronments
search success,.4) evaluation studies and usability testing, 5) intermediary studies and their implications, and 6) research methods for interaction studies.
User.Goals.and.their.Impact
User goal is an important component of OPAC interaction studies because it is the objective of IR interaction. It is important to understand user goals and their relationships with both search strategies and information-seeking behaviors. By employing questionnaire surveys, semistructured interviews, transaction logs, and unobtrusive observation, Belkin, Chang, Downs, Saracevic, & Zhao (1990) identified design principles for the third generation OPACs through an understanding of the intentions and behaviors of people as they interacted with information in different types of libraries. They classified the goals, behaviors, and intentions associated with interaction with information, and further identified relationships among goals, behaviors, and intentions. Analyzing the same data, Chang (1995) identified the underlying common dimensions of browsing: scanning, resource, goal, and object.
Based on these four dimensions, she discovered five themes and nine patterns of browsing. Goal is one key dimension that determines the browsing patterns.
Although researchers agree that user goals have an impact on users’ interactions with IR systems, there is a disagreement on whether users change their goals during their interaction with IR systems. Hert (1996, 1997) defined user goal as what a user attempts to accomplish during the interaction. She further developed the no- tion of information-seeking interaction as “situated action” in an investigation of information interactions of OPAC users. After analyzing videotapes, transaction logs, and interviews, the results of her study indicated that user goals were not greatly modified during their information interactions. Users’ actions were not completely predetermined; instead, elements of the situation, such as elements associated with the respondent, elements associated with the problem or project, and elements as- sociated with the system response, affect users’ interactions with OPAC systems.
She further suggested that system design should build on dynamic user models and focus on feedback mechanisms.
Using Daniels’ (1986) classification of goals, Xie (2000, 2002) constructed four types of user goals and eight types of subgoals, interactive intentions based on users’
interactions with OPACs in different types of libraries. However, contrary to Hert’s findings (1996, 1997), she found that users did not change their long-term goals and leading search goals, but they did change their current search goals, which correspond to the user goals, as defined by Hert (1996, 1997), and the interactive intentions in the process of achieving their leading search goals. She identified patterns between interactive intentions and information seeking strategies, and investigated the shifts in current search goals, interactive intentions and information-seeking strategies.
This study demonstrated that interactive information retrieval is the product of plans
and situations, and suggested that the design of interactive IR systems should sup- port multiple types of information-seeking strategies and shifts in levels of goals and types of information-seeking strategies. It is important to further investigate whether users change their different levels of goals for a variety of tasks in different settings, and under what circumstances they shift their levels of goals.
Slone (2002) investigated the influence of user mental models and goals on search patterns during Web interactions, including interactions with OPAC systems..She found that elements of situational goals, along with users’ experience, motivation, and mental models, affected how they searched for information. Different types of user goals led to different search approaches. Users who intended to accomplish job-related or educational goals were more highly motivated than those who tried to fulfill recreational or personal goals. Users with job-related or educational goals used a variety of tools or Web online catalog, or off-line sources depending on their Internet experience. Users with recreational or personal use goals conducted more searches by serendipity.
There are still unanswered questions about user goals. For example, what are the relationships among user goals, tasks, and problems? Do user goals change in their interactions with OPACs? Large and Beheshti (1997, p. 128) also asked several vital questions related to studies of user goals:
• Does a user pursue a single goal in one search session, or is the goal dynamic and changes in the search process?
• Does a user divide the search goal into several search sessions?
• What determines goals and search strategies?
Strategies/Behaviors.and.Affecting.Factors
Information-seeking strategies and behaviors are the center of OPAC interaction studies because they represent how interactions take place. In this section, the author reviewed the research focusing on identification of types of strategies and patterns of behaviors, and the factors affecting these strategies and behaviors.
First, the different types of information-seeking strategies are related to task dimen- sions. One dimension of tasks is type of task. Considering OPAC searching a special kind of communication between humans and computers, Slone (2000) explored and identified information-seeking strategies and behaviors based on three types of searches: unknown-item searches, area searches, and known-item searches. The results showed that term generation is the driving force for unknown-item searches, where the basic strategy is to formulate a query, evaluate the results, and reformulate the query if necessary. Speed and convenience are essential to area searches; there-
Interactve IR n OPAC Envronments
fore, users quickly look for a few records from OPAC and complete their searches by browsing the shelves. Query-matching is appropriate for known-item searches because accuracy and simplicity is most important for this type of search. This study indicates that strategies and behaviors are determined by the dimensions of tasks.
Second, information-seeking strategies and behaviors are related to affective responses. It is a complicated process for users to interact with OPACs. Their behaviors, attitudes, and feelings need to be identified in order to understand their interactions with systems. By applying factor analysis, Dalrymple and Zweizig (1992) demonstrated that benefits and frustration are the two dimensions of users’
affective responses to OPAC systems, and that both benefits and frustration affect information-searching behavior, especially reformulation behavior. Affective feelings were also investigated with the three types of searches conducted by Slone (2000), discussed above. The unknown-item searchers experienced the most frustration and doubt; the known-item searchers experienced the most disappointment; and the area searchers experienced the most confidence and contentment. In that sense, user tasks, strategies, and affective feelings are interrelated.
Third, types of information-seeking strategies are related to types of users. OPACs are designed for a variety user groups. Some of them are designed for children, so it is important to understand children’s information-seeking strategies. By observing, questioning, collecting think-aloud protocols, and analyzing documents, Soloman (1993) explored children’s information-retrieval behavior in using an OPAC in an elementary school library. A variety of factors were considered for children’s information retrieval success and breakdown in the study, such as user character- istics, the school setting, interface usability, and information access features. Most important, he identified two classes of strategies, planned strategies and reactive strategies. Planned strategies, in which users make decisions prior to and including the first move, consist of author, title, multiple concepts, external support, system features, and index function. Reactive strategies, in which users make decisions to follow up one move with another, include focus shifts, search term relations, error recovery, and external supports.
Borgman and her associates have studied children’s information-retrieval behaviors in multiple design iterations of a science library catalog. The objective of these studies is to understand children’s information-seeking behaviors and further incorporate search mechanisms facilitating their searching (Borgman, Hirsh, & Hiller, 1996).
Borgman, Hirsh, Walter, & Gallagher (1995) proposed an alternative information retrieval model for children that was built on the capabilities and knowledge of children at their respective ages. Children can recognize information, browse for information, use hierarchies, and provide a context for information. Multiple meth- ods, including interview, online monitoring, and focus groups, were applied to four experiments on four versions of the catalog. The studies shed light on how children searched for information in both a hierarchical, browsing, recognition-based system
and a keyword and Boolean system. The authors concluded that the ideal system for children may combine the browsing and keyword features that do not require children to use content spelling, or use Boolean logic.
These studies characterize the patterns of information-seeking strategies or behaviors displayed by different types of users. They also identify the task, affective, and age factors that affect information-seeking strategies and behaviors. However, further research needs to examine more in-depth the dimensions of these factors and other factors that might influence users’ strategies and behaviors. In addition, researchers also need to look into whether users exhibit different types of strategies or behaviors in interacting with different types of online IR systems.
Effect.of.Knowledge.Structure.on.Search.Success
In order to successfully interact with OPAC systems, users need to possess different types of knowledge because the existing OPAC systems do not provide assistance to enhance their knowledge structure.
First, they need to have domain knowledge for understanding the topic of their searches. Employing interviews, online monitoring techniques, observations, and card-sorting tasks, Hirsh (1997) focused her research on how children found infor- mation through different types of searches, and found that task complexity and the children’s domain knowledge affected their success in finding information in the online catalog. More IR tools that are designed specifically for children are essential to help children seek information in digital environments. Knowledge about the searching topic is important for subject searching, and knowledge about the items that users look for is essential for known-item searching.
Second, users need to have specific knowledge about the retrieved items, particu- larly for known-item searching. This specific knowledge can be an extension of the domain knowledge. After analyzing interviewing protocols, Wildemuth and O’Neill (1995) examined what information users had for known-item searches and how that affected their success in known-item searching. They found that searchers normally knew the title, publication date, page numbers, or the author of a known- item based on bibliographies, search results, published references, hand-written notes, or recalled memory, and the information was accurate for finding the known items. The results of this study suggested that OPACs can be enhanced to reduce users’ efforts to describe a known item in OPAC searching.
Third, users need to have system knowledge to understand how the system works.
Ease-of-use of OPACs is not necessarily good for users’ effective information retrieval. By applying questionnaires and transaction logs to collect data, Hildreth (1997) studied whether users understood how the system processed keyword and Boolean searching while searching an OPAC. He discovered that although users
Interactve IR n OPAC Envronments
conducted more keyword searches than other types of searches, they failed more and they did not understand the process behind keyword searching. He called for the need to improve the design of OPACs based on interactive models of informa- tion-seeking behavior.
Fourth, and most import, users need to integrate and apply the above knowledge to their interactions with OPAC systems. Connell (1995) found that metaknowledge is used in the subject searching process by experienced searchers in online catalogs.
Metaknowledge is an integration of factual, process, and experiential knowledge about the search and the search context. This finding was based on data collected from think-aloud protocols, transaction logs, and structured interviews. The study recommended the construction of aids about metaknowledge to assist users, mainly novice users, in searching online catalogs.
It is no doubt that users need different types of knowledge for successful interac- tions with OPACs. The problem is that users probably won’t be able to have all the needed knowledge. Researchers have not explored how IR systems can enhance users’ knowledge structure in the interaction process. Another question is how knowledge structure affects users’ search strategies and behaviors in addition to search performance.
Evaluation.Studies.and.Usability.Testing
Relevance is a fundamental concept for the evaluation of IR systems. O’Brien (1994) pointed out that it is essential to understand what relevance in the context of OPAC searching is, because developments in OPACs are based on more interac- tive approaches with relevance feedback capabilities. However, not all users liked relevance feedback mechanisms. Applying questionnaire and transaction logs, Han- cock-Beaulieu, Fieldhouse, and Do (1995) evaluated an interactive query expansion mechanism based on relevance feedback in an OPAC system. They found the use of the interactive query expansion option was lower and the retrieval performance less effective because this option gave users too much control. There is a need to further investigate the relationship between interactive interface environments and their impact on searching behavior.
Most of the evaluation studies have been usability studies that concentrate on the evaluation of the interfaces and features of OPAC systems. These usability tests are conducted by comparing an OPAC with another OPAC or other types of IR systems.
A browsable graphical interface, Public Access Catalogue Extension (PACE) is an alternative interface designed to enhance online catalogs. It simulates the images of books and library shelves to facilitate users’ browsing of the online catalog. The interface was evaluated and tested against a text-based OPAC. The results showed that a majority of the users preferred a visual interface to a command-driven, text- based OPAC because it required a smaller cognitive load. The familiar metaphor of
the bookshelves was more intuitive (Beheshti, Large, & Bialek, 1996). Drabenstott and Weller (1996) conducted an experiment to compare two catalogs: search tree controlled the selection of subject searching in one, and the subject-searching ap- proach was randomly selected in another. Although the results showed that there were mixed results in terms of whether search tree improved subject searches, the results did show that users preferred the OPAC system that controlled the subject search. The same system was also selected for its ease-of-use and efficiency.
In evaluating the usability of OPACs, researchers also examined the problems of subject searching. After comparing the usage of an OPAC and the card catalogue of a library, Sridhar (2004) looked into the problems of subject searching. Users are required to have technical skills and conceptual and semantic knowledge in order to articulate the query for a subject search. In order to facilitate subject searching, Sridhar (2004) called for the need to enhance interactive searching features, such as drag and drop text from hits, “more like this” features, online thesauri with clas- sification links, and so forth.
Do all users prefer the same features of an OPAC? In order to understand the re- lationships between user characteristics and OPAC features, Kim, Chung, Hong, Moon, and Park (1999) tested the correlations between user characteristics—such as age, gender, educational status, computer skills, and OPAC experience— and the preferred usability features of Web OPACs, such as interaction styles, character and image on screen, browsing and navigating style, screen layout, and ease of learning.
Although this study was based on a small-scale sample, the result discovered sig- nificant correlation between user characteristics and the preferred features of a Web OPAC. It further identified that age was the most significant variable, followed by gender, subjects’ computer skills, and OPAC experience. To sum up, OPAC system design needs to consider user demographics.
The existing evaluation studies on OPACs mainly concentrate on usability studies.
In addition, they have not identified the criteria for the evaluation of interactive IR systems. Researchers also need to pay more attention to how to take users’ charac- teristics into account for the evaluation of interactive IR systems or features.