Ph.D. – granting institutions and 50 at two-year colleges. Looking at web sites as information tools, reference tools, research tools, and instructional tools, they wanted to see how they fit into the structure of the parent in- stitution and how well the library web site reflected the college/university.
Curl, Reynolds, and Mai (2000) studied asynchronous credit courses in libraries. Purdue University offered a course titled Information Strategies using WebCT and Subramanyam’s circular model of the evolution of sci- entific information as the framework. They found a correlation between frequency of visits and grades earned, that students who used their own computers did better than those who used only computer labs, that com- puter skills that students brought to class were uneven, that students liked the flexibility, and that cooperative and collaborative learning was evident.
An interesting side note to this literature was that Lee and Teh (2000) found that studies of academic library web sites either exclusively used quantitative analyses or qualitative methods, concluding that there appears to be a lack of a mixed-method approach for the evaluation of academic library web sites that integrated the results of both kinds of studies. There- fore, this study was designed to add to the literature in terms of both re- search methodology and the specifics of library services to distance education students.
of documents and other forms of communication in a systematic approach.
Most often used in a strict quantitative fashion, a large subpopulation of research has been using the term in a more general sense, not any less systematic but more inferential in nature.
Originally used for examining textual literature in newspapers and other fixed forms, it is now also used to objectify and organize electronic media such as web sites. Content analysis of traditional media, such as newspapers and broadcast media, assume some linearity or at least commonly accepted sequencing of messages. Hypertext by its very nature changes this approach;
each person interacts with web sites in different ways, and, therefore, the final results may vary (McMillan, 2000). The rapid growth and change of the web also leads to potential problems in the third stage of content anal- ysis, data collection, and coding.
One of the problems with looking at web sites is their dynamic and changing nature. Bauer & Scharl (2000) gives three possible options to sidestep potential shortfalls. The first approach is to complete a snapshot analysis. This allows a static view similar to traditional print-based media.
Using this method, researchers analyze a large number of web sites at a given time. This allows comparison of individual criteria and the clustering of sites by any number of qualifications. The second approach is a longi- tudinal analysis, in effect a series of snapshot analyses allowing the re- searcher to see trends. This approach takes advantage of the dynamic nature of the web. The third option is a comparative analysis. By comparing web sites from one institution with those of competitors or similar types of institutions, the researcher can make note of the relative performance of the sites they are studying. This study uses both the first and the third options.
Krippendorff (1980, p. 21) defines content analysis as ‘‘a research tech- nique for making replicable and valid inferences from data to their context.’’
There are usually four steps. The first is data making: collecting data in an analyzable form which others would be able to re-evaluate. Text would be collected, speech or sounds would be recorded and then transcribed, and a collection would be formed. Second comes data reduction in order to ease computational efforts. It may be statistical, algebraic, or simply a question of omitting what turns out to be irrelevant detail. Third is inference, the reason for any content analysis, the task of finding relationships. Fourth is analysis. This is the identification and representation of patterns that are noteworthy and statistically significant.
Once the data is collected and organized, a statistical analysis is done in order to pull out important trends. According to Krippendorff (1980), the most common form of representation of data, serving primarily the
summarizing function of analysis, is in terms of frequencies: absolute fre- quencies, such as the numbers of incidents found in the sample, or relative frequencies, such as the percentages of the sample size. Volume measures such as column inches, time, space, or other frequency-based indices have the same status in content analysis.
Interviews and Surveys
Interviews are often conducted in order to provide more background in- formation or substance to a numerical finding. Explanations of why certain decisions were made as well as what elements or were not included in the website can then be added to the final analyses. This is meant to complement the data previously found.Bogdan and Biklen (1998)argue that this com- bination of approaches is ideal. Miles and Huberman (1994) state that qualitative data can be used to supplement, validate, explain, illuminate, or reinterpret quantitative data gathered from the same subjects or site. Indeed, combining quantitative and qualitative analyses seems to provide the best picture. A combination of numbers and words are both needed to really understand the world (Miles & Huberman, 1994).
It is not feasible to interview every single librarian that would fit in this study. Therefore, some type of sampling must be done. According toMax- well (1996), purposeful sampling is often ideal for small samples. In this type of strategy, particular settings, persons, or events are selected deliberately in order to provide important information that cannot be obtained as well from other choices.Miles and Huberman (1994)describe a similar type of sampling that they call stratified purposeful. For this study, this method seems ideal, as it illustrates subgroups and facilitates comparisons.
However, this study has gone one step further. As is often true of qual- itative methods, the data were also analyzed inductively. It could then be said that the research is grounded in the data (Bogdan & Biklen, 1998). This type of methodology is more open-ended, with probing questions yet flexible enough to develop as more information is gained as opposed to a preset and inflexible plan.
Procedure: Quantitative Content Analysis of Web Sites
The first part of this research consisted of a quantitative content analysis.
Seventeen diverse library web sites were initially selected. While many of the
studies found in the literature included a greater number of samples, they did not also include interviews and surveys. Furthermore, larger studies usually used smaller sets of content items for analysis.
Therefore, a smaller set of web sites was used in a more complex analysis and in combination with additional methods for this study. The web sites chosen were categorized by the amount of distance education offered to students: fully online institutions, traditional institutions now offering com- plete degrees via a distance, institutions offering at the most individual courses via a distance, and institutions not yet offering distance education to provide some comparisons. This initial number was realistically feasible with the understanding that additional sites would have been reviewed if the results warranted more data collection.
All of the institutions selected are regionally accredited, according to Peterson’s (2002) Guide to Distance Learning Programs. Table 1lists the library web sites that were coded for this study.
Table 1. Library Web Sites Coded.
Institution URLs
(1) Primarily distance education institution
(2) Full programs/degrees offered via distance education (3) Single courses offered via distance education (4) Little to none distance education
(1) City University http://www.cityu.edu/library/home.asp (1) Nova Southeastern University http://www.nova.edu/library/main/
(1) Touro University International http://support.tourou.edu/virtual_libraries/
index.htm
(1) The Union Institute http://www.tui.edu/vermontcollege/templates/
about_us.php?article_id=39 (1) Walden University http://www.lib.waldenu.edu/
(2) Central Missouri State University http://library.cmsu.edu/
(2) Ferris State University http://www.ferris.edu/library/
(2) Illinois Institute of Technology http://www.gl.iit.edu/
(2) University of Illinois http://www.outreach.uiuc.edu/aolibrary/
(2) University of Wisconsin–Madison http://www.library.wisc.edu/
(3) Central Methodist College http://www.cmc.edu/library (3) Delta State University http://library.deltastate.edu/
(3) Kentucky State University http://www.kysu.edu/library/default2.html (3) Northern Illinois University http://www.niu.edu/libraries.html (3) Southeast Missouri State University http://library.semo.edu/
(4) Dominican University http://domweb.dom.edu/library/
(4) Saint Anthony College of Nursing http://www.sacn.edu/library
The coding of web sites was done during the month of September 2002.
This time frame was necessitated due to the fact that web sites change frequently. This study was, therefore, a snapshot of specific services at that point in time. The researcher did all of the coding. Those items not found within a reasonable amount of time, five minutes, were checked off as not existing. In order to gain a better understanding of the experiences that the library users face, this researcher wanted to approach the sites from a stu- dent’s perspective and coded all of the sites himself.
The coding was accomplished by browsing and searching selected library web sites, and then adding results to an electronic coding sheet. These results were then able to be moved into other statistical software and manipulated for analysis.
The coding sheet was developed to list the types of services that are offered (Table 2). The review of the literature provided some types of ideas
Table 2. Coding Sheet.
Institution Name
- URL
- Date accessed
- Type of institution (http://nces.ed.gov/IPEDS/COOL/Search.asp) - Size of institution (http://nces.ed.gov/IPEDS/COOL/Search.asp)
- Degree or programs offered via distance
General information Mission/vision
- News (what’s new)
- New acquisitions
- Internal search engine
- Site map
- Webmaster
- Employee listing
- List of subject specialists
- Policies
Distance education Page for distance learners Specified librarian for DE
Strategies for distance learning effectiveness
Reference Phone, toll free
E-mail electronic form Chat
Video conferencing
Consultation, research assistance Online reference materials
to think about, and an initial pre-review of library web sites provided some initial insight. The coding sheet was also left open to modification as the study progressed with the option to go back and re-review web sites if a major function was not initially included.
Procedure: Qualitative Interviews and Surveys
The second part of this research consisted of interviews and surveys. The researcher contacted four libraries after the quantitative analysis was com- pleted. These sites were selected from representatives of web sites with low, medium, and high degree of service elements. This provided variation and explanation for the findings and also a better idea of what kinds of decisions were made at these institutions.
Table 2. (Continued)
Institution Name
Bibliographic instruction Tutorials
Class (credit/non-credit) Links to evaluation of resources Interlibrary loan Information about
Materials mailed Electronic forms
Circulation Books delivered
Print materials copied and delivered Book renewal form
Joint borrowers card
Arrangement with other libraries
Databases Indexes and full-text
Online catalog E-books Journal holdings Subject web sites
Reserves Electronic reserves
Feedback Electronic feedback mechanism
FAQ Password protected site Description
The interviews were conducted in October 2002. The telephone conver- sations were recorded so that transcripts of the conversations could be made. The interviews were open to any topic that the respondents wanted to discuss. The following is a generalized description of those interviewed during October 2002:
Assistant Professor of Information Services at a small, private Midwest- ern college
Distance Education Librarian at a medium-sized public university Head of Distance Education and Instructional Library Services at a large,
private university
Instructional Services Librarian at a large, public university
With this approach, questions and topics for discussion were developed prior to and after quantitative data analysis. Preparation for the interviews evolved as information was gathered and the interviews themselves were flexible depending on what the interviewee was willing and interested in discussing. Once the interviews were completed the transcripts were coded and analyzed for key words and ideas. Either themes were to emerge or the diversity of responses was to instead show a diverse set of approaches from libraries. Regardless of the specific results, additional interviews would have been considered if warranted
The third part of this research consisted of surveys of students enrolled in distance education courses. These surveys were conducted from the list of institutions where librarians were previously interviewed. Six faculties from three different institutions shared the survey with their students. This was done to gauge the satisfaction students have with their library web sites and overall distance services, and to provide a learner-based perspective.
Contacts were made at these institutions to find faculty who were cur- rently teaching a distance education course. Surveys were distributed to students through their teachers by directing them to a web-based survey form. The form was created and posted on the author’s personal web space at Central Missouri State University. The electronic form simplified the process, allowing immediate anonymous feedback from students without the costs and time involved with paper surveys.
A pilot study was conducted in September 2002 in order to gauge the amount of time that would be involved with a full study, to insure that the web-based survey form was working properly, and make sure that the methods chosen were feasible. Contacts for student surveys and library in- terviews were made through institution web sites.