We can administer surveys in a variety of ways. Until near the end of the twentieth century, the most common methods of administration were face-to-face structured in- terviews and pencil-and-paper forms completed by participants. Computers have sig- nificantly changed the ways in which surveys are administered. Today, investigators may send surveys to participants by e-mail, or participants can log into a Web site to complete a survey. Hand-held and laptop computers allow direct entry of data into the computer database by the interviewer or the participant. Each approach has its own advantages and disadvantages, of which only a few will be discussed here.
In a face-to-face interview, the interviewer and the participant are physically to- gether. The interviewer asks the questions and records the participant’s answers. When using this format, the researcher must provide the interviewer with extensive train- ing about the research study, the items themselves, and the proper manner of asking questions. The researcher also must decide which items should include probes for the interviewer to clarify information and must decide on the types of probes. The ad- vantages of this approach include the likelihood that all items will be completed or that an explanation will be given for those items that are left unanswered. A probe might clarify items that are confusing to the participant, leading to more accurate an- swers. The disadvantages include a greater possibility of socially desirable responses because the participant must provide the answers to another human being. Also, the cost of the interviewer-administered survey is generally greater than the costs associ- ated with other methods.
A second form of interviewer-administered survey is the telephone survey. In this approach, interviewers call the participant on the phone and read aloud items to which
the participant responds. In addition to providing the opportunity to use probes and to clarify questions, the telephone interview is generally more convenient for both the interviewer and the participant because it does not require travel and it is generally shorter than other interviews. However, a distinct disadvantage is the inability of the participant to see the interviewer or the response options. In the face-to-face interview, the interviewer and respondent can see each other and use the full range of verbal and nonverbal cues in their discourse. In telephone interviews, where interviewer and re- spondent are unable to see each other, pauses are more disruptive and pressure to respond quickly increases. This means that the accuracy of responses in which recall and calculation are required is diminished. Because the respondent cannot see the re- sponse options, response choices to questions are limited to what people can reason- ably remember and consider without a printed reminder.
In the third type of survey, respondents read and answer each item on their own.
The traditional approach involves the use of paper and pencil to complete the survey. The survey instrument can be provided to the participants in a number of ways. Surveys can be handed out to individuals or sent by surface mail or email. Surveys can also be administered to large preformed groups, such as classes of students or attendees at a conference. Giving the survey to members of large groups has the advantage of col- lecting information from many participants at the same time. Generally, the paper- and-pencil survey is less costly than the face-to-face and telephone surveys. Whether administered individually or to a group, the paper-and-pencil survey allows respon- dents to answer the questions on their own at their own pace. On the negative side, the respondents must figure out for themselves the meanings of terms and how to com- plete the survey. Respondents sometimes fail to follow directions, fail to complete the items in the order presented, or make careless errors that decrease the quality of the data. Surveys administered to large preformed groups suffer from the lack of random sampling, which limits the generalizability of the results. Mail surveys are expensive, and researchers must make efforts to follow up with respondents who have failed to return surveys.
As we have mentioned, an increasingly popular approach for the administration of surveys is asking participants to record their answers directly into the computer. This approach has several advantages, including the elimination of the need for a data-entry person to enter responses from paper forms into the computer. Using computers with an audio component, each participant hears the same voice reading the same instruc- tions. This feature enhances the consistency of survey administration and reduces mea- surement error related to misreading an item or related to the tone and accent of an interviewer’s voice. When using the computer, participants are required to answer the items in the order presented, eliminating the possibility of missing items. Items that should be skipped based on previous responses are never shown to the respondent, thus reducing error associated with inconsistent responses. Recent studies also show that
respondents are more likely to admit to sensitive behaviors such as drug use when using the computer as compared to participating in face-to-face interviews (Jones, 2003;
Metzger, 2000). The primary disadvantages of using computers to administer surveys are related to the computer literacy of the staff and participants and to software glitches.
Training staff and providing some initial assistance to participants are usually sufficient to overcome computer literacy issues. Software glitches, however, can result in the loss of one or more completed surveys. Procedures for monitoring and data transfer can help prevent major problems with data collection.
Summary
In this chapter, we have presented a brief description of the procedures of survey development and administration. An effective survey begins with a statement of pur- pose and a list of objectives. The objectives are used to determine the content of the survey and to guide the development of individual items. A major emphasis in this chapter was on item construction. The principles for writing high-quality items will be useful to the student when we discuss scale development in Chapter Seven. Students who are interested in more extensive discussion of survey development and adminis- tration will find the texts by Dillman (2000), Mangione (1995), Salant and Dillman (1994), and Schonlau, Fricker, and Elliott (2002) helpful. Likewise, those who are in- terested in more detailed discussion of issues related to item writing should read the classic text by Sudman and Bradburn (1982) and the more recent work by Sudman, Bradburn, and Schwarz (1996), Schwarz and Sudman (1996), and Tourangeau, Rips, and Rasinski (2000).
75
LEARNING OBJECTIVES
At the end of this chapter, the student should be able to 1. Describe the characteristics of a knowledge test.
2. Discuss the basic principles of knowledge test construction.
3. Develop a table of test specifications.
4. Apply principles to writing multiple-choice and true/false items.
5. Revise poorly written items.
6. Describe the association between item wording and measurement error.
7. Conduct three types of item analyses.
T
here is a saying in health behavior research that knowledge does not predict behavior. For example, smokers are likely to know they are at risk for lung can- cer, yet their knowledge of the risk seems to have little effect on their decision to smoke.Similarly, sunbathers are likely to understand the relationship between sun exposure and skin cancer risk. Still, many choose to spend hours in the sun without applying sunscreen. Despite the absence of a consistent association between knowledge and health behaviors, behavioral scientists occasionally want to find out what people know about health behavior. Take, for instance, a researcher studying hand-washing prac- tices of medical residents. This researcher may want to consider what the residents