Respondents
The next step in developing a survey is to consider the respondents:
• Who are the respondents?
• What are their expected levels of language and reading skills?
• How familiar are they with the topic?
• How will respondents be recruited for the study?
• How much time will they have to complete the survey?
• Will they be willing to respond to the survey?
Young children with limited reading and writing skills would need an interviewer to read questions to them, as would adult respondents with poor literacy or physical limitations that interfere with reading or writing. If the researcher believes some re- spondents might be unfamiliar with the topic, definitions of some terms should be provided.
Just as the researchers assessing epilepsy self-management made decisions about what to include in the epilepsy background information survey, researchers must make tough decisions about the inclusion of survey items based on the amount of time avail- able. If the respondents have only five minutes, ten to fifteen questions may be the maximum number that they can answer.
Nurses and other professionals working on hospital units tend to be extremely busy. Therefore, our public health professionals who want to ask them to complete a survey on hand-hygiene practices must bear in mind the respondents’ time restrictions when developing their survey. A short survey, such as that presented in Exhibit 4.1, would probably take a few minutes to complete, and nurses might be willing to com- plete it during the course of their work day. This survey would be useful to gather pre- liminary information that could be used to develop a more comprehensive questionnaire on hand-hygiene practices.
The first principle of survey development is related to the respondents’ process of completing the survey. Clark and Schober (1994) note that what appears to be a sim- ple process of providing information on the survey is actually a complex interaction.
They make clear that it is important for the researcher to think about a survey in terms of an exchange of information between a researcher and a respondent. In the sim- plest exchange, the respondent first reads and then answers the survey items. Under- standing both the role of the researcher’s intentions in asking the questions and the meaning attributed to the questions by the respondent is important in developing high- quality items that minimize measurement error.
In a normal conversation, people develop a common ground, or common under- standing of the discussion (Clark & Schober, 1994). Generally, conversation consists of the logical progression of ideas and information, strung together by the contributions of each participant in a discussion. Speakers make statements in such a way that the peo- ple to whom they speak can understand what they are saying. Unless a respondent re- quests clarification, we assume that the recipients of the speaker’s statements understand what is being said. As the conversation moves along, topics change, but those who are paying attention to the conversation realize it has headed in a different direction.
In an ordinary conversation, participants in the discussion contribute in real time.
However, with a survey, one participant in the conversation, the researcher, writes all the questions before the respondent ever answers the first one. Thus, the challenge for the researcher is to build the common ground for the survey, which includes making state- ments clear so that respondents know how to answer each question and providing suf- ficient instructions for the completion of the survey along with directions for changes in the topics within the survey. Using these principles of language along with some com- mon prescriptions for item writing, we present some do’s and don’ts for writing items.
These suggestions are based on the work of several authors who have studied the art and science of asking questions (Sudman & Bradburn, 1982; Dillman, 2000).
The Do’s of Item Writing
1. Write items that are related to the purpose of the survey and that address the objectives. When participants are told the purpose of the survey, they will ex- pect to answer questions that are related to the topic (Dillman, 2000). That is, the items should reflect the purpose of the survey and be transparent to the respondent. If the purpose of the survey is to identify the exercise habits of college students, a respon- dent will not expect to see items about attitudes toward instructors. If the researchers include items that are not transparent, they should tell the participants how these items are related to the study. For example, if a researcher is interested in familial and peer factors associated with exercise, some items are likely to refer to the exercise habits of
friends and family members. A simple statement at the beginning of the study telling the participant that the study will also examine how others influence their exercise habits helps participants understand the rationale for these questions.
2. Write each item as a complete sentence with a subject and a verb. Typically, people who are conversing speak in complete sentences rather than single words or short phrases.
Weak: Age
Revision: What is your age?
3. Write items that are short and concise when requesting information about neutral topics. Short questions are easier for respondents to read. Longer ques- tions take more time to read and are subject to more error if the respondent fails to read the items carefully and responds inappropriately. In addition, long questions are likely to include more than one idea, making it difficult for the respondent to answer.
Weak: Do you ever eat fruits and vegetables that are not washed because you don’t have time to wash them or because you don’t think that unwashed fruits and vegetables are harmful?
Yes No
Revision: Do you wash fruits and vegetables before you eat them?
Yes No
If no, which of the following are your reasons for not washing them? (Check all that apply.)
Not enough time
Not concerned about harmful bacteria Habit
Sometimes participants need background information about a topic before en- countering the questions. This background information may make the question seem very long. In this case, separating the information from the question helps the re- spondent see exactly what is being asked.
People with epilepsy may have different types of seizures. The two primary types are generalized seizures and partial seizures. Generalized seizures always involve loss of
consciousness and can involve movements of all extremities. Partial seizures may in- clude loss of consciousness; however, movements or sensations are usually limited to one part of the body.
Would you say that your most recent seizure was a generalized seizure or a par- tial seizure?
Generalized Partial
4. Write open-ended or long items when requesting information about sensi- tive topics. People tend to underreport occurrences of socially undesirable behav- iors and overreport occurrences of socially desirable behaviors. Sensitive behaviors, such as drug and alcohol use and sexual and criminal behaviors, are likely to be un- derreported, whereas healthy behaviors are likely to be overreported. Sudman and Bradburn (1982) suggest using open-ended questions and longer questions when asking about threatening or sensitive behaviors.
Respondents often infer from response alternatives that the researcher knows what is normal and is presenting them with response options within the normal range. Peo- ple reporting on sensitive behaviors may want to appear to fall within the normal range, so they may choose a middle response option. The provision of response alternatives can encourage underreporting or overreporting of some behaviors. In the example given here, we might interpret a history of six sexual partners as the upper limit of normality. However, a person who has had eight partners might select a response of 1–2 or 3–5 to appear to be within the normal range.
Weak: How many sexual partners have you had in your lifetime?
None 1–2 3–5 6 or more
Revision: How many sexual partners have you had in your lifetime?
Loading is another way to encourage accurate responses for both open- and closed-ended questions (Sudman & Bradburn, 1982). Before asking the actual ques- tion, the researcher places the behavior in a context that encourages the participant to report behavior occurrences accurately. In the following example, the loaded state- ment lets the participant know that having sex without a condom is common.
Not loaded: Did you use a condom the last time you had sex?
Yes No
Loaded: Many times people want to use a condom, but they forget or they don’t have one with them, and so they have sex without a condom. Did you use a condom the last time you had sex?
Yes No
Another approach is to use authority to justify behavior.
Recent studies suggest that drinking two or more cups of coffee with caffeine each day is good for your heart. How many cups of coffee with caffeine did you have yesterday?
5. Write response choices that are mutually exclusive and exhaustive. When respondents must make a choice in the selection of a response, the response set should include all possible choices. In addition, choices should be mutually exclusive, so that a respondent cannot select more than one.
Weak: What is your marital status?
Single Married
Living with a partner Divorced
Revision: What is your marital status?
Married Never married Divorced Widowed Separated
Are you currently living with a partner?
Yes No
If numerous possible response choices exist, a researcher may include several of the most common and then have a final choice labeled other, allowing the respondent the option of writing his or her response if it is unavailable on the list.
What is your religion?
1. Buddhist
2. Catholic
3. Jewish
4. Muslim
5. Protestant
6. Other (please specify)
6. Spell out acronyms. A search of acronyms on the Internet reveals that the same acronym may refer to many different organizations, programs, or ideas. For exam- ple, the American Medical Association and the Association of Media Agents both use AMA as an acronym. Individuals reading a question that contains an acronym are likely to interpret and thus respond to the question with their own particular understand- ings of what the acronym means to them. The same is true for abbreviations. If a sur- vey needs an abbreviation throughout, the developers should define it first in the introductory section and use the full term in the first question.
Weak: Are you a member of APHA?
Revision: Are you a member of the American Public Health Association (APHA)?
Weak: Have you ever been tested for HIV?
Revision: Acquired immunodeficiency syndrome (AIDS) is caused by the human immunodeficiency virus (HIV). There is a simple blood test for HIV. Have you ever been tested for HIV?
Yes No
7. Define unusual terms. Respondents seem to have stable opinions on issues about which they know nothing (Clark & Schober, 1994). Generally, respondents believe that if a researcher asks a question, they should understand what is being asked and will be able to answer the question. Thus, respondents will give opinions on issues about which they know little or nothing. Likewise, respondents tend to interpret vague words in the moment, choosing the closest meaning they can think of at that time; this often leads to inaccurate responses. In the weak item in the following example, respondents may be- lieve that vocational rehabilitation refers to something akin to physical therapy and re- spond accordingly. If a vague term is necessary, the researchers should define it. A better approach is to select a term or phrase that is more descriptive of the concept presented.
Weak: Have you ever attended vocational rehabilitation?
Yes No
Revision: Following your accident, have you attended sessions with a career counselor for job training?
Yes No
8. Write items that contain one idea. If, during the course of a conversation, a speaker asks, “Do you think the Atkins and South Beach diets are good?” then others in the discussion may choose to respond by commenting on either one of the two diets or by providing a comparison. However, if investigators present an item that includes two ideas (also called a double-barreled item) in the context of a survey with forced-choice options, the respondent must decide how best to respond.
If the respondent has a different opinion about each idea, he or she might choose a response based on one of the ideas or choose a response that seems to be on the mid- dle ground.
For example, a respondent might find it difficult to answer the following weak question about helmet use for drivers and passengers because he or she always wears a helmet and his or her passengers never do. Because of potential differences in responses to the two ideas, the surveyor should present the ideas separately.
Weak: How often do you and your passengers wear helmets when you ride motorcycles?
Revision: How often do you wear a helmet when your ride a motorcycle?
and
How often do your passengers wear a helmet when they ride with you?
9. Write items that are specific. In developing items, the researcher needs to con- sider how people might respond to each item. An item that seems fairly simple to the re- searcher might generate confusion for the respondent. Consider the following question about drinking. In the weak example, the respondent might wonder what the researcher meant by the word drink. More often, the respondent assumes that the researcher would agree with the respondent’s interpretation of the item. However, if the respondent thought that drink included only water, his or her response would be limited to the amount of water he or she drinks. Another respondent might believe that drink refers to alcoholic beverages only and answer accordingly. The researcher, however, might have wanted to learn about the full range of fluids an individual drinks in a day.
Weak: On average, how much do you drink per day?
Revision: On average, how many 8 ounce glasses of fluid do you drink each day? Include water, juice, coffee, tea, soft drinks, soups, and other liquids.
10. Use simple words. Survey development experts generally suggest that items be written at the sixth- through eighth-grade levels. To meet the criteria for these levels, items are generally short, with simple, common words. Researchers may dismiss this advice when they plan to survey college-educated individuals. However, even col- lege graduates find it easier to read items written at a lower reading level, and the lower reading level makes the task easier and faster.
Weak: How confident are you that you can differentiate eustress from distress in your daily life?
Revision: How confident are you that you can tell the difference between things that encourage you to succeed and things that cause you stress in your daily life?
11. Highlight terms that could easily be missed or that are particularly im- portant. In normal conversation, a speaker may emphasize a statement by changing the volume or quality of his or her voice. Written communication fulfills the same func- tion through the use of capitals, boldface, italics, underlining, or color changes to draw attention to a word or a phase. One word often missed in items is not. Therefore, if the survey includes items that contain the word not or prefixes such as non- and un-, the re- searcher should draw attention to them.
Weak: When was the last time you did not take your medicines?
Revision: When was the last time you did not take your medicines?
12. For recall items, allow a specific time period that is consistent with the behavior. In responding to items, participants feel pressure to respond as soon as a ques- tion has been asked or read. As in normal conversation, a person responds relatively quickly when asked a question, even if the question calls for information that requires recall and calculation. For example, when a respondent is asked in normal conversation,
“How many times did you eat fruit last week?” he or she is usually not expected to ac- tually take the time to recall exactly on which days fruit was eaten and how much.
This process would take too long, so respondents give a quick estimate to provide an answer in a reasonable amount of time. When responding to a survey question, the participant also uses a process to quickly estimate the amount of fruit eaten last week, unless he or she is given instructions on how to estimate the amount. To minimize mea- surement error, researchers may provide respondents with a reasonable period of time that is consistent with the behavior. When asking about frequently occurring, routine be- haviors, consider asking about the last day or the last week. For behaviors that occur less frequently or those that respondents are more likely to remember, the last month or the last year might be the most appropriate time frame.
Weak: How many fruits did you eat during the past week?
How many times have you visited the doctor in the past week?
Revision: How many fruits did you eat yesterday?
How many times have you visited the doctor in the past six months?
Additional ways to aid recall are to give specific dates and examples.
How many times have you been admitted to the hospital or treated in the emergency room since January 1, 2005?
How many college organizations, such as sororities, fraternities, political clubs, and foreign language groups, do you belong to?
13. Use ranges rather than precise values for sensitive items. When respondents reply to items about sensitive information, they are sometimes more likely to answer if they have a range of responses from which to select. This advice appears to conflict with rule 4, which suggests using open-ended questions to gather information about sensitive behavior. This example demonstrates that the researcher must understand the advantages and disadvantages of both approaches and make decisions based on the survey population and the goals of the research. In the case of income, people are much more likely to respond if they are given a range. The highest level of the range should be consistent with the population. For example, with middle-class and higher- income people, the upper range of household income might be $250,000 or more.
Weak: What was your total household income last year?
Revision: What was your total household income last year? (Please check one.)
$0–19,999
$20,000–39,999
$40,000–59,999
$60,000–79,999
$80,000 or more
14. Use response options such as don’t knowand not applicablesparingly.
One common decision researchers must make is whether to use a don’t know category.
Participants who are not offered don’t know as an option will generally choose one of the categories that are presented. Only a few will indicate that they don’t know when this option is not available. When don’t know is offered, the percentage of respondents choosing this category is likely to increase.