The item-writing process consists of the following six steps, which are shown in Figure 7.1:
1. State the theoretical definition.
2. Identify the dimensions of the concept.
3. Create a content-domain by concept-dimension matrix.
4. Decide how many items to include for each cell in the matrix.
5. Write items for each content-domain by concept-dimension cell in the matrix.
6. Write the rules for the scoring and administration of the scale.
Following we describe each of these steps, using as an example items written for a scale measuring the concept outcome expectations for healthy eating.
FIGURE 7.1. STEPS IN THE ITEM-WRITING PROCESS.
State the theoretical definition
Write items Decide on number of
items per cell
Identify the dimensions of the
concept
Draw content by concept matrix
Write rules for administration
State the Theoretical Definition
To begin the item-writing process, the researcher refers to the theoretical definition and the attributes of the concept derived from the concept analysis. For example, using the definition of outcome expectations presented by Bandura (1997), we define the concept outcome expectations for healthy eating as those outcomes one expects if one eats a diet low in saturated fat and high in fruits and vegetables. This definition suggests that the researcher is interested in outcomes associated with eating a healthy diet and has identified three content domains:—eating a low-fat diet, eating fruits, and eating vegetables. The researcher could combine these three behaviors and call the combination a healthy diet, or could conceptualize each behavior separately. In the ex- ample that appears later, the three behaviors are treated as three content domains of a healthy diet.
Identify Dimensions of the Concept
If, during the concept analysis, the researcher determines that the concept is composed of more than one dimension (is multidimensional, as opposed to unidimensional), the dimensions can be treated separately. For our example of OE, we find that Bandura (1997) conceptualized three dimensions of OE: (1) self-evaluative OE, (2) social OE, and (3) physical OE. Self-evaluative OE consists of those outcomes one expects to per- sonally experience if one performs the behavior. Social OE consists of those outcomes that one expects others to experience if one performs the behavior. Physical OE, of course, consists of those outcomes that are physical in nature. If we apply these di- mensions to OE for healthy eating, we might conclude that eating a healthy diet would result in the following: (1) feeling good about oneself (self-evaluative OE), (2) gaining the approval of others (social OE), and (3) maintaining a desired weight (physical OE).
Create a Content-Domain by Concept-Dimension Matrix
For ease of item development, the researcher creates a matrix (also called a blueprint) that visually displays the content to be addressed by the items and the dimensions of the concept that will be measured. This matrix is similar to the table of test specifi- cations for knowledge tests (Table 5.1) that we discussed in Chapter Five. The simplest matrix is composed of one content domain for a unidimensional concept (a concept with only one dimension). Table 7.1 shows the matrix for a scale to measure barriers to the use of sun protection products. Here, the researcher is interested in writing only items related to an individual’s opinion of the physical and psychosocial costs of using sunscreen products. All the items developed for this scale must correspond to the de- finition of barriers and be specific to sun protection products.
If the researcher wanted to develop items for the eight dimensions of social cap- ital using Bullen and Onyx’s (2005) framework, the matrix would consist of one row and eight columns, as shown in Table 7.2. A matrix for the concept of OE related to healthy eating using the three content dimensions of low fat, fruits, and vegetables would look like that in Table 7.3. In this matrix, we see the content domains of the concept of a healthy diet along the left side (low fat, fruits, and vegetables), and the di- mensions of the concept OE across the top (self-evaluative, social, and physical). When preparing the matrix, it is important to identify the variable for each cell. Being ex- plicit about the quantity to measure will assist with writing items. For example, for the social capital dimension of community participation, we would write the variable as
“the extent to which one joins local community organizations.”
Decide How Many Items to Include for Each Matrix Cell
The matrix helps the researcher visualize the content domains and the concept di- mensions. It also helps the researcher identify the number and type of items required to adequately measure the concept. Thus, the next step in the item-writing process is to use the matrix as a guide to determine how many items to write for each cell in the matrix. In our example of OE for healthy eating, the researcher might decide to write five items for each cell, in which case the final scale would contain forty-five items.
As we discussed earlier in Chapter Six, Matt is creating such a scale for men and women over age sixty-five. Generally, long scales are difficult for elderly people to com- plete. Thus, Matt must make some decisions about how many items to include for each of the cells. If Matt believes that a forty-five-item scale is too long, he can write fewer items for each cell. Alternatively, he might believe that the measurement of OE re- lated to fruits and vegetables is more important than that of OE related to low fat.
In that case, he will write more items to assess fruits and vegetables and fewer to assess low fat. Likewise, if Matt believes that personal OE is more important to measure than physical OE, he can write more items to assess the personal dimension of OE than items to address the physical dimension.
Generally, researchers write more items than needed for a scale. In the process of evaluating the scale for reliability and validity, most likely some items will not fare well and will be deleted. To guard against having fewer items than desired for the final scale, it is important to include a sufficient number in the original pool.
TABLE 7.1. EXAMPLE OF A 1 × 1 MATRIX.
Content Sunscreen Product Use Barriers
TABLE 7.2.EXAMPLE OF AN 8 ×1 MATRIX. ContentCommunity TrustToleranceNeighborhoodFamily and WorkValue of Proactivity ParticipationConnectionsFriend ConnectionsLifein Social ConnectionsContext Social Capital TABLE 7.3.MATRIX FOR OUTCOME EXPECTATIONS FOR HEALTHY EATING. Dimensions for Outcome Expectations for Healthy Eating Content DomainSelf-EvaluativeSocialPhysical Low fat Fruits Vegetables
Write Items for Each Content-Domain by Concept-Dimension Cell
The matrix guides the researcher in determining the content areas and the number of items to write, but for the actual phrasing of items, the researcher turns to other sources.
These include (1) experts (literature review or personal contacts), (2) qualitative inter- views with representatives of the population, and (3) other measuring instruments.
Literature Review. During the process of concept analysis, the researcher likely collected examples of phrases from the literature that embodied the concept. These phrases now become useful in developing items. For example, in reading about the reasons why people choose to eat well, the researcher might discover that people want to maintain an acceptable weight, to have more energy, to lose weight so that their clothes fit better, to receive the approval of family and friends, and to live longer. We can transform each of these facts into an item to measure OE:
If I ate a low-fat diet, I would maintain my weight.
If I ate a low-fat diet, I would have more energy.
If I ate a low-fat diet, I would lose weight.
If I ate a low-fat diet, my clothes would fit better.
If I ate a low-fat diet, my family would be pleased.
If I ate a low-fat diet, I would live longer.
Note that the first and third items are very similar. Matt might include only one of these items in the scale at this point in the development process, or he might keep both items and determine which is the better item after reliability and validity assess- ments have been completed.
Qualitative Interviews. Information collected from members of the population in which the concept appears, through either focus groups or qualitative interviews, pro- vides another rich source of phrases for the generation of items. For example, suppose Matt conducted a focus group to find out what elderly men and women eat, what bar- riers they face in eating a healthy diet, and what they believe are its benefits.
In one focus group, Matt says to the group: “So far, we have discussed what we think a healthy diet is. You mentioned that eating less fat would be considered part of a healthy diet. This might include eating only one or two servings of red meat per week and not snacking on potato chips and cookies. Now I would like to ask you what you think are some of the benefits of eating a low-fat diet.”
Respondent 1: “I have high blood pressure. Every time I go to my doctor, she tells me to eat less fat. She thinks that if I did, my blood pressure would go down, and I would not have to take so many pills.”
Respondent 2: “They say that eating fat will make cholesterol build up in the ar- teries near your heart and that when the cholesterol builds up there, it can block the arteries and cause a heart attack. So if you eat less meat and more vegeta- bles, you have less chance of having a heart attack. I am trying to eat more veg- etables because my mother and her sister both had heart attacks in their fifties.”
Respondent 3: “I really, really like chocolate, and it is hard for me not to eat it.
But if I stopped eating it and ate fruit instead, I would probably be able to lose some weight. If I could lose 20 pounds, I would feel a lot better.”
In this exchange, each of these respondents revealed an expectation about an out- come he or she might realize if he or she ate a healthy diet. In constructing items, we can draw on their comments. For example:
If I ate a low-fat diet, my blood pressure would go down.
If I ate vegetables, I would be less likely to have a heart attack.
If I ate fruits, I would lose weight.
Other Instruments. In addition to talking with members of the population, researchers can review other instruments to locate useful items during the development of the new scale.
They can obtain scales from other researchers, from written sources such as books and jour- nal articles, from companies that sell the instruments, and from Internet sources. Before using an item, researchers should first determine its copyright status and obtain permission from the developer of the instrument. Once it is obtained, they must review items to determine their fit with the purpose of the new scale and with the theoretical definition. Researchers may include items as written or they may modify them to fit the scale.
Recall that in Matt’s first attempt to develop an OE scale, he did not have the ben- efit of information obtained from a concept analysis. Thus, he and the staff members relied on their own interpretations of the concept to generate items for the scale. Some- times this approach works, particularly if the researcher is the same person who iden- tified the concept. (For example, Bandura [1997] identified the construct of self- efficacy and then taught people how to measure it.) Most of the time, however, a scale developed without sufficient understanding of the concept will demonstrate problems during reliability and validity testing, which we will discuss in Chapters Ten and Eleven.
One person or several people can write items for a scale. If more than one person is involved in item writing, all the item writers should understand the variable and agree on its meaning. Ideally, all the item writers have involved themselves in the concept analy- sis and the generation of the theoretical definition and attributes for the concept and the statement of the variable. Following the creation of the definition, the group writes several sample items. They review the sample items for consistency with the concep- tual definition. After the members of the group agree on the final forms of these sam- ple items, they assign each member to write items according to the matrix. After the items are complete, the group convenes to discuss and modify them until members agree that the items adequately measure the variable, and they express satisfaction with the phrasing of the items. During the testing of the scale, researchers identify and delete weak items, leaving only those that have functioned well during the testing process.
Write Rules for Scale Scoring and Administration
The final task in scale development is to write rules for scoring items and administer- ing the scale to respondents. Rules for scoring scale items correspond to the type of scale used. In Chapter Two, we presented a description of five types of scales:—VAS, Thurstone, Guttman, semantic differential, and Likert. Because the Likert scale (also called a summated rating scale) is the most common type of scale used in health be- havior research, we will describe the scoring rules for the Likert scale in the section that follows. Though scoring rules depend on the type of scale, rules for administra- tion depend on the preferences of the scale developer and, to some extent, correspond to the medium used to present the scale and the data collection procedures. For exam- ple, a seven-point scale might be used for scale items that participants complete using paper and pencil. In contrast, the developer might consider using three to five response options for scale items presented to participants over the telephone. Scale developers may have preferences about the order of the items, the type of response options, the use of adjectives, adverbs, or numbers on the response scale, and the format for presenta- tion. These preferences are included in the description of the scale.