Experimental designs offer researchers the format for inferring a relationship between a theory and the application of that theory. The relationship between the dependent variable can be an association or a correlation but more often than not the relationship cannot show true causality—the cause leads to the effect—in the strict sense of the word. Even in total, clinical, experimental research, randomness threatens causality (Blalock, 1964). Furthermore, if all attempts to control for random selection are observed, total randomness is always questioned. This phenomenon is increasingly
demonstrated more in quasi-experimental designs than in experimental designs; nevertheless, it appears to be more accurate to refer to the relationship between variables as an association, a relationship, or a correlation, especially in the social sciences, rather than refer to the relationship between the dependent and independent variables as one of cause and effect.
Often designs are expressed using symbols of R¼randomly chosen subjects
O¼the observation or the measurement of the effect on the dependent variable X¼the independent variable
Although looking at combinations of these symbols in describing experiments presents some confusion on the part of the student, there is little alternative to presenting a diagram of the research designs in this way. However, when a description of the experiment is plainly given, the diagram of the experiment eventually presents a visual representation of which design is desired.
Furthermore, it is important to note that experimental designs and variations and derivatives of those designs are numerous, and often the researcher uses different combinations of designs to investigate a phenomenon or prove a theory. It would be unduly cumulative to present all of the combinations that could be created and, at the same time, some combination would invariably be omitted.
The following designs are listed and described using the typicalR,X,Oformat. The threats to validity and the internal and external controls, the strength of the experimental design over the threats to validity, are also identified.
8.6.1 PREEXPERIMENTALDESIGN
One-Shot Case Study3 X O
Threatened Internally by Threatened Externally by Controls Internally
History Selection interaction None
Maturation
Selection bias Controls Externally
Mortality None
As the design states, a one-shot case study does nothing more than observe the effect of the independent variable. It reports nothing more than some contrast or difference in a group attributed to some treatment. There is little scientific value to the one-shot case study; it is at risk from most relevant threats to validity; it does not control any threats. Other than a bearing from where to begin a discussion of experimental design, one-shot case studies offer little utility in the social sciences other than single‘‘snapshots’’of a group at one point in time.
One-Group Pretest–Posttest Design O X O
Threatened Internally by Threatened Externally by Controls Internally
History Testing interaction Selection bias
Maturation Selection interaction mortality
Testing
Instrumentation Controls Externally
Selection maturation interaction None
Mortality
With its numerous threats to validity, the one-group pretest–posttest design is just slightly better than the one-shot study or as Campbell and Stanley state‘‘. . . enough better than Design 1 [One- Shot Case Study] to be worth doing when nothing better can be done’’(1963, p. 7).
An example of the one-group pretest–posttest design is the study of reading skills development, where a group is tested and then after some of time the same group is tested again.
Static-Group Comparison X O
O
Threatened Internally by Threatened Externally by Controls Internally
Selection bias Selection interaction History
Maturation Testing
Selection maturation interaction Instrumentation regression
Controls Externally None
Static-group comparison studies are done where two groups are observed, one receiving the effect of the independent variable and the other group not experiencing the treatment. Static-group comparisons are useful when comparing program participants—children who have participated in operation ‘‘Head Start’’—with the reading level of those who did not participate in the head start program. The single accomplishment of the static-group comparison is that it establishes the effect of the independent variable.
8.6.2 THECLASSICALEXPERIMENTAL DESIGN
Notwithstanding the preexperimental designs where there are drawbacks that often preclude the use of these designs to protect research from threats to validity, the experimental designs offer more insulation from internal and external threats and are more appropriate as a research design. For this discussion the focus is on those experimental designs that have the following features:
. Reflect a random selection of subjects and there is no significant difference between an experimental and control group.
. A pretest that measures the dependent variable is given to the experimental and control groups.
. Both experimental and control groups will experience equal conditions except for the treatment of the independent variable.
. The researcher controls the amount of treatment to the experimental group.
. A posttest is given after the exposure to the treatment to both the experimental and control groups.
. Changes due to the dependent variable and the differences between the dependent variable effect on the experimental and control group, evidenced by the posttest, are attributed to the independent variable (adapted from O’Sullivan and Rassel, 1995).
Pretest–Posttest Control Group Design R O X O
R O O
Threatened Internally by Threatened Externally by Controls Internally
None Selection interaction History
Maturation Testing Instrumentation Regression Selection bias Mortality
Selection interaction Controls Externally No external threats noted
The pretest–posttest control group design is also referred to as the classical experimental design.
It is the most commonly used experimental design. This design enables the researcher to choose experimental and control groups of randomly assigned subjects. One group receives the experi- mental treatment while the control group does not. After the introduction of the independent variable, both groups are observed again. Differences between the experimental and control groups are attributed to the effect of the independent variable.
From the beginning of this research design—the assignment of random subjects to experi- mental and control groups—threats to validity are being controlled. If the selection is truly random, then biases, regression toward the mean, and all other internal threats are initially controlled in the experiment. However, as the experiment progresses over time, it is practically impossible to control for maturation and maturation interaction. The following example is a description of a pretest–posttest control group design.
A random selection of mothers at the local community health station were chosen to test if there were differences in satisfaction levels between the random group of mothers who were in the experimental group and those in the control. The study wanted to determine if they could eliminate, as a cost-containment technique, nurses at the intake level and run the clinic with ancillary health- professionals, nurse practitioners, and doctors. The experimental group was to be interviewed by a nurse to take a history of the current complaint, whether this was a ‘‘well-baby care’’ visit, or a
‘‘sick-baby’’visit and answer any questions the mother may have about her child. After the initial visit by the nurse, the nurse practitioner or the doctor would come into the room to examine or treat the child. The control group would not receive the nurses’visit. Both groups would receive a pretest one month after being enrolled as patients. Then the independent variable would be introduced, and a posttest on both groups for customer satisfaction.
Solomon Four-Group Design R O X O R O O R X O
R O
Threatened Internally by Threatened Externally by Controls Internally
None None History
Maturation Testing Instrumentation Regression Selection bias Mortality
Selection interaction Controls Externally Testing interaction
The Solomon four-group design is thefirst experimental design presented that controls, to some extent, threats to generalizability or duplication. The design is set up where the component of the pretest–posttest control group design makes up thefirst two randomized groups. In addition to this experimental and control group, a third group is added that is not given a pretest but is exposed to the independent variable, and a fourth group that is given neither the pretest nor exposure to the independent variable. In the example of eliminating nurses at a clinic, there would be a group of clients who were not given the pretest for customer satisfaction but received the previsit by the nurse, and a fourth random group that received neither the pretest nor the experimental treatment of the nursing previsit.
Posttest Only Control-Group Design R X O
R O
Threatened Internally by Threatened Externally by Controls Internally
None None History
Maturation Testing Instrumentation Regression Selection bias Mortality
Selection interaction Controls Externally Testing interaction
The posttest only control-group design is also known as the randomized posttest design (O’Sullivan and Rassel, 1995). This design also protects the experiment from the same threats to internal and external validity as the Solomon four-group design. The posttest only control-group design also presents the same opportunity for generalization as the Solomon design. However, there are times when it may not be practical, feasible, or possible to administer a pretest. The option of a pretest is removed when we are studying large groups of subjects, there are no pertinent questions to be asked in a pretest, or there is not adequate funding to administer a pretest to the experiment’s participants. Furthermore, the applica- tion of a pretest takes enormous time and may not be of value. Consider the following example.
The federal government was considering changing the style of uniforms for the Air Force. Since the end of World War II, there was much discontent among members of the Air Force that the uniforms were drab and generally lacking in the type of military style that may be found in the other branches of the military. Although the discontent over the uniforms ebbed and flowed over the years, recently, recruitment quotas were consistently below expected levels and it was thought that changing the uniforms would enhance recruitment. To see if the new uniform would increase recruitment, new recruits, from random cities on the West Coast, were given the new uniforms, while new recruits from random cities on the East Coast were issued the old uniforms. The result of the experiment was that recruitment quotas were met on the West Coast but remained at a continuous low level on the East Coast.
This posttest design experiment is one that illustrates the point that it would be difficult to administer a pretest to every adult eligible to join the Air Force; nevertheless, the posttest was able to show that there was an association between the independent and dependent variables.