Guaranteeing such similarity between the two groups is far from straightforward given the difficulty that social researchers have with manipulating and controlling an individual’s circumstances. It should be borne in mind that the laboratory conditions that are available to those working in the natural sciences can often be difficult to replicate when studying phenomena in the social world.
The steps to be taken when conducting an experiment are set out in Figure 6.3.
Group Pre-test Stimulus Post-test
Experimental O1→ X →O2
Control O3→ → →O4
FIGURE 6.2 PRE-TEST – POST-TEST TWO-GROUP EXPERIMENT
1. Determine the dependent variable and independent variable in your study.
2. Choose the level of treatment to be applied (i.e. what test to use, and how often to conduct it).
3. Draw a representative sample from your target population.
4. Impose as many controls as are possible on other parameters that could affect the conditions of the experiment.
5. Divide the research participants into an experimental group and a control group.
6. Pre-test both the experimental group and the control group using an appropriate instrument.
7. Expose the experimental group to the treatment.
8. Measure both the experimental and control group again using the same instrument.
9. Collect data from both the pre-testing and post-testing of both groups.
10. Analyse data to determine the effect of the treatment on the experimental group.
FIGURE 6.3 THE 10 STEPS IN AN EXPERIMENT
experimental and control groups are as identical as is possible in every respect, with the obvious exception of the exposure of the experimental group to the treatment.
However, many events may be outside of the control of even the most careful and resourceful experimenter:
1. History – events that may occur in society between the first and second measurements which could explain the change in the dependent variable. For example, in carrying out an experiment on workplace morale something may happen in the experimental group’s workplace and not in the control group’s workplace. Even outside influences such as particularly good or bad weather (most definitely outside of the control of the experimenter) may impact on the experiment.
2. Maturation – other processes that may be influenced by the passage of time between the two tests. This obviously depends on the time that elapses between the pre-test and the post-test. Where the gap is a considerable one a variety of factors, including personal life events, may need to be taken into account.
3. Mortality – this happens when some of the experimental or control group leave the experiment thus affecting the two groups’ comparability. Again, this will be a function of the length of the experiment, and will be more of an issue where the experiment takes place over a longer period.
4. Instrumentation – any variation in the test whether between the two groups or over the two tests. It will be very important to ensure that the same instrument is used for both the pre-test and the post-test. If this is not the case, then the observed difference could be the result of a variation in the measurement process.
5. Testing – the possibility that the test itself may explain the change in the dependent variable. For example, in the course of carrying out an experiment on the extent to which exposure to party election broadcasts might affect the level of people’s political knowledge, the very act of actually taking part in the experiment itself might affect people’s test score. It might get them into the
‘mode’ of being tested – perhaps by relieving any pre-test nerves, and increasing their general pre-test level of confidence. If this is the case, it is possible that any change you record may actually be the result of your conducting the research, not of showing them the party election broadcasts.
External validity
The main threat to external validity is that the knowledge that people are participating in a study is likely to impact on the behaviour of the research participants. If, for example, the people in the study know that you are observing them to see whether their morale has improved after the introduction of a new management initiative, they may act in a particular way deliberately. This is known as the problem of reactivity. They may display markedly positive or negative
reactions, depending on their disposition towards their employer. This phenomenon has become known as the ‘Hawthorn Effect’ after a research project that was carried out at the Hawthorne Works in Chicago in the 1920s where the workers in question ‘acted up’ for the benefit of the researchers. Reactivity is used as a methodological justification for using a level of deception in experimental research. As we shall see in Chapter 7, it is an issue that confronts the researcher intent on using a qualitative participant observation approach – whether to do so overtly or covertly.
Ethical issues in experimental research
Experimental research raises a number of ethical dilemmas concerning the manner in which researchers treat people. For example, some researchers may consider it inappropriate to ‘manipulate’ human beings in the same way as laboratory animals such as mice and guinea pigs are treated. An extensive discussion of the ethical implications of experimental research can be found in Chapter 4.
An additional ethical consideration in relation to experimental research is the question of including or excluding people from a study in which some may benefit.
For example, an experiment may be designed to measure the effect that the introduction of CCTV has in reducing crime in certain residential neighbourhoods.
In this instance, it may be argued that researchers occupy a too powerful position in being able to decide which area (and therefore which residents) will benefit from the experiment and which will not benefit. One way around such a charge of unethical abuse of power by the researcher is to take a change that is occurring anyway, and collect or obtain statistics from before the change, during it, and after it. This is known as a quasi-experiment.
Defining change accurately
Another problem that confronts social scientists in using experiments is being able to accurately establish exactly what it is that they will be looking for as an outcome in their research. Experimentation in the natural sciences is not usually faced with such a problem. For example, a chemist may want to know whether heating a particular object causes its temperature to rise above a definite point. In this case, the experimenter will know exactly what she or he is seeking to measure – a precise temperature at a predetermined time. After this temperature has been taken, the experimenter will be able to state clearly the outcome of the experiment.
However, in the great majority of cases the social world does not offer such clear-cut situations. What if our social experiment wants to measure the effect of changing practices in the workplace? In implementing some new practice at work, perhaps to enhance morale, the experimenter will need to define what will count as an improvement before starting to make any measure. This must be done in
advance of the experiment. Otherwise, defining what counts as success afteryou have carried out the experiment, that is initiated the programme of workplace changes, is likely to be influenced by what you see happening in the early stages of the new initiatives.
One obvious way of determining measurement outcomes is to consult the literature in the chosen field of research to see what the expert or professional convention regards as acceptable. You may, for instance, want to establish what counts as ‘improved morale at work’. Before you initiate your experiment you will need to think very carefully about the outcome measures that you will use to identify changes in employee morale.
Activity 6.1 Experimental research design
Design an experiment to investigate the hypothesis that attending staff development seminars on equal opportunities issues will affect a person’s attitudes towards racism. As you do so, follow the steps, and consider the issues set out in Figure 6.1, earlier. What ethical issues, if any, do you think that you will need to consider in this experiment?