CHAPTER
142 EXPERIMENTAL DESIGNS
CHAPTER OBJECTIVES
After completing Chapter 7, you should be able to:
1. Distinguish between causal and correlational analysis.
2. Explain the difference between lab and field experiments.
3. Explain the following terms: nuisance variables, manipulation, experimental and control groups, treatment effect, matching, and randomization.
4. Discuss internal and external validity in experimental designs.
5. Discuss the seven possible threats to internal validity in experimental designs.
6. Describe the different types of experimental designs.
7. Discuss the Solomon four-group design and its implications for internal validity.
8. Apply what has been learned to class assignments and exams.
Consider the following three scenarios.
Scenario A For some time now, there has been the feeling that individual companies and the economy will be better served if executive compensation contracts are entered into, making the CEOs accountable for performance. Currently the top executives are compensated irrespective of their performance, making them per- manent corporate fixtures.
A switch to the new mode is likely to irk the chiefs, but is definitely worth a try if it does work. But how can we be sure that it would work?
Scenario B A study of absenteeism and the steps taken to curb it indicate that companies use the following incentives to reduce it:
14% give bonus days 39% offer cash
39% present recognition awards 4% award prizes
4% pursue other strategies Asked about their effectiveness,
22% of the companies said they were very effective 66% said they were somewhat effective
12% said they were not at all effective
CAUSAL RELATIONSHIPS 143 What does the above information tell us? How do we know what kinds of incen- tives cause people not to absent themselves? What particular incentive(s) did the 22% of companies that found their strategies to be ―very effective‖ offer? Is there a direct causal connection between one or two specific incentives and absenteeism?
Scenario C The dagger effect of layoffs is that there is a sharp drop in the commitment of workers who are retained, even though they might well understand the logic of the reduction in the workforce.
Does layoff really cause employee commitment to drop off, or is something else operating in this situation?
The answers to the questions raised in Scenarios A, B, and C might be found by using experimental designs in researching the issues.
In the previous chapter we had touched on experimental designs. In this chap- ter, we will discuss both lab experiments and field experiments in detail. Exper- imental designs, as we know, are set up to examine possible cause and effect relationships among variables, in contrast to correlational studies, which exam- ine the relationships among variables without necessarily trying to establish if one variable causes another.
To establish that variable X causes variable Y, all three of the following con- ditions should be met:
1. Both X and Y should covary [i.e., when one goes up, the other should also simultaneously go up (or down)].
2. X (the presumed causal factor) should precede Y. In other words, there must be a time sequence in which the two occur.
3. No other factor should possibly cause the change in the dependent variable Y.
It may thus be seen that to establish causal relationships between two variables in an organizational setting, several variables that might covary with the depen- dent variable have to be controlled. This would then allow us to say that vari- able X and variable X alone causes the dependent variable Y. Useful as it is to know the cause-and-effect relationships, establishing them is not easy, because several other variables that covary with the dependent variable have to be con- trolled. It is not always possible to control all the covariates while manipulating the causal factor (the independent variable that is causing the dependent vari- able) in organizational settings, where events flow or occur naturally and nor- mally. It is, however, possible to first isolate the effects of a variable in a tightly controlled artificial setting (the lab setting), and after testing and establishing the cause-and-effect relationship under these tightly controlled conditions, see how generalizable such relationships are to the field setting.
Let us illustrate this with an example. Suppose a manager believes that staffing the accounting department completely with personnel with M.Acc. (Master of Accountancy) degrees will increase its productivity. It is well nigh impossible to
144 EXPERIMENTAL DESIGNS
transfer all those without the M.Acc. degree currently in the department to other departments and recruit fresh M.Acc. degree holders to take their place. Such a course of action is bound to disrupt the work of the entire organization inasmuch as many new people will have to be trained, work will slow down, employees will get upset, and so on. However, the hypothesis that possession of a M.Acc.
degree would cause increases in productivity can be tested in an artificially cre- ated setting (i.e., not at the regular workplace) in which an accounting job can be given to three groups of people: those with a M.Acc. degree, those without a M.Acc. degree, and a mixed group of those with and without a M.Acc. degree (as is the case in the present work setting). If the first group performs exceed- ingly well, the second group poorly, and the third group falls somewhere in the middle, there will be evidence to indicate that the M.Acc. degree qualification might indeed cause productivity to rise. If such evidence is found, then planned and systematic efforts can be initiated to gradually transfer those without the M.Acc. degree in the accounting department to other departments and recruit others with this degree to this department. It is then possible to see to what extent productivity does, in fact, go up in the department because all the staff members are M.Acc. degree holders.
As we saw earlier, experimental designs fall into two categories: experiments done in an artificial or contrived environment, known as lab experiments, and those done in the natural environment in which activities regularly take place, known as the field experiment.