Basic Concepts of Probability
6.2 The Taming of Chance
Historically the process of decision making in situations of uncertainty was often dealt with by techniques designed to let the fates or the gods control the out- comes; for instance, procedures like the casting or drawing of lots is often refer- enced in ancient writings. With the rise of Christianity in the West, this practice was largely replaced through the direct petitioning to the Christian God for guidance in the making of weighty decisions. However, starting in the seven- teenth century, several natural philosophers and mathematicians started to speculate about how to deal with more mundane uncertainties associated with daily life (e.g. Reeves, 2015). One such activity that spurred a good deal of inter- est concerned gambling. Many historians of science point to an exchange of let- ters between Blaise Pascal and Pierre De Fermat regarding how to settle out the stakes involved in an interrupted gambling venture that could not be finished as a critical event in the development of probability theory (e.g. Katz, 1993; Weis- berg, 2014). We can easily imagine a gambling situation where one contestant would be ahead and would be more likely to win if the contest was to continue, but that this outcome would not be certain. Splitting the winnings in half would seem unfair to the person who is ahead. However, giving this person the full amount would seem unfair to the other contestant who, although sitting in a disadvantaged position, had not yet lost and had some hope of recovering to win. The discussion started by Pascal and Fermat generated a lot of further dis- cussion over the next 300 years regarding how to quantify the uncertainty in situations such as these. (See also Box 2.2 presented earlier in this text.)
A major breakthrough came when it was realized that“games of chance”were probably not that at all but that the outcomes of rolls of the dice and the location of a card in a deck when shuffled were regulated and law-like, if only detailed information about preconditions and processes were available. However, given that such precise information could never be fully known, the key was to becomewillfully ignorantof the myriad of small effects that influence outcomes and choose to focus on the more general truths. (See Box 6.1 for further information about the use of willful ignorance in probabilistic thinking.) If we should choose to become ignorant of any particular roll of the dice and instead focus on what tends to happen across several events (e.g. several rolls
6.2 The Taming of Chance 165
An aspect of probabilistic thinking that seems to have been lost in most modern discussions of probability, and that may be partly responsible for the reproduc- ibility problem in the social and medical sciences, is the realization that uncertainty is not merely the quantification of likelihood, but is also influenced by a clear understanding of the situation; let us use the term“clarity”for a lack of a better one. Now“likelihood”(or “risk” as it is sometimes called) is usually understood as something that can be quantified numerically, like the odds of selecting a spade from a deck of cards. This predisposes that the conditions are well understood as well as the relative frequencies of favorable and unfavorable options to be known. Every judgment call dealing only with likelihood has some known degree of risk to be wrong. Modern probability theory, since the 1930s, has almost exclusively focused on only this aspect of uncertainty. However, uncertainty, which is what probability theory was designed to address, is not restricted to merely this more objective quantifica- tion. Clarity is also a necessary component. By clarity, it is meant the degree to which the situation one is speculating about is being properly conceptualized.
For instance, when we say that the likelihood of getting snake eyes when rolling two dice is 1/36, we are assuming that we have full clarity regarding the situation (e.g. what is known as“snake eyes,”the features of the two dice, and so on). We realize that gambling on this outcome has a certain amount of risk associated with it (actually a lot of risk associated with it!), but this risk is at least quantifiable. However, if we think about it for a few minutes, it should be easy to see that we may be overlooking several aspects of the probabilistic situation out of convenience. For instance, what if each side of each die is not equally likely? What if there are actually 10 sides to each die? What if the numbers on the die do not start with 1? Now, this is a simple example where there is most likely a lot of initial clarity regarding the situation; or if not, issues revolving around the clarity of the situation could be easily resolved by simply looking at the dice to be rolled. However, in many research situations, the degree of clarity that a researcher has regarding the situation is not as easy as inspecting the dice. Further compounding the situation, a researcher’s lack of clarity may be best understood as a qualitative variable as opposed to a quantitative one, and as a result, not something that could even be factored into modern probability theory. Acknowledging this limitation, however, might leave the researcher without a clear path forward in their efforts to investigate.
So, oftentimes researchers, perhaps unknowingly, engage in what some theorists call“willful ignorance”regarding issues of clarity so as to focus only on the issue of likelihood when testing their hypotheses. By essentially ignoring issues of clarity and focusing exclusively on likelihood, modern probability theory has created a false sense of confidence in the stated outcomes of some statistical analyses. Some argue that our failure to address this fundamental issue has led to the challenging state in which many areas of scientific investi- gation currently find themselves (e.g. Byers, 2011; Weisberg, 2014).
To help illustrate this further, imagine a medical situation where a patient has arrived with a set of symptoms and an unknown diagnosis. There are two judgments that need to be made: First, what is the nature of the illness, and second, what form of treatment would be best? It might be helpful to consider
of the die), we can construct a fairly accurate representation of the set of outcomes and their likelihood. For example, Figure 6.1 shows a representation of the set of outcomes associated with the rolling of two standard dice. Histor- ians suggest this gained insight to be critical in the development of probability theory.
the first of these two judgements as being representative of the issues of clarity.
The doctor, relying on clinical expertise, past experiences, logic, and judgment, moves to increase clarity by rendering a diagnosis. Yes, some probabilistic information may have been used in this analysis, but primarily this was not an exercise in probability. Now, the second question of treatment may be much more representative of the issues related to likelihood and probability. Based on the outcomes of previous studies employing different treatments, the doctor may choose to prescribe one form of treatment over another due to the likelihood of a favorable outcome derived from a quantification of relevant data.
Research situations oftentimes involve both forms of uncertainty: likelihood and clarity. Moreover, when research replication efforts take place, issues of clarity have oftentimes not properly been noted or documented by the original researchers and are not properly considered by the replicating researcher. This disconnect can lead to contradictory findings when replication takes place and thus great confusion as to what is actually going on. In the interest of accuracy and the public confidence in science, it would be wise for researchers to begin to consider seriously the issue of clarity when setting up their investigations, writing up their procedures, and drawing conclusions from their data.
2 3 4 5 6 7 8 9 10 11 12
1/36 1/18 3/36 1/9 5/36 1/6 7/36
Dice roll outcome
Likelihood
Figure 6.1 A probability distribution representing a single roll of a pair of six-sided dice.
6.2 The Taming of Chance 167