Basic Concepts of Probability
Spotlight 6.1 Thomas Bayes and Bayesianism
Thomas Bayes (1701–1761) was a nonconformist (a term used for those who had problems with the Church of England), English cleric, statistician, and philosopher (Bellhouse, 2001). Although his interests were broad and his writings ranged from theology to a defense of Newton’s ideas regarding cal- culus, he is most well known for a posthumously published paper by a friend in which he formulated a specific case of the theorem that now bears his name (Bayes’ theorem; see Section 6.9). His theorem solved the problem of inverse probability (also known as the“confusion of the inverse”or“con- ditional probability fallacy”). As a result of his broad contributions to math- ematics, Bayes was elected as a Fellow of the Royal Society sometime in the mid-1700s and prior to his death in 1761. This was the most prestigious Brit- ish association for individuals who had been deemed to have made substan- tial contributions to the improvement of what was called “natural knowledge.”
The currently used term Bayesianism comes not only from Bayes’own writ- ings but also from the work of a French scholar named Pierre-Simon Laplace (1749–1827) who used Bayes’ideas to develop a way to think probabilistically about events that may not be part of a known“reference class”(e.g. Stigler, 1986) or what we have previously referred to in our text as the“sample space.” This way of thinking allowed probability theorists to reason about the accuracy of various speculative hypotheses by first assigning prior probabilities, which were to be later updated to posterior probabilities in the light of new and rel- evant data in a recursive system of thinking. What we now call Bayesianism (or Bayesian probability) is the standard set of procedures and formulae utilized for this sequence of calculations.
Bayes theorem
P B A = P A B P B
P A B P B +P A notB P notB (Formula 6.8)
Let us try our hand at a typical Bayes’theorem problem.
■QuestionImagine it is true that 1% of 40-year-old women who participate in a routine screening have breast cancer. Further imagine that 80% of women with breast cancer will receive a positive reading from the mammogram screen procedure. However, 9.6% of women without breast cancer will also receive a positive reading from the mammogram screening procedure (this is sometimes referred to as a“false positive”result). Now suppose a 40-year-old woman is told that her mammogram screening is positive for breast cancer. What is the likelihood that she actually has breast cancer?
Solution
Step 1.Let us first transpose the variables in our example into the terms used by Bayes’theorem, namely,P(B|A),P(A|B),P(B),P(notB), andP(A|notB). Recall that we are trying to determine the probability that a person with a positive mammogram reading does indeed have breast cancer. This can be stated in probability language asP(breast cancer|positive reading). Since the formula is set up to findP(B|A), this means that eventAcorresponds to the eventpos- itive readingand eventBcorresponds to the eventbreast cancer.
Step 2.This means the following assignments should be true:
P(A|B) =P(positive reading|breast cancer)
P(B) =P(breast cancer)
P(A|notB) =P(positive reading|not breast cancer)
It follows then that
P(A|B) =.8
P(B) =.01
P(A|notB) =.96
And we can deduce that
P(notB) =.99
Step 3.Use Bayes’theorem to solve the equation
P B A = P A B P B
P A B P B +P A notB P notB
P B A = 8 01
8 01 + 096 99 = 078or about7 8
6.9 Bayes’Theorem 187
This may seem surprising to us. We probably thought the chance of this lady actually having breast cancer was much higher. However, as we stop to think about it, we may realize that about 10% of the 40-year-old ladies getting screened who do not have breast cancer (which is about 99% of them) are going to get a positive mammogram. That’s a large number of false positives. This is intentional, as the inconvenience and sense of alarm that may result from receiving a false positive pale in comparison to the need to avoid false negatives.
In reality, positive screens for breast cancer result in secondary screening pro- cedures that are more sensitive and designed to distinguish between these initial false positives and true positives.■
Summary
This chapter and Chapter 7 serve as the theoretical “bridges” that connect descriptive statistics to inferential statistics, the remaining material in the text- book. Inferential statistics, the ability to draw inferences about populations based on known properties of samples drawn from those populations, is dependent upon several concepts related to probability theory and hypothesis testing.
Probability theory started in the seventeenth century by several key thinkers who decided it was best to approach situations with a sense of willful ignorance regarding specific outcomes and the many idiosyncratic issues associated with them and to rather focus on determining likelihood over multiple trials. Out of this thinking emerged modern probability theory.
Probability can be understood mathematically as a proportion that ranges from 0 to 1. A probability of 0 means that an event is certain to not occur; a probability of 1 means that an event is certain to occur. A distinction is made between sampling with and without replacement. Sampling with replacement is a method of sampling whereby a member of a population is randomly selected and then returned to the population before the next member is selected.
Sampling without replacement is a method of sampling in which a member of a population is not returned to the population before selecting another member of the population. Since hypothesis testing concepts are based on determining likelihood when in situations with replacement, this chapter will restrict itself to these situations.
There are various formulas that can be used to determine specific prob- abilities: the basic probability formula, the“or”formulas, the“and”formulas, the conditional probability formula, and Bayes’ theorem. To distinguish between the “or” formulas, the concept of “mutual exclusivity” is needed.
To distinguish between the “and” formulas, the concept of “independent” is needed. Bayes’ theorem allows us to avoid the problem of inverse prob- ability (also called the“confusion of the inverse”or“conditional probability fallacy”).
Key Formulas
Probability of favorable event P= number of favorable events
total number of events (Formula 6.1)
Addition rule formula for two mutually exclusive events P(A or B) =P(A) +P(B) (Formula 6.2)
Addition rule formula for more than two mutually exclusive events P(A or B or C or…Z) =P(A) +P(B) +P(C) + +P(Z) (Formula 6.3) Addition rule formula for two events
P(A or B) =P(A) +P(B)−P(A and B) (Formula 6.4) Multiplication rule formula for two independent events P(A and B) =P(A)P(B) (Formula 6.5)
Multiplication rule formula for two events P(A and B) =P(A|B)P(B) (Formula 6.6) Conditional probability formula
P A B =P A and B
P B (Formula 6.7) Bayes’theorem
P B A = P A B P B
P A B P B +P A notB P notB (Formula 6.8)
Key Terms
Inferential statistics Multiplication rule
Probability Probabilistic independence
A priori (classical) approach Probabilistic dependence A posteriori approach Conditional probability
Addition rule Sample space
Mutually exclusive events Bayes’theorem
Questions and Exercises
1 Together, this chapter and Chapter 7 allow the researcher to (please select the best answer):
a Understand how to run inferential statistics
b Determine the nature of samples from populations and the nature of populations from samples
Questions and Exercises 189