that when we don’t have any reason to think that any two basic outcomes are not equally likely to occur, we can base our measure of probability on counting basic outcomes. In classical probability, these basic outcomes are calledsimple events.
Mutually exclusive events
Finally, there is an important concept that applies to all three types of probability, but is best understood in the case of classical probability. Note that we have been considering different values (black and white, or heavy wool, fine wool, and mutton) for only a single variable (color or breed) at a time. This was a trick to ensure that all of the events were what is called mutually exclusive. Two events (and the probabilities of those events) are mutually exclusive if the fact that one happens means that the other cannot possibly have happened. If the color of the sheep we pick is black, its color cannot be white. If the sheep is a mutton merino, it cannot be a heavy wool merino. This is always true for different values of a single variable.
Things get a bit more complex when we consider more than one variable at a time. If the sheep we pick is black, it might or might not be a fine wool merino. We can’t really know unless we know the relationship between the colors and the breeds for our entire flock. If one or both of our two fine wool merinos is black, then the event of picking a black sheep is not mutually exclusive of the event of picking a fine wool merino. However, if it happens that both of our fine wool merinos are white, then picking a black sheep means we definitely did not pick a fine wool merino, and vice versa. The two events are mutually exclusive despite being defined by values on two different variables.
In cases such as these, we need a new rule for assigning values for our probabilities. This time the rule depends on Hume’s assumption (discussed in Chapter 1 ‘‘Statistics for Business’’) that the future will be like the past, which is key to the philosophy of science, which provides the model for the second type of probability, based on the theory ofrelative frequency.
In science, the assumption that the future will be like the past leads us to assume that, under the same circumstances, if we do things exactly the same way, that the results (called theoutcome) will come out the same. The basic idea behind a scientific observation or experiment is that things are done in such a very carefully specified and documented way that the next person who comes along can read what we have done and do things so similarly that she or he will get the same results that we did. When this is true, we say that the observation or experiment isreplicable.Replicability is the heart of Western science.
Frequentist theoreticians have an imaginary model of the scientific experiment called the simple experiment. They define simple experiments in terms of gambling devices and the like, where the rule of insufficient reason applies and we know how to calculate the probabilities. Then they show that, in the ideal, simple experiments, repeated many times, will produce the same numbers as classical probability. The big advantage to frequentist probability is that, mathematically, simple experiments work even when the underlying simple events arenot equally likely.
The first simple experiment that is usually given as an example is a single flip of a coin. Then the frequentist moves on to dice. (Trust us. Heads still turn up 50–50 and each side of the die shows up 1/6th of the time. Everything works.) We will skip all this and construct a simple experiment with our flock of sheep. Suppose we put all of our flock into an enclosed pen. We find someone who is handy with a lasso, blindfold her, and sit her up on the fence.
Our lassoist then tosses her lasso into the pen and pulls in one sheep at a time.
(Simple experiments are theoretical, and don’t usually make much sense.) The lassoing is our model ofsampling, which we learned about in Chapter 2
‘‘What Is Statistics?’’ Importantly, after the sheep is lassoed and we take a look at it, we then return it to the flock. (Like we said, these experiments don’t make much sense.) This is calledsampling with replacement.
TIPS ON TERMS
Sampling with replacement. In the context of an imaginary simple experiment, an act that determines a single set of one value for each variable in such a way that the likelihood of the different values does not change due to the act of sampling itself. Examples are: the flip of a coin; the roll of a pair of dice; the
drawing of a card from a deck of cards, after which the card is placed back in the deck.
Note that things like flipping a coin or rolling dice, which we might not ordinarily call ‘‘sampling’’ count as sampling in statistics. When we flip a coin, we are said to be sampling from the space of possible outcomes, which are the events, heads and tails.
This is sampling from a set of abstract events, rather than from a set of physical objects. What makes it sampling with replacement is that, once you flip a coin, the side that lands up doesn’t get used up for the next toss. In terms of the odds, nothing changes from one flip of the coin, or one roll of the dice, to the next. With cards, in order to keep the odds the same, we have to replace the card drawn into the deck, hence the expression,with replacement.
Sampling without replacement. In the context of an imaginary simple experiment, an act that determines a single value for each variable in such a way that the likelihood of the different values changes due to the act of sampling itself.
Examples are: the drawing of a card from a deck of cards, after which the card is set aside before the next draw; choosing a name from a list and then checking off the name.
The vast majority of statistical techniques, and all that we will cover here in Business Statistics Demystified assume that sampling is done with replacement.
Mathematically, sampling without replacement is very complicated because, after each subject unit is removed from the population, the size of the population changes.
As a result, all of the proportions change as well. However, sampling with replacement does not make sense for many business applications.
Consider the example of surveying our customers: we have a list of customers and are calling them in random order. In order to sample with replacement, we would have to keep a customer’s number on the list even after we’d interviewed them once.
But if we do that, we might pick the exact same phone number again and have to call that same customer! (‘‘Hi, Mr. Lee! It’s me, again. Sorry to bother you, but I need to ask you all those same questions again.’’)
Of course, in these sorts of cases, sampling with replacement is never really done, but the statistics that are used assume that statistics with replacement is always done.
The trick is that, mathematically, if the population is infinitely large, sampling without replacement works identically to sampling with replacement. If our population is finite, but very large compared to the total size of our sample, we can pretend that it is infinite, and that all sampling is sampling with replacement, without generating too much error.
What is the probability that we will lasso a black sheep? According to classical probability theory, it is 5/12. Let’s have our lassoist lasso sheep 120 times, releasing the sheep afterwards each time. We will probably not find that we have lassoed exactly 50 sheep (equal to 5/12 times 120), but we will be pretty close. In short, we can estimate the true probability by repeating our simple experiment, countingthe different types of outcomes (black sheep or
white sheep, in this case), and calculating the proportion of each type of outcome. An advantage of frequentist probability is that it uses proportions, just like classical probability. The difference is that, where classical probability involves counting the different possible types of simple events and assuming that each is equally likely, frequentist probability involves repeating a simple experiment and counting the different outcomes.
HANDY HINTS
Later on, after we have learned some additional tools, we will see that the frequentists have a better way of performing simple experiments in order to estimate the true probability. Without giving too much away, let’s just say that it turns out that it is better to do ten experiments, lassoing twelve times for each experiment, than doing one experiment lassoing 120 times.
Why is this difference important? The reason is that the outcomes of simple experiments don’t have to be equally likely. If our simple experiment is to flip a coin or roll a die, the outcomes are heads or tails, or the number on the top face of the die, and the outcomes can safely be assumed to be equally likely. But what about our simple experiment lassoing sheep? If we think of the outcome as being which of the 12 individual sheep gets lassoed, then each outcome is equally likely. But, suppose we aren’t on familiar terms with all of our sheep, and don’t know them all individually? We can think of the outcomes as lassoing any black sheep and lassoing any white sheep. Unless we count all the sheep in our flock and apply classical probability, we don’t know what the relative likelihoods of lassoing a black sheep or a white sheep are, and we certainly cannot assume they are equal. White sheep are vastly more common than black sheep, and this is a very good reason to assume the likelihoods of picking each type are notequal.
There are two sorts of cases where it is good to have the frequentist approach, one where classical probability can be very hard to apply, and one where it is impossible to apply.
First, suppose we had a huge flock of sheep. We aren’t even sure just how many sheep we have. We want to know the probability that we will pick a black sheep. If we define the outcome of our experiment as ‘‘black sheep’’ and
‘‘white sheep,’’ we can estimate the probability of picking a black sheep without having to count our entire flock, or even being able to tell one sheep from another, except for their color. This illustrates both the convenience of the frequentist approach and the power of the sampling upon which it depends.
Second, so long as we can construct a simple experiment to sample some attribute of our subjects, we can estimate the probabilities. This is very useful in cases like the weather, profits and losses, and level of education (discussed above), where we have no way of counting anything except the results of sampling. Often, we do not even have to be able to conduct the simple experiment. (No blindfolds required!) We can just collect the data for our statistical study according to best practices, and treat the numbers as if they were the outcomes of simple experiments. This illustrates how probability based on relative frequency can be very useful in real world statistical studies.