So far we have seen that ifXis a discrete random variable, the expected value isE(X) =P
k∈SkP(X=k), whereS is the sample space ofX. For a continuous random variableX, we have
E(X) = Z
S
xf(x)dx,
where f is the density function of X andS is the set of points inR for whichf(x)>0. The variance can be defined as
Var(X) =E[(E(X)−X)2],
where this definition holds for both continuous and discrete random variables.
The following propositions gives some basic properties that hold for both discrete and continuous random variables.
Proposition 7.3.1. LetX andY be random variables, and letabe a real number. Then, (a) E(a) =a
(b) E(aX) =aE(X)
(c) E(X+Y) =E(X) +E(Y) (d) Var(aX) =a2Var(X) (e) Var(X) =E(X2)−E(X)2 (f ) Var(X+a) =Var(X)
Next we need to define what it means for two random variables to have some sort of relationship to each other. We define the conditional probability that X takes on a specific valuex, given thatY is known to take on a certain value y. We denote this probability byP(X =x|Y =y).
The condition that Y = y may simply change the sample space for X. For example, suppose we are to draw 2 cards from a standard 52-card deck without replacement. The probability of selecting aKing on the second draw, given that we know a King was drawn on the first draw isP(K2nd|K1st) = 513. If instead we know a King was NOT drawn on the first draw, we haveP(K2nd|K¯ 1st) = 514.
The basicmultiplicative law of probability states that
P(X=xandY =y) =P(X =x)P(Y =y|X =x). (7.4) Definition 7.3.2. LetX andY be discrete random variables with sample spacesS(X)andS(Y)respectively.
We say thatX andY areindependentifP(X =x|Y =y) =P(X =x)for all x∈S(X), y∈S(Y).
In other words,X andY are independent if the value ofY has no effect onX. The definition is symmetric, as it is possible to prove that ifP(X =x|Y =y) =P(X =x) thenP(Y =y|X =x) =P(Y =y) as well.
From (7.4), we see thatX andY are independent if and only ifP(X =xandY =y) =P(X =x)·P(Y =y) for all x ∈ S(X), y ∈ S(Y). It also turns out that if X and Y are independent then their variances are additive, as stated in the following proposition.
Proposition 7.3.3. LetX andY be independent random variables. Then Var(X+Y) =Var(X) +Var(Y).
The Central Limit Theorem is often considered to be the most important Theorem in statistics. It says that if events are repeated many times, then the average result falls into a predictable pattern.
Theorem 7.3.4. (The Central Limit Theorem) Let{Xi}∞i=1 be a sequence of independent random variables, each having the same distribution with E(Xi) =µ and Var(Xi) =σ2. Let x¯n = n1Pn
i=1Xi be the sample mean for any combination ofn of theXi random variables. Then
(a) E(¯xn) =µ (b) Var(¯xn) =σ2/n
(c) The distribution of x¯n approaches a normal distribution asn→ ∞.
The fact that the distribution of sample means ¯xn approaches a normal distribution is difficult to prove, but it can be verified using simulation. We will discuss the normal distribution in more detail later on in this chapter.
Example 7.3.5
Consider the experiment of flipping a coinntimes. Let Xi be a random variable giving the number of heads from flipping a coin one time (either 0 or 1, each with probability 1/2).
(a) Compute the expected value and variance ofXi. (b) Let ¯xn =n1Pn
i=1Xibe the mean number of heads appearing after flipping a coinntimes. Compute the expected value and variance of ¯xn.
(c) Use simulation with 1000 trials to verify your answers to (a) and (b). Also use your code to estimate the probability of tossing more than 60% heads in 40 flips.
(d) Use your simulation to determine a 95% confidence interval for the proportion of heads that should appear after 160 flips.
. . . . (a) The expected value is
E(Xi) = 0·1 2+ 1·1
2 = 0.5, and the variance is
Var(Xi) = (0−0.5)2(1
2) + (1−0.5)2(1 2) = 1
4. (b) By Theorem 7.3.4,E(¯xn) =E(Xi) =p=12 and Var(¯xn) = Var(Xi)/n=4n1.
(c) Use the code segment
%coinFlips.m
1 numSims = 1000; %Initialize number of simulations 2 heads = zeros(numSims, 1); %Initialize vector of number of heads
3 numFlips = 40; %Initializen
4 for k=1:numSims %For all simulations
5 for i=1:numFlips %Fornflips
6 if rand<.5 %If a head was flipped
7 heads(k) = heads(k)+1; %Add 1 head tokth simulation
8 end
9 end
10 end
11 propHeads = heads/numFlips; %Convert vector from total heads to proportion 12 hist(propHeads) %Histogram of proportions (sample means) 13 ExBar = mean(propHeads) %Mean of sample means
14 VarxBar = var(propHeads) %Variance of sample means 15 sigma2 = 1/4/numFlips %Analytic variance
16 prop60 = mean(propHeads>.6) %Proportion of simulations w/ more than 60% heads We want to verify that E(¯xn) = 0.5 and Var(¯xn) = 4n1 . Line 13 displays the estimated mean of the sample means (proportion of heads from 40 trials), and line 14 displays the estimated variance of the sample means. Line 15 displays the theoretical variance to be compared with the estimated variance.
By running this code we should see that these values are close. Also, line 16 displays the proportions of simulations with more than 60% heads - this is used to estimateP(¯xn >0.6). Ifn= 40, you should see that the probability is roughly 0.077.
One way to shorten this code is to have MATLAB generate all of the numFlips random numbers in a single line. Then it is possible to set heads(k) to the number of random numbers that exceed 0.5. To do this, replace lines 5 through 9 with the following code:
r = rand(numFlips,1);
heads(k) = sum(r<.5);
It is actually possible to generate all of the numSims*numFlips random numbers in a single line which results in an even more efficient code.
(d) Suppose we want to make a 95% confidence interval for the proportion of heads that should appear after 160 flips. You may have learned how to use the normal distribution to make such an interval in a statistics class. In order to do that we would need to justify using the normal distribution, compute the mean and standard deviation of the proportion of heads, and use technology to compute probabilities (such as a z-table, TI-calculator, or computer software). However it is relatively simple to use your simulation to generate a good approximation of the confidence interval. You sort the values of the propHeads vector and determine the values in the 5th and 95th percentiles.
sortedHeads = sort(propHeads) sortedHeads(round(.05*numSims)) sortedHeads(round(.95*numSims))
In the first line, we use the sort function to sort the values in increasing order. Then we need to look at the index of the 5th percentile - to do so we can multiply the size of the vector by .05. In case this is not a whole number, use round to round to the nearest whole number. The last line computes the upper end of the confidence interval. You should see a confidence interval (0.4375,0.56875) forn= 160.
Refer to Activity A.0.56 and A.0.57 for more practice with the Central Limit Theorem.