What Are Important Characteristics of Probability Distributions and Where Do They Come From?
4.2 Variance and Covariance of Random Variables
120 Chapter 4 Mathematical Expectation distribution of X and is denoted by Var(X) or the symbolσ2X, or simply byσ2 when it is clear to which random variable we refer.
Definition 4.3: LetX be a random variable with probability distributionf(x) and meanμ. The variance ofX is
σ2=E[(X−μ)2] =
x
(x−μ)2f(x), ifX is discrete, and σ2=E[(X−μ)2] =
∞
−∞
(x−μ)2f(x)dx, ifX is continuous.
The positive square root of the variance,σ, is called thestandard deviationof X.
The quantityx−μin Definition 4.3 is called thedeviation of an observation from its mean. Since the deviations are squared and then averaged,σ2will be much smaller for a set of xvalues that are close to μthan it will be for a set of values that vary considerably fromμ.
Example 4.8: Let the random variableX represent the number of automobiles that are used for official business purposes on any given workday. The probability distribution for company A[Figure 4.1(a)] is
x 1 2 3
f(x) 0.3 0.4 0.3 and that for company B [Figure 4.1(b)] is
x 0 1 2 3 4
f(x) 0.2 0.1 0.3 0.3 0.1
Show that the variance of the probability distribution for company B is greater than that for company A.
Solution:For companyA, we find that
μA=E(X) = (1)(0.3) + (2)(0.4) + (3)(0.3) = 2.0, and then
σ2A= 3 x=1
(x−2)2= (1−2)2(0.3) + (2−2)2(0.4) + (3−2)2(0.3) = 0.6.
For companyB, we have
μB =E(X) = (0)(0.2) + (1)(0.1) + (2)(0.3) + (3)(0.3) + (4)(0.1) = 2.0, and then
σ2B= 4 x=0
(x−2)2f(x)
= (0−2)2(0.2) + (1−2)2(0.1) + (2−2)2(0.3) + (3−2)2(0.3) + (4−2)2(0.1) = 1.6.
4.2 Variance and Covariance of Random Variables 121 Clearly, the variance of the number of automobiles that are used for official business purposes is greater for companyB than for companyA.
An alternative and preferred formula for findingσ2, which often simplifies the calculations, is stated in the following theorem.
Theorem 4.2: The variance of a random variableX is
σ2=E(X2)−μ2. Proof:For the discrete case, we can write
σ2=
x
(x−μ)2f(x) =
x
(x2−2μx+μ2)f(x)
=
x
x2f(x)−2μ
x
xf(x) +μ2
x
f(x).
Since μ =
x
xf(x) by definition, and
x
f(x) = 1 for any discrete probability distribution, it follows that
σ2=
x
x2f(x)−μ2=E(X2)−μ2.
For the continuous case the proof is step by step the same, with summations replaced by integrations.
Example 4.9: Let the random variableX represent the number of defective parts for a machine when 3 parts are sampled from a production line and tested. The following is the probability distribution ofX.
x 0 1 2 3
f(x) 0.51 0.38 0.10 0.01 Using Theorem 4.2, calculateσ2.
Solution:First, we compute
μ= (0)(0.51) + (1)(0.38) + (2)(0.10) + (3)(0.01) = 0.61.
Now,
E(X2) = (0)(0.51) + (1)(0.38) + (4)(0.10) + (9)(0.01) = 0.87.
Therefore,
σ2= 0.87−(0.61)2= 0.4979.
Example 4.10: The weekly demand for a drinking-water product, in thousands of liters, from a local chain of efficiency stores is a continuous random variable X having the probability density
f(x) =
2(x−1), 1< x <2,
0, elsewhere.
Find the mean and variance ofX.
122 Chapter 4 Mathematical Expectation Solution:CalculatingE(X) andE(X2, we have
μ=E(X) = 2 2
1
x(x−1)dx= 5 3 and
E(X2) = 2 2
1
x2(x−1)dx=17 6 . Therefore,
σ2= 17 6 −
5 3
2
= 1 18.
At this point, the variance or standard deviation has meaning only when we compare two or more distributions that have the same units of measurement.
Therefore, we could compare the variances of the distributions of contents, mea- sured in liters, of bottles of orange juice from two companies, and the larger value would indicate the company whose product was more variable or less uniform. It would not be meaningful to compare the variance of a distribution of heights to the variance of a distribution of aptitude scores. In Section 4.4, we show how the standard deviation can be used to describe a single distribution of observations.
We shall now extend our concept of the variance of a random variable X to include random variables related toX. For the random variableg(X), the variance is denoted by σg(X)2 and is calculated by means of the following theorem.
Theorem 4.3: LetX be a random variable with probability distributionf(x). The variance of the random variableg(X) is
σ2g(X)=E{[g(X)−μg(X)]2}=
x
[g(x)−μg(X)]2f(x)
ifX is discrete, and
σ2g(X)=E{[g(X)−μg(X)]2}= ∞
−∞[g(x)−μg(X)]2f(x)dx ifX is continuous.
Proof:Sinceg(X) is itself a random variable with meanμg(X)as defined in Theorem 4.1, it follows from Definition 4.3 that
σ2g(X)=E{[g(X)−μg(X)]}.
Now, applying Theorem 4.1 again to the random variable [g(X)−μg(X)]2completes the proof.
Example 4.11: Calculate the variance of g(X) = 2X + 3, where X is a random variable with probability distribution
x 0 1 2 3
f(x) 14 18 12 18
4.2 Variance and Covariance of Random Variables 123
Solution:First, we find the mean of the random variable 2X+ 3. According to Theorem 4.1, μ2X+3=E(2X+ 3) =
3 x=0
(2x+ 3)f(x) = 6.
Now, using Theorem 4.3, we have
σ22X+3=E{[(2X+ 3)−μ2x+3]2}=E[(2X+ 3−6)2]
=E(4X2−12X+ 9) = 3 x=0
(4x2−12x+ 9)f(x) = 4.
Example 4.12: LetX be a random variable having the density function given in Example 4.5 on page 115. Find the variance of the random variableg(X) = 4X+ 3.
Solution:In Example 4.5, we found thatμ4X+3= 8. Now, using Theorem 4.3, σ4X+32 =E{[(4X+ 3)−8]2}=E[(4X−5)2]
= 2
−1
(4x−5)2x2
3 dx= 1 3
2
−1
(16x4−40x3+ 25x2)dx= 51 5 . Ifg(X, Y) = (X−μX)(Y−μY), whereμX =E(X) andμY =E(Y), Definition 4.2 yields an expected value called thecovariance ofX andY, which we denote byσXY or Cov(X, Y).
Definition 4.4: LetXandY be random variables with joint probability distributionf(x, y). The covariance ofX andY is
σXY =E[(X−μX)(Y −μY)] =
x
y
(x−μX)(y−μy)f(x, y)
ifX andY are discrete, and σXY =E[(X−μX)(Y −μY)] =
∞
−∞
∞
−∞(x−μX)(y−μy)f(x, y)dx dy ifX andY are continuous.
The covariance between two random variables is a measure of the nature of the association between the two. If large values ofX often result in large values ofY or small values ofX result in small values ofY, positiveX−μX will often result in positiveY−μY and negativeX−μXwill often result in negativeY−μY. Thus, the product (X−μX)(Y −μY) will tend to be positive. On the other hand, if largeX values often result in smallY values, the product (X−μX)(Y−μY) will tend to be negative. Thesignof the covariance indicates whether the relationship between two dependent random variables is positive or negative. WhenXandY are statistically independent, it can be shown that the covariance is zero (see Corollary 4.5). The converse, however, is not generally true. Two variables may have zero covariance and still not be statistically independent. Note that the covariance only describes the linear relationship between two random variables. Therefore, if a covariance betweenXandY is zero,X andY may have a nonlinear relationship, which means that they are not necessarily independent.
124 Chapter 4 Mathematical Expectation The alternative and preferred formula forσXY is stated by Theorem 4.4.
Theorem 4.4: The covariance of two random variablesX andY with meansμX andμY, respec- tively, is given by
σXY =E(XY)−μXμY. Proof:For the discrete case, we can write
σXY =
x
y
(x−μX)(y−μY)f(x, y)
=
x
y
xyf(x, y)−μX
x
y
yf(x, y)
−μY
x
y
xf(x, y) +μXμY
x
y
f(x, y).
Since
μX =
x
xf(x, y), μY =
y
yf(x, y), and
x
y
f(x, y) = 1 for any joint discrete distribution, it follows that
σXY =E(XY)−μXμY −μYμX+μXμY =E(XY)−μXμY.
For the continuous case, the proof is identical with summations replaced by inte- grals.
Example 4.13: Example 3.14 on page 95 describes a situation involving the number of blue refills X and the number of red refills Y. Two refills for a ballpoint pen are selected at random from a certain box, and the following is the joint probability distribution:
x
f(x, y) 0 1 2 h(y) 0 283 289 283 1528 y 1 143 143 0 37
2 281 0 0 281
g(x) 145 1528 283 1 Find the covariance ofX andY.
Solution:From Example 4.6, we see thatE(XY) = 3/14. Now μX =
2 x=0
xg(x) = (0) 5
14
+ (1) 15
28
+ (2) 3
28
= 3 4, and
μY = 2 y=0
yh(y) = (0) 15
28
+ (1) 3
7
+ (2) 1
28
= 1 2.
4.2 Variance and Covariance of Random Variables 125 Therefore,
σXY =E(XY)−μXμY = 3 14−
3 4
1 2
=−9 56.
Example 4.14: The fractionX of male runners and the fractionY of female runners who compete in marathon races are described by the joint density function
f(x, y) =
8xy, 0≤y≤x≤1, 0, elsewhere.
Find the covariance ofX andY.
Solution:We first compute the marginal density functions. They are g(x) =
4x3, 0≤x≤1, 0, elsewhere, and
h(y) =
4y(1−y2), 0≤y≤1,
0, elsewhere.
From these marginal density functions, we compute μX=E(X) =
1 0
4x4 dx= 4
5 andμY = 1
0
4y2(1−y2)dy= 8 15. From the joint density function given above, we have
E(XY) = 1
0
1 y
8x2y2dx dy= 4 9. Then
σXY =E(XY)−μXμY = 4 9−
4 5
8 15
= 4 225.
Although the covariance between two random variables does provide informa- tion regarding the nature of the relationship, the magnitude ofσXY does not indi- cate anything regarding the strength of the relationship, sinceσXY is not scale-free.
Its magnitude will depend on the units used to measure bothX andY. There is a scale-free version of the covariance called thecorrelation coefficientthat is used widely in statistics.
Definition 4.5: LetX and Y be random variables with covarianceσXY and standard deviations σX andσY, respectively. The correlation coefficient ofX andY is
ρXY = σXY σXσY
.
It should be clear to the reader thatρXY is free of the units ofX andY. The correlation coefficient satisfies the inequality−1≤ρXY ≤1. It assumes a value of zero whenσXY = 0. Where there is an exact linear dependency, say Y ≡a+bX,
126 Chapter 4 Mathematical Expectation ρXY = 1 if b > 0 and ρXY =−1 if b < 0. (See Exercise 4.48.) The correlation coefficient is the subject of more discussion in Chapter 12, where we deal with linear regression.
Example 4.15: Find the correlation coefficient betweenX and Y in Example 4.13.
Solution:Since
E(X2) = (02) 5
14
+ (12) 15
28
+ (22) 3
28
=27 28 and
E(Y2) = (02) 15
28
+ (12) 3
7
+ (22) 1
28
= 4 7, we obtain
σ2X= 27 28−
3 4
2
= 45
112 andσY2 =4 7 −
1 2
2
= 9 28. Therefore, the correlation coefficient betweenX andY is
ρXY = σXY
σXσY
= −9/56
(45/112)(9/28) =− 1
√5.
Example 4.16: Find the correlation coefficient ofX andY in Example 4.14.
Solution:Because E(X2) =
1 0
4x5 dx= 2
3 andE(Y2) = 1
0
4y3(1−y2)dy= 1−2 3 =1
3, we conclude that
σ2X= 2 3 −
4 5
2
= 2
75 andσY2 =1 3 −
8 15
2
= 11 225. Hence,
ρXY = 4/225
(2/75)(11/225) = 4
√66.
Note that although the covariance in Example 4.15 is larger in magnitude (dis- regarding the sign) than that in Example 4.16, the relationship of the magnitudes of the correlation coefficients in these two examples is just the reverse. This is evidence that we cannot look at the magnitude of the covariance to decide on how strong the relationship is.
/ /
Exercises 127
Exercises
4.33 Use Definition 4.3 on page 120 to find the vari- ance of the random variableXof Exercise 4.7 on page 117.
4.34 LetX be a random variable with the following probability distribution:
x −2 3 5
f(x) 0.3 0.2 0.5 Find the standard deviation ofX.
4.35 The random variableX, representing the num- ber of errors per 100 lines of software code, has the following probability distribution:
x 2 3 4 5 6
f(x) 0.01 0.25 0.4 0.3 0.04 Using Theorem 4.2 on page 121, find the variance of X.
4.36 Suppose that the probabilities are 0.4, 0.3, 0.2, and 0.1, respectively, that 0, 1, 2, or 3 power failures will strike a certain subdivision in any given year. Find the mean and variance of the random variableXrepre- senting the number of power failures striking this sub- division.
4.37 A dealer’s profit, in units of $5000, on a new automobile is a random variableX having the density function given in Exercise 4.12 on page 117. Find the variance ofX.
4.38 The proportion of people who respond to a cer- tain mail-order solicitation is a random variableXhav- ing the density function given in Exercise 4.14 on page 117. Find the variance ofX.
4.39 The total number of hours, in units of 100 hours, that a family runs a vacuum cleaner over a period of one year is a random variable X having the density function given in Exercise 4.13 on page 117. Find the variance ofX.
4.40 Referring to Exercise 4.14 on page 117, find σg2(X) for the functiong(X) = 3X2+ 4.
4.41 Find the standard deviation of the random vari- ableg(X) = (2X+ 1)2 in Exercise 4.17 on page 118.
4.42 Using the results of Exercise 4.21 on page 118, find the variance ofg(X) =X2, whereX is a random variable having the density function given in Exercise 4.12 on page 117.
4.43 The length of time, in minutes, for an airplane to obtain clearance for takeoff at a certain airport is a
random variableY = 3X−2, whereXhas the density function
f(x) = 1
4e−x/4, x >0 0, elsewhere.
Find the mean and variance of the random variableY. 4.44 Find the covariance of the random variablesX andY of Exercise 3.39 on page 105.
4.45 Find the covariance of the random variablesX andY of Exercise 3.49 on page 106.
4.46 Find the covariance of the random variablesX andY of Exercise 3.44 on page 105.
4.47 For the random variablesX andY whose joint density function is given in Exercise 3.40 on page 105, find the covariance.
4.48 Given a random variableX, with standard de- viationσX, and a random variableY =a+bX, show that ifb <0, the correlation coefficientρXY =−1, and ifb >0,ρXY = 1.
4.49 Consider the situation in Exercise 4.32 on page 119. The distribution of the number of imperfections per 10 meters of synthetic failure is given by
x 0 1 2 3 4
f(x) 0.41 0.37 0.16 0.05 0.01 Find the variance and standard deviation of the num- ber of imperfections.
4.50 For a laboratory assignment, if the equipment is working, the density function of the observed outcome X is
f(x) =
2(1−x), 0< x <1, 0, otherwise.
Find the variance and standard deviation ofX.
4.51 For the random variables X and Y in Exercise 3.39 on page 105, determine the correlation coefficient betweenX andY.
4.52 Random variablesXandY follow a joint distri- bution
f(x, y) =
2, 0< x≤y <1, 0, otherwise.
Determine the correlation coefficient between X and Y.
128 Chapter 4 Mathematical Expectation