1 2 3 variables
4.8 Data Transformations
Since the three life expectancy variables are similar, let us look at the simplified splom in Figure 4.14. The bottom row of the splom, with life.exp as the y- coordinate, shows an L-shaped pattern against both ppl.per.tv and ppl.per.physas the x-variables. We have learned (or will learn in this chapter and again in Chapter8) that straight lines are often helpful in understanding a plot.
There is no sensible way to draw a straight line here. The plot of the two potential x-variables against each other is bunched up in the lower-left corner. The bunching
Televisions, Physicians, and Life Expectancy
life.exp
70 75 80
70 75 80
55 60 65
55 60 65
ppl.per.tv
300 400 500
600 300 400 500 600
0 100 200 300
0 100 200 300
ppl.per.phys
20000 30000
20000 30000
0 10000 0 10000
Fig. 4.14 Televisions, physicians, and life expectancy.
suggests that a log transformation of theppl.*variables will straighten out the plot.
We see in Figure4.15that it has done so.
We also see that the log transformation hasstabilized the variance.By this we mean that theppl.per.phys ~ life.exppanel of Figure4.14has a range that fills the vertical dimension of the panel for values oflife.expnear 50 and that is almost constant for values oflife.explarger than 65. After the log transformation ofppl.per.physshown in Figure4.15, for any given value oflife.expwe ob- serve that the vertical range of the response is about13 of the vertical dimension of the panel.
There are several issues associated with data transformations. In the life ex- pectancy example the natural logarithm ln was helpful in straightening out the plots.
In other examples other transformations may be helpful. We will take a first look at a family of power transformations. We recommend Emerson and Stoto (1983) for a more complete discussion. We identify some of the issues here and then focus on the use of graphics to help determine which transformation in the family of power transformation would be most helpful in any given situation.
• Stabilize variance. This chapter and also Chapters6and14.
• Remove curvature. This chapter.
• Remove asymmetry. This chapter.
• Respond to systematic residuals. Chapters8and11.
102 4 Graphs log(Televisions, Physicians), and Life Expectancy
life.exp
70 75 80
70 75 80
55 60 65
55 60 65
ppl.per.tv
3 4 5
6 3 4 5 6
0 1 2 3
0 1 2 3
ppl.per.phys
8 9
10 8 9 10
6 7 8
6 7 8
Fig. 4.15 log(televisions), log(physicians), and life expectancy.
The family of power transformationsTp(x), often called theBox–Cox transfor- mationsBox and Cox (1964), are given by
Tp(x)=
⎧⎪⎪⎪⎨
⎪⎪⎪⎩
xp (p>0) ln(x) (p=0)
−xp (p<0)
(4.1)
Notice that the family includes both positive and negative powers, with the logarithm taking the place of the 0 power. The negative powers have a negative sign to maintain the same direction of the monotonicity; ifx1 <x2, thenTp(x1) <Tp(x2) for all p.
When the data are nonnegative but contain zero values, logarithms and negative powers are not defined. In this case we often add a “start” value, frequently12, to the data values before taking the log or power transformation.
When we wish to study the mathematical properties of these transformations, we use the related family of scaled power transformationsT∗p(x) given by
Tp∗(x)=
& xp−1
p (p0)
ln(x) (p=0) (4.2)
The scaling inTp∗(x) gives the same valueTp∗(1)=0 and derivative dxdT∗p(1)=1 for allp.
There is also a third family of power transformationsWp(x) given by Wp(x)=
&
xp (p0) Do not use this form,
ln(x) (p=0) the reciprocal is not negated. (4.3) that is occasionally (and incorrectly) used. This family does not negate the recipro- cals; hence, as we see in Figure4.16b, it is very difficult to read.
a. Simple Powers with Negative Reciprocals (monotonic, wrong order)
x
Tp
−2 0 2 4 6
0.0 0.5 1.0 1.5 2.0 2.5
−1−0.5 00.5 1 2
p
b. Simple Powers with Positive Reciprocals (not monotonic, wrong order)
x
Wp
−2 0 2 4 6
0.0 0.5 1.0 1.5 2.0 2.5
−1−0.5 00.5 1 2
p
c. Scaled Powers (monotonic, right order)
x
Tp∗
−2 0 2 4 6
0.0 0.5 1.0 1.5 2.0 2.5
−1−0.5 00.5 1 2
p
Tp(x) = xp
sign(p) Wp(x) =xp Tp(x) =xp− 1
p
*
Fig. 4.16 Power Transformations. The smooth transitions between the scaled curves in Fig- ure4.16c is the justification for using the family of power transformationsT∗p(x) in Equation (4.2).
This is the only one of three panels in which both (a) the monotonicity of the individual powers is visible and (b) the simple relation between the curves and the sequence of powers in the lad- der of powersp=−1,−12,0,12,1,2 is retained over the entirexdomain. Figure4.16a keeps the monotonicity but loses the sequencing. Figure4.16b, which doesn’t negate the reciprocals, is very hard to read because two of the curves are monotone decreasing and four are monotone increasing.
Figure4.16is based on Figures 4-2 and 4-3 of Emerson and Stoto (1983).
Figure4.16shows the plots of all three families: the two parameterizations of the Box–Cox power transformationsTp(x) andT∗p(x), and the third, poorly parameter- ized power familyWp(x). There are several things to note in these graphs.
1. Figure4.16a, the plots ofTp(x), correctly negates the reciprocals, thereby main- taining a positive slope for all curves and permitting the perception that these are all monotone transformations.
2. In Figure4.16b, the plots ofWp(x), we see that the plots of the two reciprocal transformations have negative slope and that all the others have positive slope.
This reversal interferes with the perception of the monotonicity of the transfor- mations.
3. Figure 4.16c, the plots of Tp∗(x), is used to study the mathematical and geo- metric properties of the family of transformations. The individual formulas in Equations (4.1) and (4.2) are linear functions of each other; hence the properties and appearance of the individual lines in the graphs based on them are equiva-
104 4 Graphs lent. Equation (4.1) is simpler for hand arithmetic. Equation (4.2) makes evident that the powers (including 0 and negative) are simply and systematically related.
Taking the negative of the reciprocal explains how the negative powers fits in.
Showing how the 0 power or logarithm fits in is trickier; we use l’Hˆopital’s rule:
limp→0
xp−1 p =lim
p→0 d
d p(xp−1)
d
d pp =lim
p→0xplnx=lnx
Theladder of powersis the sequential set of power transformations with p =
−1,−12,0,12,1,2.