ANALYSIS OF VARIANCE AND QUADRATIC FORMS
4.1 Introduction to Quadratic Forms
Consider first a sum of squares with which you are familiar from your Quadratic Form for One Contrast earlier statistical methods courses, the sum of squares attributable to a
linear contrast. Suppose you are interested in the linear contrast
C1∗ = Y1+Y2−2Y3. (4.1) The sum of squares due to this contrast is
SS(C1∗) = (C1∗)2
6 . (4.2)
The divisor of 6 is the sum of squares of the coefficients of the contrast. This divisor has been chosen to make the coefficient ofσ2in the expectation of the sum of squares equal to 1. If we reexpressC1∗so that the coefficients on theYi include 1/√
6, the sum of squares due to the contrast is the square of the contrast. Thus,C1=C1∗/√
6 can be written in matrix notation as C1 = aY =√1
6Y1+√1
6Y2− 2
√6Y3 (4.3)
by defininga= ( 1/√ 6 1/√
6 −2/√
6 ) andY = (Y1 Y2 Y3). The sum of squares forC1is then
SS(C1) = C12= (aY)(aY)
= Y(aa)Y
= YAY. (4.4)
Thus, SS(C1) has been written as a quadratic forminY whereA, the Defining Matrix defining matrix, is the 3×3 matrixA=aa. The multiplicationaa for
this contrast gives
A = aa=
√1 6
√1 6
−√26
$√1
6 √1
6 −√26%
4.1 Introduction to Quadratic Forms 103
=
16 1 6 −26
16 1 6 −26
−26 −26 46
. (4.5)
Completing the multiplication of the quadratic form gives
YAY = $
Y1 Y2 Y3 %
16 1 6 −26
16 1 6 −26
−26 −26 46
Y1
Y2
Y3
= 1
6[Y1(Y1+Y2−2Y3) +Y2(Y1+Y2−2Y3) +Y3(−2Y1−2Y2+ 4Y3)]
= 1
6Y12+1 6Y22+4
6Y32+2
6Y1Y2−4
6Y1Y3−4
6Y2Y3. (4.6) This result is verified by expanding the square ofC1∗, equation 4.1, in terms ofYiand dividing by 6.
Comparison of the elements ofA, equation 4.5, with the expansion, equa- tion 4.6, shows that the diagonal elements of the defining matrix are the co- efficients on the squared terms and thesums of the symmetric off-diagonal elements are the coefficients on the product terms. The defining matrix for a quadratic form is always written in this symmetric form.
Consider a second linear contrast on Y that is orthogonal to C1. Let C2= (Y1−Y2)/√
2 =dY whered= ( 1/√
2 −1/√
2 0 ). The sum of squares for this contrast is
SS(C2) = YDY, (4.7)
where the defining matrix is
D = dd=
12 −12 0
−12 12 0
0 0 0
. (4.8)
Each of these sums of squares has 1 degree of freedom since a single linear Degrees of Freedom contrast is involved in each case. Thedegrees of freedomfor a quadratic
form are equal to the rank of the defining matrix which, in turn, is equal to the trace of the defining matrix if the defining matrix isidempotent.
(The defining matrix for a quadratic form does not have to be idempotent.
104 4. ANALYSIS OF VARIANCEAND QUADRATIC FORMS
However, the quadratic forms with which we are concerned have idempotent defining matrices.) The defining matricesA and D in the two examples are idempotent. It is left to the reader to verify thatAA=AandDD= D (see Exercise 2.25).A and D would not have been idempotent if, for example, the 1/√
6 and 1/√
2 had not been incorporated into the coefficient vectors. Notice that tr(A) = tr(D) = 1, the degrees of freedom for each contrast.
The quadratic forms defined byA andD treated each linear function Quadratic Form—Joint Functions separately. That is, each quadratic form was a sum of squares with 1 degree
of freedom. The two linear functions can be considered jointly by defining the coefficient matrixK to be a 2×3 matrix containing the coefficients for both contrasts:
KY =
√1 6 √1
6 −√26
√1
2 −√12 0
Y1
Y2
Y3
. (4.9)
The defining matrix for quadratic formYKKY is
F = KK=
23 −13 −13
−13 23 −13
−13 −13 23
. (4.10)
In this example, the defining matrixFis idempotent and its trace indicates that there are 2 degrees of freedom for this sum of squares. (The quadratic form defined in this way is idempotent only because the two original con- trasts were orthogonal to each other,ad= 0. The general method of defin- ing quadratic forms, sums of squares, for specific hypotheses is discussed in Section 4.5.1.)
Two quadratic forms (of the same vectorY) are orthogonal if the product Orthogonal Quadratic Forms of the defining matrices is0. Orthogonality of the two quadratic forms in
the example is verified by the multiplication ofAandD:
DA =
12 −12 0
−12 12 0
0 0 0
16 1 6 −26
16 1 6 −26
−26 −62 46
=
0 0 0 0 0 0 0 0 0
, (4.11)
which equals AD since A, D, and DA are all symmetric. Note that DA = ddaa and will be zero if da = 0. Thus, the quadratic forms associated with two linear functions will be orthogonal if the two vectors of
4.1 Introduction to Quadratic Forms 105 coefficients are orthogonal—that is, if the sum of products of the coefficient vectors dais zero (see Exercise 2.26). When the two linear functions are orthogonal, the sum of sums of squares (and degrees of freedom) of the two contrasts considered individually will equal the sum of squares (and degrees of freedom) of the two contrasts considered jointly. For this addi- tivity to hold when more than two linear functions are considered, all must be pairwise orthogonal. Orthogonality of quadratic forms implies that the two pieces of information contained in the individual sums of squares are independent.
The quadratic forms of primary interest in this text are the sums of squares associated with analyses of variance, regression analyses, and tests of hypotheses. All have idempotent defining matrices.
The following facts about quadratic forms are important [see Searle (1971) for more complete discussions on quadratic forms].
1. Any sum of squares can be written asYAY, whereAis a square symmetric nonnegative definite matrix.
2. The degrees of freedom associated with any quadratic form equal the rank of the defining matrix, which equals its trace when the matrix is idempotent.
3. Two quadratic forms are orthogonal if the product of their defining matrices is the null matrix 0.
For illustration of quadratic forms, let Example 4.1
Y = ( 3.55 3.49 3.67 2.76 1.195 )
be the vector of mean disease scores for a fungus disease on alfalfa. The five treatments were five equally spaced day/night temperature regimes under which the plants were growing at the time of inoculation with the fungus.
The total uncorrected sum of squares is
YY = 3.552+ 3.492+· · ·+ 1.1952= 47.2971.
The defining matrix for this quadratic form is the identity matrix of order 5. SinceI is an idempotent matrix and tr(I)= 5, this sum of squares has 5 degrees of freedom.
The linear function ofY that gives the total disease score over all treat- ments is given by
Yi=a1Y, where
a1= ( 1 1 1 1 1 ).
The sum of squares due to correction for the mean, the correction factor, is (
Yi)2/5 = 43.0124. This is written as a quadratic form as Y(J/5)Y,
106 4. ANALYSIS OF VARIANCEAND QUADRATIC FORMS
whereJ =a1a1 is a 5×5 matrix of ones. The defining matrixJ/5 is an idempotent matrix with tr(J/5) = 1. Therefore, the sum of squares due to correction for the mean has 1 degree of freedom.
Based on orthogonal polynomial coefficients for five equally spaced treat- ments, the linear contrast for temperature effects is given by
C2∗ = a∗2Y = (−2 −1 0 1 2 )Y. Incorporating the divisor
a∗2a∗2 = √
10 into the vector of coefficients gives
a2 = $
−√210 −√110 0 √110 √210% .
The sum of squares due to the linear regression on temperature is given by the quadratic form
YA2Y = 2.9594, where
A2 = a2a2=
.4 .2 0 −.2 −.4 .2 .1 0 −.1 −.2
0 0 0 0 0
−.2 −.1 0 .1 .2
−.4 −.2 0 .2 .4
.
The defining matrixA2is idempotent with tr(A2) = 1 and, therefore, the sum of squares has 1 degree of freedom.
The orthogonal polynomial coefficients for the quadratic term, including division by the square root of the sum of squares of the coefficients, is
a3= √1
14( 2 −1 −2 −1 2 ).
The sum of squares due to quadratic regression is given by the quadratic
form YA3Y = 1.2007,
where
A3 = a3a3=
.2857 −.1429 −.2857 −.1429 .2857
−.1429 .0714 .1429 .0714 −.1429
−.2857 .1429 .2857 .1429 −.2857
−.1429 .0714 .1429 .0714 −.1429 .2857 −.1429 −.2857 −.1429 .2857
.
The defining matrixA3is idempotent and tr(A3) = 1 so that this sum of squares also has 1 degree of freedom.
It is left to the reader to verify that each of the defining matrices J/5, A2, andA3is idempotent and that they are pairwise orthogonal to each