Major topics covered in this chapter
Exercises 35 balance with a single internal reference weight.) Carefully considered procedures
F- test for the comparison of standard deviations 47
3.9 Comparison of several means
Table 3.2 shows the results obtained in an investigation into the stability of a fluo-rescent reagent stored under different conditions. The values given are the fluores-cence signals (in arbitrary units) from dilute solutions of equal concentration. Three replicate measurements were made on each sample. The table shows that the mean
Table 3.2 Fluorescence from solutions stored under different conditions
Conditions Replicate measurements Mean
A Freshly prepared 102, 100, 101 101
B Stored for 1 hour in the dark 101, 101, 104 102
C Stored for 1 hour in subdued light 97, 95, 99 97
D Stored for 1 hour in bright light 90, 92, 94 92
Overall mean 98
54 3: Significance tests
values for the four samples are different. However, we know that because of random error, even if the true value which we are trying to measure is unchanged, the sam-ple mean may vary from one samsam-ple to the next. ANOVA tests whether the differ-ence between the sample means is too great to be explained by the random error.
Figure 3.2 shows a dot-plot comparing the results obtained in the different condi-tions. This suggests that there may be little difference between conditions A and B but that conditions C and D differ both from A and B and from each other.
The problem can be generalised to consider h samples each with n members as in Table 3.3 where xijis the jth measurement of the ith sample. The means of the samples are and the mean of all the values grouped together is . The null hypoth-esis adopted is that all the samples are drawn from a population with mean and vari-ance . On the basis of this hypothesis can be estimated in two ways, one involving the variation within the samples and the other the variation between the samples.
1 Within-sample variation
For each sample a variance can be calculated by using the formula (see Eq. (2.1.2))
Using the values in Table 3.2 we have:
Variance of sample B = (101 - 102)2 + (101 - 102)2 + (104 - 102)2
(3 - 1) = 3
Variance of sample A = (102 - 101)2 + (100 - 101)2 + (101 - 101)2
(3 - 1) = 1
a 1xi - x22>1n - 12 s20 s20
m x
x1, x2, . . . , xh
Table 3.3 Generalisation of Table 3.2
Mean Sample 1
Sample 2
Sample i
Sample h
Overall mean x xh
xh2ÁxhjÁxhn
xh1
o o
o o o
xi xi2ÁxijÁxin
xi1
o o
o o o
x2
x22Áx2jÁx2n
x21
x1
x12Áx1jÁx1n
x11
A B D
Conditions 105
100
95
90
Signal
C
Figure 3.2 Dot-plot of results in Table 3.2.
Comparison of several means 55 Similarly it can be shown that samples C and D both have variances of 4. Averag-ing these values gives a within-sample estimate of . This estimate has eight degrees of freedom: each sample estimate has two degrees of freedom and there are four samples. Note that this estimate of does not de-pend on the means of the samples: for example, if all the measurements for sam-ple A were increased by say, 4, the estimate of would be unaltered. The general formula for the within-sample estimate of is:
(3.9.1)
The summation over j and division by gives the variance of each sample;
the summation over i and division by h averages these sample variances. The ex-pression in Eq. (3.9.1) is known as a mean square (MS) since it involves a sum of squared (SS) terms divided by the number of degrees of freedom. In this case the number of degrees of freedom is 8 and the mean square is 3, so the sum of the squared terms is
2 Between-sample variation
If the samples are all drawn from a population which has a variance , then their means come from a population with variance (cf. the sampling distribution of the mean, Section 2.5). Thus, if the null hypothesis is true, the variance of the means of the samples gives an estimate of From Table 3.2:
So the between-sample estimate of is This estimate has three degrees of freedom since it is calculated from four sample means. Note that this estimate of does not depend on the variability within each sample, since it is calculated from the sample means. But if, for example, the mean of sample D was changed, then this estimate of would also be changed.
In general we have:
(3.9.2) which again is a ‘mean square’ involving a sum of squared terms divided by the number of degrees of freedom. In this case the number of degrees of freedom is 3 and the mean square is 62, so the sum of the squared terms is
Summarising our calculations so far:
If the null hypothesis is correct, then these two estimates of should not differ sig-nificantly. If it is incorrect, the between-sample estimate of will be greater than the within-sample estimate because of between-sample variation. To test whether it is significantly greater, a one-sided F-test is used (see Section 3.6):
F = 62>3 = 20.7
s20 s20 Between-sample mean square = 62 with 3 d.f.
Within-sample mean square = 3 with 8 d.f.
3 * 62 = 186.
Between-sample estimate of s20 = n a
i 1xi - x22>1h - 12 s20
s20
(62>3) * 3 = 62.
s20
= 62>3
(101 - 98)2 + (102 - 98)2 + (97 - 98)2 + (92 - 98)2 (4 - 1)
Sample mean variance =
s20>n.
s20>n
s20 3 * 8 = 24.
(n - 1) Within-sample estimate of s20 = a
i a
j 1xij - xi22>h1n - 12 s20
s20
s20
s20 = (1 + 3 + 4 + 4)>4 = 3
56 3: Significance tests
(Remember that each mean square is used so no further squaring is necessary.) The numerator has three degrees of freedom and the denominator has eight degrees of freedom, so from Table A.3 the critical value of F is 4.066 (P 0.05). Since the calculated value of F is much greater than this the null hypothesis is rejected: the sample means do differ significantly.
Such a significant difference can arise for several different reasons: for example, one mean may differ from all the others, all the means may differ from each other, the means may fall into two distinct groups, etc. A simple way of deciding the rea-son for a significant result is to arrange the means in increasing order and compare the difference between adjacent values with a quantity called the least significant difference. This is given by where s is the within-sample estimate of and is the number of degrees of freedom of this estimate. For the example above, the sample means arranged in increasing order of size are:
and the least significant difference is
Com-paring this value with the differences between the means suggests that conditions D and C give results which differ significantly from each other and from the results ob-tained in conditions A and B. However, the results obob-tained in conditions A and B do not differ significantly from each other. This confirms the indications of the dot-plot in Fig. 3.2, and suggests that it is exposure to light which affects the intensity of fluorescence.
The least significant difference method described above is not entirely rigorous: it can be shown that it leads to rather too many significant differences. However, it is a simple follow-up test when ANOVA has indicated that there is a significant differ-ence between the means. Descriptions of other more rigorous tests are given in the books in the Bibliography at the end of this chapter.