Basic Concepts of Sampling
2.3 Bias, Precision and Accuracy
error or bias. This systematic error is distinct from random error because it does not balance out on average. Bias is defined as the size of the difference between the expectation of a sample estimate and the population parameter being estimated. If mis an estimate of the population mean µ, then bias is defined as E(m) µ, where E( ) denotes expectation. Bias is usually impossible to measure in practice, because we rarely know what µis. Certain types of estimators such as ratio estimators are inherently biased because of their mathematical formulation, but in sampling for pest management decision-making, a more important potential source of bias is the way in which data are collected or analysed.
Precisionrefers to how close to its own expectation we can expect one single estimate to be. If m is an estimate of the population mean µ, then precision is quantified by E(mE(m))2. With ‘E( )’ occurring twice, this may look daunting, but it can be put into words:
• (m E(m))2 is the squared difference between an estimate and its long-term average
• E(mE(m))2is the long-term average of these squared differences
E(m E(m))2 is actually more a measure of imprecision than precision, because it increases as estimates get more erratic. Note that if the difference, (mE(m)), were not squared, the result would be zero.
Accuracy denotes the closeness of population estimates to the true population parameter. Accuracy therefore incorporates both bias and precision. For example, an estimate may have high precision, but low accuracy because of high bias. The relationships among bias, precision and accuracy can be visualized by considering again the ‘random’ numbers of Example 1 and comparing sampling from these numbers to sampling from a set of truly random numbers. We simulated random sampling from the data consisting of the numbers 1–100 (Fig. 2.1a and b) and from the non-random TPE data set (Table 2.1; Fig. 2.1c and d). Both data sets were sam- pled using a small sample size (Fig. 2.1a and c) as well as a large sample size (Fig.
2.1b and d). The figure shows the resulting four distributions of sample means.
As expected, the frequencies for trials (a) and (b) are centred on 50.5, illus- trating zero bias, and the frequencies for (c) and (d) are centred to the left of 50.5, illustrating negative bias. The spread of frequencies for trials (a) and (c) is wider than in (b) and (d), illustrating less precision for (a) and (c) than for (b) and (d) due, as we shall see below, to the smaller sample size. Obviously, the sampling strat- egy for Fig. 2.1b gives the most accurate results, combining zero bias and compara- tively high precision.
EXAMPLE 3: SELECTION BIAS WHEN SAMPLING STONES. Bias can be studied in more practical instances, as was done by Yates (1960). In one study, he laid out on a table a collection of about 1200 stones (flints) of various sizes with an average weight of 54.3 g. He asked 12 observers to choose three samples of 20 stones each, which would represent as closely as possible the size distribution of the whole collection of stones. The mean weight over all 36 samples proved to be biased upwards: the aver- age weight of all 36 samples was 66.5 g. Ten of the observers chose stones whose average was greater than 54.3 g, and of their 30 samples only two had sample
means less than 54.3 g. All three of the samples of one of the two other observers had means less than 54.3 g. These results suggest that the non-random method of selecting stones by observers resulted in unrepresentative samples, leading to biased estimates of the average weight.
EXAMPLE4: SELECTION BIAS WHEN COLLECTING‘ADDITIONAL’ SAMPLE UNITS. In a second study, Yates describes how a carefully designed and tested sampling protocol for estimating wheat growth was, on two occasions, ‘adjusted’ at sampling time by an observer because the situation that he faced had not been envisaged in the proto- col. Following the protocol as far as he could, he realized that he would collect only 192 measurements of plant height, rather than the required 256, so he added 64 of his own ‘randomly chosen’ plants, fortunately keeping the values separate. On the
Basic Concepts of Sampling for Pest Management 21
Fig. 2.1. Frequency distributions of 500 sample means based on random and non- random sampling of the numbers 1–100. Samples consist of: (a) five randomly chosen numbers, low precision, no bias; (b) 20 randomly chosen numbers, high precision, no bias; (c) five numbers chosen randomly from Table 2.1, low precision, bias; (d) 20 numbers chosen randomly from Table 2.1, high precision, bias.
first date, the additional observations tended to be larger than the standard ones, while on the latter date they tended to be smaller. Yates suggests why these differ- ences might have arisen: on the first occasion, the plants were only half grown, and the observer would have been more inclined to sample larger plants; on the second occasion, the plants had come into ear, and the observer may have over-compen- sated by looking for plants closer to the average.
These examples exemplify what is called selection bias. Yates (1960) lists four possible causes of selection bias:
1. Deliberate (but unsuccessful) selection of a representative sample.
2. Selection of sample units on the basis of a characteristic which is correlated with the properties to be investigated by sampling.
3. Omission of sample units that are (more) difficult or tedious to collect or inspect than others.
4. Substitution of omitted or rejected sample units by more readily observable units.
Selection bias is an ever-present danger in pest sampling work when sample units are not selected in a truly random manner.
A second source of bias is called enumeration (counting) bias. Enumeration bias occurs when counts on sample units are systematically less than or greater than the actual number on the sample unit. This is obviously of concern when organisms are very small, but may also be an issue when organisms are mobile or are difficult to recognize. Not only can a procedure which has bias be bad in itself, but different people may act in such a way that they have their own personal amounts of bias.
Different users of a sampling plan might then arrive at different conclusions, even though they may be sampling from the same location and at the same time.
To minimize selection bias, the choice of sample units should be codified so that the opportunity for bias to creep into the sampling process is minimized. If cost and time were of no concern, each possible sample unit could be numbered and sample units drawn by generating random numbers. Of course, this is impossible in prac- tice, so a compromise must be made between selecting sample units entirely at random and making the sample process reasonable. Sampling procedures should be carefully codified to prevent any preferential selection of sample units which in some way or other are related to their pest status. We illustrate two common situa- tions in which care is needed:
1. Pests or pest injury is readily apparent on potential sample units. Leaf-rolling caterpil- lars are often regarded as pests in apple orchards. These insects create feeding sites by wrapping a leaf or leaves with silken threads. These feeding sites are readily visi- ble, even when the caterpillars are very small, so there is great danger in using a sup- posedly random collection of branch terminals or of clusters of leaves as a sample set. Observers might preferentially select or avoid leaves with feeding damage. To