We may test a pseudo-random number generator by takingMsamples{ξi}iM=1and computing thesample mean
µM := 1 M
M i=1
ξi, (4.1)
and thesample variance
σM2 := 1 M−1
M i=1
(ξi −µM)2. (4.2)
The sample mean (4.1) is simply the arithmetic average of the sample values. The sample variance is a similar arithmetic average corresponding to the expected value in (3.10) that defines the variance. (You might regard it as more natural to take the sample variance as(1/M)M
i=1(ξi −µM)2; however, it can be argued that scaling
1 All computational experiments in this book were produced in MATLAB, using the built-in functionsrandand randnto generateU(0,1)andN(0,1)samples, respectively. To make the experiments reproducible, we set the random number generator seed to 100; that is, we usedrand(‘state’,100)andrandn(‘state’,100).
4.3 Statistical tests 35 Table 4.2.Sample mean (4.1) and sample variance
(4.2) using M samples from aU(0,1)and an N(0,1)pseudo-random number generator
U(0,1) N(0,1)
M µM σM2 µM σM2
102 0.5229 0.0924 0.0758 1.0996
103 0.4884 0.0845 0.0192 0.9558
104 0.5009 0.0833 −0.0115 0.9859
105 0.5010 0.0840 0.0005 1.0030
byM−1 instead ofMis better. This issue is addressed in Chapter 15.) Results for M=102,103,104 and 105 appear in Table 4.2. We see that as M increases, the U(0,1)sample means and variances approach the true values 12 and 121 ≈0.0833 (recall Exercise 3.5) and theN(0,1)sample means and variances approach the true values 0 and 1.
A more enlightening approach to testing a random number generator is to divide thex-axis into subintervals, orbins, of lengthx and count how many samples lie in each subinterval. We takeM samples and letNi denote the number of sam- ples in the bin [ix, (i+1)x]. If we approximate the probability of X taking a value in the subinterval [ix, (i+1)x] by the relative frequency with which this happened among the samples, then we have
P(ix ≤ X ≤(i+1)x)≈ Ni
M. (4.3)
On the other hand, we know from (3.3) that, for a random variableX with density f(x),
P(ix ≤ X ≤(i+1)x)=
(i+1)x
ix
f(x)d x. (4.4) Letting xi denote the midpoint of the subinterval [ix, (i+1)x] we may use the Riemann sum approximation
(i+1)x
ix
f(x)d x ≈x f(xi). (4.5) (Here, we have approximated the area under a curve by the area of a suitable rect- angle – draw a picture to see this.) Using (4.3)–(4.5), we see that plotting Ni/(Mx)againstxi should give an approximation to the density function values
0 0.5 1 0
0.5 1 1.5
1000 samples
0 0.5 1
0 0.5 1 1.5
10 000 samples
0 0.5 1
0 0.5 1 1.5
100 000 samples
0 0.5 1
0 0.5 1 1.5
1 000 000 samples
Fig. 4.1. Kernel density estimate for aU(0,1)generator, with increasing number of samples. Vertical axis isNi/(Mx), forx=0.05.
f(xi). This technique, and more sophisticated extensions, fit into the area ofkernel density estimation.
Computational example We compute a simple kernel density estimate for a U(0,1) generator, using intervals of width x=0.05. Since f(x) is nonzero only for 0≤x ≤1, we takei =0,1,2, . . . ,19. In Figure 4.1 we plot Ni/(Mx)againstxi for the number of samplesM=103,104,105,106. These points are plotted as diamonds joined by straight lines for clarity. We see that as Mincreases the plot gets closer to that of aU(0,1)density. ♦ Computational example In Figure 4.2 we perform a similar experiment with anN(0,1)generator. Here, we took intervals in the region−4≤x≤4 and used bins of widthx =0.05. (Samples that were smaller than−4 were added to the first bin and samples that were larger than 4 were added to the last bin.) We used M =103,104,105,106. The correct N(0,1)density curve is superimposed in white. We see that the density estimate improves asMincreases. ♦ We now look at another technique for examining statistical aspects of data. For a given density function f(x)and a given 0< p<1 define the pth quantileof f asz(p), where
z(p)
−∞ f(x)d x = p. (4.6)
4.3 Statistical tests 37
−4 −2 0 2 4
0 0.1 0.2 0.3 0.4 0.5
1000 samples
−4 −2 0 2 4
0 0.1 0.2 0.3 0.4 0.5
10 000 samples
−4 −2 0 2 4
0 0.1 0.2 0.3 0.4 0.5
100 000 samples
−4 −2 0 2 4
0 0.1 0.2 0.3 0.4 0.5
1 000 000 samples
Fig. 4.2. Kernel density estimate for anN(0,1)generator, with increasing num- ber of samples. Vertical axis isNi/(Mx), forx=0.05.
Given a set of data pointsξ1, ξ2, . . . , ξM, aquantile–quantile plotis produced by (a) placing the data points in increasing order:ξ1,ξ2, . . . ,ξM,
(b) plottingξkagainstz(k/(M+1)).
The idea of choosing quantiles for equally spaced p=k/(M+1) is that it
‘evens out’ the probability. Figure 4.3 illustrates theM =9 case when f(x)is the N(0,1) density. The upper picture emphasizes that the z(k/(M+1)) break the x-axis into regions that give equal area under the density curve – that is, there is an equal chance of the random variable taking a value in each region. The lower picture in Figure 4.3 plots the function f(x)and shows thatz(k/(M+1))are the points on thex-axis that correspond to equal increments along they-axis. The idea is that, for large M, if the quantile–quantile plot produces points that lie approxi- mately on a straight line of unit slope, then we may conclude that the data points
‘look as though’ they were drawn from a distribution corresponding to f(x). To justify this, if we divide thex-axis into M bins where x is in thekth bin if it is closest toz(k/(M+1)), then, having evened out the probability, we would expect roughly oneξi value in each bin. So the smallest data point,ξ1, should be close toz(1/(M+1)), the second smallest,ξ2, should be close to z(2/(M+1)), and so on.
Computational example Figure 4.4 tests the quantile–quantile idea. Here we took M =100 samples from N(0,1) andU(0,1) random number generators.
−5 −4 −3 −2 −1 0 1 2 3 4 5 0
0.1 0.2 0.3 0.4 0.5
f (x)
−5 −4 −3 −2 −1 0 1 2 3 4 5
0 0.2 0.4 0.6 0.8
1 N(x)
Fig. 4.3. Asterisks on thex-axis mark the quantilesz(k/(M +1))in (4.6) for an N(0,1)distribution using M =9. Upper picture: the quantiles break thex-axis into regions where f(x)has equal area. Lower picture: equivalently, the quantiles break thex-axis into regions whereN(x)has equal increments.
Each data set was plotted against theN(0,1)andU(0,1)quantiles. A reference line of unit slope is added to each plot. As expected, the data set matches well with the ‘correct’ quantiles and very poorly with the ‘incorrect’ quantiles. ♦ Computational example In Figures 4.5 and 4.6 we use the techniques intro- duced above to show the remarkable power of the Central Limit Theorem. Here, we generated sets ofU(0,1) samples{ξi}ni=1, withn=103. These were com- bined to give samples of the form
n
i=1ξi−nµ σ√
n , (4.7)
where µ= 12 and σ2= 121. We repeated this M =104 times. These M data points were then used to obtain a kernel density estimate. In Figure 4.5 we used bins of widthx =0.5 over [−4,4] and plotted Ni/(Mx)against xi, as described for Figure 4.1. Here we have used ahistogram, orbar graph, so each rectangle is centred at anxi and has height Ni/(Mx). TheN(0,1)den- sity curve is superimposed as a dashed line. Figure 4.6 gives the corresponding quantile–quantile plot. The figures confirm that even though eachξi is nothing like normal, the scaled sum(n
i=1ξi −nµ)/(σ√
n)is very close toN(0,1). ♦
4.3 Statistical tests 39
−5 0 5
−5 0 5
N(0,1) samples and N(0,1) quantiles
−5 0 5
−5 0 5
N(0,1) samples and U(0,1) quantiles
−5 0 5
−5 0 5
U(0,1) samples and N(0,1) quantiles
−1 0 1 2
−1
−0.5 0 0.5 1 1.5 2
U(0,1) samples and U(0,1) quantiles
Fig. 4.4. Quantile–quantile plots using M =100 samples. Ordered samplesξ1, ξ2, . . . ,ξM on thex-axis against quantilesz(k/(M+1))on they-axis. Pictures show the four possible combinations arising fromN(0,1)orU(0,1)random num- ber samples againstN(0,1)orU(0,1)quantiles.
−5 −4 −3 −2 −1 0 1 2 3 4 5
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35
0.4 N(0,1) density
Sample data
Fig. 4.5. Kernel density estimate for samples of the form (4.7), with N(0,1) density superimposed.
−6 −4 −2 0 2 4 6
−5
−4
−3
−2
−1 0 1 2 3 4 5
Fig. 4.6. Quantile–quantile plot for samples of the form (4.7) against N(0,1) quantiles.