Anestimator is a random variable that we can use to infer the statistical properties of the parent distribution. For example, the mean of the sam- ple is an estimator of the mean of the population. Since estimators are random variables, they have their own distributions. The estimator com- puted from a sample is a realization of a random variable. An efficient estimator is one whose distribution is highly concentrated around the true value of the statistical parameter we are trying to measure. An unbi- ased estimator is one whose expectation is the statistical quantity we are estimating.
Estimation of the Mean
Assume n independent and identically distributed (IID) random variables Each Xi is a normalized (discounted) evaluation of the payoff function as a result of the ith Monte Carlo cycle. Consider the sample mean:
(5.3) The mean of the sample is an estimator of the mean of the population.
Is it an unbiased estimator of the population mean? To answer this ques- tion, we simply take the expectation of :
(5.4)
Since the expectation of each Xi is the same as the expectation of the population,
(5.5) X1, ,… Xn
{ }.
X 1
n--- Xi i=1 i n=
∑
=
X
E[ ]X 1
n---E Xi i=1 i n=
∑
=
1
n--- E[ ]Xi i=1 i n=
∑
=
1
n--- E[ ]Xi i=1 i n=
∑
= 1n---nE[ ]XThis gives
(5.6) which shows that the expectation of the sample is an unbiased estimator of the expectation of the population.
How good is this estimator? To answer this, we look at the variance of
(5.7)
Here we made use of the fact that the Xi are independent random vari- ables from a parent distribution with variance . This gives us:
(5.8) This tells us that the standard error of the mean estimator is inversely proportional to the square root of the number of samples. Notice that the estimator of the mean is the sum of independent random variables. The cen- tral limit theorem tells us that the mean estimator is normally distributed.
If we assume that the computational work involved in estimating the mean is linearly proportional to the number of cycles, then
(5.9) This tells us that if the computational time needed to get a sample Xi is independent of the total number of samples, then in order to double the accuracy we must quadruple the computational work.
However, it is perfectly possible to have a situation where in order to increase the accuracy of the estimation the computational work needed to
E[ ]X = E[ ]X
X:
X2 = var( )X var 1
n--- Xi i=1 i n=
∑
=
1 n2
--- var( )Xi
i=1 i n=
∑
=
1 n2
--- X2
i=1 i n=
∑
= 1 n2 ---nX2
=
X2
X2 X2 ---n
=
Computational work n X2 Error
( )2
---
∝ ∝
compute Xiincreases. In such a case this scaling law breaks down and the relationship between computational work and accuracy may become far worse. To visualize how this link between the number of Monte Carlo cycles and the effort within each cycle may arise, consider a case where the Xi are computed through some numerical approximation. Clearly, if we are con- ducting a relatively crude Monte Carlo with a small number of samples, it may not make sense to excessively refine the computation of each Xi. As we increase the number of cycles, and as a result we expect to get a better estima- tion of the mean, then it would make sense to refine the computation of each Xi. This means that in such a case, the more Monte Carlo cycles we run to estimate the mean, the more work we do in each cycle. This happens when the computation of the payoff function involves the numerical solution of stochastic differential equations. This link between the computational work of each cycle and the total number of cycles may lead to a computational bar- rier, where the amount of work needed to further increase the accuracy of the estimation becomes unmanageable. We will discuss this in greater detail later.
At this point we can make two significant observations.
■ In numerical analysis there is an informal concept known as the curse of dimensionality. This refers to the fact that the computational load (CPU time, memory requirements, etc.) may increase exponentially with the number of dimensions in the problem. The computational work needed to estimate the expectation through Monte Carlo does not depend explic- itly on the dimensionality of the problem. This means that there is no curse of dimensionality in Monte Carlo computations when we are only interested in a simple expectation. This is the case with European options.
Unlike other methodologies, such as finite differences, where the compu- tational burden increases with the number of dimensions, there is no such limitation with Monte Carlo. As we will see later, the situation is less rosy when we deal with early exercise.
■ The standard error of the mean estimation depends on the standard devia- tion of the population. Since we don’t know the variance of the population, we must estimate it. However, the estimator of the variance of the popula- tion is a random variable, and it has a variance. This implies that there will be an error in our assessment of the standard error of the mean estimator.
Estimation of the Variance
An obvious candidate for estimating the variance of the population is the variance of the sample:
(5.10) X2 1n--- (Xi–X)2
i=1 i n=
∑
=
We now determine whether this estimator is unbiased:
(5.11)
The expression
(5.12) says that the variance of the sample is not an unbiased estimator of the variance of the population. This also tells us that the quantity
(5.13) EX2 E 1n--- (Xi–X)2
i=1 i n=
∑
=
E 1n--- Xi2 21
n---X Xi 1 n--- X2
i=1 i n=
∑
+ i=1 i n=
∑
– i=1 i n=
∑
=
E 1
n--- Xi2–2XX 1 n---nX2 + i=1
i n=
∑
=
E 1n--- Xi2–X2 i=1
i n=
∑
=
1
n--- E[Xi2]–E[X2] i=1
i n=
∑
= 1
n---nE[Xi2]–E[X2]
=
E[Xi2]–E[X2]
=
X2 +E2[ ]X
( )–(X2+E2[ ]X )
= X2
E2[ ]X
( + ) X2
---n +E2[ ]X
–
= n–1 ---n X2
=
X2 = n---–n1X2
sX2 n n–1 ---X2
=
is an unbiased estimator of the variance of the population because the factor cancels:
(5.14)
Since the estimator s2 is a random variable, it has a variance (this is the variance of the variance estimator).
After some algebra, we can show that the variance of s2 is given by (5.15) If the parent distribution is Gaussian, then the variance of the variance estimator is
(5.16) For n large, this is approximately
(5.17) Let’s now compare the standard deviation of the mean estimator and the standard deviation of the variance estimator:
where we should keep in mind that the second expression is only valid for Gaussian parents.
n–1 ---n
E[ ]sX2 n n–1 ---X2
=
E n
n–1 ---n–1
---n X2
= n n–1 ---n–1
---En [X2]
= n n–1 ---n–1
---n X2
= X2
=
s2
2 1
n--- E[X4] n–3 n–1 --- – E2[X2]
=
s2
2 2
n–1 ---(X2)2
=
s2
2 2
n---(X2)2
=
Stand. dev. of mean est. X ---n
= Stand. dev. of var. est. 2X2
---n
=
How about estimating the standard deviation, as opposed to the vari- ance? We can use s as an estimator of the standard deviation. We can derive an expression for the standard deviation of the standard deviation estimator. For the case of a Gaussian parent distribution, it can be shown that the standard error in the estimation of the standard deviation is (Lupton, 1993)
(5.18) Assessment of this error may become important in value-at-risk com- putations.