3.6 Case 2: Unknown PDF Type (Parametric)
3.6.1 Bayesian Model Averaging Approach
Consider a particular random variable X for which data is available. Let D denote the collection of all data, which comprises of m point data xi (i = 1 to m) and n intervals [ai, bi] (i = 1 ton). Now, the aim is to quantify the model form uncertainty in the probability distribution of X. The method of Bayesian model averaging is applicable when multiple competing model forms are compared. The overall approach is to express the PDF fX(x) as a weighted sum of the competing model forms.
Without loss of generality, the method is discussed here for two competing model forms; it can be easily extended to any number of competing model forms. Let fX1(x|φ) and fX2(x|θ) denote the PDFs of the competing model forms; in each PDF, φ and θ are the unknown distribution parameters.
Using Bayesian model averaging, the PDF of X can be expressed as the sum of the above two PDFs as:
fX(x|w,φ,θ) =wfX1(x|φ) + (1−w)fX2(x|θ) (3.14)
Since there are only two competing distributions here, their weights are chosen as w and 1−w in Eq. 3.14. If there are n competing distributions, then there are n − 1 weights to be estimated, and the nth weight is estimated by imposing the condition that the sum of all the weights is equal to unity, since the area under the PDF fX(x|w,φ,θ) must be equal to unity.
A likelihood-based estimation procedure similar to that in Section 3.4.1 is used here. The difference is that the combined likelihood of the weights and the distribution parameters, i.e. L(φ,θ, w), is constructed as:
L(φ,θ, w)∝[
m
Y
i=1
fX(x=xi|w,φ,θ)][
n
Y
j=1
Z bj
aj
fX(x|w,φ,θ)dx] (3.15)
This likelihood function can be maximized to obtain the maximum likelihood estimates of φ, θ, and w. Further the uncertainty in the estimates can also be quantified using Bayesian inference, as in Eq. 3.7. A uniform prior bounded on [0, 1]
is chosen forw, and non-informative priors are chosen for the distribution parameters φ and θ. The prior distributions are multiplied by the likelihood function and then normalized to calculate the posterior distributions of φ, θ, and w.
Two illustrations are presented below. The first example considers a large amount of data and two significantly different candidate model forms. The second example considers a large amount of data and two candidate model forms that are not signif- icantly different from one another.
3.6.1.1 Illustration 1
Consider a case of 100 samples generated from an underlying normal distribution with mean and standard deviation equal to 100 units and 10 units respectively. Since the amount of data is large, it is easy to identify that the underlying distribution
is, in fact, normal. However, this example is used only to demonstrate the Bayesian averaging method.
For the sake of illustration, assume that the two competing model forms are normal (N(µ, σ)) and uniform (U(a, b)). With reference to Eq. 3.15, φ={µ, σ} and θ ={a, b}. Letwdenote the weight for the normal distribution, and 1−wis the weight for the uniform distribution. The joint likelihood is evaluated for five quantities (w,µ, σ,a, andb), and the posterior distribution is estimated for each quantity using 10,000 samples from slice sampling [39]. The correctness of these posterior distributions can be easily verified since the samples were actually generated from a normal distribution N(100,10).
First, the PDF of the weight w is shown in Fig. 3.13. The estimated statis- tics/PDFs of distribution parameters are shown in Table 3.4 and Fig. 3.14.
0.850 0.9 0.95 1 1.05
10 20 30 40 50 60
Weightw
Figure 3.13: PDF of Weight w
From Table 3.4, it can be clearly seen that the method isolates the data to come from a normal distribution. There is high confidence in this conclusion because the mean of w is high (0.98) and the standard deviation of w is small (0.015). Also,
96 98 100 102 104 106 0
0.1 0.2 0.3 0.4
µ
(a) PDF of µ
6 8 10 12 14
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
σ
(b) PDF of σ
−200 0 20 40 60
0.01 0.02 0.03 0.04 0.05 0.06
a
(c) PDF ofa
1000 150 200 250 300
0.005 0.01 0.015 0.02 0.025
b
(d) PDF ofb
Figure 3.14: PDFs of Distribution Parameters
the distribution parameters of the normal distribution are in good agreement with the actual values using which the data was simulated, and the uncertainty in the estimates of these distribution parameters is small. Since the weighting factor for the uniform distribution is small, the distribution parameter estimates for the uniform distribution have high uncertainty.
The uncertainty in the estimate of the weight wis low because of two reasons: (1) there is sufficient data to conclusively suggest a normal distribution, and (2) the two competing model forms, i.e. normal and uniform are significantly different from each other.
It is obvious that, if there is only sparse data, then the uncertainty in the estimate ofwwill be high. However, even if there is sufficient data, it is hard to uniquely isolate
Table 3.4: Normal vs. Uniform: Results of Bayesian Model Averaging Quantity Mean Standard Deviation 95% Bounds
w 0.986 0.015 [0.949, 0.999]
µ 100.887 0.969 [99.078, 102.811]
σ 9.998 0.704 [8.752, 11.534]
a 18.193 9.584 [2.997, 43.800]
b 203.767 27.6065 [157.278, 239.324]
a particular model form if the competing model forms are not significantly different from one another, as shown next.
3.6.1.2 Illustration 2
Consider 100 samples generated from an exponential distribution with parameter µ= 1. The PDF for this distribution is given by
fX(x|µ) = 1
µexp(−x
µ) (3.16)
For the sake of illustration, assume that the two competing model forms are exponential and Rayleigh. While the former has one parameter (µ) as indicated in Eq. 3.16, the latter also has only one parameter (b), and the PDF is given by:
fX(x|b) = x
b2exp(−x2
2b2) (3.17)
Note that the exponential and Rayleigh distributions are not as significantly dif- ferent from each other as the uniform and normal distributions. This is because both exponential and Rayleigh distributions can be viewed as special cases of the two-parameter Weibull distribution with shape parameters equal to one and two re- spectively. Since the Weibull distribution is commonly used to study time-dependent reliability, this example is of practical significance.
0 0.5 1 1.5 0
0.5 1 1.5 2 2.5
Weightw
Figure 3.15: PDF of Weight w
Similar to the previous example, the joint likelihoodL(w, µ, b) is used to evaluate the posterior distributions of w, µ and b respectively. First, the PDF of the weight w is shown in Fig. 3.15 where w is the weight for the exponential distribution, and 1−w is the weight for the Rayleigh distribution.
The PDFs of the distribution parameter for each model-form (µ for exponential distribution andbfor Rayleigh distribution) are shown in Fig. 3.16, and the numerical estimates are shown in Table 3.5.
Table 3.5: Exponential vs Rayleigh: Results of Bayesian Model Averaging Quantity Mean Standard Deviation 95% Bounds
w 0.746 0.158 [0.424, 0.988]
µ 0.840 0.239 [0.382, 1.255]
b 2.060 1.181 [0.561,7.793]
The mean ofwis about 0.75, which suggests a higher likelihood for the exponential distribution. However, there is significant uncertainty in w, leading to inconclusive distinction between the exponential and Rayleigh distributions. Also, the estimates
0 0.5 1 1.5 2 0
0.5 1 1.5
µ
(a) PDF ofµ
−10 0 1 2 3 4
0.5 1 1.5 2
b
(b) PDF ofb
Figure 3.16: PDFs of Distribution Parameters
of the distribution parameters suggest a higher likelihood for the exponential dis- tribution, because µ in the exponential distribution has a much smaller uncertainty compared to b in Rayleigh distribution. That is, a “narrow” estimate of µ is suffi- cient to “explain” the available data whereas a “wide” estimate of b is needed for the same. This is intuitive because the data actually originates from an exponential distribution. Also, the maximum likelihood estimate of µis one, which is exactly the same as the originally assumed value for µused to generate the data.
3.6.1.3 Quantifying Individual Contributions
Earlier, Section 3.5 developed a computational method to assess the individual contributions of variability and distribution parameter uncertainty by assuming a particular distribution type. In the present section, the distribution type is also un- certain, and this uncertainty is quantified through the PDF ofw. Hence, the method developed in Section 3.5 is now extended to quantify the individual contributions of (1) variability; (2) distribution type uncertainty; and (3) distribution parameter uncertainty.
The concept of the auxiliary variable was introduced earlier in Section 3.5.2 to facilitate the use of global sensitivity analysis for the quantification of individual
Table 3.6: Contributions of Physical Variability and Epistemic Uncertainty Illustration Effect Physical Distribution Distribution
Variability Type Parameter
Example 1 Individual 94.1% 1.0% 4.0%
Section 3.6.1.1 Overall 98.1% 1.0% 5.2%
Example 2 Individual 40.7% 12.4% 40.5%
Section 3.6.1.2 Overall 43.5% 15.4% 43.3%
contributions. This auxiliary variable is now redefined to include the distribution type uncertainty, as:
UX = Z X
−∞
fX(x|w,φ,θ)dx (3.18)
Similar to Section 3.5, there is unique one-to-one mapping between X and UX. Since Eq. 3.18 is simply the definition of CDF, by varying UX on the uniform dis- tribution U(0,1), it is possible to obtain the entire distribution of X. Now, global sensitivity analysis can be applied to calculate the individual and overall effects of physical variability (UX), distribution type uncertainty (w), and distribution param- eter uncertainty (φ and θ). For example, the results of sensitivity analysis for the illustrative examples discussed in Sections 3.6.1.1 and 3.6.1.2 are tabulated in Ta- ble 3.6.
Similar to Section 3.5, it is straightforward to quantify the contributions in a single variable as well as the contributions to the output of a response function, since the response function is a deterministic transfer function from the inputsX to the output Y. The propagation of the results of Bayesian model averaging through a response function will be discussed later in Section 3.6.3.
3.6.1.4 Summary
Conventionally, model averaging methods assign weights for competing models, and these weights are estimated in a deterministic manner. In this section, the un- certainty in the weights is also computed, thereby giving both the confidence in a particular distribution type (through the mean value of w) , and a measure of uncer- tainty in this confidence (through the standard deviation of w). One disadvantage of this approach is that it assumes spurious interactions between competing model forms while constructing the joint likelihood of weights and distribution parameters of all model forms. As a result, this approach involves multi-dimensional integration; a significant amount of computational power may be required, if there are several com- peting model forms. For example, if there were 5 competing model forms, each with two distribution parameters, then the joint likelihood needs to be constructed for 14 quantities (4 weights and 10 parameters), and a 14-dimensional integration is needed to quantify the model form uncertainty and estimate the distribution parameters.
The next section discusses the use of Bayesian hypothesis testing to quantify model form uncertainty; this approach provides a computationally efficient alternative and also directly computes the probability that the data supports a given model form.