UQ due to Modeling Parameters using Surrogate Models

III. Forward Uncertainty Quantification of Spent Fuel

3.5. Uncertainty Quantification Results

3.5.1. UQ due to Modeling Parameters using Surrogate Models

This section presents the findings of the PCE, GSA, developing the surrogate model, and forward UQ due to modeling parameter uncertainties. The following is a description of the workflow:

1) As stated in Table 3.1, identify the input parameters and their corresponding uncertainty.

The surrogate model is only based on the assembly model parameters and does not take nuclear data into account.

2) Create 100, 300, and 1,000 samples by LHS with all input parameters perturbed at the same time. Three sets of input parameters are represented here.

3) Run STREAM on the three different sets of input parameters generated in step 2. With a single core on a desktop computer, each STREAM run takes roughly six minutes. The dataset required to build the surrogate model is represented by the input and output data.

4) Develop the PCE surrogate model using the input and output data from the first set (of 100 runs). To determine the coefficients using the linear regression approach for a 2^nd order PCE, a minimum of 78 model evaluations for the 11-dimensional input parameter problem

46

are needed. A minimum of 12 and 364 STREAM calculations are needed for 1^st and 3^rd PCE, respectively.

5) To prevent overfitting, utilize the results from the second set (of 300 inputs that were not used to develop the model) to validate the surrogate model.

6) To determine the model response for the third set of 1,000 samples, use the surrogate model.

7) Using the STREAM and surrogate model results, perform the needed statistics and GSA.

Figures 3.3 – 3.4 show the surrogate model validation, which compare the PDFs and model responses to those of STREAM. The STREAM results are plotted against the surrogate model predictions for the 300 and 1,000 samples in Figure 3.4. To show the linear relationship and agreement between STREAM and the surrogate model, the plot includes a linear fit. Good agreement is observed suggesting that the surrogate can accurately project the input space into the model response. Table 3.6 shows the surrogate model relative validation error for 1^st order PCE calculated by Eq. (3.35) in which 𝜇_{𝑌𝑉𝑎𝑙} is mean response of the validation set. The validation set inputs and outputs are represented by [𝜉_𝑉𝑎𝑙, 𝑦_𝑉𝑎𝑙 = 𝑌(𝜉_𝑉𝑎𝑙)]. The relative validation error of the gamma source at 23 years is less than at discharge, as shown in Table 3.6. This is due to the fact that the surrogate gamma source agreement with STREAM is better at 23 years than at discharge.

Because the surrogate model show better agreement with STREAM at discharge than at 23 years, the relative validation error for activity, decay heat, and neutron source is larger at 23 years than at discharge. The relative discrepancies between STREAM and surrogate model at discharge and 23 years support this observation. Regardless, the general STREAM/surrogate agreement is good.

The errors present in the surrogate predictions is discussed further in this section to establish its accuracy and reliability. After building the surrogate model, it can be used to predict the output of a large number of input samples at a low computing cost. The time it takes for STREAM to run 1,000 input samples with single core on a desktop computer would be around 6,000 minutes (100 hours or 4 days). The surrogate model performs this task in roughly 3 seconds, resulting in a significant reduction in the time required.

𝜖_𝑉𝑎𝑙 =^𝑁−1

𝑁 [^{∑ (𝑌(𝜉𝑉𝑎𝑙}

(𝑖))−𝑌^𝑃𝐶𝐸(𝜉_𝑉𝑎𝑙^(𝑖)))² 𝑁𝑖=1

∑^𝑁_𝑖=1(𝑌(𝜉_𝑉𝑎𝑙^(𝑖))−𝜇_{𝑌𝑉𝑎𝑙})² ] (3.35)

47 𝜇_{𝑌𝑉𝑎𝑙} = ¹

𝑁∑^𝑁_𝑖=1𝑌(𝜉_𝑉𝑎𝑙^(𝑖)) (3.36) Table 3.6. Surrogate model relative validation error (%).

Activity Decay heat Neutron source Gamma source Burnup k-inf 1^a 3.39E-02 1.92E-02 2.09E-02 2.67E-02 8.94E-05 6.67E-01

2^b 1.10E-01 1.90E-01 3.12E-01 1.64E-02 - -

a Results correspond to discharge. ^bResults correspond to 23 years of cooling.

48

Figure 3.3. Response PDF comparison at 23 years based on 1,000 runs.

49

Figure 3.4. Surrogate predictions versus STREAM results for responses at 23 years.

50

Figure 3.5 depicts the uncertainty in SNF characteristics (evaluated as the RSD) at discharge and after 23.2 years due to modeling parameter uncertainties. It should be emphasized that the nuclide number density uncertainties in Figure 3.5 are derived from the 100 STREAM results obtained at the beginning of this section and not from the surrogate models. At 23 years, the neutron source has an uncertainty of about 6%. At 23 years, the decay heat, activity, gamma source, burnup, and infinite multiplication factor all have uncertainties of roughly 2%, 1.6%, 1.4%, 0.8%, and 0.5%, respectively. The major neutron source in the first two to three decades of cooling,

244Cm (half-life 18.1 years), causes the high neutron source uncertainty. ²⁴⁴Cm is responsible for 94.1% of the total neutron source after 23 years of cooling.

Table 3.7 compares the uncertainty of STREAM against the number of samples (N).

STREAM uncertainties are derived from statistically analyzing the results. On increasing sample size, the RSD appears to fluctuate. This is induced by the convergence of stochastically sampled values, which necessitates a large number of repeated runs to converge.

Figure 3.5. SNF uncertainties caused by modeling parameters.

Table 3.7. Uncertainty of STREAM results at 23 years of cooling (%).

N Activity Decay heat Neutron source Gamma source Burnup^a k-inf^a

100 1.43 1.73 5.25 1.32 0.86 0.57

300 1.61 2.02 6.32 1.39 0.88 0.50

1000 1.55 1.91 5.80 1.38 0.80 0.50

a See footnote of Table 3.5.

51

Statistical fluctuations and errors are present in the results due to the finite size of the sample analyzed. As the sample size grows larger, the standard error of the mean and standard deviation falls until convergence is achieved. With a sample of size n, the standard error of the mean is proportional to ^𝜎

√𝑛 (where 𝜎 is the standard deviation). The standard error of the standard deviation is proportional to ^𝜎

√2𝑛. For the decay heat at discharge, burnup and k-inf of the assembly considered, the convergence of their standard deviations as a function of the sample size is presented in Figures 3.6 and 3.7 below. Three different sets of samples have been considered, which are used in the uncertainty analysis due to modeling parameters. Figures 3.6 and 3.7include the standard error of the standard deviation calculated with the previously mentioned formula. The statistical fluctuations are seen to dampen out with increasing sample size as convergence is reached. In addition, the standard error decreases with sample size. The maximum standard error is observed at low number of samples. Thus, the increase of the RSD from 100 to 300 samples and the decrease from 300 to 1,000 samples can be attributed to these statistical fluctuations. These statistical fluctuations are because of the errors associated with the RSD due to the finite size of the sample.

Figure 3.6. Convergence of decay heat standard deviation.

52

Figure 3.7. Convergence of Burnup and k-inf standard deviation. The black band is the standard error of the standard deviation.

53

Table 3.8 compares the results of the PCE-based surrogates to those of STREAM for 1,000 samples. This table shows the mean and RSD of the SNF characteristics after 23 years of cooling.

Please keep in mind that the surrogate in Table 3.8 is constructed using 1^st order PCE, which only requires 12 simulation results. Furthermore, the surrogate results in Table 3.8 are derived by using a 1^st order PCE surrogate to predict the model responses of 1,000 samples, and then calculating the mean and standard deviation. The burnup and k-inf are obtained after discharge, while the rest of the results are obtained after 23 years of cooling. The surrogate results, as shown in Table 3.8, are very similar to the STREAM results. In a subsequent analysis, 2^nd and 3rd order surrogates are developed and applied to the 300 and 1,000 input samples, to estimate the model response and the associated uncertainty. When comparing the results to those of STREAM, we see that the first order surrogate is sufficient for the assembly under consideration, despite the fact that it was built with a smaller number of STREAM calculations. It is worth noting that the computation time for the 1^st, 2^nd, and 3^rd order surrogates is only a few seconds.

Table 3.8. Moments of the responses at 23 years of cooling.

Response Mean RSD (%)

STREAM Surrogate STREAM Surrogate Activity (Bq) 4.5697E+15 4.5695E+15 1.55 1.56 Decay heat (W) 4.1747E+02 4.1745E+02 1.91 1.92 Neutron source (n/s) 9.8740E+07 9.8718E+07 5.80 5.79 Gamma source (photons/s) 2.2331E+15 2.2331E+15 1.38 1.38 Burnup^a (GWd/tU) 3.6676E+01 3.6676E+01 0.80 0.80 k-inf^a (-) 8.8627E-01 8.8624E-01 0.50 0.50 ^a See footnote of Table 3.5.

Figure 3.8 shows the total and first order effects of the Sobol' sensitivity indices at 23 years.

The Sobol' indices show the main sources of uncertainty as well as the less important uncertain parameters, which can be left out of the study because they are nearly zero effect on the output variance. Concerning the specific powers, the Sobol' indices have been combined into one effect.

The most significant sources of uncertainty, as shown in Figure 3.8, are the fuel radius, specific power, and moderator temperature. The clad radius and the fuel density also play a role, but to a lesser extent. Enrichment, fuel temperature, and boron concentration, on the other hand, make insignificant contributions to the integral source term uncertainty.

54

Figure 3.8. First order and total Sobol’ indices for uncertain input parameters at 23 years.

By using their unperturbed values, future UQ investigations on our test case may ignore parameters with insignificant contributions to the uncertainty. This is an example of dimensionality reduction using a surrogate model and GSA, i.e., reduced order modeling. The total and first order Sobol' indices can be compared to see if interactions between the input parameters have an effect on the output variance. The expected contributions of the high order interactions between the input parameters to the response variance are negligible because the total and first order Sobol' indices are nearly identical, as illustrated in Figure 3.8. As indicated in Section 3.4, input correlations are not accounted for in the computations, however examples of such correlations, such as between power and fuel/moderator temperature, have been documented [120]. When examining the responses at discharge and during the cooling time considered, however, correlations between the output can be seen, as shown in Figure 3.9.

55

Figure 3.9. Correlation matrix of source terms for the 15 x 15 fuel assembly.

With the exception of the neutron source, the trend of the Sobol' indices in Figure 3.8 is similar to the relative sensitivities in Table 3.5. It is possible that the observed disparities between Figure 3.8 and Table 3.5 are related to the linearity assumption utilized in Table 3.5 to estimate the relative sensitivities. If the output is nonlinearly dependent on the input parameters, the linearity assumption results in an approximation. However, Sobol' indices can be used with nonlinear responses.

The uncertainties caused by the finite size of input data is estimated using the bootstrap method [121] for the point-wise predictions of the surrogate model. To construct bootstrap replications, one hundred sample sets are drawn from the first set of input data by resampling with replacement. Each replication is equal in size to the initial batch of input data and is used to calculate the PCE. This generates a collection of 100 surrogate models, each with its own set of coefficients and responses. Due to the finite size of the input data, the sets of outputs are then used to calculate the standard deviation of the surrogate point-wise predictions. Table 3.9 shows the maximum RSD of the 1st order surrogate point-wise predictions. Furthermore, the sets of PCE coefficients from the bootstrap replications were used to estimate the RSD of the surrogate mean and standard deviation. Table 3.10 summarizes these results. As the accuracy of the surrogate model is being assessed, these tables reveal that the uncertainties present in the surrogate point- wise estimates, mean, and standard deviations are minor. The interval in which the true surrogate prediction can be found at a specific confidence level can be determined from the uncertainty analysis of surrogate predictions.

56

Table 3.9. Maximum RSD of surrogate point-wise predictions at 23 years.

Activity Decay heat Neutron source RSD (%) 3.14E-02 4.91E-02 1.86E-02

Gamma source Burnup^a k-inf^a RSD (%) 8.79E-03 3.00E-04 2.15E-02

a See footnote of Table 3.5.

Table 3.10. Error of surrogate model mean and standard deviation at 23 years (%).

Activity Decay heat Neutron source RSD of mean 4.83E-03 7.06E-03 2.74E-02

RSD of σ 4.91E-01 6.56E-01 8.94E-01

Gamma source Burnup^a k-inf^a RSD of mean 1.58E-03 7.95E-05 4.07E-03

RSD of σ 1.47E-01 9.89E-03 8.64E-01

a See footnote of Table 3.5.