V. Machine Learning System for Spent Fuel Analysis
5.8. Uncertainty Quantification of Machine Learning Models
5.8.1. Uncertainty Quantification due to Perturbation of Input Features
In the first UA, the uncertainties in the selected input features of Table 5.3 are considered.
Due to uncertainties, there are studies in safeguard and non-proliferation applications [9] which aim to verify some nuclear fuel assembly model parameters even when such information are declared by the reactor operators. The input data and associated uncertainties are assumed to follow a normal distribution. The uncertainties at one sigma value in the input data are obtained from literature and presented in Table 5.11 for the PWR assemblies. Due to lack of uranium mass uncertainty information in literature, the uranium mass was considered unperturbed, and its nominal value was left unchanged in each sample. The input parameters are assumed to be independent so that correlations between them are neglected. The first UA is the conventional Monte Carlo method, and it is summarized in the following steps (see Figure 5.9): (i) assign uncertainties to all the input features as shown in Table 5.11 (ii) for each of the 91 PWR dataset records, generate 105 samples of perturbed input data following a normal distribution by LHS (iii) evaluate the decay heat using the perturbed data as inputs in the trained ML models (GPR, SVM and NN) (iv) perform the needed statistics on the decay heat evaluations. These steps are repeated for all the fuel assembly measurement data in the PWR dataset. The nominal input values are in the dataset and the input uncertainties are listed in Table 5.11. All the original PWR dataset records are used to train the models in performing the first UA i.e., UA due to perturbed inputs features.
The models developed in this work serve as cheap surrogates for fast evaluation of the decay heat, despite the large number of evaluations required in the UA. The generation of 105 samples of each perturbed input record and calculation of the decay heat by the model takes about 2 minutes for the entire dataset. This is another application of ML models to problems which cannot be dealt with by time consuming computer code systems. For the PWR dataset, a total of 9.1 x 106 perturbed inputs are generated in this first UA for which the perturbed decay heat output should be calculated.
For a computer code this will be computationally prohibitive. However, with the ML models developed in this work, the large number of perturbed outputs can be calculated in very few minutes. This demonstrates the effectiveness of the ML models suggested in this work in applications where time consuming computer code systems cannot be applied. The GPR and SVM models accurately map the input features to the decay heat response because their training errors are very small. The results of the UA due to uncertainties in the input data are shown in Figures
133
5.10 and 5.11. The uncertainties reported in Figure 5.10 corresponds to the relative standard deviation (i.e., ratio of decay heat standard deviation to the mean) and these are presented against varying fuel assembly burnup, decay time and enrichment, to identify physics based trend. Because of the presence of high multicollinearity in the input space previously discussed in Section 5.2, it is difficult to identify any trend in Figure 5.10. Figure 5.11 shows the decay heat PDFs of 105 runs of two different records from the dataset. The absolute uncertainties are also shown in Figure 5.11.
First we observe, as expected, in Figure 5.11 that the uncertainties due to perturbed input features are similar in the three models. Second observation is that the decay heat uncertainties due to perturbed input features are not large considering that the CI bands are small. Despite the small uncertainties in Figure 5.11, it is important that such error estimates can be quantified to increase confidence in ML predictions. An important aspect of the ML models developed in this thesis is that the predictions can have intervals where the true value lies at a specific confidence level.
Table 5.11. Input features of PWR dataset and uncertainties (1𝜎).
Parameter Uncertainty (%) Distribution Ref.
Burnup (GWd/tU) 1.6
Normal [10]
Decay time (days) 1.5
Enrichment (wt.% 235U) 0.05 [169]
Figure 5.9. Uncertainty analysis due to perturbed input features.
134
Figure 5.10. Relative uncertainty of decay heat caused by input feature perturbation. This applies to the PWR original dataset.
135
Figure 5.11. PDFs of decay (top). Absolute uncertainty of decay heat (bottom). The Uncertainty is caused by input feature perturbation and applies to the PWR original dataset.
136
The decay heat relative uncertainties shown in Figure 5.10, which are obtained from the ML models due to perturbed input features are now presented in detail in Tables 5.12 to 5.16.
Some of these uncertainties can be compared to those obtained from STREAM in Chapters III and IV. However, the comparison will be limited to modeling parameter induced uncertainties. It is important to mention that the surrogate model in Chapters III and IV has input parameters (see Table 3.1) different from those of the ML models (Tables 5.1 to 5.3), except for the 235U enrichment which is common in both. For the comparison of uncertainties, only the assemblies listed in Table 4.1 are considered. This is because each of these assemblies already have STREAM simulation results in which the assembly modeling parameters are perturbed 100 times. These results were used as reference solutions to build the surrogate models employed in Chapter IV.
The comparison of uncertainties between STREAM and the ML model is illustrated in Table 5.17.
This table shows that the uncertainties are comparable, although the ML model shows higher uncertainty in some cases. Recall that the input features of the ML model are sampled independently, it might thus be possible to reduce the ML model uncertainties in Table 5.17 if we first consider the input correlations shown in Table 5.1 and sample the ML input features from a joint PDF. A second way to reduce the ML model uncertainties is to calibrate its input features by the method of Chapter IV. This second approach can further improve the model performances shown in Tables 5.5 to 5.9 by reducing the RMSE and MAE. In addition, for the STREAM uncertainties shown in Table 5.17, the model parameters perturbed are the nominal ones. If the calibrated parameters of those assemblies from a posterior distribution are used, the uncertainties will be further reduced as demonstrated in Chapter IV (Table 4.6). Perhaps the most important finding from Table 5.17 is that the ML model can be used to quantify the modeling parameters induced uncertainties which is comparable to those obtainable from STREAM. This is important from the standpoint of computational efficiency. Please note that the uncertainty results in Tables 5.12 to 5.16 are based on 1,000 perturbed samples while those in Table 5.17 are obtained from 100 perturbed input samples in STREAM and the ML model calculations, except otherwise stated.
137
Table 5.12. Uncertainty of decay heat of 15 x 15 assemblies from Ringhals 2 PWR.
Assembly ID
Burnup (GWd/t)
Enrichment (wt.% 235U)
Decay time (days)
Uncertainty (%)
C01 36.7 3.1 8468 2.22
C12 36.4 3.1 8403 2.17
C20 35.7 3.1 6950 2.68
C20 35.7 3.1 6951 2.68
C20 35.7 3.1 6952 2.68
C20 35.7 3.1 6959 2.68
C42 35.6 3.1 5803 2.15
C42 35.6 3.1 5804 2.08
D27 39.7 3.3 7673 2.37
D38 39.4 3.3 8005 1.99
E38 34.0 3.2 7999 1.72
E38 34.0 3.2 8000 1.72
E40 34.3 3.2 8075 1.72
F14 34.0 3.2 7726 1.73
F14 34.0 3.2 8306 1.75
F21 36.3 3.2 7376 2.23
F21 36.3 3.2 7944 1.81
F21 36.3 3.2 9377 2.04
F21 36.3 3.2 9524 2.01
F25 35.4 3.2 7729 1.87
F25 35.4 3.2 8297 1.89
F25 35.4 3.2 9733 1.74
F25 35.4 3.2 9734 1.75
F32 51.0 3.2 5860 1.14
G11 35.5 3.2 6990 2.21
G23 35.6 3.2 6984 2.20
I20 34.3 3.2 6588 1.67
I09 40.2 3.2 5849 2.60
I09 40.2 3.2 6458 2.22
I09 40.2 3.2 7927 2.33
I24 34.3 3.2 6601 1.73
I24 34.3 3.2 7203 2.03
I25 36.9 3.2 6198 1.74
138
Table 5.13. Uncertainty of decay heat of 17 x 17 assemblies from Ringhals 3 PWR.
Assembly ID
Burnup (GWd/t)
Enrichment (wt.% 235U)
Decay time (days)
Uncertainty (%)
2A5 20.1 2.1 7297 1.45
2A5 20.1 2.1 7699 0.93
5A3 19.7 2.1 6972 2.63
5A3 19.7 2.1 6975 2.59
5A3 19.7 2.1 6977 2.57
5A3 19.7 2.1 7291 1.40
5A3 19.7 2.1 7304 1.35
5A3 19.7 2.1 7683 0.77
5A3 19.7 2.1 7691 0.80
5A3 19.7 2.1 9467 4.01
0C9 38.4 3.1 6551 1.93
1C2 33.3 3.1 6559 1.89
1C5 38.5 3.1 6593 1.98
1C5 38.5 3.1 8599 2.03
2C2 36.6 3.1 6550 2.32
2C2 36.6 3.1 6954 2.66
2C2 36.6 3.1 6955 2.66
3C1 36.6 3.1 6545 2.30
3C1 36.6 3.1 6949 2.70
3C4 38.4 3.1 6544 1.90
3C5 38.4 3.1 6543 1.96
3C5 38.4 3.1 6943 2.16
3C5 38.4 3.1 6948 2.16
3C5 38.4 3.1 8713 2.20
3C9 36.6 3.1 6552 2.32
4C4 33.3 3.1 6572 1.90
4C7 38.4 3.1 6549 1.95
0E2 41.6 3.1 5823 2.09
0E2 41.6 3.1 6389 1.98
0E2 41.6 3.1 6390 1.98
0E2 41.6 3.1 7826 1.93
0E2 41.6 3.1 7837 1.95
0E2 41.6 3.1 7970 2.19
0E6 36.0 3.1 5829 1.94
IE5 34.6 3.1 5818 1.66
5F2 47.3 3.4 4724 1.21
5F2 47.3 3.4 5332 1.42
5F2 47.3 3.4 6803 1.50
139
Table 5.14. Uncertainty of decay heat of 15 x 15 assemblies from Turkey Point PWR.
Assembly ID
Burnup (GWd/t)
Enrichment (wt.% 235U)
Decay time (days)
Uncertainty (%)
B-43 25.6 2.6 1782 1.96
D-15 28.4 2.6 957 1.65
D-15 28.4 2.6 1139 1.87
D-15 28.4 2.6 2072 1.76
D-22 26.5 2.6 958 2.56
D-34 27.9 2.6 859 1.83
Table 5.15. Uncertainty of decay heat of 14 x 14 assemblies from San Onofre 1 PWR.
Assembly ID
Burnup (GWd/t)
Enrichment (wt.% 235U)
Decay time (days)
Uncertainty (%)
C-01 26.5 3.9 3011 1.07
C-16 28.5 3.9 3012 1.85
C-19 30.4 3.9 3011 2.17
C-20 32.4 3.9 3011 2.14
D-01 31.4 4.0 2358 1.61
D-46 32.3 4.0 2360 2.00
E-18 32.4 4.0 1794 1.84
F-04 30.4 3.9 1078 0.72
Table 5.16. Uncertainty of decay heat of 14 x 14 assemblies from Point Beach 2 PWR.
Assembly ID
Burnup (GWd/t)
Enrichment (wt.% 235U)
Decay time (days)
Uncertainty (%)
C-52 25.6 3.4 1635 1.99
C-56 28.4 3.4 1634 1.60
C-64 28.4 3.4 1633 1.24
C-66 28.4 3.4 1630 2.05
C-67 26.5 3.4 1629 1.57
C-68 27.9 3.4 1630 1.79
140
Table 5.17. Comparison of uncertainties between STREAM and ML model.
Assembly
ID Burnup
(GWd/tU) Enrichment
(wt.% 235U) Decay time
(days) STREAMa (%) ML (%)
5A3 19.7 2.1 6972 1.77 2.59
5A3 19.7 2.1 6975 1.77 2.56
5A3 19.7 2.1 6977 1.77 2.53
5A3 19.7 2.1 7291 1.78 1.43
5A3 19.7 2.1 7691 1.79 0.80
5A3 19.7 2.1 9467 1.82 4.03
0C9 38.4 3.1 6551 2.18 1.96
3C5 38.4 3.1 6948 2.22 2.20
3C5 38.4 3.1 8713 2.25 2.26
4C4 33.3 3.1 6572 2.14 1.94
0E2 41.6 3.1 5823 2.14 2.11
0E2 41.6 3.1 6389 2.14 2.00
0E2 41.6 3.1 6390 2.14 2.00
0E2 41.6 3.1 7826 2.14 1.93
0E2 41.6 3.1 7837 2.14 1.94
0E2 41.6 3.1 7970 2.14 2.22
C20 35.3 3.1 6950 1.94 2.64
C20 35.3 3.1 6951 1.94 2.64
D27 39.7 3.3 7673 1.99 2.43
E38 34.0 3.2 7999 2.01 1.74
E38 34.0 3.2 8000 2.01 1.74
F21 36.3 3.2 7376 1.88 2.27
F21 36.3 3.2 9377 1.91 2.13
F21 36.3 3.2 9524 1.91 2.06
G11 35.5 3.2 6990 1.82 2.22
G23 35.6 3.2 6984 1.77 2.21
I20 34.3 3.2 6588 1.66 1.70
E40 34.3 3.2 8075 1.63 1.74
C01 36.7 3.1 8468 1.73 2.22
C01b 36.7 3.1 8468 2.02 2.21
C01c 36.7 3.1 8468 1.91 2.22
a STREAM results based on 100 perturbed input samples.
b, c STREAM results: 300 and 1,000 perturbed input samples, respectively (cf. Table 3.7).
141