DECLARATION 2: PUBLICATIONS
6. IMPACT OF MODEL CONFIGURATION AND PARAMETER ESTIMATION
6.5 Sensitivity of the CSM System to Different Sources of Input Information
6.5.3 Results and discussion
is reflected in the MRE/MARE values where this scenario produces the lowest values, indicating that the most accurate DnV and DnQp estimates are obtained for the Current CSM System scenario. In terms of the DnV for this scenario, the MRE is lower than the MARE, indicating a combination of both under and overestimation. In terms of the DnQp for this scenario, the MRE and MARE are the same, indicating consistent over-simulation of the DnQp values. It is important to highlight that the results varied from catchment to catchment, however, these results summarise the overall general performance of each scenario.
Figure 6.2 Average NSE values obtained for simulated versus observed Daily Streamflow Volumes (DyV) and Daily Peak Discharges (DyQp), averaged across all verification catchments, excluding Lambrechtsbos B (G2H010), for each model scenario
When applying the ACRU National Soils scenario, the results were slightly worse compared to those obtained from the Current CSM System in terms of both NSE (Figure 6.2) and MARE/MRE values (Figure 6.3). Therefore, when using the CSM system it is better to use the default soils information assigned to the selected SCS-SA soil group, as defined in the rules developed by Rowe (2015) and Rowe et al. (2018), and not the soils information obtained from the most updated national soils map (Schulze and Horan, 2008). This makes sense since the rules developed by Rowe (2015) and Rowe et al. (2018), and incorporated into the CSM System, are based on calibrations performed using this default soils information.
Figure 6.3 Average MARE/MRE values obtained for simulated versus observed Design Streamflow Volumes (DnV) and Design Peak Discharges (DnQp), averaged across all verification catchments, excluding Lambrechtsbos B (G2H010), for each model scenario
It is important to highlight at this stage that changes in simulated streamflow volumes have a significant influence on the simulated peak discharges, as documented in Chapters 4 and 5, i.e.
since the simulated peak discharges in the model are directly dependent on the simulated streamflow volumes. This is particularly evident in both the NSE and MARE/MRE results for the scenarios where default SCS-SA soil group information is used, i.e. Schulze 2012 SCS Soils and Schulze and Schütte 2018 SCS Soils. For example, for relatively small changes in NSE values in terms of DyV there are significant changes in the corresponding DyQp NSE values.
The same trend is seen when comparing the DnV MARE/MRE values to the DnQp MARE/MRE values. The results from these two scenarios in terms of both the NSE (Figure 6.2) and MARE/MRE values (Figure 6.3), and particularly in terms of the DyQp and DnQp values, are significantly worse compared to those obtained for the Current CSM System scenario. The Schulze and Schütte 2018 SCS Soils scenario performs substantially better than the Schulze 2012 SCS Soils scenario, however, in general both scenarios performed poorly.
This indicates the sensitivity of the CSM system to the SCS-SA soil group selected, and inherently the sensitivity of the SCS CN approach, i.e. since the ACRU model was
parameterised based on the SCS-SA CNs. Therefore, the results indicate that, in general, if the SCS-SA soil group is not correctly determined for use with the CSM system poor results may be obtained, with over-simulation of DnV and particularly significant over-simulation of DnQp.
This highlights the importance of accurately estimating the SCS-SA soil group for a catchment, when applying the CSM system. Furthermore, since the CSM System was calibrated against SCS-SA CNs, this warning is also directly transferable to the SCS-SA model. Ultimately, the results indicate that the national soils maps poorly represent the actual SCS-SA soil group information at such localised scales, i.e. the national soils maps cannot capture the site-specific soils information for such small catchments. Therefore, further work on, or refinement of, the national SCS-SA soil maps is required. Based on the sensitivity of the results to the SCS-SA soil group selected, another possible consideration is that the changes in CN for each SCS-SA soil group are too sensitive and abrupt, and that the CNs for SCS-SA soil groups and land cover classes possibly need to be recalibrated for South African conditions, realising that the CNs were adopted from the SCS (1956) classification developed in the United States many years ago. In addition, in many cases CN values were simply interpolated between and extrapolated beyond other values, with very limited verification of the CN values being performed in South Africa, prior to this study. That being said, however, the SCS CNs were derived using observed data, it is therefore possible that such changes in stormflow response for corresponding changes in SCS-SA soil groups are indeed correct. This, however, can only be verified through further research, using observed data from catchments with specific land cover and soil combinations.
In terms of the NLC 2000 scenario, the NSE (Figure 6.2) and MARE/MRE (Figure 6.3) values were similar to those obtained for the Schulze and Schütte 2018 SCS Soils scenario. In terms of the NSE values, however, the NLC 2000 scenario produced a DyV NSE value substantially lower than that obtained for the Schulze and Schütte 2018 SCS Soils scenario, the DyQp NSE values, however, were very similar with the NLC 2000 NSE value being only slightly higher than that of the Schulze and Schütte 2018 SCS Soils scenario. The MARE/MRE values between the two scenarios were very similar in terms of both the DnV and DnQp. The overall error (MARE) was slightly lower for the NLC 2000 scenario, however, with a greater tendency to overestimate design values, i.e. with a slightly higher MRE value compared to the Schulze and Schütte 2018 SCS Soils scenario. For this reason, both the MARE and MRE in terms of DnQp values were slightly higher for the NLC 2000 scenario. The results for the NLC 2000 scenario therefore indicate that the CSM system is also sensitive to the land cover information used and
the consequent land cover class selected. Similar to the use of default soils information, the use of default land cover maps and assigned land cover classes did not produce particularly good results, i.e. compared to those obtained from the Current CSM System. A degree of conservatism, however, was incorporated into the default land cover maps, as detailed in Section 6.5.1.3, which explains the deterioration in the results. This, similar to the results obtained from using national soils maps, indicates the importance of accurately estimating the actual land cover class for the catchment. In addition, based on the sensitivity of the results to the land cover class selected, the results, once again, possibly suggest that the changes in CN for each land cover class are too sensitive and abrupt, and that the CNs possibly need to be recalibrated for South Africa. Since the SCS CNs were derived using observed data it is, however, possible that such changes in stormflow response for corresponding changes in land cover classes and/or conditions are indeed correct. Once again, this can only be verified through further research, using observed data from catchments with specific land cover and soil combinations.
The final scenario assessed, was the SCS Lag Equation scenario. Since the lag equation only influences peak discharges, the NSE and MARE/MRE values, in terms of DyV and DnV respectively, are identical to those obtained for the Current CSM System scenario. In terms of the DyQp NSE and DnQp MARE/MRE values, however, the results are significantly worse for the SCS Lag Equation scenario. Therefore, for small catchments the Schmidt and Schulze (1984) lag equation produces better results. The results also, once again, indicate the sensitivity of the ACRU peak discharge computation to lag time estimates.