DECLARATION 2: PUBLICATIONS
7. A COMPARATIVE PERFORMANCE ASSESSMENT BETWEEN THE FINAL
7.5 Results and Discussion
Figure 7.1 shows the ACRU and SCS-SA simulation results obtained for two catchments, namely Cathedral Peak IV (V1H005) and DeHoek/Ntabamhlope (V7H003), and compares them to the observed data. These two catchments were selected to graphically depict the typical results obtained. The two catchments also have significantly different stormflow responses as indicated by the CNs and QFRESP parameter values in Table 4.1. Graphical plots, similar to
those presented in Figure 7.1, for the remaining verification catchments are provided in Appendix G (Chapter 16). It is important to emphasise that the ACRU model simulates total streamflow which includes both stormflow and interflow/baseflow. The SCS-SA model, on the other hand, simulates only stormflow. Both models, however, use simulated stormflow to simulate the stormflow contribution to peak discharge. In the ACRU model the baseflow/interflow volume is uniformly distributed throughout the day and converted into a constant flow rate which is added to the simulated stormflow peak discharge. This contribution to the peak discharge, however, is negligible, particularly for design events.
For catchment V1H005 (Figure 7.1), the simulated design stormflow volumes from the SCS- SA model for CN-II and the MCM are very similar, since the median (50th percentile) change in soil water for this catchment is effectively zero, therefore the adjustment to CN-II is insignificant and the CN-II value is retained for the MCM. The simulated design streamflow volumes from the ACRU model and simulated design stormflow volumes from the SCS-SA model for CN-II and the MCM are considerably different for catchment V1H005 (Figure 7.1).
These results indicate the importance of accounting for interflow/baseflow in terms of reproducing observed streamflow volumes correctly. This is particularly relevant to catchments such as catchment V1H005, which has highly permeable soils with high infiltration rates, dense vegetation and a low stormflow potential, as indicated by the low CN and QFRESP parameter values for this catchment (Table 4.1). For this catchment interflow/baseflow contributes significantly to the design streamflow values simulated by the ACRU model, i.e. when plotting only the simulated stormflow volume from the ACRU model the plot produces results very similar to the SCS-SA model for CN-II and the MCM (Figure 7.1). Therefore, since the SCS- SA model only simulates stormflow, a significant portion of the total streamflow is not accounted for, which is a limitation of the SCS-SA model. Consequently, for catchments such as catchment V1H005, the simulated stormflow from the SCS-SA model is a relatively poor approximation of the observed streamflow. For catchments such as catchment V7H003, however, which has less permeable soils with lower infiltration rates, less dense vegetation and a higher stormflow potential, as indicated by the higher CN and QFRESP parameter values in Table 4.1, the simulated stormflow volumes from the SCS-SA model are more comparable to the simulated streamflow volumes from the ACRU model. This is because stormflow dominates over interflow/baseflow for catchments such as catchment V7H003, and therefore the simulated stormflow is a closer approximation of the observed streamflow.
Figure 7.1 Observed and simulated design streamflow/stormflow volumes and design peak discharges for Cathedral Peak IV (V1H005) and DeHoek/Ntabamhlope (V7H003), applying both the ACRU and SCS-SA models
It should be noted, however, that the ability of the ACRU model to explicitly account for antecedent soil water also contributes to the differences observed between the design streamflow volumes simulated by the ACRU model and the design stormflow volumes simulated by the SCS-SA model. For catchment V7H003, the results from the SCS-SA model for the MCM are slightly better than the results from the SCS-SA model for CN-II.
In summary, since the SCS-SA model does not account for interflow/baseflow, the model underestimates design streamflow volumes. This highlights the advantage of using the ACRU CSM system over the SCS-SA model. This is supported by the results presented in Figure 7.1, where the ACRU model produces results most similar to the observed data, i.e. across the entire range of design values from 2 – 100 years. In terms of the design peak discharges very slight differences between the results simulated by the ACRU model and those simulated by the SCS- SA model for CN-II and the MCM were observed (Figure 7.1). This was expected since both the ACRU and SCS-SA models use simulated stormflow to simulate the stormflow contribution to peak discharge, and the simulated stormflow volumes from both models are very similar since the ACRU stormflow response was calibrated based on the SCS-SA stormflow response.
The similarity in the peak discharge results suggests that antecedent soil water has a limited influence on the simulated stormflow volumes from both models, and since these differences are small there are small differences in the resulting simulated peak discharges. This further emphasises the significance that interflow/baseflow has on the total simulated streamflow, i.e.
the differences in simulated stormflow volumes from the SCS-SA model and streamflow volumes from the ACRU model are predominantly due to the fact that a significant fraction of the simulated streamflow in the ACRU model comprises of interflow/baseflow.
Ultimately, both models provide reasonable estimates of the design peak discharges, however, there is a consistent over-simulation. This has been attributed to variations in daily stormflow responses, catchment lag time and rainfall intensity, all of which are approximated with estimates of average or typical conditions. Further room for improvement, particularly with the ACRU model, has been documented to account for these variations on a daily basis. With the SCS-SA design event-based approach only typical conditions or ensembles of possible scenarios can be simulated for design events, without the ability to replicate the actual conditions prior to each design event. The use of ensemble events or Monte Carlo simulations, however, with event-based models such as the SCS-SA model has large potential and is an
approach that has received increasing attention in recent years (Kjeldsen et al., 2010; Blöschl et al., 2013; Ball et al., 2016). This provides uncertainty bands and estimations of worst-case scenarios which provides more information to the design engineer to make more informed decisions. The SCS-SA model, however, does not account for the interflow/baseflow contribution to total streamflow which is a limitation of the approach.
Although not directly comparable, for the reasons stated in Section 7.2, the results from the SCS-SA JAM for both catchments are generally poor in terms of both the design stormflow volumes and design peak discharges. The results are more reasonable for return periods from 2 – 10 years. For the 20-year return period, however, there is a substantial increase in the quantiles, due to a frequency analysis being performed on simulated flows from a relatively short record (approximately 20 years), as explained in Section 7.2. In general, however, there is a significant overestimation of design values for these two catchments when applying the JAM.
The overall performance of the ACRU CSM system and the SCS-SA model for all verification catchments, excluding the Lambrechtsbos B (G2H010) Catchment, is summarised in Figure 7.2. For consistency, as performed in Chapters 4 and 6, the results from the Lambrechtsbos B (G2H010) Catchment were excluded, due to challenges associated with modelling this catchment and associated poorly simulated results, as detailed in Chapter 4.
Figure 7.2 Average MARE/MRE values obtained for simulated versus observed Design Streamflow/Stormflow Volumes (DnV) and Design Peak Discharges (DnQp), averaged across all verification catchments, excluding Lambrechtsbos B (G2H010), for both the ACRU and SCS-SA models
When comparing the average MARE/MRE values in terms of design streamflow (ACRU) and
lowest MARE (0.25), with a positive MRE (0.14), indicating a tendency to overestimate observed design streamflow volumes in general. The MARE for the SCS-SA model, when applying CN-II, is only slightly larger (0.29) compared to the ACRU CSM system, however, the MRE is significantly lower (-0.09), indicating a greater tendency to underestimate the observed design streamflow. Both the MARE and MRE when applying the SCS-SA model with the MCM, are only slightly lower, 0.26 and -0.13 respectively, compared to those obtained for the SCS-SA model applying CN-II. Therefore, in general the SCS-SA model does not seem to be very sensitive to changes in antecedent soil water for the catchments assessed.
The JAM produced the highest MARE (0.51), with a positive MRE of 0.14. This indicates that, in general, there is an overestimation of the observed design streamflow, however, in many cases there is also significant underestimation, i.e. as indicated by the relatively lower MRE compared to the MARE. Overall, however, the JAM did not perform well and, as alluded to above, this is attributed to: (i) the use of historically assigned rainfall stations with limited record lengths, and (ii) the use of frequency analyses and not extreme value analyses used in the development of the approach and results generated.
In terms of the design peak discharges, the ACRU CSM system and the SCS-SA model applying CN-II and the MCM provided similar results (Figure 7.2). This again indicates that in terms of design stormflow volumes both the ACRU CSM system and the SCS-SA model are producing similar values, and therefore similar design peak discharges are obtained. There are, however, more significant differences between the design streamflow and design stormflow volumes from each model, since the ACRU model includes the interflow/baseflow contribution to the total streamflow. Both the ACRU CSM system and the SCS-SA model when applying CN-II and the MCM generally overestimate design peak discharges, with the ACRU CSM system producing the lowest MARE (1.47) and MRE (1.46) and the SCS-SA model applying CN-II the highest MARE (1.66) and MRE (1.63) values, i.e. when comparing the results from these three scenarios, excluding those from the SCS-SA JAM. The slightly better results obtained for the ACRU CSM system compared to the SCS-SA CN-II and MCM may be attributed to explicit accounting of antecedent soil water and an extreme value analysis being performed on the AMS extracted from continuous simulations of daily peak discharges. The general overestimation of the observed design peak discharges is attributed to one or a combination of the following: (i) inaccurate simulations of stormflow volumes for certain
design values, (ii) poor approximation of the actual daily rainfall distribution for design events by the synthetic rainfall distribution selected, and (iii) inaccurate estimation of the catchment lag time, i.e. as explained in the analysis of the results from catchment V1H005 and V7H003 above. Once again, the JAM produced the highest MARE (2.55) and MRE (2.41) in terms of design peak discharges.