4.4 Data Fusion III: Non-Linear Neural Ensemble
4.4.3 Summary
164
WOA-ELM-E decreased to the 10 scale as compared to the 10 scale of the BMA-E. This means that the bias of WOA-ELM-E’s estimations of ET0 was even lower than those of BMA-E’s. Comparing the WOA-ELM-E and BMA-E, the former had better generalisation ability that could cope with data from all clusters.
Meanwhile, the R2 of the WOA-ELM-E was comparable to the BMA- E, which means that both ensembles fit well into the data of each station, whereby different input combinations have been considered. On the other hand, the error metrics, including the MAE, RMSE and MAPE showed improvement when the model centric BMA data fusion was replaced with the black-box NNE approach.
165
A scoring system introduced by Despotovic, et al. (2015), known as the GPI, was calculated for every model. The positive-oriented GPI was inclusive of all the performance evaluation metrics used in this study and can provide an overall score for the models. In this study, the GPI scores of the models were calculated based on the normalised values of the performance evaluation metrics according to the different number of input meteorological variables. The comparison of GPI scores of the models developed in this research work (MLP, SVM, ANFIS, BMLP, BSVM, BANFIS, BMA-E and WOA-ELM-E) are shown in Figure 4.29 to Figure 4.33 (from Cluster 1 to Cluster 5). The actual values of the GPI can be referred to Appendix D (Table D6 to Table D10).
Figure 4.29: GPI Scores of Different Machine Learning Models at Stations in Cluster 1
166
Figure 4.30: GPI Scores of Different Machine Learning Models at Stations in Cluster 2
Figure 4.31: GPI Scores of Different Machine Learning Models at Stations in Cluster 3
167
Figure 4.32: GPI Scores of Different Machine Learning Models at Stations in Cluster 4
Figure 4.33: GPI Scores of Different Machine Learning Models at Stations in Cluster 5
In Figure 4.29 to Figure 4.33, WOA-ELM-E appeared to be the most stable model that maintained constantly in the upper bracket in terms of the GPI values. This can be used as strong evidence to justify the NNE developed in this study as a promising tool in improving the estimation of ET0 in various regions in Peninsular Malaysia, even though with only one input meteorological
168
satisfactory performance in most cases. However, the performance of the BMA- E was strongly correlated to the performance of the base models, in which their poor performance would result in a poor BMA-E (Chen, et al., 2015). This can be clearly seen at Station 48620 (Sitiawan) in Figure 4.30. The BMA-E had low GPI as the base model also performed poorly at that station.
The base MLP appeared to be sufficient for ET0 estimation at some of the stations, such as Station 48623 (Lubok Merbau). A similar observation is obtained for the BMLP. However, the performance of the MLP and BMLP was not as stable as the BMA-E or WOA-ELM-E. In other words, the outstanding performance of MLP and BMLP are only exceptions that happened in certain regions only.
Table 4.6 tabulates the best model for daily ET0 estimation at all stations.
The WOA-ELM-E was selected as the best ET0 estimating model at most of the stations, regardless of the number of input meteorological variables. On the contrary, the BMA-E was only selected when the number of input meteorological variables was low. This is because the BMA-E could only manage to improve the model when different weights were assigned to different base models. Consequently, the favourable traits of different base models can be incorporated into the BMA-E to achieve estimations with higher quality.
Although BMA-E was also selected at Station 48601 (Bayan Lepas) and Station 48603 (Alor Setar) for six and five input meteorological variables, respectively,
169
however, the BMA-E was merely the MLP as the Bayesian weight of a unit was given to the MLP in both cases. As for the other models such as the MLP, BMLP and ANFIS, they were selected in some cases where they perform better marginally than the WOA-ELM-E and the BMA-E. Therefore, a conclusion can be reached at this stage, in which the WOA-ELM-E was the best ET0 estimating model in Peninsular Malaysia, in which it has wider spatial applicability among the eight models developed in this study.
The next section of this thesis will discuss the transferability of the locally developed WOA-ELM-E at external stations to investigate their spatial robustness. In other words, the local ET0 values were estimated using exogenous models to eliminate the need for local data collection and model development/calibration.
170
Station Number of Meteorological Variables
1 2 3 4 5 6
Station 48600 (Pulau Langkawi) WOA-ELM-E BMA-E WOA-ELM-E WOA-ELM-E WOA-ELM-E MLP
Station 48601 (Bayan Lepas) BMA-E MLP WOA-ELM-E BMLP WOA-ELM-E BMA-E
Station 48603 (Alor Setar) BMA-E WOA-ELM-E WOA-ELM-E WOA-ELM-E BMA-E WOA-ELM-E
Station 48615 (Kota Bharu) ANFIS WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E
Station 48620 (Sitiawan) WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E
Station 48623 (Lubok Merbau) MLP WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E
Station 48625 (Ipoh) MLP BMA-E WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E
Station 48632 (Cameron Highlands) WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E WOA-ELM-E
Station 48647 (Subang) WOA-ELM-E WOA-ELM-E WOA-ELM-E BMA-E WOA-ELM-E MLP
Station 48649 (Muadzam Shah) WOA-ELM-E BMA-E MLP WOA-ELM-E WOA-ELM-E WOA-ELM-E
Station 48650 (KLIA) WOA-ELM-E BMA-E WOA-ELM-E WOA-ELM-E WOA-ELM-E MLP
Station 48657 (Kuantan) WOA-ELM-E BMA-E WOA-ELM-E BMLP BMLP WOA-ELM-E
171 4.5 Models Transferability
Results and discussion from Section 4.1 to Section 4.4 focussed on Scenario 1, in which the machine learning models (both base and hybrid) were trained and tested locally. However, in order to curb the issue of qualitative data hunger, the machine learning models should be transferable. In other words, they must be spatially robust so that they can be applied elsewhere other than the location where they were trained. In this section, results involving Scenario 2 and Scenario 3 will be discussed further to prove the spatial robustness of the machine learning models developed in this research work.