2.3 Machine Learning for Evapotranspiration
2.3.1 Artificial Neural Network
21
22
learning machine (ELM) and the back-propagation neural network (BPNN), each with different architectures and estimation functions. A general illustration of ANN is provided in Figure 2.1.
Figure 2.1: General Structure of the ANN
The application of the ANN is a simulation of biological neurons in the nervous system, where neurons are connected via synapses. In the ANN, the neurons are connected between layers through weights and biases. This intrinsically establishes the relationship between the input and output layers during the learning process (Abiodun, et al., 2018).
In recent years, the use of the ANN had been evolved by increasing the number of hidden layers. This was accompanied by an improvement in the types
{ { {
Input Layer
Hidden Layers
Output Layer
23
of hidden neurons. Instead of using classical hidden neurons regulated by activation functions, advanced hidden neurons can exist in the form of long short-term memory (LSTM) cells, gated recurrent unit (GRU) and so on. This revolution sees a paradigmatic shift to the utilisation of the deep neural network (DNN) for various applications, including ET0 estimation. As compared to the classical ANN, the DNN is believed to be relatively capable of learning more complex relationships due to its ability to store different states within the cells (Hu, et al., 2018). Furthermore, the emergence of big data and cloud computing also provides a more conducive environment for the application of the DNN.
However, the computational (and time) cost of the DNN is much higher than the conventional ANN and thus unworthy when dealing with small sets of data (Nagappan, Gopalakrishnan and Alagappan, 2020). The simple and easy applications of the ANN have attracted the attention of numerous researchers to estimate ET0 using ANN, and subsequently, their research has attained tremendous achievements. Table 2.1 summarises research using ANN in ET0
study.
24
Approach Key Findings Reference
MLP • MLP was used to estimate mean monthly ET0 in humid subtropical environment
• MLP was trained using PM model as target and compared with calibrated HS model
• When Tmax, Tmin and Ra were used as input meteorological variables, MLP six hidden nodes achieved the best performance
(Rahimikhoob, 2009)
BPNN • Daily Tmax, Tmin, Ra, u, RH and sunshine hours were used to train the BPNN using PM model as reference in the area with semi-arid climate pattern
• Different combinations of input meteorological variables were tested
• Temperature-based BPNN performed better than HS model
• Inclusion of u and RH data further enhanced the estimation accuracy
• Ra and sunshine hours did not play a significant role in the improvement of model accuracy
(Traore, Wang and Kerh, 2010)
25
Table 2.1 (continued): Summary of Research using the ANN in ET0 Study
Approach Key Findings Reference
RBF GRNN
• Daily Tmax, Tmin, Tmean, u and sunshine hours were used to train the networks to estimate ET0 based on PM model in the temperate zone with mild Mediterranean climate
• Different combinations of input meteorological variables were tested to investigate the best possible dataset
• GRNN generally performed better than RBF
• Removal of u caused a sudden drop in prediction accuracy
• The performance of the models could be further improved by the addition of precipitation data
(Ladlani, et al., 2012)
MLP • Daily Tmax, Tmin, Tmean, Rs, u, RHmean, RHmin and sunshine hours were used to train the MLP using PM model as reference in the area with semi-arid climate pattern
• The MLP was trained using different learning algorithms, including Levenberg-Marquardt, Delta-Bar-Delta, Step, Momentum, Conjugate Gradient and Quickprop
• Levenberg-Marquardt, Delta-Bar-Delta and Conjugate Gradient algorithms with hyperbolic tangent transfer functions performed the best
• The combinations of input meteorological variables with the highest prediction accuracy were Tmean, RHmin, u and sunshine hours, which well explained the properties of ET
(Tabari and
Hosseinzadeh Talaee, 2012)
26
Approach Key Findings Reference
Generalised ANN
Multi Linear Regression (MLR)
• Daily Tmax, Tmin, Rs, u and RH were used to train the models using PM model as reference in humid, sub-humid, arid and semi-arid areas
• One station was used to train in one climate region, while another was used for testing
• Different combinations of input meteorological variables were tested
• As the number of input meteorological variables decreased, the prediction accuracy decreased gradually
• Generalised ANN performed better than MLR
(Wang, et al., 2013)
ELM
Feed-Forward Back-Propagation (FFBP) Network
• Daily Tmax, Tmin, Rn, u and RH were used to train and test the models using PM model as target in arid and semi-arid regions
• Different combinations of input meteorological variables were tested for both networks
• ELM and FFBP estimated daily ET0 with comparable accuracy
• ELM was claimed to be more efficient
• Reduction of input meteorological variables did not significantly affect the prediction accuracy
(Abdullah, et al., 2015)
27
Table 2.1 (continued): Summary of Research using the ANN in ET0 Study
Approach Key Findings Reference
MLP
Probabilistic Neural Network (PNN) Generalised Feed-Forward (GFF) Linear Regression (LR)
• Study was carried out to forecast daily ET0 using forecasted temperature in humid subtropical environment
• The four models were simulated with forecasted daily Tmax and Tmin
• Performances of MLP, PNN and GFF were slightly better than LR
• Errors were accumulated from the inaccuracy of forecasted temperature and short of climate data
(Luo, et al., 2015)
RBF MLP
Support Vector Machine (SVM)
• Monthly Tmax, Tmin, ea, u and sunshine hours were used to train the MLP using PM models as reference in moderate Mediterranean region
• Optimisation of RBF network was modified, either by back-propagation or particle swarm optimisation
• RBF performed better than MLP and SVM, but method of optimisation did not show a significant difference
(Petković, et al., 2015)
ELM • Monthly Tmax, Tmin, ea, u and sunshine hours were used to train ELM based on HS model, PT model as well as Turc model in the temperate region
• The three models did not show notable differences; however, ELM could be employed as it reduced the complexity in calculating ET0
(Gocic, et al., 2016)
28
Approach Key Findings Reference
ELM MLP
Genetic Programming SVM
• Monthly Tmax, Tmin, u, pan-evaporation rate, rainfall and sunshine hours were used to train the models using PM model as reference in humid subtropical region
• Estimation of ELM was more accurate as compared to other models and also required lesser computational time
• ELM using sigmoid transfer function performed better than its hard limit transfer function counterpart
(Kumar, et al., 2016)
ELM MLP
Least-Square Support Vector Machine (LS-SVM)
• Weekly Tmax, Tmin and Ra were used to train the networks to estimate ET0
based on HS model in arid region
• Further investigation also included ET0 value from other stations as input to the model
• ELM was proved to have the best performance as it required less human intervention with good estimation efficiency
• Inclusion of external ET0 value further enhanced the accuracy of estimation
(Patil and Deka, 2016)
29
Table 2.1 (continued): Summary of Research using the ANN in ET0 Study
Approach Key Findings Reference
MLP (back-propagation) PNN
GFF LR
• Models were trained using daily Tmax, Tmin, Rs and Ra using PM model in humid subtropical environment
• Forecasted climate data with different combinations were feed into the networks
• It was found that MLP with Tmin, Tmax and Rs as input meteorological variables could produce the most accurate results, up to 15 days forecast horizon
• Tmax was claimed to be the most significant factor for ET0 estimation
(Traore, Luo and Fipps, 2016)
MLP • Daily Tmean, Rs, u and RH were used to train the MLP using PM model as reference in humid continental region
• Different combinations of input meteorological variables were tested
• MLP network with complete set of meteorological variables performed the best, followed by models that contained temperature and radiation data
(Antonopoulos and Antonopoulos, 2017)
ELM GRNN
• Daily Tmax, Tmin, and Ra were used to train the networks to estimate ET0
based on PM model in warm humid region
• ELM and GRNN outperformed HS model by reduction of deviation from actual ET0 value
(Feng, et al., 2017b)
30
Approach Key Findings Reference
Random Forest (RF)
Extreme Gradient Boosting (XGBoost) MLP
Convolutional Neural Network (CNN)
• Only temperature and humidity data were used to estimate ET0 on an hourly basis
• The MLP outperformed XGBoost and RF, while the CNN performed better in relation to the MLP
• The authors recommended further studies on the deep learning approaches
(Ferreira and da Cunha, 2020)
RBF CNN
• Tmax, Tmin and u were used as the predictors after performing principal component analysis (PCA)
• CNN performed better than RBF in ET0 estimation, however, the time needed for training was longer
(Nagappan,
Gopalakrishnan and Alagappan, 2020)
DNN • DNN was designed from MLP with four hidden layers
• It was found that input combinations that only included the temperature and radiation data were able to produce ET0 estimation with accuracy close to that of complete dataset
(Sowmya, Santosh Kumar and Ambat, 2020)
31
As shown in Table 2.1, a considerable number of studies had been done on the utilisation of the ANN as an ET0 estimation tool. However, the trend of the studies generally focused the following few aspects:
• Minimisation of mandatory inputs
• Generalisation of the ANN for wider applications
• Application of novel input features
• Improvisation of ANN’s ability to forecast
It is believed that the four aspects stated above could revolutionise the prediction of ET0, with a more general model without the need for much climate data. On top of that, a longer forecasting horizon acts as an important pre- requisite for a pro-active water management strategy. Unfortunately, using ANN alone seems to be insufficient in providing the solution. Hence, the coming subsections will be focussed on the discussion of other machine learning models that were employed in ET0 estimation.