Chapter 7: Results of the LSTM forecasting models
7.6 Results of the LSTM models
7.6.4 Predictions based on Multivariate LSTM Model
Figure 7.10: Multivariate LSTM model training error for the NZX 50 Index (sample period three: nonlinear movement with COVID-19)
Figure 7.8 shows the history of the multivariate LSTM model training for the NZX 50 Index during the upward trend period (22/01/2009 to 31/12/2017). Figure 7.9 shows the multivariate LSTM model training error history for the NZX 50 Index data related to the nonlinear trend period [without the COVID-19 impact (1/1/2007 to 31/12/2017)]. Figure 7.10 represents the history of the multivariate LSTM model training error for the NZX 50 Index data during the nonlinear trend period with the COVID-19 exposure (01/01/2007 - 31/12/2020)].
Figure 7.8 – Figure 7.10 follows the characteristics of a “good fit model” as the plot of training loss and test loss in each figure decreases to the point of stability. Therefore, it is possible to conclude that the formulated multivariate LSTM model is a good fit for both the 'training data set' (training cohort) and the 'testing data set' (testing cohort).
Table B.16 in Appendix B show the prediction results of Multivariate LSTM originating from its application to the NZX 50 Index from 2009 to 2017 (sample period one). Table C.16 in Appendix C show the forecast results of Multivariate LSTM originating from the models’
application to the NZX 50 Index from 2007 to 2017 (sample period two). Likewise, Table D.16 in Appendix D provide the extrapolation results of Multivariate LSTM applied to NZX 50 Index from 2007 to 2020 (sample period three). Examination of the actual and predicted series in Tables B.16, C.16 and D.16 show that Multivariate LSTM efficiently predicts the NZX 50 Index for all the sample periods investigated.
Figures 7.11 – 7.13 portray the comparative time plots displaying the NZX 50 Index actual and forecast series. More precisely, Figure 7.11 represents the time plots showing observed values against the predicted values where the predictions originated from the application of Multivariate LSTM to sample period one (2009 to 2017). Similarly, Figure 7.12 portrays the time plot showing the actuals against forecasts where the predictions are obtained from Multivariate LSTM’s application to sample period two (2007 to 2017). Likewise, Figure 7.13 displays the comparative time plot showing the actual series against the forecasts where the predictions are derived from Multivariate LSTM’s application to sample period three (2007 to 2020).
Figure 7.11: Multivariate LSTM Actual versus Prediction – Period one (2009 – 2017)
Figure 7.12: Multivariate LSTM Actual versus Prediction – Period two (2007 – 2017)
Figure 7.13: Multivariate LSTM Actual versus Prediction – Period three (2007 – 2020)
Residuals that are derived from the applications of Multivariate LSTM to the three samples are then further scrutinised. The descriptive summary statistics of the residuals are presented in Table B.17 in Appendix B, Table C.17 in Appendix C, and Table D.17 in Appendix D. More precisely, Tables B.17, C.17, and D.17 display the residuals of Multivariate LSTM applied to sample periods one, two, and three, respectively. For example, during sample period one (2009 – 2017), the centre measures of the residuals are clustered around 0, which reinforces the prediction accuracies of the Multivariate LSTM.
In addition, Tables B.18, C.18, and D.18 provide the OLS regression results based on the predictions from the Multivariate LSTM applied to different sample periods (periods one, two, and three) as a function of the actual observations. The analyses performed here are similar to evaluations conducted in Univariate LSTM, ARIMA and HWES discussions. This performance evaluation uniformity allows me to compare and contrast the ideal prediction model under different tested samples and conditions, which will be examined in Chapter 8.
Three widely used hypothesis tests are performed to determine the effectiveness of the OLS regressions. Firstly, I evaluate the overall joint significance of each of the estimated regressions. A null hypothesis of no collective explanatory power of the estimated regression (all regression parameters are zero) is tested using the F-Test at a 5 percent significance level.
The P-value of each F-statistic for the estimated OLS regressions is statistically significant at 5 percent significance, thus, concluding that each estimated OLS regression presented in Tables B.18, C.18, and D.18 have joint explanatory power. Secondly, the slope coefficients in these tables are statistically significant as the P-values are less than the 5% significance level. Thus, this evidence confirms the rejection of the null hypothesis that the actual (true) slope coefficient is zero. This finding justifies that the variable named “actual” has a statistically significant effect on the “predicted” series in each OLS regression presented in Tables B.18, C.18, and D.18. Finally, the R2,[304]–[306], which measures the explanatory power of the estimated
regressions, is used to assess the goodness of fit of the estimated OLS models. For example, the R2 of Table B.18 is 0.999988, which confirms that the Multivariate LSTM is an efficient forecasting model for the NZX 50 Index for period one (2009 – 2017). The findings from the descriptive summaries of the residuals (presented in Tables B.17, C.17, and D.17) and the OLS regressions (given in Tables B.18, C.18, and D.18) validate the predictive precision of the Multivariate LSTM models when they are employed to the NZX 50 Index in the tested periods.
When LSTM models are comparatively evaluated, it is evident that the univariate LSTM outperforms the Multivariate LSTM for every sample period we examined. This conclusion is reached because the error measures (RMSE, MAE, and MAPE) of Univariate LSTM for each sample period tested are significantly lower than the error measures of Multivariate LSTM.