Chapter 2: Factor Augmented Artificial Neural Network Model
2.4 Forecasting models
In this section, the basic concepts and modeling approaches of the dynamic factor model (DFM), autoregressive model (AR) and artificial neural networks (ANNs) models for time series forecasting are presented. The section also introduces the formulation of the proposed model.
19
2.4.1 Dynamic Factor model forecast
The estimated factors will be used to forecast the variables of interest. The forecasting model is specified and estimated as a linear projection of an h-step ahead transformed variable into t- dated dynamic factors. The forecasting model follows the setup in Stock and Watson (2002a) and Froni et al. (2003) which takes the form:
̂ where ̂ are dynamic factors estimated using the method by Stock and Watson (2002b) while are the lag polynomials, which are determined by the Schwarz Information Criterion (SIC). The is an error term. The coefficient matrix for factors and autoregressive terms are estimated by ordinary least square (OLS) for each forecasting horizon .
2.4.2 Autoregressive (AR) Forecast
The AR model is given by
(2.5) where is the variable to forecast, is a constant, is the iteratively estimated lag polynomial, the lag order is chosen by SIC and is the error term.
The h-step ahead forecast from this model is
(2.6) where is the h-step ahead forecast2 of , are the iteratively estimated lag polynomials, is the h-month ahead forecast error term.
2In this paper we choose iterated forecast instead of direct forecast. Marcellino et. al (2006) found that iterated forecast using AIC lag length selection performed better than direct forecasts, especially when forecast horizon increases. They argued that iterated forecast models with lag length selected based on information criterion are good estimates to the best linear predictor.
20
The benchmark AR forecast is individually applied to our variables of interest, namely, Deposit rate, Gold mining share prices and Long term interest rate. The optimal lag length is chosen by SIC.
2.4.3 The ANN and the formulation of the FAANN model
Artificial neural networks (ANNs) are model free dynamic, which are widely used for forecasting. One of the important advantages of the ANN models over other classes of nonlinear models is that ANNs are universal approximators that can approximate a large class of functions with a high degree of accuracy. See Chen et al. (2003) for more details. There is no need for prior assumptions about the model form during the model building process.
Figure 2.1 Neural network model ( )
Fig. 2.1 shows a popular three-layer feed-forward neural network model. It consists of one input layer with input variables, one hidden layer with hidden nodes, and one output layer with a single output node. The hidden layers perform nonlinear transformations on the inputs from the input layer and feed the transformed values to the output layer. The connection weights and node
Hidden layer
𝑦𝑡 𝑦𝑡
𝛼 𝛼𝑗
𝛽𝑖 𝑗
𝛽𝑗
Output layer Input layer
𝑦𝑡 𝑦𝑡 𝑝
…
21
biases are the model parameters. The model estimation process is called network training.
Usually in applications of ANNs, the total available data are split into a training set and a test set.
The training set is used to calibrate the network model, while the test set is used to evaluate its forecasting ability. During the training procedure, an overall error measure is minimized to get the estimates of the parameters of the models. The mathematical representation of the model in Fig. 2.1 that show the relationship between output ( ) and the inputs ( ) is given by;
∑ ( ∑ ) (2.7) where ( and are the model parameters often called the connection weights. As we stated above and are the number of input nodes and hidden nodes respectively, is error term. The logistic function is usually used as the hidden layer transfer function, which is generally given by;
(2.8) There are many different approaches to find the optimal networks but these approaches are quite complicated and are difficult to implement, and in addition there is no guarantee that the optimal solution of these approaches is optimal for all real forecasting problems. Thus the procedure often used to determine ( ) is to test numerous networks with different numbers for ( ) to select the network that minimize the error. The minimization is done with some efficient nonlinear optimization algorithm; in our case we use Broyden, Fletcher, Goldfarb and Shanno (BFGS) algorithm, see Nocedal and Wright (2006).
22
2.4.4 Formulation of the FAANN model
The unique properties of ANN models motivated us to augment the factors to the ANN models to produce a more accurate forecast. The ANN models properties include; the relationships between input and output variables do not need to be specified in advance, since the method itself establishes these relationships through a training process. The ANN models do not require any assumptions on the underlying population distributions.
Time series forecasting research has demonstrated that the combined models improve forecasting performance substantially. For example see Khashei and Bijari (2010) and Zhang (2003). These combined models reduce the risk of failure compared to a single model where the underlying process cannot easily be determined or a single model may not be adequate to identify all the characteristics of the series.
In this chapter, we introduce the factor augmented artificial neural network (FAANN) model; the proposed model is a hybrid model of artificial neural network and factor model in order to produce more accurate forecasts. In the FAANN model the series is considered as nonlinear function of several past observations and the factors - that are extracted from large dataset that relate to the series under consideration – are as follows:
( ) (2.9) where is nonlinear functional form determined via ANN. In the first stage, the factor model is used to extract factors from a large related dataset. In the second stage, a neural network model is used to model the nonlinear and linear relationships existing in factors and original data. Thus,
∑
( ∑
∑
)
23
As previously noted, the ( and are the model parameters often called the connection weights; as we stated before and are the number of input and hidden nodes respectively, is the error term. Fig. 2.2 represents the FAANN model architecture.
Figure 2.2: The FAANN model architecture (