BAYES ESTIMATION FOR ARMA MODEL FORECASTING UNDER NORMAL-GAMMAR PRIOR.

Bayes

Estimation

for

ARMA

Model

Forecasting

under

Normal-Gamma Prior

Zul

Amryl'*

and

Adam

Baharum2

1

Department

of

Mathematics,

State

University

of

Medan,

lndonesia

2

School

of

Mathematical

Sciences,

Universiti

Sains Malaysia, Pulau Pinang, Malaysia *Corresponding

Author:

zu l.a

ryrrv@gmpil,cop

Presented

On

The

3'd

International

Seminar on

Operation

Research.

To

Celebrate The

50th

Department

of Mathematics

University

of

Bayes

Estimation

for

ARMA

Moder

Forecasting

under

Normal-Gamma

Prior

Zul

Amryl,-

and Adam Baharum2

2schoo,",T:ffi:ll:1iT:*H',il1,::?:i,"Ji;Jil:,$H:fr1;i,Ti#x'ilMa'aysia

-Corresponding

Author: zJl.am{r@gmail.com

Abstract

This paper presents a _{Bayesian approach to} find the Bayes estimator for the point forecast and forecast variance of

ARMA model under normal-gamma prior assumption with quadratic loss function in the form mathematical expression. The

conditional posterior predictive density be obtained based the posterior under normal-gamma prior and the conditional predictive density' whereas the marginal conditional posterior predictive density be obtain-ed based the conditional posterior

predictive _{density. Fufihermore, neither the point forecast nor the forecast variance are}

both derived from the marginal conditional posterior predictive density.

Keywords

ARMA Model, Bayes Theorem, Normal-Gamma prior

1.

Introduction

Bayes theorem calculates the posterior distribution as proportional _{to the} product of a prior distribution and the likelihood function' The prior distribution is a probability model desiribing the knowledge about ihe parameters before observing the

currently _{by the available data [1 ]. Main idea of Baye_sian forecaiting is the}

prrii.tiu"

_distribution

of the future given thJfast

data follows directly from the joint probabilistic model. The predictlve distiibution is derived from the sampling predictive

density, weighted by the posterior _{dishibution [4]. This pup"i ir done refer to Liu [15]}

discussed the Bayesian u"nutyiri, ro, one-step ahead forecast in ARMA model and _{Liu [16]} also discussed the multiperiod ftrecast for ARX model employed the Bayesian approach. others the paper related to this research are Amry and Baharum _[2],Fan& yao _{[7], Kleibergen & Hoek}

[13]' and Uturbey _{[20] also discussed the Bayesian analysis for ARMA model. This'paper focuses to}

find the mathematical expression of the Bayes estimator for the point forecast and forecast variance of

ARMI

model under normal-gamma prior assumption with quadratic loss function

2.

Materials

and

Methods

The materials in this paper are some theories in mathematics and statistics such as the ARMA repeated integration, gamma distribution, _{and the univariate student's t-distribution. The method}

applying the Bayesian analysis under normal-gamma prior assumption.

2.1. ARMA model

The ARMA (1t, _{q)model [15] defined by:}

model, Bayes theorem,

is study of literature by

(2.1)

0,

and

Q

are

written

(2.2) y, _=f.o y,-,

*

fe,r,

_, * r,

i=t j=t '

where

{e'}

is

sequence

of i

i

d

_{normal random variables with e,-N(0,rt1,}

r>0

_{and unknown,} parameters.

2.2.Bayes theorem

Suppose there are

two

discrete random variables

X

atd.

)r,

then

p( x,y) = p( xl y) p,( y) and the marginal probability density function

of

p*(x)=

_z,

p(x,y)

:

2,p(x|y)

p,( y)

Bayes rule for the conditional p( y I x

)

is

the

joint

probability function

X

[18] is:

p(

ylx)=

p(x,y)

:

p(xly)

p,(y) _

p(xly)

p,(y)

P*(r)

p,(x)

Z,p(xly)pr(y)

IfIis

continuous, the Bayes theorem can be stated as:

,r

r'

r1=- PQ-2)-P'1!

t-j

p(xly)

p,r y )dy

by using the proportional notation (oc _{) which} can be expressed by:

p(ylx)

6

p(x)y)p,(y)

where p(

y

_I x) is posterior _{distribution, p(x}

I

I

is likelihood function and p,( y _)is prior distribution.

/

-\

,n'k-t'

p I

_.1,-r

,f

p

,

t,ll

L\v,

r s])x

r ,

^rl-iLLlr,-Lo,t

-2t,,

))j

The equation (3.3) can be expressed as:

/ \ (n.k-ttp ( _1._o_,

r(p.rlsjJ

cr , *fl-tllti

I z I t=p+r

where B,

:

(y,,

!tt,

_{..., lt+t_p, e6} er_t, ..., €*t_q) 2.3. Gamma distribution

A positive random quantity

/

is said to have a gamma _{distribution with paramet er n}

>0

_{and d>0}

if

_{it has the probability}

density _{function [17]:}

e@=#ia*'

expf-4al

the notation isQ

-

c(n,d)

the mean is

o(a)=]

and the variance

is

var(Q)=ft

2.4. _{Univariate student-t distribution}

A _{r >} random _{0if it} quantity,

x,

is said to have a student-t distribution on n degrees of freedom with mode p and scale parameter

has the probability density function _[17]:

r( n*1),1

-

.,

-

,,, p1x

y)

l)-1,*6-

r')'

|

'

rl;)@:

r r

l

Thenotationisx-t,(p,r),tnemeanis

r(x)=p

_{andthevariance}

isvar{x)=J7-,ifn>2

3. Results

3.1. _{Likelihood Function for ARMA model}

The k-step-ahead point forecast

ofl

,*

_{t ,} defined by :

i(k)=E(Y,*ls]:)

where Sj =(y,, y,,..., y,*o_,)

Based the equation (2.

l)

be obtained residuals as:

e, _{= y,}

-tt,y,-,

-f,o,r,_,

_(3.2) By conditioning the first p _{observations and} lerling _eo:eo-t:...:

€,:

e

_where

r:

_min(0,

p+l-q),

_{one may approximate}

byBox&Jenkins[4],thelikelihoodfunctionfory:(Q1,Q2

,000t,02,..,0)andr

based

sj t,

(2.6) (2.3)

(2.4)

(2.s)

(2.7)

(3.1)

(3.3)

nrk-t n+k- | ll -

2y'

L

/,8,_,+

y

@, a,_,)' _ll

By

letting

U =

p - _c_r

where

6,=y,-2d,y,,-ZF,aFj,t:p+l,p+2,...,n,

_6,

&

d,

aremaximumlikelihoodestimator

of

Q,

&

0,,and

a,, a,-,,..., a,-oo. lutua.a

iru'

A, _{=y, -Fr} B,_,

the likelihood function in equation (3.4) can be expressed as: Yp !p+t !n*-2

lp-t Yp !n+k,3

/ t /z /n+k-t-p €o €p*t €n+k-2

€ _p-t €, €n+k-j

€ _p*t-q € _p*z-q en+k-t-p

W:U(.f,and

V:Uxo

.'li';'i:l

(3.5)

(3.6)

3.2. Posterior distribution under normal-gamma prior

According to Broemeling and Shaarawy's suggestion _[5], the normal-gamma prior of parameters Pand ris:

€(Y,r)=6,(Y )r).

(,(r)

n

r'f-'*o{-1lr'g*-v'ep-p'ev+pre1.t+2Bj}

'l 2'

,.tl

)

where

f, -

,Ut,Gd-')

,

I , - c.tu (a, p) _{, Q} is apositive definite matrix of the order (p + q),

a

_{and B} are parameters.

By applying Bayes theorem to equation (3.6) and (3.7), the posterior

of

!o

and r-,

is: (n-k-tt-p { -f*r-r ll

t\v.r

s))ne,

rrpl-=llyl

-zy'v+v'wvll

|

2I,A;

lj

n(Y,r-t1si 1=

r(v,r1s)),

a(v,,)

n+k-t-p'2d+p_t _{( ,t -}

n

q.

r,

*pI-jlv,ev-v,

(v+ep)-(y+q1,fv+x)j

where

p:w+e

and. K _{=')t*,,ri + 1tr e1t} + 2 p .

3.3. Conditional posterior predictive density Based

on

", =

,,

-f,O,r,-,

-fr,",-,

with e,

-

u(o,r-,

)

be obtained f (et1s:,y,r-t )= (zo,

Ifexpressed in y, is:

f(y,lS),V,r-' 1=(2r,

'

''-t LI

)'

uP\-jl,

_-LQ,Y,-, P _-

P,tr,,f'

I

Based the equation (3.9), be obtained the conditional predictive density of y _{, *} p..

f

( y.,o I s),y, t

-'

) = (z o,-' I

i

*r{-

}lr.

"

- fit, r.. 0,, -

f

_,,

n,-,)'

}

L f-f

p

o

l'l

*,'

*,

I_

;Lt,.r

_l,.,

_t,-o-,

_\t,,,*_,

) ]

*

': *r{-+l

,.-r

-lf.r*.---,

.ir,"..--,ll'l

[ "L \,=i i-t )))

Bychanging

f4,y,,o-,+f,o,r,*-,

to:

,)r *o{_LrG),}

(3.8)

(3.e)

Qt!*t-t *02!n,*-z t .,.1-dp!,*-p _{+ete,+k-t t0z€n+*-z +,..+eqe,+k_q : (Ot Q, , 0,}

e j e2 0r)

where B*k-t=0*u,, _{!n+k-2,..., !o+r-p,e,11-1,} en+k-2, ..., e,*o_o), theequation (3.10) can be written as:

-f 6,.0 | s;, v,

"

-,

)

*,i

*o

{- !r[ r,. r -

r,

u, * _,]']

*

,1

*p7-ilf,-r

_- zv, a,-* _{t rn+t}

*

_@,

u*r-,)'lj

I f

,r

*

,i

*pl-ily',-o

*v,

Ry

-

2v7 t,,r_,

,,-r

lI

where R ₌ Bn+r_t@ BI*o_, and (v, n,*o_,)t _=v,

nv

Basedontheequation(3.8)and(3'll),beobtainedtheconditionalposteriorpredictivedensity

ofy,t

f

_{o(t,*ls:,,Y,r-t}

)

n

o(v,

'"

_{lsj ) f}

(y,.ols),v)1 I

!n+k-l

! n+k-2

!n+*-p

€ n+k-t

€ _n+k-z € _n*-q

: _V, B,*O-'

,-o-,-o-r".0-,

I

lv'av

-Yr 1lt +Q1t+ B,u-,

y*o)-

f)

0.,

T'

*ofil(y

_{+ep)r + BI**., !,.* 1v}

*

_yl.r

*

_rcll

t"Lll

(n+k-l- p+2d _)+l

2

(3.1l)

(3.12)

(3.1 3)

whereG:P+R

3.4. _{Marginal conditional posterior predictive density}

The marginal conditional posterior predictive density

of

Y

,*

_r be obtained

by

integrating the conditional posterior predictive density in equation (3.12):

_f

,(t,.rls,)

=

I [

_{-f ,f}

t,,rls:,y,r1)

dydr

_:-=

ii''-*'-{-

;l:;,o.*r;'.'

:;r':)l

;;

i;-,:r'

:

-)}

n

*

o,

71,--x-4:]:!_,

_{^'_l(r_o-,(1tt+g1t1+n,+k.trn+k)), c(v_c_,(1v+e1t)+8,*r,y,_r)\_f ,...,}

*

_J

J

'

z--'

"-'l'i,

)nu,*

B,t*-t _!n-t

),

c-, (rt +gpt* B,-*-r

t,,*

_)*y,,,0 +

K

)o'

o'

*u -Q - Bl,o-, G-t B,*r-t

f'

(t:,0-, c-, g, * p1.,1)]

_K-{V+ep)t

Gotv+gp1

+

(n+ k _- t - p+ 2") _{U -}

B|*,

B,;J

The marginal conditional _{posterior predictive density}

of

y,*1

_{is a univariate}

student,s t_distribution on (n+k_l_p+2a)

degrees of freedom with mode _tr= (t

-

_{a1-o-, G-' Bn*o-,)" (u:.r-,}

c'

1v + g1t1) and scale parameter

_

K-(V+eu)rG^(V+Ou)

T= -" v\

(n + k _-

t-

p+ za)Q _-

ffi+;J'

where co =G-t *Q

-

Bl.o-,G-'8,-r-,)-'

(c-'ao

'1

3.5. Point _{forecast and forecast variance}

For quadratic loss function, the point forecast of Y , *p is the posterior mean of the marginal conditional posterior predictive, that is:

*l'.0-'-0.'"

and the forecast variance of

I,

* r is:

(n+k-1- p+2a) K-(V+ 1r Golv +gpy

r*(v,,.r _1s')=

+k-l-p+2a

=(n + k _- 3 _- p + za)-' (x - p, + gp _{1, c o1v} + gp 1) _{Q -} ul,*, G -, B,*o_,), (3. 1 5)

4.

Conclusion and

computational

procedure

This paper analyzes how to find mathematical expression ofthe point forecast in equation (3.14) and the forecast variance in equation (3.15) based the marginal conditional posterior prediitive density in aquation

f:.r:1.

The conditional posterior predictive density be obtained by multiplying the normal-gamma prior to the condiiional piedictive density. By iniegrating

the _{conditional posterior predictive density to paramaters Y and}

t

_{respectively, obtained tire marginal .onditionut}

po-sterior predictive density. Furthermore, the point forecast and the forecast uuiiun.. are derived based the mean and the variance

of

marginal conditional posterior predictive density that has the univariate student's t-distribution. procedure _{to compute the} point forecast and the forecast variance are as follows:

Defines:

Yp !p+t !n+r-: lpt Yp !n+k,3

::::

y. _, v_{h+x t_p}

€ _p*l €n+k-2

€ _o € _n+k-3

:::

€ _p*z-q en+k-t-q

ll. XO:

U= !t

ao

6

o_,

e

l';:.:,1

l:l

lr..r

I

iii.

iv.

Compute:

v.

vi. vii.

viii. K ₌

ix. x.

xi. xii.

xiii.

xiv.

It, Q, a

and

_B

B,* -, =\y *o -,' !.nr-l,..., / n+t-t- p, € _n+k,t, € _n+*-2, ..., r,*-,u)

W:ULf

V:Ux6

P:IY+Q

^.f'

yl

_*

p'ep

_{+ 2p}

R= B,,r_,& BI*_,

G:P+R

G-1

Go = G-t + _{Q -} Bl.o_,G-'a,,r_,)-'(c-'nc-')

E(y,.k ) s: )

:

Q - a:.r-, G-' B,*0,,)-' (u:.,,, c-, 1v + gp 1)

t5l

References

lll

Alba, E. And Mendoza, M. Bayesian Forecasting Methods for Short Time Series. Article

l2l

Amry,

Z'

and

Baharum, A. (2015)' Bayesian Multiperiod Forecasting for Arma Model under Jeftey,s prior. Mathematics and Statistics 3(3), 65-70.

13l

Bain, L'J.

and

Engelhardt, M'

(2006).

Introduction to Probability and Mathematical statistics, 2nd. Duxbury Press, Belmont, California.

I4l

Bijak, _{J' (2010). Bayesian Forecasting and Issues}

of

_{Uncertainty. Centrefor population}

change, (Jniversity

of

Southampton, I-19.

Broemeling,L.and Shaarawy(1988).TimeSer_iesA_BayesianAnalysisinTimeDomain, _inBayesian analysis of Time series and Dynamic Moders, (J.c, spail, ed.; l-2l,MarcelDekker, New york. DeGroot, M.H'

(2004).

optimql Statistical decisions. John Wiley

&

Son. Inc. New yersey

{an' _Reseqrch,

c'

a,nd _1(4),49-55.Yao, s'(2008). Bayesian Approach for ARMA Process and Its Application. International Business

t8l

Gelman, A.(2008), objections to Bayesian statistics, Boyesian Anatysis 3(3), 445_ 450.

t9]

Geweke, J and whiteman, C., (2004), Bayesian _{Forecasting,Deparfment of Economics, University of Iowa.}

[10] Hussein, H' M' A' (2008)' _{Bayesian Analysis_of the Autoregressive-Moving Average}

Model with Exogenous Inputs

Using

Gibb

Sampling. Journar of Appried sciences Reseirch,

a(r2),

tss-5-rsg2.

ll

ll

_Department, Ismail' M'A' and Amin, A.A. (2010). Gibbs Sampling for SARMA Models. paper, Faculty of Commerce, statistics

Menoufia (Jniversity,

Egpt,

_{l_15.}

tl2l

Ismail, M'A' and Charif, H'A. (2003). Bayesian _{Inference for Thresold Moving Average Models.}

METR7N-Internqtional Journal ofStatistics, vol.

LXI,

Ilg_132.

tl3l

Kleibergen, F.

and

Hoek, H.

(1996).

Bayesian

Analysis

of

ARMA

_{model using Non informative prior.} Paper, _{Econometric Institute, Erasmus University, Rotterdqm, I_24}

ll4]

Lee, J'c', Lin, T.I. and Hsu, Y.L' (2004). Bayesian Analysis of Box-cox _{transformed linear mixed models with} ARMA(p,q) dependence. Journal of Statiitical Planning and Inference, t33(2005), qiS-.aSl.

I I

5]

Liu, S. I. ( 1995). Comparison of _{Forecasts for ARMA Models Between A Random coefficient}

Approach and A Bayesian Approach. Commun. Statist.

_

Theory Meth., 24(2), 3l 9_333

tl6l

Liu,

S' I. (1995). _Bayesian

Multiperiod

_Forecasts

for

_ARX

Models . Ann. Inst. statist. _{Mqth. vol. 47, no.2,} 2 I 1_224.

[17]

Pole, A', west, M' and Harrison _{, J.}

(lgg4).

Apptied

_{Bayesian Forecasting}

and

Time series Analysis. Chapman and Hall, New york.

tl8l

Ramachandran, _K'M'

and

_{Tsokos, c.P.}

(2009).

_Mathemqtical

statistics

with Applications, Elsevier Academic press. _{San Diego, California.}

[19]

_sankhya: Safadi, T' and Morettin, P.A.

(2000)'

Bayesian

Analysis

of

Threshold Autoregressive Moving Average Models.

The Indian Journar _{ofstatistics, vorume}

6i,

_{series B,} pt.

3,

_353-37r.

I20l

_^u**t9ty'

w

(2006).

Identification

of

ARMA Model _{by Bayesian Methods}

Applied

to

_{Srreamflow Data.}

9''

Intetnational

_conference

on

_{Probqbilistic}

uethLi-'ippti"d

to

power

systems

KTH

(

l

l-15 June 2006, Stockholm, Sweden), t_7

t6l

@

;

E @ H

s

(t)

o

jj

F-csP

ER

()rJ 9r,

cg C)

>8

({< ()

vc)

>C/]

€

==

2!u cd r,'l

TIr

E

\z

ud

at

\a

(g

7-E

'..--lt c= i) \\ o ll

.\---'-.,

r,,:;ffi

;

5 E o

E

_e E-r

=

CA 63 H tr C€

U

&

k C) I ca

o

-l

F

tn

t-i IJ

c.l

&

^

\J 'F{ L.r C) .l_)

(-F

\/

r-l C)

t{

-)

tv

()

a

C.)

^7

:

P

_F

io

.Fl

.E

,k

_C)

+

c

a1 ts

o

$-r

€tn

=F

;cg9

h€cl

qo".i

a

rfrrt

NE-^c..

-sxg:*

\'vrx5

'F!

r-.,,

bO

s\

i6 E

=,

\h

*)trtr

-'1

{t!20.8

Ql {_) rh

q.s2o

--=

-SLo

*t

'i

6

i-

-i-t €

-!=:''e'==

-\

SFTfi

?;e

Judul Makalah

Penulis Makalah

ldentitas Makalah

Kategori Publikasi Makalah (beri

/pada

kategori yang tepat)

a. Judul Prosiding

b. ISBN

c. Tahun Terbit d. Penerbit e. Jumlah halaman

f.

WEB Laman

{

Prosiding Forum llmiah lnternasional

Lampiran 8

lnternational Seminar on Operational Research (lnteriOR 2015)

August 2015

USU Medan LEMBAR

HASIL PENITAIAN SEJAWAT SEBIDANG ATAU PEER REVIEW

KARYA ILMIAH : PROSIDING INTERNASIONAL

"Bayes Estimation for ARMA Model Forecasting Under Normal-Gamma Prior"

Zul Amry and, Adam Baharum

l-l

Prosiding Forum llmiah Nasional

M.S 986011001

Medan,

September 2016

Reviewer 1,

Prof. Dr. Tulus, M.Si

NrP. 19620901 198803 1002

Unit Kerja : Guru Besar FMIPA USU Hasil Penilaia n Peer Review :

Komponen

Yang Dinilai

Nilai Maksimal Prosiding

Nilai Akhir Yang

Diperoleh lnternasional

{

Nasaional

E

d. Kelengkapan unsur isi artikel (10%)

&f

0,f

b. Ruang lingkup dan kedalaman pembahasan (30%)

2,7

2,f

c. Kecukupan dan kemutahiran data/informasi dan

metodoloei (30%)

7,V

2

d. Kelengkapan unsur dan kualitas penerbit (30%)

2,7

2,f

Totat =

(lOO%l

f

Nilai Pengusul

/,d

\'\

ri

JJ

Judul Makalah

Penulis Makalah

ldentitas Makalah

Kategori Publikasi Makalah {beri

/pada

kategori yang tepat)

Mengetahui:

,E€kanF,KlP N Medan,

NtP, 19551-0251985031002

a. Judul Prosiding

b. ISBN

c. Tahun Terbit d. Penerbit e. Jumlah halaman

f.

WEB Laman

{

Prosiding Forum llmiah lnternasional

|_l

Prosiding Forum llmiah Nasional

Lampiran 8

lnternational Seminar on Operational Research (lnteriOR 2015)

August 201.5

USU Medan TEMBAR

HASIL PENILAIAN SEJAWAT SEBIDANG ATAU PEER REVIEW

KARYA ILMIAH : PROSIDING INTERNASIONAL

"Bayes Estimation for ARMA Model Forecasting Under Normal-Gamma Prior"

Zul Amry and, Adam Baharum

Medan,

September 2016

Reviewer 2,

Dr. Firmansyah, M.Si

NrP. 19671110 199303 1003

Unit Kerja: Univ. Muslim Nusantara Hasil Penilaia n Peer Review :

Komponen

Yang Dinilai

Nilai Maksimal Prosiding

Nilai Akhir Yang

Diperoleh lnternasional

./

Nasaional

tr

a. Kelengkapan unsur isi artikel (10%)

0,9

a(

b. Ruang lingkup dan kedalaman pembahasan (30%)

L+

2,t

c. Kecukupan dan kemutahiran data/informasi dan

metodoloei(30%)

AT

248

d. Kelengkapan unsur dan kualitas penerbit (30%)

2,+

v{

Total =

(LOO%I

I

Nilai Pengusul

8

LEIVIBAR

HASIL PENILAIAN SEJAWAT SEBIDANG ATAU PTER REVIEW KARYA ILMIAH : PROSIDING INTERNASIONAL

Lampiran

I

lnternational Seminar on Operational Research

(lnteriOR 2015i

August 2015 USU Medan Judul Makalah

Penulis Makalah

ldentitas Makalah

Kategori Publikasi Makalah (beri _"'pada kategori yang tepat)

Hasil Penilaian Peer Review :

"Bayes Estirnation for ARMA Model Forecasting Under Normal-Gamma

Prior"

Zul Amry and, Adam

a. Judul Prosiding

b" tsBN

c. Tahun Terbit d. Fenerbit

e. Jumlah halaman

f"

WEB Laman

Baharum

./

Prosiding Forum llmiah lnternasional

|_l

Prosiding Forum llmiah Nasional

Nasaional

n

September 2016

3

Nilai Akhir Yang

Diperoleh

\f

L,f

Ltl

L

Medan, Reviewe

NtP. 19570804 198503 1002

Unit Kerja : Guru Besar FMIPA UNIMED

Komponen Yang Dinilai

;:lK"l""s@

Nilai Maks

l"t"r*si"*a

./

otl

b. Ruang lingkup dan kedalaman pembahasan (30%)

g,T

a,T

L,T

L

c. Kecukupan dan kemutahiran data/informasi dan

metodoloei (30%)

d. Kelengkapan unsur dan kualitas penerbit (30%)

Totat =

(LOO%I