Directory UMM :Data Elmu:jurnal:M:Mathematical Biosciences:Vol168.Issue2.Dec2000:

A note on integrals for birth±death processes

Valeri T. Stefanov

*

_{, Song Wang}

Department of Mathematics and Statistics, The University of Western Australia, Nedlands, WA 6907, Australia

Received 21 December 1999; received in revised form 21 August 2000; accepted 30 August 2000

Abstract

A general integral for birth±death Markov processes is considered. A closed form expression is provided for its expected value in terms of the birth and death rates. A simple route for numerical evaluation of its variance is also suggested. Ó _{2000 Elsevier Science Inc. All rights reserved.}

Keywords:Birth±death process; Hitting time; Integral of Markov process

Hernandez-Suarez and Castillo-Chavez [1] discusses a methodology for evaluating the

expec-tation of the integral under the stochastic path of a birth±death process up to extinction time and the expected time to extinction. Their main result is a closed form expression for the expectation of the aforementioned integral. In this note we demonstrate a more general result, which embraces and substantially extends that in Ref. [1], ¯ows from Stefanov's [2] results on ®nite-state birth± death processes. The relevance of such results to biological sciences is spelled out in Ref. [1] (see also the references therein). Furthermore, we provide a simple route for numerical evaluation of the variance of this integral together with a code for implementing such evaluation. In what

follows, we will use similar notation to that adopted in Ref. [1], letting X…t† be a birth±death

Markov process on the state spacef0;1;_{. . .};g, with birth and death rateski andli, respectively

for statei. Further, denote byZ…k† the waiting time till reaching state 0 from statek, that is

Z…k† ˆinfft>0_:X…t† ˆ0jX…0† ˆkg

and de®ne

Yf…k† ˆ Z Z…k†

0

f…X…t††dt;

*_{Corresponding author. Tel.: +61-8 9380 1870; fax: +61-8 9380 1028.}

E-mail address:stefanov@maths.uwa.edu.au (V.T. Stefanov).

0025-5564/00/$ - see front matter Ó _{2000 Elsevier Science Inc. All rights reserved.}

wherefis a non-negative real function on the state space of the chainX; we assume additionally

that there exists a positive real numberx0, such thatfis a non-decreasing function forxPx0. Note

that iff…x† 1, thenYf…k† ˆZ…k†, and iff…x† ˆx, thenYf…k†is the integral considered in Ref. [1].

Denote

Hj…i† ˆ

kjkj‡1. . .kiÿ1

l_j‡1l_{j‡2. . .}l_i; i>jP1

with the conventionHi…i† ˆ1; iˆ1;2;. . ., and Hj…i† ˆ0 ifi<j. Our result is stated in the

fol-lowing.

Proposition.The expected value ofYf…k†has the following closed form expression in terms of theki

andl_i:

E…Yf…k†† ˆ Xk

jˆ1 1 l_j

X1 iˆj

f…i†Hj…i† !

: …1†

Note that in the particular case whenk ˆ1 andf…x† ˆx the aforementioned formula reduces

to the main result in Ref. [1] (see (5) on p. 98 of Ref. [1]; there is a conspicuous misprint in (5)).

Also iff…x† ˆ1 the formula reduces to a well-known result (see (3.58) on p. 67 of Ref. [3]).

Proof of Proposition. LetSi…k† be the total time spent in statei within the random time interval

‰0;Z…k†Š. Clearly

Yf…k† ˆ X1

iˆ1

f…i†Si…k†:

From the monotone convergence theorem we have

E…Yf…k†† ˆlimE Xm

iˆ1

f…i†Si…k† !

…2†

asm! ‡1. Denote byZ…m†_{k†, Y}…m†

f …k†andS

…m†

i …k†,…mPk†, the counterparts ofZ…k†,Yf…k†and

Si…k† in a …m‡1†-state birth±death process with birth and death rates ki andli, respectively for

each statei,iˆ0;1;. . .;m;…m<‡1; kmˆ0†. It is easy to see that for largem(mPx0, wherex0

has been introduced above)

E…Yf…k††PE…Y… m†

f …k††PE Xm

iˆ1

f…i†Si…k† !

: …3†

In view of the results in Ref. [2] (Theorem 2, cf. the second and fourth identities in Theorem 1) we have

E…Z…m†_{k†† ˆ}Xk jˆ1

1 l_j

Xm

iˆj

Hj…i† !

ˆX

k

jˆ1 1 l_j

X1 iˆj

Hj…i† !

where the second equality is due tokmˆ0. Moreover, as a byproduct from the results in Ref. [2]

To see this, start with the last few lines on p. 848 and derive the identities, corresponding to those

at the bottom of p. 847, for the stopping time sn;nÿ1 that has been introduced in Ref. [2]. Thus,

(1) translates into the following closed form expressions for the four models considered in Ref. [1]

(qˆk=l<1 and the conventionPÿ1

(note that there is a misprint in the expression forY…k†provided in Ref. [1]),

(b)ki ˆk; liˆil(M=M=1 queue),

(note that the suggested expression forEY…k†in [1] is incorrect),

It seems that a neat closed form expression for the variance of the random quantity Yf…k† is not

derivable. However, the tools from Ref. [2] provide a simple route for numerical evaluation of this

variance. First note that, in view of our assumptions for the functionf…†and the fact that the Si

are non-negative, (2) and (3) remain valid if we replace `E' by `Var'. Therefore,

Var…Yf…k†† ˆlim

number of one-step transitions from state i to state j within the time interval ‰0;Z…m†_{k†Š (recall}

that kmˆ0). Observe that the following system of linear equations hold (m and k have been

suppressed for the sake of brevity):

Var…Ni;i‡1†

with the convention N_0…;m₁†…k† ˆ0. These equations are derived after di€erentiating twice, with

respect to theki andli, the continuous-time counterpart of the integral identity on p. 847 of Ref.

[2], bearing in mind that the di€erentiation and integration signs can be interchanged, the easily noticeable relations

N₁…m_;0†…k† ˆ1;

N_i…;m_i‡†₁…k† ˆN_i…‡m₁†_;i…k† ÿ1; iˆ1;_{. . .};kÿ1; N_i…;m_i‡†₁…k† ˆN_i…‡m₁†_;i…k†; iˆk;_{. . .};mÿ1;

The expected values of theN_i…;m_i‡†₁…k†are known coecients, which are derived from (4) and (7). The solution of system (6) produces the Cov…S_i…m†…k†;S_j…m†…k††and subsequently the variance of Yf…k†.

Note that system (6) is sparse (up to four unknown variables per equation), and can therefore be solved numerically even when the order of the associated matrix is very large. Since the system matrix is not diagonally dominant and it is not clear whether it is positive-de®nite or not, iterative algorithms, such as those based on preconditioned conjugate gradient methods, are not appli-cable. Therefore, we use the LU decomposition for the solution of (6). There are a number of FORTRAN 77/90 and C codes for solving sparse linear systems based on LU decomposition.

Such are also available on the Web (cf.http://www.netlib.org). For the example provided

below we choose a reliable FORTRAN 77 sparse linear system solver, Harwell MA28, which is

freely available for non-commercial use (cf.http://www.netlib.org/harwell/). We have

written a Fortran 77 code for assembling the relevant matrix and the right-hand side vector in (6),

and have made it available at http://www.maths.uwa.edu.au/stefanov. Numerical

results for the SIS model, with di€erent values of the parameters, are provided in Table 1.

References

[1] C.M. Hernandez-Suarez, C. Castillo-Chavez, A basic result on the integral for birth±death Markov processes, Math. Biosci. 161 (1999) 95.

[2] V.T. Stefanov, Mean passage times for tridiagonal transition matrices, J. Appl. Prob. 32 (1995) 846.

[3] E. Renshaw, Modelling Biological Populations in Space and Time, Cambridge University, Cambridge, UK, 1991. Table 1

E…Y…k††and 

Var…Y…k††

p

for di€erent values of the parameters in the SIS model

Initial number of infectives

kˆ0:8;lˆ1 kˆ1:2;lˆ1

k Expected cost Standard deviation of cost Expected cost Standard deviation of cost

Nˆ150

1 4.4948 11.8384 263.5898 928.6760

2 8.8927 16.4846 483.8832 1216.2008

5 21.5417 24.9267 958.9993 1577.8288

10 41.0037 32.9079 1394.4095 1743.6087

Nˆ200

1 4.5989 12.4200 672.1968 2400.7958

2 9.1202 17.3503 1234.3382 3145.9029

5 22.2418 26.4731 2443.6815 4083.3338

10 42.7658 35.4092 3533.4096 4508.8569

Nˆ250

1 4.6670 12.8167 1650.1987 5888.8914

2 9.2691 17.9419 3030.0504 7714.3959

5 22.7031 27.5379 5990.3266 10 000.3876

10 43.9425 37.1583 8620.6198 11 018.9165

Nˆ300

1 4.7151 13.1057 3960.3590 14 044.2136

2 9.3745 18.3735 7270.8599 18 387.0272

5 23.0308 28.3188 14 355.8171 23 791.1018