Lecture-09: Regenerative Processes

Lecture-09: Regenerative Processes

1 Regenerative processes

Let(Ω,F,P)be a probability space, andS:Ω→R^N+ be a renewal sequence, with the associated inter-renewal sequenceX:Ω→R^N+and the counting processN:Ω→Z^R+⁺. That is, we haveS_n,∑ⁿi=1X_ifor eachn∈Nand N(t) =∑n∈N1{S_n6t}for eacht∈R+.

Consider a stochastic processZ:Ω→R^R+defined over the same probability space, where thenth segment of the joint process(N,Z):Ω→(Z+×R)^R+is defined as the sample path in thenth inter-renewal duration, written

ζn,(X_n,(Z_t:t∈[Sn−1,S_n)), n∈N.

Definition 1.1. The processZis regenerative over the renewal sequenceS, if its segments(ζ_n:n∈N)arei.i.d.

The processZis delayed regenerative, ifSis a delayed renewal sequence and the segments of the joint process are independent with(ζ_n:n>2)being identically distributed.

Remark 1. LetFt=σ(N(u),Z(u),u6t)be the history until timet∈R+. The renewal sequence S is there- generation timesfor the process Z, and the process Z possesses the regenerative property that the process (Z_Sn−1+t:t>0)is independent of historyFSn−1.

Remark2. The definition says that probability law is independent of the past and shift invariant at renewal times.

That is after each renewal instant, the process becomes an independent probabilistic replica of the process starting from zero.

Remark3. If the stochastic processZis bounded, then for any Borel measurable function f:R→R, we have E[f(Z_Sn−1+t)|FSn−1] =Ef(Z_t).

Example 1.2 (Age process). LetN:Ω→R^R+⁺ be the renewal counting process for the renewal sequence (S_n:n∈N), then the age at timet is defined asA(t) =t−S_N(t). Then the age processA:Ω→R^R+⁺ is regenerative. To see this, we observe that the sample path of age innth renewal interval is given by

A(S_n−1+t) =t, t∈[0,X_n).

Since the segments(X_n,(t:t∈[0,X_n)))arei.i.d., the result follows.

Example 1.3 (Markov chains). For a discrete time irreducible and positive recurrent homogeneous Markov chainX:Ω→X^Non finite state spaceX⊂R, we can inductively define the recurrent times for statey∈X asτ_y⁺(0) =0, and

τ_y⁺(n) =inf

k>τ_y⁺(n−1):X_k=y .

From the strong Markov property of Markov chain X, it follows thatτ_y⁺:Ω→N^N is a delayed renewal sequence. For alln∈N, we define thenth excursion time to the stateyasI_n,

τ_y⁺(n−1), . . . ,τ_y⁺(n)−1 and length of this excursion asT_y(n),τ_y⁺(n)−τ_y⁺(n−1).

We show that the Markov processX is regenerative over renewal sequence τ_y⁺. Let thenth excursion time to the state y be denoted by I_n=

τ_y⁺(n−1), . . . ,τ_y⁺(n)−1 , then we can write thenth segment as ζn= (T_y(n),(X_k,k∈I_n)).

Independence of the segments follows from the strong Markov property. Further, in thenth segment of the joint process, we can write the joint distribution for(T_y(n),X_τ+

y(n−1)+k)fork<T_y(n)andx6=yas Pn

T_y(n) =m,X

τy⁺(n−1)+k=x,k∈[m−1]o

=P_y

τ_y⁺(1)>k,X_k=x P_x

τ_y⁺(1) =m−k . The equality follows f rom the strong Markov property and the homogeneity of processX.

1

2 Renewal equation

LetZ:Ω→R^R+ be a regenerative process over renewal sequenceS:Ω→R^N+defined on the probability space (Ω,F,P), andF be the distribution of inter-renewal times. The counting process associated with the renewal sequenceSis denoted byN, and we define the history of the joint processZ,Nuntil timetbyFt. For any Borel measurable setA∈B(R)and timet>0, we are interested in computing time dependent marginal probability

f(t) =P{Z_t∈A}. We can write the probability of the event{Z_t∈A}by partitioning it into disjoint events as P{Z_t∈A}=P{Z_t∈A,S1>t}+P{Z_t∈A,S₁6t}.

We define the kernel functionK(t) =P{S₁>t,Z_t ∈A}which are typically easy to compute for any regenerative process. By the regeneration property applied at renewal instantS₁, we have

E[E[1{Z_t∈A,S₁6t}|σ(S₁)] =E[E[1{Z_t∈A,S₁6t}|FS₁]|σ(S₁)] =1{S₁6t}E[1{Zt−S1∈A}|σ(S₁)] =f(t−S1)1{S₁6t}. Hence, we have the following fixed pointrenewal equationfor f

f(t) =K(t) + Z t

0

dF(s)f(t−s) =K+F∗f.

We assume that the distribution functionF and the kernelKare known, and we wish to find f, and characterize its asymptotic behavior.

Example 2.1 (Age and Excess time processes). For a renewal sequenceS:Ω→R^N+with associated counting processN:Ω→Z^R+⁺, we can define the age processA:Ω→R^R+⁺ where the ageA(t)at timet is the time since last renewal, i.e.

A(t),t−S_N(t),t∈R+.

Similarly, we can define the excess time processY:Ω→R^R+⁺where the excess timeY(t)at timetis the time until next renewal, i.e.

Y(t),S_N(t)+1−t, t∈R+.

Since the age process is regenerative for the associated renewal sequence, we can write the renewal equa- tion for its distribution function as

P{A(t)>x}=P{A(t)>x,S₁>t}+ Z t

0

dF(y)P{A(t−y)>x}.

Theorem 2.2. The renewal equation has a unique solution f = (1+m)∗K, where m(t) =∑n∈NF_n(t) is the renewal function associated with the inter-renewal time distribution F.

Proof. It follows from the renewal equation thatF∗(1+m)∗K=∑n∈NF_n∗K=m∗K. Hence, it is clear that (1+m)∗Kis a solution to the renewal equation. For uniqueness, let fbe another solution, thenh=f−K−m∗K satisfiesh=F∗h, and henceh=F_n∗hfor alln∈N. From finiteness ofm(t), it follows thatF_n(t)→0 asngrows.

Hence, limn∈N(F_n∗h)(t) =0 for eacht.

Proposition 2.3. Let Z be a regenerative process with state spaceX⊂R, over a renewal sequence S with renewal function m. For a Borel measurable set A∈B(R)and the kernel function K(t) =P{Z_t∈A,S₁>t}, we can write for any t>0

P{Z_t∈A}=K(t) + Z _t

0

dm(s)K(t−s). (1)

Example 2.4 (Age process). Since the age process is regenerative for the associated renewal sequence, we can write the kernel functionK(t)in the renewal equation for its distribution function in terms of the comple- mentary distribution function ¯Fof the inter-arrival times, asK(t) =P{A(t)>x,S₁>t}=1_{t>x}F¯(t). From the solution of renewal equation it follows that

P{A(t)>x}=1_{t>x}F(t) +¯ Z t

0

dm(y)1_{t−y>x}F(t¯ −y). (2)

2