Lecture-09: Limit Theorems

Lecture-09: Limit Theorems

1 Growth of renewal counting processes

Lemma 1.1. Consider the counting process N:Ω→_Z^R₊⁺ associated withi.i.d.inter-renewal time sequence X:Ω→_R^N₊ with finite meanEXn<_{∞. Let N∞}≜limt→∞Nt, then P{N_∞=_∞}=1.

Proof. It suffices to showP{N_∞<_∞}=0. SinceE[Xn]<_{∞, we have}P{Xn=_∞}=0 and P{N_∞<_∞}=P ^[

n∈N

{N_∞<n}=P ^[

n∈N

{Sn=_∞}=P ^[

n∈N

{Xn=_∞}⩽

∑

n∈N

P{Xn=_∞}=0.

Corollary 1.2. For delayed renewal processes with finite mean of first renewal instant and subsequent inter- renewal times, P

limt→∞N_t^D=_∞ =1.

1.1 Strong law for renewal processes

We observed that the number of renewalsNtincreases to infinity with the length of the durationt. We will show that the growth ofNtis asymptotically linear with timet, and we will find this coefficient of linear growth ofNtwith timet.

Theorem 1.3 (Strong law). For a renewal counting process N:Ω →_Z^R₊⁺ with i.i.d. inter-renewal times X:Ω→_R^N₊ having a finite meanµ, we have

t→lim∞

Nt

t = ¹

µalmost surely.

Proof. Note thatSN_t represents the time of last renewal beforet, andSN_t+1represents the time of first renewal after timet. Clearly, we haveSN_t⩽t<SN_t+1. Dividing byNt, we get

SN_t

Nt ⩽ ^t Nt

< ^SNt+1

Nt . (1)

SinceNtincreases monotonically to infinity astgrows large, we can apply strong law of large numbers to the sumSN_t=_∑^N_i=1^t Xi, to get limt→∞S_Nt

N_t =µalmost surely. Hence the result follows.

Corollary 1.4. For a delayed renewal process with finite inter-arrival durations,limt→∞N_D(t) t = _µ¹

F.

Example 1.5. Suppose, you are in a casino with infinitely many games. We assume thatX:Ω→ [0, 1]^N is ani.i.d.uniform sequence whereX_iis the random probability of win in the gamei∈_N.

One can continue to play a game or switch to another one. We are interested in a strategy that maximizes the long-run proportion of wins. LetN(n)denote the number of losses innplays. Then the fraction of winsP_W(n)is given byP_W(n) =^n−N(n)_n .

We pick a strategy where any game is selected to play, and continue to be played till the first loss. We show that limn→∞PW(n) =1 for this proposed strategy. LetT_ibe the number of times a gameiis played. We observe that the conditional probability mass function for the number of plays for each gameiis geometrically distributed as

E[_1{Ti=k}|σ(Xi)] =X_i^k−1(1−Xi), k∈_N.

t Nt

S1 S2 SNs s SNs+1

1 2 3 4

Figure 1: Time of last renewal

Hence, it follows thatTiarei.i.d.random variables with meanETi=E[E[Ti|Xi]] =Eh

1 1−X_i

i

=∞.

It follows that each loss is a renewal event, and from the strong law of renewal process, we obtain

n→lim∞

N(n)

n = ¹

E[Time till first loss] = ¹ ETi

=0.

1.2 Wald’s lemma for renewal processes

Basic renewal theorem implies ^N_t^t converges to ¹_µalmost surely. We are next interested in convergence of the ratio ^m_t^t. Note that this is not obvious, since almost sure convergence doesn’t imply convergence in mean. To illustrate this, we have the following example.

Example 1.6. Consider a Bernoulli random sequenceX:Ω→ {0, 1}^Nwith probabilityP{Xn=1}=

1

n, and another random sequenceY:Ω→Z^N₊ defined asYn≜nXn forn∈N. Then,P{Yn=0}= 1−¹_n. That isYn→0 a.s. However,E[Yn] =1 for alln∈_{N. SoE}[Yn]→1.

Even though, basic renewal theorem doesNOTimply it, we still have ^m_t^t converging to _µ¹. We first need this technical Lemma.

Proposition 1.7 (Wald’s Lemma for renewal process). Let m:R+→R+ be the renewal function for a renewal counting process N:Ω→Z^R₊⁺ with i.i.d.inter-arrival times X:Ω→R^N₊ having finite meanµ= E[X1]<∞. Then, Nt+1is a stopping time for the sequence X, and

E

"_N

t+1 i=1

∑

X_i

#

=µ(₁+mt)_.

Proof. We observe that for anyn∈N, the event{Nt+1=n}belongs toσ(X₁, . . . ,Xn), since {Nt+1=n}={Sn−1⩽t<Sn}=

(_n−1

i=1

∑

X_i⩽t<

n−1

∑

i=1

X_i+Xn

)

∈σ(X₁, . . . ,Xn).

Thus Nt+1 is a stopping time with respect to the random sequenceX, and the result follows from Wald’s Lemma.

Theorem 1.8 (Elementary renewal theorem). For a renewal process with finite mean inter-arrival times, the renewal function satisfies

m 1

Proof. By the assumption, we have meanµ<∞. Further, we know thatS_Nt+1>t. Taking expectations on both sides and using Proposition 1.7, we haveµ(mt+1)>t. Dividing both sides byµtand taking lim inf on both sides, we get

lim inf

t→∞

mt

t ⩾ ¹ µ.

We employ a truncated random variable argument to show the reverse inequality. We define trun- cated inter-renewal times ¯X:Ω→[0,M]^Ndefined as ¯Xn≜Xn∧Mfor eachn∈N, and with common mean denoted byµM. SinceXisi.i.d., so is the truncated sequence ¯X, and hence we can define the corresponding renewal sequence ¯S:Ω→R^N₊ and the counting process ¯N:Ω→Z^N₊ defined as

S¯n≜

∑

n i=1

X¯_i, n∈N, and ¯Nt≜

∑

n∈N

1{S^¯n⩽t}^{, t}∈R+.

Note that sinceSn⩾S^¯n, the number of arrivals would be higher for renewal process ¯Ntwith truncated random variables. That is,Nt⩽N^¯t, and hencemt⩽m¯tfrom the monotonicity of expectation. Further, due to truncation of inter-arrival time, next renewal happens within Munits of time, that is ¯SN¯t+1⩽ t+M. From the monotonicity of expectation and Wald’s Lemma for renewal processes, we get

(1+m¯t)µM⩽t+M.

Dividing both sides bytµMand the fact thatmt⩽m¯tfor all timest∈R+, we obtain lim sup

t→∞

mt

t ⩽lim sup

t→∞

¯ mt

t ⩽ ¹ µ_M. The result follows from recognizing that limM→∞µM=µ.

Corollary 1.9. For a delayed renewal process with finite inter-arrival durations, we havelimt→∞m_D(t)

t = ¹

µF.

Example 1.10 (Markov chain). Consider a positive recurrent discrete time Markov chainX:Ω→ X^N taking values in a discrete setX⊂R. Let the initial state be X₀=x∈_Xand τ_y⁺(0) =0 for y̸=x∈X, then we can inductively define thenth recurrent time to stateyas a stopping time

τ_y⁺(n) =infn

k>τ_y⁺(n−1):X_k=yo .

Since any discrete time Markov chain satisfies the strong Markov property, it follows thatτ_y⁺:Ω→ N^Nform a delayed renewal process with the first arrival distributionPx

n

τ_y⁺(1) =ko

= fxy^(k), and the common distribution of the inter-arrival durationXn,n⩾2 in terms of first return probability as

Py

n

τ_y⁺(1) =ko

=fyy^(k),k∈_N.

We denote the associated counting process by Ny:Ω→Z^N₊, where Ny(n) =_∑i∈N1{τy⁺(i)⩽n}=

∑ⁿ_k=11{X_k=y}denotes the number of visits to stateyup to timen. Letµyy=Eyτ_y⁺(1)be the finite mean inter-arrival time for the renewal process, also the mean recurrence time to statey. From the strong law for delayed renewal processes it follows that

Py

n∈limN

Ny(n)

n = ¹

µyy

=1.

Since Ny(n) is number of visits to state y in first n time steps, we have ExNy(n) =

∑ⁿ_k=1Px{Xk=y}=_∑ⁿ_k=1p^(k)xy From the basic renewal theorem for delayed renewal process it fol- lows that

n∈limN

∑ⁿ_k=1p^(k)_xy

n = lim

n∈N

Ex[Ny(n)]

n = ¹

µyy.

1.3 Central limit theorem for renewal processes

Theorem 1.11. For a renewal process with inter-arrival times having finite meanµand finite varianceσ², the associated counting process converges to a normal random variable in distribution. Specifically,

t→lim∞P





 Nt−_µ^t

σ q _t

µ³

<y







= √¹ 2π

Z y

−∞e^−x

2 2 dx.

Proof. Takeu= _µ^t +yσq _t

µ³. We shall treatuas an integer and proceed, the proof for generaluis an exercise. Recall that{Nt<u}={Su>t}. By equating probability measures on both sides, we get

P{Nt<u}=P

Su−uµ σ

√u > ^t−uµ σ

√u

=P

(Su−uµ σ

√u >−y

1+√^yσ tµ

−1/2) . By central limit theorem, ^Su−uµ

σ

√u converges to a normal random variable with zero mean and unit variance astgrows. We also observe that

t→lim∞−y

1+√^yσ tu

−1/2

=−y.

These results combine with the symmetry of normal random variable to give us the result.

2 Patterns

LetX:Ω→X^Nbe ani.i.d.sequence with common probability mass function p∈ M(X). We denote the natural filtration of process X byF•≜(Fn:n∈N)whereFn ≜^σ(X1, . . . ,Xn)for all n∈N. Let x= (x1, . . . ,xm)∈X^mbe a pattern and inductively definenth hitting times of the patternxasS₀^x≜0 and

S^x_n≜inf

n>S^x_n−1:Xn=xm,Xn−1=xm−1, . . . ,Xn−m+1=x1 .

It is easy to check that S_n^x is adapted toF• and one can verify that S_n^x is almost surely finite for all n∈N. It follows thatS^xis a sequence of stopping times adapted toF•. SinceXisi.i.d., it follows that S^x:Ω→R^N₊ is a delayed renewal sequence with inter-renewal durationsT_n^x≜S^x_n−S_n−1^x beingi.i.d.for n⩾2 and independent ofT₁^x.

2.1 Hitting time to pattern ( 1 )

First we consider the simplest example when the alphabetX={0, 1}, with the common meanEX1=p, and the patternx= (1). One way to solve this problem is to considerS¹₁as a random variable and find its distribution. We can write

Pn

S¹₁=ko

=p¯^k−1p.

We observe thatS¹₁ is a geometric random variable of the time to first success, with its mean as the reciprocal ofi.i.d. success probabilityp. An alternative way to solve this is via renewal function ap- proach. Recall that

S¹₁=1 ={X1=1}andS¹₁1_{X1=0}= (1+S₁¹)1_{X0=0}in distribution whereS¹₁is independent ofX0. The result follows from writing

ES¹₁=ES¹₁1{^S1₁^>1}+ES₁¹1{^S1₁⁼¹}=pE¯ (1+S¹₁) +p=1+pES¯ ¹₁.

2.2 Hitting time to pattern ( 0, 1 )

For the alphabet X={0, 1} with common meanEX1= p, we consider the two length pattern x = (0, 1), then S₁^x=inf{n∈_N:Xn=1,Xn−1=0}. We can again model this hitting time as a random variable, however directly finding the distribution ofS^xis slightly more complicated. We next attempt the renewal function approach. Recall thatS^x₁ is independent ofX₀, and the following equality holds in distributionS^x₁1{X₁=1}= (1+S^x₁)_1{X0=1}. In addition, the following equality holds in distribution S^x₁1{X₁=0,X₂=0}= (1+S₁^x)_1{X1=0,X0=0}. Hence, we can write

We recognize that the second term on the right hand side can be written as

ES₁^x1{X₂=0,X₁=0}=pE¯ (1+S^x₁)1{X₁=0}=p¯²+pES¯ ₁^x1{X₁=0}=p¯²+pES¯ ^x₁−ppE¯ (1+S₁^x). Combining the above two results, we can write

ES₁^x=2pp¯+p¯²+pES¯ ^x₁+p²E(1+S₁^x) =1+ (p¯+p²)_ES1^x. It follows thatES^x₁= _p¹_p¯.

2.3 Hitting time to pattern x

For ani.i.d.sequenceX:Ω→_X^N, a general approach is to modelX^m_n = (Xn,Xn−1,Xn−m+1)∈_X^mas an m-dimensional time homogeneous irreducible positive recurrent Markov chain. DefiningY≜X^mWe are interested in the mean hitting time to statex of the joint processX^m:Ω→_Y^N. It follows that the successive times for processX^mto hit a patternx∈_Yis a delayed renewal process in general. Defining the on times when X^m hits x, it follows from the strong law for renewal processes that the average number of visits to statexis the reciprocal of mean inter-renewal duration. That is,

n→lim∞

1 N

∑

N n=1

1{X^m_n=x}= lim

n→∞

1 N

∑

N n=1

1{X_n=x_m,...,X_n−m+1=x₁}=

∏

m i=1

px_i= ¹ ET_k^x.

For each patternx∈Y, we define initial sub-patternsx^k≜(x1, . . . ,xk)fork∈[m]. If the initial sub-pattern is not one of the final sub-patterns, i.e. (x₁, . . . ,x_k)̸= (x_m−k+1, . . . ,xm)_{for any}k∈[m], then we observe thatS^xis a renewal sequence andES₁^x= _∏m¹

i=1p_xi.

Example 2.1. Consider patterns(1)and(01)fori.i.d.Bernoulli sequenceX:Ω→ {0, 1}^Nwith common meanEX1=p. Clearly,(1)has no sub-pattern and henceES¹₁=¹_p. Similarly,(01)has a sub-pattern(0) but(₀₀)is not a sub-pattern of(₀₁)_{and henceES}⁰¹₁ = _pp¯¹_.

If there exists a non empty I⊆[m]such that for each k∈I there is an initial sub-patternx^k such that(x^k) = (x^m_m−k+1)is a final sub-pattern ofx, then the mean hitting time to patternxis equal to the telescopic sum of mean hitting time to sub-patterns That is, denotingI≜{i₁, . . . ,i_k}, we can write

ES^x₁=

∑

k j=1

E_xijS^x₁^ij+1.

The mean time duration between two successive hits tox^ij+1isET₂^xij+1= ¹

∏^ijℓ=⁺1¹pxℓ

. This is the same mean time fromx^ij tox^ij+1. Therefore,

ES^x₁=

∑

k j=1

1

∏ⁱ_ℓ=1^j+1 pxℓ

.

Example 2.2. Consider pattern (101) and (1011) for i.i.d. Bernoulli sequence X:Ω → {0, 1}^N with common meanEX1=p. Pattern(101)has sub-patterns(1)and(10), where(1)appears at the end as well. Therefore,

ES⁽¹⁰¹⁾₁ =ES¹₁+E1S¹⁰¹₁ = ¹ p+ ¹

p²p¯.

Pattern(1011)has sub-patterns(1),(10),(101), where(1)appears at the end. Thus, Therefore, ES₁⁽¹⁰¹⁾=ES¹₁+E1S¹⁰¹¹₁ = ¹

p+ ¹ p³p¯.