Lecture-25: Random walks
1 GI/GI/1 Queueing Model
Consider aGI/GI/1 queue. Customers arrive in accordance with a renewal process having an arbitrary interarrival distributionF, and the service distributionG.
Proposition 1.1. Let Dn be the delay in the queue of the nth customer in a GI/GI/1 queue with independent inter-arrival times Xnand service times Yn. For a random walk Snwith steps Un=Yn−Xn+1for all n∈N,
P{Dn+1>c}=P{Sj>c, for some j∈[n]}. (1) Proof. The following recursion forDnis easy to verifyDn+1= (Dn+Yn−Xn+1)1{Dn+Yn−Xn+1>0}=max{0,Dn+ Un}. Iterating the above relation withD1=0 yields
Dn+1=max{0,Un+max{0,Dn−1+Un−1}}=max{0,Un,Un+Un−1+Dn−1}.
For the random walkSnwith stepsUn, we can write delay in terms of random walk SnasDn+1=max{0,Sn−
Sn−1,Sn−Sn−2, . . . ,Sn−S0}. Using the duality principle, we can rewrite the following equality in distribution
Dn+1=max{0,S1, . . . ,Sn}.
Corollary 1.2. IfEUn>0, then for all c∈R, we have P{D∞>c},limn∈NP{Dn>c}=1.
Proof. It follows from Proposition 1.1 thatP{Dn+1>c}is nondecreasing inn. Hence, by MCT the limit exists and is denoted byP{D∞>c}=limn∈NP{Dn>c}. Therefore, by continuity of probability, we have from (1), that
P{D∞>c}=P{Sn>cfor somen}. (2)
IfEUn=EYn−EXn+1is positive, then by strong law of large numbers the random walkSnwill converge to positive infinity with probability 1. The above will also be true whenE[Un] =0, then the random walk is recurrent.
Remark1. Hence, we get thatEYn<EXn+1implies the existence of a stationary distribution.
Proposition 1.3 (Spitzer’s Identity). Let Mn=max{0,S1,S2, . . . ,Sn}for n∈N, thenEMn=∑nk=11kES+k. Proof. We can writeMn=1{Sn>0}Mn+1{Sn60}Mn. We can rewrite first term in decomposition as,
1{Sn>0}Mn=1{Sn>0}max
i∈[n]Si=1{Sn>0}(X1+max{0,S2−S1, . . . ,Sn−S1})
Hence, taking expectation and using exchangeability, we get E[Mn1{Sn>0}] =E[X11{Sn>0}] +E[Mn−11{Sn>0}].
SinceXi,Snhas the same joint distribution for alli, we haveES+n =E[Sn1{Sn>0}] =E∑ni=1Xi1{Sn>0}] =nE[X11{Sn>0}].
Therefore, it follows thatE[1{Sn>0}Mn] =E[1{Sn>0}Mn−1] +1nE[S+n]. FurtherSn60 implies thatMn=Mn−1, and hence 1{Sn60}Mn=1{Sn60}Mn−1. Thus, we obtain the following recursion,
E[Mn] =E[Mn−1] +1 nE[S+n].
Result follow from the fact thatM1=S+1.
Remark2. SinceDn+1=Mn, we haveE[Dn+1] =E[Mn] =∑nk=11kE[S+k].
1
2 Martingales for Random Walks
Proposition 2.1. A random walk Snwith step size Xn∈ {−M,M}for some M∈Zis a recurrent DTMC iffEX=0.
Proof. IfEX6=0, the random walk is clearly transient since, it will diverge to±∞depending on the sign ofEX.
Conversely, ifEX=0, thenSnis a martingale. Assume that the random walk starts at stateS0=i. We define A={−M,−M+1,· · ·,−2,−1}, Aj={j+1, . . . ,j+M}, j>i.
LetNdenote the hitting time toAorAjby random walkSn. SinceNis a stopping time andSN∧n6|M|+j, from optional stopping theorem we haveEi[SN] =Ei[S0] =i. Thus we have
i=Ei[SN]>−MPi{SN∈A}+j(1−Pi{SN∈A}).
Rearranging this, we get a bound on probability of random walkSnhittingAoverAjas
Pi{Sn∈Afor somen}>Pi{SN ∈A}> j−i j+M.
Taking limit j→∞, we see that for any i>0, we have Pi{Sn∈Afor somen}=1. Similarly takingB= {1,2,· · ·,M}, we can show thatPi{Sn∈Bfor somen}=1 for anyi60. Result follows from combining the above two arguments to see thatPi{Sn∈A∪Bfor somen}=1 for anyi∈Z.
Proposition 2.2. Consider a random walk Sn with mean step sizeE[X]6=0. For a,b>0, let Pa denote the probability that the walk hits a value greater than a before it hits a value less than−b. Then, forθ6=0such that EeθX1=1, we have
Pa≈ 1−e−θb eθa−e−θb.
Approximation is an equality when step size is unity and a and b are integer valued.
Proof. For anya,b>0, we can define stopping times
Ta=inf{n∈N:Sn>a}, T−b=inf{n∈N:Sn6−b}.
We are interested in computing the probabilityPa=P{Ta<T−b}. Now letZn=eθSn. We can see thatZnis a martingale with mean 1. Define a stopping timeN=Ta∧T−b. From Doob’s Theorem,E[eSN] =1. Thus we get
1=E[eθSN|SN>a]Pa+E[eθSN|SN6−b](1−Pa).
The result follows from obtaining an approximation forPaby neglecting the overshoots pastaor−b. Thus we get E[eθSN|SN >a]≈eθa, E[eθSN|SN6−b]≈e−θb.
2
