Lecture-16: Weak convergence of random variables

(1)

Lecture-16: Weak convergence of random variables

1 Convergence in distribution

Definition 1.1. A sequence (Xn:n∈N)of random variablesconverges in distributionto a random vari- ableXif

limn FXn(x) =FX(x)

at all continuity pointsxofFX. Convergence in distribution is denoted by limnXn=Xin distribution.

Proposition 1.2. Let (Xn :n∈N)be a sequence of random variables and let X be a random variable. Then the following are equivalent:

(a) limnXn=X in distribution.

(b) limnE[g(Xn)] =_E[g(X)]for any bounded continuous function g.

(c) Characteristic functions converge point-wise, i.e.limnΦX_n(u)→_Φ_X(u)for each u∈R.

Proof. Let(Xn:n∈_N)be a sequence of random variables and letXbe a random variable. We will show that(a) =⇒ (b) =⇒ (c) =⇒ (a).

(a) =⇒ (b): Let limnXn=Xin distribution, then limnR

x∈Rg(x)dFXn(x) =R

x∈Rg(x)limndFXn(x)by the bounded convergence theorem for any bounded continuous functiong.

(b) =⇒ (c): Let limnE[g(Xn)] =E[g(X)]for any bounded continuous functiong. Taking g(x) =e^jux, we get the result.

(c) =⇒ (a): The proof of this part is technical and is omitted.

Example 1.3 (Convergence in distribution but not in probability). Consider a sequence of non- degenerate continuousi.i.d. random variables(Xn:n∈N)and independent random variableXwith the common distribution F_X. Then F_X_n =F_X for all n∈N, and hence limnXn =X in distribution.

However, for anyn∈_Nande>0, from the monotonicity of distribution function, we have P{|Xn−X|>e}=_E1_{X_n∈[X−e,X+e]}_/ =_EF_X(X+e)−_EF_X(X−e)>0.

Lemma 1.4 (Convergence in probability implies in distribution). Consider a sequence (Xn:n∈_N)of random variables and a random variable X, such thatlimnXn=X in probability, thenlimnXn=X in distribution.

Proof. Fixe>0, and consider the eventEn,{ω∈_Ω:|Xn−X|(ω)>e}={Xn∈/[X−e,X+e]} ∈_{F. We} further define eventsAn(x),{Xn6x}andA(x),{X6x}, then we can write

An(x)∩A(x+e)⊆A(x+e), An(x)∩A^c(x+e)⊆En, A(x−e)∩An(x)⊆An(x), A(x−e)∩A^c_n(x)⊆En.

1

(2)

From the above set relations, law of total probability, and union bound, we have F(x−e)−P(En)6Fn(x)6F(x+e) +P(En). From the convergence in probability, we have limnP(En) =0 get

F(x−e)6lim inf

n Fn(x)6lim sup

n Fn(x)6F(x+e). We get the result at the continuity points ofFX, since the choice ofewas arbitrary.

Theorem 1.5 (Central Limit Theorem). Consider ani.i.d. random sequence(Xn:n∈_N)withEXn =µand Var(Xn) =σ², defined on the probability space (_Ω,F,P). We define the n-sum as Sn =_∑ⁿ_i=1X_i and consider a standard normal random variable Y with density function f_Y(y) =^√¹

2πe⁻^y

2

2 for all y∈_{R. Then,} limn

Sn−nµ σ

√n =Y in distribution.

Proof. The classical proof is using the characteristic functions. LetZ_i,^Xⁱ_σ⁻^µfor alli∈N, then the shifted and scaledn-sum^Sⁿ^−nµ

σ

√n =_∑ⁿ_i=1ZiWe use the third equivalence in Proposition 1.2 to show that the characteristic function of converges to the characteristic function of the standard normal. We define the characteristic functions

Φn(u),Eexp

ju(Sn−nµ) σ

√n

, ΦZ_i(u),Eexp(juZ_i), ΦY(u),Eexp(juY). We can compute the characteristic function of the standard normal as

ΦY(u) =√¹ 2π

Z

y∈Re⁻^u

2

2 exp

−(y−ju)² 2

dy=e⁻^u

2 2 .

Since(Zi:n∈N)is a zero meani.i.d. sequence, and using the Taylor expansion of the characteristic function, we have

Φn(u) =

∏

n i=1

Eexp

ju(Xi−µ) σ

√n

=

ΦZ₁

√u n

n

=

1− ^u

2

2n +o u²

n ⁿ

. For anyu∈R, taking limitn∈N, we get the result.

2 Strong law of large numbers

Definition 2.1. For a random sequence(Xn:n∈_N)with bounded meanE|Xn|<∞, we define then-sum asSn,∑ⁿi=1X_iand the empiricaln-mean ^S_nⁿ for eachn∈N. For eachn∈N, we define event

En,{ω∈_Ω:|Sn−_ES_n|>ne} ∈_F.

Theorem 2.2 (L⁴strong law of large numbers). Let(Xn:n∈_N)be a sequence of pair-wise uncorrelated random variables with bounded meanEXnand uniformly bounded fourth central momentE(Xn−EXn)⁴6B<∞for all n∈N. Then, the empirical n-mean converges tolimnESn

n almost surely.

Proof. From the Markov’s and Minkowski’s inequality, we have P(En)6 ^∑ⁿⁱ⁼¹^E(X_n4e²ⁱ^−µ)⁴ 6 ^B+µ_n3e²⁴. It follows that the∑n∈NP(En)<∞, and hence by Borel Canteli Lemma, we haveP{E^c_nfor all but finitely manyn}= 1. Since, the choice ofewas arbitrary, the result follows.

Theorem 2.3 (L²strong law of large numbers). Let(Xn:n∈_N)be a sequence of pair-wise uncorrelated random variables with meanEXn and uniformly bounded variance Var(Xn)6B<_∞for all n∈N. Then, the empirical n-mean converges tolimnES_n

n almost surely.

2

(3)

Proof. For eachn∈N, we define eventsFn,E_n2, and Gn,

( max

n²6k<(n+1)²

|S_k−S_n2−_E(S_k−S_n2)|>n²e )

= ^[

n²6k<(n+1)²

( ω∈_Ω:

∑

k i=n²

X_i−_EX_i

>n²e )

.

From the Markov’s inequality and union bound, we have P(Fn)6 ^∑

n²

i=1Var(Xi) n⁴e² 6 ^B

n²e², P(Gn)6

∑

2n k=0

kB

n⁴e6 (2n+1)B n³e . Therefore,∑n∈NP(Fn)<_∞_and_∑_n∈_NP(Gn)<∞, and hence by Borel Canteli Lemma, we have

limn

S_n2−ES_n2

n² =lim

n

S_k−S_n2−E(S_k−S_n2)

n² =0 a.s.

The result follows from the fact that for anyk∈N, there existsn∈_Nsuch thatk∈n², . . . ,(n+1)²−1 and hence

|Sk−ESk|

k 6

|S_n2−ES_n2|

n² +|Sk−S_n2−E(Sk−S_n2)|

n²

.

Theorem 2.4 (L¹strong law of large numbers). Let(Xn:n∈N)be a sequence of pair-wise uncorrelated random variables such thatE|Xn|6B<∞for all n∈N. Then, the empirical n-mean converges tolimnESn

n almost surely.

3