A random variable Z is said to be infinitely divisible provided that, for each integern≥1, there are independent and identically distributed random variablesZn,1, . . . , Zn,n whose sum has the same distribution asZ. A prob- ability measure is said to beinfinitely divisible if it is the distribution of an infinitely divisible variable. It follows that the Fourier transformf of such a measure is characterized by the property thatf1/nis the Fourier transform of a probability measure for eachn.
Poisson distributions, Gaussian distributions, and gamma distributions are all infinitely divisible; see Exercise II.5.22 for a proof. It is obvious that the sums of infinitely divisible random variables are infinitely divisible. We shall show later that, in fact, every infinitely divisible random variable is the sum of a Gaussian variable and a limit of linear combinations of independent Poisson variables.
The theorems above have shown that the row sums of the array8.2con- verge in distribution to Gaussian or Poisson variables under certain condi- tions. The following shows that, in general, the limits are infinitely divisible.
8.26 Theorem. Suppose that the array8.2is strictly triangular (that is, kn =n) and that Xn1, . . . , Xnn are independent and identically distributed.
If (Zn) converges in distribution to some variable Z, then Z is infinitely divisible. Conversely, ifZis an infinitely divisible random variable, then there is an array[Xni]whose row sumsZn converge in distribution toZ.
Proof. The converse statement is trivial: ifZ is infinitely divisible, there are independent and identically distributed variablesXnj, j= 1, . . . , n, whose sum has the same distribution asZ; then, we define the array by theseXnj
and the row sumsZn have the same distribution asZ for alln.
Sec. 8 Central Limits 135 Assume that the array is as described and thatZn →Z in distribution;
we need to show thatZ is infinitely divisible. Fixm; we need to show that there are independent and identically distributed variablesY1, . . . , Ymwhose sum has the same distribution asZ.
Letnk =mk, k≥1. Consider the nthk row of the array and let S1,k be the sum of the firstk variables there,S2,k the sum of the next k variables, and so on, ending withSm,k, the sum of the lastkvariables at or before the diagonal. Then, these m sums are independent and identically distributed and their sum isZnk=Zmk.
In Lemma 8.27below we shall show that there is a subsequenceK such that (S1,k) converges in distribution along K to some random variable Y1. Then, sinceSjk has the same distribution asS1,k, the sequence (Sjk) con- verges in distribution along the sameKto some variableYj forj= 1, . . . , m, and moreover the variables Y1, . . . , Ym are identically distributed and can be made to be independent. The independence of the Sj,k for j = 1, . . . , m implies (see Exercise 5.22) that
Zmk=S1,k+· · ·+Sm,k→Y1+· · ·+Ym
in distribution ask → ∞alongK. In other words, (Zn) has a subsequence that converges toY1+· · ·+Ymin distribution. SinceZn→Z in distribution by assumption, this implies thatZhas the same distribution asY1+· · ·+Ym,
and the proof is complete.
The Lemma needed in the proof is next, with the same assumptions and notations.
8.27 Lemma. The sequence (S1,k) has a subsequence that is convergent in distribution.
Proof. In view of Theorem5.13on tightness and existence of subsequences that converge, it is enough to show that the distributions ofS1,k,k≥1, form a tight family.
Suppose, to the contrary, that the family is not tight. Then, there exists ε >0 such that for everybinR+there is a subsequenceJ such thatP{S1,k∈ [−b, b]} ≤1−2ε for everyk in J, which implies that eitherP{S1,k > b} or P{S1,k<−b}exceedsε. SinceS1,k, . . . , Sm,kare independent and identically distributed and sum toZmk, it follows that eitherP{Zmk> mb}orP{Zmk<
−mb}exceedsεm. Thus,
P{|Zmk|> mb} ≥εm
for every k in J. Since (Zn) converges in distribution to Z, this implies that P{|Z| > mb} ≥ εm for every b such that mb and −mb are points of continuity for the distribution function of Z. Letting b → ∞ we see that P{S ∈R} ≤1−εm<1, which contradicts the definition of convergence in
distribution.
Finally, returning to the array8.2with the assumptions stated there, we state the following general theorem which also identifies the limiting distri- bution. We do not assume identical distribution for the elements of a row.
Recall that∂Adenotes the boundary ofA. We omit the proof.
8.28 Theorem. Suppose that the array[Xnj]satisfies the “infinitesimal- ity” condition8.22. In order that(Zn−an)converge in distribution for some (an)in R, it is necessary and sufficient that there exist a constant b≥0and a measureλ on the Borel σ-algebra ofRsatisfying
ˆ
R
λ(dx)(x2∧1)<∞ 8.29
such that
limn
j
P{Xnj∈A}=λ(A)
for every Borel setA withλ(∂A) = 0, and
εlim→0lim inf
n→∞
j
VarXnj¯iε◦|Xnj|= lim
ε→0lim sup
n→∞
j
VarXnj¯iε◦|Xnj|=b . In the case of convergence, the constantsan can be chosen as
an=
j
EXnj1[−c,c]◦Xnj
wherec >0 is an arbitrary constant such that neither c nor −c is an atom of λ. With this choice of (an) and assuming convergence, the limit variable Z is infinitely divisible and its characteristic function is
exp
−1 2r2b+
ˆ
R
λ(dx)
eirx−1−irx1[−c,c](x)
, r∈R. In ChapterVIIwe shall construct infinitely divisible variables. The mea- sureλappearing in the theorem is called the L´evy measure; it is defined to satisfy8.29, which is used to show that certain series converge.
Exercises
8.30 Lyapunov’s condition. Suppose that limn
j
E|Xnj|2+δ = 0
for someδ >0. Show that this implies Lindeberg’s condition.
8.31 Lindeberg’s theorem. The classical case of Theorem8.1 can be put in the form of an array8.2 by definingXnj =Xj/√
n, j = 1, . . . , n, for some sequence (Xn) of independent and identically distributed variables. Show that, assuming Var Xn = b < ∞, then, Lindeberg’s condition holds. So, Lindeberg’s theorem subsumes the classical case.
Sec. 8 Central Limits 137 8.32 Convergence to Poisson. Suppose that the nth row of the array8.2 is (Xn,1, . . . , Xn,n, 0,0, . . .) where the Xnj are independent and identically distributedN-valued variables with
pn=P{Xnj≥1}, qn=P{Xnj ≥2} .
Ifnqn→0 andnpn →cfor some constantc >0, thenZn →Pc in distribu- tion. Show this directly.
Chapter IV
Conditioning
This chapter is a continuation of Chapter II. We start with conditional expectations, namely, estimation of a random variable in the presence of partial information. Then, the notion of conditional independence follows as a natural generalization of independence, where the role played by the expectation operator is now played by the conditional expectation opera- tor. Finally, we discuss various ways of constructing random variables and stochastic processes.
1 Conditional Expectations
Let (Ω,H,P) be a probability space. LetFbe a sub-σ-algebra of H, and Xan ¯R-valued random variable. As usual, we regardFboth as a collection of events and as the collection of allF-measurable random variables. Moreover, we think heuristically and regardF as a body of information, namely, the information that determines the valuesV(ω) for allV in F and allω in Ω.
Recall from ChapterII, Section4, that knowing the valueV(ω) is much less than knowingω but, rather, it affords us to infer various properties ofω.
Heuristically, the conditional expectation ofX givenFis a random vari- able ¯X such that, for every possibilityω, ¯X(ω) is the “best” estimate ofX(ω) based on the informationF. So, ¯X must be determined byF, and among all random variables determined by Fit must be the “closest” to X, the terms
“best” and “closest” having a definite meaning at least when X is square integrable, that is, when EX2 < ∞. Then, E(X −Y)2 can be regarded as a measure of the error committed by usingY as an estimate of X, and we wantE(X−X)¯ 2≤E(X−Y)2 for allY inF.