www.elsevier.com/locate/spa
Large deviations for the Fleming–Viot process with neutral
mutation and selection, II
(Donald A. Dawson
a;∗, Shui Feng
baThe Fields Institute, 222 College Street, Toronto, ONT, Canada M5T 3J1
bDepartment of Mathematics and Statistics, McMaster University, Hamilton, ONT, Canada L8S 4K1 Received 5 January 2000; received in revised form 10 July 2000; accepted 6 August 2000
Abstract
Large deviation principles are established for the Fleming–Viot process with neutral mutation and with selection, and the associated equilibrium measures as the sampling rate approaches zero and when the state space is equipped with the weak topology. The path-level large deviation results improve the results of Dawson and Feng (1998, Stochastic Process. Appl. 77, 207–232) in three aspects: the state space is more natural, the initial condition is relaxed, and a large deviation principle is established for the Fleming–Viot process with selection. These improvements are achieved through a detailed study of the behaviour near the boundary of the Fleming–Viot process with nite types. c2001 Elsevier Science B.V. All rights reserved.
MSC:primary 60F10; secondary 92D10
Keywords:Fleming–Viot process; Large deviations; Weak and-topology
1. Introduction
The Fleming–Viot process (henceforth, FV process) is a probability-valued stochas-tic process describing the evolution of the distribution of genotypes in a population under the inuence of mutation, replacement sampling, and selective advantages among various genotypes.
Let E= [0;1], C(E) be the set of continuous functions on E, and M1(E) denote
the space of all probability measures on E equipped with the usual weak topology
and Prohorov metric . Let A be the generator of a Markov process on E with
do-mainD(A). DeneD={F:F() =f(h; i); f∈C∞
b (R); ∈D(A); ∈M1(E)}, where
Cb∞(R) denotes the set of all bounded, innitely dierentiable functions on R. Then
the generator of the FV process in this article has the form
LF() =
Z
E
AF()
(x)
(dx) + 2
Z
E
Z
E
2F()
(x)(y)
Q(; dx;dy)
(Research supported by the Natural Science and Engineering Research Council of Canada. ∗Corresponding author.
E-mail addresses: don@elds.elds.utoronto.ca (D.A. Dawson), shuifeng@mcmail.cis.mcmaster.ca (S. Feng).
=f′(h; i)h; Ai+ 2
Z Z
f′′(h; i)(x)(y)Q(; dx;dy); (1)
where
F()=(x) = lim
→0+
−1{F((1−)+
x)−F()};
2F()=(x)(y)= lim
1→0+; 2→0+(12)
−1
{F((1−1−2)+1x+2y)−F()};
Q(; dx;dy) =(dx)x(dy)−(dx)(dy);
andx stands for the Dirac measure at x∈E. The domain of Lis D.E is called the
type space or the space of alleles, A is known as the mutation operator, and the last
term describes the continuous sampling with sampling rate. If the mutation operator
has the form
Af(x) = 2
Z
(f(y)−f(x))0(dy)
with0∈M1(E), we call the process a FV process with neutral mutation.
For any symmetric bounded measurable functionV(z; y) on E⊗2, let
V() =
Z
E
Z
E
V(z; y)(dz)(dy)
and
V()
; F
=
Z
E
Z
E
Z
E
F
(z)[V(z; y)−V(y; w)](dz)(dy)(dw):
Then the generator of a FV process with neutral mutation and selection takes the form
L
VF() =L
F() +
V()
; F
; (1.2)
whereV is called the tness function which is assumed to be continuous in the sequel.
A nice survey on FV process and their properties can be found in Ethier and Kurtz (1993). In particular, it is shown in Ethier and Kurtz (1993) that the martingale problem
associated with generators L andLV are well-posed.
Let T ¿0 be xed, and C([0; T]; M1(E)) denote the space of all M1(E)-valued,
continuous functions on [0; T]. For any inM1(E), letP; ; 0 andP; ; V; 0 be the laws
of the FV process with neutral mutation and FV process with neutral mutation and
selection, respectively.; ; 0 and; ; 0;V will represent the corresponding equilibrium
measures.
Let X(E) be the space of all nitely additive, non-negative, mass one measures
on E, equipped with the smallest topology such that for all Borel subset B of E,
(B) is continuous in . The -algebra B of space X(E) is the smallest -algebra
such that for all Borel subset B of E, (B) is a measurable function of . It is clear
that M1(E) is a strict subset of X(E). In Dawson and Feng (1998), large deviation
principle (henceforth, LDP) is established for equilibrium measures on spaceX(E) and
measures on spaceM1(E), and compare these with results obtained in Dawson and Feng
(1998). Secondly, we establish the LDPs for the FV process with neutral mutation and
with selection as →0 on space C([0; T]; M1(E)). These improve the corresponding
results in Dawson and Feng (1998) in three aspects: the space is more natural, the initial condition is relaxed, and a full large deviation principle is established for the FV process with selection. This type of LDPs can be viewed as the innite-dimensional generalization of the Freidlin–Wentzell theory. We prove the results through a detailed analysis of the boundary behaviour of the FV process with nite types.
In Section 2, we list some preliminary results to make the paper self-contained. LDPs for the equilibrium measures are the content of Section 3. The detailed study of the Fleming–Viot process with nite type space is carried out in Section 4. Finally, in Section 5, we prove the LDPs for the FV process with neutral mutation and with selection.
2. Preliminary
In this section, we give some denitions and results in the theory of large deviations. A more complete introduction to the theory is found in Dembo and Zeitouni (1993). Properties of nite additive measures will also be discussed.
Let X be a Hausdor space with -algebra F. Here F could be smaller than the
Borel -algebra on X.{P: ¿0} is a family of probability measures on (X;F).
Denition 2.1. The family{P: ¿0}is said to satisfy a LDP with a good rate
func-tion I if
1. for all x∈X; I(x)¿0;
2. for anyc¿0, the level set (c) ={x∈X:I(x)6} is compact in X;
3. for anyF-measurable closed subset F of X,
lim sup
→0
logP(F)6−inf x∈FI(x);
4. for anyF-measurable open subset G of X,
lim inf
→ logP(G)¿−xinf∈GI(x):
Functions satisfying the rst two conditions are called good rate functions.
Denition 2.2. The family{P: ¿0}is said to be exponentially tight if for any ¿0
there exists a compact subset K of X such that
lim sup
→0
logP(Kc)6−;
whereKc
is the complement of K in X.
Theorem 2.1 (Pukhalskii). Assume that the space X is a Polish space with metric
d and F is the Borel -algebra. Then the family {P: ¿0} satises a LDP with
good rate functionI if and only if the family{P: ¿0} is exponentially tight; and
for anyx∈X
−I(x) = lim
→0lim sup→0
logP{y∈X: d(y; x)6}
= lim
→0lim inf→0 logP{y∈X: d(y; x)¡ }: (2.3)
If the family {P: ¿0} is exponentially tight; then for any sequence (n)
con-verging to zero from above as n goes to innity; there is a subsequence (nk) of (n)
such that{Pnk} satises a LDP with certain good rate function that depends on the
subsequence.
Proof. Part one and part two are Corollary 3:4 and Theorem (P) of Pukhalskii (1991) respectively.
LetX be a Polish space,M1(X) denote the space of all probability measures on X
equipped with the weak topology. Let Cb(X) and Bb(X) denote the set of bounded
continuous functions onX, and the set of bounded measurable functions on X,
respec-tively. For any ; in M1(X), the relative entropy of with respect to is dened
and denoted by
H(|) =
Z
X
logd if.;
∞ otherwise;
(2.4)
where is the Radon–Nikodym derivative of with respect to . It is known (cf.
Donsker and Varadhan, 1975) that
H(|) = sup
g∈Cb(X)
Z
X
gd−log
Z
X
egd
= sup
g∈Bb(X)
Z
X
gd−log
Z
X
egd
: (2.5)
The relative entropy is closely related to the rate functions discussed in Section 3. Finally, we present some results on nitely additive measures in Yosida and Hewitt
(1952). Assume that the spaceX is a Polish space and F is the Borel -algebra. Let
N(X) denote the set of all nonnegative, nitely additive measures on (X;F). For any
; in N(X), we write 6 if for any B in F we have(B)6(B).
Denition 2.3. Let be any element in N(X). If 0 is the only countably additive
measure satisfying 066;then is called a pure nitely additive measure.
Here is an example of a pure nitely additive measure. Let X = [0;1]; F be the
Borel -algebra. Dene measure such that it only assumes the values of 0 and 1,
(A) = 1 if A contains [0; a) for some a ¿0, (A) = 0 if A has Lebesgue measure
in Yosida and Hewitt (1952). Since limn→∞([0;1=n)) = 16= 0 =({0}); is not
countably additive. To see that it is pure nitely additive, let be any countably
additive measure satisfying 066. Then
([0;1]) =({0}) + lim
n→∞(1−((0;1=n])) = 0
which implies that≡0.
Theorem 2.2 (Yosida and Hewitt, 1952, Theorem 1:19). Let be any non-negative
pure nitely additive measure; and be any non-negative countably additive
mea-sure on space (X;F). Then for any ¿0; there exists A in F such that (Ac) =
0; (A)¡ ; where Ac is the complement of A.
Theorem 2.3. Any non-negative measure on space (X;F) can be uniquely written
as the sum of a non-negative;countably additive measure cand a non-negative;pure
nitely additive measure p.
Proof. This is a combination of Theorems 1:23 and 1:24 in Yosida and Hewitt (1952).
3. Equilibrium LDPs
LetX(E) be the space of all nitely additive, non-negative, mass one measures onE
equipped with the projective limit topology, i.e., the weakest topology such that for all
Borel subsetB ofE,(B) is continuous in . Under this topology,X(E) is Hausdor.
The-algebraBof spaceX(E) is the smallest-algebra such that for all Borel subset
B of E; (B) is a measurable function of .
It was incorrectly stated in Dawson and Feng (1998) thatX(E) can be identied with
M1(E) equipped with the-topology for large deviation purposes. In this section we will
rst clarify the issues associated with equilibrium LDPs on the spaceX(E), and then
establish the LDPs for equilibrium measures onM1(E) under the weak topology. Recall
that the-topology onM1(E) is the smallest topology such thath; fi=RE f(x)(dx) is
continuous in for any bounded measurable functionf onE. This topology is clearly
stronger than the weak topology, and is the same as the subspace topology inherited
from X(E). We use M
1(E) to denote space M1(E) equipped with the-topology.
Theorem 3.1. Every element inX(E) has the following unique decomposition:
=ac+s+p; (3.1)
where p is a pure nitely additive measure; ac and s are both countably additive
with ac.0; s⊥0.
Proof. The result follows from Theorem 2.3 and the Lebesgue decomposition theorem.
For anyinX(E) satisfyingac(E)¿0, letac(·|E)=ac(·)=ac(E). Then it is clear
thatac(·|E) is inM1(E). is said to be absolutely continuous with respect to0, still
denoted by .0, if 0(B) = 0 implies (B) = 0. For any two probability measures
; ; H(|) denotes the relative entropy of with respect to . Dene
I() =
[H(0|ac(·|E))−logac(E)] if .0; ac(E)¿0; 6∈M1(E);
H(0|) if .0; ∈M1(E);
∞ else:
(3.2)
Remark. .0 implies that s= 0.
Theorem 3.2. The family{; ; 0}satises a LDP onX(E)with good rate functionI.
Proof. Let
P={{B1; : : : ; Br}:r¿1; B1; : : : ; Br is a partition of [0;1]
by Borel measurable sets}: (3.3)
Elements ofP are denoted by ; —;and so on. We say —≻ i — is ner than. Then
P partially ordered by ≻ is a partially ordered right-ltering set.
For every—= (B1; : : : ; Br)∈P, let
X—=
(
x= (xB1; : : : ; xBr):xBi¿0; i= 1; : : : ; r; r
X
i=1
xBi= 1 )
:
For any = (C1; : : : ; Cl); —= (B1; : : : ; Br)∈P; —≻, dene
—:X—→X; (xB1; : : : ; xBr)→ X
Bk⊂C1
xBk; : : : ; X
Bk⊂Cl
xBk !
:
Then {X—; —; ; —∈J;≻} becomes a projective system, and the projective limit of
this system can be identied asX(E).
For any nite partition= (B1; : : : ; Br) of E, and any in X(E), let
I() =
Xr
k=1 0(Bk)log
0(Bk)
(Bk)
if.0;
∞ else;
where we treat c=0 as innity for c ¿0.
Let
˜
I() =sup
I(); (3.4)
where the supremum is taken over all nite partitions of E.
By the standard monotone class argument indicator function over an interval can be approximated by bounded continuous functions pointwise. Hence, the restriction of
the -algebra B on M1(E) coincides with the Borel -algebra generated by the weak
topology, and ; ; 0 is well-dened on space (X(E);B).
By using Theorem 3:3 of Dawson and Gartner (1987), one gets that; ; 0 satisties
I() = ˜I(). This is true if is not absolutely continuous with respect to 0 since
intersection, the sequence{An} can be chosen to be decreasing. For any nite partition
= (B1; : : : ; Br) we introduce a new nite partition —= (B1∩An; B1∩Acn; : : : ; Br∩An;
Letting n go to innity, we get
Lemma 3.3. Assume that there is a sequence of decreasing intervals An such that
Remark. Clearly a large class of probability measures including Lebesgue measure satisfy the condition in Lemma 3.3. But pure atomic measures with nite atoms do not satisfy the condition.
Proof. We will construct a counter example. Assume thatI() is a good rate function.
Then for any ¿0, the level set () ={∈M1(E):I()6} is a -compact set.
whereA is the indicator function of set A, and Ac denotes the complement of A. By
denition, we have
sequential compactness (see Ganssler, 1971, Theorem 2:6), the sequencen converges
intopology to a measure in() and thus in weak topology to the same measure.
For any continuous functiong on E, one has
lim
Lemma 3.4. Assume0 satises the condition in Lemma3:3.Then it is impossible to
establish a LDP for the Poisson–Dirichlet distribution with respect to 0 on M1(E)
with a good rate function.
Proof. Being the projective limit of a system of Hausdor space, the space X(E) is
also Hausdor. By Tychono theorem, the product space ∈PX is compact. since
topological space. Assume that a LDP is true on M
1(E) with a good rate function
J(). By Lemma 3.3, J must dier from I. But the following arguments will lead to
the equality of the two which is an obvious contradiction.
Fix an 0 in M1(E). The LDP on M1(E) with good J implies the corresponding
LDP on M1(E) with good J. Since M1(E) is regular, we get that for any ¿0 there
is an open neighborhood Gw of
0 such that
inf
∈Gw
J()¿(J(0)−)∧
1
;
whereGw is the closure in space M
1(E).
LetGX(E) be an open set in X(E) such that GX(E)∩M
1(E) =Gw. This is possible
because the subspace topology on M1(E) inherited from X(E) is stronger than the
weak topology. Now from the two LDPs, we get
−inf
Gw I()6−infGX I()6lim inf→0 log; ; 0(G X(E)
) = lim inf
→0 log; ; 0(G
w)
6lim sup
→0
log; ; 0(Gw)6−inf
Gw
J();
which implies that
I(0)¿inf
Gw I()¿inf Gw
J()¿(J(0)−)∧
1
:
Letting approach zero we end up with I(0)¿J(0): On the other hand, sinceX(E)
is also regular, we get that for any ¿0, there exists open set GX(E) containing
0
such that
inf
∈GX(E)
I()¿(I(0)−)∧
1
andGX(E)
is the closure in X(E).
LetG=GX(E)
∩M1(E). Then as before we get
−inf
G J()6lim inf→0 log; ; 0(G) = lim inf→0 log; ; 0(G
X(E) )
6lim sup
→0
log; ; 0(G X(E)
)6− inf
GX
(E)
I();
which implies that
J(0)¿inf
G J()
¿ inf
GX(E)
I()¿(I(0)−)∧
1
andJ(0)¿I(0).
From Theorem 3.1 and Lemma 3.4 we can see that in order to get an equilibrium
LDP in thetopology, one has to expandM1(E) to a bigger space. Next we are going
to show that under a weaker topology, the weak topology, the equilibrium LDP holds on M1(E).
First note that the space M1(E) is a compact, Polish space with Prohorov metric .
for ; ; 0 with a good rate function it suces to verify that there exists a function J
such that for every ∈M1(E),
lim
→0lim inf→0 log; ; 0{(; )¡ }
= lim
→0lim sup→0
log; ; 0{(; )6}=−J(): (3.5)
By Theorem 2.1, the functionJ is the good rate function.
Let supp() denote the support of a probability measure , and M1; 0(E) ={∈
M1(E): supp()⊂supp(0)}. Let {n} be an arbitrary sequence in M1; 0(E) that
con-verges to a in M1(E). Since supp(0) is a closed set, we get
1 = lim sup
n→∞
n{supp(0)}6{supp(0)};
which implies that ∈M1; 0(E). Hence M1; 0(E) is a closed subset of M1(E).
Next we prove (3.5) for
J() =
(
H(0|) if ∈M1; 0(E);
∞ else;
We will treat 0 log0
0 as zero.
For any∈M1(E), deneE={t∈(0;1):({t})=0}. For anyt1¡ t2¡· · ·¡ tk∈E,
sett1; :::; tk= (([0; t
1)); : : : ; ([tk;1])) which can be viewed as a probability measure on
space{0;1; : : : ; k} with a probability ([ti; ti+1)) at i6=k and ([tk;1]) atk. Set t0= 0.
Lemma 3.5. For any ; ∈M1(E);
H(|) = sup
t1¡t2¡···¡tk∈E; k¿1
H(t1; :::; tk|t1; :::; tk): (3.6)
Proof. By Lemma 2:3 in Dawson and Feng (1998), we have
H(|)¿ sup
t1¡t2¡···¡tk∈E; k¿1
H(t1; :::; tk|t1; :::; tk): (3.7)
On the other hand, by (2.5) for any ¿0, there is a continuous function g on E
such that
H(|)6
Z
gd−log
Z
egd
+:
Now chooset1n¡ t2n· · ·¡ tnkn inE such that
lim
n→∞i=0max;:::; kn−1
|tin+1−tin|+ max
t; s∈[tn i; t
n i+1]
|g(t)−g(s)|
= 0:
This is possible because E is a dense subset of E. Choose n large enough and let
tn0= 0, we get
H(|)6 kn X
i=0
−log
"kn−1 X
i=0
eg(tni)([tn
i; tni+1)) + eg (tknn)
([tknn;1])
#
++n(g)
6 sup
i; i=0;:::; kn ( kn
X
i=0
i([tin; tin+1)) +kn([t n kn;1])
−log
"kn−1
X
i=0
ei([tn
i; tin+1)) + ekn([tknn;1]) #)
++n(g)
=H(tn1;:::;tkn|t1n;:::;tkn) ++ n(g);
wheren(g) converges to zero asn goes to innity. Lettingn go to innity, then go
to zero, we get
H(|)6 sup
t1¡t2¡···¡tk∈E; k¿1
H(t1; :::; tk|t1; :::; tk):
This combined with (3.7) implies the result.
For any ¿0; ∈M1(E), let
B(; ) ={∈M1(E):(; )¡ }; B(; ) ={∈M1(E):(; )6}:
Since the weak topology on M1(E) is generated by the family
{∈M1(E):f∈Cb(E); x∈R; ¿0;|h; fi −x|¡ };
there exist f1; : : : ; fm in Cb(E) and ¿0 such that
{∈M1(E):|h; fji − h; fji|¡ :j= 1; : : : ; m} ⊂B(; ):
Let
C= sup{|fj(x)|:x∈E; j= 1; : : : ; m};
and choose t1; : : : ; tk∈E such that
sup{|fj(x)−fj(y)|:x; y∈[ti; ti+1]; i= 0;1; : : : ; k; tk+1= 1; j= 1; : : : ; m}¡ =4:
Choosing 0¡ 1¡ =2(k+ 1)C, dene
Vt1;:::;tk(; 1) ={∈M1(E):|([tk;1])−([tk;1])|¡ 1;
|([ti; ti+1))−([ti; ti+1))|¡ 1; i= 0; : : : ; k−1}:
Then for any inVt1;:::; tk(; 1) and any fj, we have
|h; fji − h; fji|=
Z
[tk;1]
fj(x)((dx)−(dx))
+
k−1
X
i=0
Z
[ti; ti+1)
fj(x)((dx)−(dx))
¡
2 +
k
X
i=0
which implies that
Vt1;:::; tk(; 1)⊂{∈M1(E):|h; fji − h; fji|¡ :j= 1; : : : ; m} ⊂B(; ):
Let
F() = (([0; t1)); : : : ; ([tk;1])):
Then ; ; 0 ◦ F
−1 is a Dirichlet distribution with parameters (=)(([0; t
1)); : : : ;
([tk;1])). By applying Theorem 2.2 in Dawson and Feng (1998) we get that for
in M1; 0(E),
−J()6−H(t1; :::; tk
0 |
t1; :::; tk)
6lim inf
→0 log; ; 0{Vt1;:::; tk(; 1)}
6lim inf
→0 log; ; 0{B(; )}: (3.8)
Letting go to zero, we end up with
−J()6lim
→0lim inf→0 log; ; 0{B(; )}: (3.9)
For other , (3.9) is trivially true.
On the other hand, for any t1; : : : ; tk in E, we claim that the vector function F is
continuous at . This is because all boundary points have -measure zero. Hence for
any2¿0, there exists ¿0 such that
B(; )⊂Vt1;:::; tk(; 2):
Let
Vt1;:::; tk(; 2) ={∈M1(E):|([tk;1])−([tk;1])|61;
|([ti; ti+1))−([ti; ti+1))|61; i= 0; : : : ; k−1}:
Then we have
lim
→0lim sup→0
log; ; 0{B(; )}6lim
→0lim sup→0
log; ; 0{Vt1;:::;tk(; 2)}: (3.10)
By letting2 go to zero and applying Theorem 2.2 in Dawson and Feng (1998) to
; ; 0◦F
−1 again, one gets
lim
→0lim sup→0
log; ; 0{B(; )}6−J
t1; :::; tk
0
(t1; :::; tk); (3.11)
where
J
t1; :::; tk 0
(t1; :::; tk) = (
H(t1; :::; tk
0 |t1; :::; tk) if t1; :::; tk.
t1; :::; tk
0 ;
∞ else:
Finally, taking supremum over the setE and applying Lemma 3.5, one gets
lim
→0lim sup→0 log; ; 0{
B(; )}6−J(); (3.12)
Theorem 3.6. The family {; ; 0} satises a LDP on M1(E) with good rate
function J().
Remark. 1. By (3.2), for any inM1(E), we have thatI()=∞ifis not absolutely
continuous with respect to0. On the other hand, if we choose=120+120, then
is not absolutely continuous with respect to0 butJ()¡∞. Hence the restriction of
I() on M1(E) is not equal to J().
2. Let (1; 2; : : :) be a probability-valued random variable that has the Poisson–
Dirichlet distribution with parameter = (cf. Kingman, 1975), and 1; 2; : : : ; be i.i.d.
with common distribution 0. Then ; ; 0 is the distribution of
P∞
i=1ii (cf. Ethier
and Kurtz, 1994, Lemma 4:2), and the LDP we obtained describes the large deviations
in the following law of large numbers:
∞
X
i=1
ii ⇒0:
The new features are clearly seen by comparing this with the Sanov theorem that describes the large deviations in the law of large numbers:
n
X
i=1
1
ni ⇒0:
Corollary 3.1. The family {; ; 0;V} satises a LDP on space M1([0;1])with good
rate function JV() = sup{V()−J()} −(V()−J()).
Proof. By Lemma 4.2 of Ethier and Kurtz (1994), one has
; ; 0;V(d) =Z −1exp
V()
; ; 0(d); (3.13)
whereZ is the normalizing constant.
Since V(x) is continuous, we get that V()∈C(M1([0;1])). By using Varadhan’s
Lemma, we have
lim
→0logZ= lim→0log
Z
eV()=
; ; 0(d) = sup
{
V()−J()}: (3.14)
By direct calculation, we get that for any∈M1(E)
lim
→0lim inf→0 log
Z
B(;)
eV()=; ; 0(d)
=V() + lim
→0lim inf→0 log; ; 0{B(; )}
=V() + lim
→0lim sup→0 log; ; 0{
B(; )}
= lim
→0lim inf→0 log
Z
B(;)
eV()=; ; 0(d)
which combined with (3.14) implies
lim
→0lim inf→0 log; ; 0;V{(; )¡ }
= lim
→0lim sup→0 log; ; 0;V{(; )6}=−JV(): (3.15)
Since the family {; ; 0;V} is also exponentially tight, using Theorem 2.1 again,
we get the result.
4. LDP for FV process with nite types
We now turn to the study of LDP at the path-level. In this section, we focus on FV process with nite types.
LetEn={1;2; : : : ; n}, and dene
Sn=
(
x= (x1; : : : ; xn−1): xi¿0; i= 1; : : : ; n−1; n−1
X
i=1
xi61
) ;
with S◦
n denoting its interior. Then the FV process with n types is a FV process
with neutral mutation and type space En. It is a nite-dimensional diusion process
statisfying the following system of stochastic dierential equations:
dxk(t) =bk(x(t))dt+√ n−1
X
l=1
kl(x(t))dBl(t); 16k6n−1; (4.1)
where x(t) = (x1(t); : : : ; xn−1(t)), bk(x(t)) ==2(pk − xk(t)), and (x(t)) =
(kl(x(t)))16k; l6n−1 is given by
(x(t))′(x(t)) =D(x(t)) = (x
k(t)(kl−xl(t)))16k; l6n−1;
wherepk=0(k), and Bl(t); 16l6n−1 are independent Brownian motions.
LetPx denote the law of x(·) starting atx. The LDP for Px on space C([0; T];Sn)
as goes to zero has been studied in Dawson and Feng (1998) under the assumptions
thatpk is strictly positive for allkandxis in the interior ofSn. In this section we will
consider the LDP forPx when these assumptions are not satised, i.e., some of pk are
zero orxis on the boundary. This creates serious diculties because of the degeneracy
and the non-Lipschitz behaviour of the square root of the diusion coecient on the boundary. Letpn= 1−Pni=1−1pi,xn= 1−Pni=1−1xi, andbn(x) = (=2)(pn−xn). If for a
particular k, pk=xk∈ {0;1}, then xk(t) will be zero (or one) for all positivet. Thus,
without the loss of generality, we assume thatpk+xk is not zero or two for all k. For
any givenp= (p1; : : : ; pn−1), dene
Zp={x= (x1; : : : ; xn−1)∈Sn: 0¡ xk+pk¡2; k= 1; : : : ; n}:
For any x in Zp, let Hx; is the set of all absolutely continuous element in
C([; ];Sn) starting atx, i.e.,
H; x =
’∈C([; ];Sn):’(t) =x+
Z t
˙
’(s) ds
Dene
Lebesgue integrals. We denote I0; T
x (Hx0; T) by Ix (resp. Hx ).
from (A) and (B), respectively.
For any ¿0; N¿1, and 06a6b6T, set
result holds in the case of’(0)6=x. If’ is not absolutely continuous, then there exist
c ¿0 and disjoint subintervals [am
=−
depending on . Here
sign(c) =
which combined with Doob’s inequality implies
For any N¿1, choose small enough such that no functions in the set
B(’;2; 1=N; T) hit zero or one in the time interval [1=N; T]. Let be the law of
x1=N under Px. Then one gets
logPx{B(’; ; 0; T)}6logPx{B(’; ; 1=N; T)}
=log
Z 1
0
Py{B(’; ; 1=N; T)}(dy)
6log sup
|y−’(1=N)|6
Py{B(’; ; 1=N; T)}
=logPy{B(’; ; 1=N; T)} for some|y−’(1=N)|6;
where in the last equality we used the property that the supremum of an upper semi-continuous function over a closed set can be reached at certain point inside the set.
Noting that Py coincides with a non-degenerate diusion over the interval [1=N; T]
on any set that does not hit the boundary of [0;1]. By the uniform large deviation
principle for non-degenerate diusions (cf. Dembo and Zeitouni, 1993), we get
lim sup
→0
logPx{B(’; ; 0; T)}6− inf
|y−’(1=N)|6 ∈B(inf’;;1=N;T)I 1=N; T
y ( ): (4.5)
Assume that inf|y−’(1=N)|6inf ∈B(’; ;1=N; T)Iy1=N; T( ) is nite for small . Otherwise
Ix(’)¿I’1(1=N; T=N)(’) =∞, and the upper bound is trivially true. For any y satisfying
|y−’(1=N)|6, inB(’; ; 1=N; T) satisfying (1=N) =y,Iy1=N; T( )¡∞, we dene
for t in [1=N; T]
(t) = (t) + (’(1=N)−y):
Then it is clear that is in B(’;2; 1=N; T) and thus does not hit zero or one. By
direct calculation, we get that
I’1=N; T(1=N)( )6Iy1=N; T( ) +N;
whereN goes to zero as goes to zero for any xed N. This combined with (4.5)
implies that
lim sup
→0
lim sup
→0
logPx
sup
06t6T|
x(t)−’(t)|6
6−lim
→0 ∈B(’;inf2;1=N;T)I 1=N; T
’(1=N)( ) =−I 1=N; T
’(1=N)(’); (4.6)
where the equality follows from the lower semicontinuity ofI’1=N; T(1=N)(·) at non-degenerate
paths. Finally by lettingN go to innity we end up with (4.3).
More generally, if ’(t) is in (0;1) over [; ]⊂[0;1], then the above argument
leads to
lim sup
→0
lim sup
→0
logPx
sup
06t6T|
x(t)−’(t)|6
6−I’; ()(’): (4.7)
Let
1= inf{t∈[0; T]:’(t) = 1}; = inf{t∈[0; T]:Ix0; t(’) =∞}:
If ¡ 1, (4.3) is proved by applying the arguments in Case II to the time interval
[0;(+1)=2]. Now assume that ¿1.
Since p ¡1, one can nd 0¡ t3¡ t2¡ 1 satisfying infs∈[t3;1]’(s)¿ p. By using
result in Case II, we have
lim sup
→0
logPx (
sup
t∈[0;T]|
x(t)−’(t)|6 )
6−I0; t2
x (’):
Since I0; t2
x (’) is nite for all t2, we get
Ix(’)¿Ix0; t2(’) =
1 2
Z t2
0
( ˙’(t)−(=2)(p−’(t)))2
’(t)(1−’(t)) dt
¿
2
Z t2
t3
(’(t)−p) ’˙(t)
1−’(t)dt→ ∞ ast2 ր1; (4.8)
which implies (4.3).
CaseIV: A second visit to zero by ’(t) occurs at a strictly positive time.
Let
0= inf{t ¿0:’(t) = 0}¿0:
Choosing 0¡ t1¡ t2¡ 0 such that inft∈[t1;0](p−’(t))¿0. Then we have
Ix(’)¿ lim t2ր0
1 2
Z t2
t1
( ˙’(t)−(=2)(p−’(t)))2
’(t) dt
¿− lim
t2ր0
2
Z t2
t1
(p−’(t))’˙(t)
’(t)dt=∞:
This combined with (4.7) implies the result.
(B) 06x ¡1; p= 1. First assume thatIx(’) is nite (i.e.=∞). For small, dene
(’) ={t∈[0; T]:’(t)¡1−}=
∞
[
i=1
(ai; bi):
Since the set {t∈[0; T]:( ˙’(t)−(=2)(1−’(t)))2=’(t)(1−’(t)) = 0} has no
con-tribution to the value of Ix(’) and the niteness of Ix(’) implies that the set N=
{t∈[0; T]: ( ˙’(t)−(=2)(1−’(t)))2=’(t)(1−’(t)) =∞} has zero Lebesgue measure,
we may redene the value of ( ˙’(t)−(=2)(1−’(t)))2=’(t)(1−’(t)) to be zero onN
without changing the value of the rate function. After this modication we can apply
the monotone convergence theorem and get that I
x (’) converges to Ix(’) as goes
to zero.
By the Markov property and an argument similar to that used in deriving (4.5) and
(4.6), we have for anym¿1 and ¿0,
lim sup
→0
lim sup
→0
logPx (
sup
t∈[0;T]|
6
Next we assume the rate function is innity. By an argument similar to that used in Case IV, we get the result for all paths that hit zero at a positive time. We now
assume that ’ does not hit zero at any positive time. If ¡ 1 the result is true by
using the argument in Case III. Let us now assume that166T.
If’(t) = 1 over [; +] for some ¿0, then we have limtրIx0; t(’) =∞ and the
result follows by approaching from below.
If ’() is in (0;1), then the result is obtained by using (4.7) in a small two-sided
neighborhood of since the rate function over the neighborhood is innity.
The only possibility left is that ’() = 1, and 0¡ ’(t)¡1 over (; ] for some
¡ 6T. By applying (4.7) over [; ] with ∈(; ) and letting approach from
above, we get the result.
6( ˙’k(t)−bk(’(t)))
loss of generality, let us assume that k= 1. Thus by (4.12) and by applying Lemma
4.1 to the two-type process (x1(·);Pnj=2xj(·)) we get both (4.10) and (4.11) when
By an argument similar to that used in the proof of (B) of Lemma 4.1, one can apply the monotone convergence theorem and get
Ix(’) = lim
and so (4.10) follows.
Next assume thatx is in S◦
By direct calculation, one gets
61
theorem, we get that
lim
This, combined with (4.14) and (4.15), implies lim sup
→0
Ix(’)6Ix(’): (4.18)
Finally, by letting go to zero, then go to zero in (4.13), we get (4.11).
Remark. More detailed information about the way that a path leaves or approaches the boundary can be obtained by an argument similar to that used in Feng (2000), where the one-dimensional continuous branching processes are studied.
Proof. If xi=pi= 0 for some i= 1; : : : ; n−1, then xi(t)≡0 for all t in [0; T]. This
explains the denition of the set Hx; p. By projection to lower dimension, the result
is reduced to the case wherexi+pi¿0 for all i. Then by applying Lemma 4.2 and
Theorem 2.1, we get the result.
Remark. This result has been proved in Theorem 3:3 in Dawson and Feng (1998)
under the assumption that xi¿0; pi¿0 for all i= 1; : : : ; n. Here we removed all
restrictions on x and p in the upper bound, and extend the lower bound to cases
wherepi can be zero for some i= 1; : : : ; n.
5. LDP for FV processes
Path level LDPs are established in this section for FV processes with neutral muta-tion, and with selection.
5.1. LDP for FV processes with neutral mutation
Let C1;0([0; T]×E) denote the set of all continuous functions on [0; T]×E with
continuous rst order derivative in timet. For any ∈M1(E), (·)∈C([0; T]; M1(E)),
dene
Y
0={∈M1(E): supp(0)⊂supp()} (5.1)
and
S((·))
= sup
g∈C1;0([0;T])×E)
h(T); g(T)i − h(0); g(0)i −
Z T
0 h
(s);
@ @s +A
g(s)ids
−1
2
Z T
0
Z Z
g(s; x)g(s; y)Q((s); dx;dy) ds
: (5.2)
Recall that for any inM1(E), we have Q(; dx;dy) =(dx)x(dy)−(dx)(dy).
LetBb1;0([0; T]×E) be the set all bounded measurable functions on [0; T]×E with
continuous rst-order derivative in timet, and stepwise constant inx. For every function
finC1;0([0; T]×E) and anyn, letfn(t; x) =f(t; i=n) forxin [i=n;(i+ 1)=n),fn(t;1) =
fn(t;(n−1=n)). Then it is clear that fn is in Bb1;0([0; T]×E) and fn converges to f
uniformly as n goes to innity. On the other hand, by interpolation, every element in
B1
b([0; T]×E) can be approximated almost surely by a sequence in C1;0([0; T]×E).
Hence, we get
S((·)) = sup g∈B1;0([0;T])×E)
h(T); g(T)i − h(0); g(0)i
−
Z T
0 h
(s); @
@s +A
g(s)ids
−1
2
Z T
0
Z Z
g(s; x)g(s; y)Q((s); dx;dy) ds
Denition 5.1. Let C∞(E) denote the set of all continuous functions onE possessing
continuous derivatives of all order. An element (·) in C([0; T]; M1(E)) is said to be
absolutely continuous as a distribution-valued function if there exist M ¿0 and an
absolutely continuous functionhM: [0; T]→R such that for all t; s∈[0; T]
sup
supx∈E|f(x)|¡M
|h(t); fi − h(s); fi|6|hM(t)−hM(s)|:
Let H be the collection of all absolutely continuous paths in C([0; T]; M1(E))
starting at, and dene
K((·)) =
Z T
0 k
˙
(s)−A∗((s))k2
(s)ds if (·)∈H;
∞ elsewhere;
whereA∗ is the formal adjoint of Adened through the equality hA∗(); fi=h; Afi,
and for any linear functional #on space C∞(E)
k#k2= sup
f∈C∞(E)
h#; fi −1
2
Z
E
Z
E
f(x)f(y)Q(; dx;dy)
:
Then we have the following.
Theorem 5.1. Assume ∈Y
0.Then for any (·) inC([0; T]; M1(E)); we have
S((·)) =K((·)): (5.4)
Proof. By denition, we have
S((·))6K((·)):
To get equality, we only need to consider the case when S((·)) is nite. For any
[s; t]⊂[0; T]; and any f inC1;0([s; t]×E), let
ls; t(f) =h(t); f(t)i − h(s); f(s)i −
Z t
s
h(u);
@ @u+A
fidu:
By introducing an appropriate Hilbert space structure and applying the Riesz
Repre-sentation Theorem, one can nd a square integrable function h such that
ls; t(f) =
Z t
s
Z
E
Z
E
f(u; x)h(u; y)Q((u); dx;dy) du; (5.5)
(f) =
Z t
s
Z
E
Z
E
inf
f∈C1;0([s; t]×E)(h(u; x)−f(u; x))(h(u; y)
−f(u; y))Q((u); dx;dy) du
6 inf
f∈C1;0([s; t]×E)
Z t
s
Z
E
Z
E
(h(u; x)−f(u; x))(h(u; y)
−f(u; y))Q((u); dx;dy) du
and
From (5.5), we can see that (·) is absolutely continuous as a distribution-valued
function. Applying (5.6),(5.7), and the fact that (f)¿0, we get
For further details, please refer to the appendix in Dawson and Feng (1998).
Remark. If E ={1; : : : ; n} for some n¿2, then we have C∞(E) =C(E), and the
absolute continuity of (·) as a distribution-valued function is the same as the usual
If ’k(s) = 0;’˙k(s)−bk(’(s)) 6= 0, then by choosing f(i) = 0 for i 6= k, f(k) =
nsign( ˙’k(s)−bk(’(s))), and letngo to innity, we get that k˙(s)−A∗((s))k2(s)=∞.
Assume that’i(s)¿0 for all 16i6n. Noting thatPni=1bi(’(s))=0, andPni=1’i(s)
= 1, we get
k˙(s)−A∗((s))k2(s)= sup
f∈C(E)
n−1
X
i=1
(f(i)−f(n))( ˙’i(s)−bi(’(s)))
−1
2
n−1
X
i; j=1
(f(i)−f(n))(f(j)−f(n))’i(s)(ij−’j(s))
:
Since the matrix (’i(s)(ij−’j(s)))16i; j6n−1 is invertible and its inverse is given by
Eq. (3.4) in Dawson and Feng (1998), we get that
k˙(s)−A∗((s))k2(s)=
n
X
i=1
( ˙’i(s)−bi(’(s)))2
’i(s)
:
Since we treat 0=0 as zero, c=0 as innity for c ¿0 in (4.2), we get that for nite
type model,
Ix; p(’) =S((·)); (5.8)
wherep= (0(1); : : : ; 0(n)). We thus derive a variational formula for the rate function
obtained in Theorem 4.3.
For any ∈M1(E), let P; ; 0 be the law of the FV process with neutral mutation
on space C([0; T]; M1(E)) starting at .
Lemma 5.2. The family {P; ; 0
}¿0 is exponentially tight; i.e.; for any a ¿1; there
exists a compact subsetKa of C([0; T]; M1(E)) such that
lim sup
→0
logP; ; 0
{Kca}6−a; (5.9)
where Kca is the complement of Ka.
Proof. Let A be the neutral mutation operator. For anyf in C(E), let
Mt(f) =h(t); fi − h(0); fi −
Z t
0 h
(s); Afids:
ThenMt(f) is a martingale with increasing process
hhM(f)iit=
Z t
0
Z Z
f(x)f(y)Q((s); dx;dy)ds:
More generally, let M(dt;dx) denote the martingale measure obtained from Mt(f).
In other words, M(dt;dx) is a martingale measure such that for any f in C(E),
Z t
0
Z
E
f(x)M(dt;dx) =Mt(f):
For any t in [0; T], let 0 =t0¡ t1¡· · ·¡ tm=t be any partition of [0; t]. For any
functiong inC1;0([0; T]×E) and any u; v in [0; T], let
Mu; v(g) =h(v); g(v)i − h(u); g(u)i −
Z v
u
(s);
@
@s+A
g
Then we have
-martingale with increasing process
Z t
0
Z Z
g(s; x)g(s; y)Q((s); dx;dy)ds:
Hence by the exponential formula, for any real number , the following
6
where the Markov property is used in the last inequality. For any xed f in C(E),
the constant
By Chebyshev’s inequality and the martingale property, we get
P; ; 0
Choosing == and minimizing with respect to ¿0 in (5.14), and by symmetry,
Then we have
and applying (5.18), we get (5.9).
Lemma 5.3. For any (·) in C([0; T]; M1(E)); we have
Proof. By denition, we have
S((·)) = sup
which, combined with the relationZT(1
g;1) = (Z
Lemma 5.4. For any (·) in C([0; T]; M1(E)) and in Y0; we have lim inf
→0 lim inf→0 logP
; ; 0
{d((·);(·))¡ }¿−S( (·)): (5.21)
Proof. If S( (·)) =∞, the result is clear. Next we assume that S( (·))¡∞. By
Theorem 5.1, (·) is absolutely continuous in t as a distribution-valued function which
implies the absolute continuity of (t;[a; b)) and (t;[a; b]) int for a; b∈E.
Let{fn∈C(E):n¿1} be a countable dense subset of C(E). Dene another metric
ˆ
d on C([0; T]; M1(E)) as follows:
ˆ
d((·); (·)) = sup
t∈[0;T]
∞
X
n=1
1
2n(|h(t); fni − h(t); fni| ∧1): (5.22)
Then d and ˆd generate the same topology on C([0; T]; M1(E)). Hence it suces to
verify (5.21) with ˆd in place of d. Clearly for any ¿0; ∈C([0; T]; M1(E)), there
exists ak¿1 such that
{(·)∈C([0; T]; M1(E)):|h(t); fni − h(t); fni|6=2; n= 1; : : : ; k}
⊂{dˆ((·);(·))¡ }:
Next choosex0= 0¡ x1¡ x2¡· · ·¡ xm¡1 =xm+1 such that
max
06i6m; x; y∈[xi; xi+1]
{|fn(y)−fn(x)|: 16n6k}6
6: Let
Ux1;:::; xm
(·);
6
=
(·)∈C([0; T]; M1(E)):
sup
t∈[0;T];06n6m−1{|
(t)([xn; xn+1))−(t)([xn; xn+1))|;
|(t)([xm;1])−(t)([xm;1])|}6
6
;
where = supx;16n6k|fn(x)|. Then we have
Ux1;:::; xm
(·);
6
⊂{dˆ((·);(·))¡ }: (5.23)
Set
((·)) = ((·)([0; x1)); : : : ; (·)([xm;1])):
The partition property (cf. Ethier and Kurtz, 1994) of the FV process implies that
P; ; 0
mu ◦−1 is the law of a FV process with nite type space. This combined with
Theorem 4.3, implies
lim inf
→0 lim inf→0 logP
; ; 0
mu {dˆ((·);(·))¡ }
¿lim inf
→0 lim inf→0 logP
; ; 0
{Ux1;:::; xm( (·);
6 )}
¿−IF(); F(0)(( ))¿−S( (·)); (5.24)
Lemmas 5.2–5.4, combined with Theorem 2.1, imply the following.
5.2. LDP for FV processes with selection
For any∈M1(E), letP; ; V; 0 be the law of the FV process onC([0; T]; M1(E)) with
tness functionV and initial point. By the Cameron–Martin–Girsanov transformation
(see Dawson, 1978) we have that,
dP; ; V; 0
Then we have the following theorem.
−1
which combined with the exponential tightness of the family {P; ; 0
}¿0 implies that
the family {P; ; V; 0
}¿0 is also exponentially tight.
By choosing C={:d((·); (·))6} in (5.30), we get
By an argument similar to that used in the derivation of (3:38) and (3:40) in Dawson
+1
lim sup→0
logP; ; 0
{d((·); (·))6}; (5.33)
lim inf
→0 logP
; ; V; 0
{d((·); (·))¡ }¿ V((·))−
2 +
2
+lim inf
→0 logP
; ; 0
{d((·); (·))¡ }: (5.34)
Letting go to zero, then go to zero and nally go to 1, we get that (5.32) is
also true for(·) satisfying S; V((·))¡∞. Applying Theorem 2.1 again, we get the
result.
Acknowledgements
We thank the referee for helpful comments and suggestions, and W. Stannat for suggesting the type of example used in the proof of Lemma 3.3.
References
Dawson, D.A., 1978. Geostochastic calculus. Canad. J. Statist. 6, 143–168.
Dawson, D.A., Feng, S., 1998. Large deviations for the Fleming–Viot process with neutral mutation and selection. Stochastic Process. Appl. 77, 207–232.
Dawson, D.A., Gartner, J., 1987. Large deviations from the McKean–Vlasov limit for weakly interacting diusions. Stochastics 20, 247–308.
Dembo, A., Zeitouni, O., 1993. Large Deviations and Applications. Jones and Bartlett Publishers, Boston. Donsker, M.D., Varadhan, S.R.S., 1975. Asymptotic evaluation of certain Markov process expectations for
large time, I. Comm. Pure Appl. Math. 28, 1–47.
Ethier, S.N., Kurtz, T.G., 1993. Fleming–Viot processes in population genetics. SIAM J. Control Optim. 31 (2), 345–386.
Ethier, S.N., Kurtz, T.G., 1994. Convergence to Fleming–Viot processes in the weak atomic topology. Stochastic Process. Appl. 54, 1–27.
Feng, S., 2000. The behaviour near the boundary of some degenerate diusions under random perturbation. In: Gorostiza, L.G., Gail Ivano, B., (eds.), Stochastic Models. CMS Conference Proceedings, Vol. 26, pp. 115 –123.
Ganssler, P., 1971. Compactness and sequential compactness in spaces of measures. Z.Wahrsch, Verw. Geb. 17, 124–146.
Kingman, J.F.C., 1975. Random discrete distributions. J. Roy. Statist. Soc. Ser. 37, 1–20.
Pukhalskii, A.A., 1991. On functional principle of large deviations. In: Sazonov, V., Shervashidze, T. (Eds.), New Trends in Probability and Statistics, VSP Moks’las, Moskva.
