Gaussian measures in Hilbert Spaces and their applications.

Boby Gunarso. 2016. Gaussian Measures In Hilbert Spaces And Their Applications. Skripsi. Program Studi Matematika, Jurusan Matematika, Fakultas Sains dan Teknologi, Universitas Sanata Dharma, Yogyakarta.

Boby Gunarso. 2016. Gaussian Measures In Hilbert Spaces And Their Applications. A Thesis. Mathematics Study Program, Department of Mathematics, Faculty of Science and Technology, Sanata Dharma University, Yogyakarta.

The Lebesgue measure plays a fundamental role in ℝ �. It is uniquely determined (up to some constant) by the properties of being locally finite and invariant under translation. One may ask a question whether the Lebesgue measure makes sense in an infinite dimensional space. The answer is negative. In order to build a well defined measure in infinite dimensional spaces, we can incorporate a rapidly decreasing exponential factor to the Lebesgue measure and hence we obtain the so-called Gaussian measure. Unfortunately, Gaussian measure with identity operator as a covariance function still cannot be defined in infinite dimensional separable Hilbert spaces. We have at least 2 ways to remedy this situation. First we can use trace class operator as a covariance function to show the existence of a Gaussian measure in infinite dimensional separable Hilbert spaces. The second way is by retaining the identity covariance operator but the consequence is the measure does exist only on a topological dual of a nuclear space. In this thesis, we will focus only on the first approach, i.e. we construct a Gaussian measure in infinite dimensional separable Hilbert spaces by using a trace class operator as a covariance function. We start the construction of Gaussian measure on the real line, on the finite dimensional Euclidean space and finally in an arbitrary infinite dimensional separable Hilbert space. We also use Gaussian measure to study Gaussian random variables, white noise mapping, Malliavin derivative, and a construction of a Brownian motion in a Gaussian Hilbert space.

GAUSSIAN MEASURES IN HILBERT SPACES AND THEIR

APPLICATIONS

BACHELOR THESIS

Presented as Partial Fulfillment of the Requirements to Obtain the Degree ofSarjana Matematika

Written by: Boby Gunarso

Student Number: 123114002

MATHEMATICS STUDY PROGRAM MATHEMATICS DEPARTMENT FACULTY OF SCIENCE AND TECHNOLOGY

SANATA DHARMA UNIVERSITY YOGYAKARTA

GAUSSIAN MEASURES IN HILBERT SPACES AND THEIR

APPLICATIONS

BACHELOR THESIS

Presented as Partial Fulfillment of the Requirements to Obtain the Degree ofSarjana Matematika

Written by: Boby Gunarso

Student Number: 123114002

MATHEMATICS STUDY PROGRAM MATHEMATICS DEPARTMENT FACULTY OF SCIENCE AND TECHNOLOGY

SANATA DHARMA UNIVERSITY YOGYAKARTA

ABSTRACT

Boby Gunarso. 2016. Gaussian Measures In Hilbert Spaces And Their

Appli-cations. A Thesis. Mathematics Study Program, Departement of Mathematics,

Faculty of Science and Technology, Sanata Dharma University, Yogyakarta.

The Lebesgue measure plays a fundamental role in Rn. It is uniquely determined (up to some constant) by the properties of being locally finite and invariant under

translation. One may ask a question whether the Lebesgue measure makes sense in

an infinite dimensional space. The answer is negative. In order to build a well

de-fined measure in infinite dimensional spaces, we can incorporate a rapidly decreasing

exponential factor to the Lebesgue measure and hence we obtain the so-called

Gaus-sian measure. Unfortunately, GausGaus-sian measure with identity operator as a covariance

function still cannot be defined in infinite dimensional separable Hilbert spaces. We

have at least 2 ways to remedy this situation. First we can use trace class operator as

a covariance function to show the existence of a Gaussian measure in infinite

dimen-sional separable Hilbert spaces. The second way is by retaining the identity covariance

operator but the consequence is the measure does exist only on a topological dual of

a nuclear space. In this thesis, we will focus only on the first approach, i.e. we

con-struct a Gaussian measure in infinite dimensional separable Hilbert spaces by using a

trace class operator as a covariance function. We start the construction of Gaussian

measure on the real line, on the finite dimensional Euclidean space and finally in an

arbitrary infinite dimensional separable Hilbert space. We also use Gaussian measure

to study Gaussian random variables, white noise mapping, Malliavin derivative, and a

ACKNOWLEDGEMENTS

First of all, I would like to thank my Lord, Jesus Christ, for the blessing so that I can

finally finish my bachelor thesis. I thank for all the ways, strengths, and courage that

He gave me during the process of finishing this thesis.

During the writing of this thesis, I recieve supports and assistance from many

peo-ple. Therefore, I would like to thank especially to:

1. Dr.rer.nat. Herry Pribawanto Suryawan, M.Si., as my thesis advisor, for his

guidance, suggestion, and correction during the writing of this thesis.

2. Y.G. Hartono, S.Si., M.Sc., Ph.D, as my academic advisor, for his guidance and

support during my bachelor study.

3. Prof. Dr. Frans Susilo, SJ and Prof. Dr. Christiana Rini Indrati, M.Si., as the

examiners, for their valuable correction and suggestions of this thesis.

4. All lecturers of the mathematics study program, for all the knowledge and

sup-port given to me during my bachelor study.

5. My family, for their support during my bachelor study.

6. All my friends in mathematics study program, for sharing happiness, support,

and knowledge during our study.

Finally, I realize that this thesis is still far from perfect. I welcome any critics and

suggestions to make this thesis better. I hope this thesis will be useful for everyone

who would like to read and have interest in this thesis topic.

Yogyakarta, 18 July 2016

TABLE OF CONTENTS

TITLE PAGE i

APPROVAL PAGE ii

ENDORSEMENT PAGE iii

STATEMENTS OF WORK’S ORIGINALITY iv

ABSTRACT v

PERNYATAAN PERSETUJUAN PUBLIKASI KARYA ILMIAH vi

ACKNOWLEDGEMENTS vii

TABLE OF CONTENTS viii

CHAPTER 1 INTRODUCTION 1

A. Research Background . . . 1

B. Research Problems . . . 2

C. Problem Limitation . . . 2

D. Research Objectives . . . 3

E. Research Benefits . . . 3

F. Research Methods . . . 3

G. Systematics Writing . . . 3

CHAPTER 2 TOOLS FROM MEASURE THEORY AND FUNCTIONAL ANALYSIS 5 A. Tools from Measure Theory . . . 6

B. Tools from Functional Analysis . . . 20

D. Motivation to Measure in Infinite Dimensional Spaces . . . 41

CHAPTER 3 GAUSSIAN MEASURES IN HILBERT SPACES 46 A. Mean and Covariance of Probability Measures in Hilbert Spaces . . . 46

B. Law of a Random Variable . . . 48

C. Gaussian Measures . . . 50

1. Gaussian Measures inR . . . 51

2. Gaussian Measures inRn . . . 52

3. Gaussian Measures in Hilbert Spaces . . . 54

CHAPTER 4 APPLICATIONS OF GAUSSIAN MEASURES 60 A. Random Variables on a Gaussian Hilbert Spaces . . . 60

B. Linear Random Variables on a Gaussian Hilbert Spaces . . . 63

C. Equivalence Classes of Random Variables . . . 64

D. The Cameron Martin Space and The White Noise Mapping . . . 67

E. The Malliavin Derivative . . . 73

F. Approximation by Exponential Functions . . . 74

G. The Malliavin-Sobolev SpaceD1,2♣H _,µq _{. . . . 76}

H. Brownian Motion in Gaussian Hilbert Space . . . 79

CHAPTER 5 CONCLUSIONS AND RECOMMENDATIONS 85 A. Conclusions . . . 85

B. Recommendations . . . 86

CHAPTER 1

INTRODUCTION

A. Research Background

In Euclidean geometry we need the concept of ameasureto know the measureµ♣Sqof

a solid bodyS. The Lebesgue measure generalizes the concept of the usual measure in

Euclidean geometry. In one, two, and three dimensional Euclidean spaces, we refer to

this measure aslength,area, andvolumeofS, respectively. The classical idea to build

measure in Euclidean space is by partitioning the body we measured into a finitely

many components and then applying some rigid motions into those components to

form a simpler body which presumably has the same measure. After the presence of

analytic geometry, Euclidean geometry became interpreted as the study of Cartesian

productRnof the real line R. Using this analytic foundation rather than the classical geometrical one, it is no longer intuitively obvious how to measure any subsets ofRn. This is what we learn inthe theory of measure, since we want to keep some nice

prop-erties of a measure (e.g. invariant under isometries), so we restrict ourselves to only

measure some nice subsets ofRn (i.e. σ-algebra) instead of all subsets ofRn. In the real line, the existence of such non-measurable set can be seen as follows: sinceQis

an Abelian additive subgroup ofR, then the quotient groupR④Qform a partition ofR

into disjoint cosets. Next, by the denseness of each cosetAPR④Q, we can have the

following set of all coset representativesV ✏ txA:xAPA❳ r0,1sandAPR④Q✉called

Vitali set. Using the countable additivity and translation invariant properties of a

mea-sure, it is easy to show that this set is indeed a non-measurable set. Unfortunately, not

all measures we have in finite dimensional space can be extended in a naive way to

the infinite dimensional spaces. As we will discuss in the next chapter that the only

locally finite and translation invariant Borel measureµ on an infinite dimensional

measure in infinite dimensional spaces. In this thesis we also give an overview to the

above discussed problem on measures in infinite dimensional spaces. First we will

recall some important concepts from measure theory and functional analysis and then

we shall construct the Gaussian measures in the finite and infinite dimensional spaces.

B. Research Problems

There are several problems to be answered in this research:

1. What is the motivation behind measure in infinite dimensional spaces?

2. How to construct a Gaussian measure in infinite dimensional separable Hilbert

spaces?

3. How to define random variables in Gaussian Hilbert spaces and what are their

properties?

4. How to use Gaussian measure to define and study the Cameron-Martin space

and white noise mapping?

5. How to formulate the Malliavin derivative of functions in Malliavin-Sobolev

space?

6. How to construct Brownian motion B ✏Bt, t P r0,Ts, in a Gaussian Hilbert

space?

C. Problem limitation

There are two limitations in this research as follows:

1. The state space of the Gaussian measure discussed in this thesis is limited to

2. The applications of the Gaussian measure discussed in this thesis are limited

to the use of Gaussian measure to study random variables on a Gaussian Hilbert

space, the Cameron-Martin space and white noise mapping, the Malliavin

deriva-tive, and construction of a Brownian motion in a Gaussian Hilbert space.

D. Research Objectives

The objective of this research is to understand the theoretical background of Gaussian

measures in infinite dimensional Hilbert spaces and some of their applications.

E. Research Benefits

The benefits of this research are the writer can obtain more knowledge about measure

in infinite dimensional spaces and its applications and also as a reference for further

study on this topics.

F. Research Methods

The method of this research is by literature review method, that is, by reading some

books and papers related to the topic of this research.

G. Systematics Writing

CHAPTER I. INTRODUCTION

A. Research Background

B. Research Problems

C. Problem Limitation

E. Research Benefits

F. Research Methods

G. Systematics Writing

CHAPTER II. TOOLS FROM MEASURE THEORY AND FUNCTIONAL

ANALY-SIS

A. Tools from Measure Theory

B. Tools from Functional Analysis

C. Events and Random Variables

D. Motivation to Measure in Infinite Dimensional Spaces

CHAPTER III. GAUSSIAN MEASURES IN HILBERT SPACES

A. Mean and Covariance of Measures in Hilbert Spaces

B. Law of a Random Variable

C. Gaussian Measures

CHAPTER IV. APPLICATIONS OF GAUSSIAN MEASURES

A. Random Variables on a Gaussian Hilbert Spaces

B. Linear Random Variables on a Gaussian Hilbert Spaces

C. Equivalence Classes of Random Variables

D. The Cameron Martin Space and The White Noise Mapping

E. The Malliavin Derivative

G. The Malliavin-Sobolev SpaceD1,2♣H,µq

H. Brownian Motion in Gaussian Hilbert Space

CHAPTER V. CONCLUSIONS AND RECOMMENDATIONS

A. Conclusions

B. Recommendations

CHAPTER 2

TOOLS FROM MEASURE THEORY AND FUNCTIONAL

ANALYSIS

In this chapter we will discuss some important concepts from measure theory and

functional analysis that play important role in the next chapters.

A. Tools from Measure Theory

Definition 2.1.1 (Sigma Algebra). Let _Ω be a nonempty set, we call F ❸₂Ω _{a σ}

-algebra on_Ωif it satisfies the following axioms

(i) (Empty set)❍ PF.

(ii) (Complement) If APF_{, then A}c_PF.

(iii) (Countable unions) If A1, ...,An, ...PF, then

➈_✽

n✏1AnPF.

We also call the ordered pair♣_Ω,Fq_{as a measurable space and A}PF _{as a}

measur-able set.

This definition shows that aσ-algebraF _{is a countable version of a (concrete) Boolean}

algebra on _Ω. From these axioms, it is clear that the trivial algebra t❍,_Ω✉ is the smallestσ-algebra and the discrete σ- algebra 2Ω is the largest σ-algebra on_Ω. We

can also deduce from the generalized de Morgan’s law thatF _{is closed under countable}

intersections.

It is easy to see that the intersection (finite, countably infinite, or even uncountable)

of arbitraryσ-algebras on_Ωis again a σ-algebra on_Ω. Therefore, we can obtain the

following definition.

Definition 2.1.2 (Generation ofσ-algebras). Let _Ω be a nonempty set and G ❸2Ω.

we call it theσ-algebra generated byG. Clearly, σ♣Gqis the smallestσ-algebra on

ΩcontainingG_{. In particular, we call theσ-algebra generated by all open subsets of}

Ω, denoted byB♣_Ωq_{, as the Borel σ-algebra onΩ, i.e.,}B♣_Ωq ✏_σ♣_τq _{where τ is a}

topology on_Ω.

From the definition above, we can directly show that if P♣Aq=”the setAhas

prop-ertyP” and satisfies the following axioms

(i) P♣Aqis true for everyAPG_.

(ii) P♣❍qis true.

(iii) IfP♣Aqis true for someA❸_Ω, thenP♣Acqis also true. (iv) IfAn❸ΩandP♣Anqis true for everynPN, thenP♣

➈

nPNAnqis also true,

thenP♣Aqis true for everyAPσ♣Gq.

Definition 2.1.3 (Dynkin’s π✁λ system). Let _Ω be a nonempty set and A,B be

collections of subsets of_Ω. We callA _{aπ-system if A1}❳_A2PA _{for every A1,A2}PA_.

We callB _{aλ-system if it satisfies the following axioms}

(i) (Empty set)❍ PB_.

(ii) (Complement) If BPB_{, then B}cPB_.

(iii) (Countable disjoint unions) If B1, ...,Bn, ...PBand if Bi❳Bj✏ ❍for i✘ j, then

➈_✽

n✏1BnPB.

Clearly a λ-system which is also a π-system is a σ-algebra since ➈_n✽_✏1Bn ✏B1❨

♣B2③B1q ❨ ♣B3③B2③B1q ❨... PB for any sequence ♣BnqnPN in B. Moreover, every

λ-systemB is closed under proper differences, i.e., ifB1,B2PB withB2❸B1, then

B1③B2PB. Now we will prove the following lemma before stating the Dynkin’sπ✁λ

Lemma 2.1.1. LetA be aπ-system of_Ωand l♣Aqbe the smallestλ-system contain-ingA _{as a subset. Then, l}♣Aq_{is aσ-algebra.}

Proof. It suffices to prove that l♣Aqis a π-system. Set lA✏ tB❸Ω:A❳BPl♣Aq✉

for any AP l♣Aq. It is clear that ❍ PlA and lA is closed under countable disjoint

unions. Next, ifBPlA, then A❳Bc✏A③♣A❳Bqis a proper difference of sets in the

λ-system l♣Aq_{, and so is inlA. This shows thatlA is aλ-system for anyA}P_l♣Aq_.

Now supposeAPA_{. ThenA}❳_BPA ❸_l♣Aq_{for everyBin theπ-system}A_{. Hence,}

BPlA and thuslA❹A. But sincel♣Aqis the smallestλ-system containingA, then

we havelA❹l♣Aq. Therefore,A❳BPl♣Aqfor everyBPl♣Aq. Finally, letBPl♣Aq

and considerlB✏ tA❸Ω:A❳BPl♣Aq✉. Using our result before, we conclude that

lB is aλ-system andlB❹A. But this meansA❳BPl♣Aqfor everyAPl♣Aq. Since

BPl♣Aq_{is arbitrary, then the result follows.}

Theorem 2.1.1 (Dynkin’sπ✁λ theorem). LetA _{be aπ-system of subsets ofΩand}

B_{aλ-system of subsets ofΩsuch that}A ❸B_{. Then,σ}♣Aq ❸B_.

Proof. Let _Ωbe a nonempty set andA,B be π-system and λ-system of_Ω, respec-tively, such that A ❸B_{. Then, by definition of l}♣Aq_{as the smallest λ-system}

con-taining A_{, we have l}♣Aq ❸B_{. From the previous lemma, we know that l}♣Aq_{is a}

σ-algebra, and since A ❸_l♣Aq_{, we have σ}♣Aq ❸_l♣Aq_{. This completes the proof,}

sincel♣Aq ❸B_.

Definition 2.1.4(Measure). Let♣_Ω,Fq_{be a measurable space. Ifµ:}F Ñ_{R , where}

R is the extended nonnegative real number lineR ✏ r0, ✽s, satisfies the following axioms

(i) (Null empty set)µ♣❍q ✏0

(ii) (Countable additivity) If A1, ...,An, ...is a sequence of pairwise disjoint sets in

F_{, thenµ}♣➈✽

n✏1Anq ✏

➦_✽

then we call µ a measure on ♣_Ω,Fq and ♣_Ω,F,µq a measure space. Ifµ♣Ωq ✏1, then we callµ a probability measure and♣_Ω,F_,µq_{a probability space.}

Definition 2.1.5(Outer Measure). Let_Ωbe an arbitrary set. A set functionµ✝: 2ΩÑ R is called an outer measure on_Ωif it satisfies the following axioms

(i) (Null empty set)µ✝♣❍q ✏0.

(ii) (Monotonicity) If A1,A2P2Ω such that A1❸A2, thenµ✝♣A1q ↕µ✝♣A2q.

(iii) (Countable subadditivity) If A1, ...,An, ...P2Ω, thenµ✝♣

➈_✽

n✏1Anq ↕

➦_✽

n✏1µ✝♣Anq.

From the third axiom, an outer measure µ✝ is countably subadditive on theσ-algebra 2Ω. If it is also additive on_Ω, i.e.,µ✝♣A1❨A2q ✏µ✝♣A1q µ✝♣A2qfor everyA1,A2P

2Ωsuch thatA1❳A2✏ ❍, then we haveµ✝♣

➈_✽

n✏1Anq ➙µ✝♣

➈N

n✏1Anq ✏

➦N

n✏1µ✝♣Anq

for every N PN and pairwise disjoint sets A1, ...,An, ...P2Ω so that µ✝♣

➈_✽

n✏1Anq ➙

➦_✽

n✏1µ✝♣Anq. Using this result together with the countable subadditivity of µ✝, we

conclude thatµ✝ is countably additive on 2Ω so that it is a measure on theσ-algebra 2Ω. The above definition also allow us to define the concept of measurability. A subset

Aof_Ωis said to be measurable with respect toµ✝ (orµ✝-measurable) if it satisfies the following Caratheodory condition

µ✝♣Tq ✏µ✝♣T❳Aq µ✝♣T❳Acqfor any setT P2Ω.

Definition 2.1.6(Covering class of a set). Let_Ωbe an arbitrary set andA ❸2Ω be a collection of subsets of_Ωsuch that

(i) (Empty set)❍ PA_.

(ii) (Countable covering) _Ω has a countable covering in A_{, i.e., there exist a}

se-quence A1, ...,An, ...PA such that

➈_✽

n✏1An✏Ω,

Clearly, the first axiom implies that every subsetAof_Ωhas a countable covering, i.e.,

containing all numbers in R on which we take the infimum to obtain µ✝♣A1q and

µ✝♣A2q, respectively. Then,M❸Nand consequentlyµ✝♣A1q ✏infM↕infN✏µ✝♣A2q

and sinceε→0 is arbitrary, the conclusion follows.

Definition 2.1.7(Lebesgue measure). Let I0be the collection of❍and all open

measure onR,µ✝: 2RÑ r0, ✽s, as follows

We call the restriction of µ✝ to the collection of all Lebesgue measurable (i.e. mea-surable with respect to µ✝) sets inRas the Lebesgue measure onR. We also call the corresponding measure space as the Lebesgue measure space.

Lemma 2.1.2. For every countable subset A ofR, we haveµ✝♣Aq ✏0.

Proof. First we will show that every singleton has measure zero. Let xPRand ε →

If not, then am 1➙bm for some m. Since Jn❳I ✘ ❍ for every n, then there exist

xm PJm❳I and xm 1 PJm 1❳I such that am ➔xm ➔bm↕am 1 ➔xm 1 ➔bm 1.

SinceI is an interval andxm,xm 1PI, we haverxm,xm 1s ⑨I. Consider the following

two possibilities. If bm✏am 1, then bmPI but bm❘

➈N

i✏1Jni. If bm➔am 1, then

rbm,am 1s ⑨I butrbm,am 1s ❶

➈N

i✏1Jni which contradicts the fact that♣Jniq

N

i✏1is an

open cover ofI. Thus we have am 1➔bmfor every mP tn1, ...,nN✉. So we have the

following inequalities

➳

nPN

δ♣Inq ➙

➳

nPN

δ♣Jnq ➙ N

➳

i✏1

δ♣Jniq ✏ ♣bn1✁an1q ... ♣bnN✁1✁anN✁1q ♣bnN✁anNq

➙ ♣an2✁an1q ... ♣anN✁anN✁1q ♣bnN✁anNq

➙bnN✁an1

➙b✁a

✏δ♣Iq,

i.e., δ♣Iq is a lower bound of ➦_nPNδ♣Inq so that µ✝♣Iq ➙δ♣Iq and the conclusion

follows. Second, letI✏ ♣a,bqbe a finite open interval. Using the previous result and the monotonicity and subadditivity ofµ✝ as an outer measure, we have the following inequalities

µ✝♣♣a,bqq ↕µ✝♣ra,bsq ↕µ✝♣ta✉q µ✝♣♣a,bqq µ✝♣tb✉q ✏µ✝♣♣a,bqq

which implies thatµ✝♣♣a,bqq ✏µ✝♣ra,bsq ✏δ♣ra,bsq ✏δ♣♣a,bqq. Third, ifIis a finite interval of the form I ✏ ♣a,bs, then we have µ✝♣♣a,bqq ↕ µ✝♣♣a,bsq ↕ µ✝♣♣a,bqq

µ✝♣tb✉q ✏µ✝♣♣a,bqqso that µ✝♣♣a,bsq ✏µ✝♣♣a,bqq ✏δ♣♣a,bqq ✏δ♣♣a,bsq. Clearly forI✏ ra,bqwe also have µ✝♣ra,bqq ✏δ♣ra,bqq. Last, ifI is an infinite interval, say of the formI✏ ♣a, ✽q, then we haveµ✝♣♣a, ✽qq ➙µ✝♣♣a,nqq ✏δ♣♣a,nqq ✏n✁a

for any other infinite intervalI.

Definition 2.1.8(Translation Invariance). Let_Ωbe a vector space,♣_Ω,F_,µq_a

mea-sure space, and A x✏ ta x:aPA✉for any APF _{and x}P_{Ω. Then}

(i) Theσ-algebra F _{is called invariant under translation if for every measurable}

set A and xP_Ω, A x is also measurable.

(ii) The measure µ is called invariant under translation if F is invariant under translation and for every measurable set A and xP_Ω, A x has the same measure

as A itself.

(iii) ♣Ω,F,µqis called a translation invariant measure space if both F _{and µ are}

invariant under translation.

Lemma 2.1.3(Translation invariance of the Lebesgue outer measure). Lebesgue outer

measureµ✝onRis invariant under translation.

Proof. Clearly, theσ-algebra 2Ris invariant under translation. LetAP2RandxPR.

Take an arbitrary open interval covering ♣InqnPN of Ain I0, then we have

➈_✽

n✏1♣In

xq ✏ ♣➈✽_n✏1Inq x❹A and hence

➦_✽

n✏1δ♣Inq ✏

➦_✽

n✏1δ♣In xq ➙µ✝♣A xq. Since

♣InqnPN is an arbitrary open interval cover of A, then µ✝♣Aq ➙µ✝♣A xq from the

definition of µ✝ as the infimum of ➦✽_n✏1δ♣Inq. Conversely, by applying the similiar

method to the setA xand its translate♣A xq✁x✏A, we have the reverse inequality

µ✝♣A xq ➙µ✝♣Aqand the conclusion follows.

Theorem 2.1.4(Translation invariance of the Lebesgue measure space). The Lebesgue

measure space is translation invariant.

Proof. LetAbe a Lebesgue measurable set andxPR, then for any subsetT ofR, we

have

✏µ✝♣♣T✁xq ❳Aq µ✝♣♣T✁xq ❳Acq

✏µ✝♣T✁xq

✏µ✝♣Tq

by using Lemma 2.1.2 and the fact that Ais measurable. This shows that A xalso

satisfies the Caratheodory condition and therefore it is Lebesgue measurable. By using

this result together with lemma 2.1.2, we also conclude that

µ♣A xq ✏µ✝♣A xq ✏µ✝♣Aq ✏µ♣Aq,

i.e., the Lebesgue measureµ is translation invariant.

Definition 2.1.9 (Complete measure space, complete extension, and completion of a

measure space). A measure space ♣_Ω,F,µq is called a complete measure space if every subset A0 of a null set (i.e. a set with measure zero) APF has measure zero.

If there exist a complete measure space ♣_Ω,F_0,µ0q _{such that}F₀❹F _{and µ0}✏_µ

onF_{, then we say that} ♣_Ω,F_0,µ0q_{is a complete extension of} ♣_Ω,F_,µq_{. If it is the}

smallest complete extension of♣_Ω,F_,µq_{, i.e, if for any complete extension}♣_Ω,F_1,µ1q

of♣_Ω,F,µqwe haveF₁❹F_0andµ1✏_µ0onF_{0, then we call it the completion of}

the corresponding measure space.

Theorem 2.1.5. The Lebesgue measure space is complete.

Proof. First, notice that ifAP2Rhas Lebesgue outer measure 0, then it is Lebesgue

measurable. Clearly, for any testing setT P2R, we have 0↕µ✝♣T❳Aq ↕µ✝♣Aq ✏0 and henceµ✝♣T❳Aq ✏0. Therefore,µ✝♣Tq ↕µ✝♣T❳Acq ✏µ✝♣T❳Aq µ✝♣T❳Acq

so thatAis Lebesgue measurable. Next, assume thatAhas Lebesgue measure zero and

B⑨A, thenµ✝♣Bq ↕µ✝♣Aq ✏µ♣Aq ✏0 so thatBis Lebesgue measurable.

count-able open intervals, it follows that every Borel set is Lebesgue measurcount-able. The Borel

measure space onRis obtained by restricting the domain of the Lebesgue measureµ

from the Lebesgueσ-algebra to the Borelσ-algebra onR. We call such a measure as

Borel measure onRand denote it by µB. The previous theorem then implies that the

Lebesgue measure space is a complete extension of the Borel measure space.

Definition 2.1.10(Measurable function). Let♣X,Aq_and♣_Y,Bq_{be measurable spaces.}

A function f :X ÑY is called measurable (B✁A-measurable) if for all BPB, we have f✁1♣Bq PA_.

The following theorem simplifies our problem of checking the measurability of a

function from the entireσ-algebraB _{to only its generator.}

Theorem 2.1.6. Let♣X,Aq_,♣_Y,Bq_{be measurable spaces and}G ❸₂Y _{be the generator}

ofB_{, i.e.,}B✏_σ♣Gq_{. Then a function f} :XÑY is measurable if and only if f✁1♣Gq P

A for every GPG.

Proof. Clearly if f is measurable, then f✁1♣Gq PA _{for every G}PG_{. Define} M ✏

✥

MPB_{: f}✁1_{♣Mq PA}✭_{. It is easy to show that M is aσ-algebra as follows: since}

f✁1♣Yq ✏XPA_{, thenY} PM_{. IfM}PM_{, then f}✁1♣_Mq PA_{, and hence}♣_f✁1♣_Mqqc✏

f✁1♣Mcq PA _{so that M}cPM_{. Finally, ifM1}, ...,Mn, ...PM, then f✁1♣Miq PA for

everyiPNwhich implies that➈n_i✏1f✁1♣Miq ✏ f✁1♣

➈n

i✏1♣Miqq PA. Therefore, since

G ❸M, thenσ♣Gq ❸_σ♣Mq, i.e,B❸M and the conclusion follows.

Definition 2.1.11(Simple function). A function is called a simple function if its range

is a finite set.

AnR -valued simple functionφ always has a representation φ ✏➦_kn_✏1ak1Ek where

ak PR and Ek✏φ✁1♣tak✉q. This definition of simple functions play a fundamental

Theorem 2.1.7(Approximation by simple functions). If f :XÑR is a nonnegative

measurable function, then there is a monotone increasing sequence of nonnegative

simple functionsϕn:XÑR such thatϕnÑ f pointwise as nÑ ✽.

Note that every function f can be written as a sum of two positive functions,

i.e., f ✏ f ✁f✁ where f ✏maxtf,0✉and called the positive part of f and f✁✏

maxt✁f,0✉and called the negative part of f. This means that we can prove that some properties hold for f just simply by proving that it is true for the corresponding simple

functions.

Definition 2.1.12(Lebesgue Integral). Let♣_Ω,F_,µq_{be a measure space. The Lebesgue}

integral over_Ωof a measurable simple functionφ :_ΩÑR is defined as

➺

If f :_ΩÑR is a nonnegative measurable function, then the Lebesgue integral of f

over_Ωis defined as➩_Ω f dµ ✏sup✥➩_Ωφdµ:φis simple and 0↕φ ↕ f✭. Finally, for

any measurable function f :_ΩÑR✏R❨t✁✽, ✽✉, the Lebesgue integral of f over

Ωis defined as➩_Ω f dµ ✏➩_Ω f dµ✁➩_Ω f✁dµ.

Theorem 2.1.8(Monotone Convergence Theorem). Let♣fnqnPNbe a sequence of

Proof. Since fn is measurable and increasing pointwise to f, then f is measurable.

thermore, from the continuity of a measure and monotonicity of the Lebesgue integral,

we have

Ω fndµ. Finally by taking the supremum ofϕin the last inequality we obtain

the desired result.

Definition 2.1.13(Integral of an almost everywhere defined function). Let♣_Ω,F,µq

be a measure space and f :A③NÑR be a measurable function on A③N where A,NP

extended real valued measurable function defined on A by setting

The definition above of➩_A f dµ for a function f which is defined only a.e. (almost everywhere) onAis for convenience of not having to write➩_A③N f dµ.

Theorem 2.1.9(Fatou’s lemma). Let♣_Ω,F_,µq_{be a measure space and}♣_fnq_{nPNbe an}

arbitrary sequence of nonnegative extended real valued measurable functions on a set

APF_{, then we have}

increasing and hence by the monotone convergence theorem

➺

mea-sure space and fn:X ÑR be a sequence of measurable functions such that fnÑ f

pointwise as nÑ ✽. Suppose that there exist an integrable function g such that⑤fn⑤ ↕

g for every nPN. Then fnand f are also integrable andlimnÑ ✽

Since fnÑ f, the left hand side of the inequality above is just

➩

Remark 2.1.1. Here are some remarks about the Lebesgue dominated convergence

theorem

(i) The hypothesis can be relaxed becomes fnÑ f a.e. or⑤fn⑤ ↕g a.e.

(ii) By the triangle inequality, it is clear that limnÑ ✽

➩

Ω fndµ ✏

➩

Ω f dµ.

Theorem 2.1.11(Beppo-Levi). Let♣_Ω,F,µqbe a measure space and fn:ΩÑ r0, ✽s

be a sequence of nonnegative measurable functions. Then

Proof. Let gN ✏

➦N

n✏1fn and g✏

➦

nPNfn. By the monotone convergence theorem,

we have

➺

Ω

gdµ ✏ ➺

Ω

lim

NÑ ✽gNdµ✏NÑ ✽lim

➺

Ω

gNdµ ✏ lim

NÑ ✽

N

➳

n✏1

➺

Ω

fndµ ✏

✽

➳

n✏1

➺

Ω

fndµ.

B. Tools from Functional Analysis

Definition 2.2.1 (Normed space). A normed space is a vector space X over a field

F P tR,C✉equipped with a function⑥ ☎ ⑥:X ÑR, called a norm, such that for every x,yPX andλ PF the following axioms hold

(i) (Nullity)⑥x⑥ ✏0implies x✏0.

(ii) (Homogeinity)⑥λx⑥ ✏ ⑤λ⑤⑥x⑥.

(iii) (Triangle inequality)⑥x y⑥ ↕ ⑥x⑥ ⑥y⑥.

From these axioms, we can easily deduce that⑥x⑥ ➙0 and⑥x⑥ ✏0 if and only ifx✏0.

Definition 2.2.2(Metric space). A metric space is a nonempty set X equipped with a

function d :X✂X Ñ R, called a metric, such that for every x,y,zPX the following axioms hold

(i) (Nullity) d♣x,yq ✏0if and only if x✏y.

(ii) (Symmetry) d♣x,yq ✏d♣y,xq.

(iii) (Triangle inequality) d♣x,yq ↕d♣x,zq d♣z,yq.

From the third axiom, we also can easily deduce thatd♣x,yq ➙0 for everyx,yPX. Also notice that every normed space ismetrizable, i.e., we can generate a metricd♣x,yq:✏

used when we talk about some metric-related objects (e.g. convergence of a sequence)

in a normed space.

Definition 2.2.3 (Sequence in Metric Space, Complete Metric Space). A sequence

♣xnqnPNin a metric space X is called converges to a point xPX if for everyε→0, there

exist NPNsuch that d♣xn,xq ➔ε for every n➙N. A sequence♣xnqnPNin X is called a

Cauchy sequence if for everyε→0, there exist NPNsuch that d♣xn,xmq ➔εfor every

n,m➙N. In the case where every Cauchy sequence in X converges to a point in X , we called X a complete metric space.

Notice that although in the real/complex space every Cauchy sequence is convergent,

a Cauchy sequence in a metric space does not need to be convergent. For example,

consider the sequencexn✏1_n,nPNin the metric space of real interval♣0,1qwith the

usual metric d♣x,yq ✏ ⑤x✁y⑤ which is Cauchy but not convergent to an element in

♣0,1q.

Definition 2.2.4 (Inner Product). Let X be a vector space over a field F. An inner

product on X is a map ①☎,☎②:X✂X Ñ F that satisfies the following axioms for any x,y,zPX andλ PF.

(i) (Conjugate Symmetry)①x,y② ✏ ①y,x②.

(ii) (Distributive)①x y,z② ✏ ①x,z② ①y,z②.

(iii) (Homogeneity)①λx,y② ✏λ①x,y②.

(iv) (Positive definiteness)①x,x② ➙0and①x,x② ✏0if and only if x✏0.

Definition 2.2.5 (Banach and Hilbert space). Banach space is a complete normed

space, i.e., a normed space where is also a complete metric space. In the case that

the norm is induced by an inner product, i.e.,⑥x⑥ ✏ ①x,x②12 for some inner product①☎,☎②

Definition 2.2.6(Orthonormal Basis of Hilbert Spaces). Let♣eiqiPI be a collection of

vectors inH_{. We call}♣_eiq_{iPI an orthonormal basis of}H _if①_ei,ej② ✏_{0for i}✘ _{j and}

①h,ei② ✏0for every iPI implies that h✏0.

Denote byCb♣Hqthe space of all continuous bounded mappings fromH intoR

endowed by the norm⑥f⑥0✏supxPH ⑤f♣xq⑤for any f PCb♣H q.

Proposition 2.2.1. Cb♣Hqis a Banach space.

Proof. Let ♣fnqnPN be a Cauchy sequence inCb♣Hq. Since ⑤fn♣xq ✁ fm♣xq⑤ ↕ ⑥fn✁

fm⑥0, the sequence ♣fn♣xqqnPNis a Cauchy sequence for any xPH. SinceRis

com-plete, its limit is in R and hence the pointwise limit f♣xq ✏limnÑ ✽fn♣xq is an

R-valued function. Now we will show that the limit f is a bounded function as

fol-lows: Letε→0 andN be a positive integer such that⑥fn✁fm⑥0➔ε for alln,m➙N.

Then, the inequality ⑤f♣xq⑤ ↕ ⑤f♣xq ✁ fN♣xq⑤ ⑤fN♣xq⑤ ↕ ⑥f ✁fN⑥0 ⑥fN⑥0 holds for

allxPH . But since forn➙N,⑤fn♣xq ✁ fN♣xq⑤ ➔ε for everyx, then⑤f♣xq ✁fN♣xq⑤ ✏

limnÑ ✽⑤fn♣xq ✁ fN♣xq⑤ ↕ε so that ⑥f ✁ fN⑥0 ✏supxPH ⑤f♣xq ✁ fN♣xq⑤ ↕ε. This

shows that the function f is bounded. Letx0in H. By the continuity of fN, choose

δ →0 such that ⑤fN♣xq ✁ fN♣x0q⑤ ➔ε whenever ⑥x✁x0⑥ ➔δ. Then if ⑥x✁x0⑥ ➔δ,

we have ⑤f♣xq ✁f♣x0q⑤ ↕ ⑤f♣xq ✁ fN♣xq⑤ ⑤fN♣xq ✁ fN♣x0q⑤ ⑤fN♣x0q ✁f♣x0q⑤ ↕ ⑥f ✁

fN⑥0 ⑤fN♣xq ✁ fN♣x0q⑤ ⑥fN✁f⑥0 ➔ε ε ε ✏3ε. This shows the continuity of

f. To finish the proof we need to show fn converges in norm, i.e. ⑥f ✁ fn⑥0 Ñ0

as n Ñ ✽. This is clear by using the triangle inequality as follows: ⑥f ✁ fn⑥0 ↕

⑥f✁fN⑥0 ⑥fN✁fn⑥0↕ε ε✏2ε so that⑥f✁fn⑥0Ñ0.

Theorem 2.2.1(The orthogonal decomposition theorem). LetH _{be a Hilbert space}

and S❸H _{a closed subspace of}H_{. Then the orthogonal complement S}❑ _{defined by}

S❑✏ txPX :①x,y② ✏0 for every y PS✉is also a closed subspace ofH _andH _can

Proof. It is clear thatS❑is a vector subspace ofH . LetxnPS❑,nPN, be a sequence

of vectors such thatxnÑx. Using the continuity of an inner product, we have for any

yPS, 0✏limnÑ ✽①xn,y② ✏ ①x,y② which implies that xPS❑. To prove the second

statement, we need to show thatS❳S❑✏ t0✉andS S❑✏H _{. Clearly, ifx}P_S❳_S❑_,

then①x,x② ✏0 which implies thatx✏0. The case thatS✏H _{is trivial sincex}✏_{x 0}

for everyxPH _andS❑✏ t₀✉_{. Assume thatS}⑨H _andxPH ③_{S, then we haved}✏

infyPS⑥x✁y⑥ →0. Now take a sequence♣ynqnPNinSsuch thatd✏limnÑ ✽⑥x✁yn⑥.

We will show that♣ynqnPNis Cauchy as follows

⑥yn✁ym⑥2✏ ⑥♣yn✁xq ♣x✁ymq⑥2

✏2♣⑥x✁yn⑥2 ⑥x✁ym⑥2q ✁ ⑥2x✁yn✁ym⑥2

✏2♣⑥x✁yn⑥2 ⑥x✁ym⑥2q ✁4⑥y✁

yn✁ym

2 ⑥

2

↕2♣⑥x✁yn⑥2 ⑥x✁ym⑥2q ✁4d2

which is converges to zero as n,mÑ ✽. This implies that there exist an element

y0PSsuch thatd✏ ⑥x✁y0⑥. Now consider the decompositionx✏y0 ♣x✁y0q. Letz

be an arbitrary vector inS, then for anycPR, we havey0 czPSand hence

d2↕ ⑥♣x✁y0q ✁cz⑥2✏ ①♣x✁y0q ✁cz,♣x✁y0q ✁cz②

✏ ⑥x✁y0⑥2✁c①x✁y0,z② ✁c①z,x✁y0② c2⑥z⑥2

✏d2✁2cRe①x✁y0,z② c2⑥z⑥2.

Therefore, we have✁2cRe①x✁y0,z② c2⑥z⑥2➙0 and so Re①x✁y0,z② ✏0 sincecis

arbitrary. On the other side, we also have Im①x✁y0,z② ✏ ✁Re①x✁y0,iz② ✏0 which

shows thatx✁y0PS❑.

Definition 2.2.7 (Linear Map). Let X and Y be two vector spaces over a field F. A

c1,c2 PF, we have T♣c1x1 c2x2q ✏c1T♣x1q c2T♣x2q. We will call a linear map

T :X ÑF as a linear functional.

Now since a normed space has both topological and algebraic structure, we can

talk about boundedness and continuity of a linear operator as follows.

Definition 2.2.8 (Bounded linear map). Let ♣X,⑥ ☎ ⑥Xq and ♣Y,⑥ ☎ ⑥Yq be two normed

spaces. A linear map T : X ÑY is called bounded if there exist M →0 such that

⑥T x⑥ ↕M for every xPX with⑥x⑥ ↕1. We denote the smallest such M by⑥T⑥and call

it the operator norm of T . Furthermore, we denote the set of all bounded linear maps

from X to Y by the symbolL♣_X,Yq_{. For X} ✏_{Y , we denote such a set by}L♣_Xq_.

It is easy to see that this definition is equivalent with saying thatT maps every bounded

sets to bounded sets. If the codomain of the mapT is complete, i.e., if it is a Banach

space, then the spaceL♣_X,Yqform a Banach space under operator norm as stated in the following proposition.

Proposition 2.2.2. If X is a normed space and Y is a Banach space, then L♣_X,Yq_is

a Banach space with respect to the operator norm⑥T⑥ ✏sup_⑥x⑥↕1⑥T x⑥.

Proof. LetL♣_X,Yqbe a vector space over fieldF P tR,C✉such that for any bounded linear operatorT,Swe have♣αT βSq♣xq ✏αT♣xq βS♣xqfor anyα,βPF_{. It is easy}

to show thatL♣X,Yqtogether with the operator norm⑥☎⑥is a normed space. To show its completeness, let♣TnqnPN be a Cauchy sequence inL♣X,Yq. Then for anyxPX,

the sequence♣TnxqnPNis a Cauchy sequence inY because of the following inequality:

⑥Tnx✁Tmx⑥ ↕ ⑥Tn✁Tm⑥⑥x⑥and⑥Tn✁Tm⑥ Ñ0 asm➙nandnÑ ✽. It follows that

limnÑ ✽Tnxexist for anyxPX. Denote the limit byT x, i.e.,T x✏limnÑ ✽TnxforxP

X. To show thatT is indeed a member ofL♣_X,Yq, we first show the linearity ofT. Let

x,yPX, thenT♣x yq ✏limnÑ ✽Tn♣x yq ✏limnÑ ✽♣Tnx Tnyq ✏limnÑ ✽Tnx

Cauchy in L♣X,Yq, then ♣⑥Tn⑥qnPN is also Cauchy in R by the triangle inequality.

Also, for eachxPX we have the inequalities ⑥T x⑥ ↕sup_nPN⑥Tnx⑥ ↕ ♣supnPN⑥Tn⑥q⑥x⑥

which shows thatT is bounded and⑥T⑥ ↕sup_nPN⑥Tn⑥.

Now we will prove an important proposition that gives a connection between bounded

and continuous operator.

Proposition 2.2.3. Let♣X,⑥ ☎ ⑥Xqand♣Y,⑥ ☎ ⑥Yqbe two normed spaces and T :X ÑY

a linear map. Then the following statements are equivalent

(i) T is uniformly continuous on X .

(ii) T is continuous at zero.

(iii) T is bounded.

Proof. (i)ñ(ii) trivial.

(ii) ñ (iii) Assume the contrary, i.e., T is continuous at zero but unbounded. The

unboundedness ofT implies that for every nPN, there existxnPX such that⑥xn⑥ ↕1

but⑥T xn⑥ →n. Since⑥x_nn⑥ ↕ 1_n for everyn, then x_nn converges to zero inX and hence

our assumption thatT is continuous at zero implies that⑥T xn

n ⑥ converges to zero. But

this contradict the fact that⑥T xn⑥ →nfor everyn.

(iii)ñ(i) Assume thatT is bounded, then⑥T x⑥ ↕ ⑥T⑥for every⑥x⑥ ↕1. Since_⑥x_x⑥↕1 for every xPX, then we have ⑥T_⑥x_x⑥⑥ ↕ ⑥T⑥. Consequently, ⑥T x⑥ ↕ ⑥T⑥⑥x⑥ and by additivity of T, ⑥T x✁Ty⑥ ✏ ⑥T♣x✁yq⑥ ↕ ⑥T⑥⑥x✁y⑥. Therefore, T is a Lipschitz

continuous function which is uniformly continuous.

Definition 2.2.9(Dual space). Let X be a normed space over field F. The (topological)

dual space of X is the spaceL♣_X,Fq_{, i.e., the space of all bounded linear functionals}

Definition 2.2.10(Separable space). A metric space X is called separable if it contains

a countable dense subset, i.e., there exist a sequence ♣xnqnPNin X such that for every

xPX andε→0, there exist nPNsuch that d♣xn,xq ➔ε.

Now we will present one of the famous theorem in Hilbert space that we will use

extensively throughout this thesis.

Theorem 2.2.2(Separability of Hilbert space). A Hilbert spaceH _{is separable if and}

only if it admits a countable orthonormal basis.

Proof. (ñ) First assume thatH is separable, then there exist a sequence♣xnqnPN⑨H

dense in H . Now we will construct a maximal subsequence B of ♣xnqnPN which is

linearly independent by using the induction method. Letn1 be the least positive

in-teger such that xn1 is nonzero. Next, assume that k✁1 linearly independent vectors

have already been selected. If for all n→nk✁1, the vectors xn1, ...,xnk_✁1,xn are

lin-early dependent, then we stop since we already get a maximal linlin-early independent

subsequence of ♣xnqnPN which is finite in this case. On the other side, if there exist

the smallest index n_k→n_{k 1} such that the vectors xn1, ...,xnk✁1,xnk are linearly

inde-pendent, then replacexn1, ...,xnk✁1 with the new subsequencexn1, ...,xnk✁1,xnk. Indeed,

using this construction we finally can obtain a maximal subsequenceBof♣xnqnPN, i.e.,

B✏ ♣xn1, ...,xnk_✁1, ...q which is linearly independent. Then, apply the Gram-Schmidt

process to transform the sequence of linearly independent vectors B into a sequence

of orthonormal vectorsB0✏ ♣un1, ...,unk✁1, ...qwhich satisfiesspanB✏spanB0. Also,

sinceBis a Hamel basis (i.e., a maximal linearly independent subset of♣xnqnPN), then

we obtain the following equalities

spanB0✏spanB✏spant♣xnqnPN✉

but this implies that the linear span ofB0is dense inV and hence its orthogonal

(ð)Now assume that♣unqnPNis a countable orthonormal basis forH , then for every

hPH _{can be represented as}

h✏

✽

➳

n✏1

knun

where the series being finite ifH _{is finite dimensional. Consider the set}

B✏

It is a direct consequence of the Cantor diagonalization theorem that the set Q2n ✏

t♣♣p1,q1q,♣p2,q2q, ...,♣p2n,q2nqq:pi,qiPQ✉ is countable for every positive integern

and hence we can regard B as a countable union of countable sets Q2n in order to deduce that B is countable. Furthermore, sinceQ is dense in R, then for anyε →0,

there existrn✏pn iqnwherepn,qnPQsuch that⑤kn✁rn⑤ ↕ ₂εn. Therefore,

Proposition 2.2.4 (Parseval Identity). Let ♣ejqjPN be an orthonormal basis for the

Hilbert spaceH_{. Then, for any x}PH _{, we have}➦

Proof. Consider the Fourier series representation of the vectorx. Then we have

⑥x⑥2✏ ①x,x② ✏ ①➳

jPN

xjej,

➳

jPN

xjej② ✏ lim NÑ ✽①

N

➳

j✏1

xjej, N

➳

j✏1

xjej②

✏ lim

NÑ ✽

N

➳

j✏1

xjxj✏

➳

jPN

xjxj

✏ ➳

jPN

①x,ej②①x,ej②

✏ ➳

jPN

⑤①x,ej②⑤2

Definition 2.2.11(Banach algebra). Banach algebra is a Banach space A which is also

an associativeF-algebra,FP tR,C✉, i.e., an additive Abelian group A which has the structure of both unital ring andF-module such that the scalar multiplication satisfies

α♣xyq ✏ ♣αxqy✏x♣αyq for every α PF and x,yP A. Moreover, the norm and the algebra multiplication satisfies the inequality: ⑥xy⑥ ↕ ⑥x⑥⑥y⑥for every x,yPA.

Definition 2.2.12 (Unital sub-algebra and separating points). Let K be a compact

metric space and C♣K,Rq be a Banach algebra equipped with the sup-norm ⑥f⑥0 ✏

sup_xPK⑤f♣xq⑤. We say that A ❸C♣K,Rq a unital sub-algebra if 1PA and if αf

βg,f gPA for any f,gPA andα,β PR. We say that A separates point of K if for any two elements x✘y of K, there exist f PA such that f♣xq ✘ f♣yq.

Definition 2.2.13 (Lattice). Let K be a compact metric space and A _{a subset of}

C♣K,Rq. We say that A _{is a lattice if for every f,g}PA_{, we have f} ❴_{g and f} ❫_g

are inA_{, where}♣_f❴_gq♣_xq ✏_maxt_f♣_xq_,g♣_xq✉_and♣_f❫_gq♣_xq ✏_mint_f♣_xq_,g♣_xq✉_.

Before we state the Stone-Weierstrass theorem, we will prove the following useful

Lemma 2.2.1. Let A ❸C♣K,Rq be a closed unital sub-algebra. Then we have the following three statements

(i) If f is a nonnegative function in A, then❄f is also an element of A.

(ii) If f PA, then⑤f⑤ PA.

(iii) A is a lattice.

Proof. (i) Since A is closed under scalar multiplication then we can assume that

0↕ f ↕1 by normalization. Hence, f can be written as f ✏1✁g for some

This Maclaurin series approximates❄f uniformly in⑥ ☎ ⑥0. Indeed,

✎✎

algebra. Moreover, by the closedness ofA, we conclude that❄f PAas desired.

(iii) This result comes directly from the following identities and by applying (ii)

f❴g✏ 1

2♣f g ⑤f✁g⑤q and f❫g✏ 1

2♣f g✁ ⑤f✁g⑤q.

The following theorem proved by Marshall H. Stone in 1937 generalizes the

Weier-strass approximation theorem which states that the polynomials are uniformly dense in

the space of countinuous real-valued function onra,bs,C♣ra,bs,Rq.

Theorem 2.2.3 (Stone-Weierstrass theorem). Let K be a compact metric space and

A⑨C♣K,Rq a unital sub-algebra which separates points of K. Then A is dense in C♣K,Rq.

Proof. Let f PC♣K,RqandA❸C♣K,Rqbe a closed unital sub-algebra which separates points ofK. Letε→0 be given, we will show that there existsgPAsuch that⑥f✁g⑥0➔

ε. Take arbitrary two different pointst1,t2PK. SinceAseparates points, there exists

hPAsuch thath♣t1q ✘h♣t2q. Now fors1,s2PR, defineh:KÑRgiven byh♣xq ✏s2

♣s1✁s2q_hh_♣♣_tx₁q✁_q✁h_h♣_♣t_t2₂q_q for everyxPK. ClearlyhPA,h♣t1q ✏s1, andh♣t2q ✏s2. Therefore,

for any t1✘t2, there exists ft1,t2 PA such that ft1,t2♣t1q ✏ f♣t1q and ft1,t2♣t2q ✏ f♣t2q.

Since ft1,t2 is continuous, ft1,t2 approximates f in neighborhoods oft1andt2. Now let

t1be fixed andt2 vary. DefineAt2 ✏ txPK⑤ft1,t2♣xq ➔ f♣xq ε✉, thenAt2 is open as a

preimage of an open set. Moreover, it also containst2which implies thattAt2✉t2PK is

an open cover ofK. SinceKis compact then there exists a finite subcoverAt2,1, ...,At2,n

such that K❸➈n_j✏1At2.j. Define ht1 ✏min1↕j↕nft1,t2,j, then from Lemma 2.2.1, we

haveht1PA. Clearly, we also haveht1♣t1q ✏ f♣t1qandht1 ➔ f ε. Now define an open

setBt1 ✏ txPK⑤ht1♣xq → f♣xq ✁ε✉. In a similar way as before, we haveK❸

➈

t1PKBt1.

Again, by the compactness ofK, there exists a finite subcover such thatK❸➈m_j✏1Bt1,j.

Putg✏max1↕j↕mht1,j, thengPAand f✁ε➔g➔ f ε, i.e.,⑥f✁g⑥0➔ε. Thus,Ais

Definition 2.2.14(Baire categories). Let X be a topological space. We say that X is of

the first category if X ✏➈_nPNXnwhere Xnis a nowhere dense subset (i.e., a set whose

closure has empty interior) of X . Otherwise, we say that X is of the second category.

Lemma 2.2.2. Let X and Y be two normed spaces and T PL♣_X,Yqbe such that the

range of T , R♣Tq, is of the second Baire category in Y. If N0X is a neighborhood of the

zero vector0X in X , then the closure T♣N0Xqis a neighborhood of the zero vector0Y

in Y .

Proof. Let 0X,0Y be the zero vector of X andY, respectively. It is clear that for an

arbitrary neighborhood N0X of 0X, we can always find a ball B✏Bε♣0Xq such that

B B❸N0X by taking the radius ε small enough. Also, since for every xPX, we

have limnÑ ✽1_nx✏0, then for everyxPX, there is a corresponding positive integer

nx such that xPnxB. This implies that the space X can be written as the union X ✏

➈

nxPN♣nxBq and hence the range ofT, R♣Tq ✏

➈

nxPNT♣nxBq. From the hypothesis,

sinceR♣Tq is of the second category, then there existn0PNsuch that T♣n0Bq is not

nowhere dense which means that it has nonempty interior. Because of the continuity

of multiplication by a nonzero scalar and multiplication by a nonzero scalar c has

a continuous inverse, namely multiplication by 1_c, then multiplication by a non-zero scalar is a homeomorphism so thatT♣n0Bq ✏n0T♣Bq ✏n0T♣Bq. This shows that the

interior ofT♣Bqis not empty. Therefore, there existy0PT♣Bqsuch thatBδ♣y0q ❸T♣Bq

for someδ →0. Now consider the ballB ✏_{Bδ, then we have}B ✏ ✁_{y0 Bδ}♣_y0q ❸

✁y0 T♣Bq ✏T♣✁x0q T♣Bq ✏T♣✁x0 Bq ❸T♣N0Xqsince✁x0 B❸B B❸N0X

and the conclusion follows.

In the following we will state one of the fundamental result in functional analysis

which is also known as the Banach-Schauder theorem.

Theorem 2.2.4(The open mapping theorem). Let X and Y be two Banach spaces and

for every open set A in X .

Proof. We shall denoteXε ✏Bε♣0Xqfor the open ball inX centered at 0X with radius

ε, andYε the same inY. For an arbitraryε→0, letεn✏₂εn,n✏ t0✉❨N. Using Lemma

2.2.2, there existλn whereλnÑ0 as nÑ ✽ such thatYλn ❸T♣Xεnq. Let yPBλ0.

Let us claim that there exist an element xPX2ε0 such that T x✏y. Indeed, from the

above assumption, we know that there existsx0PXε0 such that⑥y✁T x0⑥ ↕λ1 so that

y✁T x0PYλ1. Also, there existsx1PXε1 such that⑥y✁T x0✁T x1⑥ ➔λ2. In a similar

way, we obtain a sequencexnPXεnsuch that⑥y✁T♣

➦n

i✏0xiq⑥ ➔λn 1. Ifyn✏

➦n k✏1xk,

then⑥yn✁ym⑥ ✏ ⑥

➦n

k✏m 1xk⑥ ↕

➦n

k✏m 1⑥xk⑥ ↕

➦n

k✏m 1εk↕

➦n

k✏m 12✁k

✟

ε0which

shows thatynis Cauchy. Hence, by the completeness ofX,ynÑxasnÑ ✽. Also,

since norm is continuous, we have⑥x⑥ ✏limnÑ ✽⑥

➦n

k✏0xk⑥ ↕limnÑ ✽

➦n

k✏0⑥xk⑥ ↕

➦n k✏02✁k

✟

ε0 ✏2ε0. Passing to the limit as nÑ ✽, we get ⑥y✁T x⑥ ✏0 so that

y✏T x. Sinceyis an arbitrary element ofBλ0, then we have shown thatBλ0❸T♣X2εq.

LetGbe a nonempty set inX and letxPG. By the openness ofG, there existε →0

such thatx X2ε ❸G. ConsequentlyT x Bλ0 ❸T x T♣X2εq ✏T♣x X2εq ❸T♣Gq,

i.e., for any xPG, T♣Gq contains a neighborhood of T x which shows that T♣Gq is

open.

The following theorem is a direct consequence of the open mapping theorem.

Theorem 2.2.5(Bounded inverse theorem). Let X,Y be two Banach spaces and T P

L♣X,Yqis bijective, then T is a homeomorphism.

Proof. Since R♣Tq ✏Y, then T is open by the open mapping theorem, so that the

preimage of every open set inXunderT✁1:Y ÑX is an open set inY and the conclu-sion follows.

If X,Y are two normed spaces with norms⑥ ☎ ⑥X and⑥ ☎ ⑥Y, respectively, thenX✂

⑥y⑥_Ypq1p_{for 1}↕ _p➔ ✽_{. Until now we only consider linear transformations from a}

vector space X to a vector spaceY, i.e., their domain of definition coincide with the

entire spaceX. Now we will also consider a transformation defined on a proper

sub-space ofX which we also called an operator.

Definition 2.2.15(Graph, closed, and closable operators). Let X and Y be two normed

spaces and T :D♣Tq ⑨X ÑY an operator defined on its domain of definition D♣Tq.

The graph of T , denoted by G♣Tq, is defined as G♣Tq ✏graph T ✏ t♣x,T xq:xPD♣Tq✉

❸X✂Y . Moreover, the operator T is said to be closed if its graph G♣Tqis a closed

subspace of X✂Y . If the closure of graph of T , G♣Tq ❸X✂Y , can be identified as a

graph of a linear operator T , then T is said to be closable and we call T the closure of

T .

Proposition 2.2.5. Let X,Y be two normed spaces and T :D♣Tq ❸X ÑY a linear

operator. Then T is closable if and only if for every sequence ♣xnqnPN ❸D♣Tq such

that xnÑ0and T xnÑy implies y✏0.

Proof. Let♣xnqnPNbe a sequence inD♣Tqsuch thatxnÑ0 andT xnÑy, then♣xn,T xnq Ñ

♣0,yqasnÑ ✽. LetT be the closure ofT, then we have ♣0,yq PG♣Tq ✏G♣Tqso thaty✏T♣0q ✏0 by the linearity ofT. For the converse, define

D♣Uq ✏

✦

xPX:❉yPY,♣x,yq PG♣Tq

✮

,

i.e., the projection ofG♣Tq onX. It is easy to see thatD♣Uqis a subspace ofX. Let

xPD♣Uqand assume that there existy1,y2PY such that♣x,y1q,♣x,y2q PG♣Tq. Then,

there exist two sequencespn,qninD♣Tqsuch that♣pn,T pnq Ñ ♣x,y1qand♣qn,T qnq Ñ

♣x,y2q asnÑ ✽. By letting rn✏ pn✁qn, we have rnÑ0 andTrnÑy1✁y2✏0

by our hypothesis. Therefore, y1 ✏y2 which shows that for every xP D♣Uq there

where Sx✏y such that ♣x,yq PG♣Tq. Similar as before, let p,qPD♣Uq, then there exist two sequencespn,qninD♣Tqsuch that♣α1pn α2qn,T♣α1pn α2qnqq Ñ ♣α1p

α2q,α1U p α2U qq. This shows the linearity ofU, and the conclusion follows.

We will state a fundamental result concerning closed operators due to Banach

which shows that the domain of every noncontinuous closed operators T ✘0 cannot

be the entire space.

Theorem 2.2.6(Closed graph theorem). Let X and Y be Banach spaces and T a closed

linear operator from X to Y with D♣Tq ✏X , then T is continuous.

Proof. If♣xn,ynqnPNis a Cauchy sequence inX✂Y, then♣xnqnPNis a Cauchy sequence

inX and♣ynqnPNis a Cauchy sequence inY. Hence if bothX andY are complete, then

there existxPX and yPY such that♣xn,ynq Ñ ♣x,yqby using the definition of norm

in the Cartesian spaceX✂Y. Thus, we conclude that ifX andY are Banach spaces,

then X✂Y is also a Banach space. Moreover, since the operator T is closed, then

G♣Tqis a closed subspace of a complete spaceX✂Y which is also complete. Define a

projectionpX :G♣Tq ÑXwherepX♣♣x,T xqq ✏x, then clearlypXis a bijection between

G♣Tqand X and is continuous. Moreover, the bounded inverse theorem implies that

pX has a continuous inverse p✁_X1. Define also the second projection pY :G♣Tq ÑY

defined by pY♣♣x,T xqq ✏T x. Then, clearly we haveT ✏pY✆p✁_X1which is continuous

as a composition of two continuous operators.

From now on, H _{will denote a separable Hilbert space and}⑥ ☎ ⑥_{will be the norm}

induced by the inner product ①☎,☎② onH _{. Furthermore,} B♣H q_{denote the Borel σ}

-algebra onH_{. LetL}♣Hq_{be the set of all bounded linear operator on}H_.

Definition 2.2.16 (Trace class operator). A trace class operator is an operator T P

L♣H q_{such that there exist two sequences}♣_ajq_jPN,♣bjqjPNinH with

➦_✽

j✏1⑤aj⑤⑤bj⑤ ➔

linear operator T onH we define the trace of T by

Proposition 2.2.6. If T PL♣H q_{is a trace class operator, then}

(i) T is compact.

(ii) For any orthonormal basis♣ejqjPNinH , Tr T=

➦_✽

j✏1①Tej,ej②is absolutely

con-vergent and equal to➦✽_j✏1①aj,bj②.

Proof. First, recall that an operatorT onH _{is compact if and only ifT is a strong limit}

of a sequence of finite rank operators ♣TnqnPN in H . Choose Tnx✏

➦n

j✏1aj①x,bj②.

Then by using both triangle and Cauchy-Schwarz inequalities, it is easy to see that

⑥T✁Tn⑥ ↕

✽

➳

j✏n 1

⑤aj⑤⑤bj⑤,

where⑥ ☎ ⑥is the operator norm onH _{. From the definition of a trace class operator, it}

is obvious that right hand side of the inequality converges to zero asnÑ ✽.

For the second statement, again it is easy to show that after some computations

using the additivity and homogeneity property of an inner product and then using the

Cauchy-Schwarz inequality we obtain

From this point, by applying the Parseval identity yields

can be proven by rewritting➦✽_j✏1①Tej,ej②as

✽

➳

k✏1

①

✽

➳

j✏1

①ak,ej②ej,bk②

and then using the fact that♣ejqjPNis an orthonormal basis inH .

We will denote the set of all positive, symmetric, trace class operatorQonH _{by the}

symbolL₁♣Hq_.

Definition 2.2.17 (Fréchet derivative). Let X,Y be two normed spaces. The Fréchet

derivative (or strong derivative) of a function f :X ÑY at x is a bounded linear map

D f♣xq:XÑY such that

lim

hÑ0

⑥f♣x hq ✁ f♣xq ✁D f♣xqh⑥

⑥h⑥ ✏0.

We will denote the k-th order derivative of f by the symbol Dkf .

This definition can be seen as generalization of the derivative of a function f :RÑR

and the Jacobian of a function f :RnÑRm to the derivative in an arbitrary normed space.

Let us now recall the Riesz representation theorem which is very important in the

construction of a measure in an infinite dimensional spaces.

Theorem 2.2.7(The Riesz representation theorem). LetH✝ _{denote the dual space of}

a separable Hilbert spaceH _{consisting of all continuous linear functionals from} H

into its field. Then for any element T ofH✝_{, there exist a unique element y}PH _such

that

T♣xq ✏ ①x,y②, ❅xPH

Proof. LetTPH ✝. First we will show that the representation ofT is well-defined. Let

the sequence aj must itself be square summable. To see this, first note that since

⑥T♣xq⑥ ↕ ⑥T⑥⑥x⑥, then we have ⑤➦✽_j✏1ajcj⑤ ↕ ⑥T⑥

Now letN be a fixed positive integer and define a sequence

cj✏

by the least upper bound property of R. This implies that, as an element of H , the series y✏➦✽_j✏1ajej is well-defined which means that we can have the equality

Theorem 2.2.8(Riesz Representation Theorem for Bilinear Form). Letϕbe a bounded

bilinear form onH . Then, there exist a unique bounded linear operator QPL♣Hq

Proof. Obviously, ifQexist, then it is unique. Now letxPH be fixed. Then,ϕ♣x,yq

is conjugate linear iny, so that ϕ♣x,yq is linear in y. Using the Riesz representation theorem, there exist a unique zP H _{such that ϕ}♣_x,yq ✏ ①_y,z②_{, and hence ϕ}♣_x,yq ✏

①z,y②. We now define the mapQ:H ÑH _byQx✏_{z. Since}

①Q♣αx1 βx2q,y② ✏ϕ♣αx1 βx2,yq ✏αϕ♣x1,yq β ϕ♣x2,yq

✏α①Qx1,y② β①Qx2,y②

✏ ①αQx1,y② ①βQx2,y②

This implies that①Q♣αx1 βx2q,y② ✁ ①αQx1,y② ✁ ①αQx2,y② PH ❑✏ t0✉so thatQis

linear. To show thatQis bounded, notice that

⑥Q⑥ ✏sup

x✘0

⑥Qx⑥

⑥x⑥ ✏_x✘0sup_,Qx✘0

①Qx,Qx②

⑥x⑥⑥Qx⑥

↕ sup

x✘0,y✘0

①Qx,y②

⑥x⑥⑥y⑥

✏ sup

x✘0,y✘0

ϕ♣x,yq

⑥x⑥⑥y⑥

✏ ⑥ϕ⑥,

and the conclusion follows.

We are going to use both of the Riesz representation theorem to define the mean

and covariance of a probability measureµ in the next chapter.

C. Events and Random Variables

We say that two eventsAandBare (stochastically) independent if the occurence ofA

does not change the probability that the eventBalso occurs. In a more precise way, we

Definition 2.3.1(Independence of events). Let I be an arbitrary index set. A collection

Now suppose that we roll a dice infinitely many times, what is the probability that

the face shows a one infinitely many ? We guess that this should be equal to one since

otherwise there would be a last point in time when we see a one and after which the face

only shows a number two to six. However, this is not very plausible. The following

theorem confirms the conjecture we mentioned before and also gives conditions under

which we cannot expect that infinitely many of the events occur.

Theorem 2.3.1(Borel-Cantelli lemma). Let♣AnqnPNbe a sequence of events and A✏

lim sup_{nÑ ✽}An. Then

(i) Letµ be an arbitrary measure. If➦✽_n✏1µ♣Anq ➔ ✽, thenµ♣Aq ✏0.

(ii) If♣AnqnPNis independent and

➦_✽

n✏1P♣Anq ✏ ✽, thenP♣Aq ✏1.

Proof. By the continuity from above andσ-subadditivity ofµ, respectively, we have

µ♣Aq ✏ lim

For the second part, by using the extended De’Morgan’s rule and continuity from below

then we have the following inequality

✽

➵

n✏m

♣1✁P♣Anqq ✏exp♣

✽

➳

n✏m

ln♣1✁P♣Anqq ↕exp♣✁

✽

➳

n✏m

P♣Anqq ✏0

and the conclusion follows.

Now we extend the definition of independence from a collection of events to a

family of collection of events.

Definition 2.3.2(Independence of a collection of events). Let♣_Ω,F,Pq be a proba-bility space and I be an arbitrary index set. LetA_i❸F _{for every i}P_{I. The collection}

♣A_iq_{iPI is called independent if for any finite subset J of I and any choice Aj}PA_{j, we}

have

P

✄ ↔

jPJ

Aj

☛

✏➵

jPJ

P♣Ajq.

Let ♣_Ω,F,Pq be a probability space, ♣_Ωˆ,Fˆq a measurable space, and B♣Ωˆq the Borelσ-algebra of ˆ_Ω.

Definition 2.3.3(Random variable). Let_Ωˆ be a complete metric space. An _Ωˆ-valued

random variable defined on the measure space ♣_Ω,F_,Pq _{is a measurable function}

X :_ΩÑ _Ωˆ, i.e, X✁1♣Iq PF _{for every I}P_B♣_Ωˆq_{. If Ω}ˆ ✏_{R, then we call X as a real}

random variable.

Remark 2.3.1. For FPB♣_Ωˆq, we denotetX PF✉ ✏X✁1♣FqandP♣XPFq ✏PrX✁1♣Fqs. In particular, we lettX➙a✉ ✏X✁1♣ra, ✽qq,tX ➔a✉ ✏X✁1♣♣✁✽,aqq, and so on.

Definition 2.3.4(Distribution of random variables). Let X be a random variable. Then

(i) The probability measurePX ✏P✆X✁1is called the distribution of X .

(ii) For a real random variable X , the function FX which maps x intoP♣X ↕xqis

called the distribution function of X . We write X✒µ if X has distributionµ, i.e,