Chapter 7 Hilbert Space and Elements of Fourier Analysis
7.1 Hilbert Space
7.1.2. Orthogonal Projection and Bases
It is interesting to observe that, as distinguished from the preceding con- struction of a complex from a real Hilbert space, the passage from a complex Hilbert space to a real one of which it is the complexification is not canonical.
See Exercise 7.1.10 for more details.
§7.1.2. Orthogonal Projection and Bases: Given a closed linear sub-
7 Hilbert Space and Fourier Analysis
of {yn : n ≥1} has a limit y0, and it is obvious that any such limit y0 will satisfykx−y0kH=δ.
However, whenHis infinite-dimensional, one has to find another argument.
It simply is not true that every bounded subset of an infinite-dimensional space is relatively compact. For example, take`2(N;R) to beL2(µ;R) where µ is the counting measure on N (i.e., µ({n}) = 1 for each n ∈ N), and let xn=1{n} be the element of`2(N;R) that is 1 atnand 0 elsewhere. Clearly, kxnk`2(N;R)= 1 for alln∈N, and therefore{xn: n∈N}is bounded. On the other hand,kxn−xmk`2(N;R)=√
2 for alln6=m, and therefore{xn: n∈N} can have no limit point in`2(N;R).
As the preceding makes clear, in infinite dimensions one has to base the proof of existence of ΠLx on something other than compactness, and so it is fortunate that completeness comes to the rescue. Namely, I will show that every minimizing sequence {yn : n ≥ 1} is Cauchy convergent. The key to doing so is the parallelogram equality, which says that the sum of the squares of the lengths of the diagonals in a parallelogram is equal to the sum to the squares of the lengths of its sides. That is, for any a, b ∈ H, ka+bk2H+ka−bk2H = 2kak2H+ 2kbk2H, an equation that is easy to check by expanding the terms on the left-hand side. Applying this when a =x−yn
andb=x−ym, one gets 4
x−yn+ym
2
2
H
+kyn−ymk2H = 2kx−ynk2H+ 2kx−ymk2H, and therefore, since yn+y2 m ∈L, that
kyn−ymk2H≤2kx−ynk2H+ 2kx−ymk2H−4δ2.
Thus, the Cauchy convergence of {yn : n≥1} follows from the convergence of{kx−ynkH : n≥1}toδ.
Before moving on, I will summarize our progress thus far in the following theorem, and for this purpose it is helpful to introduce a little additional terminology. A map Φ taking H into itself is said to be idempotent if Φ◦Φ = Φ, it is called acontractionifkΦ(x)kH ≤ kxkH for allx∈H, and it is said to besymmetric(or sometimes, in the complex case,Hermitian) if Φ(x), y
H = x,Φ(y)
H for allx, y∈H.
Theorem 7.1.3. Let Lbe a closed, linear subspace of the real or complex Hilbert space H. Then, for each x ∈ H there is a unique ΠLx ∈ L for which (7.1.1) holds. Moreover, ΠLx is the unique element of L for which (7.1.2)holds. Finally, the mapx ΠLxis a linear, idempotent, symmetric contraction.
Proof: Only the final assertions need comment. However, linearity follows from the obvious fact that if, depending on whetherH is real or complex,α1
andα2are elements ofRorC, then for anyx1, x2∈H,α1ΠLx1+α2ΠLx2∈L
and α1x1+α2x2−α1ΠLx1−α2ΠLx2 ⊥L. As for idempotency, use (7.1.1) to check that ΠLx=xifx∈L. To see that ΠL is symmetric, observe that, because x−ΠLx⊥Landy−ΠLy⊥L,
x,ΠLy
H= ΠLx,ΠLy
H = ΠLx, y
H
for allx, y∈H. Finally, becausex−ΠLx⊥L,kxk2H =kΠLxk2H+kx−ΠLxk2H, and so kΠLxkH ≤ kxkH.
In view of its properties, especially (7.1.2), it should be clear why the map ΠL is called theorthogonal projection operatorfrom H ontoL.
An immediate corollary of Theorem 7.1.3 is the following useful criterion.
In its statement, and elsewhere, S⊥ denotes theorthogonal complement a subset of S of H. That is, x∈ S⊥ if and only ifx⊥S. Clearly, for any S ⊆H, S⊥ is a closed linear subspace. Also, the span, denoted by span(S), ofS is the smallest linear subspace containingS. Thus, span(S) is the set of vectorsPn
m=1αmxm, wheren∈Z+,{xm: 1≤m≤n} ⊆S, and, depending on whether H is real or complex,{αm: 1≤m≤n}is a subset of RorC. Corollary 7.1.4. IfS is a subset of a real or complex Hilbert space H, thenS spans a dense subset ofH if and only ifS⊥={0}.
Proof: Let L denote the closure of the subspace of H spanned by S, and note that S⊥ = L⊥. Hence, without loss in generality, we will assume that S =Land must show thatL=H if and only ifL⊥={0}. But ifL=H and x⊥L, thenx⊥xand thereforekxk2H = 0. Conversely, ifL6=H, then there exists anx /∈L, and sox−ΠLxis a non-zero element of L⊥.
Elements of a subset S are said to be linearly independent if, for each finite collection F of distinct elements from S, the only linear combination P
x∈Fαxxthat is 0 is the one for whichαx= 0 for eachx∈F. AbasisinH is a subsetSofH whose elements are linearly independent and whose span is dense inH. A Hilbert spaceH isinfinite-dimensionalif it admits no finite basis.
Lemma 7.1.5. Assume thatH is an infinite-dimensional, separable Hilbert space. Then there exists a countable basis in H. Moreover, if {xn : n≥0}
is linearly independent inH, then there exists a sequence{en: n≥0} such that (em, en)H =δm,n for allm, n∈Nand
span {x0, . . . , xn}
= span {e0, . . . , en}
for eachn∈N. In particular, if{xn: n≥0} is a basis forH, then{en: n≥0}is also.
Proof: To produce a countable basis, start with a sequence {yn : n ≥ 0}
⊆H\ {0} that is dense inH, and filter out its linearly dependent elements.
That is, take n0 = 0, and, proceeding by induction, take nm+1 to be the smallest nfor whichyn∈/span {y0, . . . , yn−1}
. Ifxm=ynm, then it is clear
7 Hilbert Space and Fourier Analysis
that thexm’s are linearly independent and that they span the same subspace as theyn’s do. Hence,{xm: m≥0}is a countable basis forH.
Now suppose that {xn : n ≥ 0} is linearly independent, and set Ln = span {x0, . . . , xn}
. ThenLn is finite-dimensional and therefore (cf. part (ii) in Exercise 7.1.9) closed. Becausexn+1−ΠLnxn+16= 0 for anyn≥0, we can take
e0= x0
kx0kH and en+1= xn+1−ΠLnxn+1
kxn+1−ΠLnxn+1kH. If we do so, then it is obvious that span {e0, . . . , en}
is contained in Ln
for each n ≥ 0. To see that it is equal to Ln, one can work by induction.
Obviously, there is nothing to do whenn= 0, and ifLn= span {e0, . . . , en} , then, becausexn+1−kxn+1−ΠLnxn+1kHen+1∈Ln, the same is true forn+1.
Finally, because, by construction, kenkH = 1 anden+1 ⊥Ln for all n∈ N, (em, en)H =δm,n.
A sequence {en : n≥0} ⊆ H is said to beorthonormal if (em, en)H = δm,n, and the preceding construction of an orthonormal sequence from a lin- early independent one is called the Gram–Schmidt orthonormalization procedure.
Lemma 7.1.6. Suppose that{en : n≥0}is an orthonormal sequence inH.
Then, depending on whetherH is real or complex, for each{αm: m≥0} ∈
`2(N;R) or{αm: m≥0} ∈`2(N;C), the seriesP∞
n=0αnen converges inH.
That is, the limit
∞
X
n=0
αnen≡ lim
N→∞
N
X
n=0
αnen exists inH.
Moreover,
αm=
∞
X
n=0
αnen, em
!
H
for allm∈N and
∞
X
n=0
αnen
2
H
=
∞
X
n=0
|αn|2.
Finally, the closed linear span1 Lof{en: n≥0}in H coincides with the set of sumsP∞
n=1αnen as{αn: n≥1}runs over`2(N;R)or`2(N;C). In fact, if x∈H, then, depending or whetherH is real or complex,{(x, en)H : n≥0}
is an element of`2(N;R)or`2(N;C), and ΠLx=
∞
X
n=0
x, en
Hen.
1The closed linear span of a set is the closure of the subspace spanned by that set. It is easy to check that an equivalent description is as the smallest closed linear subspace containing the set.
In particular,
∞
X
n=0
x, en
H
2=kΠLxk2H ≤ kxk2H for allx∈H,
and
ΠLx, y
H =
∞
X
n=0
(x, en)H(y, en)H for allx, y∈H, where the series on the right is absolutely convergent.
Proof: To prove thatP∞
n=0αnen converges, note that, because (em, en)H = δm,n,
N
X
n=0
αnen−
M
X
n=0
αnen
2
H
=
N
X
n=M+1
αnen
2
H
=
N
X
n=M+1
|αn|2
for all M < N. Hence, since P∞
n=0|αn|2<∞, n PN
n=0αnen: N ≥0o satis- fies Cauchy’s convergence criterion. Furthermore,
αm= lim
N→∞
N
X
n=0
αnen, em
!
H
=
∞
X
n=0
αnen, em
!
H
, and
∞
X
n=0
αnen
2
H
= lim
N→∞
N
X
n=0
αnen
2
H
= lim
N→∞
N
X
n=0
|αn|2=
∞
X
n=0
|αn|2. Obviously, P∞
n=0αnen ∈L for all{αn : n≥0}from `2(N;R) or `2(N;C).
Conversely, if LN = span {e0, . . . , eN}
, then it is obvious that
N
X
n=0
x, en
Hen, em
!
H
= x, em
H for all 0≤m≤N, from which it is an easy step to see that x−PN
n=0(x, en)Hen ⊥ LN and therefore that ΠLNx=PN
n=0 x, en)Hen for allN ≥0 andx∈H. Hence,
N
X
n=0
x, en
H
2=
N
X
n=0
x, en
Hen
2
H
= ΠLNx
2
H ≤ kxk2H, and therefore
∞
X
n=0
x, en
H
2≤ kxk2H <∞.
7 Hilbert Space and Fourier Analysis
In particular, the series {(x, en)H : n ≥ 0} is an element of `2(N;R) or
`2(N;C),P∞
n=0(x, en)Hen converges inH to an element ofL, and x, em
H =
∞
X
n=0
x, en
Hen, em
!
H
for allm∈N.
Since this means that x−P∞
n=0(x, en)Hen is orthogonal to {em : m ≥ 0}, and therefore to L, we now know that ΠLx = P∞
n=0(x, en)Hen for x ∈ H.
Finally, if x, y∈H, then
ΠLx, y
H = lim
N→∞
N
X
n=0
(x, en)Hen, y
!
H
= lim
N→∞
N
X
n=0
(x, en)H(en, y) =
∞
X
n=0
(x, en)H(y, en)H,
and the absolute convergence from the Schwarz’s inequality for`2(N;R).
The inequality P∞
n=0|(x, en)H|2 ≤ kxk2H in Lemma 7.1.6 is often called Bessel’s inequality.
By combining Lemmas 7.1.5 and 7.1.6, one arrives at a structure theorem for infinite-dimensional, separable Hilbert spaces.
Theorem 7.1.7. LetH be an infinite-dimensional, separable Hilbert space.
Then, there exists a linear isometryΦfromH onto, depending on whetherH is real or complex,`2(N;R)or `2(N;C).
Proof: Choose, via Lemma 7.1.5, an orthonormal basis{en : n≥0}forH, and define Φ(x) ={(x, en)H: n≥0}. Now apply Lemma 7.1.6 to check that Φ has the required properties.
I will finish this survey of Hilbert spaces with a procedure for constructing a basis out of other bases. In order to avoid the introduction of the general notion of tensor products, I will state and prove this result only forL2-spaces.
If f1 :E1 −→Cand f2 :E2−→C, then f1⊗f2 is the C-valued function on E1×E2 given by f1⊗f2(x1, x2) =f1(x1)f2(x2). Of course, if f1 and f2
areR-valued, then so is f1⊗f2.
Theorem 7.1.8. Let(E1,B1, µ1)and(E2,B2, µ2)be a pair ofσ-finite mea- sure spaces whose σ-algebras are countably generated. If {e1,n : n ≥ 0}
and {e2,n : n ≥ 0} are orthonormal bases for L2(µ1;R) (or L2(µ1;C)) and L2(µ2;R) (or L2(µ2;C)) respectively, then {e1,n1 ⊗e2,n2 : (n1, n2)∈ N2} is an orthonormal basis forL2(µ1×µ2;R)(orL2(µ1⊗µ2;C).
Proof: Since the proof is the same in both cases, we will deal only with the R-valued case.
By Fubini’s Theorem,
e1,m1⊗e2,m2, e1,n1⊗e2,n2
L2(µ1×µ2;R)
= e1,m1, e1,n1
L2(µ1;R) e2,m2, e2,n2
L2(µ1;R),
and so it is clear that{e1,n1⊗e2,n2 : (n1, n2)∈N2}is orthonormal. Thus, it suffices to check that L= span {e1,m1⊗e2,m2 : (m1, m2)∈N2}
is dense in L2(µ1×µ2;R). Since, forf1∈L2(µ1;R) andf2∈L2(µ2;R), one has
Mlim→∞
M
X
m1=0
f1, e1,m1
L2(µ1;R)e1,m1
!
⊗
M
X
m2=0
f2, e2,m2
L2(µ2;R)e2,m2
!
= lim
M→∞
M
X
m1,m2=0
f1, e1,m1
L2(µ1;R) f2, e2,m2
L2(µ2;R)e1,m1⊗e2,m2,
where the convergence is in L2(µ1×µ2;R), f1⊗f2 ∈L. Hence, L contains the linear span of such functions, and, by Exercise 6.2.10, that span is dense in L2(µ1×µ2;R).
Exercises for §7.1
Exercise 7.1.9. Let H be a real or complex Hilbert space, and note that every closed subspace of H becomes a Hilbert space with the inner product obtained by restriction. Here are a few more simple facts about subspaces of a Hilbert space.
(i) Show that for anyS⊆H,S⊥ is always a closed linear subspace of H.
(ii) The proof of Lemma 7.1.5 used the fact that the subspaces Ln there are closed. Of course, once one knows that Ln admits an orthonormal basis {em : 0 ≤m ≤n}, this can be easily checked by noting that x∈ Ln =⇒ x=Pn
m=0(x, em)Hem, and therefore if{xk : k ≥1} ⊆Ln and xk −→xin H, then
x−
n
X
m=0
(x, em)Hem
H
= lim
k→∞
xk−
n
X
m=0
(xk, em)Hem
H
= 0.
More generally, without using the existence of orthonormal bases, show that any finite dimensional subspace LofH is closed.
Hint: IfL={0}there is nothing to do. Otherwise, choose a basis{b1, . . . , b`} for L, and show that
P`
k=1αkbk
H ≥ P`
k=1|αk|212
for some > 0.
Conclude from this that if {xn : n ≥ 1} ⊆ L converges to x, then x = P`
k=1αkbk ∈Lfor some real or complex coefficients{αk : 1≤k≤`}.
(iii) Show that C([0,1];R) is a non-closed subspace of L2(λ[0,1];R). Hence, when dealing with infinite-dimensional subspaces, closedness is something that requires checking.
7 Hilbert Space and Fourier Analysis
Exercise 7.1.10. Given a complex, infinite-dimensional, separable Hilbert space H, there are myriad ways in which to produce a real Hilbert space of which H is the complexification. To wit, choose an orthonormal basis {en : n ≥0} for H, and let L be the set of x∈ H such that (x, en)H ∈ R for all n≥0. Show thatL becomes a real Hilbert space when one takes the restriction of (·,·)H to Lto be its inner product. In addition, show thatH is the complexifiction ofL.
Exercise 7.1.11. Suppose that Π is a linear map from the Hilbert spaceH into itself. Show that Π is the orthogonal projection operator onto the closed subspace Lif and only ifL= Range(Π) and Π is idempotent and symmetric (i.e., Π2= Π and (Πx, y)H= (x,Πy)H for allx, y∈H). Also, show that ifL is a closed, linear subspace ofH, then ΠL⊥ =I−ΠL, whereI is the identity map.
Exercise 7.1.12. It may be reassuring to know that, in some sense, the dimension of a separable, infinite-dimensional Hilbert space is well-defined and equal to the cardinality of the integers. To see this, show that if E is an infinite subset ofH that is orthonormal in the sense that
e, f)H=
1 iff =e
0 otherwise fore, f ∈E,
then the elements ofE are in one-to-one correspondence with the integers.
Exercise 7.1.13. Assume thatH is a separable, infinite-dimensional, real or complex Hilbert space, and letLbe a closed, linear subspace ofH. Show that Lis also a separable Hilbert space and thatevery orthonormal basis forLcan be extended to an orthonormal basis for H. That is, ifE is an orthonormal basis for L, then there is an orthonormal basis ˜E forH withE⊆E. In fact,˜ show that ˜E=E∪E0, where E0 is an orthonormal basis forL⊥.