Every nontrivial inner-product space has an orthonor

mal basis.

Proof.

Call the space X. Since it is not 0, it contains a nonzero vector

x.

The set consisting solely of

x/llxll

is orthonormal. Now order the family of all orthonormal subsets of X in the natural way (by inclusion). In order to use Zorn's Lemma, one must verify that each chain of orthonormal sets has an upper bound. Let C be such a chain, and put

A'

⁼

U{A : A

^{E C} .} It is obvious that

A'

is an upper bound for C, but is

A'

orthonormal? Take

x

and y in

A'

^such

that

x ;6 y.

Say

x

E

Al

^{E C and}

^y

^E

A2

^{E C. Since C} is a chain, either

Al

^C

A2

or

A2

^C

AI.

Suppose the latter. Then

x, y

E

AI.

^Since

Al

is orthonormal,

(x, y) = O.

Obviously,

IIxll =

1. Hence

A'

is orthonormal. • Theorem 6. The Orthonormal Basis Theorem.

For an or­

thonormal family ^Iu;! (not necessarily /inite or countable) in a Hilbert space

^X,

the following properties are equivalent:

a.

Iu;! is an orthonormal basis for

^X.

b.

Ifx

E X

and ^{x .l Ui} for all i, then ^{x =}

^o.

c.

For each ^x

^{E X,}

x = L(X, Ui)Ui.

d.

For each ^x and ^y in

^X,

^{(x, y)}

⁼

L(X, Ui)(y, Ui)'

e .

For each ^x in

^X,

IIxll2 = L I (x, Ui)j2 . (Parseval Identity)

Proof.

To prove that a implies b, suppose that b is false. Let

x ;6 0

and

x

.1

Ui

for all

i.

Adjoin

x/ llxll

to the family

Iu;)

to get a larger orthonormal family. Thus the original family is not maximal and is not a basis.

To prove that b implies c, assume b and let

x

be any point in X. Let Y

= L (X, Ui)Ui'

By Bessel's inequality (Theorem

4),

we have

By Theorem

2,

the series defining

y

converges. (Here the completeness of X is needed.) Then straightforward calculation (as in the proof of Theorem

3)

shows that

x - y

.1

Ui

for all

i.

By b,

x - y = O.

74 Chapter 2 Hilbert Spaces

To prove that c implies d, assume c and writ.e

Straightforward calculation then yields To prove that d implies e , assume d

(x,y)

and let

⁼ L(X,Ui)(y,Ui)' y

⁼

x

^{i n d .} The result is the assertion in e.

To prove that e implies a, suppose that a is false. Then

lUi]

is not a maximal orthonormal set. Adjoin a new element,

x,

to obtain a larger orthonormal set.

Then

1 ⁼ IIxl12

¹⁼

L I(x, ui)12

⁼

0,

showing that e is false. • Example 1. One orthonormal basis in

e2

is obtained by defining

unU) ⁼ 6nj.

Thus

Ul

⁼

[1, 0, 0,

^{. .}

^.

^{J ,}

U2 = [0, 1, 0,

^{. . .} J , etc.

To see that this is actually an orthonormal base, use the preceding theorem, in particular the equivalence of a and

b.

Suppose

x

^E

e2

^and

(x, un)

⁼

0

for all n.

Then

x ⁽

ⁿ

)

⁼

0

for all n, and

x ⁼

^{o. •}

Example 2. An orthonormal basis for

L2[0, ^1]

is provided by the functions

un(t) ⁼ e2rrint,

where n E Z. One verifies the orthonormality by computing the appropriate integrals. To show that

[un]

is a base, we use Part

b

of Theorem

6.

Let

x

^E

L2[0, ^1]

^and

x

¹⁼

^O.

It is to be shown that

(x,un)

¹⁼

0

^{for some n. Since}

the set of continuous functions is dense in

L2,

there is a continuous

y

^{such that}

Ilx - YII ^< Ilxli/5.

^Then

lIyll

^�

Ilxll - llx - YII ^> �llxll·

By the Weierstrass Approximation Theorem, the linear span of

[unJ

is dense in the space

C[O, lJ,

furnished with the supremum norm. Select a linear combination

p

^of

[un]

^such

that

�lIxli. lip - YII

^{Then 00}

^< IIxli/5.

^Then

lip - YII ^< IIxll/5.

^Hence

IIpl! ^> IIYII - IIY -pll ^>

l(x,p)1

^�

l(p,p)I - I(y -p,p)I - I(x - y,p)1

�

IIpll2 - IIY - pll IIpil - IIx - yll IIpll ^> 0

Thus it is not possible to have

(x, un) ⁼ 0

for all n. • Recall that we have defined the orthogonal projection of a Hilbert space

X

onto a closed subspace

Y

to be the mapping

P

such that for each

x

^E

^{X, P} x

is the point of

Y

closest to

x.

Theorem 7. The Orthogonal Projection Theorem.

The ortllOgonal projection P of a Hilbert space X onto a closed subspace Y has tlJese properties:

a.

It is well-defined; i.e., Px exists and is unique in Y.

b. It is surjective, i.e., P(X)

⁼

Y.

c.

It is linear.

d. If _e.

x

^--

Y is not Px .1 Y for all 0 (the zero subspace), then x. IIpll ⁼ 1.

Section

2.2

Orthogonality and Bases

f.

P is Hermitian; i.e., (Px, w) ⁼ (x, Pw) ^{[or all} x ^and w.

g.

If[Yd is an orthonormal basis [or Y, ^then Px ⁼ "L(X,Yi)Yi.

h.

P is idempotent; i.e., p2 ⁼ P.

i.

Py ⁼ Y ^{[or all} Y

^E

Y. ^Thus PlY ⁼

^fl"

j.

IIxll2

⁼

IIpxll2

⁺

llx - Px1J2.

Proof. This is left to the problems.

75

• The Gram-Schmidt process, familiar from the study of linear algebra, is an algorithm for producing orthonormal bases. It is a recursive process that can be applied to any linearly independent sequence in an inner-product space, and it yields an orthonormal sequence, as described in the next theorem.

Theorem 8. The Gram-Schmidt Construction.

Let

[VI, V2, V3,

^..

^{. J} be a linearly independent sequence in an inner product space. Having set UI ⁼ vI/llvlll, define recursively

Vn - "L (vn, Ui)Ui n-I i=1 Un

⁼⁼

---'-n--:"'I IIVn - "L (Vn, Ui)udl

i=1 ^--

--

n =

2, 3, . . .

Then [UI' U2, U3, ^.

^..

^J is an orthonormal sequence, and [or each

^n, span{

u I

^'

U2, .. .

^,

^u n}

^{== span{}

VI, V2, · .. , vn}.

Notice that in the equation describing this algorithm there is a normalization process: the dividing of a vector by its norm to produce a new vector pointing in the same direction but having unit length. The other action being carried out is the subtraction from the vector

Vn

of its projection on the linear span of the orthonormal set presently available,

UI, U2,'" ,Un-I.

This action is obeying the equation in Theorem

3,

and it produces a vector that is orthogonal to the linear span just described. These remarks should make the formulas easy to derive or remember.

Example 3. (A nonseparable inner-product space) . A normed linear space (or any topological space) is said to be separable if it contains a countable dense set. If an inner-product space is nonseparable, it cannot have a count­

able orthonormal base. For an example, we consider the uncountable family of functions

u>. (t) = ei>'t,

^where

^t

^{E IR and ,\ E IR.} This family of functions is linearly independent (Problem

5

)

,

and is therefore a Hamel basis for a linear space X. We introduce an inner product in X by defining the inner product of two elements in the Hamel base:

This is the value that arises in the following integration:

1

j

^{T - 1}

j

^{T .}

lim

2T u>.(t) uq(t) dt

== lim

2T e,(>,-q)t ^dt

T_� -T T-� _T

76 Chapter 2 Hilbert Spaces

If ^A

=

^a, this calculation produces the result

1.

If ^A i-a , we get

O.

Elements of X have the property of almost periodicity. (See Problem

1.)

• Example 4. (Other abstract Hilbert spaces). A higher level of abstraction can be used to generate further inner product spaces and Hilbert spaces. Let us create at one stroke a Hilbert space of any given dimension. Let

5

be any set.

The notation

res

denotes the family of all functions from

5

to the field IC. This set of functions has a natural linear structure, for if

x

and

y

belong to

res, x

+

y

can be defined by

(x

+

y)(s) = x(s)

+

y(s)

A similar equation defines ^AX for ,\ E IC. Within

res

we single out the subspace X of all

x

E

res

such that

( 1) 2: [ lx(sW : s

E

5 ] < 00

(Here we are using the notion of unordered sum as defined previously.) This . construction is familiar in certain cases. For example, if

S =

^{{ I ,}

2, .

. . ^{, n}

}

^, then the space X just constructed is the familiar space

ren.

On the other hand, if

S =

^N, then X is the familiar space £2 . In the space X, addition and scalar multiplication are already defined, since X C

reS.

Naturally, we define the inner product by

(2) (x, y)

⁼

2: [x(s)y(s) : s

E

S]

r..luch of what we are doing here loses its mystery when we recall (from the Corollary to Theorem

4)

that the sums in Equations

(1)

and

(2)

are always countable. The space discussed here is denoted by £2(S). • Example 5. (Legendre polynomials.) An important example of an orthonor­

mal basis is provided by the Legendre polynomials. We consider the space

C! - 1 , 1 J and use the simple inner product

( J

^,

^g ) = [II f(t)g(t) dt

Now apply the Gram-Schmidt process to the monomials

t

^o-t

1 , t, t2, t3,

^{. • .} The un-normalized polynomials that result can be described recursively, using the classical notation

Pn:

Po(t) =

¹

2n - 1 n

- 1

Pn(t)

⁼

--tPn_l (t) - --Pn-2(t)

n n

(

ⁿ

= 2, 3, . . . )

The orthonormal system is, of course,

P ⁿ = PniIIPn ll.

The completion of the space G[- I , 1] with respect to the norm induced by the inner product is the space L2 [- I, IJ. Every function

f

in this space is represented in the L2 -sense by the series

f = 2:(J,Pk)Pk 00

k=O

Section

2.2

Orthogonality and

Bases