Summary of operator-adapted wavelets

Chapter II: Denoising

2.2 Summary of operator-adapted wavelets

We proceed by reviewingoperator-adapted waveletsas in [153, Sec. 2], also named gambletsin reference to their game theoretic interpretation, and their main properties [99,101,104,119]. They are constructed with a hierarchy of measurement functions and an operator. Theorem2.2.4shows these gamblets are simultaneously associated with Gaussian conditioning, optimal recovery, and game theory. By selecting these measurement functions to bepre-Haar wavelets, the gamblets are localized both in space and in the eigenspace of the operator.

Hierarchy of measurement functions

Let𝑞 ∈ N^∗ (used to represent a number of scales). Let(I^(𝑘))_1≤𝑘≤𝑞 be a hierarchy of labels defined as follows. I^(𝑞) is a set of 𝑞-tuples consisting of elements 𝑖 = (𝑖₁, . . . , 𝑖_𝑞). For 1 ≤ 𝑘 ≤ 𝑞 and𝑖 ∈ I^(𝑞),𝑖^(𝑘) := (𝑖₁, . . . , 𝑖_𝑘) andI^(𝑘) is the set of

𝑘-tuplesI^(𝑘) = {𝑖^(𝑘)|𝑖 ∈ I^(𝑞)}. For 1 ≤ 𝑟 ≤ 𝑘 ≤ 𝑞 and 𝑗 = (𝑗₁, . . . , 𝑗_𝑘) ∈ I^(𝑘), we write 𝑗^(𝑟) = (𝑗₁, . . . , 𝑗_𝑟). We say that 𝑀 is aI^(𝑘) × I^(𝑙) matrix if its rows and columns are indexed by elements ofI^(𝑘) andI^(𝑙), respectively.

Let {𝜙^(𝑘)

𝑖 |𝑘 ∈ {1, . . . , 𝑞}, 𝑖 ∈ I^(𝑘)} be a nested hierarchy of elements of H^−𝑠(Ω) such that(𝜙^(𝑞)

𝑖 )_𝑖∈I(𝑞) are linearly independent and 𝜙^(𝑘)

𝑖 = Õ

𝑗∈I^(𝑘+1)

𝜋^{(𝑘 , 𝑘+1)}

𝑖, 𝑗 𝜙^(𝑘+1)

𝑗 (2.2.1)

for𝑖 ∈ I^(𝑘),𝑘 ∈ {1, . . . , 𝑞−1}, where𝜋^{(𝑘 , 𝑘+1)} is anI^(𝑘) × I^(𝑘+1) matrix and

𝜋^{(𝑘 , 𝑘+1)}𝜋^{(𝑘+1, 𝑘)} = 𝐼^(𝑘). (2.2.2)

In (2.2.2), 𝜋^{(𝑘+1, 𝑘)} is the transpose of 𝜋^{(𝑘 , 𝑘+1)} and 𝐼^(𝑘) is the I^(𝑘) × I^(𝑘) identity matrix.

Hierarchy of operator-adapted pre-wavelets Let(𝜓⁽

𝑘)

𝑖 )_𝑖∈I(𝑘)be the hierarchy of optimal recovery splines associated with(𝜙⁽

𝑘) 𝑖 )_𝑖∈I(𝑘), i.e., for𝑘 ∈ {1, . . . , 𝑞}and𝑖 ∈ I^(𝑘),

𝜓⁽

𝑘)

𝑖 = Õ

𝑗∈I^(𝑘)

𝐴⁽

𝑘) 𝑖, 𝑗 L⁻¹𝜙⁽

𝑘)

𝑗 , (2.2.3)

where

𝐴^(𝑘) :=(Θ^(𝑘))⁻¹ (2.2.4)

andΘ^(𝑘) is theI^(𝑘)× I^(𝑘) symmetric positive definite Gramian matrix with entries (writing [𝜙, 𝑣] for the duality pairing between𝜙 ∈ H^−𝑠(Ω) and𝑣 ∈ H^𝑠

0(Ω)) Θ_{𝑖, 𝑗}^(𝑘) =[𝜙⁽

𝑘)

𝑖 ,L⁻¹𝜙⁽

𝑘)

𝑗 ]. (2.2.5)

Note that 𝐴^(𝑘) is the stiffness matrix of the elements(𝜓⁽

𝑘)

𝑖 )_𝑖∈I(𝑘) in the sense that 𝐴⁽

𝑘) 𝑖, 𝑗 =

𝜓⁽

𝑘) 𝑖 , 𝜓⁽

𝑘) 𝑗

. (2.2.6)

Writing Φ^(𝑘) :=span{𝜙^(𝑘)

𝑖 | 𝑖 ∈ I^(𝑘)} and𝔙^(𝑘) := span{𝜓^(𝑘)

𝑖 | 𝑖 ∈ I^(𝑘)}, Φ^(𝑘) ⊂ Φ^(𝑘+1) and Ψ^(𝑘) = L⁻¹Φ^(𝑘) imply Ψ^(𝑘) ⊂ Ψ^(𝑘+1). We further write [𝜙^(𝑘), 𝑢] =

[𝜙^(𝑘)

𝑖 , 𝑢]

𝑖∈I^(𝑘) ∈R^I

(𝑘). The(𝜙^(𝑘)

𝑖 )_𝑖∈I(𝑘) and (𝜓^(𝑘)

𝑖 )_𝑖∈I(𝑘) form a bi-orthogonal system in the sense that [𝜙^(𝑘)

𝑖 , 𝜓^(𝑘)

𝑗 ] =𝛿_{𝑖, 𝑗} for𝑖, 𝑗 ∈ I^(𝑘) (2.2.7) and the

·,·

-orthogonal projection of𝑢 ∈ H^𝑠

0(Ω)onΨ^(𝑘) is 𝑢^(𝑘) := Õ

𝑖∈I^(𝑘)

[𝜙⁽

𝑘) 𝑖 , 𝑢]𝜓⁽

𝑘)

𝑖 . (2.2.8)

Multiple interpretations of operator adapted pre-wavelets Using operator-adapted pre-wavelets,𝜓^(𝑘)

𝑖 , we summarize the connections between optimal recovery, game theory, and Gaussian conditioning. First, we define Gaussian fields, a generalization of Gaussian processes.

Definition 2.2.1. The canonical Gaussian field 𝜉 associated with operator L : H^𝑠

0(Ω) → H^−𝑠(Ω) is defined such that 𝜙 ↦→ [𝜙, 𝜉] is the linear isometry from H^−𝑠(Ω) to a Gaussian space characterized by











[𝜙, 𝜉] ∼ N (0,k𝜙k_∗²) Cov [𝜙, 𝜉],[𝜑, 𝜉]

= h𝜙, 𝜑i_∗,

(2.2.9)

where k𝜙k_∗ =sup_𝑢∈H^𝑠

0(Ω)

∫

Ω𝜙𝑢

k𝑢k is the dual norm ofk · k.

Remark 2.2.2. When𝑠 > 𝑑/2, the evaluation functional𝛿_𝑥(𝑓) = 𝑓(𝑥) is continu- ous. Hence,𝜉|𝛿_𝑥,𝑥∈Ωis naturally isomorphic to a Gaussian process with covariance function𝑘(𝑥 , 𝑥⁰) =h𝛿_𝑥, 𝛿_𝑥0i_∗.

Several notable properties of these pre-wavelets are summarized in the following result. Recall we write[𝜙^(𝑘), 𝑢] = [𝜙^(𝑘)

𝑖 , 𝑢]

𝑖∈I^(𝑘) ∈R^I

(𝑘). Theorem 2.2.3. Consider pre-wavelets 𝜓⁽

𝑘)

𝑖 adapted to operator L constructed with measurement functions 𝜙^(𝑘)

𝑖 . Further, suppose that for 𝑢 ∈ H^𝑠

0(Ω) we define 𝑣^†(𝑢) =𝑢^(𝑘) =Í

𝑖∈I^(𝑘)[𝜙^(𝑘)

𝑖 , 𝑢]𝜓^(𝑘)

𝑖 . 1. For fixed𝑢 ∈ H^𝑠

0(Ω),𝑣^†(𝑢) is the minimizer of











Minimizek𝜓k Subject to𝜓 ∈ H^𝑠

0(Ω)and [𝜙^(𝑘), 𝜓] =[𝜙^(𝑘), 𝑢].

(2.2.10)

2. For fixed𝑢 ∈ H^𝑠

0(Ω),𝑣^†(𝑢) is the minimizer of











Minimizek𝑢−𝜓k Subject to𝜓 ∈span{𝜓^(𝑘)

𝑖 :𝑖 ∈ I^(𝑘)}.

(2.2.11)

3. For canonical Gaussian field𝜉 ∼ N (0,L⁻¹), 𝑣^†(𝑢) =E

𝜉

[𝜙^(𝑘), 𝜉] = [𝜙^(𝑘), 𝑢]

. (2.2.12)

4. It is true that2

𝑣^†∈argmin_{𝑣∈𝐿(Φ,H}^𝑠

0(Ω)) sup

𝑢∈H^𝑠

0(Ω)

k𝑢−𝑣(𝑢) k

k𝑢k (2.2.13)

Proof. (1) is a result of [101, Cor. 3.4] (2) is equivalent to [101, Thm. 12.2] (3) and

(4) are results in [101, Sec. 8.5].

This result shows that the operator-adapted pre-wavelets transform defined by 𝑣^†(𝑢) = 𝑢^(𝑘) is an optimal recovery in the sense of Theorem 2.2.3.1-2. Simul- taneously, 𝑣^†(𝑢) are conditional expectations of the canonical Gaussian field with respect to the measurements [𝜙^(𝑘),·] as in Theorem 2.2.3.3. Another interpreta- tion of the transform is game theoretic as expressed in Theorem2.2.13.4. Equation (2.2.13) represents the adversarial two player game where player I selects𝑢 ∈ H^𝑠

0(Ω) and player II approximates𝑢 with𝑣(𝑢) with measurements [𝜙^(𝑘), 𝑢]. Player I and II aim to maximize and minimize the recovery error of𝑣(𝑢). This game theoretic interpretation inspires the name gamblets, referring to operator-adapted wavelets.

Note that the pre-wavelets𝜓^(𝑘)

𝑖 lie on only one level of the hierarchy. The following addresses the construction of a wavelet decomposition ofH^𝑠

0(Ω)on all hierarchical levels.

Operator-adapted wavelets

Let(J^(𝑘))_{2≤𝑘≤𝑞}be a hierarchy of labels such that, writing|J^(𝑘)|for the cardinal of J^(𝑘),

|J^(𝑘)|=|I^(𝑘)| − |I^(𝑘−1)|. (2.2.14) For𝑘 ∈ {2, . . . , 𝑞}, let𝑊^(𝑘) be a J^(𝑘) × I^(𝑘) matrix such that3

Ker(𝜋^{(𝑘−1, 𝑘)}) =Im(𝑊^(𝑘),𝑇). (2.2.15)

For𝑘 ∈ {2, . . . , 𝑞}and𝑖 ∈ J^(𝑘) define 𝜒^(𝑘)

𝑖 := Õ

𝑗∈I^(𝑘)

𝑊^(𝑘)

𝑖, 𝑗 𝜓^(𝑘)

𝑗 , (2.2.16)

and write 𝔚^(𝑘) := span{𝜒^(𝑘)

𝑖 | 𝑖 ∈ J^(𝑘)}. Then 𝔚^(𝑘) is the

·,·

-orthogonal complement of𝔙^(𝑘−1) in𝔙^(𝑘), i.e. 𝔙^(𝑘) = 𝔙^(𝑘−1) ⊕𝔚^(𝑘),and

𝔙^(𝑞) = 𝔙⁽¹⁾ ⊕𝔚⁽²⁾ ⊕ · · · ⊕𝔚^(𝑞). (2.2.17)

2𝐿(Φ,H^𝑠

0(Ω)) is defined as the set ofH^𝑠

0(Ω) → H^𝑠

0(Ω) functions that are of form 𝑣(𝑢) = Ψ [𝜙^(𝑘), 𝑢])

with measureableΨ:R^I

(𝑘) → H₀^𝑠(Ω).

3We write𝑀^(𝑘),𝑇 and𝑀^(𝑘),−1for the transpose and inverse of a matrix𝑀^(𝑘).

For𝑘 ∈ {2, . . . , 𝑞}write

𝐵^(𝑘) :=𝑊^(𝑘)𝐴^(𝑘)𝑊^(𝑘),𝑇 . (2.2.18) Note that𝐵^(𝑘) is the stiffness matrix of the elements(𝜒^(𝑘)

𝑗 )_𝑗∈J(𝑘), i.e., 𝐵^(𝑘)

𝑖, 𝑗 = 𝜒^(𝑘)

𝑖 , 𝜒^(𝑘)

𝑗

. (2.2.19)

Further, for𝑘 ∈ {2, . . . , 𝑞}, define

𝑁^(𝑘) := 𝐴^(𝑘)𝑊^(𝑘),𝑇𝐵^(𝑘),−1 (2.2.20)

and, for𝑖 ∈ J^(𝑘),

𝜙^{(𝑘), 𝜒}

𝑖 := Õ

𝑗∈I^(𝑘)

𝑁^(𝑘),𝑇

𝑖, 𝑗 𝜙^(𝑘)

𝑗 . (2.2.21)

Then defining𝑢^(𝑘) as in (2.2.8), it holds true that for𝑘 ∈ {2, . . . , 𝑞},𝑢^(𝑘)−𝑢^(𝑘−1) is the

·,·

-orthogonal projection of𝑢 on𝔚^(𝑘) and 𝑢^(𝑘) −𝑢^(𝑘−1) = Õ

𝑖∈J^(𝑘)

[𝜙⁽

𝑘), 𝜒 𝑖 , 𝑢]𝜒⁽

𝑘)

𝑖 . (2.2.22)

To simplify notations, write J⁽¹⁾ :=I⁽¹⁾, 𝐵⁽¹⁾ := 𝐴⁽¹⁾, 𝑁⁽¹⁾ := 𝐼⁽¹⁾, 𝜙⁽¹⁾

, 𝜒

𝑖 := 𝜙⁽¹⁾

𝑖

for𝑖 ∈ J⁽¹⁾,J := J⁽¹⁾∪ · · · ∪ J^(𝑞), 𝜒_𝑖 := 𝜒⁽

𝑘)

𝑖 and𝜙

𝜒 𝑖 :=𝜙⁽

𝑘), 𝜒

𝑖 for𝑖 ∈ J^(𝑘) and4 1≤ 𝑘 ≤ 𝑞. Then the𝜙

𝜒

𝑖 and 𝜒_𝑖 form a bi-orthogonal system, i.e., [𝜙

𝜒

𝑖 , 𝜒_𝑗] =𝛿_{𝑖, 𝑗} for𝑖, 𝑗 ∈ J (2.2.23) and

𝑢^(𝑞) =Õ

𝑖∈J

[𝜙

𝜒

𝑖 , 𝑢]𝜒_𝑖. (2.2.24)

Simplifying notations further, we will write [𝜙^𝜒, 𝑢] for the J vector with entries [𝜙

𝜒

𝑖 , 𝑢]and 𝜒for theJ vector with entries 𝜒_𝑖so that (2.2.24) can be written 𝑢^(𝑞) = [𝜙^𝜒, 𝑢] ·𝜒 . (2.2.25) Further, define the J by J block-diagonal matrix 𝐵 defined as 𝐵_{𝑖, 𝑗} = 𝐵^(𝑘)

𝑖, 𝑗 if 𝑖, 𝑗 ∈ J^(𝑘) and 𝐵_{𝑖, 𝑗} = 0 otherwise. Note that it holds that 𝐵_{𝑖, 𝑗} =

𝜒_𝑖, 𝜒_𝑗

. When 𝑞 =∞and∪^∞

𝑘=1Φ^(𝑘) is dense inH^−𝑠(Ω), then, writing𝔚⁽¹⁾ := 𝔙⁽¹⁾, H^𝑠

0(Ω) =⊕^∞

𝑘=1𝔚^(𝑘), (2.2.26)

4The dependence on𝑘is left implicit to simplify notation, for𝑖 ∈ Jthere exists a unique𝑘such that𝑖∈ J^(𝑘).

𝑢^(𝑞) = 𝑢, and (2.2.24) is the corresponding multi-resolution decomposition of 𝑢. When 𝑞 < ∞, 𝑢^(𝑞) is the projection of 𝑢 on ⊕^𝑞

𝑘=1𝔚^(𝑘) and (2.2.25) is the corresponding multi-resolution decomposition. Note that the optimal recovery, game theory, and Gaussian conditioning results in Theorem 2.2.3 also holds for wavelets.

Theorem 2.2.4. Consider pre-wavelets 𝜒_𝑖 adapted to operatorL constructed with measurement functions𝜙^𝜒. Further, suppose that for𝑢 ∈ H^𝑠

0(Ω), we define𝑣^†(𝑢)= 𝑢^(𝑞) = [𝜙^𝜒, 𝑢] ·𝜒.

1. For fixed𝑢 ∈ H^𝑠

0(Ω),𝑣^†(𝑢) is the minimizer of











Minimizek𝜓k Subject to𝜓 ∈ H^𝑠

0(Ω) and[𝜙^𝜒, 𝜓] =[𝜙^𝜒, 𝑢].

(2.2.27)

2. For fixed𝑢 ∈ H^𝑠

0(Ω),𝑣^†(𝑢) is the minimizer of











Minimizek𝑢−𝜓k

Subject to𝜓 ∈span{𝜒_𝑖 :𝑖 ∈ J }.

(2.2.28)

3. For canonical Gaussian field𝜉 ∼ N (0,L⁻¹), 𝑣^†(𝑢) =E

𝜉

[𝜙^𝜒, 𝜉] = [𝜙^𝜒, 𝑢]

. (2.2.29)

4. It is true that5

𝑣^† ∈argmin𝑣∈𝐿(Φ,H₀^𝑠(Ω)) sup

𝑢∈H^𝑠

0(Ω)

k𝑢−𝑣(𝑢) k

k𝑢k . (2.2.30)

Pre-Haar wavelet measurement functions

The gamblets used in the subsequent developments will use pre-Haar wavelets (as defined below) as measurement functions𝜙⁽

𝑘)

𝑖 and our main near-optimal denoising estimates will be derived from their properties (summarized in Thm.2.2.5).

Let𝛿, ℎ ∈ (0,1). Let(𝜏⁽

𝑘)

𝑖 )_𝑖∈I(𝑘)be uniformly Lipschitz convex sets forming a nested partition of Ω, i.e., such that Ω = ∪_𝑖∈I(𝑘)𝜏^(𝑘)

𝑖 , 𝑘 ∈ {1, . . . , 𝑞} is a disjoint union except for the boundaries, and 𝜏^(𝑘)

𝑖 = ∪_𝑗∈I(𝑘+1):𝑗^(𝑘)=𝑖𝜏^(𝑘+1)

𝑗 , 𝑘 ∈ {1, . . . , 𝑞−1}.

5Here 𝐿(Φ,H₀^𝑠(Ω)) is defined as the set ofH₀^𝑠(Ω) → H₀^𝑠(Ω) functions that are of form 𝑣(𝑢)= Ψ [𝜙^𝜒, 𝑢])

with measureableΨ:R^J → H₀^𝑠(Ω).

Assume that each𝜏^(𝑘)

𝑖 , contains a ball of radius𝛿 ℎ^𝑘, and is contained in the ball of radius𝛿⁻¹ℎ^𝑘. Writing |𝜏⁽

𝑘)

𝑖 |for the volume of𝜏⁽

𝑘) 𝑖 , take 𝜙⁽

𝑘)

𝑖 :=1

𝜏^(𝑘)

𝑖

|𝜏⁽

𝑘)

𝑖 |⁻¹² . (2.2.31)

The nesting relation (2.2.1) is then satisfied with 𝜋^{(𝑘 , 𝑘+1)}

𝑖, 𝑗 := |𝜏^(𝑘+1)

𝑗 |¹²|𝜏^(𝑘)

𝑖 |⁻¹² for 𝑗^(𝑘) =𝑖and𝜋⁽

𝑘 , 𝑘+1)

𝑖, 𝑗 :=0 otherwise.

For𝑘 ∈ {2, . . . , 𝑞}, let J^(𝑘) be a finite set of 𝑘-tuples of the form 𝑗 = (𝑗₁, . . . , 𝑗_𝑘) such that{𝑗^(𝑘−1) | 𝑗 ∈ J^(𝑘)} =I^(𝑘−1), and for𝑖 ∈ I^(𝑘−1), Card{𝑗 ∈ J^(𝑘) | 𝑗^(𝑘−1) = 𝑖} =Card{𝑠 ∈ I^(𝑘) |𝑠^(𝑘−1) =𝑖} −1. Note that the cardinalities of these sets satisfy (2.2.14).

Write 𝐽^(𝑘) for the J^(𝑘) × J^(𝑘) identity matrix. For 𝑘 = 2, . . . , 𝑞, let 𝑊^(𝑘) be a J^(𝑘) × I^(𝑘) matrix such that Im(𝑊^(𝑘),𝑇) =Ker(𝜋^{(𝑘−1, 𝑘)}),𝑊^(𝑘)(𝑊^(𝑘))^𝑇 =𝐽^(𝑘) and 𝑊^(𝑘)

𝑖, 𝑗 =0 for𝑖^(𝑘−1) ≠ 𝑗^(𝑘−1).

Theorem 2.2.5. With pre-Haar wavelet measurement functions, it holds true that 1. For𝑘 ∈ {1, . . . , 𝑞} and𝑢 ∈ L⁻¹𝐿²(Ω),

k𝑢−𝑢^(𝑘)k ≤𝐶 ℎ^{𝑘 𝑠}kL𝑢k_𝐿2(Ω). (2.2.32) 2. Writing Cond(𝑀) for the condition number of a matrix 𝑀, we have for

𝑘 ∈ {1,· · · , 𝑞}

𝐶⁻¹ℎ^{−2(𝑘−1)𝑠}𝐽^(𝑘) ≤ 𝐵^(𝑘) ≤ 𝐶 ℎ^{−2𝑘 𝑠}𝐽^(𝑘) (2.2.33) andCond(𝐵^(𝑘)) ≤𝐶 ℎ^−2𝑠.

3. For𝑖 ∈ I^(𝑘) and𝑥⁽

𝑘)

𝑖 ∈𝜏⁽

𝑘)

𝑖 ,

k𝜓_𝑖k_H𝑠(Ω\𝐵(𝑥^(𝑘)

𝑖 ,𝑛 ℎ)) ≤ 𝐶 ℎ^−𝑠𝑒^−𝑛/𝐶. (2.2.34)

4. The wavelets𝜓⁽

𝑘) 𝑖 , 𝜒⁽

𝑘)

𝑖 and stiffness matrices 𝐴^(𝑘), 𝐵^(𝑘) can be computed to precision 𝜖 (in k · k-energy norm for elements of H^𝑠

0(Ω) and in Frobenius norm for matrices) inO(𝑁log^{3𝑑 𝑁}_𝜖) complexity.

Furthermore the constant𝐶 depends only on𝛿,Ω, 𝑑 , 𝑠, kL k := sup

𝑢∈H₀^𝑠(Ω)

kL𝑢k_H^−𝑠_(Ω) k𝑢k_H^𝑠

0(Ω)

and kL⁻¹k := sup

𝑢∈H₀^𝑠(Ω)

k𝑢k_H^𝑠

0(Ω)

kL𝑢k_H^−𝑠_(Ω) .

(2.2.35)

Proof. (1) and (2) follows from an application of Prop. 4.17 and Theorems 4.14 and 3.19 from [100]. (3) follows from Thm. 2.23 of [100]. 4 follows from the complexity analysis of Alg. 6 of [100]. See [101] for detailed proofs.

Remark 2.2.6. The wavelets 𝜓⁽

𝑘) 𝑖 , 𝜒⁽

𝑘)

𝑖 and stiffness matrices 𝐴^(𝑘), 𝐵^(𝑘) can also be computed in O(𝑁log²𝑁log^{2𝑑 𝑁}_𝜖) complexity using the incomplete Cholesky factorization approach of [119].

Theorem 2.2.5.2-3 implies that the gamblets are localized both in the eigenspace of operator L and inΩspace. Further, Theorem2.2.5.1 shows the accuracy of the recovery,𝑢^(𝑘), in L norm is bounded by𝐿²norm of L𝑢. This result is used in the proofs of the denoising result shown in the following section.

2.3 Denoising by truncating the gamblet transform