Numerical Example for MMD

MULTIRESOLUTION MATRIX DECOMPOSITION

3.4 Numerical Example for MMD

We have used the fact that

kP_Ψ^A(k)xk²_A= x^TAΨ^(k) (Ψ^(k))^TAΨ^(k)−1(Ψ^(k))^TAx

= x^TΦ^(k) (Φ^(k))^TA⁻¹Φ^(k)−1(Φ^(k))^Tx, ∀k ≥ 0. Therefore we have

kx− P_Φ^(K)xk₂² ≤

K

X

k=0

ε(P^(k),q^(k))² kxk²_A.

Remark 3.3.2.

- Though the compression error is in a cumulative form, if we assume that ε(P^(k),q^(k)) increases with k at a certain ratio _ε(P^ε(P(k−^(k)1)^,q,q^(k(k−))1)) ≥ γ for some γ >1, then it is easy to see that

K

X

k=1

ε(P^(k),q^(k))² ≤ γ²

γ²−1ε(P^(K),q^(K))², which is an error of scaleε(P^(K),q^(K))²as we expected.

- Again one shall be aware of the difference between the one-level compres- sion witherror factor ε(P^(K),q^(K))and the multi-level compression in Sec- tion 3.1.3. A one-level compression witherror factor ε(P^(K),q^(K))requires to constructP^(K),Φ^(k)and so on directly with respect toAin the whole space Rⁿ, which involves solving eigenvalue problems on considerably large patches in P^(K) when ε(P^(K),q^(K)) is a coarse scale. But the multi-level compres- sion in Section 3.1.3 is computed hierarchically with bounded compression ratio between levels, and thus only involves eigenvalue problems on patches of well-bounded size(s = O(log¹ +logn)^d+l) in each reduced space R^N

(k), and is thus more tractable in practice.

- One can also analyze the compression error when localization of each Ψ^(k) is considered. The analysis would be similar to the one in Theorem 3.3.1.

(a) Front view (b) Top view Figure 3.2: A “Roll surface” constructed by (3.33).

are randomly distributed around a 2-dimensional roll surface of area 1 inR³. The distribution is a combination of a uniform distribution over the surface and an up to 10% random displacement off the surface. More precisely, the two-dimensional roll is characterized as

(x(t),y(t),z) = (ρ(t)cos(θ(t)), ρ(t)sin(θ(t)),z), t ∈[0,1],z ∈[0,1], where

θ(t)= 1 alog

1+t e^4πa−1

, ρ(t)= a

√1+a²

t+ 1 e^4πa−1

, and sop

(ρ⁰(t))²+ (ρ(t)θ⁰(t))²= 1. Each vertex(xi,yi,zi)is generated by

(xi,yi,zi) = (ηiρ(ti)cos(θ(ti)), ηiρ(ti)sin(θ(ti)),zi), i =1,2,· · ·,n, (3.33) whereti

i.i.d

∼ U[0,1], zi i.i.d

∼ U[0,1], and ηi i.i.d

∼ U[0.9,1.1]. In this example we taken = 10000 anda = 0.1. Figure 3.2a and Figure 3.2b show the point cloud of all vertices. This explicit expression, however, is considered as a hidden geometric information, and is not employed in our partitioning algorithm.

The Laplacian L = A₀ and the energy decomposition E are given as in Exam- ple 2.1.10, and we apply a 4-level multiresolution matrix decomposition with lo- calization using Algorithm 6 to decompose the problem of solving L⁻¹. In this particular case we have λ_max(L) = 1.93×10⁷and λ_min(L) = 1. Again for graph Laplacian, we choose q = 1. On each level k, the partition is constructed sub- ject toε(P^(k),1)² = 10^k−6(i.e. {ε(P^(k),1)²}⁴_k=1 = {0.00001,0.0001,0.001,0.01}) and δ(P^(k),1)ε(P^(k),1)² ≤ 50. The compliment space U^(k) are extended from

Level Size #Nonzeros Condition Number Complexity L 10000×10000 128018,m 1.93×10⁷ 2.47×10¹²

BH⁽¹⁾ 1898×1898 9812≈0.07m 1.72×10² 1.69×10⁶ BH⁽²⁾ 6639×6639 391499≈3.06m 1.80×10¹ 7.04×10⁶ BH⁽³⁾ 1244×1244 417156≈3.26m 7.27×10¹ 3.03×10⁷ BH⁽⁴⁾ 186×186 34596≈0.27m 4.47×10¹ 1.55×10⁶ AH⁽⁴⁾ 33×33 1025≈0.008m 2.83×10³ 2.90×10⁶

total - 854088≈6.67m - 4.34×10⁷

Table 3.1: Complexity results of 4-level MMD with localization using Algorithm 6.

(a) Spectrum (b) Complexity

Figure 3.3: Spectrum and complexity of each layer in a 4-level MMD obtained from Algorithm 6.

Φ^(k) using a patch-wise QR factorization. So according to Corollary 3.2.1, each κ(BH^(k)), k = 2,3,4 is expected to be bounded by

δ(P^(k−1),1)ε(P^(k),1)² =δ(P^(k−1),1)ε(P^(k−1),1)² ε(P^(k),1)²

ε(P^(k−1),1)² ≤500, κ(BH⁽¹⁾) is expected bounded by λ_max(L)ε(P⁽⁰⁾,1)² = 1.93×10², and κ(HA⁽⁴⁾) is expected to be bounded by

δ(P⁽³⁾,1)λ_min(L)⁻¹= δ(P⁽³⁾,1)ε(P⁽³⁾,1)²λ_min(L)⁻¹

ε(P⁽³⁾,1)² ≤ 5000.

Since we will use a CG type method to compare the effectiveness of solving L⁻¹ directly and using the 4-level decomposition, the complexities of both approaches are proportional to the product of the number of nonzero (NNZ) entries and the condition number of the matrix concerned, given a fixed prescribed relative accuracy [102].

Therefore we define the complexity of a matrix as the product of its NNZ entries and

its condition number. Though here we use the sparsity ofB^(k) = (U^(k))^TA^(k−1)U^(k), in practice only the sparsity of A^(k−1) matters and the matrix multiplication with respect toU^(k) can be done by using the Householder vectors from the implicit QR factorization [102]. The results not only satisfy the theoretical prediction, but also turn out to be much better than expected as shown in Table 3.1 and Figure 3.3. We

Level Size #Nonzero Condition Number Complexity L 10000×10000 128018,m 1.93×10⁷ 2.47×10¹²

BH⁽¹⁾ 1898×1898 9812≈0.07m 1.72×10² 1.69×10⁶ BH⁽²⁾ 6639×6639 391499≈3.06m 1.80×10¹ 7.04×10⁶ BH⁽³⁾ 1014×1014 237212≈1.85m 2.56×10¹ 6.09×10⁶ BH⁽⁴⁾ 313×313 86323≈0.67m 1.62×10¹ 1.40×10⁶ BH⁽⁵⁾ 114×114 12996≈0.10m 5.54×10¹ 7.20×10⁵ AH⁽⁵⁾ 22×22 442≈0.004m 1.60×10³ 7.07×10⁵

total - 738284≈5.77m - 1.76×10⁷

Table 3.2: Complexity results of a 5-level MMD with localization using Algorithm 6.

(a) (b)

Figure 3.4: Spectrum and complexity of each layer in a 5-level MMD obtained from Algorithm 6.

now verify the performance of the 4-level and the 5-level MMDs by solving two particular systemsLu^∗ =b, where

Case 1: u_i^∗ = (x²_i + y_i²+ z²_i)¹², i= 1,2,· · ·,n,b= Lu^∗; Case 2: u_i^∗ = xi+ yi+sin(zi), i =1,2,· · ·,n,b= Lu^∗.

For both cases, we set the prescribed relative accuracy to be = 10⁻⁵ such that kuˆ−u^∗k_L ≤ kbk₂. By Theorem 3.2.3, this accuracy can be achieved by imposing

a corresponding accuracy control on each level’s linear system relative error (i.e., err^(K)_A and errB ) and localization error (errloc). In practice, instead of imposing a hard error control, we relax the localization error toε(P^(k),1)(instead of)in order to ensure sparsity, which is actually how we obtain the 4-level decomposition with localization. Such relaxed localization error can be fixed by doing a compensation correction at level-0, which takes the output of Algorithm 7 as an initialization to solveLu =b. As shown in the gray columns “# Iteration" in Table 3.3 and Table 3.4, the number of iterations in the compensation calculation (which is the computation at Level 0) are fewer than 25 in both cases, which indicate that the localization error on each level is still small even when relaxed.

Scale Matrix # Iteration Main Cost

Direct λ⁻_max¹ (L)∼λ_min⁻¹(L) L 572 7.32×10⁷

Level 4-level 5-level 4-lvl 5-lvl 4-lvl 5-lvl 4-lvl 5-lvl

0 10⁻⁵∼λ⁻¹_max(L) 10⁻⁵∼λ_max⁻¹ (L) BH⁽¹⁾ BH⁽¹⁾ 22 22 2.15×10⁵ 2.15×10⁵ 1 10⁻⁵∼10⁻⁴ 10⁻⁵∼10⁻⁴ BH⁽²⁾ BH⁽²⁾ 11 11 4.31×10⁶ 4.31×10⁶ 2 10⁻⁴∼10⁻³ 10⁻⁴∼3×10⁻³ BH⁽³⁾ BH⁽³⁾ 25 14 1.04×10⁷ 3.32×10⁶

3×10⁻³∼10⁻³ BH⁽⁴⁾ 14 1.21×10⁶

3 10⁻³∼10⁻² 10⁻³∼10⁻² BH⁽⁴⁾ BH⁽⁵⁾ 23 22 7.96×10⁵ 2.86×10⁵ 4 10⁻²∼λ⁻_min¹(L) 10⁻²∼λ⁻_min¹(L) AH⁽⁴⁾ AH⁽⁵⁾ 31 22 3.18×10⁴ 9.72×10³

0 - - L L 25 30 3.20×10⁶ 3.84×10⁶

Total - - - - 137 135 1.90×10⁷ 1.31×10⁷

Parallel - - - - - - 1.36×10⁷ 8.15×10⁶

Table 3.3: Case 1: Performance of the 4-level and the 5-level decompositions.

Scale Matrix # Iteration Main Cost

Direct λ⁻¹_max(L)∼λ_min⁻¹(L) L 586 7.50×10⁷

Level 4-level 5-level 4-lvl 5-lvl 4-lvl 5-lvl 4-lvl 5-lvl

0 10⁻⁵∼λ⁻_max¹ (L) 10⁻⁵∼λ_max⁻¹ (L) BH⁽¹⁾ BH⁽¹⁾ 24 24 2.35×10⁵ 2.35×10⁵ 1 10⁻⁵∼10⁻⁴ 10⁻⁵∼10⁻⁴ BH⁽²⁾ BH⁽²⁾ 11 11 4.31×10⁶ 4.31×10⁶ 2 10⁻⁴∼10⁻³ 10⁻⁴∼3×10⁻³ BH⁽³⁾ BH⁽³⁾ 25 14 1.04×10⁷ 3.32×10⁶

3×10⁻³∼10⁻³ BH⁽⁴⁾ 14 1.21×10⁶

3 10⁻³∼10⁻² 10⁻³∼10⁻² BH⁽⁴⁾ BH⁽⁵⁾ 23 23 7.96×10⁵ 2.99×10⁵ 4 10⁻²∼λ⁻¹_min(L) 10⁻²∼λ⁻¹_min(L) AH⁽⁴⁾ AH⁽⁵⁾ 30 22 3.08×10⁴ 9.72×10³

0 - - L L 16 23 2.05×10⁶ 2.94×10⁶

Total - - - - 129 131 1.78×10⁷ 1.23×10⁷

Parallel - - - - - - 1.24×10⁷ 7.25×10⁶

Table 3.4: Case 2: Performance of the 4-level and the 5-level decompositions.

In particular, we use a preconditioned CG method to solve any involved linear systems Au = b in Algorithm 7. The precondition matrix D is chosen as the

diagonal part of Aand we take0(all-zeros vector) as initials if no preconditioning vector is provided. The main computational cost of a single use of the PCG method is measured by the product of the number of iterations and the NNZ entries of the matrix involved (See the gray column “Main Cost” in both Table 3.3 and Table 3.4).

In both cases, we can see that the total computational costs of our approach are obviously reduced compared to the direct use of PCG. Moreover, since the downward level-wise computation can be done in parallel, the effective computational costs of our approach are even less, which is the sum of the maximal cost among all levels and the cost of the compensation correction (See “Parallel" row in Table 3.3 and Table 3.4 respectively).

Further, from the results we can see that the costs among all levels in both cases are mainly concentrated on level 2, namely, the inverting of BH⁽³⁾. This observation implies that the structural/geometrical details of L have more proportion on the scale corresponding to level 2 than on other scales, which is consistent to the fact that BH⁽³⁾ on level 2 has the largest complexity of all. Though in practice we do not have the information in Table 3.3 and Table 3.4, we may observe the dominance of time complexity on level 2 after numbers of calls of our solver. As a natural improvement, we can simply further decompose the problem at level 2. More precisely, to relieve the dominance of level 2, we add one extra scale of 0.0003 between the scales of 0.0001 and 0.001. Consequently, we obtain a similar 5-level decomposition with{ε(P^(k),1)²}⁵_k=1 = {0.00001,0.0001,0.0003,0.001,0.01}(See Table 3.2 and Figure 3.4). From the “Main Cost 5-level” columns of Table 3.3 and Table 3.4), we can see that this simple improvement of further decomposition based on the feedback of computational results does reduce the computational cost.

C h a p t e r 4