Energy Decomposition

OPERATOR COMPRESSION VIA ENERGY DECOMPOSITION

2.1 Energy Decomposition

We start by considering the linear system L x = b, where L is the Laplacian of an undirected, positive-weighted graphG= {V;E,W}, i.e.

Li j =



















P(i,j⁰)∈Ewi j⁰ ifi= j;

−wi j ifi, jand(i, j) ∈E;

0 otherwise.

(2.1)

We allow for the existence of self-loops(i,i) ∈E. When Lis singular we mean to solve x = L^†b, where L^† is the pseudo-inverse of L. Our algorithm will base on a fast clustering technique using local spectral information to give a good partition of the graph, upon which special local basis will be constructed and used to compress

the operatorL⁻¹into a low dimensional approximationL⁻_com¹ subject to a prescribed accuracy.

As we will see, our clustering technique exploits local path-wise information of the graphG by operating on each single edge inE, which can be easily adapted to a larger class of linear systems with symmetric, positive semidefinite matrix. Notice that the contribution of an edge(i, j) ∈E with weightwi j to the Laplacian matrix Lis simply

Eii ,

i

* . . ,

+ / / -

0

wii i

0

, i = j; Ei j ,

i j

* . . . . . . . . . ,

+ / / / / / / / / / -

0

wi j −wi j i . . .

−w_{i j} w_{i j} j

0

, i , j.

(2.2) And we have L = P

(i,j)∈EEi j. In view of such matrix decomposition, our algo- rithm works for any symmetric, positive semidefinite matrix A that has a similar decomposition A = Pm

k=1Ek with each Ek 0. Here Ek 0 means Ek is posi- tive semidefinite. Therefore, we will theoretically develop our method for general decomposable PD matrices. Also we assume that Ais invertible, as we can easily generalize our method to the case when A^†bis pursued.

We therefore introduce the idea of energy decomposition and the corresponding mathematical formulation which motivates the methodology for solving linear sys- tems with energy decomposable linear operator. LetAbe an×nsymmetric, positive definite matrix. We define theenergy decompositionas follows:

Definition 2.1.1(Energy Decomposition). We call {Ek}^m_k=1anenergy decomposi- tionof Aand Ek to be anenergy elementof Aif

A=

m

X

k=1

Ek, Ek 0∀k =1, . . . ,m. (2.3)

Intuitively, the underlying structural(geometric) information of the original matrix Acan be realized through an appropriate energy decomposition. And to preserve as much detailed information of Aas possible, it is better to use the finest energy

decomposition that we can have, which actually comes naturally from the generat- ing of Aas we will see in some coming examples. More precisely, for an energy decompositionE = {Ek}_k^m₌₁of A, if there is someEkthat has its own energy decom- positionEk = Ek,1+Ek,2that comes naturally, then the finer energy decomposition E_{f ine} = {Ek}^m_k=2∪ {Ek,1,Ek,2}is more preferred as it gives us more detailed informa- tion of A. However one would see that anyEk can have some trivial decomposition Ek = ¹₂Ek+¹₂Ek, which makes no essential difference. To make it clear what should be the finest underlying energy decomposition ofAthat we will use in our algorithm, we first introduce the neighboring relation between energy elements and basis.

LetE ={Ek}^m_k=1be an energy decomposition of A, andV ={vi}ⁿ_i=1be an orthonor- mal basis ofRⁿ. we introduce the following notation:

• For any E ∈ E and anyv ∈ V, we denote E ∼ v if v^TEv > 0 ( or equivalently Ev ,0, sinceE 0);

• For anyu,v ∈ V, we denoteu∼ vif∃E ∈ E such thatu^TEv ,0.

As an immediate example, if we take V to be the set of all distinct eigen vectors of A, thenv / ufor any two v,u ∈ V, namely all basis functions are isolated and everything is clear. But such choice ofV is not trivial in that we know everything about A if we know its eigen vectors. Therefore, instead of doing things in the frequency space, we assume the least knowledge of A and work in the physical space, that is we will chooseV to be the natural basis{e_i}_iⁿ₌₁ ofRⁿ in all practical use. But for theoretical analysis, we still use the general basis notationV = {vi}_iⁿ₌₁. Also, for those who are familiar with graph theory, it is more convenient to un- derstand the setsV,E from graph perspective. Indeed, one can keep in mind that G= {V,E }is the generalized concept of undirected graphs, whereV stands for the set of vertices, andE stands for the set of edges. For any vertices (basis)v,u ∈ V, and any edge (energy)E ∈ E,v ∼ Emeans thatEis an edge ofv, andv∼ umeans that v and u share some common edge. However, different from the traditional graph setting, here one edge(energy)Emay involve multiple vertices instead of just two, and two vertices(basis)v,umay share multiple edges that involve different sets of vertices. Further, the spectrum magnitude of the “multi-vertex edge" E can be viewed as a counterpart of edge weight in graph setting. Conversely, if the problem comes directly from a weighted graph, then one can naturally construct the setsV andEfrom the vertices and edges of the graph as we will see in Example 2.1.10.

Definition 2.1.2 (Neighboring). Let E = {Ek}^m_k=1 be an energy decomposition of A, and V = {vi}_iⁿ₌₁ be an orthonormal basis of Rⁿ. For any E ∈ E, We define N(E;V) := {v ∈ V : E ∼v}to be the set ofv ∈ V neighboringE. Similarly, for anyv ∈ V, we defineN(v;E) := {E ∈ E : E ∼ v}andN(v) := {u ∈ V :u ∼ v} to be the set of E ∈ E and the set of u ∈ V neighboring v ∈ V respectively.

Furthermore, for anyS ⊂ V and anyE ∈ E, we denoteE ∼ SifN(E;V)∩ S ,∅ andE ∈ S ifN(E;V) ⊂ S.

In what follows, we will see that if two energy elements Ek,Ek⁰ have the same neighbor basis, namelyN(Ek;V)= N(Ek⁰;V), then there is no need to distinguish between them, since it is the neighboring relation between energy elements and basis that matters in how we make use of the energy decomposition. Therefore we say an energy decompositionE = {Ek}^m_k=1 is the finest underlying energy decomposition of Aif no Ek ∈ E can be further decomposed as

Ek = Ek,1+Ek,2,

where eitherN(Ek,1;V) &N(Ek;V)orN(Ek,2;V) & N(Ek;V). From now on, we will always assume thatE = {Ek}^m_k=1is the finest underlying energy decomposi- tion of Athat comes along with A.

Using theneighboringconcept between energy elements and orthonormal basis, we can then define various energies of a subsetS ⊂ V as follows:

Definition 2.1.3 (Restricted, Interior and Closed energy). Let E = {Ek}^m_k=1 be a energy decomposition of A. Let S be a subset of V, and PS be the orthogonal projection ontoS. Therestricted energyofS with respect toAis defined as

AS := PSAPS; (2.4)

Theinterior energyofSwith respect to AandEis defined as A^E_S = X

E∈S

E; (2.5)

Theclosed energyofS with respect to AandE is defined as A^E_S = X

E∈S

E+ X

E<S,E∼S

PSE^dPS, (2.6)

where

E^d = X

v∈V

X

u∈V

v^TEu

vv^T = X

v∼E

X

u∼v

v^TEu

vv^T (2.7)

is called thediagonal concentrationofE, and we have PSE^dPS = X

v∈S,v∼E

X

u∼v

v^TEu

vv^T (2.8)

Remark 2.1.4. The restricted energy ofScan be simply viewed as the restriction ofA on the subsetS. The interior energy (closed energy) ofSisASexcluding (including) contributions from other energy elements E < S neighboring S. The following example illustrates the idea of various energies introduced in Definition 2.1.3 by considering the 1-dimensional discrete Laplace operator with Dirichlet boundary conditions.

Example 2.1.5. Consider A to be the (n +1) × (n+ 1) tridiagonal matrix with entries -1 and 2 on off-diagonals and diagonal respectively. Let

E₁= _{2 −1}

−1 1

0

, En=

0

_{1 −1}

−1 2

, and Ek = * ,

0

_{1 −1}

−1 1

0

+ -

(2.9) for k = 2, . . .n− 1. Let V = {ei}_iⁿ₌₀ to be the standard orthonormal basis for Euclidean space Rⁿ⁺¹. Formally Ek is the edge between ek−1 and ek. If S = {e₃,e₄,e₅,e₆}, then we have

AS =* . . . ,

0

_{2 −1}

−1 2 −1

−1 2 −1

−1 2

0

+ / / / -

, A^E_S =* . . . ,

0

_{1 −1}

−1 2 −1

−1 2 −1

−1 1

0

+ / / / - ,

and A_S^E =* . . . ,

0

_{3 −1}

−1 2 −1

−1 2 −1

−1 3

0

+ / / / - .

Recall that the interior energy A^E_S =P

E_k∈SEk = P₆

k=4Ek, while the closed energy A_S^E = A^E_S+ X

E<S,E∼S

PSE^dPS

= A^E_S+|e^T₃E₃e₂|e₃e^T₃+|e^T₃E₃e₃|e₃e^T₃ +|e^T₇E₇e₆|e₆e^T₆+|e^T₆E₇e₆|e₆e^T₆ includes the partial contributions from other energy elements E < S neighboring S, which areE₃andE₇respectively.

Remark 2.1.6.

- Notice that any eigenvector x of AS (or A^E_S, A_S^E) corresponding to nonzero eigenvalue must satisfy x ∈ span(S). In this sense, we also say AS (or A^E_S, A^E_S) is local toS.

- For any energyE, we haveE E^d since for anyx = Pn

i=1civi, we have x^TE x = X

i

c_i²v_i^TEvi+X

i,j

2cicjv_i^TEvj

≤ X

i

c_i²v_i^TEvi+X

i,j

(c_i²+c²_j) v^T_i Evj

= X

i

X

j

c_i² v_i^TEvj

= x^TE^dx.

Proposition 2.1.7. For anyS ⊂ V, we have that A^E_S AS A^E_S. Proof. We have

A^E_S = X

E∈S

E X

E∈S

E+X

E<S, E∼S

PSE PS X

E∈S

E+X

E<S, E∼S

PSE^dPS = A^E_S.

Notice thatPSE PS = E forE ∈ S, andPSE PS =0 forE / S, thus AS = PSAPS = X

E∈S

E+X

E<S, E∼S

PSE PS,

and the desired result follows.

Definition 2.1.8(Partition of basis). LetV = {v_i}_iⁿ₌₁be an orthonormal basis ofRⁿ. We sayP = {Pj}^M_j=1is apartitionofV = {vi}_iⁿ₌₁if (i) Pj ⊂ V ∀j; (ii)Pj∩Pj⁰ =∅ if j , j⁰; and (iii)SM

j=1Pj =V.

Again one can see the partition of basis as partition of vertices. This partitionPis the key to construction of local basis for operator compression purpose. The following proposition serves to bound the matrix A from both sides with blocked(patched) matrices, which will further serve to characterize properties of local basis.

Proposition 2.1.9. Let E = {Ek}^m_k=1 be an energy decomposition of A, and P = {Pj}^M_j=1be a partition ofV. Then

M

X

j=1

A^E_P

j A

M

X

j=1

A^E_Pj. (2.10)

Proof. LetE_P = {E ∈ E : ∃Pj ∈ P such that E ∈ Pj}, and E_P^c = E\E_P. Recall that E ∈ Pj ifN(E,V) ⊂ Pj (See Definition 2.1.2). We will usePj to denote the orthogonal projection onto Pj. SincePj ∩Pj⁰ = ∅for j , j⁰, we haveP

jPj = Id. Then

X

j

A^E_P

j = X

j

X

E∈P_j

E

X

j

X

E∈P_j

E+ X

E∈E^c_P

E

X

j

X

E∈P_j

E+ X

E∈E^c_P

f X

j

Pj

E^d X

j⁰

Pj⁰g

= X

j

X

E∈P_j

E+ X

E∈E^c

P

X

j

PjE^dPj

= X

j

X

E∈P_j

E+ X

E<P_j,E∼Pj

PjE^dPj

= X

j

A^E_Pj.

We have used the fact thatPjE^dPj⁰ =0 for j , j⁰. Notice that A= X

E∈EP

E+ X

E∈E^c_P

E =X

j

X

E∈P_j

E+ X

E∈E^c_P

E,

and the desired result follows.

Throughout the thesis, we will always assume that Ahas a finest energy decompo- sitionE = {Ek}^m_k=1, and all the other discussed energies of Aare constructed from E with respect to some orthonormal basisV (by taking interior or closed energy).

Therefore we will simply use A_S,AS to denote A^E_S,A^E_S for anyS ⊂ V.

Example 2.1.10. ConsiderLto be the graph Laplacian matrix of the graph given in Figure 2.1. For graph Laplacian, an intrinsic energy decomposition arises during the assembling of the matrix in which the energy element is defined over each edge(see Equation (2.2)). Now suppose we have given the partition P = (

Pj

)₃

j=1

withP₁ =[e₁,e₂,e₃], P₂ =[e₄,e₅,e₆,e₇]andP₃ =[e₈,e₉,e₁₀,e₁₁], whereeiare the standard basis ofR¹¹. Then we can obtain L_P

j andLP_j as follows:

L_P1 = _{4 −2−2}

−2 4 −2

−2−2 4

P₁, L_P2 = ⁻−^{52 4}1⁻−²2 5 2^{−−12 0−2}

−2 0 −2 4

!

P₂

, L_P3 = ⁻−^{42 5}2⁻−²1 5^{−−2 01−}−²2 0 −2−2 4

!

P₃

LP₁ = _{6 −2−2}

−2 6 −2

−2−2 8

P₁, LP₂ = ⁻−^{72 4}1⁻−²2 7 2^{−−12 0−2}

−2 0 −2 8

!

P₂

, LP₃ = ⁻−^{62 5}2⁻−²1 9^{−−2 01−}−²2 0 −2−2 6

!

P₃

(a) (b)

Figure 2.1: (a) An illustration of a graph example . (b) An illustration of a partition P = {{1,2,3},{4,5,6,7},{8,9,10,11}}.

Here we denote the matrix(·)_Pj to be the matrix inR^11×11but with nonzero entries onPj only.