Asynchronous Polynomial Filters on Graphs

RANDOM NODE-ASYNCHRONOUS UPDATES ON GRAPHS

2.6 Asynchronous Polynomial Filters on Graphs

Some updates might be adversarial that increase the variation of the signal over the graph. See Figure 1 of [184] for an illustrative example. Nevertheless, some other updates cancel them out in the long run as proven by Theorem 2.2.

Then, random updates on ℎ(A) as in(2.70) converge to an eigenvector of Awith eigenvalue𝜆_𝑗 for any amount of asynchronicity0≤ 𝛿_T ≤ 1.

Proof. Sinceℎ(·)is assumed to satisfy (2.72), ℎ(A)has a unit eigenvalue, and the non-unit eigenvalues are strictly less than one in magnitude. Hence, Corollary 2.2 ensures that the updates converge to a point in the eigenspace of ℎ(A) with the eigenvalue 1, which is equivalent to the eigenspace ofAwith the eigenvalue𝜆_𝑗 due

to the property in (2.72).

Polynomial filters play an important role in the area of graph signal processing.

Starting from the early works [153, 151, 160], polynomial filters are designed to achieve a desired frequency response on the graph. On the contrary, the condition (2.72) disregards the overall response of the filter since its objective is to isolate a single eigenvector. Thus, the design procedure and the properties of the polynomials (to be presented in the subsequent sections) differ from the polynomial approximation ideas considered in [153, 151, 160].

Theorem 2.4 tells that an arbitrary eigenvector of the graph can be computed in a decentralized manner if a low order polynomial satisfying (2.72) is constructed.

In this regard the updates on polynomial filters resemble the beam steering in antenna arrays [202]: assume that the order of the polynomial, 𝐿, is fixed. Then, the communication pattern between the nodes is completely determined by the operator A and the order 𝐿 (See Algorithm 1). Once the nodes start to update their values randomly and asynchronously, the behavior of the signal is controlled by the polynomial coefficients. By changing the coefficients in the light of (2.72), one cansteer the signal over the graph to desired “directions,” which happen to be the eigenvectors of the operator. Here, 𝐿 corresponds to the number of elements (sensors) in the array, and the filter coefficients serve the purpose of steering in both cases.

Note that the condition in (2.72) is not necessary in general for the convergence of random updates with a fixed amount of asynchronicity. (See Section 2.4.) However, (2.72) is necessary to guarantee the convergence for all levels of asynchronicity including the most restrictive case of power iteration.

Notice that if the operator A is the graph Laplacian, and the target eigenvalue is 𝜆_𝑗 =0, then the corresponding eigenvector is the constant vector. In this case the problem reduces to the consensus problem, and the condition in (2.72) becomes a relaxed version of the polynomial considered in [150].

In the following sections we will assume that eigenvalues ofAare real valued, which is the case for undirected graphs, and assume that ℎ_𝑛 ∈R. Although the complex case can also be considered in this framework, as we shall see, some of the results do not extend to the complex case.

2.6.1 The Optimal Polynomial

In this section we consider the construction of the polynomial that has the largest gap between the unit eigenvalue and the rest. In order to represent the condition (2.72) in the matrix-vector form we will use a vector of length𝐿+1 to denote the polynomial in (2.69), that is, h=[ℎ

0 · · · ℎ_𝐿]^T. In addition, let𝚽be a Vandermonde matrix constructed with the eigenvalues ofAin the following form:

𝚽=





 1 𝜆

1 𝜆²

1 · · · 𝜆^𝐿

1 𝜆

2 𝜆²

2 · · · 𝜆^𝐿

.. .

. · · · . .. 1 𝜆

𝑁 𝜆²

𝑁 · · · 𝜆^𝐿

𝑁







∈R^𝑁×(𝐿+¹⁾. (2.73)

In the case of repeated eigenvalues, we assume that the repeated rows of the matrix 𝚽are removed. Letφ(𝑗)denote the row of𝚽corresponding to the target eigenvalue 𝜆_𝑗, and let ¯𝚽𝑗 denote the remaining rows of𝚽.

In order to find the optimal𝐿^{𝑡 ℎ} order polynomial satisfying (2.72), we consider the following optimization problem:

max 𝑐, h

𝑐 s.t.

φ(𝑗) h=1,

𝚽¯ 𝑗 h

≤ (1−𝑐) 1. (2.74) First of all notice that the constraints in (2.74) are linear due to the fact that𝚽and hare real valued. The objective function is linear as well. Hence, (2.74) is a linear programming that can be solved efficiently given the eigenvalue matrix𝚽.

The constraints of (2.74) enforce the polynomial to satisfy the desired condition in (2.72) while the objective function maximizes the distance between the unit and non-unit eigenvalues ofℎ(A). Therefore, the formulation in (2.74) searches for the polynomial that yields the fastest rate of convergence onℎ(A)among all polynomials of order 𝐿 satisfying (2.72). Hence, we will refer to the solution of (2.74) asthe optimal polynomial.

It should be noted that the solution of (2.74) is optimal with respect to the worst case scenario of the synchronous updates. In general, the polynomial generated

via (2.74) may not give the fastest rate of convergence for an arbitrary amount of asynchronicity.

2.6.2 Sufficiency of Second Order Polynomials

In order to make use of the construction in (2.74), the order of the polynomial, 𝐿, should be selected appropriately so that the problem is feasible and admits a solution. One way to guarantee the feasibility is to select𝐿 = 𝑁−1, in which case a solution always exists due to the invertibility of the Vandermonde matrix𝚽 (by disregarding the multiplicity of eigenvalues). In this case, however, updates are no longer localized, which prevents the asynchronous model of (2.5) from being useful.

At the other extreme, the case of𝐿 =1 is insufficient to ensure the condition (2.72) in general. Therefore, nonlocal updates are required for the sake of flexibility in the eigenvectors. Nevertheless, the locality of the updates needs only to be compromised marginally, as the following theorem shows that𝐿 =2 is in factsufficientto satisfy (2.72).

Theorem 2.5. Assume that the operatorAhas real eigenvalues𝜆_𝑖 ∈R. For a given target eigenvalue 𝜆_𝑗, the condition in (2.72) is satisfied by the following second order polynomial:

ℎ(𝜆) =1−2𝜖 (𝜆−𝜆_𝑗)²/𝑠²

𝑗, (2.75)

for any𝜖 in0< 𝜖 < 1and𝑠_𝑗 satisfying the following:

𝑠_𝑗 ≥ max

1≤𝑖≤𝑁

|𝜆_𝑖−𝜆_𝑗|. (2.76)

Proof. It is clear thatℎ(𝜆_𝑗) =1. In the following we will show that−1< ℎ(𝜆_𝑖) < 1 for all𝜆_𝑖 ≠𝜆_𝑗. For the upper bound note that(𝜆_𝑖−𝜆_𝑗)²> 0 for all𝜆_𝑖 ≠𝜆_𝑗. There- fore,

1−ℎ(𝜆_𝑖)=2𝜖 (𝜆_𝑖−𝜆_𝑗)²/𝑠²

𝑗 > 0, (2.77)

which proves thatℎ(𝜆_𝑖) < 1 for all𝜆_𝑖 ≠𝜆_𝑗. For the lower bound notice that we have 𝑠²

𝑗 ≥ (𝜆_𝑖−𝜆_𝑗)²for all𝜆_𝑖by the condition in (2.76). Therefore we have ℎ(𝜆_𝑖) =1−2𝜖 (𝜆_𝑖−𝜆_𝑗)²/𝑠²

𝑗 ≥1−2𝜖 > −1, (2.78)

for all𝜆_𝑖.

Notice that 𝜖 in (2.75) is a free parameter which can be tuned to increase the gap between the eigenvalues. Thus, the polynomial given in (2.75) isnotguaranteed to

be optimal in general. It merely shows that a second order polynomial satisfying (2.72) always exists, which also implies the feasibility of (2.74) in the case of𝐿 =2, or larger.

An important remark is as follows: the sufficiency of second order polynomials does not extend to the complex case in general. To see this consider the following set of 𝑁 complex numbers: 𝜆_𝑛=𝑒^𝑗²^{𝜋𝑛/(𝑁}^-1⁾ for 1≤ 𝑛 ≤ 𝑁-1 and𝜆_𝑁 =0. As shown in the supplementary document, no polynomial of order 𝐿 ≤ 𝑁-2 (possibly with complex coefficients) can satisfy|ℎ(𝜆_𝑛) | < 1 for 1≤ 𝑛 ≤ 𝑁-1 andℎ(𝜆_𝑁) =1. This adversarial example shows not only that second order polynomials are insufficient, but also that a polynomial of order 𝑁-1 is in fact necessary in the complex case in general. Although no guarantee can be provided, low order polynomials might still exist in the complex case depending on the values of the eigenvalues of a given operatorA.

2.6.3 Spectrum-Blind Construction of Suboptimal Polynomials

Although the solution of (2.74) provides the optimal polynomial, it requires the knowledge of all the eigenvalues of A. Such information is not available and difficult to obtain in general. By compromising the optimality, we will discuss a way of constructing second order polynomials satisfying (2.72) withoutusing the knowledge of all eigenvalues ofA, except the target eigenvalue𝜆_𝑗.

First of all notice that a value for the coefficient 𝑠_𝑗 used in (2.75) can be found using only the minimum and the maximum eigenvalues of the operator. That is, the following selection

𝑠_𝑗 =max{𝜆

max−𝜆_𝑗, 𝜆_𝑗 −𝜆

min}, (2.79)

satisfies (2.76). Therefore, the minimum, the maximum and the target eigenvalues suffice to construct the polynomial (2.75).

In fact (2.76) can be satisfied by using an appropriate upper bound for 𝜆

max and lower bound for𝜆

min. For example if Ais the adjacency matrix or Laplacian, we can use the largest degree𝑑

maxof the graph to select𝑠_𝑗 as follows:

The Laplacian

In this case the eigenvalues are bounded as 0 ≤𝜆_𝑖 ≤ 2𝑑

max. Hence, 𝑠_𝑗 =𝑑

max+ |𝜆_𝑗−𝑑

max|. (2.80)

The Adjacency

In this case the eigenvalues are bounded as−𝑑

max≤ 𝜆_𝑖 ≤ 𝑑

max. Hence, 𝑠_𝑗 = 𝑑

max+ |𝜆_𝑗|. (2.81)

The Normalized Laplacian

In this case the eigenvalues are bounded as 0 ≤𝜆_𝑖 ≤ 2. Hence,

𝑠_𝑗 =1+ |𝜆_𝑗−1|. (2.82)

Since the selections in (2.80), (2.81), and (2.82) do not use the eigenvalues of the (corresponding) operatorA, the polynomial in (2.75) can be constructed using only the target eigenvalue𝜆_𝑗.

Aforementioned constructions also provide a trade-off between the available information and the rate of convergence. This point will be elaborated in Section 2.7.

2.6.4 Inexact Spectral Information and The Use of Nonlinearity

In this subsection, we will focus on the construction of a second order polynomial when the target eigenvalue𝜆_𝑗 is not known exactly. In this regard, we first assume that we are given an interval [𝑎 𝑏] to whichonlythe eigenvalue𝜆_𝑗 belongs. More precisely, we assume the following:

𝜆_𝑗

-1 < 𝑎 ≤𝜆_𝑗 ≤ 𝑏 < 𝜆_𝑗

+1, (2.83)

where we consider only the distinct eigenvalues indexed in ascending order, and assume that eigenvalues are real. Then, we consider the following polynomial:

ℎ(𝜆)=1−2𝜖

(𝜆−𝑎) (𝜆−𝑏) max{𝜆

upp−𝑏, 𝑎−𝜆

low} + (𝑏−𝑎)/2₂ (2.84) for some 𝜖 in 0< 𝜖 < 1. It should be noted that when the target eigenvalue 𝜆_𝑗 is known exactly, we can take 𝑎 =𝑏=𝜆_𝑗, in which case the polynomial in (2.84) reduces to the one in (2.75) with𝑠_𝑗 selected as in (2.79). In the case of 𝑎≠ 𝑏, one can observe that the polynomial in (2.84) maps the eigenvalues of the operatorAas follows:

ℎ(𝜆_𝑗) >1 and |ℎ(𝜆_𝑖) | < 1 ∀𝜆_𝑖 ≠𝜆_𝑗, (2.85) which shows that ℎ(A) does not have a unit eigenvalue, thus the asynchronous updates running on ℎ(A) do not converge. In fact, the signal would diverge due

to the dominant eigenvalue, ℎ(𝜆_𝑗), being strictly larger than 1. (See Figure 2.2).

In order to prevent the signal from diverging, we consider the followingsaturated update model:

x𝑘

𝑖 =









 𝑓

ℎ(A)x𝑘-1

𝑖

, 𝑖 ∈ T𝑘, x𝑘-1

𝑖, 𝑖∉T𝑘,

(2.86) where 𝑓(·)is the “saturation nonlinearity” defined as follows:

𝑓(𝑥) =sign(𝑥) ·min{|𝑥|, 1}, (2.87) which is visualized in Figure 2.6.

-5 -4 -3 -2 -1 0 1 2 3 4 5

x -1

0 1

f(x)

Figure 2.6: Visualization of the saturation nonlinearity defined in (2.87).

Notice that the boundedness of the function 𝑓(·)ensures that the signalx𝑘 in (2.86) does not diverge. In fact, it is numerically observed that randomized asynchronous updates in (2.86) indeed converge to the fixed point of the model. That is, x𝑘

converges tobx, wherebxsatisfies the following equation:

𝑓 ℎ(A)bx

=bx, (2.88)

where 𝑓(·)is assumed to operate element-wise on a vector.

The key observation regarding the solution of (2.88) is the following: when the interval [𝑎 𝑏] in (2.83) is small, then bx is a good approximation of the target eigenvectorv𝑗. That is,

bx≈ 𝛾 v𝑗 (2.89)

for some scale factor𝛾 ∈R. In fact, when the target eigenvalue is known exactly, i.e., 𝑎=𝑏 =𝜆_𝑗, then the approximation in (2.89) becomes equality. Therefore, we conclude that the asynchronous saturated update scheme in (2.86) allows us to find an arbitrary eigenvector approximately when the corresponding eigenvalue is not known exactly. Furthermore, as we have a better approximation of the eigenvalue, we get a better approximation of the corresponding eigenvector. Section 2.7 will make use of this observation to obtain an autonomous clustering of a network.

2.6.5 Implementation of Second Order Polynomials

In Section 2.6.2 graph signals are shown to converge to an arbitrary eigenvector of the underlying graph operatorAthrough random asynchronous updates running on an appropriate second order polynomial filter. In this section, we will show that asynchronous updates on a second order polynomial can be implemented as a node-asynchronous communication protocol in which nodes follow a collect- compute-broadcast scheme independently from each other. For this purpose, we first write a second polynomial ofAexplicitly as follows:

ℎ(A)= ℎ

0I+ℎ

1A+ℎ

2A², (2.90)

where the filter coefficients ℎ

0, ℎ

1, ℎ

2 are assumed to be pre-determined such that (2.72) is satisfied for the eigenvalue of the target eigenvector. Then, we define an auxiliary variableyas:

y=A x, (2.91)

where xdenotes the signal on the graph, and yis the “graph shifted” signal. We will assume that the𝑖^{𝑡 ℎ} node stores𝑥_𝑖 and 𝑦_𝑖 simultaneously. Thus, (𝑥_𝑖, 𝑦_𝑖) can be considered as the state of the𝑖^{𝑡 ℎ} node. Then, we can write the following:

(ℎ(A)x)𝑖 = ℎ

0𝑥_𝑖+ℎ

1 𝑦_𝑖+ℎ

2 (A y)𝑖. (2.92) Using (2.92), asynchronous updates with 𝛿_T =1 (only one node is updated per iteration) running onℎ(A)can be equivalently written in the following three steps:

𝑢 ← (ℎ

0−1)𝑥_𝑖+ℎ

1𝑦_𝑖+ℎ

2A[𝑖,:]y, (2.93)

𝑥_𝑖 ←𝑥_𝑖+𝑢, (2.94)

y←y+A[:,𝑖]𝑢, (2.95)

whereA[𝑖,:] andA[:,𝑖] denote the𝑖^{𝑡 ℎ} row and column ofA, respectively.

It is important to note that equations in (2.93)-(2.95) are in the form of a collect- compute-broadcast scheme. In (2.93), the termA[𝑖,:]yrequires the𝑖^{𝑡 ℎ}node to collect 𝑦_𝑗’s from all𝑗 ∈ N_in(𝑖). In (2.94), the node simply updates its own signal. In (2.95), the termA[:,𝑖]𝑢 requires the𝑖^{𝑡 ℎ} node to broadcast𝑢to all 𝑗 ∈ N_out(𝑖). These three steps can be converted into a node-asynchronous communication protocol as in Algorithm 1.

Algorithm 1 consists of three procedures: initialization, active stage, and passive stage. In the initialization, the 𝑖^{𝑡 ℎ} node assigns a random value to its own signal

Algorithm 1Node-Asynchronous Communication Protocol procedureInitialization(𝑖)

Initialize𝑥_𝑖 randomly.

Collect𝑥_𝑗 from nodes 𝑗 ∈ N_in(𝑖). 𝑦_𝑖 ←Í

𝑗∈N_in(𝑖) 𝐴_{𝑖, 𝑗} 𝑥_𝑗. procedurePassive Stage(𝑖)

ifa broadcast𝑢is received from the node 𝑗 then 𝑦_𝑖 ← 𝑦_𝑖+ 𝐴_{𝑖, 𝑗} 𝑢.

ifthe node 𝑗 sends a requestthen Send𝑦_𝑖 to node 𝑗.

procedureActive Stage(𝑖)

Collect𝑦_𝑗 from nodes 𝑗 ∈ N_in(𝑖). 𝑢← (ℎ

0−1)𝑥_𝑖+ℎ

1𝑦_𝑖+ℎ

𝑗∈N_in(𝑖) 𝐴_{𝑖, 𝑗} 𝑦_𝑗. 𝑥_𝑖 ←𝑥_𝑖+𝑢.

Broadcast𝑢to all nodes 𝑗 ∈ N_out(𝑖).

𝑥_𝑖, then it constructs the auxiliary variable 𝑦_𝑖 by collecting𝑥_𝑗 from its neighbors.

Once the initialization is completed, the𝑖^{𝑡 ℎ}node waits in the passive stage, in which the graph signal𝑥_𝑖 is not updated. However, its neighbors can request 𝑦_𝑖, or send a broadcast. When a broadcast is received, the𝑖^{𝑡 ℎ} node updates only its auxiliary variable 𝑦_𝑖. When the𝑖^{𝑡 ℎ} node wakes up randomly, it gets into the active stage, in which it collects the auxiliary variable 𝑦_𝑗 from its neighbors, updates its signal𝑥_𝑖, and then broadcasts the amount of update to its neighbors. Then, the node goes back to the passive stage.

Five comments are in order: 1) The random update model in (2.9) implies that all nodes have the same probability of going into the active stage. 2) As the signal x converges to the target eigenvector ofA, the ratio 𝑦_𝑖/𝑥_𝑖 converges to the corresponding eigenvalue ofA(assuming𝑥_𝑖 is non-zero), thus ℎ(𝑦_𝑖/𝑥_𝑖) converges to 1 due to (2.72). 3) In the active stage, the broadcast (the step in (2.95)) is essential to ensure thatxandysatisfy (2.91). 4) The amount of update for𝑥_𝑖is computed by the𝑖^{𝑡 ℎ} node itself. The amount of update for𝑦_𝑖 is dictated by the neighbors of the 𝑖^{𝑡 ℎ} node. Thus,𝑦_𝑖can also be considered as a buffer. 5) Since edges are allowed to be directed, a node may collect data from the 𝑗^{𝑡 ℎ} node in the active stage, but may not send data back to the 𝑗^{𝑡 ℎ} node.

As a final remark we note that Algorithm 1 assumes reliable communication between the nodes, i.e., no link failures. In this case Algorithm 1 is exactly equivalent to the model in (2.5) running onℎ(A). As long as the polynomial coefficients are selected

properly (see Theorem 2.4), the signalxin Algorithm 1 is guaranteed to converge to the eigenvector targeted by the polynomial. In the case of link failures, Algorithm 1 deviates from the model in (2.5), thus the convergence guarantees presented here are not applicable. Nevertheless, we have numerically observed that Algorithm 1 converges even in the case of random link failures. This case will be studied in future.

Dalam dokumen Signals on Networks: Random Asynchronous and Multirate Processing, and Uncertainty Principles (Halaman 69-78)