Convergence of the Proposed Algorithm For convenience of analysis we defineFor convenience of analysis we define

IIR FILTERING ON GRAPHS WITH RANDOM NODE-ASYNCHRONOUS UPDATES

3.4 Convergence of the Proposed Algorithm For convenience of analysis we defineFor convenience of analysis we define

uniformly randomly. However, the computational stage of [74] consists of a linear mapping followed by a sigmoidal function, whereas Algorithm 5 uses a linear update model. More importantly, aggregations are done synchronously in [74], that is, all nodes are required to complete the necessary computations before proceeding to the next level of aggregation. On the contrary, nodes aggregate information repetitively andasynchronouslywithout waiting for each other in Algorithm 5.

3.4 Convergence of the Proposed Algorithm

follows:

x⁰_𝑖 = Õ

𝑗∈N_in(𝑖)

𝐺_{𝑖, 𝑗} x𝑗 =Õ

𝑗

X₍𝑘-1) e_𝑗 e^T_𝑗 G^Te_𝑖 =X₍𝑘-1) G^T e_𝑖. (3.40) Therefore, the next value for its state vector is given as follows:

X(𝑘)e𝑖 =

A X(𝑘-1)G^T+b(u+w(𝑘-1))^T

e𝑖, 𝑖 ∈ T𝑘. (3.41) On the other hand, if the node𝑖does not get into the active stage at the𝑘^{𝑡 ℎ} iteration, i.e.,𝑖∉T𝑘, its state vector remains unchanged. Thus, we can write the following:

X(𝑘) e𝑖 =X(𝑘-1) e𝑖, 𝑖 ∉T𝑘. (3.42) Since both (3.41) and (3.42) are linear in the augmented state variable matrixX(𝑘), we can transpose, and then vectorize both equations and represent them as follows:

(x¯𝑘)𝑖 =











(A¯ x¯𝑘-1)𝑖+ (u¯𝑘-1)𝑖, 𝑖 ∈ T¯𝑘, (x¯𝑘-1)𝑖, 𝑖 ∉T¯𝑘,

(3.43) where the variables of the vectorized model are as follows:

x¯𝑘 =vec X^T_(𝑘)

, A¯ =A⊗G,

u¯ =b⊗u, w¯𝑘 =b⊗w(𝑘), (3.44) and ¯u𝑘 is defined similar to (3.2) as ¯u𝑘 =u¯ +w¯𝑘. Furthermore, the update set ¯T𝑘 of the vectorized model is defined as follows:

¯ T𝑘 =

𝑖+ 𝑗 𝑁 | 𝑖 ∈ T𝑘, 0≤ 𝑗 < 𝐿 , (3.45) which follows from the fact that when a node gets into the active stage, it updates all elements of its own state vector simultaneously according to Line 10 of the algorithm.

We note that the mathematical model in (3.43) appears as a pull-like algorithm, in which nodes retrieve data from their incoming neighbors. However, with the use of a buffer, the model (3.43) can be implemented in a collect-compute-broadcast scheme as proposed in Algorithm 5. See also Figure 3.1.

When the algorithm is implemented in a synchronous manner, the state recursions of (3.43) reduce to the following form:

x¯𝑘 =A¯ x¯𝑘-1+u¯𝑘-1, (3.46) and the following theorem (whose proof is provided in Section 3.7.3) presents the mean-squared error of the algorithm:

Theorem 3.2. In Algorithm 5, assume that all the nodes on the graph get into the active stage synchronously, and the matrixA¯ does not have an eigenvalue equal to 1. Then,

E

y(𝑘) −eu ²

2

=

c⊗G

A¯^𝑘-1 x¯₀−x¯^★

2 2

+

𝑘−1

Õ

𝑛=0

|ℎ_𝑛|² tr

G^𝑛𝚪(G^𝑛)^H

, (3.47) where x¯^★ is the fixed point of (3.46), and ℎ_𝑛’s are the coefficients of the impulse response of the digital filter as in(3.34).

In (3.47) it is clear that as long as

𝜌(A)¯ <1, (3.48)

the first term of (3.47) converges to zero irrespective of the initial vector ¯x₀, as the iteration progresses. So, from Theorem 3.2 the residual error approaches an error floor:

𝑘→∞lim E

y(𝑘) −eu ²

2

=tr(H𝚪), (3.49)

where

H=

∞

Õ

𝑛=0

|ℎ_𝑛|²(G^𝑛)^H G^𝑛. (3.50) Thus, the error floor in the synchronous case depends on theimpulse responseof the underlying digital filter as well as the graph operator, but the similarity transform Tdoes not affect the error floor. In short, the similarity transform does not affect either the convergence or the error floor in the synchronous case. Note that the stability condition in (3.48) ensures the convergence of (3.50). Note also that 𝜌(A¯) = 𝜌(A)𝜌(G)in view of (3.44).

Next consider the asynchronous case. The equivalent model of the algorithm in (3.43) is in the form of (3.12), thus the results presented in Section 3.2 (Corollary 3.1 in particular) can be used to study the convergence of the algorithm. In this regard, we present the following theorem, whose complete proof is given in Section 3.7.4:

Theorem 3.3. In Algorithm 5, letPdenote the average node selection matrix and𝚪 the covariance matrix of the measurement noise. If the state transition matrixAof the filter, and the operatorGof the graph satisfy the following:

kAk²

2G^H P G ≺P, (3.51)

then

𝑘lim→∞ E

ky(𝑘)− euk²

2

≤ tr(R𝚪), (3.52)

where

R= kbk²

2 kck²

2 kGk²

2

𝜆min

P− kAk²

2G^HP G P+ |𝑑|²I. (3.53) Theorem 3.3 presents an upper bound on the mean-squared error. In the noise-free case (𝚪=0), the right-hand-side of (3.52) becomes zero, and the condition (3.51) ensures the convergence of the output signal to the desired filtered signal in the mean-squared sense. We note also that the right-hand-side of (3.52) is linear in the noise covariance matrix, which implies that the error floor of the algorithm increases at most linearly with the input noise. This will be numerically verified later in Section 3.5.1. (See Figure 3.4b.) In fact, it is possible to integrate stochastic averaging techniques studied in [6, 7] into Algorithm 5 in order to overcome the error due to noise at expense of a reduced convergence rate.

We conclude by noting that graph filtering implementations considered in [158, 149, 150, 154, 157, 87, 88, 108, 109] are likely to tolerate asynchronicity up to a certain degree. In fact, [109] presented numerical evidences in this regard. This is not surprising because linear asynchronous fixed-point iterations are known to converge under some conditions [32, 14]. The main difference of Algorithm 5 studied in this chapter is due to its proven convergence under some mild and interpretable conditions with the assumed random asynchronous model (Theorem 3.3).

3.4.1 Selection of the Similarity Transform

In addition to the dependency on the graph operator and the average node selection matrix, the sufficiency condition (3.51) depends also on the realization of the filter of interest. Thus, in the asynchronous case, both the condition for convergence and the error bound depend on the similarity transform. Since the condition becomes more relaxed as the state transition matrixAhas a smaller spectral norm, it is important to select the similarity transformTin (3.35) in such a way thatAhas the minimum spectral norm.

Due to their robustness, minimum-norm realizations of digital filters have been studied extensively in signal processing [12, 120, 198]. A minimum-norm imple- mentation corresponds to an appropriate selection of the similarity transformTin (3.35) due to the following inequality:

kAk₂≥ 𝜌(A)= 𝜌(bA). (3.54)

The lower bound𝜌(bA)depends only on the coefficients of the polynomial𝑞(𝑥)due to the definition ofAbin (3.36).

The lower bound in (3.54) maynotbe achieved with equality in general, and we will consider one such example in the next section. Nevertheless, it is known that the companion matrixAbis diagonalizable if and only if the digital filter in (3.34) has𝐿 distinct poles [85]. That is to say, when there are 𝐿 distinct nonzero𝑧_𝑛’s such that 𝑞(𝑧^-1

𝑛) =0, we can write the following eigenvalue decomposition:

Ab=V

bA 𝚲

bA V^-1

Ab, (3.55)

where𝚲

bAis a diagonal matrix with𝑧^-1

𝑛’s on the diagonal, andV

bAis a Vandermonde matrix corresponding to𝑧^-1

𝑛’s. If the similarity transformTis selected according to (3.55), then the bound in (3.54) is indeed achieved. More precisely,

T=V

bA =⇒ A=𝚲

bA =⇒ kAk₂=𝜌(bA). (3.56) Thus, the most relaxed version of the sufficiency condition of Theorem 3.3 is obtained when the updates of Algorithm 5 are implemented using the similarity transform given in (3.56).

When the filter (3.34) has repeated poles, the companion matrix Ab is not diago- nalizable, hence an implementation achieving the bound (3.54) does not exist [12].

Nevertheless, the study [12] discussed that for any𝜖 > 0, there exists a realization with a state transition matrixAsuch that

kAk₂≤ 𝜌(bA) +𝜖 . (3.57)

Therefore, it is always possible to obtain “almost minimum" realizations with the spectral norm arbitrarily close to the lower bound in (3.54). As a particular example, the case of FIR graph filters will be considered in the next section.

3.4.2 The Case of Polynomial Filters

Polynomial (FIR) graph filters can be considered as a special case of the rational graph filter (3.29), in which the denominator is selected as𝑞(𝑥) =1 so that𝑞(G) =I, and the filtered signal in (3.30) reduces toeu= 𝑝(G)u. In this case, the companion

matrixbA(direct form implementation) has the following form:

bA=



























0 1 0 · · · 0 0 0 1 · · · 0

.. .

.. .

.. .

... .. . 0 0 0 · · · 1 0 0 · · · 0



























∈R^𝐿×𝐿, (3.58)

which has all eigenvalues equal to zero, so that𝜌(bA) =0. As a result, no realization of a polynomial filter can achieve the lower bound (3.54) since kAk₂ =0 implies A=0. However, the spectral norm of a realization can be made arbitrarily small.

In particular, consider the following similarity transform:

T=diag [1 𝜖 𝜖² · · · 𝜖^𝐿-1]

, (3.59)

where𝜖 is an arbitrary nonzero complex number. Then, the corresponding realiza- tionAcan be found as follows:

A=T^-1A Tb =𝜖Ab =⇒ kAk₂ =|𝜖|. (3.60) Thus, it is possible to select a value for 𝜖 (with a sufficiently small magnitude) in order to satisfy the condition (3.51). (See [101, Fact 2.5.4].) Such a selection is not unique in general, and one can easily find a value for𝜖 satisfying the following:

|𝜖| <

kGk₂

q kP

2

P^-1 2

_-1

, (3.61)

which ensures that the condition (3.51) is met.

As a result, for any graph operator G and average node selection matrix P, it is always possible to implementanypolynomial filter in a random node-asynchronous manner that is guaranteed to converge in the mean-squared sense. However, we note thatTgiven in (3.59) may not be the optimal similarity transform in general.

We also note that when a polynomial filter is implemented in a synchronous manner, Theorem 3.2 shows that the algorithm reaches the error floor after𝐿iterations since Ab in (3.58) is a nil-potent matrix and Ab^𝑛 =A^𝑛 =0 for 𝑛 ≥ 𝐿. This convergence behavior will be verified numerically later in Section 3.5.4. The error bound still depends onTbecause of kAk₂.