Interpretation in the Context of Graph Signal Processing

RANDOM NODE-ASYNCHRONOUS UPDATES ON GRAPHS

2.5 Interpretation in the Context of Graph Signal Processing

In Sections 2.2 and 2.3 we defined and studied the random asynchronous updates from a linear algebraic point of view where the results (Theorems 2.1, 2.2, and their corollaries) are general enough to apply them to an arbitrary normal matrix. WhenA is assumed to be a graph operator, the asynchronous iterations offer an insightful way to interpret a “typical” graph signal. Consider a network of fully autonomous nodes (agents) and assume that a node updates its value via retrieving information from its neighbors. This update scheme can be modeled with (2.5) where the update set T has only one element (𝛿_T =1). In this model, the local graph operatorAmodels the way nodes update their values. If a node computes the sum of its neighbors, thenAis the adjacency matrix; if a node computes the sum of differences with its neighbors, then Ais the graph Laplacian; if a node computes a weighted average of its neighbors, then Ais a weighted adjacency matrix. Therefore, the matrix A describes what the nodes compute, the update scheme in (2.5) describes the time dynamics of the nodes, and the values held by the nodes are considered as a signal on the graph (with respect to the operator A). In the following, we will interpret random asynchronous updates in this context and argue that a “typical signal” on a graph is necessarily a “smooth signal” with respect to the graph operator. For this purpose first notice that the GFT ofAx−xis given by(𝚲−I)

bx. Then, we define a notion of smoothness accordingly as follows:

Definition 2.1(Smoothness Set). A graph signalxbelongs to the setS𝜖 if its graph Fourier transformbx=V^Hxsatisfies

|b𝑥_𝑖| |𝜆_𝑖−1| ≤ 𝜖 ∀𝑖, (2.58) for the given graph operatorA.

A signal x belongs to S𝜖 if the differences between the graph Fourier coefficients of x and Ax are not larger than 𝜖 in absolute sense. That is, small values of 𝜖 implies that x and Ax are similar to each other. Hence, we can interpret 𝜖 as a scale of the smoothness of the signal xwith respect to the operator A. Here, the smoothness is quantified according to the total variation (TV) of the eigenvectors, TV(v𝑖) = |𝜆_𝑖−1|, as introduced in [153]. For a given value of𝜖, the setS𝜖describes the signals with|

b𝑥_𝑖| ≤ 𝜖/ |𝜆_𝑖−1|. So, for a smooth signal, the Fourier coefficientb𝑥_𝑖

should be smaller for those𝜆_𝑖 whosev𝑖 has large total variation. The condition in (2.58) is equivalent to bounding a weighted max-norm of the GFT ofx, that is,

x ∈ S𝜖 ⇐⇒ k (𝚲−I)bxk_∞ ≤ 𝜖 , (2.59) where the weight matrix is selected as|𝚲−I|. The setS𝜖 depends on the underlying graph operator. A signal that is smooth on one graph may not be smooth on another graph.

In the following we will consider the effect of a single asynchronous update on the smoothness of the signal. For this purpose let xbe the initial signal and y be the signal after an update. According to (2.7) they are related asy=Q(T )x, which can be equivalently written in the graph Fourier domain as follows:

by=V^HQ(T )Vbx=bx+Õ

𝑖∈T

V^He_𝑖e^H_𝑖 V(𝚲−I)bx. (2.60)

The following theorem reveals a relation between smooth graph signals and a single asynchronous update of (2.5).

Theorem 2.3. Assume that the signalxbelongs toS𝜖 of a graph with operatorA.

Then, the signalycomputed as in(2.5)satisfies the following:

kby−

bxk_∞ ≤ 𝜖 |T | kVk_max kVk_∞. (2.61) Proof. Assume thatx∈ S𝜖. Then, we can write the following set of inequalities for a fixed index 𝑗:

|b𝑦_𝑗− b𝑥_𝑗|=

e^H_𝑗 Õ

𝑖∈T

V^He𝑖e^H_𝑖 V(𝚲−I)bx

, (2.62)

≤ Õ

𝑖∈T

e^H_𝑗V^He𝑖

e^H_𝑖 V(𝚲−I)bx

, (2.63)

≤ kVk_max Õ

𝑖∈T

e^H_𝑖 V(𝚲−I)bx

, (2.64)

≤ kVk_max Õ

𝑖∈T

V^He𝑖

1

(𝚲−I)bx

_∞, (2.65)

≤ kVk_max |T | kVk∞ 𝜖 , (2.66)

where (2.62) follows from (2.60), (2.63) follows from the triangle inequality, (2.64) follows from the definition of kVk_max, (2.65) follows from the Hölder inequality,

and (2.66) follows from the fact that kVk_∞ is the largestℓ

1-norm of the rows ofV. Then, we have the following:

kby−bxk∞ =max

𝑗

|b𝑦_𝑗 −

b𝑥_𝑗| ≤ 𝜖 |T | kVk_max kVk∞, (2.67) where the inequality follows from the fact that the bound in (2.66) is valid for any

index 𝑗.

If the underlying graph is circulant, then V is the normalized DFT matrix, and Theorem 2.3 reduces to the following simple form as a corollary: k

by−

bxk_∞ ≤ 𝜖 |T | [184].

The bound given by Theorem 2.3 is not tight in general. Nevertheless, it provides a useful interpretation: if the initial signal is smooth on the graph (belongs to S𝜖

with 𝜖 being small), then the amount of change in each GFT coefficient is also small, hence a smooth signal remains to be (relatively) smooth on the graph after a single asynchronous update. If the initial signal is not smooth (𝜖 is large), then the right-hand-side of (2.61) is a large quantity, and thus we cannot reach a conclusion regarding the effect of an asynchronous update.

We will say thata signalxis typical to the graph (with respect to the operatorA) if it satisfiesAx=x. This definition is motivated by the following three observations.

First of all Lemma 2.2 shows that a typical signal is invariant (stationary) under any asynchronous update. Secondly, a signal is typical if and only if it is the smoothest signal (w.r.t. Definition 2.1):

x∈ S₀ ⇐⇒ A x=x. (2.68)

This is consistent with the studies in [152, 160, 113] that consider typical signals to be smooth on the graph. Thirdly, and most importantly, Corollary 2.2 proves that when the nodes communicate autonomously for a sufficiently long time, the signal becomes typical to the graph irrespective of the starting point. Once the signal gets typical it stays the same over the network. Therefore, a steady signal over an autonomous network is necessary typical with respect to the operatorA.

The equivalence given in (2.68) also shows the smoothing effect of the asynchronous updates. An arbitrary initial signal is not expected to be smooth over a graph, but Corollary 2.2 proves that the signal over an autonomous network eventually becomes typical, hence the smoothest. However, it should be noted that the smoothing (convergence) does not happen monotonically unlike the power iteration of (2.3).

Some updates might be adversarial that increase the variation of the signal over the graph. See Figure 1 of [184] for an illustrative example. Nevertheless, some other updates cancel them out in the long run as proven by Theorem 2.2.