INTRODUCTION
1.3 Outline and Scope of the Thesis
This thesis studies various aspects of signal processing techniques in network set- tings. The first part of the thesis (Chapters 2, 3, 4, and 5) focuses on randomized and asynchronous variants of state recursions and studies their behavior in a statistical sense. The main theoretical finding of these sections is that random asynchronous implementations can stabilize systems that are otherwise unstable. The second part of the thesis studies extensions (and re-interpretation) of classical signal processing techniques to the case of graphs. Namely, Chapter 6 considers multirate signal processing techniques (and filter banks) for the graph case. Chapter 7 visits un- certainty principles and shows that time-frequency localization phenomena does not always extend to the case of graphs. Chapter 8 studies the optimal polynomial filter design problem (that maximizes the bass-band energy) for the case of graphs.
Finally, Chapter 9 considers the heat-diffusion process over networks, and studies the estimation of the starting time of a diffusion. In this section, the scope of each chapter will be briefly outlined.
1.3.1 Random Node-Asynchronous Updates (Chapter 2)
This chapter introduces a node-asynchronous communication protocol in which an agent in a network wakes up randomly and independently, collects states of its neighbors, updates its own state, and then broadcasts back to its neighbors. This protocol differs from consensus algorithms and it allows distributed computation of an arbitrary eigenvector of the network, in which communication between agents is allowed to be directed. In order to analyze the scheme, this chapter studies a random asynchronous variant of the power iteration where the update matrix is selected to be the graph operator of interest. Under this random asynchronous model, an initial signal is proven to converge to an eigenvector of eigenvalue 1 (a fixed point) even in the case of operator having spectral radius larger than unity.
In particular, the convergence region for the eigenvalues gets larger as the updates get less synchronous. The rate of convergence is shown to depend not only on the eigenvalue gap but also on the eigenspace geometry of the operator as well as the amount of asynchronicity of the updates. In particular, the rate of convergence is affected by the phase of the eigenvalues, and the randomized updates favor negative eigenvalues over positive ones. Random asynchronous updates are also interpreted from the graph signal perspective, and it is shown that a non-smooth signal on the graph converges to the smoothest signal under the random model. Polynomials of the operator are used to achieve convergence to an arbitrary eigenvector of the operator. When the eigenvalues are real, second order polynomials are shown to be sufficient for this. Using second order polynomials in the randomized update model, the chapter formalizes the node-asynchronous communication model whose convergence is readily proven. As an application, the protocol is used to compute the Fiedler vector of a network to achieve autonomous clustering. As another application, this chapter considers a reformulation of the component-wise updates revealing a randomized algorithm that is proven to converge to the dominant left and right singular vectors of a normalized data matrix. The algorithm is also extended to handle large-scale distributed data when computing an arbitrary rank approximation of an arbitrary data matrix. Numerical simulations verify the convergence of the proposed algorithms under different parameter settings.
1.3.2 IIR Filtering with Random Node-Asynchronous Updates (Chapter 3) This chapter proposes a node-asynchronous implementation of rational filters on arbitrary graphs. In the proposed algorithm nodes follow a randomized collect- compute-broadcast scheme: if a node is in the passive stage it collects the data
sent by its incoming neighbors and stores only the most recent data. When a node gets into the active stage at a random time instance, it does the necessary filtering computations locally, and broadcasts a state vector to its outgoing neighbors.
For the analysis of the algorithm, this chapter first considers a general case of randomized asynchronous state recursions and presents a sufficiency condition for its convergence. Based on this result, the proposed algorithm is proven to converge to the filter output in the mean-squared sense when the filter, the graph operator and the update rate of the nodes satisfy a certain condition. The proposed algorithm is simulated using rational and polynomial filters, and its convergence is demonstrated for various different cases, which also shows the robustness of the algorithm to random communication failures.
1.3.3 Random Asynchronous Linear Systems (Chapter 4)
This chapter extends the random asynchronous state recursions studied in Chap- ters 2 and 3 to a setting where the input is time-dependent, such as complex si- nusoids. It is based on a randomized asynchronous variant of linear discrete-time state-space models, in which each state variable gets updated with a non-zero prob- ability independently (and asynchronously) in every iteration. This chapter shows that such randomized systems behave very similar to the synchronous non-random counterpart in a statistical sense. In particular, the output of the randomized system with a sinusoidal input is still a sinusoid in expectation with the same frequency.
So, it is possible to consider the “frequency response” of such systems. This chapter also presents the necessary and sufficient condition for the mean-squared stability of the randomized system. It is shown that stability of the underlying state transi- tion matrix is neither necessary nor sufficient for the mean-squared stability of the randomized asynchronous recursions. However, randomization introduces an error depending on the update probabilities and the amount of variation (frequency) in the input signal. It is also shown that eigenvectors (and not just eigenvalues) of the state transition matrix are important in determining the stability of the randomized system, i.e., stability (or instability) property can be altered with a similarity trans- form. The chapter also revisits the special case of constant input, in which case the randomized system becomes randomized fixed-point iterations studied in Chapter 3 and stability conditions are less restrictive than in the non-random case.
1.3.4 Randomized Algorithms as Switching Systems (Chapter 5)
This chapter considers the random asynchronous update model studied in Chapters 2, 3, and 4 from the viewpoint of switching systems and shows that convergence (and stability) properties of these models follow directly from the stability theory already developed for switching systems. The chapter further shows that randomized versions of Kaczmarz’s method, Gauss-Seidel iterations, and asynchronous fixed- point iterations can be represented as specific instances of randomly switching systems. Then, the chapter presents alternative proofs for the mean-squared and almost sure convergence of randomized Kaczmarz and Gauss-Seidel methods. The necessary and sufficient condition for the mean-squared convergence of random asynchronous fixed-point iterations is also provided.
1.3.5 Extending Classical Multirate Signal Processing Theory (Chapter 6) This chapter extends classical multirate signal processing ideas to graphs and revisits ideas such as noble identities, aliasing, and polyphase decompositions in graph multirate systems. It is shown that the extension of classical multirate theory to graphs is nontrivial, and requires certain mathematical restrictions on the graph.
For example, classical noble identities cannot be taken for granted. Similarly, one cannot claim that the so-called delay chain system is a perfect reconstruction system (as in classical filter banks). It will also be shown that 𝑀-partite extensions of the bipartite filter bank results will not work for 𝑀-channel filter banks, but a more restrictive condition called𝑀-block cyclic property should be imposed. Such graphs are studied in detail. Building upon the basic theory of multirate systems for graph signals, 𝑀-channel polynomial filter banks on graphs are studied. The behavior of such graph filter banks differs from that of classical filter banks in many ways, the precise details depending on the eigenstructure of the adjacency matrix.
If the adjacency matrix is actually𝑀-block cyclic then perfect-reconstruction (PR) filter banks become practical, i.e., arbitrary filter polynomial orders are possible, and there are robustness advantages. In this case the PR condition is identical to PR in classical filter banks – any classical PR example can be converted to a graph PR filter bank on an𝑀-block cyclic graph. Polyphase representations are developed for graph filter banks and utilized to develop alternate conditions for alias cancellation and perfect reconstruction, again for graphs with specific eigenstructures. It is then shown that the eigenvector condition on the graph can be relaxed by using similarity transforms.
1.3.6 Uncertainty Principles and Sparse Eigenvectors (Chapter 7)
This chapter advances a new way to formulate the uncertainty principle for signals defined over graphs, by using a non-local measure based on the notion of sparsity.
To be specific, the total number of nonzero elements of a graph signal and its corresponding graph Fourier transform (GFT) is considered. A theoretical lower bound for this total number is derived, and it is shown that a nonzero graph signal and its GFT cannot be arbitrarily sparse simultaneously. When the graph has repeated eigenvalues, the graph Fourier basis (GFB) is not unique. Since the derived lower bound depends on the selected GFB, a method that constructs a GFB with the minimal uncertainty bound is provided. In order to find signals that achieve the derived lower bound (i.e. the most compact on the graph and in the GFB), sparse eigenvectors of the graph are investigated. It is shown that a connected graph has a 2-sparse eigenvector (of the graph Laplacian) when there exist two nodes with the same neighbors. In this case the uncertainty bound is very low, tight, and independent of the global structure of the graph. For several examples of classical and real-world graphs it is shown that 2-sparse eigenvectors, in fact, exist.
1.3.7 Energy Compaction Filters (Chapter 8)
In classical signal processing spectral concentration is an important problem that was first formulated and analyzed by Slepian. The solution to this problem gives the optimal FIR filter that can confine the largest amount of energy in a specific bandwidth for a given filter order. The solution is also known as the prolate sequence. This chapter investigates the same problem for polynomial graph filters.
The problem is formulated in both graph-free and graph-dependent fashions. The graph-free formulation assumes a continuous graph spectrum, in which case it becomes the polynomial concentration problem. This formulation has a universal approach that provides a theoretical reference point. However, in reality graphs have discrete spectrum. The graph-dependent formulation assumes that the eigenvalues of the graph are known and formulates the energy compaction problem accordingly.
When the eigenvalues of the graph have a uniform distribution, the graph-dependent formulation is shown to be asymptotically equivalent to the graph-free formulation.
However, in reality eigenvalues of a graph tend to have different densities across the spectrum. Thus, the optimal filter depends on the underlying graph operator, and a filter cannot be universally optimal for every graph.
1.3.8 Time Estimation for Heat Diffusion (Chapter 9)
This chapter studies the estimation of the starting time of a diffusion process from its noisy measurements when there is a single point source located on a known vertex of a graph with unknown starting time. The diffusion process is assumed to be governed by the heat equation. In particular, the Cramér-Rao lower bound (CRLB) for the problem is derived. It is shown that the problem has a larger CRLB for graphs with higher connectivity. Closed form expression of the bound is derived for some graphs. The Maximum Likelihood estimator is numerically verified to be unbiased, and achieves the CRLB for some graphs.
C h a p t e r 2