Further Discussions

FREQUENCY RESPONSE AND MEAN-SQUARED STABILITY

4.6 Further Discussions

50 100 150 200 250

k

-1 -0.5 0 0.5 1

I m{u_k} Re{u_k}

50 100 150 200 250

k

0 1 2 3

tr(Qk) tr(Q^rk)

(a)

50 100 150 200 250

k

-1 -0.5 0 0.5 1

I m{u_k} Re{u_k}

50 100 150 200 250

k

0 1 2 3

tr(Qk) tr(Q^rk)

(b)

Figure 4.3: Comparison between inputs (a)𝑢_𝑘 = 𝑒^{𝑗 𝜔 𝑘}, and (b)𝑢_𝑘 =cos(𝜔 𝑘) with frequency𝜔 =2𝜋/100. The first row shows the input signal, and the second row shows the trace of the error correlation matrix, which is obtained by averaging over 10⁶independent runs.

simplicity, in the rest of this section we assume that the input signal is𝑢_𝑘 =𝑒^{𝑗 𝜔 𝑘}. So, we have kx^ss_𝑘k = k (𝑒^{𝑗 𝜔}I−A¯)^-1B¯k, and the randomization error is given in (4.49).

Thus, SRR^𝑘 remains constant as a function of the iterations. However, the value of SRR^𝑘 depends on the input frequency 𝜔 as well as the update probabilities P. In order to demonstrate this point, we compute SRR numerically for the system in (4.27) for different values of𝜔and 𝑝. These results are presented in Figure 4.4.

Generally speaking, SRR tends to be larger as the input signal varies more or the state variables are updated less frequently.

We first note that the matrixAgiven in (4.27) has𝜌(A) > 1. However, for the case ofP= 𝑝I, we have numerically verified that the stability condition (4.41) is satisfied for 0< 𝑝 ≤ 0.9542. This is why the case of 𝑝 > 0.9542 is excluded in Figure 4.4.

Heuristically speaking, Figure 4.4 shows that state variables in the randomized case are trying to “keep up with the input signal.” As the input signal varies faster (larger values of𝜔), or the state variables are updated slower (small values of𝑝), the randomization error tends to be larger. However, SRR^𝑘isnotmonotonic in terms of the input frequency𝜔 and the update probabilitiesPin general. This complicated behavior follows from the fact that signal power and randomization error change

Figure 4.4: SRR of the example in (4.27) for different values of the input frequency 𝜔andP= 𝑝I. The plot is in dB scale, i.e., 10 log₁₀(SRR)is plotted. Dotted black curve corresponds to the case of 0 dB.

simultaneously with the input frequency and the update probabilities.

4.6.3 The Distribution of the Random Variables

Due to random selection of the indices, the state vector x𝑘 in the asynchronous model (4.13) is a discrete random vector with at most 2^{𝑁 𝑘} distinct values, as there are 2^𝑁 different ways of selecting an update set in any iteration. We note that the first and the second order statistics ofx𝑘 are described previously in Theorem 4.1 and Corollary 4.2, respectively. In this section, we will consider the distribution of the random vectorx𝑘 (as well as the outputy𝑘).

One can consider approximatingx𝑘 with a multivariate complex Gaussian random vector with mean x^ss_𝑘 and covariance Q𝑘 due to the independent selection of the indices and the central limit theorem. Contrary to the anticipation, we have numer- ically observed thatthe random vectorx𝑘 does not have a Gaussian distributionin general as we demonstrate next.

In this regard, we consider the system in (4.27) with the input𝑢_𝑘 =cos(2𝜋 𝑘/100), and we takeP= 𝑝Iand𝚪=0. Since the output is a scalar real random variable in this case, we will focus ony𝑘 for the sake of simplicity. In particular, we present the empirical distribution ofy𝑘 for several different values of𝑘 for the probabilities 𝑝 ={0.3, 0.6, 0.9}in Figure 4.5.

-14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 yk

0 0.10 0.20 0.30 0.40 0.50

PDF

y100

y120

y130

y140

y150

(a)

-14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 yk

0 0.25 0.50 0.75 1.00 1.25 1.50

PDF

y100

y120

y130

y140

y150

(b)

-14 -12 -10 -8 -6 -4 -2 0 2 4 6 8 10 12 14 yk

0 1.00 2.00 3.00 4.00

PDF

y100

y120

y130

y140

y150

(c)

Figure 4.5: Empirical distributions of the output random variable for the update probabilities (a) 𝑝 =0.3, (b) 𝑝 =0.6, (c) 𝑝=0.9. Dotted lines denote the mean of the corresponding random variables. Distributions are obtained via 10⁹independent samples of the random variables.

Numerical results in Figure 4.5 show that the distributions are not necessarily symmetric with respect to their means, and they can be multimodal. So, it is clear that the random variabley𝑘 does not have a Gaussian distribution in general.

In addition to the mean and the variance being a function of the iteration index 𝑘 (see Theorem 4.1 and Corollary 4.2), Figure 4.5 shows that overall “shape”

of the distributions also change with iterations. In particular, Figures 4.5b and 4.5c show that the distributions can be multimodal or unimodal depending on 𝑘. Similarly, update probabilities also affect the distributions. As discussed previously in Section 4.6.2, distributions tend to be “narrower” as the update probabilities get larger. However, the precise relationship between the update probabilities and the distributions is not know at this point.

4.6.4 Effect of the Stochastic Model

Although linear systems show an unexpected behavior under randomized asyn- chronicity, precise details of the stochastic model are also important in determining the mean-squared stability. The results presented in this chapter are valid only for the stochastic index selection model described in Section 4.2.1, and the stability con- dition can be different under different models. In this section we will demonstrate this point.

We start by considering the model in Section 4.2.1 with all the indices being updated with probability 𝜇/𝑁, so that 𝜇 uniformly randomly selected indices are updated per iteration on average, that is,E[|T𝑘|] = 𝜇for all𝑘. More precisely,

P= 𝜇 𝑁

I =⇒ A¯ =I+ 𝜇 𝑁

A−I

. (4.76)

In this case, the matrix𝚽in (4.38) has the following form:

𝚽=A¯^∗⊗A¯ + 𝜇(𝑁− 𝜇) 𝑁2 J

(A^∗−I) ⊗ (A−I)

. (4.77)

We now consider a slightly different model where we update exactly 𝜇 indices per iteration. So, we have |T𝑘| =𝜇 for all 𝑘, and the set T𝑘 is selected uniformly randomly among all possible ^𝑁𝜇

subsets of size 𝜇. In this case the average state transition matrix ¯Astill has the form in (4.76). However, it can be shown (using the identity (2.2)) that the error correlation matrix evolves according to the following function:

𝜑⁰(X) =A X¯ A¯^H− 𝜇(𝑁−𝜇)

𝑁2(𝑁−1) (A−I)X(A^H −I) + 𝜇(𝑁− 𝜇)

𝑁(𝑁−1)

(A−I)X(A^H −I)

I. (4.78)

By vectorizing both sides of (4.78), matrix representation of the function𝜑⁰(·)can be found as follows:

𝚽⁰=A¯^∗ ⊗A¯ + 𝜇(𝑁− 𝜇) 𝑁(𝑁−1)

J− 1

𝑁

I (A^∗−I) ⊗ (A−I)

. (4.79)

It is clear that𝚽and𝚽⁰are different from each other although the difference𝚽−𝚽⁰ approaches zero as𝑁gets larger. More importantly, there is no clear relation between 𝜌(𝚽) and 𝜌(𝚽⁰). Thus, random asynchronous updates may be stable under one stochastic model, but the iterations may get unstable under the other model although both models update 𝜇uniformly selected random indices per iteration on average.