System Identification for Randomized LTI Systems

LEARNING AND CONTROL IN LINEAR TIME-VARYING SYSTEMS

4.3 Stability and Identification of Random Asynchronous LTI Systems In this section, we introduce the random asynchronous LTI systems, a new form ofIn this section, we introduce the random asynchronous LTI systems, a new form of

4.3.4 System Identification for Randomized LTI Systems

mean-square stability of the randomized LTI system and the stability of Ado not imply each other, i.e.,𝜌(S)<1 and𝜌(A)<1 are not equivalent in general. Note that Lemma 4.4 provides the precise characterization of the rightmost axes of the plots in Figure 4.4.

In order to visualize the convergence behavior of the randomized updates, we consider a numerical test example of size 𝑑_𝑥 =2 with a constant input (i.e., fixed- point iteration), and initialize𝑥₀with independent Gaussian random variables (left- most block in Figure 4.5). Then, the distribution of the state vector 𝑥_𝑡 (at time 𝑡) follows a Gaussian mixture model (GMM) due to the randomized nature of the updates (See Figure 4.5). Furthermore, the stability of the matrix S ensures that the mean of𝑥_𝑡 converges to the fixed-point of the system while the variance of 𝑥_𝑡 converges to zero.

The key insight to the convergence behavior in Figure 4.5 is as follows: When rep- resented as a switching system, the randomized LTI model (4.29) switches between 2^𝑑𝑥 systems randomly, and it can be shown that all these 2^𝑑𝑥 systems (including the original system) have the same fixed-point. It should also be noted that not all 2^𝑑𝑥 systems are stable by themselves, and an arbitrary switching does not necessarily ensure the convergence. Nevertheless, with a careful selection of the probability, the randomized model can obtain convergence even when the synchronous system is unstable.

When the system is mean-square stable, the steady-state covariance matrix, 𝚪, is given as

vec(𝚪) =(I−S)⁻¹ 𝑝²I+ (𝑝− 𝑝²)J

vec(B U B^⊤) +𝜎²

𝑤vec(I)

. (4.33) When𝑝 =1, we have𝜙(𝑥) =A𝑥A^⊤, which implies that𝚪=A𝚪A^⊤+B U B^⊤+𝜎²

𝑤I and 𝜌(S) =𝜌²(A). So, we have 𝜌(S) < 1 if and only if𝜌(A) < 1 for synchronous LTI systems, which recovers the well-known stability result in the classical systems theory.

Figure 4.5: Evolution of the state vector for a mean-square stable (but synchronously unstable) 2-dimensional randomized LTI system with a fixed input and Gaussian initialization

First, recall the Markov chain central limit theorem (MC-CLT). Assume that we have a Markov chain at its stationary distribution. MC-CLT states that, the sample average of any measurable, finite-variance and real-valued function of a sequence of 𝑛variables from this Markov chain converge to a Gaussian distribution as𝑛→ ∞, where mean is the expected value of this function at the stationary distribution and the variance linearly decays in𝑛[129].

Notice that the randomized updates of (4.29) form an ergodic Markov chain (due to independent selection in every iteration) and the stability of the system guar- antees the stationary distribution. We also know that the stable systems converge exponentially fast to their steady state, i.e., Markov chain formed by (4.29) quickly approaches to its stationary distribution. In light of these observations, we can deduce that, as the number of collected input-output samples𝑇 increases, the sam- ple state correlation and input-output cross correlation matrices converge to their expected values with the rate of 1/√

𝑇. In particular, given a sequence of inputs and outputs{𝑥₀, 𝑢₀, 𝑥₁, . . . , 𝑢_𝑇−1, 𝑥_𝑇}, let

C0= 1 𝑇

∑︁𝑇−1 𝑡=0

"

𝑥_𝑡 𝑢_𝑡

# "

𝑥_𝑡 𝑢_𝑡

#⊤

, C1= 1 𝑇

∑︁^𝑇

𝑡=1

𝑥_𝑡

"

𝑥_𝑡−1 𝑢_𝑡−1

#⊤

. (4.34)

According to MC-CLT, as 𝑇 → ∞, C0 and C1 converge to E[C0] and E[C1] respectively, where

E[C0] =

"

𝚪 0 0 U

#

, E[C1] = h

A𝚪 BU i

. (4.35)

Therefore, usingC1C⁻₀¹converges to the average state transition and input matrices h

A B i

. Notice that C1C⁻₀¹ is in fact the solution of the following least squares problem:

arg min

Θ

∑︁𝑇

𝑡=1tr 𝑥_𝑡−Θ

"

𝑥_𝑡−1 𝑢_𝑡−1

#

𝑥_𝑡 −Θ

"

𝑥_𝑡−1 𝑢_𝑡−1

# ⊤

. (4.36)

Thus, we are guaranteed to recover the average system consistently via (4.36). This result could be extended to recover first 𝐾 Markov parameters of the randomized partially observable LTI systems. Define E = [I^𝑑𝑥 0] ∈ R^{𝑑𝑥×(𝑑𝑥+𝑑𝑢)}. Then, the extended Lyapunov equation can be written as

E

"

𝚪 0 0 U

# E^⊤=

h A B

i

"

𝚪0 0 U

# h

A B i⊤

+

1/𝑝−1 h

A−I B i

"

𝚪0 0 U

# h

A−I B i⊤

⊙I+𝜎²

𝑤I, (4.37) We know that covariance matrices of the state variables 𝚪 and inputs U must satisfy (4.37) for a stable randomized LTI system. The central idea for our system identification method is to exploit this fact and recover the randomization probability 𝑝, the noise covariance𝜎²

𝑤 and the system parametersA,Bof a stable randomized LTI system. Therefore, we can write extended Lyapunov equation (4.37) in terms ofC0andC1and due to (4.35) expect to havely(C0,C1) =0, where

ly(C0,C1)BEC0E^⊤−C1C⁻¹0 C^⊤1−1 𝑝

−1 (C1C⁻¹0 −E)C0(C1C⁻¹0 −E)^⊤

⊙I−𝜎²

𝑤I. Thus, to identify the underlying system dynamics, we propose to solve the following:

𝑝,b b𝜎²

𝑤 =arg min

𝑝,𝜎_𝑤²

∥ly(C0,C1) ∥²_F. (4.38) This problem can be further simplified to

b𝑝, b𝜎²

𝑤 =arg min

𝑝,𝜎_𝑤²

𝑴₁− (1/𝑝) 𝑴₂−𝜎²

𝑤I

2

F, (4.39)

where𝑴₂=

(C1C⁻¹₀ −E)C0(C1C⁻¹₀ −E)^⊤

⊙Iand𝑴₁=E C0E^⊤−C₁C⁻¹₀ C^⊤₁+𝑴₂. Notice that𝑝and𝜎²

𝑤appear decoupled in (4.39). Therefore, we can first solve (4.39) forb𝜎²

𝑤 for a fixed value of 𝑝 to get an optimal solution. Then, substitutingb𝜎²

𝑤 into the problem and solving for b𝑝we obtain the optimal estimate for 𝑝. The described procedure yields the following optimal estimates:

𝑝b=

𝑑_𝑥 tr(𝑴^⊤₂ 𝑴₂) −tr²(𝑴₂) 𝑑_𝑥tr(𝑴^⊤₁ 𝑴₂) −tr(𝑴₁)tr(𝑴₂),

b𝜎²

𝑤 = tr(𝑴₁) − (1/𝑝b) tr(𝑴₂) 𝑑_𝑥

. (4.40) Using the estimate of randomization probability b𝑝 and C1C⁻¹₀ = [b

A bB], i.e., the estimate of average system transition parameters, the underlying system parameters could be recovered as Ab=(1/𝑝b)b

A+ (1−1/b𝑝)I^𝑑𝑥 and bB=(1/b𝑝)b

B. To study the performance of the proposed system identification method, we consider a random- ized LTI system with state transition matrix of A1 and a random B with 𝑝 = 0.5

10⁰ 10¹ 10² 10³ 10⁴ 10⁵ 10⁶

T(number of samples)

10^-3 10^-2 10^-1 10⁰ 10¹

AverageEstimationError

|p−pb|

|σ_w−bσ_w| 1/√

T

10⁰ 10¹ 10² 10³ 10⁴ 10⁵ 10⁶

T(number of samples)

10^-3 10^-2 10^-1 10⁰ 10¹

AverageEstimationError

kA−AkbF/kAkF

kB−BkbF/kBkF

1/√ T

Figure 4.6: Average estimation error for the unknown system parameters of the stable randomized LTI system with state transition matrix ofA1and randomBfor 100 independent single trajectories

which guarantees the stability (verified by Lemma 4.4) and set 𝜎_𝑤 = 1. We run 100 independent single trajectories and present the average rate of decay for the estimation errors of 𝑝, 𝜎²

𝑤 in the first plot andAandBin the second plot of Figure 4.6. Notice that the estimation errors behave irregularly at the beginning where there are few samples, corresponding to burn-in period to converge to steady-state.

On the other hand, Figure 4.6 show that, as predicted by MC-CLT, the estimation errors decay with 1/√

𝑇 rate as we get more samples. This estimation error rate is the optimal behavior in linear regression problems with independent noise and covariates [106]. This depicts the consistency and efficiency of the proposed system identification method for randomized LTI systems.