LOCALIZED LINEAR QUADRATIC REGULATOR
5.1 Problem Statement
5.1.1 Interconnected System Model
……
……
Subsystem
: physical interaction : communication : sensing and actuation
: sub-controller : subsystem’s state
……
……
Figure 5.1: An Example of Interconnected System
Considerndynamically coupled discrete time linear time invariant (LTI) subsystems that interact with each other according to an interaction graph G = (V,E). Here V = {1, . . . ,n} denotes the set of subsystems. We denotexiandui the state vector and control action of subsystem i. The set E ⊆ V × V encodes the physical
interaction between these subsystems — an edge (i, j) is in E if and only if the statexj of subsystem j directly affects the statexiof subsystemi. The dynamics of subsystemiis assumed to be given by
xi[t+1]= Aiixi[t]+ Õ
j∈Ni
Ai jxj[t]+Biiui[t]+wi[t], (5.1) where Ni = {j|(i,j) ∈ E} is the (incoming) neighbor set of subsystem i, Aii, Ai j, Bii some matrices with compatible dimension, and wi the process disturbance of subsystem i. Figure 5.1 shows an example of such interconnected distributed system — each subsystemihas a sub-controller that takes the locally available state measurement xi, exchanges information with some other sub-controllers through a communication network, and generates the control actionui to control the state xi
of the physical system.
Define x = [x1. . .xn]>,u = [u1. . .un]>, andw = [w1. . .wn]> the stacked vectors of the subsystem states, controls, and process disturbances, respectively. The n interconnected system models (5.1) can be combined into a global system model given by
x[t+1]= Ax[t]+B2u[t]+w[t], (5.2) with
A=
A11 · · · A1n
... ... ...
An1 · · · Ann
and B2=
B11 · · · 0 ... ... ...
0 · · · Bnn
.
Note that the topology of the graphGis encoded in the sparsity pattern of the global system matrices A. In particular, the block sparsity pattern of A is equal to the adjacency matrix of its underlying graphG.
Remark 6. When each xi and ui in (5.1) are scalar variables, we call the model (5.1)-(5.2)ascalar subsystem model.
Remark 7. Note that the matrix B2 in(5.2) is diagonal. If theith subsystem does not have an actuator that can directly alter its state, we simply assign Bii = 0. We will lift these assumptions in Chapter 7 to consider the most general output feedback interconnected system model with arbitrary sparsity pattern.
We assume that the pair(A,B2)in (5.2) is controllable. In addition, the disturbances w are drawn i.i.d. from a zero mean unit covariance Gaussian distribution, i.e., we haveE(w[k])= 0 andE(w[i]w[j]>)= δi jIfor alli, j,k, whereδi j is the Kronecker
delta andE(·)is the expectation operator. The objective is to find a control strategy, which is a mapping from state measurementxto control actionu, to minimize the expected value of the average quadratic cost
E
N→∞lim 1 N
N
Õ
k=1
"
x[k] u[k]
#>"
C>
1C1 C>
1D12 D>
12C1 D>
12D12
# "
x[k] u[k]
#
(5.3) for some cost matrices (C1,D12). The traditional infinite horizon stochastic LQR problem can then be formulated as
minimize
{x[k],u[k]}∞k=1 E
N→∞lim 1 N
N
Õ
k=1
"
x[k] u[k]
#>"
C>
1C1 C>
1D12 D>
12C1 D>
12D12
# "
x[k] u[k]
#
subject to x[k+1]= Ax[k]+ B2u[k]+w[k] x[0]=0, u[0]=0
E(w[k])=0, E(w[i]w[j]>)=δi jI. (5.4) 5.1.2 Challenges on Scalability
Traditionally, the solution of the stochastic LQR problem (5.4) is obtained by solving a discrete time algebraic Riccati equation (DARE). The optimal solution is given by a static feedbacku[k]= K x[k], where the gain matrixKcan be found by solving the DARE. The solution to the LQR problem is one of the most important and elegant result in modern optimal control theory [81] due to the following reasons: (i) the optimal solution is proven to belinear(the control action uis a linear function of the state measurement x), (ii) the optimal solution isstatic(the control actionu[k] at timek depends only on the measurement x[k]at time k), and (iii) the controller K can be obtained by solving a convex program in polynomial time. In addition to its simplicity, the LQR method has proven to be useful in extremely diverse applications [81].
However, there are some limitations of the LQR method for large-scale systems:
1. Communication delays: The LQR gain matrix K is generally dense even when the system matrices(A,B2)that specify the system dynamics (5.2) are sparse. This means that the measurements from all states need to be shared instantaneously, which requires infinite (or impractically fast) communication speed.
2. Scalability of controller implementation: A dense LQR gain also implies that the measurements from all sensor need to be collected by every sub- controller in the network, which is not scalable to implement.
3. Scalability of controller synthesis: To compute the LQR gain, one need to solve a large-scale DARE. The complexity of solving the DARE isO(n3), where n is the dimension of the matrix A. Even though the DARE can be solved in polynomial time, the complexity blows up quickly for a largen.
4. Scalability of controller re-synthesis subject to model change: When a few entries of the global plant model(A,B2)change, one needs to recompute the solution of (5.4) to resynthesize the global LQR optimal controller. This is not scalable for incremental design when the physical system expands.
To solve the above limitations, a common approach is to incorporate structured constraint on the controller, which leads to a constrained optimal control problem as in (2.8). In particular, the stochastic LQR problem (5.4) is a special case of aH2 optimal control problem, with plant model given by
P=
A I B2 C1 0 D12
I 0 0
=
"
P11 P12 P21 P22
# .
A structuredH2optimal control is given by (cf., Section 2.3) minimize
K
kP11+P12K(I−P22K)−1P21kH2
2 (5.5a)
subject to Kinternally stabilizesP (5.5b)
K ∈ C, (5.5c)
where the subspace constraint K ∈ C enforces information sharing constraints between the sub-controllers.1 As mentioned in Chapter 2, it was shown in [53] that if the subspace constraint set C is quadratically invariant (QI) with respect to P22 then the optimal control problem admits a convex reformulation. Loosely speaking, this condition requires that sub-controllers be able to share information with each other at least as quickly as their control actions propagate through the plant [52].
The QI framework therefore provides a tractable way to address the challenge of communication delays mentioned above.
However, the distributed optimal control framework based on QI has less emphasis on the challenges of scalability mentioned above. We note that the implementation complexity of controllerKis determined by the densest rowKiofK. Specifically, if
1Note that the controllerKhere does not need to be static. Therefore, we use a boldface Latin letter to emphasize thatKis a transfer matrix.
we consider the control action taken at subsystemi, which is specified byui =Kix, then if Ki is completely dense then subsystem i must collect measurements from every other subsystem j ∈ V. In order to design a controller that is scalable to implement, a natural solution is to impose sparsity constraints on the controllerK such that each row only has a small number of nonzero terms — in this way each subsystemi only needs to collect a small number of measurements to compute its control action. Unfortunately, this naive approach fails if the underlying topology of G of the networked system is strongly connected. Specifically, when the topology ofGis strongly connected, any sparse constraint setC is not QI (cf. Example 1 in Section 2.4), which means that the constrained optimal control problem is always non-convex. Although recent methods based on convex relaxations [19] can be used to solve certain cases of the non-convex optimal control problem (5.5) with sparse constraint set C, the underlying synthesis optimization problem is itself still large- scale and does not admit a scalable reformulation. The need to address scalability, both in the synthesis and implementation of a controller, is the driving motivation of the LLQR framework.