Finite State Approximation of Stochastic Differential
Equations with Switching Dynamics
Iwan R. Setiawan and Arjon Turnip
Technical Implementation Unit for InstrumentationDevelopment, Indonesian Institute of Sciences, Kompleks LIPI Gd. 30, Bandung, Indonesia
Email: [email protected]
T.A. Tamba
Research Group on Instrumentation & Control, Faculty of Industrial Technology, Institute of Technology Bandung, Indonesia
Email: [email protected]
Abstract—This paper discusses a method for con-structing a finite state approximate model for stochas-tic differential equation with switching behaviors. We used a finite state Markov Chain model to describe the dynamics of such an abstract model. In particular, the transition matrix of the obtained Markov Chain model characterizes the switching probability between alternative equilibria in the system. When extended to the study of interconnecting system, we show some preliminary idea about the equivalence between the original stochastic differential equation and its Markov Chain approximation.
I. Introduction
Comples systems are prevalent in real life and engi-neering applications. These systems are tipically large in scale and consist of many subsystems that interacts be-tween each other under strong coupling or nonlinearity ef-fects. These characteristics of complex system often make their study become very difficult and computationally intractable. As a result, model approximation or abstrac-tion methods are often employed in order to find simple descriptions about the essential behaviors of these systems. In this regard, model approximation can be seen as a mean to extract the most essential information from the system by leaving away all the details that are irrelevant with the main objective of the analysis. Depending on the objective and purpose of the study, various criteria can be used to measure the quality of an abstract model. These include the amount of data or variables in the model, the types of behaviors that should be included in the abstract model, or the efficiency of computation efforts required to generate the abstract model.
Several methods for abstracting or approximating the dynamics of complex system have been proposed in lit-erature. For example, in the field of systems theory and computer science, model checking has been one of the most popular abstraction methods which has gained tremen-dous interests from research community. In particular, the model checking approach has also been implemented in the verification process of embedded systems or other systems whoxe models can be described by finite state machine [1], [5]. Despite these progresses, it is still difficult to apply the model checking technique or its variants
for verifying systems with continous dynamics. The well-known state explosion problem is one of the main issue that is oftenly encountered during the partition of the continuous state space into its corresponding (symbolic) finite state equivalent. One may then expect that as the size of the system is increased, the state explosion problem will become more apparent.
In tackling this issue, there have been a parallel ap-proach from control literatures for doing system verifi-cation without the need to compute the finite state (or discrete) abstraction of the original model explicitly. Such an indirect method often uses optimization technique to compute some certificates (termed Barrier certificates) that proves the satisfaction of the properties to be verified [6], [11]. The absence of needs to compute the discrete abstraction in this approach seems to gives some hope that the issues of state explosion problem encountered when using the model checking technique can be resolved. The recent development on computation methods using the sums-of-square optimization has also been the major support for the success of this method [12], [13].
This paper aims at examining the implementation of such an indirect verification method for computing an approximate model for nonlinear stochastic differential equation (SDE). In particular, we consider the case when the the SDE model has multiple stable equilibria and its dynamics are governed by switching behaviors between competing equilibria. We argue that an abstract model of this system in the form of finite state markov chain model can be obtained. In particular, we show that the transition probability matrix of the approximate Markov Chain model can be computed using efficient sums-of-square optimization method [3]. We also used the abstract markov chain model to show that the study of inter-connecting SDEs can be abstracted to the study of its approximate Markov Chain model. We show that, under some conditions on the structure of the chains’ intercon-nection, the obtained system has statinary distribution that are equivalent with Gibbs random field [3], [9]. With the existing equivalence, one can then use the tools from Gibbs random field to study the abstract model of the system.
2015 International Conference on Automation, Cognitive Science, Optics, Micro Electro-Mechanical System, and Information Technology (ICACOMIT), Bandung, Indonesia, October 29–30, 2015
II. System Description
We consider a stochastic version of an intraguild pre-dation (IGP) system model between crayfish and bass in a closed lake ecosystem described in [7]. Denoting x
and y for crayfish and bass densities, respectively, the nondimensional model after rescaling is given as follows.
dx =
x(1−x−ay)− dyx
2
k2+x2
dt+g(x)dW (1)
dy =
ry(1−bx−y) + edyx
2
k2+x2
dt+g(y)dW (2) where constants a= 0.7, b= 0.9, d= 0.075, e= 0.01, k= 0.1, r = 1.5, and W is the standard Wiener process. An IGP system models a simultaneous interactions between species in which both species compete for the same re-source while at the same time one species predates the other. Interested readers are referred to [7] for complete description and exposition of this model.
System (1)-(2) is a stochastic differential equation (SDE) interpreted in Ito’s sense [10]. When g(.) = 0, we recover the deterministic version of the model. The analysis of the deterministic version of (1)-(2) has shown that the coexistence of both species in the system is guaranteed when the system have three equilibrium points: two of them are stable while the other is unstable [7]. This shows that the system has a multiple equilibria charac-teristic in which the region of attraction of each stable equilibrium is separated by a separatrix. Thus, depending on from which region of attraction the system’s trajectory is initialized, the trajectory will eventually move towards the equilibrium of the corresponding basin of attraction and stay there forever.
The introduction of Wiener-type noiseg(.)dW to model (1)-(2), however, makes it possible for the trajectory to switch between the two stable equilibria. In other words, even a small noise intensity will result in a (possibly small) positive probability for the trajectory to switch between the two stable states. As the noise level is increasing, it
Fig. 1: Switching behaviors in system (1)-(2).
is reasonable to expect that the switching frequency of the trajectory will also becomes higher. In this case, the system can be seen as having a switching behavior: the tra-jectory starts to move randomly around the neighborhod of one stable equilibrium and then after some time it jumps to another stable equilibrium and stay there for some other period of time before it then jumps back again and repeats the same behavior. Fig. 1 illustrates an example of such behavior.
Analyzing such a complex behavior of systems with multiple equilibria. However, by noticing that the qual-itative behavior of the system trajectory is more likely to either stay in one of the equilibriums for a period of time or switch between them, we aim to abstract the original model into a two states markov chain (MC) model. The states of the MC are then describe the two stable equilibrium points and its transition matrix describe the probabilities of staying within one state or transitioning to another state. The computation of the entry of the MC’s transition matrix will be done using an approach in stochastic verification methods as introduced in [11].
III. Main Results
This section describe the main results of this paper. The first result is an abstraction method for the original SDE model using finite state MC. We then use the obtained abstraction to present our second result on the analysis of aggregate behaviors of an interconnecting SDEs. This second result basically shows that the interconnection of the obtained abstract model posses the property of a Gibbs random field (GRF). Thus, rather than analysing the original interconnecting SDE directly (which in many cases can be difficult), we can instead analyze the abstract model using the well-known tools in GRF.
A. Markov Chain Abstraction of Switching SDE
Consider a two states Markov Chain (X, P, π0) where
X = {E1, E2} is the set of states, P = [0,1]2×2 is the
transition matrix, and π0 = [0,1]2 is the initial
distri-bution. The states E1, E2 correspond to the two stable
equilibriums of the SDE in (1)-(2). Let P = 1−p p q 1−q
such that
p = P r{Xt=E2,for somet≥0|X0=E1} (3)
q = P r{Xt=E1,for somet≥0|X0=E2}. (4)
The computation of p, q in (3)-(4) can be done using a convex optimization technique from stochastic safety veri-fication methods proposed in [11]. Essentially, the value of
Lemma 1 [11] Consider a SDE of the form dx(t) =
f(x(t))dt+g(x(t))dW on a bounded state spaceX. Let the sets X,X0,Xu be given and consider the stopped process
{x˜(t) =x(t) for t < τ and ˜x(t) =x(τ) for t≥τ}, where
τ is the exit time of x(t) from the open set Int(X). Sup-pose that there exists a twice continuously differentiable functionB:Rn →Rsuch that
0 ≤ B(x, t),∀x∈ X (5)
1 ≤ B(x, t),∀x∈ Xu (6)
γ ≥ B(x, t),∀x∈ X0 (7)
0 ≥ ∂B∂t +∂B
∂x +
1 2Tr(g
T∂ 2
B
∂x2g),∀x∈ X (8)
Then P r{x˜(t)∈ Xu for somet≥0|x˜(0) =x0} ≤γ.
The search for a barrier functionB(x, t) or upper bound probability γ is in general difficult. However when the system is in polynomial form and the sets X,X0,Xu are semi algebraic, then this problem can be solved efficiently using the sums-of-squares polynomial toolbox SOSTOOLS [12], [13]. Since (1)-(2) is of polynomial type equation and conditions (3)-(4) are basically equivalent as in Lemma 1, we compute the value ofp(respectively q) as the solution of the the following optimization problem.
min p (9)
s. t. (10)
0 ≤ B(x, t),∀x∈ X (11) 1 ≤ B(x, t),∀x=E2 (12)
p ≥ B(x, t),∀x=E1 (13)
0 ≥ ∂B
∂t + ∂B
∂x (14)
+1 2Tr(g
T∂ 2B
∂x2g),∀x∈ X (15)
To compute the approximate MC model for system (1)-(2), we solved the above optimization problem using SOS-TOOLS and obtained the followingP matrix for our MC model.
P =
0.7502 0.2498 0.6517 0.3483
It can be easily seen that P is aperiodic and irreducible, thus P is ergodic [3], [8] and have a unique stationary distribution given byπ∗
M C = (0.7229,0.2771).
To compare the obtained MC stationary distribution with the original SDE, we simulate (1)-(2) using the Milstein algorithm over 106 steps and obtained an
ap-proximation to the stationary distribution around the neighborhood of each equilibriumπ∗
SDE= (0.7084,0.2916) (Fig. 2). Since the value ofπ∗
M Cis relatively close toπ
∗
SDE, we conclude the obtained MC model gives a good approx-imation of the stationary dostribution of the original SDE model.
B. Spatially Interconnected SDE as Gibbs Random Field In this section, we will show that - using the approxi-mate two state MC model - a spatially distributed SDE model can be model as Gibbs Random Field.
Let the SDE in (1)-(2) describe the dynamic of a site that are spatially interconnected with other sites within a bounded state space. Assume that each of this other site has a dynamic described analogously by (1)-(2). One can think that each site is a partition within an aquatic ecosys-tem (lake or stream) within which there exist dynamic interaction between species of the form given in (1)-(2). The union of all sites is then corresponds to the total area of the lake or stream. Formally, for given state space X and a finite integer i = 1, . . . , n, we said Xi is a site in
X if and only if Xj∩ Xk = ∅ for j 6= k and ∪Xi = X fori= 1, . . . , n. Moreover, the dynamics in each siteXi is given by (1)-(2). Towards the end, we will use the notation
si instead ofXi
Using the MC abstraction described in the previous section, each site can then be in one of the two states:E1
orE2. We assignE1= +1 andE2=−1 such that the state
variable is given bysi=±1. The interconnection between sites within the whole state space induce a configuration of the state that change dynamically at each time step. Let
Cl denotes the configuration of the overal state at time l and is given byCl={s1, s2, . . . , sn}. The total number of configuration is then 2n.
For a configurationCl, letei(Cl) denotes the net effect received byi−site from through interaction with its neigh-bor. This effect is assumed tobe a random variable with normal distribution with mean ¯ei(Cl) = PjCi,jsj(Cl) and variance ν. Now, assume that the transition of site
i from +1 to -1 (and vice versa) is dependent upon the value of the net effect ei(Cl) from its neighbors when the configuration of the state is Cl. Assume that this dependency is given as some threshold constant θi such that the transition probability of sitei change from -1 to +1 in the next time step given the current configuration
of the state is where erf(.) denotes the error function,G(ei) is a Gaussian density with mean ¯ei and variance ν, and the integration variables y, z are given by
y= (ei√−e¯i)
Now the one step configuration transition probability from Cl toCl+1 can be written as be seen that the transition probability of configuration in (24) is a Gibbs-type transition probabilities in which µi act as the potential of the i−th configuration and the denominator in (24) is the normalizing term. One can verify (see [?] for instance) that the transition probability (24) is in fact satisfies the detailed balanced condition and hence has a unique stationary distribution which is given by Gibbs distribution. Thus we can conclude that the use of MC abstraction to describe the interconnecting SDE will result in a random field that satiesfies the Gibbs stationary distribution. Using this result, one can then do the analysis using the tools available in the area of Gibbs Random Field.
IV. Discussion
We have presented the construction of a finite state markov chain model as an abstract representation of stochastic differential equation model. The extension of this result for the case of interconnecting system, how-ever, still needs to be further pursued. In particular, the abstraction method for interconneting SDEs described in this paper has taken an assumption that the knowledge of the threshold for transition between the two states of the Markov chain is known. In reality, this threshold is difficult to be obtained. Verification through simulation of the obtained abstract model for the interconnecting system is also needs to be done so that its comparison with the original model can be carried out. Finally, comoputing a metric that shows how good the obtained approximation is will also be beneficial.
References
[1] R. Alur, T. Henzinger, G. Laffereire, and G. J. Pappas ”Discrete abstraction of hybrid system,” Proceedings of the IEEE, vol. 88, no. 7, pp. 971-984, 2000.
[2] M. Aoki, Modeling Aggregate Behavior and Fluctuation in Economics: Stochastic Views of Interacting Agents, Cambridge University Press (1st Ed.), 2002.
[3] P. Bremaud, Markov Chains: Gibbs Fields, Monte Carlo Simu-lation, and Queues, Springer, 1999.
[4] S. R. Carpenter, D. Ludwig, and W. A. Brock, ”Management of eutrophication for lakes subject to potentially irreversible change,” Ecological Applications, vol. 9, no. 3, pp. 751-771, 1999.
[5] E. M. Clarke, Model Checking, MIT Press, 1999.
[6] H. El-Samad, S. Prajna, A. Papachristodoulou, J. Doyle, and M. Khammash, ”Advanced methods and algorithms for biological networks analysis,” Proceedings of the IEEE, vol. 94, no. 4, pp. 832-853, 2006.
[7] R. D. Horan, E. P. Fenichelb, K. S. Drury, and D. M. Lodge, ”Managing ecological thresholds in coupled environmen-talâĂŞhuman systems,” PNAS, vol. 108, no. 18, pp. 7333-7338, 2011.
[8] F. P. Kelly, Reversibility and Stochastic Network, John Wiley & Sons Ltd., 1979.
[9] R. Kindermann and J. L. Snell, Markov Random Fields and Their Applications, Contemporary Mathematics, American Mathematical Society, 1980.
[10] B. Oksendal, Stochastic Differential Equations: An Introduction with Applications. Berlin, Germany: Springer-Verlag, 2000. [11] S. Prajna, A. Jadbabaie and G. J. Pappas, ”A framework
for worst-case and stochastic safety verification using barrier certificates,” IEEE Transactions on Automatic Control, vol. 52, no. 8, pp. 1415-1428, 2007.
[12] S. Prajna, A. Papachristodoulou, P. Seller, and P. A. Par-rilo, âĂIJSostools and its control applications,âĂİ in Positive Polynomials in Control. New York: Springer-Verlag, 2005, pp. 273âĂŞ292.
[13] S. Prajna, A. Papachristodoulou, and P. A. Parrilo, âĂIJIntroducing SOSTOOLS: A general purpose sum of squares programming solver,âĂİ in Proc. IEEE Conf. Decision and Control, 2002, pp. 741âĂŞ746 [Online]. Available: http://www.cds.caltech.edu/sostools [Online]. Available: http://www.mit.edu/parrilo/sostools.
