Black Boxes and Representation Nuisances
4.2. Describing Black Boxes
ProbMeasures(A) is the set of probability measures on A (Definition B.1). “x ∼ µ”
means that xis a random variable andµis its probability distribution. Conditional(B;A) is the set of conditional distributions fromAtoB(Definition B.3) A stochastic process is a particular type of probability distribution on infinite sequences (Definition B.7).StocProcesses(A) is the set of all stochastic processes with values inA.
4.2. Describing Black Boxes
DEFINITION 4.1 (Black box system). Given an input spaceAand an output space B, a black-boxsystemDis a function that assigns a probability distribution for the next output as a function of the previous history of input and output:
D:Sequences(B×A)→ProbMeasures(B), (4.1)
such that the distribution of the outputbk, after the inputa:k =ha0, . . . ,akihas been given, and the past outputb:k−1 =hb0, . . . ,bk−1ihas been decided, is given byD(b:k−1,a:k):
bk ∼ D(b:k−1,a:k). (4.2)
(See Figure4.1b.)
D(B;A)denotes the set of all black box systems with output inBand input inA.
4.2. DESCRIBING BLACK BOXES 65
(a)Graphical notation for a black box systemsD∈D(B;A)between two signalsa ∈Sequences(A)andb∈Sequences(B).
(b) Realization of a black-box as a map D : Sequences(B×A) → ProbMeasures(B)that maps histories to the probability distribution of the next output. The symbol refers to a buffer accumulating pre- vious values (Definition 4.22) and∆is a unit delay (Definition 4.21).
Figure 4.1. The “black boxes” described in this chapter are possibly stateful systems, but the internal state is not modeled explicitly. Rather, the system is defined only by the recursive distribution of the observations given the history of observations and commands.
REMARK4.2. (Black box systems include instantaneous relations.) The outputbk depends on the input up to instantkincluded. While an alternative definition would have beenbk+1∼ D(b:k,a:k), it is useful to include in this class of systems also instantaneous transformations of the input, in whichbkis a function ofak.
REMARK4.3. (Conventions are compatible with left composition.) In the definition “D(B;A)”, the setBis the output space, andAis the input space. This will prove to be the most co- herent with the definition of series as left composition, whereED ∈ D(C;A)indicates the series ofE∈D(C;B)followingD∈D(B;A).
REMARK4.4. (Priors are explicit.) This formalization includes priorson the state as an integral part of the system; that is, two systems Dand D0 which have the same dynam- ics but different priors are considered different points of the spaceD(B;A). To see this, note that the domain of the mapsDisSequences(B×A), which includes sequences of all lengths. The output of the systemsb0andb00fork=0 is a function of only the inputa0:
b0b00∼ D(a0), ∼ D0(a0).
4.2. DESCRIBING BLACK BOXES 66
The formalization forces the outputb0to be well defined even if one just started interacting with the system. In other words, it coincides with some idea of “prior” for the system.
REMARK4.5. (States are implicit.) The formalization does not mention an internal state space explicitly. The state is implicit in the fact that the map depends on the complete his- tory of commands and observations. This is, of course, the traditional way to define a state from an input-output relation, as a set of equivalence classes between sequences [24]. This view is very close to the epistemological perspective of an agent that starts with zero infor- mation about the world: the knowledge of the state space is inaccessible, and uncertainty has a dominant role.
REMARK4.6. (Continuous-time modeling) This formalization can be extended to continuous- time. This allows extending most of the definitions in this chapter to be valid for the con- tinuous time setting.
First, replace the discreteSequences(B×A)with the set of continuous sequencesContSequences(B× A)(Definition A.30). The dependence of (4.2) ona:kis replaced by the dependence ona[0,t]
(tincluded), andb:k is replaced byb[0,t)(texcluded).
DEFINITION4.7 (Black box system, continuous time definition.). Given an input spaceA and an output spaceB, ablack-boxsystemDis a function that assigns a probability distri- bution for the next output as a function of the previous history of input and output:
D:ContSequences(B×A)→ProbMeasures(B) (4.3)
such that the distribution of the outputbt, after the inputsa[0,t] have been given, and the past outputb[0,t)have been observed, is given byD(b[0,t),a[0,t]):
bt ∼D(b[0,t),a[0,t]).
4.2. DESCRIBING BLACK BOXES 67
EXAMPLE4.8 (Instantaneous transformations). Any mapd:A→Binduces an instan- taneous, deterministic black box Dd ∈ D(B;A) defined bybk ∼ δd(ak), whereδd(ak) is an impulse centered atd(ak).
EXAMPLE 4.9 (Deterministic systems with hidden state space, and fixed initial state).
A discrete-time dynamical system is usually defined as a tuple hA,B,X,f,h,i with f : X×A→Xandh:X×A→B, such that
xk = f(xk−1,ak), bk = h(xk,ak).
To be converted to the representation we use, it is necessary to specify also the initial state (see Remark 4.4). Letx∗∈ Xbe the initial state. The complete dynamics is
x0 = x∗,
xk = f(xk−1,ak), fork ≥1, bk = h(xk,ak),
which can be put in the form (4.1) with the following technical construction.
This kind of deterministic system induces a deterministic mapF: Sequences(A)→B, constructed as follows. First define the evolution of the state as a mapG:Sequences(A)→ Xas
Gf,x∗(a0) = x∗,
Gf,x∗(a:k) = f(Gf,x∗(a:k−1),ak)).
4.3. SERIES 68
Then the mapF:Sequences(A)→Bis defined recursively as follows:
FhA,B,X,f,h,x∗i(a:k) = h(G(a:k),ak).
The induced black box is given by an impulse distribution centered atFhA,B,X,f,h,x∗i(a:k):
DhA,B,X,f,h,x∗i(b:k−1,a:k) =δFhA,B,X,f,h,x
∗i(a:k).
EXAMPLE4.10 (Stochastic systems with state space). In the case of a discrete-time sto-
chastic system with observations historyy:k−1and command historyu:k, the measureD(hy:k−1,u:ki) is simply the posterior distributionp(yt|y0:t−1,u0:t), which can be written as a function of
the observation modelp(y|x), the transition modelp(xt|xt−1,ut), and the prior p(x0). 4.3. Series
If two systems Dand E have compatible input and output spaces, then they can be composed in a seriesED, If Efollows D. If D ∈ D(B;A)and E ∈ D(C;B), thenED ∈ D(C;A). The notation “output; input” is that it is compatible with the definition of series:
|{z}E
D(C;B)
|{z}D
D(B;A)
∈D(C;A).
The only problem with this choice is that usually diagrams are drawn with signals going left to right (Figure 4.1).
REMARK 4.11. (D(A;A)is a monoid.) In the set D(A;A) of systems with same input
Figure 4.1. The series of two systemsDandE, ifEfollows D, is written asED. This clashes with the convention of drawing diagrams with signals flowing left to right.