Nuisances Acting on the Observations

A Catalog of Representation Nuisances

7.2. Nuisances Acting on the Observations

the largest group that preserves it.

7.2. Nuisances Acting on the Observations

Nuisance 1: Perm(n_y)– Permutations of the sensels

preconditions on format:Y=^Yny group:Perm(n_y)

action: y_i 7→^yπ(i),π∈^Perm(n_y)

This is the basic example of a representation nuisance: would your agent work if the sensels were scrambled?

If an agent has some assumptions on the identity of any signal (e.g., the first sensel encode the reward) then it will only be invariant with respect to a subgroup of this group.

Note that the observations space Y is supposed to be a set of discrete sensels, each taking value in the same setY. This nuisance would not apply to the case where the obser- vations are an infinite dimensional field.

equivariant: Pairwise statistics such as the correlation or covariance matrix are equivariant with respect to permutations.

is largest preserving: Assumption 4 (Sensels have similar statistics) Nuisance 2: GL(n_y)– Linear transformations of the observations

preconditions on format:Y=R^ny

A generic linear transformation of the observations can represent a variety of filtering operations.

preserves: Linearity and bilinearity of the dynamics.

equivariant: Covariance matrix (transforms asP7→^APAT)

7.2. NUISANCES ACTING ON THE OBSERVATIONS 125

perturbs: Correlation matrix

Nuisance 3: Homeo(^Y)– Continuous transformations of the observations

preconditions on format:Uis a topological space group:Homeo(Y)

action: y7→ ^ϕ(y)

is largest preserving: Assumption 8 (Observations are continuous in the states) Nuisance 4: Isom(Y)– Isometries of the observations

preconditions on format:Yis a metric space group:Isom(Y)

action: y7→ ^ϕ(y)

Invariance to isometries is a natural requirement in many problems. It can be often interpreted as an invariance to the data reference frame.

Note: do not confuse with Nuisance 5 (Isom(S)) .

preserves: Assumption 3 (Observations have “continuous” dynamics) Nuisance 5: Isom(S)– Isometries of the observations field

preconditions on format:Yis a field onS group:Isom(S)

action: y(s)7→^y(ϕ(s))

We assume that the observationsyare a function over some manifoldS, and we con- sider the isometries ofS. An example of this nuisance would be to mount a robot’s camera

7.2. NUISANCES ACTING ON THE OBSERVATIONS 126

(a)Isometries (b) Diffeo- morphisms

(c) Homeomor- phisms

Figure 7.1. Difference between isometries, homeomorphisms, and diffeomorphisms, in the caseS =R².

upside down. Isometries are a relatively small set of transformations, and typically finite- dimensional (Figure 7.1). For example, the isometries of the hyper sphereSⁿ⁻¹are the or- thogonal transformationsO(n)(D.2); the isometries ofRⁿform the Euclidean groupE(n) (Definition D.3).

Note: do not confuse with Nuisance 4 (Isom(Y)) .

equivariant: Isometries commute with many operations that are expressed through the met- ric ofS, such as spherical smoothing of an image (Definition 19.1).

is largest preserving: Assumption 7 (The spatial field is homogenous) Nuisance 6: Diff(S)– Diffeomorphisms of the observations

preconditions on format:Yis a field onS group:Diff(S)

action: y(s)7→^y(ϕ(s))

We assume that the observationsyare a function over some manifoldS, and we con- sider all diffeomorphisms ofS. This is a much larger set of transformations than only the isometries ofS; diffeomorphisms are an infinite-dimensional topological group (not a Lie

topological group: A group whose operations are continuous in a given topology. See

7.2. NUISANCES ACTING ON THE OBSERVATIONS 127

Figure 7.2. Contrast transformations. The human visual system is extremely robust to contrast transformations, represented by the action of the groupHomeo+(^Y).

group).

preserves: Because they are homeomorphisms, diffeomorphisms preserve the topology ofS. In addition to that, they preserve the differentiability of the signal; this allows con- sidering statistics of the spatial gradient∇sy(s)(Figure 7.1).

perturbs: Any statistics that depends on the metric ofS is perturbed by diffeomorphisms.

is largest preserving: Assumption 6 (The observations correspond to a spatial field)

Nuisance 7: Homeo+(Y) – Acting jointly on all observations

preconditions on format:Y=Y^ny

preconditions on format:Yis totally ordered

preconditions on format:(equivalently:Yis a field onS⁾ group:Homeo+(Y)

action: y_i 7→ ^f(y_i), for f ∈^Homeo+(Y)

action:(equivalently: y(s)7→ ^f(y(s)), for f ∈^Homeo+(R))

This nuisance is often called acontrasttransformation (Figure 7.2).

Nuisance 8: Aff(_R)^ny– Affine transformation of the single sensel

Definition C.43.

7.2. NUISANCES ACTING ON THE OBSERVATIONS 128

preconditions on format:Y=R^ny group:Aff(R)^ny

action: y_i 7→ ^aiy+b_i, fora_i 6=0,b_i ∈R

preserves: This is the largest nuisance that preserves the correlation between sensel values.

Nuisance 9: Perm(n_y)×^Homeo+(Y)^ny– Positive saliency

preconditions on format:Y=^Yny

preconditions on format:Yis totally ordered group:Perm(n_y)×^Homeo+(Y)^ny

action: y_i 7→ ^fi(y_π(i))

is largest preserving: Assumption 2 (Larger (or smaller) values are more salient) Nuisance 10:D^?_fm(Y)– Dynamical nuisances with finite memory acting on observations

preconditions on format: None group:D^?_fm(^Y)

action: y7→^Dy(system-signal product), forD∈^D?fm(^Y) is largest preserving: Assumption 17 (The world has finite memory)

perturbs: Any statistics ofy defined as an expectation over time (e.g., correlation) is dis- rupted by this and other dynamical nuisances.

Nuisance 11:DSMPLTI(1)– Stable, minimum phase systems

preconditions on format:Y=R^ny

action: y_i 7→ ^Dyi, forD∈ ^DSMPLTI(1)a stable, minimum phase system

7.2. NUISANCES ACTING ON THE OBSERVATIONS 129

intuition: This is a dynamical nuisance, which acts on the observations by filtering them

using a discrete-time stable, minimum-phase finite-dimensional linear time-invariant dy- namical system. While this is a mouthful, this is the minimum set of attributes to describe a linear system such that it is always possible to find a causal inverse. See Example 4.32.

perturbs: Even if the same dynamical system filters all the sensels, most statistics such as the correlation are perturbed by this nuisance.

preserves: It is possible to work in the Laplace domain to find robust statistics. LetL^in- dicate the Laplace transform of a signal. Then the transform of the filtered signal is the product of the transfer function of the filter and the transform of the signal:

L(Dy_i) =TF(D)L(y_i).

This implies that one might take statistics such as the ratios L(y_i)/L(y_j)to be invariant with respect to the action of this nuisance.

Nuisance 12:Aut(Y)^nu – Known labeling of observations

preconditions on format:Y=^Yny group:Aut(^Y)^ny

action: y_i 7→ ^fi(y_i), for f_i ∈^Aut(Y)

The action of this nuisance changes the representation of each sensel in an independent way. The order of the sensels is not changed, and no sensel values are mixed together.

perturbs: Every simple statistics like the correlation is perturbed by this nuisances.

transfer function: An equivalent representation of linear systems from an input-output perspective.