Learning from automatic supervision

of the terrain types during training, so the mechanical measurements zi, which are assumed to have come from one of the unknown nonlinear models, will act as the only supervision to the whole system. The main problem is that using the mechanical measurements as the only ground truth, or supervision, we have to learn both the terrain classification and the unknown nonlinear functions for each terrain (note that the models for the particular mechanical behavior might not be known beforehand, as is the case with slip). The difficulty of the formulated problem lies in the fact that a combinatorial enumeration problem needs to be solved as a subproblem, which is known to be computationally intractable [66].

Figure 3.4: Schematics of the main idea of incorporating automatic ambiguous super- vision into terrain classification. Examples for which the supervision signal is distinc- tively different can propagate this information to examples of ambiguous or noisy supervision through their similarity in vision space.

slip feedback to supervise the vision-based terrain classification. Our goal now is to make those two different sets of information interact.

We provide a solution to (3.1) in a maximum likelihood framework. The problem is that classifying the terrain types in visual space and learning their slip behavior are not directly related (i.e., they are done in different, decoupled spaces) but they do refer to the same terrains. So, we introduce hidden variables L (from a multi- nomial distribution with a parameter π) which will define the class-membership of each training example, similar to MoG [24]. In our case L_ij = 1 if the i^th training example (xi,y_i, zi) has been generated by the j^thnonlinear model and belongs to the j^th terrain class. Now, given that the labeling of the example is known, we assume that the mechanical measurements and the visual information are independent. So, the complete likelihood will factor as follows:

P(X, Y, Z, L|Θ) =P(X|L,Θ)P(Y, Z|L,Θ)P(L|Θ),

Figure 3.5: The graphical model for the maximum likelihood density estimation for learning from both vision and automatic mechanical supervision. The observed ran- dom variables are displayed in shaded circles.

where Θ, Θ = {µj,Σj, θj, σj, πj}^K_j=1, contains all the unknown parameters that need to be estimated in the system (to be described below).

The graphical model corresponding to this case is shown in Figure 3.5. The graph- ical model [25, 97] represents the assumptions about dependencies we have made for the variables in our problem. In particular, it states our abovementioned assumption that the visual part of the data is independent of the mechanical data, given the class label of the data point is known. We have also assumed that the number of terrain types K is known and that we have a fixed appearance representation x which is good enough for our purposes. The particular forms of the distributions P(X|L,Θ), P(Y, Z|L,Θ) will be described below. πj =P(Lj = 1|Θ_j) are the prior probabilities of each terrain class.

Using the hidden variables, the complete data likelihood function for the data D={xi,y_i, zi}^N_i=1 can be written as follows¹:

P(X, Y, Z, L|Θ) =

YN

i=1

YK

j=1

[P(xi|Lij = 1,Θ)P(yi, zi|Lij = 1,Θ)P(Lij = 1|Θ)]^Lij,

1SinceLiis an indicator variable for class membership of thei^thexample,P(Lij) =QK

j=1[P(Lij= 1)]^Lij. SimilarlyP(xi|Lij) =QK

j=1[P(xi|Lij = 1]^Lij [25].

from which the complete log likelihood function (CL) for the data is written as follows:

CL(X, Y, Z, L|Θ) =

XN

i=1

XK

j=1

LijlogP(xi|Lij = 1, µj,Σj) + (3.2)

XN

i=1

XK

j=1

LijlogP(yi, zi|L_ij = 1, θj, σj) +

XN

i=1

XK

j=1

Lijlogπj.

The complete log likelihood will be optimized iteratively in the EM algorithm [32]

to obtain the optimal set of parameters. As seen, the introduction of the hidden variables simplifies the problem and allows for it to be solved efficiently with the EM algorithm.

The vision information X and the mechanical information Y, Z are considered to come from particular probability distributions, conditioned on the label. Those distributions are modeled, so that a tractable solution to the complete maximum likelihood problem is achieved. The vision data is assumed to belong to any of the K clusters (terrain types). For each of them, the mean and covariance parameters need to be estimated. The probability of a data point x_i belonging to a terrain class j is expressed as:

P(xi|L_ij = 1, µj,Σj) = e^{−12(xi−µj)TΣ−j1(xi−µj)}

(2π)^d/2|Σ_j|^1/2 , (3.3) where µj,Σj are the means and covariances of the K clusters of vision data and d is the dimensionality of the vision space.

The mechanical measurement data is assumed to come from a nonlinear fit, which is modeled as a General Linear Regression (GLR) [105]. GLR is appropriate for expressing nonlinear behavior and is convenient for computation because it is linear in terms of the parameters to be estimated. For each terrain type j, the regression function ˜Z(Y) = E(Z|Y) is assumed to come from a GLR with Gaussian noise:

fj(Y) ≡ Z(Y) = ˜Z(Y) +²j, where ˜Z(Y) = θ_j⁰ +^PR_r=1θ_j^rgr(Y), ²j ∼ N(0, σj), and gr are several nonlinear functions selected before the learning has started. Some example functions are: x, x², e^x, logx, and tanhx(those functions are used later on in our experiments with the difference that the input parameter is scaled first). The parametersθ_j⁰, ..., θ_j^R, σj are to be learned for each modelj. The following probability

model for zi belonging to the j^th nonlinear model (conditioned on y_i), is assumed:

P(zi|yi, Lij = 1, θj, σj) = 1 (2π)^1/2σj

e⁻

1 2σ2

j

(zi−G(yi,θj))²

, (3.4)

where θj are the parameters of the nonlinear fit of the mechanical data, σj are the covariances (here it is the standard deviation, as the final measurement is one di- mensional), G(y, θj) = θ_j⁰+^PR_r=1θ^r_jgr(y), and θj = (θ¹_j, ..., θ_j^R, θ⁰_j). P(yi) is given an uninformative prior (here, uniform over a range of slopes).

The parametrization of the slip models is simpler than the one in Chapter 2. This is needed because the slip measurements assumed to have come from these models act as supervision, but at the same time the models are unknown and have to be learned from the data. In this case, using a simpler parametrization requires fewer parameters to be learned and is more feasible in our current problem.

3.4.3 Algorithm for learning from automatic supervision

The EM algorithm applied to our formulation of the problem is shown in Figure 3.6.

The detailed derivations are provided in Appendix A. In the E-step, the expected values of the unobserved label assignments Lij are estimated. In the M-step, the parameters for both the vision and the mechanical side are selected, so as to maximize the complete log-likelihood. As the two views are conditionally independent, the parameters for the vision and the mechanical side are selected independently in the M-step, but they do interact through the labels, as they both provide information for estimation in the E-step. Some notations in Figure 3.6 are as follows (see Appendix A for details): L^t+1_j is a diagonal NxN matrix which has L^t+1_1j , ...L^t+1_{N j} on its diagonal, Gis a Nx(R+ 1) matrix, such thatG_ir =g_r(y_i),G_i(R+1) = 1, andZ is a Nx1 vector containing the measurements zi.

3.4.4 Discussion

Within our maximum likelihood framework, it can be seen that the algorithm copes naturally with examples providing ambiguous supervision (i.e., belonging to areas of

Input: Training data {xi,y_i, zi}^N_i=1, wherex_i are the vision domain data, y_i are the mechanical domain data,zi are the supervision measurements Output: Estimated parameters Θof the system

Algorithm:

1. Initialize the unknown parameters Θ⁰. Set t= 0.

2. Repeat until convergence:

2.1. (E-step) Estimate the expected value of Lij

L^t+1_ij = ^{P(xi|Lij=1,Θ}

t)P(yi,zi|Lij=1,Θ^t)π_j^t

PK

k=1P(xi|Lik=1,Θ^t)P(yi,zi|Lik=1,Θ^t)π_k^t.

2.2. (M-step) Select the parameters Θ^t+1 to maximize CL(X, Y, Z, L|Θ^t):

µ^t+1_j =

PN

i=1L^t+1_ij xi

PN

i=1L^t+1_ij ; Σ^t+1_j =

PN

i=1L^t+1_ij (xi−µ^t+1_j )(xi−µ^t+1_j )^T

PN

i=1L^t+1_ij ; π_j^t+1 =^PN_i=1L^t+1_ij /N;

θ^t+1_j = (G^TL^t+1_j G)⁻¹G^TL^t+1_j Z; (σ²_j)^t+1 =

PN

i=1L^t+1_ij (zi−G(yi,θ_j^t+1))²

PN

i=1L^t+1_ij . 2.3. t=t+ 1.

Figure 3.6: EM algorithm updates for learning from automatic supervision.

overlap of several different nonlinear models). For those examples the algorithm falls back to using the visual input only, because the probability of belonging to each of the overlapping models is almost equal.

The proposed maximum likelihood approach solves the abovementioned combi- natorial enumeration problem [66] approximately by producing a solution which is guaranteed to be a local maximum only. Indeed, the EM solution is prone to get- ting stuck in a local maximum. For example, one can imagine creating adversarial mechanical models to contradict the clustering in visual space. In practice, for the autonomous navigation problem we are addressing, our intuition is that the mechan- ical measurements are correlated to a large extent with the vision input and will be only improving the vision-based classification. This is seen later in the experiments.