Figure 2.6: The Mars prototype rover Pluto in the JPL Mars Yard (left). An image showing the types of terrains available in this dataset (right).
rock’, is also present in the dataset. The rover did not drive over the rocks during the data collection, since they are obstacles, but it may still be desirable for them to be recognized and avoided.3 This dataset contained a variety of slopes as well.
This setting is used for demonstrating the results of integrating the slip prediction algorithm on the rover in Section 2.11.
2.6 General framework for slip learning and pre-
2.6.1 General framework
The dependence of slip on terrain slopes, called earlier slip behavior, is known to be highly nonlinear [81], but the precise relationship varies with the terrain type [15].
So, we cast the problem into a framework similar to the Mixture of Experts frame- work [64], in which the input space is partitioned into subregions, corresponding to different terrain types, and then several functions, corresponding to different slip be- haviors, are learned for each subregion. That is, in each region one model of slip behavior would be active, i.e., when the terrain type is known, slip will be a function of terrain geometry only.
More formally, let I be all the information available from stereo pair images, I = (A, G). Let f(Z|I) =E(Z|I) be the regression function of slip Z (Z can be any of the slip in X, in Y, or in Yaw) on the input variables A, G (used interchangeably with the image information I). Now, considering that we have several options for a terrain type T, each one occurring with probability P(T|A, G), given the information from the image in question A, G, we can write f(Z|I) as follows:
f(Z|I) = f(Z|A, G) = X
T
P(T|A, G)f(Z|T, A, G), (2.1) where PT P(T|A, G) = 1. This modeling admits one exclusive terrain type to be selected per image, or a soft partitioning of the space, which allows for uncertainty in the terrain classification. We assume that the terrain type is independent of terrain geometry P(T|A, G) = P(T|A) and that, given the terrain type, slip is independent of appearancef(Z|T, A, G) = f(Z|T, G). Assuming independence of appearance and geometry is quite reasonable because, for example, a sandy terrain in front of the rover, will appear approximately the same, no matter if the rover is traversing a level or tilted surface. So we get:
f(Z|I) = X
T
P(T|A)f(Z|T, G). (2.2) In summary, we divide the slip learning problem into a terrain recognition part
(P(T|A), i.e., the probability of a terrain type, given some appearance information) and a slip prediction part (f(Z|T, G), i.e., the dependence of slip on terrain geometry, given a fixed terrain typeT). For simplicity, instead of the mixing coefficientsP(T|A), we use a single winner-take-all terrain classification output:
T(A) = argmaxTP(T|A). (2.3)
However, using the probabilistic output P(T|A), if available, has more advantages.
For example, it can implement smooth transitions between terrains and can provide confidence intervals for the final slip prediction.
The terrain classification output T(A) will be learned and predicted by a terrain classifier (Section 2.8). The regression functions fT(Z|G) = f(Z|T, G) for different terrain typesT will be learned and predicted by a nonlinear regression method (Sec- tion 2.9). More precisely, suppose we are given training data D = {(xi,yi), zi}Ni=1, where xi is the i-th appearance input vector, yi is the i-th geometry input vector, zi is the corresponding slip measurement, and N is the number of training examples (x,yare particular representations of the appearanceAand geometryGinformation in the image, respectively). We will independently train a texture classifier T(x) to determine the terrain type, using the appearance information x in Section 2.8 and a nonlinear function approximation ZT(y) = fT(Z|G = y) for a particular terrain type T in Section 2.9. When doing testing we will use the full input vector (x,y), recognize the terrain type T0 = T(x), and then predict slip, as a function of slopes, from the slip behavior function ZT0(y) learned for the terrainT0.
We believe this approach is adequate for our slip prediction problem because ter- rain types do not represent a continuum in appearance space and, in general, would form separate regions in the input space (experts), each one of potentially different slip behavior. In the case of making a winner-take-all decision (Equation (2.3)), the framework implements this underlying ‘switching’ behavior of slip. The probabilis- tic decision additionally allows several experts to be active at the same time and can make smooth transitions in borderline terrain cases. In both cases, we have ex-
ploited information about the structure of the problem, i.e., that the slip behavior can change depending on terrain [15]. The alternative to introducing structure in the problem is pooling appearance and geometry features, which will not only make the problem more complex because of increased dimensionality, but will also require a formidable amount of training data. This framework is general and, in principle, allows for different ways of addressing the problems of learning to recognize terrain types from appearance, and different algorithms for learning of slip behavior from terrain geometry.
2.6.2 Architecture
In this section we briefly describe the architecture of our system, summarized in Figure 2.7. We will be using the stereo imagery as input, as well as the IMU of the vehicle and its wheel encoders (the latter is needed only for training). Stereo imagery is used to create a 2D cell map of the environment from its range data. It also provides appearance information for each cell in the map. The 2D map contains geometry information about the terrain (G) and, as we are interested in terrain slopes with respect to gravity, we use the vehicle’s IMU to retrieve an initial gravity-leveled pose. In fact, a filtered IMU signal is used, often in conjunction with other onboard sensors. The appearance information from color imagery (A) will be used to decide which terrain type corresponds to a cell or a neighborhood of cells. This is all the information necessary to perform slip prediction with our algorithm. The advantage of such a system is that it can sense the terrain remotely and that it needs only passive, cheap, and self-contained sensors on the vehicle, such as stereo vision.
In order to learn slip we have added slip feedback. The mechanism to measure slip is as follows. The actual motion between two frames is estimated by VO which only needs two consecutive stereo pairs as input [88]. The motion which the vehicle thinks it has performed is given by the vehicle’s forward kinematics. For example, the LAGR vehicle has a differential drive model, so the wheel encoders are sufficient to compute its full kinematics. A more complex kinematic model, which needs additional angle
Figure 2.7: Slip prediction algorithm framework.
sensors, is needed for a rocker-bogie type of vehicle, such as Rocky8, Pluto, or MER, but it is well understood how to compute it [113, 54]. Differencing the actual motion and the motion estimated by the kinematic model gives a measurement of slip for a particular step. This feedback is used for collecting training examples to learn slip from stereo imagery.