7.5 Experiments
7.5.2 Real data
An experiment with real data is carried out in this section. An image sequence is captured by 8 cameras mounted on a vehicle. The vehicle is shown in Figure 7.9. All 8 cameras are firmly mounted on the vehicle, and 4 of them are assigned on the left side and the other 4 cameras are assigned on the right side of the vehicle to have wide field of view. The distance between a set of 4-camera on the left and a set of 4-camera on the right is about 1.9 metres. The position of 8 cameras is shown in Figure 7.10. These cameras have little field of overlapping views with each other. So, it is an example of a real implementation of a non-overlapping multi-camera systems. The size of the images is 1024×768 pixels, and the number of frames in the image sequence for each camera is about 1,000 frames. So, a total of 8,000 frames are dealt with in this real experiment. In Figure 7.9, a sample of captured images from 8 cameras is shown.
Note that there is a very little overlapping field of view. In this experiment, only two cameras, one from the left side and another from the right side, are selected.
§7.5 Experiments 96
0 0.5 1 1.5 2 2.5 3
0 5 10 15 20 25
Variance of Gaussian distribution as noise on data
Rotation error by degree
Figure 7.8: Rotation error produced by our method for a generalized camera having two centres where 5 rays meet on one centre and other ray meets on the other centre in two views. Gaussian distribution of noise has been added to the data.
Figure 7.9: An 8-camera system of non-overlapping multi-camera rigs on a vehicle and a sample of 8 images. (Images: Courtesy of UNC-Chapel Hill)
§7.5 Experiments 97
3 2 1 0 1
1
0. 5
0
3 2. 5 2 1. 5 1 0. 5 0 0.5 1 1.5
Right Left
−3
−2.5
−2
−1.5
−1
−0.5 0 0.5
−3
−2.5
−2
−1.5
−1
−0.5 0 0.5 1
Left Right
3 2. 5 2 1. 5 1 0. 5 0 0.5 1 1.5
1
0. 5
0
3 2. 5
2 1. 5
1 0. 5
0 0.5
1 1.5
Left Right
3 2 1 0 1
1
0
0
3 2. 5 2 1. 5 1 0. 5 0 0.5 1 1.5
Right Left
Figure 7.10: Position of 8 cameras in the system. Red, green, blue and black colour indicate backward, side, forward and up direction of cameras, respectively. There are a little overlapping of field of view across cameras.
§7.5 Experiments 98
Figure 7.11: Five points selected from the left-backward camera in two views, frame 120 and frame 125. Epipolar lines corresponding the five points are plotted. An essential matrix is estimated from the selected five points. The five points from the first view is indicated as red circles and the five epipolar lines corresponding to the five points are shown as red lines in the second image. In the same way, green circles for 5 points in the second view and green lines for the corresponding epipolar liners.
First, features in image sequences are found and tracked across two views. We have used a commercial feature tracker, Boujou, to obtain robust feature tracks [1]. Then, an essential matrix from a camera on the left side of the vehicle (a backward camera is selected in this experiment) is estimated from five point correspondences using the five point minimal solution method [71]. The best five points are selected by the RANSAC algorithm and the estimated result is refined from inliers. In Figure 7.11, the five points and estimated epipolar lines are shown.
With the estimated essential matrix, the scale of translation direction is estimated from one point selected from the other camera on the right side of the vehicle. Like the five point method, the RANSAC approach is also used to find the best one point from the right side camera.
For refinement of the scale estimation, all inliers on the right side camera are used to find a solution of the least-squares of liner equations, and non-linear optimization by minimizing the geometric reprojection errors is applied. In Figure 7.12(a) and Figure 7.12(b), the best one point and estimated epipolar lines from all inliers on the right side are shown, respectively.
For evaluation of the result, the ground truth for the position of the vehicle, in other words, the position of cameras, is provided from a global positioning system (GPS) and inertial mea-
§7.5 Experiments 99
(a)
(b)
Figure 7.12: (a) One point selected from the right-backward camera in two views, frame 120 and frame 125 (indicated as red and green circles). This one point is used to estimate the scale of translation direction for multi-camera rigs. (b) All inliers used for the scale estimation and its epipolar lines. Note that there are no inliers found around a car in the image because the car in the image was moving and points on the car are identified as outliers. A total of 343 points out of 361 are found as inliers, and they contribute to find a solution of the scale by a refinement method. Red circles indicate the inliers in the first view and red lines show the epipolar lines corresponding to the red circles in the second view.
Green circles indicate the inliers in the first view and green lines show the epipolar lines corresponding to the green circles in the second view.
§7.5 Experiments 100
100 150 200 250 300
0 2 4 6 8 10 12
14 Critical Angles
Frame numbers
In degrees
(a)
100 150 200 250 300
10−2 10−1 100 101 102
Scale estimation ratio
Frame numbers
Ratio of the estimate and the true value
(b)
Figure 7.13: (a) Critical angles in real experiments: From frame number 150 to 250, it is larger than 2 degrees of critical angles. (b) Scale estimation in real experiments: From frame number 150 to 250, the scale estimation shows values close to the ground truth.
surement unit (IMU) device of POSLV, Applanix which is equipped in the vehicle system [2, 86].
From the geometric interpretation, we found that there is a critical configuration where our method cannot solve the scale of translation in non-overlapping multi-camera systems. Let us define critical angles as the angle between the translation vector of the first camera and the translation vector of the second camera. If the critical angle is equal to zero, it means that the motion of the multi-camera system is in a critical configuration. So, in this case, we cannot solve the scale of translation. Therefore, it is reasonable to examine how many times our 8- camera system on a vehicle has critical motions. In Figure 7.13(a), angles between the two translation vectors of two cameras are shown in each frame. From frame number 150 to 250, the angles are greater than about 2 degrees, and the rest of frames are less than 2 degrees. It means, unfortunately, most of motions of the vehicle are likely to be critical.
In Figure 7.14, the ground truth position of cameras is shown. The vehicles moved straight forward first, and then turned left and crossed over a speed bump. The speed bump mainly caused a large value of the critical angles and this motion corresponds to the frame numbers 150 to 250. Therefore, the scale of translation can be estimated correctly between these frame numbers.
In Figure 7.13(b), a ratio of scale estimation is shown. If the ratio is equal to one, then it tells us that the estimation of the scale for the translation is close to the correct solution.