8.5 Experiments
8.5.2 Real experiments
An experiment with real data is carried out. The real data is obtained from a spherical imag- ing device, LadybugTM2 camera system [32]. The LadybugTM2 camera system consists of 6 cameras in the head unit. There are 5 cameras along the ring of the head unit and one cam- era on top of the head unit as shown in Figure 8.11. Although this camera system is mainly used to capture images of spherical or omnidirectional vision, the total 6 cameras are consid- ered as a multi-camera system. Accordingly, the LadybugTM2 camera is a real example of the
“locally-central” case of generalized cameras.
To acquire the ground truth, a trajectory of the LadybugTM2 camera is generated from a computer aided drawing tool (Xfig) as shown in Figure 8.12. This trajectory is a∞-shape and it has marked positions for the LadybugTM2 camera to be aligned at every frame. As seen in Figure 8.11, the bottom of the LadybugTM2 camera is flat. So, one of the edges on the bottom of the head unit can be aligned with the marked positions in the experiment. For the alignment, a target point on the edge is marked with a label. Then, the trajectory is printed on a piece of A2-size paper and the printed trajectory is attached under a piece of half-transparent paper with 1mm grids. All the marked positions can be measured in millimetres in 2-dimensional coordinates, and they provide us the ground truth for the motion of the LadybugTM2 camera in
§8.5 Experiments 123
0 10 20 30 40 50
0 5 10 15 20 25 30
iterations
residuals
Residual convergence curve (noise = 0.05 ,100 points, average of 50 runs )
Figure 8.5: An average convergence curve of the alternation procedure, i.e. residual error v.s. number of iterations. The curve was generated by averaging 50 runs with 0.05 degrees of the standard deviation noise.
§8.5 Experiments 124
0 0.05
0 10 20 30 40 50
Error in rotation
in degrees
0 0.05 0.1
0 10 20 30 40 50
in degrees
Error in translation direction
0.950 1 1.05
50 100 150 200
scales Estimated scale
Figure 8.6: Histograms of estimation accuracy based on 1,000 randomly simulated tests for non-axial multi-camera rig. In all these tests, we introduce angular noise at the level of standard deviation 0.05 degrees. The number of rays is 100.
0 0.02 0.04 0.06
0 10 20 30 40 50
Error in rotation
in degrees
0 0.05 0.1
0 10 20 30 40 50
in degrees Error in translation direction
0.950 1 1.05
50 100 150 200
scales Estimated scale
Figure 8.7: Histograms of estimation accuracy based on 1,000 randomly simulated tests for an axial camera rig. In all these tests, we introduce angular noise at the level of standard deviation 0.05 degrees.
The number of rays is 100.
§8.5 Experiments 125
0.02 0.04 0.06 0.08 0.1
−0.005 0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045
Standard deviations of noise (in degrees) Error v.s. noise level, for non−axial multi−camera rig Rotation error
Translation error Scale error
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Standard deviations of noise (in degrees) Error v.s. noise level (when the noise level is relatively higher)
Rotation error Translation error Scale error
Figure 8.8: This figure shows estimation accuracy (in rotation, translation, scale) as a function of noise level. The error in scale estimate is defined ask1−||||tˆt||||k. Results for simulated non-axial camera rigs.
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1
−0.01 0 0.01 0.02 0.03 0.04 0.05
Standard deviations of noise (in degrees) Error v.s. noise level, for axial multi−camera rig Rotation error
Translation error Scale error
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Standard deviations of noise (in degrees) Error v.s. noise level (when the noise level is relatively higher)
Rotation error Translation error Scale error
Figure 8.9: This figure shows estimation accuracy (in rotation, translation, scale) as a function of noise level. The error in scale estimate is defined ask1−||||tˆt||||k. Results for simulated axial camera rigs.
§8.5 Experiments 126
0 0.05 0.1 0.15
0 5 10 15 20 25
Monocular
Error in rotation
in degrees
0 10 20
0 50 100 150 200
in degrees
Error in direction of translation
0 0.05 0.1 0.15
0 10 20 30
Our algorithm
in degrees
0 0.01 0.02 0.03 0.04 0
5 10 15 20
in degrees
Figure 8.10: Experiment results for a 2-camera stereo system. Top row: estimation errors in rotation and translation direction by using one camera only (i.e., monocular). Bottom row: estimation errors obtained by the proposed method.
§8.5 Experiments 127
(a) (b)
Figure 8.11: (a) LadybugTM2 camera system consisting of 5 cameras on the side and 1 camera on the top of the head unit. A label is attached on the left-side edge of the bottom of the head unit, which is just under the red light-emitting diode (LED). The label is used to align the camera with a trajectory printed on a piece of paper. (b) Positions of the 6 cameras in LadybugTM2 camera. The positions are retrieved from calibration information provided by Point Grey Inc. The order of cameras is indicated as colour red, green, blue, cyan, magenta and black, respectively. The label for the alignment is indicated as a cyan dot at the bottom of the head unit. (All copyrights of the original CAD drawing are reserved to Point Grey Inc. Modified and reprinted with permission fromhttp://www.ptgrey.com)
Start/end position
40 45 50 55
60 65 70 75
80 85 95 90
100 105 0
10
15 5
20 25
30 35
Figure 8.12: A∞-shape trajectory produced by a drawing tool. The trajectory is printed on a piece of paper and is used for the path of the LadybugTM2 camera in the experiment. The trajectory is a closed- loop and has 108 positions. A starting position and end position are shown as a red line segment, and the frame numbers are shown.
§8.5 Experiments 128
Figure 8.13: Experiment setup with a LadybugTM2 camera and books surrounding the camera. The LadybugTM2 camera is placed on a piece of A2-size paper on which the trajectory of 108 positions of cameras is printed.
Figure 8.14: A sample of 6 images taken by the LadybugTM2 camera placed on a piece of paper and surrounded by books in the experiment. The first 5 images from the left are from the camera id number 0 to 5, which are on a ring of the head unit, and the last picture is from the camera id 6, which is on the top of the head unit.
this experiment.
For features to track in this real experiment, static objects such as books and boxes are placed around the LadybugTM2 camera, as shown in Figure 8.13. Then, the LadybugTM2 cam- era is manually moved and aligned with the marked positions at every frame.
A set of six images is captured by the LadybugTM2 camera at each marked position. The number of the marked positions is 108, so a total of 648 images are captured in this experiment.
The size of the captured images is1024×768pixels and all calibration information is provided by Point Grey Inc [32]. A sample of 6 images captured by the LadybugTM2 camera in the experiment is shown in Figure 8.14.
Features in the images are detected, and tracking of the features is performed throughout
§8.5 Experiments 129
−120
−100
−80
−60
−40
−20 0 20 40 60
−350
−300
−250
−200
−150
−100
−50 0
50
Linear Ground truth Start frame
End frame
End frame
Figure 8.15: Estimated motion of the LadybugTM2 camera in the real experiment using our proposed
“linear method” which is indicated as blue dots and lines. The ground truth of the motion is superim- posed as red dots and lines. All the estimated positions go well until the frame number 92 out of total 108 frames. At the moment of the frame number 93, the linear method gives a large amount of displace- ment error. However, after that frame, the estimation goes well again until the last frame. The estimated loop would be closed if there were no large error at the frame 93. It tells us our linear method needs to find some other ways or non-linear estimation using bundle adjustment to improve the result. Accord- ingly, the linear method serves as a good initial estimate for the bundle adjustment. The measurement unit in this figure is millimetre.
6 image sequences by Boujou 2d3 software [1]. Because of the wide-angle lenses of the LadybugTM2 camera –2.5mm focal length high quality micro lenses– there is a large amount of radial distortion in the captured images. So, radial distortion correction is applied to the coordinates of the features. After the radial distortion correction, a RANSAC algorithm is used to get rid of outliers from the features [13].
Given all inliers at every frame and camera calibration information, Pl¨ucker line coordi- nates for the inliers are represented in a local coordinate system. One of the six cameras in the LadybugTM2 camera system is selected and aligned with the origin of the local coordi- nate system. With all these real data, the estimated motion of the LadybugTM2 camera and its comparison with the ground truth are shown in Figure 8.15. We showed a 3D view of the esti- mated motion and positions of all 6 cameras of the LadybugTM2 camera system in Figure 8.16.
Specifically, note that the trajectory is a closed loop and the estimated positions of the cameras