Scannerless laser range
camera
Wilfried Schroeder
Ernst Forgber and
SteÂphane Estable
Automated control of processes like naviga-tion, rendezvous and docking of spacecrafts, collision avoidance and navigation of mobile rovers for planetary research missions as well as manipulation and navigation of mobile robotic systems require 3D information about the environment. These applications there-fore can be effectively supported by vision-based distance measurement techniques. The enormous field of applications would be increased even further by the availability of 3D vision systems with high frame rates by automotive and multimedia applications.
In the past laser radar techniques have been investigated in detail and demonstrations of the powerful capabilities of this technique have been performed. Laser radar techniques show advantages for moderate to large ranges and image update rates in the order of seconds. Pulse diode lasers cover the distance range up to some 100m and diode pumped solid-state lasers measure up to several 100km. However, the mechanical scanning principle limits the obtainable lateral resolu-tion and image repetiresolu-tion rate.
A new scannerless 3D laser range camera system has been introduced, which has the capability to overcome the lateral resolution and frame rate constraint of conventional scanning systems. The range camera gener-ates video images as well as robust range images, providing intensity and distance information for each pixel.
Scannerless laser range camera
description
Principle of operation
The range image is achieved by means of an indirect time of flight measurement. The principal setup of the sensor system is sketched in Figure 1.
The complete scene of interest is illumi-nated with laser light pulses of typically 10-70ns duration. Laser diodes are taken as illumination source, as they provide high power density even at pulse length of 10ns. For each image a pulse train of some thousand pulses is emitted with 1 per cent duty cycle in order to conveniently operate pulsed diode lasers. Laser optics widen the light pulses to homogeneously illuminate the camera field of view, thus making the laser The authors
Wilfried Schroeder,Ernst Forgberand
SteÂphane Estableare with DaimlerChrysler Aerospace AG Space-Infrastructure, Bremen, Germany.
Keywords
Lasers, Cameras, Navigation
Abstract
The requirement to quickly obtain 3D measurements is common to a variety of tasks in different fields of applications, i.e. robotics, space craft rendezvous and docking scenarios and industrial assemblies. In this paper, a novel type of non-scanning optical 3D laser camera is presented. This is useful for both space and industrial applications. Avoiding any moving mechanical parts, the new sensor is able to exceed the frame rate and lateral resolution limits of common scanning devices, while providing both a grey-scale image and a dense range image of the scene at the same time.
.
Electronic access
The research register for this journal is available at
http://www2.mcb.co.uk.mcbrr.sr.asp
The current issue and full text archive of this journal is available athttp://www.emerald-library.com
285
Sensor Review
Volume 19 . Number 4 . 1999 . pp. 285±291
light safe for the human eye, as shown in Figure 2. The laser safety class is either 1 or 3a, depending on the size of the radiating aperture and on absolute laser power, respectively.
The light pulses are backscattered by various objects in the scene and arrive at the camera after individual time of flight, corre-sponding to the distance of the object to the sensor head.
In Figure 3, three objects are shown at increasing distance. In order to estimate the time of flight of the laser echo for each image pixel, three consecutive frames of the scene are grabbed.
The grey scale image ``G'' and distance image ``D'' both register the laser echos, differing only in trigger time of the fast electronic camera shutter function relative to laser pulse emission. The camera shutter
window in the ``G'' image is arranged so as to detect 100 per cent of all laser echos back-scattered from objects within the
measurement volume; thus ``G'' may be used to normalise absolute intensity. The shutter window in the ``D'' image is triggered some-what earlier relative to laser pulse emission than in ``G'' image mode. Incoming laser echoes are registered partially due to the shutter window, the amount of clipping being proportional to time of flight, thus converting object range into intensity of the correspond-ing pixel in the ``D'' image.
The background image ``B'' (not shown in Figure 3) is taken with laser power switched off, in order to eliminate any ambient light in ``G'' and ``D'' by subtraction, thus improving signal to noise ratio.
Since the intensity of a pixel in a ``D'' image also depends on the absolute laser power and
Figure 1Principal setup
object surface characteristics, the intensity of the corresponding pixel in the ``G'' image is used to compute a normalised quotient image ``Q = D/G'', in which intensity is linearly scaled by object range, as depicted in
Figure 4. The range measure for each pixel is finally found by look-up table operation on the ``Q'' image.
Apart from being used for normalisation purposes, ``G'' serves as the normal mono-chrome video image of the illuminated scene.
Field prototype
The measurement principle described above has been implemented in field prototypes and used in various test environments. The range
Figure 3Fast shutter operation
Figure 4Normalizing absolute intensity (schematic)
camera system consists of four major components, as shown in Figure 5:
(1) CCD camera equipped with a fast elec-tronic shutter facility with shutter operations down to some 10ns.
(2) Control unit responsible for camera and laser trigger generation of some 10ns duration.
(3) Laser light source, illuminating the scene with short pulses of 850nm wavelength. (4) Image processor to compute the range
image from the sequence of ``G'', ``D'', ``B'' images.
Owing to the measurement principle, scan-ning is avoided and the design does not include any moving parts. The sensors field of view can be widely adjusted by selecting appropriate laser optics and standard camera objectives. Figure 6 shows the sensor head consisting of camera and laser illuminator. A summary of the technical data is given in Table I.
The sensors operating range depends on absolute laser power and the selected field of view. Its depth of field depends on the laser pulse length and may be adjusted by setting parameters in the control unit during system operation.
For the current field prototype, acquisition of the ``G'', ``D'', ``B'' images is frame sequential, leading to a range image frame
rate of 6.7Hz at maximum lateral resolution and to 10Hz with binning at pixel level. The distance resolution of 2 per cent is limited by the photon noise level, which is directly correlated to the full well capacity of the CCD used. However, first experiments have shown that we are able overcome this limit by merging several CCD pixels, i.e. accepting a reduction of the lateral resolution.
Applications
The 3D camera provides a grey scale image and a range image of the illuminated scene, as shown in Figure 7. The grey scale image is the
Figure 5Laser range camera system architecture
``G'' frame mentioned above, sensing full laser echo intensity for objects inside the measure-ment volume. The backscattered light is in the infra-red part of the spectrum, not visible to the human eye.
Intensity and range image are correlated on the pixel level, thus a sensor fusion operation is not necessary.
Figure 7 demonstrates the usefulness of the laser range camera for the orientation of an automated mobile robot in its environment. For instance, mobile planetary rovers may use the sensor for navigation tasks, to identify and locate obstacles on the planned trajectory and to handle mission relevant objects. A much more practical use of the laser range camera may be the exact estimation of palette position before loading by a fork-lift truck.
The camera has successfully been used for robot hand-eye co-ordination. Figure 8 shows a set-up, which has been demonstrated at the Hannover Fair '98. Considering the range image data provided by the camera and applying advanced image processing tech-niques, a robot was used to grasp the uppermost ping-pong ball out of a bin. This application proved the high precision of the sensor to determine the lateral position of the ball's centre and demonstrated the achievable range resolution.
Using appropriate image processing algorithms the position and orientation of objects in a bin have been determined, serving as input data for the grasping process of a 6-DOF robot. Two different approaches, one of them model-based, the other model-free, have been used to achieve this goal. Figure 9 shows the results of model-based image processing algorithm applied to boxes in a bin.
Model-based approach
One of the main problems to solve in order to recognize boxes in a bin is to select mean-ingful object features, e.g. corners, edges, or region, respectively. The regions extracted by the selected plane segmentation could not be directly used as the basic feature for the recognition since the boxes are randomly positioned in the bin occluding each other. Therefore, surfaces of adjacent boxes may merge together into the same region, i.e. one region does not necessarily correspond to a single box. In order to improve object
Table I3D camera technical data
Technical data
Operating range (depending on used optics) 0.5-30m
Depth of field
Minimum 1.5m
Maximum 10m
Field of view (depending on used optics)
Horizontal 428, 298, 218, 108
Vertical 328, 228, 168, 88
Lateral resolution (standard) 6406480 pixels
Horizontal binning 3206480 pixels
Vertical binning 6406240 pixels
Horizontal/vertical binning 3206240 pixels
Image frequency with full lateral resolution 6.7Hz
Image frequency with horizontal/vertical binning 10Hz
Wavelength 850nm
Distance resolution +2 per cent of depth of field
I/Fs to PC
Control V 24
Grey scale and range images Serial, digital
Correction scale Serial, digital
Figure 7Indoor scene, monochrome and range image
detection, the 3D contour feature was included in the recognition and definition of the grasping point. Extracting regions boundaries guaranteed closed contours, which simplify and speed up further proces-sing.
The recognition and grasping problem was decomposed into two steps:
(1) Generation of a list of box candidates.
(2) Selection of one box and computation of its grasp position.
A list of box candidates was created by finding instances of the box model in the contour description of the boxes.
The contour analysis problem was reduced to 2D by projecting the contour to the box surface plane, thus avoiding any projective deformation. In a split and merge process, the
main segments of the closed projected con-tour were extracted.
A graph-matching algorithm performed the last step for isolating boxes along a contour. The model of a box was described as a graph with two nodes (length of two consecutive sides of the box) and one edge (distance and angle between these two sides). For a given
contour, the graph was generated and a back propagation search algorithm was used to detect sub-graph models in this graph, pro-ducing a list of box candidates. Finally, one box was selected from the list of candidates and the a priori object knowledge was used to compute its grasp position.
Grasp skill
Another way to find suitable grasp positions besides using a model as mentioned above is a model free approach called a grasp skill. The grasp skill is able to calculate and compare different grasp positions on arbitrarily shaped objects for a great variety of parallel jaw gripper shapes. The whole process consists of two main sections where the first section is the calculation (simulation) of a number of possible grasp positions and the second section is the comparison of the different grasp positions by quality features. Finally a best grasp position with the highest quality value is chosen and executed by the robot.
Conclusion
A new active, non-scanning 3D imaging system has been presented, which is able to generate range images at high frame rate and high lateral resolution. It has been demon-strated, that tasks in robotics, industrial assembly, navigation of autonomous vehicles in terrestrial, as well as in space applications, will benefit from the video and range
information generated by the laser range camera. Current activities concentrate on completion of an industrial version of the laser range camera and the provision of high power laser sources. Both will be available before the end of the year. Future develop-ments will focus on higher frame rates of about 20Hz with improved robustness with respect to moving objects in the scene. Tests with a first 20Hz version show quite encouraging results.
Figure 9Box with rectangular solids, monochrome image, depth image and results of range image segmentation with detected grasping positions (large boxes)
