The problem of recovering the three-dimensional geometry of space f1'0111 two-dimensional projectiolls can be broken down into several related subtasks, such as feature extraction, eye vergence control, computation of real distance [rOl11 image disparity and eye position, etc.
The subtask solved by the circuitry described in this chapter is called stereo correspondence, which allows the determination of image disparity. A more complete description of the problem of stereocolTespondence can be found in [14, 37J.
Binocular vision generates two images of a scene, one from each eye. Because the two eyes regard the scene frolll dilTerent points of view, they differ in their impression of the relationships between objects. Figure 4.2 shows two eyes of an observer in cross-section.
The lens of the eye focuses an image of the scene composed of discrete targets located in three-dimensional space onto the surface of the retina. Stereo correspondence is the pairing of features in one retinal image with features on the other retinal ilnage that a.rose from the SaIne target in three-dimensional space.
The task of finding matching features in each eye would be straightforward if features could be identified uniquely. However, random dot stereograms [13J demonstrate that the hUlnan visual system call compute disparity even when there al'e many identical features in close proximity (sec Figure 4.1). necause no pattern is visible monocularly, the deter- mination of correspondence must take place without cognitive assistance. Furthernlorc, no single pair of targets is sufficient to determine the appropriate correspondences; since all
Figure 4.2: Stereopsis. This figure illustrates the projection of images of four identical targets (dark disks) onto the right and left eyes of an observer. The lines going through the lenses
connecting each target with the retinas are lines of sight. The intersections of the lines of sight indicate possible target positions in space. False targets (transparent disks) are located at the intersections of lines of sight that originate from different targets in the two eyes.
targets are identical, they could be matched with any of the others. The intersection of lines of sight for features that do not correspond represents false targets. There is no way to differentiate a false target from a real target without making some assumptions about the three-dimensional structure of the targets. The determination of appropriate correspon- dence is a cooperative process that must consider simultaneously many possible feature pall"mgs.
Calculation of stereo correspondence is simplified by the fact that the search need not take place over the entire two-dimensional image. The features in the right and left image corresponding to the sanle target are confined to lie along lines on each of the retinae, as shown in Figure 4.3. These lines, called epipolar lines, are the locus of points that must be searched to establish the stereo correspondence of the features that line on them. The origin of the epipolar lines can be understood by imagining that the image position of a single feature is known, as are the positions of the two eyes, but that the 3-D position of the target giving rise to the image feature is not. The feature in the image projects through the nodal point of the eye along a line of sight. The target could lie anywhere along this line.
That line of sight is imaged through the nodal point of the other eye to the corresponding cpipolar line. Corresponding epipolar lines in the two images result frOlll intersection of the plane defined by the nodal points of the two eyes and the target, with the image planes.
When the eyes arc verged to infinity so that the optical axes of the eyes are parallel, the epipolar lines are all parallel to the horizontal axis (assuming that the image planes are flat).
Features in the right image at a particular elevation must correspond to features in the left image at that same elevation. However, as vergence changes, the epipolar lines tilt. All of the epipolar lines intersect at the point defined by intersection of the line connecting the nodal points of the eyes and the (infinitely extended) image plane [lOJ. In this case, the search for corresponding features must extend over different ve,·tical displacements, depending on the state of vergence of the eyes. Although the region of possible correspondence shifts as the cpipolar lines arc tilted, for any given state affixation, the search for possible correspondence
is a one-dimensional problem.
Once the stereo correspondence of the targets has been determined, the disparity can be calculated. In this chapter, dispari ty is defined geometrically, as if the points in the retina were assigned coordinates (Xl.
yil
in the left eye and (Xc, y,) in the right eye. The geometryFigure 4.3: The epipolar lines for two targets at different elevations. The retinas (vertically oriented image planes) are shown symmetrically verged about the midline. The nodal points of the lenses arc shown as filled circles. Two targets (filled squares), one above the other, are shown with associated lines of sight (dotted lines). Each target, along with the nodal points, defines a plane, which intersects the retinas to form epipolar lines. The epipolar lines intersect at the point of intersection between the line joining the nodal points of the eyes and the image plane.
of the epipolar lines depicted in Figure 4.3 demonstrates that, in general, receptors with the same coordinates on the two retinae cannot be stimulated by the same target. The locus of points in space that stimulate the same coordinates on the two retinae is called the horopter and is the zero-disparity surface of fixation. The horopter exists over an entire two-dimensional image only when the optical axes of the eyes are parallel. Otherwise, the geometrical horopter exists only in the horizontal plane that intersects the nodal points of the eye and is perpendicular to the image plane. This horopter is known as the Vieth- Miiller circle. The simplest interpretation of the one-dimensional stereocorresponciencc ch.ip is that it is computing correspondence on the epipolar line of this circle. All the image features corresponding to targets in the plane of the Vieth-Muller circle have the same retinal y-coordinate. Targets closer to the viewer than the horopter have crossed (negative) disparity, (XI