Occlusion occurs in an image when a portion of an object is hidden by another object in the scene.
Due to this, some portions of an object is present in one image, while absent in another image in a stereo image pair. It happens due to the fact that a three-dimensional scene is viewed by two cameras from different viewpoints. In this scenario, it is difficult to find the matching pixels in the stereo image pairs by the methods described in the previous section. To obtain an accurate disparity map, firstly occluded pixels need to be identified, and subsequently appropriate disparity values have to
2.4 Occlusion Detection and Filling
be assigned to the detected pixels. This process is called occlusion detection and filling for finding a stereo correspondence.
2.4.1 Occlusion detection
Spoerri and Ullman presented a histogram-based occlusion detection method [103–105]. In this method, if a pixel is occluded then its local neighboring pixels have two different disparity values. One disparity value corresponds to the occluding region, while the other disparity value corresponds to the occluded region. Hence, the histogram of this local region will have two peaks. In contrary to this, a local region from a single object produces only one mode or peak in the histogram. Therefore, the ratio of the second highest peak to the first highest peak gives the bimodal information. Analytically, it is given by:
Bimodality in the histogram = M2
M1 (2.30)
whereM1 andM2 denote the first and second highest peak in the histogram. If the bimodality value is close to one, then it indicates that the pixel is from the occluded region.
Another occlusion detection method based on the correlation value is proposed in [106]. In this method, if a pixel is from a non-occluded region then the correlation value will be relatively high, while it is low for an occluded pixel. This is called match goodness jumps. A particular pixel is considered as an occluded pixel when the neighboring pixels have relatively high matching score with respect to the pixel under consideration.
In another occlusion detection scheme, pixel ordering (ORD) constraint is considered [45, 70].
Ordering constraint ensures that the order of a set of pixels in the reference image is maintained by the corresponding matching pixels in the other image. Violating this constraint indicates that a particular pixel is occluded.
Occlusion (OCC) constraint is used for occlusion detection [67]. Occlusion constraint states that disparity map is continuous over the entire image except near the object boundaries. There is a sudden change in the disparity value of the occluded surface with respect to the background. This sudden change in the right disparity map corresponds to an occlusion region in the left disparity map and vice versa.
Left right check (LRC) is another popular method to detect occluded pixels based on left right consistency verification. [107, 108]. In this method, if the disparity values of a pair of matching pixels are different, then the pixel in the reference image is considered as the occluded pixel. Thex-coordinate pest1 of estimated left pixelpest corresponding tox-coordinate p′1 of pixelp′ in the right image is given
by:
pest1 =p′1+dp′ (2.31)
where dp′ is the calculated horizontal disparity value of pixel p′ with right image as reference. LRC detects a pixel as occluded, if the following condition is not satisfied:
p′1−(pest1 −dpest) = 0 (2.32) where dpest is the disparity value of pixel pest computed by considering left image as reference. It is to be mentioned that LRC is the most widely used occlusion detection algorithm for last few decades [10, 11, 41, 109]. LRC detects both occluded pixels as well as pixels having wrong disparity values.
Bimodality and match goodness jumps can detect the border occluded regions while LRC, ORD, and OCC algorithms can detect the entire half-occluded regions [110]. Though the abovementioned methods are able to detect the occluded pixels, these methods also detect many correctly matched pixels as the occluded pixels. Hence, these methods produce more false positives for some of the cases.
2.4.2 Occlusion filling
In stereo vision, a portion of an object is only present in any one of the stereo images and occluded in the other image. For the pixels present in these regions, finding the corresponding matching pixels is a difficult task. To tackle this problem, these pixels are explicitly detected and subsequently assigned disparity values. The task of assigning appropriate disparity value to an occluded pixel is termed as occlusion filling.
The basic and simple occlusion filling algorithm considers the disparity value of the closest hori- zontally left non-occluded neighborhood pixel with respect to the occluded pixel under consideration.
It assigns the disparity value of this selected neighborhood pixel to the occluded pixel [111]. Another approach assigns minimum disparity value of the closest left dl and right dr horizontal non-occluded pixels [112]. This representation is expressed as follows:
D(p) = min{dl, dr} (2.33)
where p is the occluded pixel to be filled up. This approach is similar to the previous approach as the disparity value of the closest horizontal left non-occluded pixel always have minimum disparity value among the disparity values of both left and right horizontal non-occluded pixels. However, these