2.7 Objective of the Thesis
3.1.3 A brief overview of existing stereo correspondence algorithms
3.1 Introduction
d=0d=1
d=dmax-1
.. .
Aggregated cost
disparity Aggregated
cost
Winner disparity
0 dmax-1
Figure 3.7: Disparity computation.
Global algorithms perform this step by minimizing a global cost function comprising data and smoothness terms. Minimization is performed using the optimization techniques. In global algorithms, the disparity value of a particular pixel is influenced by the disparity values of the neighbouring pixels.
3.1.2.4 Disparity map refinement
This is the post processing step which is done to obtain an accurate disparity map. It includes occluded regions detection, filling of appropriate disparity values to these detected regions, and finally performing refinement of the obtained disparity map.
Based on the abovementioned general steps of a stereo correspondence algorithm, we proposed a local stereo matching algorithm. The details of our method are discussed in the following section.
census transform for stereo matching. The limitation of both these methods is that the local measures depend on the intensity value of the center pixel. In addition to this, Histograms of Oriented Gradient (HOG) descriptors are also used for stereo matching [119]. A rectangular region around the pixel of interest is divided into smaller subregions, and HOG is computed for all the subregions. The concatenation of all HOG represents the feature of the considered pixel. The disadvantage of this method is that the size of the feature vector depends on the size of the rectangular and small subregions.
Also, the size of the feature vector depends on the number of bins of the histogram. Hence, the size of feature vector is large.
Utilization of the adaptive support weights significantly improves the performance of the local stereo matching algorithms. Yoon and Kweon proposed a stereo matching method by using adaptive support weights (ASW) [10]. In this method, weights are assigned to the pixels within the correlation window, which are inversely proportional to the spatial distance and the colour dissimilarity from the center pixel. The support window must be sufficiently large for better accuracy. But, a larger window severally affects the algorithm in real time implementation.
Gerrits et al. proposed a cost aggregation method based on the mean shift-based colour segmen- tation of the reference image [92]. Pixels in the support and the target windows are partitioned into two disjoint sets. Pixels that belong to the same segment are given weight 1, and 0 is assigned to the remaining pixels. It is based on the assumption that pixels within the same segment are likely to have similar disparity values. Mean-shift segmentation adds computational overhead to the weighted cost aggregation.
A cost aggregation method using geodesic distance is proposed in [112]. In this adaptive weight algorithm, weight of a pixel in the correlation window is defined by computing the geodesic distance to the center pixel. Pixels having a short geodesic distance to the center pixel (i.e.,pixels having an approximately constant colour path to the center pixel) are given relatively high weights, whereas the pixels having long distance are assigned low weights. Performance of this method depends on the size of the support window.
Hosniet al. performed cost aggregation using a guided filter (GF) [11]. The computational com- plexity of this method depends on the size of the image and the number of labels used. De-Maeztu et al. proposed a cost aggregation method based on a linear model [120]. This cost aggregation method relies on the symmetric strategy, and it performs cost aggregation using both the input colour images.
In this method, execution time depends on the size of the input image. The major difficulties faced by stereo correspondence methods is the computation of disparity at occluded, textureless, and disconti- nuities regions. Most of the existing local stereo matching algorithms suffer from all these problems,
3.1 Introduction
which eventually affect the performance of the algorithms in terms of real-time implementation. In the last few decades, a number of stereo matching methods were proposed to overcome some of these problems. In view of this, we propose a new feature-based stereo matching method. The proposed method uses a local Gabor wavelet feature for matching cost computation. Subsequently, two-pass cost aggregation is performed by combined use of Kuwahara filter and median filter.
Major contributions of this chapter are highlighted as follows.
Matching cost computation:
• Gabor wavelet in spatial domain is applied to the input images for computing corresponding points in the given stereo pairs. The motivation behind using Gabor wavelet for disparity computation is as follows:
– The simple cells of the visual cortex of mammalian brains are best modelled as a family of self-similar two-dimensional Gabor wavelets [121]. So, the extracted features by the proposed method will convey almost similar information as that of human visual system.
Gabor feature can extract texture information [122].
– Two-dimensional Gabor wavelet has good spatial localization, orientation selectivity, and frequency selectivity property. So, the features extracted in the proposed method have local and discriminative characteristics.
– The proposed local Gabor wavelet feature differs from the existing Gabor features in two ways:
(i) Existing Gabor feature methods apply Gabor wavelet to the entire image [123] or extracts Gabor feature at a particular point [124], whereas the proposed method applies Gabor wavelet to overlapping local image patches. Hence, the method is termed as local Gabor wavelet-based feature extraction method. The proposed method extracts feature for all the pixels in the input image. In general, local approach extracts the maximum information as compared to holistic approaches [125].
(ii) The proposed method only uses real coefficients obtained after convolving the local image patches with Gabor wavelet, whereas existing Gabor feature-based methods use both real and imaginary coefficientsi.e., magnitude or phase information.
• Dimensionality of the local Gabor wavelet coefficients depends on number of orientations and scalings. Hence in the proposed method, PCA is used to reduce the dimensionality of the Gabor wavelet coefficients.
Cost aggregation:
• Cost aggregation method is implemented by filtering the matching cost volume with Kuwahara filter followed by median filter.
– The motivation of using Kuwahara filter is that this filter can perform image smoothing by preserving the edges, and its computational complexity is O(1).
– Median filter is applied to remove the blocking artifacts produced by Kuwahara filter.
Experimental evaluation of the proposed method clearly shows that the output disparity maps gener- ated by the proposed method are qualitatively and quantitatively better than the existing methods.