75 Figure 4.9: The performance classifier of developed algorithm. true classes of the objects are indicated in the column while the predicted classes are indicated in the row. 77 Figure 4.10: Sample results of detected ground level in rainforest. terrain for simple unstructured terrain. a)(i) and (b)(i) show the sample images and the detected ground levels are shown in (a)(ii) and (b)(ii) respectively.
Research Motivation
The color stereo camera provides 2-dimensional (2D) and 3-dimensional (3D) scene information non-invasively. The highly unstructured nature of rainforest terrain also presents a problem in the segmentation process as it is difficult to obtain a significant or salient region from the scene.
Scope of Research
However, the appearance of the water may vary depending on lighting conditions and angle. Features such as colors and textures are analyzed to determine their suitability for detecting the territory of the image.
Thesis Outline
In this thesis, the tree trunk is detected using range data and color information. Based on the tree trunk characteristic, the tree trunk detection module has been developed and presented.
Overview
Review of Visual Guidance for Outdoor Terrain: Structured
Methods based on NAVLAB and ALVINN
It attempted to extract the edge of the road assuming that the road is well structured and to classify the road base according to the color on road and non-road area. The texture feature was used to aid in the road detection with the assumption that the road region will appear smoother than the non-road region.
Methods based on ARGO
Unlike rainforest terrain, a structured road cannot be expected to be present, and the condition applied to structured road cannot be applied. Similar to Navlab and the ARGO project, the systems cannot be fully used for rainforest terrain.
Review of Visual Guidance for Outdoor Terrain: Unstructured
- Methods by DEMO III
- Methods by Darpa PerceptOR
- Methods by LAGR
- Methods by Manduchi et al (2005)
One of the most prominent projects is DEMO III from the Jet Propulsion Laboratory (JPL) reported by Shoemaker et al (1998) and Bellutta et al (2000). Both modules are merged to calculate the cost of traversability and then to determine the roughness of the terrain.
Challenges in Rainforest Terrain
Some of the objects appear to be similar in color and difficult to distinguish based on color alone. The uneven ground will affect the tilt and roll of the vehicle and sensors making it difficult to detect and estimate the ground level.
Summary
To our knowledge, no work has been done to determine the depth of the water bodies present in rainforest terrain. The terrain classification to determine the traversability of certain cover was carried out based on color feature.
Overview
The approach to solving the challenges in rainforest terrain by problem-based model is explained.
Visual Guidance System
Proposed System Architecture
The proposed visual guidance system has the ability to determine the ground plane of the scene and detect obstacles and water hazards. A general visual guidance system has been developed with three main modules which perform the task-specific processing of interest in this thesis.
Sensor Description
- Bumblebee 2 Stereo Vision System
- Triclops Stereo Vision SDK (PGR Software Development Kit)
- Intel OpenCV Library
- Hardware Setup
The purpose of the Triclops library is to provide the Bumblebee 2 Stereo Vision with accurate and fast depth map generation. It is not feasible to assume that the pitch angle of the camera is the same during the operation time.
Overview of Stereo Vision Principles
Visual Depth using Bumblebee 2 Stereo Vision System
The pre-processing block of the Triclops library prepares the raw image pair from the stereo camera for stereo processing. The stereo processing block performs the computational stereo process where the image pair correspondence is determined.
Summary
The library exploits the neighboring pixel matching results of the resulting mismatch to determine an approximation that is within a pixel fraction.
Introduction
Overview of Ground Plane Detection
Methods using V-Disparity by Labayrade et al (2002)
- Property of V-disparity Image
- Ground Profile Extraction from V-Disparity Image
- Challenges of Ground Plane Estimation in Rainforest Terrain
The ground in the image pair is represented by the slanted segment in the V disparity image. The ground plane can be determined from the V disparity image if the ground correlation line can be extracted. This characteristic can be used to determine the ground correlation line when there are multiple profiles found in the V-difference image.
Feature Extraction
Analysis on Disparity Image and V-Disparity Image of Rainforest Terrain
However, this shows that the V disparity image can be used to extract the ground plane and obstacles. The V disparity correspondence image illustrates the strong ground plane profile and clear obstacle profile. It can be observed in the V disparity image that the soil profile is quite visible.
Color Feature from Stereo Imaging
Colour Distribution of Rainforest Scene
The similarity of color made it difficult to distinguish between two different clusters locked together. It can be observed that the color distribution of tree trunk and soil are close and overlapping, while the green vegetation can be easily separated. Any brown-in-color object or terrain appearing on the ground correlation profile of V-disparity is expected to be ground level.
K-means Clustering
This index measures the sum of each sample's distances from their respective cluster means. On the third iteration, assign each pixel to one of the K clusters depending on the relationship. The choice of the number of clusters for this thesis is based on some assumptions and validated with experimental data.
K-means Clustering: Number of Cluster Determination
The image contains the scene which can be divided into two main classes, the traversable ground region and the obstacle region. However, there can be many objects of an object class appearing on the scene; therefore there can be many regions of an object class. The number of predicted regions is approximately six, so it can be seen that the highest percentage of six regions is generated when the number of clusters is set to five.
Developed Ground Plane Detection Algorithm
Each of the cluster points is mapped to the V disparity image and each cluster will generate a candidate soil correlation line. The selected ground correlation profile was then applied to the V-disparity image of the whole image to extract the ground pixels. From the ground correlation profile, the points of difference corresponding to the ground level can be mapped.
Experimental Results and Discussions
Simple Unstructured Terrain Experiment
As discussed in Section 4.3.1, for the scene where ground plane occupies the majority of the image, the ground correlation line in V-disparity image is relatively clear. The ground correlation line is quite straight and it is easier to extract since the disparity data in V-disparity image mostly represents the ground plane. It can be observed that developed module succeeded in identifying the land area in the scene.
Average Error versus Distance
Moderate Terrain Experiment
Next, the ground plane detection algorithm is tested on a typical rain forest scene of moderate complexity.
Performance Matrix of Ground Plane Detection (Simple Terrain)
The lack of green vegetation on the ground is due to the canopies preventing sunlight from reaching the ground. The stereo camera's minimum disparity setting limits the distance of the object that the stereo camera can detect. This is because the scene is covered by canopies that prevent sunlight from illuminating the scene.
Performance Matrix for Ground Plane Detection (Moderate Complexity)
Complex Terrain Experiment
Thus, the disparity data representing the ground plane in the disparity image V is minimal compared to . The plane view is small, and the obstacles are too close to the camera. However, the ground plane is not the main object in the image and there are obstacles near the camera.
Performance Matrix for Ground Plane Detection (Complex Terrain)
Summary
It was able to determine the ground plane of the scene and mapped the corresponding ground pixels to the image. Based on observation at the detected ground level, the false positive alarm usually occurs at the edge of the ground level. In rainforest terrain, the terrain can be similar to the color of the tree trunks.
Overall Performance Matrix of Ground Plane Detection
- Introduction
- Overview of Obstacle Detection
- Method by Huertas et al (2005)
- Scene Consideration
- Feature Extraction
- U-disparity Image
- Sobel Edge Detector
- Tree Trunk Detection Algorithm
- Experimental Results and Discussions
The disparity information of the tree trunks will have large continuities in the image lines. Thus, the change in disparity information at the edge of the tree trunks and the background is quite abrupt. This feature in the U-disparity image is used as one of the clues to detect tree trunks.
Performance Matrix of Tree Trunks Detection (on each sample)
The tree detection module fails to identify tree trunks when the tree trunks are too far away. A false positive is usually caused by other obstacles that are too high in height. This true positive rate is directly affected by the false negative result, as the false negative increases, the true positive rate decreases.
Performance Matrix of Tree Detection Module (Distance)
Summary
It was able to determine the tree trunks of the scene and map the corresponding tree trunk pixels to the image. Stereo correspondence cannot be found and may fail to detect distant tree trunks. However, the proposed one will be able to detect the tree trunks when it is closer to the tree trunks.
Introduction
A detailed description of the state-of-the-art together with complications in the water body detection is presented in the next section.
Overview of Water Body Detection
- Method by The Jet Propulsion Laboratory (JPL)
- Method using Polarization-Based Camera
In addition, a ground detector is designed to detect the ground plane to improve the positive detection of the water body. JPL's works provide a detailed feature of the water body and clues that can be used to detect different appearances of the water body. Water hazards can be detected by comparing the degree of polarization and the similarity of the polarization phases.
Mathematical Description for Appearance of Water Body
- Water Reflectance Model
- Partial Linear Polarization
The width of the distribution will depend on the roughness of the surface (Nayar, Fang and Boult, 1993). The fraction of incident power reflected from the air/water interface is given by Fresnel's equations for light polarized perpendicular to and parallel to the plane of incidence. The most important factors that can affect the refractive index of water are the wavelength of light entering it and its salinity.
Feature Extraction
- Water Cue from Partial Polarization Feature
- Water Cue from Stereo Disparity Feature
- Water Cue from Texture Feature
Example in figure 6.5 shows that the brightness of the two images differs especially in the water area. One of the considerations is water mass under the influence of the shadow of the surroundings. It can be observed that under the influence of the shadow, the texture of the water mass is low, since there is less.
Algorithm: Fusing Water Cue
- Sky Reflection Detection Module
- Low-Texture Detection Module
- Object Reflection and Invalid Disparity Pixel Detection Module
Note that the reflection of the sky in the water has low saturation values and high brightness values. Note that in his approach, the detection of the sky is done in the first ten lines of the image. Then the region of the water body with relative smoothness less than the set threshold will be selected. Table 6.2 shows the values of relative softness of water bodies.
Experimental Results and Discussions
- Running Water Detection
Water body detection is an application extension based on the ground plane detection discussed in Chapter 4. Once the ground plane is detected, the aircraft is searched for the presence of the water body. False negative detection is increasing after 7 meters due to the limitation of our developed ground plane detection.
ROC Analysis on Running Water Detection
Standing Water Body Detection
In the detection of standing water, the scene is more susceptible to reflection from the environment around the water body. The blue region corresponds to the water region, while the red region corresponds to the land region. By considering only the land area, the algorithm will not include the air region in the result, since air region usually looks similar to air reflection on water area.
Performanace Matrix of Standing Water Detection
Back-Lighting Error
The frame pixels lose color information of the scene when the scene is illuminated with a high intensity of light. In outdoor terrain this problem can occur in circumstances where the lighting is in front of the camera. As a result, the color distribution of the image will be very close and difficult to separate during clustering step.
Summary
If the light source is too bright and in front of the stereo camera, the pixels will be oversaturated. Thus, it can be said that the performance of the proposed algorithm is limited by the stereo camera sensor. The use of a polarizer with a stereo camera shows promising results, but has not yet been fully exploited.
Overall Performance of Water Body Detection Module
Contributions
In particular, we identified that the essential components are ground plane detection, tree trunk detection and water body detection. It is proposed to be scaled down, this work focuses on the nearby vertical tree trunks obstacles. Based on the assumption, the edges of the tree trunks are taken using the Sobel edge detector.
Conclusion Remarks
The water body detection module focuses on detecting the water area without taking into account the depth of the water body.
Future works
