CHAPTER 3: ENHANCED URBAN CLASSIFICATION USING MULTI-SPECTRAL
3.3 Results
47
coverage per class (area), producer’s accuracy, user’s accuracy, overall accuracy and McNemar’s tests.
3.2.7 Significance of the differences in accuracy between the classification methods The significance of the differences in accuracy between the methods was tested based on the confusions tables, using the McNemar’s test. The McNemar’s test was used to compare each of the methods with the traditional method which uses only the reflective bands for classification to assess whether the other methods significantly differed in terms of accuracy.
The McNemar’s test is a better statistic for comparing accuracies of classification methods than the Kappa index and it is simple to compute (Petropoulos et al., 2012; Adelabu, et al., 2013).
The Kappa chi-squared requires that independent data are used to assess accuracies, but in this study, the same points are used in all methods thus the McNemar’s test was more appropriate as it is also more precise and sensitive (Manandhar et al., 2009).
Table 3.4: Comparison of two methods using the McNemar’s test Method 2
Correctly classified Wrongly classified
Method 1 Correctly classified f11 f12
Wrongly classified f21 f22
McNemar’s Chi squared statistic was computed using Equation 3.1 as:
𝒁𝟐 =(𝒇𝟏𝟐−𝒇𝟐𝟏)𝟐
𝒇𝟏𝟐+𝒇𝟐𝟏 Equation 3.1
where f12 denotes the number of cases that are wrongly classified by classifier 1 but correctly classified by classifier 2 (Table 3.4) and f21 denotes the number of cases that are correctly classified by classifier 1 and wrongly classified by classifier 2 (Petropoulos, et al., 2012). The difference in accuracies were tested at 95% significant level and deemed different if Z > 1.96.
By comparing error matrix of each analysis with that of Analysis I, we obtained total number of cases correctly classified by the analysis and wrongly classified by Analysis I (f12) and vice versa (f21). The values of f12 and f21 thus obtained were used in equation one to test whether the accuracy of each analysis was significantly different with that of analysis I at 95% confidence intervals.
48
produced an overall accuracy of 82.65% and Kappa index was 0.81. Further, producer accuracies greater than 75.0% for all the classes were obtained based on the use of reflective bands as independent datasets. Comparatively, the same dataset produced slightly lower user accuracy (65.7%) for the high density residential class and above 75.0% for the remaining classes. For example, densely built, forested and water classes, had significantly higher user and producer accuracies above 80.0%.
3.3.2. Analysis II: Classification results using TIRS spectral bands
Table 3.6 illustrates classification results (i.e. overall, producer and user accuracies) obtained from using Landsat 8 thermal bands. The use of thermal bands as standalone datasets overall yielded lower user and producer accuracies for almost all the classes considered under study except for the water class which had a producer accuracy of 87.5% and 86.5% user accuracy.
For example, for grasslands, forested and high density residential classes, user accuracies were 33.3%, 35.6% and 45.5% respectively. Similarly, the standalone use of Landsat 8 thermal bands yielded low producer accuracies of 28.0% and 44.6% for forested and grassland classes respectively. The study produced 53.40% and 0.46 kappa index value as overall accuracy, significantly lower (i.e. McNemar’s score was 9.98 at 95% confidence interval) when compared to the use of the traditional visible or reflective bands of the Landsat 8 OLI.
Compared with the other methods, Analysis II produced areas per LULC class which were mostly very different from those obtained with the other methods (Table 3.5). For example, the development class had an area of 429.69km2 using Analysis II while the area ranged between 287 and 300km2 with the six other analysis.
3.3.3. Analysis III: Classification results using OLI & TIRS spectral bands
Table 3.6 demonstrates the urban landscape classification results based on the integration of thermal and reflective bands of the Landsat 8 sensor. Based on this analysis, an overall accuracy (84.03% and kappa index was 0.81) comparatively similar to the one obtained in the Analysis I using reflective bands as a standalone dataset. For example, high producer accuracies, mostly greater than 80%, were obtained for most of the classes i.e. water, forested and densely built classes except for development class which had producer accuracy of 72.6% (Table 3.6). User accuracies were also mostly above 80% except for development and grassland classes which had 62.8% and 76.7%, respectively.
49
3.3.4. Analysis IV: Classification results using spectral vegetation indices
The urban landscapes classification results obtained using Landsat 8 derived vegetation indices are shown in Table 3.6. Comparatively, the results indicate that the use of Landsat 8 derived vegetation indices produced slightly lower classification results (i.e. overall, user and producer accuracies), when compared to the use of traditional reflective bands (detail see Analysis I).
For instance, user accuracies greater than 75% obtained for the majority of the classes, except for high density residential and grasslands classes where the user accuracies of 70.1% and 72.6% were respectively observed. Similarly, good producer’s accuracy results (i.e. above 75%) for all the classes considered in this study were observed from the use of vegetation indices as standalone datasets. Furthermore, high overall accuracy (81.96%) and Kappa index (0.79) comparable to those obtained in Analysis I were obtained (McNemar’s score was 6.93 at 95% confidence interval).
3.3.5. Analysis V: Classification results using TIRS spectral bands and VIs
Table 3.6 provides a summary of urban landscape classification results obtained based on the integration of Landsat 8 derived vegetation indices and thermal bands. The integration of Landsat 8 derived vegetation indices and thermal bands overall produced significantly higher classification accuracies (i.e. overall, user and producer accuracies). For example, an overall accuracy 82.97% and a kappa index of 0.82 was slightly higher than the result obtained in Analysis I based on the use of the traditional reflective bands as standalone dataset (McNemar’s Z score of 20.70 at 95% confidence interval). The results also indicate high producer accuracies i.e. above 80% for almost all he classes except high density residential and development classes which had producer accuracies of 78.7% and 72.3%, respectively. Only the high density residential class had user accuracy below 75%.
50
Figure 3.2: Urban landscapes lands cover classification results for obtained based on the classification models derived from analysis III and VII respectively.
51 Table 3.5: Areas per class obtained in the 7 analysis tested in this study
Area covered by class (km2)
Analysis I Analysis II Analysis III Analysis IV Analysis V Analysis VI Analysis VII
Densely Built 54.63 0.25 61.94 64.82 67.91 63.85 65.26
Low-medium residential 174.05 154.16 154.23 170.78 164.33 167.69 162.97
High density residential 139.96 126.05 156.77 143.48 142.79 130.81 133.68
Forested 129.80 31.78 126.39 124.04 121.17 126.39 126.75
Development 287.20 429.69 288.74 295.88 299.00 299.57 299.87
Grassland 53.08 107.79 54.84 49.71 50.71 55.89 55.47
Water/wetlands 17.81 5.80 12.61 6.82 9.62 11.71 11.52
McNemar’s Z score - 9.98 3.47 6.93 20.70 10.00 9.00
Table 3.6: Accuracies obtained and used to assess the impact of the inclusion of thermal band and vegetation indices on urban mapping accuracy (UA=User’s accuracy, PA=Producer’s accuracy and OA is the Overall Accuracy of the classification)
Analysis I Analysis II Analysis III Analysis IV Analysis V Analysis VI Analysis VII
PA UA PA UA PA UA PA UA PA UA PA UA PA UA
DB 84.8 87.1 0.0 0.0 88.8 88.7 80.0 77.6 83.0 79.4 88.1 87.2 90.9 88.8
FR 90.1 86.3 28.0 35.6 86.4 88.2 87.6 89.2 90.0 89.7 96.9 89.1 96.9 89.0 WT 89.2 90.6 87.5 86.5 82.1 98.6 89.4 91.2 96.6 90.2 92.7 98.8 97.0 98.9 HDR 79.2 65.7 61.7 45.5 80.4 62.8 78.7 70.1 78.7 71.7 79.4 75.7 79.8 75.0 Dv 75.9 84.4 71.3 54.9 72.6 82.7 76.2 86.7 72.3 86.4 79.3 86.4 79.1 86.4 LMR 84.5 84.6 66.6 58.7 81.4 85.4 84.0 81.2 83.3 82.8 87.2 86.0 86.4 85.9 GR 81.1 77.4 44.6 33.3 81.8 76.7 81.1 72.6 82.6 80.8 83.9 76.9 83.3 76.8
OA 82.65 53.40 84.03 81.96 82.97 85.49 89.33
Kappa 0.81 0.46 0.81 0.79 0.82 0.84 0.86
52
3.3.6. Analysis VI: Classification results using OLI spectral bands and VIs
The use of OLI spectral bands and the derived vegetation indices yielded high and comparative similar results with those obtained in Analysis I (i.e. use of reflective bands as standalone dataset) and Analysis V (i.e. use of Landsat 8 thermal bands and the derived vegetation indices).
For example, high overall user (85.49%) and producer accuracies greater than 78% were obtained for all the classes considered under this study. Moreover, an overall accuracy and kappa index of 0.84 was attained (Table 3.6 and Figure 3.2). Producer and user accuracies were greater than 75% for all the LULC classes. A comparison of the results obtained from this analysis (i.e. OLI spectral bands and the derived vegetation indices) and those obtained from Analysis I (i.e. use of the traditional reflective bands as standalone datasets) show significant differences with the McNemar’s Z score of 10 at 95% confidence interval.
3.3.7. Analysis VII: Classification results using OLI, TIRS spectral bands and VIs Table 3.6 shows the urban landscape classification results obtained from running the model based on the integration of Landsat 8 derived OLI reflective bands, TIRS spectral bands and computed vegetation indices. The classification results demonstrate great improvement on the overall, user and producer accuracies for all the classes considered under this study. For example, significantly high user and producer accuracies, greater than 85% for low-medium density residential, water, forested and densely built classes were obtained. The results showed high overall accuracy of 89.33% and a kappa index of 0.86 (Table 3.6). Furthermore, when compared to Analyses 1, II, III, IV, V and VI, the urban landscape classification results obtained from the integration of Landsat 8 derived OLI reflective bands, TIRS spectral bands and computed vegetation indices (i.e. Analysis VII) yielded higher accuracies with McNemar’s Z score of 9 at 95% confidence interval. Overall, these results demonstrate that the integration of TIRS spectral bands from the Landsat 8 sensor with the sensor’s derived reflective bands and computed vegetation indices, improves the classification accuracy of urban landscapes compared to the use of these datasets as standalone datasets.