*3.7. Algorithms Errors*

A ROC curve analysis was performed to graphically assess the sensitivity and specificity of the classifications carried out. From the analysis of Figure 8, it can be seen that, as the score point increases, the discriminating power also increases, which is, the curve is closer to the upper left corner and, consequently, a greater area is obtained below the ROC curve. In both classifiers, the largest value was recorded for ASTER and the lowest for MODIS, corroborating the results obtained by the OE and CE.

**Figure 8.** Receiver operating characteristic (ROC) curve graphs for (**a**) kNN and (**b**) RF.

#### **4. Discussion**

This study assesses the application of an automatic methodology for mapping burned areas in Portugal through the supervised classification algorithms kNN and RF using multispectral satellite images of different technical specifications. The integration of the use of these images increases the temporal accuracy of imaging a target that is susceptible to extreme events, which often require intense monitoring.

In this study we show, in detail, the quality, errors, and incompatibilities in the classification of a burned area at a local scale, which, in turn, can be used to explain phenomena

of non-resistance (edge effects, unexpected artifacts, or underestimation related to low intensity fires) that are often propagated or masked when applied on a continental or global scale. This is widely discussed in Randerson et al. [99] who observed an underestimation of 4–15% of the burned area missing in global products, slightly below the 30% that is normally assumed. This underestimation occurs due to the absence or small overlap in the detection of small fires (<270 ha) derived from different global burned area products. Such problems are also found in Nogueira et al. [119], Chuvieco et al. [120], and Roteta et al. [56]). Therefore, our analysis demonstrates the importance of accurate mapping of a burned area at a local scale, which still remains the most accurate base of reference data in protocols for validations of global burned area, after evaluation by photointerpretation [121,122] or in the field [61].

#### *4.1. Separability Analysis*

The errors that were found in the classification of burned areas were caused by several factors, one of which was the spectral similarity of burned areas with other surface elements, mainly darker bodies, in addition to the technical disparities of the kNN and RF classifiers. However, the spatial accuracy of the images was the most important agent in reducing the performance of the products. This behavior can be seen in the maps that are generated by MODIS sensor, due to its coarse spatial resolution.

The assessment of the ability to detect burned areas was performed using the JM separability index in the different bands (Equations (1) and (2) and Table 3) and the results of the confusion matrices represented by OE, CE, OA, DC, and AUC (Figures 7 and 8 and Table 5).

In agreement with previous studies [123–125], less separability is observed for the visible bands in all sensors in our results, mainly for the bands B1 and B2 for ASTER [126] and especially in the green range. This occurred because forest fires affect the leaf structure and photosynthetic capacity. They also decrease the green pigment of the leaf (chlorophyll) and increase the brown-yellow pigment (carotenoids, pheophytin, and xanthophyll) [124]. In the visible-NIR transition bands, there was high separability corroborating the studies conducted by Fernández-Manso et al. [127]. The authors proved that recent fires in healthy vegetation show a characteristic increase in the reflectance from red to NIR, associated with variations in chlorophyll content.

The analysis was able to show good discrimination of the burned areas. This approach improved the spatial homogeneity of the affected areas (even if random) of the classification thresholds, as shown by the high values of AUC (>0.88), reducing the dependence on having information on land cover, usually used in automatic burned area algorithms. Although it is important to emphasize that the lack of information on land use for adapting the algorithms behavior can imply the recurrence of systematic errors, increasing the uncertainty of the final burned area classification, as shown in Figure 9. As already mentioned, we note the presence of features that presented spectral behavior that was similar to the burned area (for example, low reflectance values in the NIR), which can be caused by topography shadows and changes in land cover not associated with fires, such as very humid soils. Therefore, it is recommended to take special care in regions where these characteristics and events occur close to the area that is affected by the fire, in addition to controlling the photointerpretation with the size of the samples of interest, especially in applications with sensors of different spatial and spectral resolutions [128]. Thus, as a future study in the study area, assessing the separability for different classes of land use and the influence of sample size may be a good alternative.

**Figure 9.** Visual analysis of the errors presented in the different land cover in the study area: highways (**a**,**b**,**k**,**l**), pasture and agriculture (**g**,**h**), soil degradation (**i**,**j**), and water bodies (**c**–**f**).