of gradient boosting and SHAP applied to a LR model, or [238], where [238]
compare the use of I*G and DeepLift in several layers.

As can be derived from the results of this review, the high and only usage of
the SHAP model-agnostic method, or similar, suggests that this method is used
regardless of the problem. We want to draw attention to this trend since there
is _no single method that fits every problem_ [320]. The practitioner should
keep in mind that each method makes certain assumptions (e.g., SHAP assumes
feature independence that simplifies the problem’s definition). These
assumptions can often be violated, for example, with time series data, and
harm the interpretation results. Therefore, the assumptions of the specific
method need to be considered and aligned with the data structure, task, and
underlying model. Additionally, each model-agnostic method represents one
approach or paradigm to generate explanations, regardless of the trained model
and input dataset. For specific tasks, specialized methods exist that fit
their need and a method should not only be used because it is popular and
easily accessible [322]. Furthermore, the choice of the _xAI method and its
evaluation should be discussed_ in each paper. This should be as much a part
of the scientific dialogue as the model selection and evaluation. Numerous
papers in this review did not mention their arguments for selecting their xAI
methods. The question arises if the authors have considered the specificities
of the used methods. Many of the existing xAI methods, like SHAP, IG, or LRP,
are only an approximation of the underlying model which comes with simplifying
assumptions. We encourage the authors to address this issue to lay the basis
for the understanding and scientific evaluation of the chosen approach. Good
perspectives in this respect are provided by [242] in [242].

Some of the modified Backpropagation (BP) methods, like Guided BP, Deep Taylor
Decomposition (DTD), and variants of LRP, have been proven to be _unfaithful
to the model_ [99, 323]. [99] [99] demonstrate that perturbing the weights of
the neural network does not significantly change the resulting explanations of
the network. Another work builds a foundational theoretical framework, showing
that many methods fail to provide class-sensitive explanations and highlight
only low-level features [323]. Nevertheless, these methods are used in many
works, as seen in Section V-A. If such a method is used, it is advisable to
use one of the sanity checks mentioned in Section IV-C to verify the strategy
applied. But, as pointed out in several comparisons of different methods [324,
325, 208], there exists no single xAI technique that satisfies all evaluation
metrics and is an obvious choice for all EO problems. Rather, the methods
should be chosen for the specific task and goal, taking into account
specifications such as the need for global or local explanations, their model-
agnostic or model-specific nature, their input type, the computational time
required for the generation of explanations, the quality or detail of these
explanations, or the suitability for the end-user, among others. A short
discussion on how to evaluate these methods can be found in the following
section. When using methods where the input is altered, like IG or SHAP, the
baseline is the neutral input, which should capture the absence of any
meaningful feature, and all outcomes are compared to the output for this
input. Hence, the choice of the baseline is crucial and some methods have a
built-in baseline by construction. This is especially important when different
methods need to be compared [326, 327].

Often, the practitioner is interested in the causal relationships and the
explanation results of black-box models might lead to _unjustified causal
claims_ [320]. xAI methods do not provide the cause. Even when the explanation
supports the hypothesis from the research objective, it does not mean that
this hypothesis has been proven. Standard ML approaches approximate
correlations from data; therefore, the model cannot assess the causal
structure of the data. However, it can help experts find unknown confounders
and correlations by giving them a tool to investigate the learned
correlations. The user must be very careful when making such assumptions to
discover underlying structures and be aware that observational data often
lacks common confounders, has strong feature correlations and its causal
structure is usually unknown. Causal inference is a complementary research
field to xAI and is not covered in this review. Nevertheless, we identified
several works discussing the causality problem in xAI [214, 136, 241, 279].

####  VI-A2 Evaluation of xAI

As mentioned in Section V-B2, anecdotal evidence is frequently reported as the
evaluation of the applied approaches. Cherry-picking and qualitative
evaluation of explanations represent a challenge to humans. Because human
perception is mainly visual, humans are biased toward certain types of xAI
explanations. Besides others, humans introduce cognitive distortion by drawing
more attention to negative examples and looking for simple but complete
explanations [328]. That is accompanied by other human biases, like the
confirmation bias, which is well-known in psychology and favors explanations
fitting the expectations, while contrary explanations are ignored [329]. These
self-introduced biases can lead to wrong reasoning about the explanations and
promoting a particular type of visualization. Hence, it is hard to quantify
the results objectively through anecdotal evaluation. Therefore, a
quantitative evaluation of xAI methods is recommended. In addition,
qualitative evaluation can be done, but it should not be the only type of
evaluation. When users are expected to interact with explanations, attention
should be paid to creating understandable and appropriate explanations for the
specific end user, their requirements, and their level of knowledge [330].
Then, keeping in mind these biases is essential. Nevertheless, cherry-picking
tends to be insufficient for evaluating all types of explanations and, as the
only assessment criteria, it does not lead to a trustworthy evaluation of xAI.

Beyond evaluation based on anecdotal evidence, there also exist quantitative
evaluation metrics (see Section IV-C) to assess the explanation methods.
However, only a few works listed in Section V-B2 evaluate their methods
against these metrics. In contrast to these studies, it has been observed that
a more detailed quantitative evaluation of xAI is conducted in climate
science, given climate models. For instance, [325] [325] evaluate seven
Backpropagation methods, among them three CAM-based methods. They use a
climate model to get reliable ground truth data for their explanations and
compare the xAI methods with functional explanation evaluation by computing
the following metrics: robustness, faithfulness, randomization, complexity,
and localization. For their classification task, LRP and I*G are the best-
performing methods for the evaluated metrics. Some of these backpropagation
methods are also compared for their fidelity in [324]. First, the methods are
evaluated on a synthetically created dataset. Then, the results are related to
those on a climate simulation classification dataset. While Gradient,
SmoothGrad, I*G, IG, and $\text{LRP}_{z}$ are very noisy and may not provide
insights into the model, $\text{LRP}_{comp}$ and SHAP seem to highlight the
attributions in a more meaningful way. IG and SHAP are also evaluated with
different baselines in [327]. The results confirm that the baseline needs to
be chosen carefully because a wrong baseline can lead to different
interpretations, both in attribution magnitude and highlighted feature. Also,
the authors suggest that all methods should be evaluated on the same baseline
for comparison purposes.

There is no standard way to evaluate xAI until now, and discussions on how to
properly do this are still ongoing. Nonetheless, different toolboxes have been
developed for comparing xAI methods using quantitative metrics [101, 331,
332]. Last but not least, only one survey [145] evaluates the usefulness of
the explanations to experts. However, with the emphasis on human-centered AI
in the current research landscape [333], user studies are becoming significant
to quantify the benefit of the explanations and the understanding of the end-
users. Because they have been largely unexplored in the context of xAI in RS,
they pose a promising research direction.

####  VI-A3 xAI benchmark datasets

Real-world datasets lack a controlled ground truth for evaluating xAI methods
and a regulated environment is needed to lay the foundation for fully
transparent datasets. Similar to [334], there are efforts to create synthetic
datasets for EO tasks. [335] generated a fully synthetic dataset where they
leverage local piece-wise linear functions to create a non-linear response to
the input drawn from a Gaussian distribution [335]. This method allows for
creating a regional climate prediction task from Sea Surface Temperature
(SST). They show that a simple FNN can approximate this function and evaluate
different post-hoc xAI methods. To get reliable ground truth data, it is also
possible to use a simulation. For example, climate models [336] can be
leveraged. Other approaches to generate fully-synthetic RS images include
[337, 338, 339]. While [337] simulate different RS sensors, atmospheres, and
scenes, including different terrains, materials, and weather conditions [337],
[338] add different compositions of the vehicles [338] and [339] show that the
motions of vehicles can be included in the simulation [339]. A framework for
generating synthetic EO datasets is presented in [340], demonstrating the
efficiency of these datasets for ML on off-shore wind farm detection. The
framework aims to extract expert knowledge about the objects to be modeled in
a machine-readable format (e.g., structure, relationships, etc.), which can be
combined to create new datasets. However, there is no work comparing xAI
methods on these synthetic RS datasets.

###  VI-B RQ5: Challenges, limitations, and future directions

We identified several challenges that emerge from xAI, RS, or the combination
of them. Figure LABEL:fig:chall_mindmap gives an overview of these challenges,
which are categorized by color according to the area from which they emerge,
i.e., those emerging from RS and xAI are in green and blue, respectively.

RQ5: Challenges and Limitations of xAI in RS Challenges from RS properties
Scale Topology Temporal dependencies Restricted availability of labeled data
Combination of xAI with related fields Physics-based ML Causal inference
Uncertainty Lack of standardized and objective evaluation of xAI Towards
interpretable DNNs Concept discovery Intrinsic interpretable DNNs Graph Neural
Networks (GNN)

####  VI-B1 Combination of xAI with related fields

##### Physics-aware ML

Physics-aware ML and xAI share the objective of enhancing model reliability,
trustworthiness, and transparency. Both rely on domain expertise: xAI enables
experts to discover new insights into physical processes. This is illustrated
by studies such as [181], which explores runoff and evapotranspiration
parameterization, and [216], where the errors of an Earth system model are
predicted to hypothesize about wrong model assumptions using xAI. Conversely,
integrating expert knowledge from physics helps to understand the models
because the incorporated prior knowledge is human-understandable. The
combination of both fields can yield physically sound and interpretable
models. We have identified several approaches relying on physics-aware
features applied for knowledge extraction from SAR data. For instance, [341]
introduce a CNN operating in the complex domain which aims to predict physical
scattering properties from Polarimetric SAR (PolSAR) images [341]. These
properties are derived with the $H-A-\alpha$ target decomposition method and
describe various urban, vegetation, and ocean surfaces. [270] inject similar
physics-aware concepts into a CNN model for sea-ice classification and
conducts a xAI analysis to show that the fusion provides a better separation
of the different classes based on their physical properties [270]. Given the
vehicle classification task, [297] extract vehicle parts with the attribute
scattering center (ASC) model and construct images for the extracted parts
based on their scattering centers selected with k-means clustering [297]. The
image parts are used as filters for the convolution of the original image.
With occlusion analysis, the authors demonstrate that incorporating the
vehicle parts in the model inputs helps to improve model robustness, enabling
the practitioners to obtain intuitive explanations and validate the model
workings with domain knowledge. In more recent work, [290] also proposes an
intrinsically interpretable CNN model that first processes the vehicle parts
and the original image with a CNN encoder [290]. Next, an attention module
uses the encoder output given the original image as queries, while the keys
and the values are derived from the encoder output for the target parts.
Finally, a convolution operation is applied to factor the contribution of the
vehicle parts into the class logits, thus explaining the model prediction
based on the different parts. All these works support that physics-aware ML
and xAI could complement each other. Further design of these approaches and
their extension to other types of RS data, such as optical or hyperspectral
images, is a promising research direction for model improvement and may also
constitute an important step towards consolidating the usage of explainable
architectures.

##### Uncertainty

The uncertainty intervals give insight into the generalization and training
process of the model, which is especially useful when dealing with
extrapolation and anomaly detection. While explanations can give more detailed
information about the models’ internal representations, they lack an important
metric: model confidence or explanation reliability. Explanations which have a
high variability or are inaccurate may lead to misinterpretations. The
combination of uncertainty and xAI can enhance the information we can get from
only one approach. For instance, [342] [342] and [343] [343] propose methods
to provide uncertainty intervals around the explanations. RS has not been
excluded from the convergence of these two disciplines [344, 345]. Through the
adaptation of a perturbation method for object detection (D-RISE) and its
combination with Deep Gaussian models, [344] get attribution maps for model
uncertainty [344]. They show the efficiency of their approach on SAR object
detection, where trustworthy predictions are especially needed since SAR
images are hard to interpret for humans. An uncertainty aware, interpretable-
by-design model frequently applied here is GPs [206, 253, 152, 151]. Further,
[206] analytically derive the input features’ sensitivity to the variance
estimate of a GP model that can be later used for uncertainty evaluation and
selection of relevant features [206].

Hence, complementing xAI methods with uncertainty constitutes an interesting
research direction to ensure more reliable explanations and better
interpretation results.

##### Causal inference

Understanding causality in xAI is fundamental for uncovering the cause-and-
effect relationships within the model’s decision-making process. Techniques
like causal inference, counterfactual reasoning, and causal graphical models
are employed to trace the causal relationships between input features and
model predictions [346]. These methods aim to not just highlight correlations
but elucidate the direct causal links, enabling better comprehension of how
the AI system arrives at its decisions. Moreover, the integration of causal
reasoning into interpretable ML models, like causal Bayesian networks or
causal decision trees, has shown promise in elucidating causal links between
input features and predictions [347]. In RS, causality involves understanding
the cause-and-effect dynamics between environmental processes and the
measurements acquired by RS instruments, with the aim of monitoring natural
resources and ecosystems. Approaches such as structural equation modeling,
directed acyclic graphs, and Granger causality have been utilized to untangle
causal relationships within RS datasets [24]. These methods aim to identify
causal pathways between environmental variables, allowing scientists to
comprehend how changes in one variable may cause alterations in others.
However, causality for explaining trained models in RS is yet to be
extensively explored, with many opportunities ahead in environmental studies,
land-use planning, disaster management, and climate change research.

####  VI-B2 Challenges from RS properties

xAI in CV applications does not account for RS image properties such as the
existence of different sources, scales, geographic relationships, and temporal
dependencies. The _scale_ in RS describes the resolution and scope of input
data [8]. While the scope determines the geographic extent, the resolution
specifies to which degree information can be captured, defining the shape,
granularity, and boundaries of the objects, among others. Additionally, some
concepts or objects, like landcover or mountains, have no distinct boundaries
or areas. Instead, changes are more continuous and have irregular shapes.
Hence, spatial resolution automatically has semantic implications since the
presence or absence of information in the data influences the ability to
distinguish and interpret specific features or objects within the scene. This
remains a challenge for RS and xAI [16]: most methods have problems with the
high granularity of features, highlight an increased spatial extent, or are
very noisy. New methods are constantly being developed to overcome these
disadvantages. For instance, the well-known Grad-CAM has this problem, and
many studies want to enhance its capabilities. Recent works leverage the
lower-level feature representations to create much more fine-grained saliency
maps [196, 192]. [202] addresses the challenge of scale by decoupling the xAI
method from spatial scale, yielding in a permutation method which attributes
across various distances [202]. The model can also already tackle the scaling
problem: a scale-invariant autoencoder has been recently proposed in [348].
The scale of RS observations also determines the spectral resolution. Every
sensor captures reflectance data across different wavelengths, extracting
different ground-level information. Until now, a comprehensive xAI evaluation
for multispectral and hyperspectral RS images is missing, taking into account
the spectral property that distinguishes them from conventional images.
Additionally, the current literature does not consider the varying spectra and
spectral resolutions between the platforms. [197] use a 3D visualization
technique originating from medical imaging, together with a CAM variant
adapted for hyperspectral images, to show the importance of different spectral
bands per pixel [197]. This provides insights both into the spatial and
spectral attribution, representing an advantage over the current CAM variants.
Current CAM methods only visualize the spatial attribution despite the
importance of the different spectral bands for specific objects and
environments.

Another challenge is the _topology_ of the data [16]. Geographic relations can
be a hidden confounder not captured in the data. For example, social
structures are strongly tied to the location or have symbiotic relationships
that exist in and between the biosphere, lithosphere, cryosphere, and
hydrosphere. That makes analyzing and using xAI in RS difficult. Therefore, a
proper set of training features is very important. One approach to tackling
this challenge focuses on location-awareness [200], where the prototypes are
specific to locations in the input. This assumes a static location in the
input image and considers the relative location from the input. While this
approach works for global images or images for the same location, it does not
take into account changing locations. For example, there is a trade-off
between the complexity or size of a model and the resolution of the RS images.
Working with static locations and high-resolution images would require huge
resources for the model. Moreover, the flexibility of using varying locations
in the input and being location-aware could not be achieved. A different
strategy that facilitates the learning of spatial relations based on a U-Net
model is presented in [224] for monitoring the exposure to surface ozone. A
grid cell is used as input to the U-Net model and consists of high-level
features for a geographical area. Hence, this setup allows the usage of the IG
attribution method to uncover the contributions of the neighboring areas for
surface ozone pollution. Currently, there is an increasing interest in the
integration of topological data analysis (TDA) [349] with ML [350, 351]. [352]
attempts to discover the topology of the data in an unsupervised way by
mapping each pixel through a lens function, dividing it into subsets, and
clustering within these subsets [352]. Edges are drawn between the subsets if
they share at least one pixel. Different groups are generated according to the
number of pixels in each subset and the connectivity, which classify the
pixels into different categories. This approach is also applied in [353] to
classify and hierarchically visualize the data in a dendrogram.

Furthermore, RS is often composed of _sequential data_. Since most xAI methods
are developed for natural images, they do not account for temporal
dependencies [354]. However, besides the standard methods, several
possibilities designed for sequential data exist in the literature [355], one
of them is the attention mechanism which we found to be frequently utilized
for such problems The only other approach identified in this review that
explicitly tackles the problem of time dependencies is presented by [228].
They learn regression coefficients per feature, time point, and location.
Additionally, the spatiotemporal dependencies are disentangled with a random
effect component where the latent variables follow a temporal Markovian
process. This approach is evaluated for air quality estimation where the
regression coefficients are used to reveal the temporal importance of various
meteorological and landcover features. Besides these approaches, the other
works typically rely on approaches like backpropagation that do not explicitly
consider temporal relations. For instance, [256] aggregate saliency maps for
individual time steps into cyclical saliency maps to identify sea surface
temperature areas relevant for river flow prediction [256]. These are computed
as an average of individual saliency maps over periods of months, seasons, or
calendar years. The authors argue that such aggregation is meaningful in the
climate context as it allows summarizing the spatial importance across
different time frames while improving the robustness towards gradient
fluctuation and noise. A similar strategy is also followed in [149], where a
CNN is used on the temporal dimension and CAM is applied for attribution
purposes.

####  VI-B3 Towards interpretable Deep Neural Networks

The initial research for interpreting NNs focused on developing saliency map
methods. In Section V-A, we have seen that this trend is also prominent in RS,
as methods like Grad-CAM are among the most utilized xAI approaches for
explaining DL models. The saliency maps highlight the relevant features
without providing additional insights into how these features are used by the
model [356]. Moreover, these highlighted features usually do not correspond to
high-level concepts that humans can easily interpret [61]. In recent years,
these limitations have been addressed with alternative explanation paradigms
such as concept-based explanations and intrinsically interpretable DNNs [357].

The concept-based explanations described in Section IV-A3 enable global
interpretation of the model workings in terms of intuitive high-level concepts
understandable to humans. Amid the growing number of these approaches in CV
[61, 62, 358, 356, 359], we did not identify any works in RS. The lack of
usage of these approaches originates from the properties of RS data described
in Section LABEL:sec:rs_properties_challenge. First, the concepts defined in
CV range from simple primitives like colors and textures to more complex
visual patterns like object parts [61, 62]. While simple visual primitives can
also be used as concepts in RS, defining complex visual patterns requires
modeling the topological relations inherent in RS data. In addition, concepts
in CV are typically extracted from an external probe dataset and a recent
study [360] recommends using a probe dataset with a similar distribution as
the dataset used for the learning task. Hence, to ensure faithful concept
explanations in RS, the probe dataset should conform to the topological
relations of the dataset on which the ML model was trained.

Even though the concept-based explanations can offer more intuitive
explanations than the saliency maps, they are still post-hoc methods, and as
already described in Section IV-C, they are not always faithful to the model
workings. Moreover, these methods often approximate the underlying model
behavior and do not make the model more transparent. While mainly simple and
easily understandable ML models are meant with interpretable models, we want
to emphasize the efforts of trying to make DNNs inherently interpretable.
Current xAI literature is going in that direction, either by introducing
layers in the neural network, like the attention mechanism that can offer
insights into the relevant features, or by designing intrinsically
interpretable DNNs.

While transformers were developed for one-dimensional sequential inputs, their
CV counterpart, ViTs [92], adapt their structure for the two dimensions of an
image by splitting the image into smaller parts. Spatio-temporal inputs are
often encountered in the EO domain. The Earthformer [361] tackles
spatiotemporal inputs through cuboid attention, i.e., by applying self-
attention on input tensors decomposed into local cuboids, and then attending
to global cuboids which summarize the overall status of the system. Another
ViT adaption to the RS domain is shown in [362], where a temporal transformer
encoder followed by a spatial one is used to process all patches composing the
time series. Additional ideas for ViTs relying on related concepts and
suitable for RS space-time data can be found in [363, 364, 365, 366]. Further
exploration of the attention mechanisms in these new transformer architectures
is still missing in the EO literature and could be a promising direction for
latent space analysis-based xAI. It should be noted that attention mechanisms
are not solely used in transformers but also in CNNs and RNNs [209, 267, 268,
260, 240, 191, 290, 121].

Other intrinsically interpretable DNN approaches for generating ante-hoc
explanations recently explored in RS are prototype networks, BagNets, and
Graph Neural Networks. As described in Section IV-A3, prototype neural
networks enforce a reasoning process that classifies input examples based on
their similarity to prototypical parts of images of a given class. In this
review, we have identified the following approaches [200, 168] that have
already been described in Sections V-A2 and V-A1, respectively. A recent work
introduces a more interpretable CNN model, the BagNet [87], which, in contrast
to the traditional CNN, is only applied to a small and local receptive field.
Therefore, the activations for each class in each receptive field are combined
to provide a prediction by addition at the image level. The magnitude of the
receptive fields’ activations corresponds to the importance of the features,
and an attribution map can be directly compiled from the activations. Examples
of this methodology applied to SAR vehicle detection are in [287, 292]. A
similar approach integrates NNs in GAMs for landslide susceptibility
prediction, where the NNs approximates the GAM’s spline functions. The model
enhances its interpretability but leverages the capabilities of a NN [162].
Another study equipped a DL model with an interpretable-by-design component, a
linear regression model is jointly training onto the same targets via
backpropagation [122].

In contrast to the NN approaches, graphs provide an intuitive representation
of many real-world problems regarding objects and their relations. GNNs
emerged as a popular paradigm for learning high-level object representation by
message-passing from the object neighbors [367]. This reasoning process
enables GNNs to represent complex patterns of relationships between objects
and generalizes their applicability to arbitrary geometric structures as
opposed to CNN, which operate on two-dimensional grids (images) and one-
dimensional sequences (text) [368]. Moreover, Graph Convolutional Networkss
(GCNs) [369], which perform message-passing with the convolution operation,
enable efficient semi-supervised learning in scenarios where not all objects
in the graph are labeled. These properties have fostered the usage of GNNs in
RS, especially in hyperspectral image classification tasks. Concretely, to
overcome the limitation of the popular CNN approaches to capture the
topological relations and the irregular object shapes inherent to
hyperspectral data, various GCN approaches have been proposed [370, 371, 372],
which demonstrate the potential of using GNNs for improving hyperspectral
image classification results. Recent works have also tackled the scarcity of
labels challenge with approaches based on GCN, such as [373, 374]. Although
GNNs produce an intuitive representation of the problem, they are still
considered black-box models [375]. A promising research direction is adopting
the novel xAI methods for interpreting GNNs [376] to the approaches used in
RS.

####  VI-B4 Restricted availability of labeled data

The high number of active airborne platforms like Sentinel-2 generates vast
amounts of data. However, only a small portion of this data can be labeled.
The lack of labels represents a challenge in the typical knowledge extraction
pipeline in RS, which is based on training supervised ML algorithms on a
labeled dataset. Hence, one recently popular approach to tackle this issue is
to apply the _Self-Supervised Learning (SSL)_ paradigm. SSL allows learning
latent representations from the unlabeled data. These are useful in
efficiently solving the downstream learning task for which only limited labels
are available. SSL is achieved by optimizing a NN to solve a pretext task with
pseudo-labels [377]. We have identified a promising research direction for
improving model interpretability: combining the traditional ML modeling
approaches with the recent DL approaches. A pretext task for the problem of
SAR sea ice classification is proposed in [270], where the pseudo-labels are
derived from a mixture of LDA topics. Topics are based on physical scattering
properties extracted from unlabeled SAR images. Thus, the model is able to
provide an interpretable representation of the sea-ice types in terms of their
physical properties (e.g., water bodies and sea floating ice are represented
with a similar set of properties that match the actual semantic definition of
these classes). SSL pretraining also preserves the physics consistency of the
features throughout the NN layers. As a result, the different-looking images
of the same class are positioned close in the latent space, leading to
improved classification results compared to a supervised CNN. [271] leverage
contrastive learning to create prototypes for landcover classification [271].
Their online learning approach fuses the images with their enhanced
counterparts at different resolutions. After a ResNet encodes them into a
feature space, they get mapped into a unit sphere where the prototypes are
clustered. Then, the resulting codes of the prototypes corresponding to the
differently enhanced images are swapped to compute the loss. Another
unsupervised prototype approach is provided in [150].

####  VI-B5 Lack of standardized and objective evaluation of xAI

As mentioned in Section LABEL:sec:rq4_recommended_partices, the qualification
of the interpretability is essential for xAI approaches [84]. Contrary to
this, there is no definition of a sufficient explanation or interpretation.
Furthermore, no standardized and objective evaluation for xAI methods has been
established, posing a challenge for potential partitioners outside of the xAI
domain. Although metrics and tools for the evaluation exist, incorporating xAI
into a ML pipeline would mean acquiring the domain knowledge from xAI to
provide a good evaluation for the methods used by a RS partitioner. An
alternative approach to the evaluation metrics is the conduction of user
studies. These studies evaluate how useful the interpretation appears to the
participants. However, these user studies can introduce subjectivity because
preferences for interpretability can vary between different groups. Hence,
conducting a well-designed user study involves a lot of effort, costs, and
domain knowledge, which again would need to be acquired by the partitioner.
Even though frameworks for xAI have been developed recently [378, 379], they
are tailored towards a specific type of method or problem [378]. Others do
focus more on the life-cycle of the systems than the evaluation [379]. In
summary, the evaluation of xAI lacks a standardized methodology, potentially
limiting non-experts applying the methods. This circumstance might have
contributed to the large number of anecdotal evidence we encountered in this
review.

##  VII Conclusion

This paper provides a detailed overview of the state-of-the-art of xAI in RS
by conducting a systematic review of the existing work in the field. First, we
collected a large set of publications by executing extensive search queries in
the established literature databases for the last six years. Subsequently, we
introduced a categorization for the existing xAI methods to structure these
works. Our analysis reveals that a substantial amount of work is motivated by
the assessment of the trustworthiness of the ML models for traditional EO
tasks like landcover mapping and agricultural monitoring. More recently, xAI
has been increasingly utilized for the discovery of scientific insights for
critical EO problems related to climate change, extreme events, or
urbanization. Although dominantly established xAI methods like SHAP or Grad-
CAM are frequently used, we observe the increased development of adapted xAI
methods to capture the specifics of RS data. Further, we summarize the works
combining xAI with other fields, like physics and uncertainty, that produce
scientifically sound ML models, ultimately enhancing the quality of the
extracted explanations.

These highlights clearly illustrate that xAI in RS is a young field with high
potential to augment the ML knowledge extraction process. At the same time,
comparing how the identified practices in this review relate to the latest
developments in xAI and the current limitations in EO gives a hint to the
fundamental challenges that need to be addressed in future work. One research
direction of paramount importance is the development of interpretable DNNs to
address the shortcomings of the widely-used saliency methods in RS. Further,
as most studies conduct only anecdotal evaluation, verifying the reliability
of the xAI outputs supports future work on quantitative evaluation and user
studies. Moreover, the adapted methods indicate that the traditional xAI
approaches do not conform to the properties of RS data. Therefore, we
encourage future work on developing models to address those drawbacks related
to scale, topology, and temporal dependencies in RS data. Another challenge is
the lack of labeled data, which is currently tackled by combining SSL with xAI
to design approaches that outperform supervised ML models, additionally
offering intuitive model interpretability. Summarizing our contributions, we
hope that the insights provided in this review enable the researchers to
better understand the state-of-the-art in this field and promote the
development of novel methods by tackling the research directions proposed.

###  -A Search queries

Due to the implementation of the search in the databases, they cannot handle
deep and nested search queries. For this reason, we repeated different
versions of different words, like AI in ”explain* AI” and ”interpret* AI”.

####  -A1 IEEE

In the IEEE search, abstract, title, and keywords have the search terms
”Abstract”, ”Document Title”, and ”Index Terms”, respectively. Unfortunately,
it is not possible to search with more than 8 wildcards (”*”) in one search
query. Hence, we split the queries into different parts and merge the results
in a second step. The search query cannot include the year or the language, so
the results were filtered in the masks of their website.

⬇

("Abstract":earth observation OR "Abstract":remote sensing OR "Abstract":earth
science OR "Abstract":satellite data OR "Abstract":satellite image OR
"Abstract":aerial image OR "Abstract":aerial data OR "Abstract":airborne image
OR "Abstract":airborne data OR "Abstract":radar image OR "Abstract":radar data
OR "Abstract":spaceborne image OR "Abstract":spaceborne data OR
"Abstract":LiDAR OR "Abstract":SAR OR "Abstract":UAV OR "Abstract":Sentinel OR
"Abstract":Landsat OR "Abstract":MODIS OR "Abstract":gaofen OR
"Abstract":ceres) AND ("Abstract":"xai" OR "Abstract":"interpret* model" OR
"Abstract":"interpret* deep learning" OR "Abstract":"interpret* machine
learning" OR "Abstract":"interpret* artificial intelligence" OR
"Abstract":"interpret* dl" OR "Abstract":"interpret* ml")

("Abstract":earth observation OR "Abstract":remote sensing OR "Abstract":earth
science OR "Abstract":satellite data OR "Abstract":satellite image OR
"Abstract":aerial image OR "Abstract":aerial data OR "Abstract":airborne image
OR "Abstract":airborne data OR "Abstract":radar image OR "Abstract":radar data
OR "Abstract":spaceborne image OR "Abstract":spaceborne data OR
"Abstract":LiDAR OR "Abstract":SAR OR "Abstract":UAV OR "Abstract":Sentinel OR
"Abstract":Landsat OR "Abstract":MODIS OR "Abstract":gaofen OR
"Abstract":ceres) AND ("Abstract":"interpret* AI" OR "Abstract":"explain*
model" OR "Abstract":"explain* deep learning" OR "Abstract":"explain* machine
learning" OR "Abstract":"explain* artificial intelligence" OR
"Abstract":"explain* dl" OR "Abstract":"explain* ml" OR "Abstract":"explain*
AI")

("Index Terms":earth observation OR "Index Terms":remote sensing OR "Index
Terms":earth science OR "Index Terms":satellite data OR "Index
Terms":satellite image OR "Index Terms":aerial image OR "Index Terms":aerial
data OR "Index Terms":airborne image OR "Index Terms":airborne data OR "Index
Terms":radar image OR "Index Terms":radar data OR "Index Terms":spaceborne
image OR "Index Terms":spaceborne data OR "Index Terms":LiDAR OR "Index
Terms":SAR OR "Index Terms":UAV OR "Index Terms":Sentinel OR "Index
Terms":Landsat OR "Index Terms":MODIS OR "Index Terms":gaofen OR "Index
Terms":ceres) AND ("Index Terms":"xai" OR "Index Terms":"interpret* model" OR
"Index Terms":"interpret* deep learning" OR "Index Terms":"interpret* machine
learning" OR "Index Terms":"interpret* artificial intelligence" OR "Index
Terms":"interpret* dl" OR "Index Terms":"interpret* ml")

("Index Terms":earth observation OR "Index Terms":remote sensing OR "Index
Terms":earth science OR "Index Terms":satellite data OR "Index
Terms":satellite image OR "Index Terms":aerial image OR "Index Terms":aerial
data OR "Index Terms":airborne image OR "Index Terms":airborne data OR "Index
Terms":radar image OR "Index Terms":radar data OR "Index Terms":spaceborne
image OR "Index Terms":spaceborne data OR "Index Terms":LiDAR OR "Index
Terms":SAR OR "Index Terms":UAV OR "Index Terms":Sentinel OR "Index
Terms":Landsat OR "Index Terms":MODIS OR "Index Terms":gaofen OR "Index
Terms":ceres) AND ("Index Terms":"interpret* AI" OR "Index Terms":"explain*
model" OR "Index Terms":"explain* deep learning" OR "Index Terms":"explain*
machine learning" OR "Index Terms":"explain* artificial intelligence" OR
"Index Terms":"explain* dl" OR "Index Terms":"explain* ml" OR "Index
Terms":"explain* AI")

("Document Title":earth observation OR "Document Title":remote sensing OR
"Document Title":earth science OR "Document Title":satellite data OR "Document
Title":satellite image OR "Document Title":aerial image OR "Document
Title":aerial data OR "Document Title":airborne image OR "Document
Title":airborne data OR "Document Title":radar image OR "Document Title":radar
data OR "Document Title":spaceborne image OR "Document Title":spaceborne data
OR "Document Title":LiDAR OR "Document Title":SAR OR "Document Title":UAV OR
"Document Title":Sentinel OR "Document Title":Landsat OR "Document
Title":MODIS OR "Document Title":gaofen OR "Document Title":ceres) AND
("Document Title":"xai" OR "Document Title":"interpret* model" OR "Document
Title":"interpret* deep learning" OR "Document Title":"interpret* machine
learning" OR "Document Title":"interpret* artificial intelligence" OR
"Document Title":"interpret* dl" OR "Document Title":"interpret* ml")

("Document Title":earth observation OR "Document Title":remote sensing OR
"Document Title":earth science OR "Document Title":satellite data OR "Document
Title":satellite image OR "Document Title":aerial image OR "Document
Title":aerial data OR "Document Title":airborne image OR "Document
Title":airborne data OR "Document Title":radar image OR "Document Title":radar
data OR "Document Title":spaceborne image OR "Document Title":spaceborne data
OR "Document Title":LiDAR OR "Document Title":SAR OR "Document Title":UAV OR
"Document Title":Sentinel OR "Document Title":Landsat OR "Document
Title":MODIS OR "Document Title":gaofen OR "Document Title":ceres) AND
("Document Title":"interpret* AI" OR "Document Title":"explain* model" OR
"Document Title":"explain* deep learning" OR "Document Title":"explain*
machine learning" OR "Document Title":"explain* artificial intelligence" OR
"Document Title":"explain* dl" OR "Document Title":"explain* ml" OR "Document
Title":"explain* AI")

####  -A2 Scopus

Scopus has the most powerful search engine for its database. Therefore, it was
possible to specify the year of the publication and the language directly in
the search query. The search term for abstract, title, and keywords is ”TITLE-
ABS-KEY”.

⬇

TITLE-ABS-KEY((earth observation) OR (remote sensing) OR (earth science) OR
(satellite data) OR (satellite image) OR (aerial image) OR (aerial data) OR
(airborne image) OR (airborne data) OR (radar image) OR (radar data) OR
(spaceborne image) OR (spaceborne data) OR (LiDAR) OR (SAR) OR (UAV) OR
(Sentinel) OR (Landsat) OR (MODIS) OR (gaofen) OR (ceres)) AND TITLE-ABS-
KEY("xai" OR "interpret* model" OR "interpret* deep learning" OR "interpret*
machine learning" OR "interpret* artificial intelligence" OR "interpret* dl"
OR "interpret* ml" OR "interpret* AI" OR "explain* model" OR "explain* deep
learning" OR "explain* machine learning" OR "explain* artificial intelligence"
OR "explain* dl" OR "explain* ml" OR "explain* AI") AND ( PUBYEAR > 2016 ) AND
( LIMIT-TO ( LANGUAGE,"English" ) )

####  -A3 Springer

The Springer link search could not be done only on abstract, title, and
keywords. Hence, a full-text search was done. The search query cannot include
the year or the language, so the results were filtered in the masks of their
website.

⬇

(earth observation OR remote sensing OR earth science OR satellite data OR
satellite image OR aerial image OR aerial data OR airborne image OR airborne
data OR radar image OR radar data OR spaceborne image OR spaceborne data OR
LiDAR OR SAR OR UAV OR Sentinel OR Landsat OR MODIS OR gaofen OR ceres) AND
("xai" OR "interpret* model" OR "interpret* deep learning" OR "interpret*
machine learning" OR "interpret* artificial intelligence" OR "interpret* dl"
OR "interpret* ml" OR "interpret* AI" OR "explain* model" OR "explain* deep
learning" OR "explain* machine learning" OR "explain* artificial intelligence"
OR "explain* dl" OR "explain* ml" OR "explain* AI")

####  -A4 Search query, ML in EO

This search query was executed in all databases for comparison purposes, its
results are shown in Figure 1.

⬇

$\Bigl{[}$earth observation OR remote sensing OR earth science OR
$\bigl{(}$(satellite OR aerial OR airborne OR spaceborne OR radar) AND (image
OR data)$\bigl{)}$ OR LiDAR OR SAR OR UAV OR Sentinel OR Landsat OR MODIS OR
gaofen OR ceres$\Bigl{]}$

AND

$\Bigl{[}$deep learning OR machine learning OR artificial intelligence OR dl
OR ml OR ai$\Bigl{]}$

###  -B Models, xAI categories and methods

Figure LABEL:fig:xai_models_categories_methods shows the number of times xAI
categories, methods, and models are mentioned throughout this review.

###  -C Glossary

We have categorized related EO tasks into groups to provide a better overview.
Here, we provide a glossary of these groups and the tasks they include.

Agricultural Monitoring:

All crop-related tasks, like crop yield prediction, crop type classification,
irrigation scheme classification, and crop lodging detection.

Atmosphere Monitoring:

The prediction of atmospheric phenomena, like air quality, aerosol optical
depth, and dust storm indices.

Building Mapping:

All tasks related to buildings and urban structures, like building footprint
classification and building damage mapping.

Ecosystem Interactions:

All interactions of the ecosystem with other systems, e.g., the atmosphere and
the hydrosphere. This includes the ecosystem CO2 exchange or the sun-induced
fluorescence prediction.

Human Environment Interaction:

The monitoring of human structures and the environment, like human footprint
estimation, socioeconomic status estimation, or well-being prediction.

Hydrology Monitoring:

Tasks related to hydrology, like runoff forecasting, water quality, streamflow
prediction, and water segmentation, but excluding floods.

Landcover Mapping:

The most common EO task includes mainly landcover classification but also
related tasks like slum mapping.

Natural Hazard Monitoring:

Monitoring of natural hazards, like landslides, wildfires, floods,
earthquakes, and volcanos.

Soil Monitoring:

Monitoring soil properties, like soil texture, respiration, moisture, and
salinity.

Surface Temperature Prediction:

The prediction of the Earth’s surface temperature.

Target Mapping:

Tasks related to the mapping of specific targets, like vehicles and objects.

Vegetation Monitoring:

Monitoring of vegetation, excluding crops, like vegetation regeneration, tree
monitoring, tree classification, and tree mapping.

Weather Climate Prediction:

Forecasting of weather and climate variables, like precipitation, temperature,
and drought.

Other:

All the tasks which did not fit into the other groups. This includes change
detection, urban mobility, sea ice classification, mosquito modeling, and
satellite product quality tasks.

## Acronyms

AI

    Artificial Intelligence
ALE

    Accumulated Local Effects
ASC

    attribute scattering center
BP

    Backpropagation
CAM

    Class Activation Mapping
CAP

    Common Agricultural Policy
CNN

    Convolutional Neural Network
CV

    Computer Vision
DeepLIFT

    Deep Learning Important FeaTures
DEM

    Digital Elevation Model
DL

    Deep Learning
DNN

    Deep Neural Network
DSM

    Digital Surface Model
DTD

    Deep Taylor Decomposition
EBM

    Explainable Boosting Machines
EG

    Expected Gradients
EO

    Earth Observation
FLS

    Fuzzy Logic System
FNN

    Feed Forward Neural Network
GAM

    Generalized Additive Model
GAN

    Generative Adversarial Network
GB

    Gradient Boosting
GCN

    Graph Convolutional Networks
GLM

    Generalized Linear Model
GNN

    Graph Neural Networks
GP

    Gaussian Process
Grad-CAM

    Gradient-weighted Class Activation Mapping
GRU

    Gated Recurrent Unit
I*G

    Input*Gradient
IG

    Integrated Gradients
IML

    interpretable Machine Learning
KL

    Kullback-Leiber
kNN

    k-Nearest-Neighbor
LDA

    Latent Dirichlet Allocation
LiDAR

    Light Detection and Ranging
LIME

    Local Interpretable Model-agnostic Explanation
LR

    Linear Regression
LRP

    Layer-wise Relevance Propagation
LSTM

    Long Short-Term Memory neural network
MDI

    Mean Decrease in Impurity
ML

    Machine Learning
MLP

    Multilayer Perceptron
MoRef

    Most Relevant First
NDVI

    Normalized Difference Vegetation Index
NN

    Neural Network
OOD

    Out-Of-Distribution
OWA

    Ordered Weighted Averaging
PCA

    Principal Component Analysis
PDP

    Partial Dependence Plot
PFI

    Permutation Feature Importance
PolSAR

    Polarimetric SAR
RAM

    Regression Activation Mapping
RF

    Random Forest
RGB

    Red-Green-Blue
RNN

    Recurrent Neural Network
RS

    Remote Sensing
SAR

    Synthetic Aperture Radar
SHAP

    SHapley Additive exPlanations
SmoothGrad

    Smooth Gradient
SOTA

    state-of-the-art
SSL

    Self-Supervised Learning
SST

    Sea Surface Temperature
SVM

    Support Vector Machine
TCAV

    Testing with Concept Activation Vectors
TDA

    topological data analysis
t-SNE

    t-distributed Stochastic Neighbor Embedding
UAV

    Unmanned Aerial Vehicles
UMAP

    Uniform Manifold Approximation and Projection
ViT

    Vision Transformer
xAI

    explainable AI
XGBoost

    eXtreme Gradient Boosting

## References

  * [1] Gustau Camps-Valls, Devis Tuia, Xiao Xiang Zhu and Markus Reichstein “Deep Learning for the Earth Sciences: A Comprehensive Approach to Remote Sensing, Climate Science, and Geosciences” In _Deep Learning for the Earth Sciences_ 1, 2021
  * [2] Xiao Xiang Zhu et al. “Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources” In _IEEE Geoscience and Remote Sensing Magazine_ 5.4 Institute of Electrical and Electronics Engineers Inc., 2017, pp. 8–36 DOI: 10.1109/MGRS.2017.2762307
  * [3] Markus Reichstein et al. “Deep Learning and Process Understanding for Data-Driven Earth System Science” In _Nature_ 566.7743 Nature Publishing Group, 2019, pp. 195–204 DOI: 10.1038/s41586-019-0912-1
  * [4] Hannah Ruschemeier “AI as a challenge for legal regulation–the scope of application of the artificial intelligence act proposal” In _ERA Forum_ , 2023, pp. 1–16 Springer
  * [5] Sarvam P TerKonda and Eric M Fish “Artificial intelligence viewed through the lens of state regulation” In _Intelligence-Based Medicine_ Elsevier, 2023, pp. 100088
  * [6] Devis Tuia et al. “Toward a Collective Agenda on AI for Earth Science Data Analysis” In _IEEE Geoscience and Remote Sensing Magazine_ 9.2, 2021, pp. 88–104 DOI: 10.1109/MGRS.2020.3043504
  * [7] Esther Rolf, Konstantin Klemmer, Caleb Robinson and Hannah Kerner “Mission Critical – Satellite Data Is a Distinct Modality in Machine Learning” arXiv, 2024 arXiv:2402.01444 [cs]
  * [8] Michael F. Goodchild “Scale in GIS: An Overview” In _Geomorphology_ 130.1, Scale Issues in Geomorphology, 2011, pp. 5–9 DOI: 10.1016/j.geomorph.2010.10.004
  * [9] Thomas Lillesand, Ralph W Kiefer and Jonathan Chipman “Remote sensing and image interpretation” John Wiley & Sons, 2015
  * [10] Gabrielle Ras, Ning Xie, Marcel Gerven and Derek Doran “Explainable Deep Learning: A Field Guide for the Uninitiated” In _Journal of Artificial Intelligence Research_ 73, 2022, pp. 329–396 DOI: 10.1613/jair.1.13200
  * [11] Dang Minh, H Xiang Wang, Y Fen Li and Tan N Nguyen “Explainable artificial intelligence: a comprehensive review” In _Artificial Intelligence Review_ Springer, 2022, pp. 1–66
  * [12] Timo Speith “A review of taxonomies of explainable artificial intelligence (XAI) methods” In _2022 ACM Conference on Fairness, Accountability, and Transparency_ , 2022, pp. 2239–2250
  * [13] Caroline M. Gevaert “Explainable AI for Earth Observation: A Review Including Societal and Regulatory Perspectives” In _International Journal of Applied Earth Observation and Geoinformation_ 112, 2022, pp. 102869 DOI: 10.1016/j.jag.2022.102869
  * [14] Carolin Leluschko and Christoph Tholen “Goals and Stakeholder Involvement in XAI for Remote Sensing: A Structured Literature Review” In _Artificial Intelligence XL_ , Lecture Notes in Computer Science Cham: Springer Nature Switzerland, 2023, pp. 519–525 DOI: 10.1007/978-3-031-47994-6˙47
  * [15] Ola Hall, Mattias Ohlsson and Thorsteinn Rögnvaldsson “A Review of Explainable AI in the Satellite Data, Deep Machine Learning, and Human Poverty Domain” In _Patterns_ 3.10, 2022, pp. 100600 DOI: 10.1016/j.patter.2022.100600
  * [16] Jin Xing and Renee Sieber “The Challenges of Integrating Explainable Artificial Intelligence into GeoAI” In _Transactions in GIS_ 27.3, 2023, pp. 626–645 DOI: 10.1111/tgis.13045
  * [17] R. Roscher, B. Bohn, M.F. Duarte and J. Garcke “Explain It to Me-Facing Remote Sensing Challenges in the Bio-and Geosciences with Explainable Machine Learning” 5, ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2020, pp. 817–824 DOI: 10.5194/isprs-Annals-V-3-2020-817-2020
  * [18] Ribana Roscher, Bastian Bohn, Marco F. Duarte and Jochen Garcke “Explainable Machine Learning for Scientific Insights and Discoveries” In _IEEE Access_ 8 Institute of Electrical and Electronics Engineers Inc., 2020, pp. 42200–42216 DOI: 10.1109/ACCESS.2020.2976199
  * [19] Christoph Molnar “Interpretable Machine Learning. A Guide for Making Black Box Models Explainable.” In _Book_ , 2019 URL: https://christophm.github.io/interpretable-ml-book
  * [20] Anuj Karpatne et al. “Machine Learning for the Geosciences: Challenges and Opportunities” In _IEEE Transactions on Knowledge and Data Engineering_ 31.8 IEEE Computer Society, 2019, pp. 1544–1554 DOI: 10.1109/TKDE.2018.2861006
  * [21] A. Mamalakis, I. Ebert-Uphoff and E.A. Barnes “Explainable Artificial Intelligence in Meteorology and Climate Science: Model Fine-Tuning, Calibrating Trust and Learning New Science” In _Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)_ 13200 LNAI, 2022, pp. 315–339 DOI: 10.1007/978-3-031-04083-2˙16
  * [22] George Em Karniadakis et al. “Physics-Informed Machine Learning” In _Nature Reviews Physics_ 3.6, 2021, pp. 422–440 DOI: 10.1038/s42254-021-00314-5
  * [23] Jakob Gawlikowski et al. “A Survey of Uncertainty in Deep Neural Networks” arXiv, 2022 DOI: 10.48550/arXiv.2107.03342
  * [24] Adrian Perez-Suay and Gustau Camps-Valls “Causal Inference in Geoscience and Remote Sensing from Observational Data” In _IEEE Transactions on Geoscience and Remote Sensing_ 57.3 Institute of Electrical and Electronics Engineers Inc., 2019, pp. 1502–1513 DOI: 10.1109/TGRS.2018.2867002
  * [25] Jakob Runge et al. “Inferring Causation from Time Series in Earth System Sciences” In _Nature Communications_ 10.1 Nature Publishing Group, 2019, pp. 2553 DOI: 10.1038/s41467-019-10105-3
  * [26] Matthew J. Page et al. “PRISMA 2020 Explanation and Elaboration: Updated Guidance and Exemplars for Reporting Systematic Reviews” In _BMJ_ 372 British Medical Journal Publishing Group, 2021, pp. n160 DOI: 10.1136/bmj.n160
  * [27] Michael Gusenbauer and Neal Haddaway “Which Academic Search Systems Are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed and 26 Other Resources [OPEN ACCESS]” In _Research Synthesis Methods_ 11, 2020, pp. 181–217 DOI: 10.1002/jrsm.1378
  * [28] Giulia Vilone and Luca Longo “Explainable Artificial Intelligence: A Systematic Review” arXiv, 2020 arXiv:2006.00093 [cs]
  * [29] Hans Hersbach et al. “The ERA5 Global Reanalysis” In _Quarterly Journal of the Royal Meteorological Society_ 146.730, 2020, pp. 1999–2049 DOI: 10.1002/qj.3803
  * [30] Amina Adadi and Mohammed Berrada “Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)” In _IEEE Access_ 6, 2018, pp. 52138–52160 DOI: 10.1109/ACCESS.2018.2870052
  * [31] Andreas Holzinger et al. “Explainable AI methods-a brief overview” In _xxAI-Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers_ , 2022, pp. 13–38 Springer
  * [32] Wojciech Samek et al. “Explaining deep neural networks and beyond: A review of methods and applications” In _Proceedings of the IEEE_ 109.3 IEEE, 2021, pp. 247–278
  * [33] Scott M. Lundberg et al. “From Local Explanations to Global Understanding with Explainable AI for Trees” In _Nature Machine Intelligence_ 2.1 Nature Publishing Group, 2020, pp. 56–67 DOI: 10.1038/s42256-019-0138-9
  * [34] Mattia Setzu et al. “GLocalX - From Local to Global Explanations of Black Box AI Models” In _Artificial Intelligence_ 294, 2021, pp. 103457 DOI: 10.1016/j.artint.2021.103457
  * [35] Huan Liu “Feature Selection” In _Encyclopedia of Machine Learning_ Boston, MA: Springer US, 2010, pp. 402–406 DOI: 10.1007/978-0-387-30164-8“˙306
  * [36] Dumitru Erhan, Aaron Courville, Yoshua Bengio and P O Box “Understanding Representations Learned in Deep Architectures”, 2010
  * [37] K Simonyan, A Vedaldi and A Zisserman “Deep inside convolutional networks: visualising image classification models and saliency maps” In _Proceedings of the International Conference on Learning Representations (ICLR)_ , 2014 ICLR
  * [38] Mukund Sundararajan, Ankur Taly and Qiqi Yan “Axiomatic Attribution for Deep Networks” In _Proceedings of the 34th International Conference on Machine Learning_ PMLR, 2017, pp. 3319–3328
  * [39] Matthew D. Zeiler, Graham W. Taylor and Rob Fergus “Adaptive Deconvolutional Networks for Mid and High Level Feature Learning” In _2011 International Conference on Computer Vision_ , 2011, pp. 2018–2025 DOI: 10.1109/ICCV.2011.6126474
  * [40] Bolei Zhou et al. “Learning deep features for discriminative localization” In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 2921–2929
  * [41] Ramprasaath R. Selvaraju et al. “Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization” In _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_ , 2017
  * [42] Sebastian Bach et al. “On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation” In _PloS one_ 10.7 Public Library of Science San Francisco, CA USA, 2015, pp. e0130140
  * [43] Matthew D Zeiler and Rob Fergus “Visualizing and understanding convolutional networks” In _Computer Vision–ECCV 2014_ , 2014, pp. 818–833 Springer
  * [44] Jerome H Friedman “Greedy function approximation: a gradient boosting machine” In _Annals of statistics_ Institute of Mathematical Statistics, 2001, pp. 1189–1232
  * [45] Daniel W Apley and Jingyu Zhu “Visualizing the effects of predictor variables in black box supervised learning models” In _Journal of the Royal Statistical Society Series B: Statistical Methodology_ 82.4 Oxford University Press, 2020, pp. 1059–1086
  * [46] Marco Tulio Ribeiro, Sameer Singh and Carlos Guestrin “”Why Should I Trust You?”: Explaining the Predictions of Any Classifier” In _Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ San Francisco California USA: ACM, 2016, pp. 1135–1144 DOI: 10.1145/2939672.2939778
  * [47] Scott M Lundberg and Su-In Lee “A Unified Approach to Interpreting Model Predictions” In _Advances in Neural Information Processing Systems_ 30 Curran Associates, Inc., 2017
  * [48] W. Murdoch and Arthur Szlam “Automatic Rule Extraction from Long Short Term Memory Networks” In _International Conference on Learning Representations_ , 2017 URL: https://openreview.net/forum?id=SJvYgH9xe
  * [49] Michael Harradon, Jeff Druce and Brian Ruttenberg “Causal learning and explanation of deep neural networks via autoencoded activations” In _arXiv preprint arXiv:1802.00541_ , 2018
  * [50] Nicholas Frosst and Geoffrey Hinton “Distilling a neural network into a soft decision tree” In _arXiv preprint arXiv:1711.09784_ , 2017
  * [51] Sarah Tan et al. “Considerations when learning additive explanations for black-box models” In _arXiv preprint arXiv:1801.08640_ , 2018
  * [52] Quanshi Zhang, Yu Yang, Haotian Ma and Ying Nian Wu “Interpreting cnns via decision trees” In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 6261–6270
  * [53] Quanshi Zhang et al. “Interpreting CNN knowledge via an explanatory graph” In _Proceedings of the AAAI conference on artificial intelligence_ 32.1, 2018
  * [54] Quanshi Zhang, Ruiming Cao, Ying Nian Wu and Song-Chun Zhu “Growing interpretable part graphs on convnets via multi-shot learning” In _Proceedings of the AAAI Conference on Artificial Intelligence_ 31.1, 2017
  * [55] Trevor Hastie, Robert Tibshirani, Jerome H Friedman and Jerome H Friedman “The elements of statistical learning: data mining, inference, and prediction” Springer, 2009
  * [56] J Dobson Annette “Introduction to generalized linear models” Chapman & Hall CRC, 2018
  * [57] Trevor J Hastie “Generalized additive models” In _Statistical models in S_ Routledge, 1992, pp. 249–307
  * [58] David M Blei, Andrew Y Ng and Michael I Jordan “Latent dirichlet allocation” In _Journal of machine Learning research_ 3.Jan, 2003, pp. 993–1022
  * [59] Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio “Neural machine translation by jointly learning to align and translate” In _arXiv preprint arXiv:1409.0473_ , 2014
  * [60] Ashish Vaswani et al. “Attention is all you need” In _Advances in neural information processing systems_ 30, 2017
  * [61] Been Kim et al. “Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)” In _International conference on machine learning_ , 2018, pp. 2668–2677 PMLR
  * [62] Amirata Ghorbani, James Wexler, James Y Zou and Been Kim “Towards automatic concept-based explanations” In _Advances in Neural Information Processing Systems_ 32, 2019
  * [63] Pang Wei Koh et al. “Concept bottleneck models” In _International conference on machine learning_ , 2020, pp. 5338–5348 PMLR
  * [64] Diego Marcos, Sylvain Lobry and Devis Tuia “Semantically Interpretable Activation Maps: what-where-how explanations within CNNs” In _2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)_ , 2019, pp. 4207–4215 IEEE
  * [65] Jacob Bien and Robert Tibshirani “Prototype selection for interpretable classification”, 2011
  * [66] Chaofan Chen et al. “This looks like that: deep learning for interpretable image recognition” In _Advances in neural information processing systems_ 32, 2019
  * [67] Meike Nauta, Ron Van Bree and Christin Seifert “Neural prototype trees for interpretable fine-grained image recognition” In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 14933–14943
  * [68] Jinkyu Kim et al. “Textual explanations for self-driving vehicles” In _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 563–578
  * [69] Sandra Wachter, Brent Mittelstadt and Chris Russell “Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR” In _Harvard Journal of Law & Technology (Harvard JOLT)_ 31, 2017, pp. 841
  * [70] Tomas Mikolov et al. “Distributed Representations of Words and Phrases and Their Compositionality” In _Advances in Neural Information Processing Systems_ 26 Curran Associates, Inc., 2013
  * [71] Maximilian Kohlbrenner et al. “Towards best practice in explaining neural network decisions with LRP” In _2020 International Joint Conference on Neural Networks (IJCNN)_ , 2020, pp. 1–7 IEEE
  * [72] Grégoire Montavon et al. “Layer-wise relevance propagation: an overview” In _Explainable AI: interpreting, explaining and visualizing deep learning_ Springer, 2019, pp. 193–209
  * [73] Leila Arras et al. “Explaining and Interpreting LSTMs” In _Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)_ 11700 LNCS Springer Verlag, 2019, pp. 211–238 DOI: 10.1007/978-3-030-28954-6“˙11/FIGURES/7
  * [74] Aaron Fisher, Cynthia Rudin and Francesca Dominici “All Models Are Wrong, but Many Are Useful: Learning a Variable’s Importance by Studying an Entire Class of Prediction Models Simultaneously” In _Journal of Machine Learning Research_ 20.177, 2019, pp. 1–81
  * [75] Lloyd S Shapley “A Value for n-Person Games” In _Contributions to the Theory of Games II_ Princeton: Princeton University Press, 1953, pp. 307–317
  * [76] Scott M Lundberg, Gabriel G Erion and Su-In Lee “Consistent individualized feature attribution for tree ensembles” In _arXiv preprint arXiv:1802.03888_ , 2018
  * [77] M. Augasta and T. Kathirvalavakumar “Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems” In _Neural Processing Letters_ 35.2, 2012, pp. 131–150 DOI: 10.1007/s11063-011-9207-8
  * [78] Zhi-Hua Zhou and Yuan Jiang “Medical Diagnosis with C4.5 Rule Preceded by Artificial Neural Network Ensemble” In _IEEE Transactions on Information Technology in Biomedicine_ 7.1, 2003, pp. 37–42 DOI: 10.1109/TITB.2003.808498
  * [79] Xuan Liu, Xiaoguang Wang and Stan Matwin “Improving the Interpretability of Deep Neural Networks with Knowledge Distillation” In _2018 IEEE International Conference on Data Mining Workshops (ICDMW)_ , 2018, pp. 905–912 DOI: 10.1109/ICDMW.2018.00132
  * [80] David Alvarez-Melis and Tommi S. Jaakkola “A Causal Framework for Explaining the Predictions of Black-Box Sequence-to-Sequence Models” arXiv, 2017 arXiv:1707.01943 [cs]
  * [81] Bernadette Bouchon-Meunier and Christophe Marsala “Learning Fuzzy Decision Rules” In _Fuzzy Sets in Approximate Reasoning and Information Systems_ , The Handbooks of Fuzzy Sets Series Boston, MA: Springer US, 1999, pp. 279–304 DOI: 10.1007/978-1-4615-5243-7“˙5
  * [82] Lotfi A Zadeh “Fuzzy sets” In _Information and control_ 8.3 Elsevier, 1965, pp. 338–353
  * [83] Leo Breiman, Jerome H Friedman, Richard A Olshen and Charles J Stone “Classification and regression trees” In _Wadsworth, Belmont, CA_ , 1984
  * [84] Zachary C Lipton “The Mythos of Model Interpretability: In Machine Learning, the Concept of Interpretability Is Both Important and Slippery.” In _Queue_ 16.3 ACM New York, NY, USA, 2018, pp. 31–57
  * [85] Laurens Van der Maaten and Geoffrey Hinton “Visualizing data using t-SNE.” In _Journal of machine learning research_ 9.11, 2008
  * [86] Leland McInnes, John Healy, Nathaniel Saul and Lukas Großberger “UMAP: Uniform Manifold Approximation and Projection” In _Journal of Open Source Software_ 3.29, 2018
  * [87] Wieland Brendel and Matthias Bethge “Approximating CNNs with Bag-of-local-Features Models Works Surprisingly Well on ImageNet” In _International Conference on Learning Representations_ , 2019
  * [88] Tim Miller “Contrastive Explanation: A Structural-Model Approach” In _The Knowledge Engineering Review_ 36 Cambridge University Press, 2021, pp. e14 DOI: 10.1017/S0269888921000102
  * [89] Peng-Tao Jiang et al. “LayerCAM: Exploring Hierarchical Class Activation Maps for Localization” In _IEEE Transactions on Image Processing_ 30, 2021, pp. 5875–5888 DOI: 10.1109/TIP.2021.3089943
  * [90] Avanti Shrikumar, Peyton Greenside and Anshul Kundaje “Learning important features through propagating activation differences” In _International conference on machine learning_ , 2017, pp. 3145–3153 PMLR
  * [91] Gianni Brauwers and Flavius Frasincar “A general survey on attention mechanisms in deep learning” In _IEEE Transactions on Knowledge and Data Engineering_ IEEE, 2021
  * [92] Alexey Dosovitskiy et al. “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale” In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_ OpenReview.net, 2021 URL: https://openreview.net/forum?id=YicbFdNTTy
  * [93] Petar Veličković et al. “Graph Attention Networks” In _6th International Conference on Learning Representations_ , 2017
  * [94] Kevin Clark, Urvashi Khandelwal, Omer Levy and Christopher D. Manning “What Does BERT Look at? An Analysis of BERT’s Attention” In _Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ Florence, Italy: Association for Computational Linguistics, 2019, pp. 276–286 DOI: 10.18653/v1/W19-4828
  * [95] Mathilde Caron et al. “Emerging properties in self-supervised vision transformers” In _Proceedings of the IEEE/CVF international conference on computer vision_ , 2021, pp. 9650–9660
  * [96] Sneha Chaudhari, Varun Mithal, Gungor Polatkan and Rohan Ramanath “An Attentive Survey of Attention Models” In _ACM Transactions on Intelligent Systems and Technology (TIST)_ 12.5 ACM New York, NY, 2021, pp. 1–32
  * [97] Finale Doshi-Velez and Been Kim “Towards a rigorous science of interpretable machine learning” In _arXiv preprint arXiv:1702.08608_ , 2017
  * [98] Meike Nauta et al. “From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI” Just Accepted In _ACM Comput. Surv._ New York, NY, USA: Association for Computing Machinery, 2023 DOI: 10.1145/3583558
  * [99] Julius Adebayo et al. “Sanity Checks for Saliency Maps” In _Advances in Neural Information Processing Systems_ 31 Curran Associates, Inc., 2018
  * [100] Umang Bhatt, Adrian Weller and José M.. Moura “Evaluating and Aggregating Feature-based Model Explanations” Main track In _Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20_ International Joint Conferences on Artificial Intelligence Organization, 2020, pp. 3016–3022 DOI: 10.24963/ijcai.2020/417
  * [101] Anna Hedström et al. “Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond” In _Journal of Machine Learning Research_ 24.34, 2023, pp. 1–11
  * [102] Chih-Kuan Yeh et al. “On the (in) fidelity and sensitivity of explanations” In _Advances in Neural Information Processing Systems_ 32, 2019
  * [103] Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans and Been Kim “A benchmark for interpretability methods in deep neural networks” In _Advances in neural information processing systems_ 32, 2019
  * [104] Yao Rong et al. “A consistent and efficient evaluation strategy for attribution methods” In _arXiv preprint arXiv:2202.00449_ , 2022
  * [105] Harini Suresh, Kathleen M Lewis, John Guttag and Arvind Satyanarayan “Intuitively assessing ml model reliability through example-based explanations and editing model inputs” In _27th International Conference on Intelligent User Interfaces_ , 2022, pp. 767–781
  * [106] Ahmed Alqaraawi et al. “Evaluating saliency map explanations for convolutional neural networks: a user study” In _Proceedings of the 25th International Conference on Intelligent User Interfaces_ , 2020, pp. 275–285
  * [107] Yao Rong et al. “Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations” In _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2023, pp. 1–20 DOI: 10.1109/TPAMI.2023.3331846
  * [108] Leander Weber, Sebastian Lapuschkin, Alexander Binder and Wojciech Samek “Beyond explaining: Opportunities and challenges of XAI-based model improvement” In _Information Fusion_ 92 Elsevier, 2023, pp. 154–176
  * [109] Laura N. Sotomayor, Matthew J. Cracknell and Robert Musk “Supervised Machine Learning for Predicting and Interpreting Dynamic Drivers of Plantation Forest Productivity in Northern Tasmania, Australia” In _Computers and Electronics in Agriculture_ 209, 2023 DOI: 10.1016/j.compag.2023.107804
  * [110] Haifei Chen, Liping Yang and Qiusheng Wu “Enhancing Land Cover Mapping and Monitoring: An Interactive and Explainable Machine Learning Approach Using Google Earth Engine” In _Remote Sensing_ 15.18, 2023 DOI: 10.3390/rs15184585
  * [111] T. Fisher et al. “Uncertainty-Aware Interpretable Deep Learning for Slum Mapping and Monitoring” In _Remote Sensing_ 14.13, 2022 DOI: 10.3390/rs14133072
  * [112] Sadeeka Layomi Jayasinghe and Lalit Kumar “Causes of Tea Land Dynamics in Sri Lanka between 1995 and 2030” In _Regional Environmental Change_ 23.4, 2023, pp. 127 DOI: 10.1007/s10113-023-02123-1
  * [113] Z. Li et al. “Advancing Satellite Precipitation Retrievals with Data Driven Approaches: Is Black Box Model Explainable?” In _Earth and Space Science_ 8.2, 2021 DOI: 10.1029/2020EA001423
  * [114] S.S. Matin and B. Pradhan “Earthquake-Induced Building-Damage Mapping Using Explainable Ai (Xai)” In _Sensors_ 21.13, 2021 DOI: 10.3390/s21134489
  * [115] Carles Milà et al. “Estimating Daily Air Temperature and Pollution in Catalonia: A Comprehensive Spatiotemporal Modelling of Multiple Exposures” In _Environmental Pollution_ 337, 2023 DOI: 10.1016/j.envpol.2023.122501
  * [116] Oladimeji Mudele et al. “Modeling the Temporal Population Distribution of Ae. Aegypti Mosquito Using Big Earth Observation Data” In _IEEE access : practical innovations, open solutions_ 8, 2020, pp. 14182–14194 DOI: 10.1109/ACCESS.2020.2966080
  * [117] S.J. Newman and R.T. Furbank “Explainable Machine Learning Models of Major Crop Traits from Satellite-Monitored Continent-Wide Field Trial Data” In _Nature Plants_ 7.10, 2021, pp. 1354–1363 DOI: 10.1038/s41477-021-01001-0
  * [118] A. Orynbaikyzy, U. Gessner, B. Mack and C. Conrad “Crop Type Classification Using Fusion of Sentinel-1 and Sentinel-2 Data: Assessing the Impact of Feature Selection, Optical Data Availability, and Parcel Sizes on the Accuracies” In _Remote Sensing_ 12.17, 2020 DOI: 10.3390/RS12172779
  * [119] J… Rubí and Paulo R.. Gondim “A Performance Comparison of Machine Learning Models for Wildfire Occurrence Risk Prediction in the Brazilian Federal District Region” In _Environment Systems and Decisions_ , 2023 DOI: 10.1007/s10669-023-09921-2
  * [120] Yarui Wu, Honglei Liu, Shuangyue Liu and Chunhui Lou “Estimate of Near-Surface NO2 Concentrations in Fenwei Plain, China, Based on TROPOMI Data and Random Forest Model” In _Environmental Monitoring and Assessment_ 195.11, 2023, pp. 1379 DOI: 10.1007/s10661-023-11993-1
  * [121] J. Xu et al. “Towards Interpreting Multi-Temporal Deep Learning Models in Crop Mapping” In _Remote Sensing of Environment_ 264, 2021 DOI: 10.1016/j.rse.2021.112599
  * [122] X. Yan et al. “A Spatial-Temporal Interpretable Deep Learning Model for Improving Interpretability and Predictive Accuracy of Satellite-Based PM2.5” In _Environmental Pollution_ 273, 2021 DOI: 10.1016/j.envpol.2021.116459
  * [123] B. Chen et al. “Estimation of Atmospheric PM10 Concentration in China Using an Interpretable Deep Learning Model and Top-of-the-Atmosphere Reflectance Data from China’s New Generation Geostationary Meteorological Satellite, FY-4A” In _Journal of Geophysical Research: Atmospheres_ 127.9, 2022 DOI: 10.1029/2021JD036393
  * [124] Bin Chen et al. “Estimation of Near-Surface Ozone Concentration and Analysis of Main Weather Situation in China Based on Machine Learning Model and Himawari-8 TOAR Data” In _Science of the Total Environment_ 864 Elsevier B.V., 2023 DOI: 10.1016/j.scitotenv.2022.160928
  * [125] Bin Chen et al. “Exploring High-Resolution near-Surface CO Concentrations Based on Himawari-8 Top-of-Atmosphere Radiation Data: Assessing the Distribution of City-Level CO Hotspots in China” In _Atmospheric Environment_ 312, 2023 DOI: 10.1016/j.atmosenv.2023.120021
  * [126] Ozlem Sen and Hacer Yalim Keles “On the Evaluation of CNN Models in Remote-Sensing Scene Classification Domain” In _PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science_ 88.6, 2020, pp. 477–492 DOI: 10.1007/s41064-020-00129-6
  * [127] Bhavan Vasu and Andreas Savakis “Resilience and Plasticity of Deep Network Interpretations for Aerial Imagery” In _IEEE access : practical innovations, open solutions_ 8, 2020, pp. 127491–127506 DOI: 10.1109/ACCESS.2020.3008323
  * [128] Anastasios Temenos et al. “Interpretable Deep Learning Framework for Land Use and Land Cover Classification in Remote Sensing Using SHAP” In _IEEE Geoscience and Remote Sensing Letters_ Institute of Electrical and Electronics Engineers Inc., 2023, pp. 1–1 DOI: 10.1109/LGRS.2023.3251652
  * [129] S.-C. Hung, H.-C. Wu and M.-H. Tseng “Remote Sensing Scene Classification and Explanation Using RSSCNet and LIME” In _Applied Sciences (Switzerland)_ 10.18, 2020 DOI: 10.3390/app10186151
  * [130] S.N. Elliott, A.J.B. Shields, E.M. Klaehn and I. Tien “Identifying Critical Infrastructure in Imagery Data Using Explainable Convolutional Neural Networks” In _Remote Sensing_ 14.21, 2022 DOI: 10.3390/rs14215331
  * [131] Antonio Manuel Burgueño et al. “Scalable Approach for High-Resolution Land Cover: A Case Study in the Mediterranean Basin” In _Journal of Big Data_ 10.1, 2023 DOI: 10.1186/s40537-023-00770-z
  * [132] Alexander Brenning “Interpreting Machine-Learning Models in Transformed Feature Space with an Application to Remote-Sensing Classification” In _Machine Learning_ 112.9, 2023, pp. 3455–3471 DOI: 10.1007/s10994-023-06327-8
  * [133] Cassio F. Dantas, Thalita F. Drumond, Diego Marcos and Dino Ienco “Counterfactual Explanations for Remote Sensing Time Series Data: An Application to Land Cover Classification” In _Machine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo Track_ , Lecture Notes in Computer Science Cham: Springer Nature Switzerland, 2023, pp. 20–36 DOI: 10.1007/978-3-031-43430-3˙2
  * [134] Shin-nosuke Ishikawa et al. “Example-Based Explainable AI and Its Application for Remote Sensing Image Classification” In _International Journal of Applied Earth Observation and Geoinformation_ 118 Elsevier B.V., 2023 DOI: 10.1016/j.jag.2023.103215
  * [135] Pattathal V. Arun and Arnon Karnieli “Learning of Physically Significant Features from Earth Observation Data: An Illustration for Crop Classification and Irrigation Scheme Detection” In _Neural Computing and Applications_ 34.13, 2022, pp. 10929–10948 DOI: 10.1007/s00521-022-07019-5
  * [136] L. Han et al. “An Explainable XGBoost Model Improved by SMOTE-ENN Technique for Maize Lodging Detection Based on Multi-Source Unmanned Aerial Vehicle Images” In _Computers and Electronics in Agriculture_ 194, 2022 DOI: 10.1016/j.compag.2022.106804
  * [137] Camilla Broms et al. “Combined Analysis of Satellite and Ground Data for Winter Wheat Yield Forecasting” In _Smart Agricultural Technology_ 3, 2023, pp. 100107 DOI: 10.1016/j.atech.2022.100107
  * [138] Florian Huber, Artem Yushchenko, Benedikt Stratmann and Volker Steinhage “Extreme Gradient Boosting for Yield Estimation Compared with Deep Learning Approaches” In _Computers and Electronics in Agriculture_ 202, 2022, pp. 107346 DOI: 10.1016/j.compag.2022.107346
  * [139] Edward J. Jones et al. “Identifying Causes of Crop Yield Variability with Interpretive Machine Learning” In _Computers and Electronics in Agriculture_ 192, 2022, pp. 106632 DOI: 10.1016/j.compag.2021.106632
  * [140] Harpinder Singh, Ajay Roy, R.K. Setia and Brijendra Pateriya “Simulation of Multispectral Data Using Hyperspectral Data for Crop Stress Studies” In _Lecture Notes in Electrical Engineering_ 970 Springer Science and Business Media Deutschland GmbH, 2023, pp. 43–52 DOI: 10.1007/978-981-19-7698-8˙5
  * [141] Patrick Filippi, Brett M. Whelan, R. Vervoort and Thomas F.. Bishop “Identifying Crop Yield Gaps with Site- and Season-Specific Data-Driven Models of Yield Potential” In _Precision Agriculture_ 23.2, 2022, pp. 578–601 DOI: 10.1007/s11119-021-09850-7
  * [142] Hari Sankar Nayak et al. “Interpretable Machine Learning Methods to Explain On-Farm Yield Variability of High Productivity Wheat in Northwest India” In _Field Crops Research_ 287, 2022, pp. 108640 DOI: 10.1016/j.fcr.2022.108640
  * [143] Manuel Campos-Taberner et al. “Understanding Deep Learning in Land Use Classification Based on Sentinel-2 Time Series” In _Scientific Reports_ 10.1 Nature Publishing Group, 2020, pp. 17188 DOI: 10.1038/s41598-020-74215-5
  * [144] Anna Mateo-Sanchis et al. “Interpretable Long-Short Term Memory Networks for Crop Yield Estimation” In _IEEE Geoscience and Remote Sensing Letters_ , 2023, pp. 1–1 DOI: 10.1109/LGRS.2023.3244064
  * [145] Dilli Paudel et al. “Interpretability of Deep Learning Models for Crop Yield Forecasting” In _Computers and Electronics in Agriculture_ 206, 2023, pp. 107663 DOI: 10.1016/j.compag.2023.107663
  * [146] Ivica Obadic, Ribana Roscher, Dario Augusto Borges Oliveira and Xiao Xiang Zhu “Exploring Self-Attention for Crop-type Classification Explainability” arXiv, 2022 DOI: 10.48550/arXiv.2210.13167
  * [147] Marc Rußwurm and Marco Körner “Self-Attention for Raw Optical Satellite Time Series Classification” In _ISPRS Journal of Photogrammetry and Remote Sensing_ 169, 2020, pp. 421–435 DOI: 10.1016/j.isprsjprs.2020.06.006
  * [148] Vivien Sainte Fare Garnot, Loic Landrieu, Sebastien Giordano and Nesrine Chehata “Satellite Image Time Series Classification With Pixel-Set Encoders and Temporal Self-Attention” In _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020, pp. 12322–12331 DOI: 10.1109/CVPR42600.2020.01234
  * [149] A. Wolanin et al. “Estimating and Understanding Crop Yields with Explainable Deep Learning in the Indian Wheat Belt” In _Environmental Research Letters_ 15.2, 2020 DOI: 10.1088/1748-9326/ab68ac
  * [150] Laura Elena Cue La Rosa, Dario Augusto Borges Oliveira and Pedram Ghamisi “Learning Crop Type Mapping from Regional Label Proportions in Large-Scale SAR and Optical Imagery” In _IEEE Transactions on Geoscience and Remote Sensing_ , 2023, pp. 1–1 DOI: 10.1109/TGRS.2023.3321156
  * [151] A. Mateo-Sanchis et al. “Learning Main Drivers of Crop Progress and Failure in Europe with Interpretable Machine Learning” In _International Journal of Applied Earth Observation and Geoinformation_ 104, 2021 DOI: 10.1016/j.jag.2021.102574
  * [152] Laura Martínez-Ferrer, Maria Piles and Gustau Camps-Valls “Crop Yield Estimation and Interpretability With Gaussian Processes” In _IEEE Geoscience and Remote Sensing Letters_ 18.12, 2021, pp. 2043–2047 DOI: 10.1109/LGRS.2020.3016140
  * [153] Mehmet Furkan Celik et al. “Explainable Artificial Intelligence for Cotton Yield Prediction with Multisource Data” In _IEEE Geoscience and Remote Sensing Letters_ 20, 2023, pp. 1–5 DOI: 10.1109/LGRS.2023.3303643
  * [154] Johnny Vega, Fabio Humberto Sepúlveda-Murillo and Melissa Parra “Landslide Modeling in a Tropical Mountain Basin Using Machine Learning Algorithms and Shapley Additive Explanations” In _Air, Soil and Water Research_ 16, 2023, pp. 11786221231195824 DOI: 10.1177/11786221231195824
  * [155] H.A.H. Al-Najjar et al. “A Novel Method Using Explainable Artificial Intelligence (XAI)-Based Shapley Additive Explanations for Spatial Landslide Prediction Using Time-Series SAR Dataset” In _Gondwana Research_ , 2022 DOI: 10.1016/j.gr.2022.08.004
  * [156] Saeed Alqadhi, Javed Mallick and Meshel Alkahtani “Integrated Deep Learning with Explainable Artificial Intelligence for Enhanced Landslide Management” In _Natural Hazards_ , 2023 DOI: 10.1007/s11069-023-06260-y
  * [157] Ashok Dahal and Luigi Lombardo “Explainable Artificial Intelligence in Geoscience: A Glimpse into the Future of Landslide Susceptibility Modeling” Earth and Space Science Open Archive, 2022 DOI: 10.1002/essoar.10512130.1
  * [158] Muhammad Sakib Khan Inan and Istiakur Rahman “Explainable AI Integrated Feature Selection for Landslide Susceptibility Mapping Using TreeSHAP” In _SN Computer Science_ 4.5, 2023, pp. 482 DOI: 10.1007/s42979-023-01960-5
  * [159] Junyi Zhang et al. “Insights into Geospatial Heterogeneity of Landslide Susceptibility Based on the SHAP-XGBoost Model” In _Journal of environmental management_ 332 NLM (Medline), 2023, pp. 117357 DOI: 10.1016/j.jenvman.2023.117357
  * [160] D. Sun et al. “Assessment of Landslide Susceptibility along Mountain Highways Based on Different Machine Learning Algorithms and Mapping Units by Hybrid Factors Screening and Sample Optimization” In _Gondwana Research_ , 2023 DOI: 10.1016/j.gr.2022.07.013
  * [161] A.E. Maxwell, M. Sharma and K.A. Donaldson “Explainable Boosting Machines for Slope Failure Spatial Predictive Modeling” In _Remote Sensing_ 13.24, 2021 DOI: 10.3390/rs13244991
  * [162] Haoran Fang et al. “A New Approach to Spatial Landslide Susceptibility Prediction in Karst Mining Areas Based on Explainable Artificial Intelligence” In _Sustainability (Switzerland)_ 15.4 MDPI, 2023 DOI: 10.3390/su15043094
  * [163] Khaled Youssef, Kevin Shao, Seulgi Moon and Louis-Serge Bouchard “Landslide Susceptibility Modeling by Interpretable Neural Network” arXiv, 2022 DOI: 10.48550/arXiv.2201.06837
  * [164] Cheng Chen and Lei Fan “An Attribution Deep Learning Interpretation Model for Landslide Susceptibility Mapping in the Three Gorges Reservoir Area” In _IEEE Transactions on Geoscience and Remote Sensing_ , 2023, pp. 1–1 DOI: 10.1109/TGRS.2023.3323668
  * [165] Halit Enes Aydin and Muzaffer Can Iban “Predicting and Analyzing Flood Susceptibility Using Boosting-Based Ensemble Machine Learning Algorithms with SHapley Additive exPlanations” In _Natural Hazards_ , 2022 DOI: 10.1007/s11069-022-05793-y
  * [166] Junfei Liu, Kai Liu and Ming Wang “A Residual Neural Network Integrated with a Hydrological Model for Global Flood Susceptibility Mapping Based on Remote Sensing Datasets” In _Remote Sensing_ 15.9, 2023 DOI: 10.3390/rs15092447
  * [167] Mo Wang et al. “An XGBoost-SHAP Approach to Quantifying Morphological Impact on Urban Flooding Susceptibility” In _Ecological Indicators_ 156, 2023, pp. 111137 DOI: 10.1016/j.ecolind.2023.111137
  * [168] Ziyang Zhang et al. “An Interpretable Deep Semantic Segmentation Method for Earth Observation” In _2022 IEEE 11th International Conference on Intelligent Systems (IS)_ , 2022, pp. 1–8 DOI: 10.1109/IS57118.2022.10019621
  * [169] Abolfazl Abdollahi and Biswajeet Pradhan “Explainable Artificial Intelligence (XAI) for Interpreting the Contributing Factors Feed into the Wildfire Susceptibility Prediction Model” In _Science of the Total Environment_ 879, 2023 DOI: 10.1016/j.scitotenv.2023.163004
  * [170] Roberto Cilli et al. “Explainable Artificial Intelligence (XAI) Detects Wildfire Occurrence in the Mediterranean Countries of Southern Europe” In _Scientific Reports_ 12.1 Nature Publishing Group, 2022, pp. 16349 DOI: 10.1038/s41598-022-20347-9
  * [171] Alan H. Taylor et al. “Spatial Patterns of Nineteenth Century Fire Severity Persist after Fire Exclusion and a Twenty-First Century Wildfire in a Mixed Conifer Forest Landscape, Southern Cascades, USA” In _Landscape Ecology_ 35.12, 2020, pp. 2777–2790 DOI: 10.1007/s10980-020-01118-1
  * [172] Nandini Saini, Chiranjoy Chattopadhyay and Debasis Das “E2AlertNet: An Explainable, Efficient, and Lightweight Model for Emergency Alert from Aerial Imagery” In _Remote Sensing Applications: Society and Environment_ 29 Elsevier B.V., 2023 DOI: 10.1016/j.rsase.2022.100896
  * [173] Teo Beker, Qian Song and Xiao Xiang Zhu “An Analysis of the Gap between Hybrid and Real Data for Volcanic Deformation Detection” In _IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium_ , 2023, pp. 825–828 DOI: 10.1109/IGARSS52108.2023.10281964
  * [174] Zhe Chen et al. “Tunnel Geothermal Disaster Susceptibility Evaluation Based on Interpretable Ensemble Learning: A Case Study in Ya’an–Changdu Section of the Sichuan–Tibet Traffic Corridor” In _Engineering Geology_ 313 Elsevier B.V., 2023 DOI: 10.1016/j.enggeo.2023.106985
  * [175] Ratiranjan Jena et al. “Explainable Artificial Intelligence (XAI) Model for Earthquake Spatial Probability Assessment in Arabian Peninsula” In _Remote Sensing_ 15.9, 2023 DOI: 10.3390/rs15092248
  * [176] A. Levering, D. Marcos and D. Tuia “On the Relation between Landscape Beauty and Land Cover: A Case Study in the U.K. at Sentinel-2 Resolution with Interpretable AI” In _ISPRS Journal of Photogrammetry and Remote Sensing_ 177, 2021, pp. 194–203 DOI: 10.1016/j.isprsjprs.2021.04.020
  * [177] Alex Levering, Diego Marcos, Jasper van Vliet and Devis Tuia “Predicting the Liveability of Dutch Cities with Aerial Images and Semantic Intermediate Concepts” In _Remote Sensing of Environment_ 287, 2023, pp. 113454 DOI: 10.1016/j.rse.2023.113454
  * [178] Songyan Zhu et al. “Investigating Impacts of Ambient Air Pollution on the Terrestrial Gross Primary Productivity (GPP) from Remote Sensing” In _IEEE Geoscience and Remote Sensing Letters_ 19, 2022, pp. 1–5 DOI: 10.1109/LGRS.2022.3163775
  * [179] Callie B. Lambert, Lynn M. Resler, Yang Shao and David R. Butler “Vegetation Change as Related to Terrain Factors at Two Glacier Forefronts, Glacier National Park, Montana, U.S.A.” In _Journal of Mountain Science_ 17.1, 2020, pp. 1–15 DOI: 10.1007/s11629-019-5603-8
  * [180] Huimin Zhou et al. “Relative Importance of Climatic Variables, Soil Properties and Plant Traits to Spatial Variability in Net CO2 Exchange across Global Forests and Grasslands” In _Agricultural and Forest Meteorology_ 307, 2021, pp. 108506 DOI: 10.1016/j.agrformet.2021.108506
  * [181] Shujie Cheng et al. “Improved Understanding of How Catchment Properties Control Hydrological Partitioning Through Machine Learning” In _Water Resources Research_ 58.4, 2022, pp. e2021WR031412 DOI: 10.1029/2021WR031412
  * [182] C. Karmakar, C.O. Dumitru, G. Schwarz and M. Datcu “Feature-Free Explainable Data Mining in SAR Images Using Latent Dirichlet Allocation” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 14, 2021, pp. 676–689 DOI: 10.1109/JSTARS.2020.3039012
  * [183] Chandrabali Karmakar and Mihai Datcu “A Framework for Interactive Visual Interpretation of Remote Sensing Data” In _IEEE Geoscience and Remote Sensing Letters_ 19, 2022, pp. 1–5 DOI: 10.1109/LGRS.2022.3161959
  * [184] D. Stroppiana et al. “A Fully Automatic, Interpretable and Adaptive Machine Learning Approach to Map Burned Area from Remote Sensing” In _ISPRS International Journal of Geo-Information_ 10.8, 2021 DOI: 10.3390/ijgi10080546
  * [185] Hugo Leon-Garza et al. “A Big Bang-Big Crunch Type-2 Fuzzy Logic System for Explainable Semantic Segmentation of Trees in Satellite Images Using HSV Color Space” In _2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)_ , 2020, pp. 1–7 DOI: 10.1109/FUZZ48607.2020.9177611
  * [186] Bryce Murray et al. “Explainable AI for Understanding Decisions and Data-Driven Optimization of the Choquet Integral” In _2018 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)_ , 2018, pp. 1–8 DOI: 10.1109/FUZZ-IEEE.2018.8491501
  * [187] Muhammad Aminul Islam et al. “Enabling Explainable Fusion in Deep Learning With Fuzzy Integral Neural Networks” In _IEEE Transactions on Fuzzy Systems_ 28.7, 2020, pp. 1291–1300 DOI: 10.1109/TFUZZ.2019.2917124
  * [188] B.J. Murray et al. “Information Fusion-2-Text: Explainable Aggregation via Linguistic Protoforms” In _Communications in Computer and Information Science_ 1239 CCIS, 2020, pp. 114–127 DOI: 10.1007/978-3-030-50153-2˙9
  * [189] Derek T. Anderson et al. “Fuzzy Choquet Integration of Deep Convolutional Neural Networks for Remote Sensing” In _Computational Intelligence for Pattern Recognition_ Cham: Springer International Publishing, 2018, pp. 1–28 DOI: 10.1007/978-3-319-89629-8˙1
  * [190] Yansheng Li, Yongjun Zhang, Xin Huang and Alan L. Yuille “Deep Networks under Scene-Level Supervision for Multi-Class Geospatial Object Detection from Remote Sensing Images” In _ISPRS Journal of Photogrammetry and Remote Sensing_ 146, 2018, pp. 182–196 DOI: 10.1016/j.isprsjprs.2018.09.014
  * [191] Wei Xiong et al. “An Interpretable Fusion Siamese Network for Multi-modality Remote Sensing Ship Image Retrieval” In _IEEE Transactions on Circuits and Systems for Video Technology_ , 2022, pp. 1–1 DOI: 10.1109/TCSVT.2022.3224068
  * [192] Xianpeng Guo et al. “Visual Explanations with Detailed Spatial Information for Remote Sensing Image Classification via Channel Saliency” In _International Journal of Applied Earth Observation and Geoinformation_ 118, 2023, pp. 103244 DOI: 10.1016/j.jag.2023.103244
  * [193] Jia Deng et al. “Imagenet: A large-scale hierarchical image database” In _2009 IEEE conference on computer vision and pattern recognition_ , 2009, pp. 248–255 Ieee
  * [194] Zhenpeng Feng, Mingzhe Zhu, Ljubiša Stanković and Hongbing Ji “Self-Matching CAM: A Novel Accurate Visual Explanation of CNNs for SAR Image Interpretation” In _Remote Sensing_ 13.9 Multidisciplinary Digital Publishing Institute, 2021, pp. 1772 DOI: 10.3390/rs13091772
  * [195] Haofan Wang et al. “Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks” In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2020, pp. 24–25
  * [196] Seyed Mojtaba Marvasti-Zadeh, Devin Goodsman, Nilanjan Ray and Nadir Erbilgin “Crown-CAM: Interpretable Visual Explanations for Tree Crown Detection in Aerial Images” In _IEEE Geoscience and Remote Sensing Letters_ 20, 2023, pp. 1–5 DOI: 10.1109/LGRS.2023.3271649
  * [197] G. De Lucia, M. Lapegna and D. Romano “Towards Explainable AI for Hyperspectral Image Classification in Edge Computing Environments” In _Computers and Electrical Engineering_ 103, 2022 DOI: 10.1016/j.compeleceng.2022.108381
  * [198] Wei Song et al. “Median-Pooling Grad-CAM: An Efficient Inference Level Visual Explanation for CNN Networks in Remote Sensing Image Classification” In _MultiMedia Modeling_ Cham: Springer International Publishing, 2021, pp. 134–146
  * [199] Ji Ge et al. “Interpretable Deep Learning Method Combining Temporal Backscattering Coefficients and Interferometric Coherence for Rice Area Mapping” In _IEEE Geoscience and Remote Sensing Letters_ 20, 2023, pp. 1–5 DOI: 10.1109/LGRS.2023.3321770
  * [200] Elizabeth A. Barnes, Randal J. Barnes, Zane K. Martin and Jamin K. Rader “This Looks Like That There: Interpretable Neural Networks for Image Tasks When Location Matters” In _Artificial Intelligence for the Earth Systems_ 1.3, 2022, pp. e220001 DOI: 10.1175/AIES-D-22-0001.1
  * [201] X. Huang et al. “Better Visual Interpretation for Remote Sensing Scene Classification” In _IEEE Geoscience and Remote Sensing Letters_ 19, 2022 DOI: 10.1109/LGRS.2021.3132920
  * [202] Alexander Brenning “Spatial Machine-Learning Model Diagnostics: A Model-Agnostic Distance-Based Approach” In _International Journal of Geographical Information Science_ 37.3, 2023, pp. 584–606 DOI: 10.1080/13658816.2022.2131789
  * [203] Giuseppina Andresini, Annalisa Appice and Donato Malerba “SILVIA: An eXplainable Framework to Map Bark Beetle Infestation in Sentinel-2 Images” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , 2023, pp. 1–17 DOI: 10.1109/JSTARS.2023.3312521
  * [204] J.L. Abitbol and M. Karsai “Interpretable Socioeconomic Status Inference from Aerial Imagery through Urban Patterns” In _Nature Machine Intelligence_ 2.11, 2020, pp. 684–692 DOI: 10.1038/s42256-020-00243-5
  * [205] Xun Li et al. “Explainable Dimensionality Reduction (XDR) to Unbox AI ‘Black Box’ Models: A Study of AI Perspectives on the Ethnic Styles of Village Dwellings” In _Humanities and Social Sciences Communications_ 10.1 Palgrave, 2023, pp. 1–13 DOI: 10.1057/s41599-023-01505-4
  * [206] Katalin Blix, Gustau Camps-Valls and Robert Jenssen “Gaussian Process Sensitivity Analysis for Oceanic Chlorophyll Estimation” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 10.4, 2017, pp. 1265–1277 DOI: 10.1109/JSTARS.2016.2641583
  * [207] Wenyi Zhang et al. “Feature Importance Measure of a Multilayer Perceptron Based on the Presingle-Connection Layer” In _Knowledge and Information Systems_ , 2023 DOI: 10.1007/s10115-023-01959-7
  * [208] I. Kakogeorgiou and K. Karantzalos “Evaluating Explainable Artificial Intelligence Methods for Multi-Label Deep Learning Classification Tasks in Remote Sensing” In _International Journal of Applied Earth Observation and Geoinformation_ 103, 2021 DOI: 10.1016/j.jag.2021.102520
  * [209] Taewoo Kim et al. “Federated Onboard-Ground Station Computing with Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis” In _IEEE access : practical innovations, open solutions_ 10, 2022, pp. 117315–117333 DOI: 10.1109/ACCESS.2022.3219879
  * [210] Jan-Peter Kucklick and Oliver Müller “Tackling the Accuracy-Interpretability Trade-off: Interpretable Deep Learning Models for Satellite Image-Based Real Estate Appraisal” In _ACM Transactions on Management Information Systems_ 14.1, 2023 DOI: 10.1145/3567430
  * [211] Qi Su et al. “Which CAM Is Better for Extracting Geographic Objects? A Perspective From Principles and Experiments” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 15, 2022, pp. 5623–5635 DOI: 10.1109/JSTARS.2022.3188493
  * [212] A. Abdollahi and B. Pradhan “Urban Vegetation Mapping from Aerial Imagery Using Explainable AI (XAI)” In _Sensors_ 21.14, 2021 DOI: 10.3390/s21144738
  * [213] Spyros Kondylatos et al. “Wildfire Danger Prediction and Understanding with Deep Learning” In _Geophysical Research Letters_ n/a.n/a, 2022, pp. e2022GL099368 DOI: 10.1029/2022GL099368
  * [214] S. Biass et al. “Insights into the Vulnerability of Vegetation to Tephra Fallouts from Interpretable Machine Learning and Big Earth Observation Data” In _Natural Hazards and Earth System Sciences_ 22.9, 2022, pp. 2829–2855 DOI: 10.5194/nhess-22-2829-2022
  * [215] Timo T. Stomberg, Taylor Stone, Johannes Leonhardt and Ribana Roscher “Exploring Wilderness Using Explainable Machine Learning in Satellite Imagery” In _arXiv:2203.00379 [cs]_ , 2022 arXiv:2203.00379 [cs]
  * [216] Sam J. Silva, Christoph A. Keller and Joseph Hardin “Using an Explainable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence” In _Journal of Advances in Modeling Earth Systems_ 14.4, 2022, pp. e2021MS002881 DOI: 10.1029/2021MS002881
  * [217] B. Hosseiny, A.M. Abdi and S. Jamali “Urban Land Use and Land Cover Classification with Interpretable Machine Learning – A Case Study Using Sentinel-2 and Auxiliary Data” In _Remote Sensing Applications: Society and Environment_ 28, 2022 DOI: 10.1016/j.rsase.2022.100843
  * [218] Jakob Gawlikowski, Patrick Ebel, Michael Schmitt and Xiao Xiang Zhu “Explaining the Effects of Clouds on Remote Sensing Scene Classification” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 15, 2022, pp. 9976–9986 DOI: 10.1109/JSTARS.2022.3221788
  * [219] Neelesh Rampal et al. “High-Resolution Downscaling with Interpretable Deep Learning: Rainfall Extremes over New Zealand” In _Weather and Climate Extremes_ 38, 2022, pp. 100525 DOI: 10.1016/j.wace.2022.100525
  * [220] Patrick W Keys, Elizabeth A Barnes and Neil H Carter “A Machine-Learning Approach to Human Footprint Index Estimation with Applications to Sustainable Development” In _Environmental Research Letters_ 16.4, 2021, pp. 044061 DOI: 10.1088/1748-9326/abe00a
  * [221] Teo Beker et al. “Deep Learning for Subtle Volcanic Deformation Detection with InSAR Data in Central Volcanic Zone” In _IEEE Transactions on Geoscience and Remote Sensing_ 61, 2023, pp. 1–20 DOI: 10.1109/TGRS.2023.3318469
  * [222] Zhitong Xiong, Sining Chen, Yilei Shi and Xiao Xiang Zhu “Disentangled Latent Transformer for Interpretable Monocular Height Estimation” arXiv, 2022 DOI: 10.48550/arXiv.2201.06357
  * [223] Miltiadis Iatrou, Christos Karydas, Xanthi Tseni and Spiros Mourelatos “Representation Learning with a Variational Autoencoder for Predicting Nitrogen Requirement in Rice” In _Remote Sensing_ 14.23 MDPI, 2022 DOI: 10.3390/rs14235978
  * [224] Nana Luo et al. “Explainable and Spatial Dependence Deep Learning Model for Satellite-Based O3 Monitoring in China” In _Atmospheric Environment_ , 2022, pp. 119370 DOI: 10.1016/j.atmosenv.2022.119370
  * [225] Julio J. Valdés and Antonio Pou “Explainable AI Applied to the Analysis of the Climatic Behavior of 11 Years of Meteosat Water Vapor Images” In _2022 IEEE Symposium Series on Computational Intelligence (SSCI)_ , 2022, pp. 846–853 DOI: 10.1109/SSCI51031.2022.10022301
  * [226] X. Yan et al. “New Interpretable Deep Learning Model to Monitor Real-Time PM2.5 Concentrations from Satellite Data” In _Environment International_ 144, 2020 DOI: 10.1016/j.envint.2020.106060
  * [227] X. Yan, Z. Zang, C. Zhao and L. Husi “Understanding Global Changes in Fine-Mode Aerosols during 2008–2017 Using Statistical Methods and Deep Learning Approach” In _Environment International_ 149, 2021 DOI: 10.1016/j.envint.2021.106392
  * [228] Paolo Maranzano, Philipp Otto and Alessandro Fassò “Adaptive LASSO Estimation for Functional Hidden Dynamic Geostatistical Models” In _Stochastic Environmental Research and Risk Assessment_ 37.9, 2023, pp. 3615–3637 DOI: 10.1007/s00477-023-02466-5
  * [229] Zohre Ebrahimi-Khusfi et al. “Determining the Contribution of Environmental Factors in Controlling Dust Pollution during Cold and Warm Months of Western Iran Using Different Data Mining Algorithms and Game Theory” In _Ecological Indicators_ 132, 2021, pp. 108287 DOI: 10.1016/j.ecolind.2021.108287
  * [230] Yanchuan Shao et al. “Estimation of Daily NO2 with Explainable Machine Learning Model in China, 2007–2020” In _Atmospheric Environment_ 314, 2023 DOI: 10.1016/j.atmosenv.2023.120111
  * [231] Shuai Wang et al. “Estimating Particulate Matter Concentrations and Meteorological Contributions in China during 2000–2020” In _Chemosphere_ 330, 2023 DOI: 10.1016/j.chemosphere.2023.138742
  * [232] L. Zipfel, H. Andersen and J. Cermak “Machine-Learning Based Analysis of Liquid Water Path Adjustments to Aerosol Perturbations in Marine Boundary Layer Clouds Using Satellite Observations” In _Atmosphere_ 13.4, 2022 DOI: 10.3390/atmos13040586
  * [233] Lu Liang et al. “Integrating Low-Cost Sensor Monitoring, Satellite Mapping, and Geospatial Artificial Intelligence for Intra-Urban Air Pollution Predictions” In _Environmental Pollution_ 331, 2023 DOI: 10.1016/j.envpol.2023.121832
  * [234] Yang Zhen and Guoping Shi “Evaluation of MACC Total Aerosol Optical Depth and Its Correction Model Based on the Random Forest Regression” In _Theoretical and Applied Climatology_ 152.3-4, 2023, pp. 1243–1258 DOI: 10.1007/s00704-023-04455-8
  * [235] Julio J. Valdés and Antonio Pou “A Machine Learning - Explainable AI Approach to Tropospheric Dynamics Analysis Using Water Vapor Meteosat Images” In _2021 IEEE Symposium Series on Computational Intelligence (SSCI)_ , 2021, pp. 1–8 DOI: 10.1109/SSCI50451.2021.9660188
  * [236] S. Zhang et al. “A Data-Augmentation Approach to Deriving Long-Term Surface SO2 across Northern China: Implications for Interpretable Machine Learning” In _Science of the Total Environment_ 827, 2022 DOI: 10.1016/j.scitotenv.2022.154278
  * [237] C.-S. Cheng, A.H. Behzadan and A. Noshadravan “Uncertainty-Aware Convolutional Neural Network for Explainable Artificial Intelligence-Assisted Disaster Damage Assessment” In _Structural Control and Health Monitoring_ 29.10, 2022 DOI: 10.1002/stc.3019
  * [238] Erfan Hasanpour Zaryabi et al. “Unboxing the Black Box of Attention Mechanisms in Remote Sensing Big Data Using XAI” In _Remote Sensing_ 14.24 MDPI, 2022 DOI: 10.3390/rs14246254
  * [239] Seyd Teymoor Seydi, Mahdi Hasanlou, Jocelyn Chanussot and Pedram Ghamisi “BDD-Net+: A Building Damage Detection Framework Based on Modified Coat-Net” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 16, 2023, pp. 4232–4247 DOI: 10.1109/JSTARS.2023.3267847
  * [240] Seyd Teymoor Seydi, Mahdi Hasanlou, Jocelyn Chanussot and Pedram Ghamisi “Leveraging Involution and Convolution in an Explainable Building Damage Detection Framework” In _European Journal of Remote Sensing_ 0.0 Taylor & Francis, 2023, pp. 2252166 DOI: 10.1080/22797254.2023.2252166
  * [241] Adrià Descals et al. “Local Interpretation of Machine Learning Models in Remote Sensing with SHAP: The Case of Global Climate Constraints on Photosynthesis Phenology” In _International Journal of Remote Sensing_ 44.10, 2023, pp. 3160–3173 DOI: 10.1080/01431161.2023.2217982
  * [242] L. Li et al. “Exploring the Individualized Effect of Climatic Drivers on MODIS Net Primary Productivity through an Explainable Machine Learning Framework” In _Remote Sensing_ 14.17, 2022 DOI: 10.3390/rs14174401
  * [243] Songyan Zhu et al. “Explainable Machine Learning Confirms the Global Terrestrial CO2 Fertilization Effect from Space” In _IEEE Geoscience and Remote Sensing Letters_ 20, 2023, pp. 1–5 DOI: 10.1109/LGRS.2023.3298373
  * [244] Sanja Scepanovic, Sagar Joglekar, Stephen Law and Daniele Quercia “Jane Jacobs in the Sky: Predicting Urban Vitality with Open Satellite Data” In _Proceedings of the ACM on Human-Computer Interaction_ 5.CSCW1, 2021, pp. 1–25 DOI: 10.1145/3449257
  * [245] Liujia Chen et al. “Measuring Impacts of Urban Environmental Elements on Housing Prices Based on Multisource Data—A Case Study of Shanghai, China” In _ISPRS International Journal of Geo-Information_ 9.2 Multidisciplinary Digital Publishing Institute, 2020, pp. 106 DOI: 10.3390/ijgi9020106
  * [246] Zhongao Ding et al. “Residential Greenness and Cardiac Conduction Abnormalities: Epidemiological Evidence and an Explainable Machine Learning Modeling Study” In _Chemosphere_ 339, 2023 DOI: 10.1016/j.chemosphere.2023.139671
  * [247] M. Kim and G. Kim “Modeling and Predicting Urban Expansion in South Korea Using Explainable Artificial Intelligence (XAI) Model” In _Applied Sciences (Switzerland)_ 12.18, 2022 DOI: 10.3390/app12189169
  * [248] Sanja Scepanovic et al. “MedSat: A Public Health Dataset for England Featuring Medical Prescriptions and Satellite Imagery” In _Thirty-Seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track_ , 2023
  * [249] A. Temenos et al. “Novel Insights in Spatial Epidemiology Utilizing Explainable AI (XAI) and Remote Sensing” In _Remote Sensing_ 14.13, 2022 DOI: 10.3390/rs14133074
  * [250] Siqin Wang et al. “Unpacking the Inter- and Intra-Urban Differences of the Association between Health and Exposure to Heat and Air Quality in Australia Using Global and Local Machine Learning Models” In _Science of The Total Environment_ 871, 2023, pp. 162005 DOI: 10.1016/j.scitotenv.2023.162005
  * [251] P. Wójcik and K. Andruszek “Predicting Intra-Urban Well-Being from Space with Nonlinear Machine Learning” In _Regional Science Policy and Practice_ 14.4, 2022, pp. 891–913 DOI: 10.1111/rsp3.12478
  * [252] Salvin S. Prasad et al. “Enhanced Joint Hybrid Deep Neural Network Explainable Artificial Intelligence Model for 1-Hr Ahead Solar Ultraviolet Index Prediction” In _Computer Methods and Programs in Biomedicine_ 241, 2023 DOI: 10.1016/j.cmpb.2023.107737
  * [253] Katalin Blix, Ana Belen Ruescas, Juan Emmanuel Johnson and Gustau Camps-Valls “Learning Relevant Features of Optical Water Types” In _IEEE Geoscience and Remote Sensing Letters_ 19, 2022, pp. 1–5 DOI: 10.1109/LGRS.2021.3072049
  * [254] Lifu Chen et al. “Towards Transparent Deep Learning for Surface Water Detection from SAR Imagery” In _International Journal of Applied Earth Observation and Geoinformation_ 118, 2023 DOI: 10.1016/j.jag.2023.103287
  * [255] S.M. Hong et al. “Monitoring the Vertical Distribution of HABs Using Hyperspectral Imagery and Deep Learning Models” In _Science of the Total Environment_ 794, 2021 DOI: 10.1016/j.scitotenv.2021.148592
  * [256] Yumin Liu, Kate Duffy, Jennifer G. Dy and Auroop R. Ganguly “Explainable Deep Learning for Insights in El Niño and River Flows” In _Nature Communications_ 14.1 Nature Research, 2023 DOI: 10.1038/s41467-023-35968-5
  * [257] Neelesh Rampal, Tom Shand, Adam Wooler and Christo Rautenbach “Interpretable Deep Learning Applied to Rip Current Detection and Localization” In _Remote Sensing_ 14.23 MDPI, 2022 DOI: 10.3390/rs14236048
  * [258] Elif Ozlem Yilmaz, Hasan Tonbul and Taskin Kavzoglu “Marine Mucilage Mapping with Explained Deep Learning Model Using Water-Related Spectral Indices: A Case Study of Dardanelles Strait, Turkey” In _Stochastic Environmental Research and Risk Assessment_ , 2023 DOI: 10.1007/s00477-023-02560-8
  * [259] Xin Jing, Jungang Luo, Ganggang Zuo and Xue Yang “Interpreting Runoff Forecasting of Long Short-Term Memory Network: An Investigation Using the Integrated Gradient Method on Runoff Data from the Han River Basin” In _Journal of Hydrology: Regional Studies_ 50, 2023, pp. 101549 DOI: 10.1016/j.ejrh.2023.101549
  * [260] J. Pyo et al. “Cyanobacteria Cell Prediction Using Interpretable Deep Learning Model with Observed, Numerical, and Sensing Data Assemblage” In _Water Research_ 203, 2021 DOI: 10.1016/j.watres.2021.117483
  * [261] Ziming Hu et al. “Water Storage Changes (2003–2020) in the Ordos Basin, China, Explained by GRACE Data and Interpretable Deep Learning” In _Hydrogeology Journal_ , 2023 DOI: 10.1007/s10040-023-02713-7
  * [262] Vahideh Saeidi et al. “Water Depth Estimation from Sentinel-2 Imagery Using Advanced Machine Learning Methods and Explainable Artificial Intelligence” In _Geomatics, Natural Hazards and Risk_ 14.1, 2023 DOI: 10.1080/19475705.2023.2225691
  * [263] S.-C. Hung, H.-C. Wu and M.-H. Tseng “Integrating Image Quality Enhancement Methods and Deep Learning Techniques for Remote Sensing Scene Classification” In _Applied Sciences (Switzerland)_ 11.24, 2021 DOI: 10.3390/app112411659
  * [264] Minsu Jeon et al. “Recursive Visual Explanations Mediation Scheme Based on DropAttention Model with Multiple Episodes Pool” In _IEEE access : practical innovations, open solutions_ 11 Institute of Electrical and Electronics Engineers Inc., 2023, pp. 4306–4321 DOI: 10.1109/ACCESS.2023.3235332
  * [265] Jianda Cheng et al. “PolSAR Image Land Cover Classification Based on Hierarchical Capsule Network” In _Remote Sensing_ 13.16 MDPI AG, 2021 DOI: 10.3390/rs13163132
  * [266] Daniel Guidici and Matthew L. Clark “One-Dimensional Convolutional Neural Network Land-Cover Classification of Multi-Seasonal Hyperspectral Imagery in the San Francisco Bay Area, California” In _Remote Sensing_ 9.6 Multidisciplinary Digital Publishing Institute, 2017, pp. 629 DOI: 10.3390/rs9060629
  * [267] Dino Ienco, Yawogan Jean Eudes Gbodjo, Raffaele Gaetano and Roberto Interdonato “Weakly Supervised Learning for Land Cover Mapping of Satellite Image Time Series via Attention-Based CNN” In _IEEE access : practical innovations, open solutions_ 8, 2020, pp. 179547–179560 DOI: 10.1109/ACCESS.2020.3024133
  * [268] N. Méger et al. “Explaining a Deep SpatioTemporal Land Cover Classifier with Attention and Redescription Mining” 43, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives, 2022, pp. 673–680 DOI: 10.5194/isprs-archives-XLIII-B3-2022-673-2022
  * [269] J. Feng, D. Wang and Z. Gu “Bidirectional Flow Decision Tree for Reliable Remote Sensing Image Scene Classification” In _Remote Sensing_ 14.16, 2022 DOI: 10.3390/rs14163943
  * [270] Z. Huang et al. “Physically Explainable CNN for SAR Image Classification” In _ISPRS Journal of Photogrammetry and Remote Sensing_ 190, 2022, pp. 25–37 DOI: 10.1016/j.isprsjprs.2022.05.008
  * [271] Siteng Ma et al. “Multicrop Fusion Strategy Based on Prototype Assignment for Remote Sensing Image Scene Classification” In _IEEE Transactions on Geoscience and Remote Sensing_ 60, 2022, pp. 1–12 DOI: 10.1109/TGRS.2022.3216831
  * [272] Muskan Verma, Nayan Gupta, Bhavishya Tolani and Rishabh Kaushal “Explainable Custom CNN Architecture for Land Use Classification Using Satellite Images” In _2021 Sixth International Conference on Image Information Processing (ICIIP)_ 6, 2021, pp. 304–309 DOI: 10.1109/ICIIP53038.2021.9702698
  * [273] Muzaffer Can Iban and Suleyman Sefa Bilgilioglu “Snow Avalanche Susceptibility Mapping Using Novel Tree-Based Machine Learning Algorithms (XGBoost, NGBoost, and LightGBM) with eXplainable Artificial Intelligence (XAI) Approach” In _Stochastic Environmental Research and Risk Assessment_ , 2023 DOI: 10.1007/s00477-023-02392-6
  * [274] Feini Huang et al. “Interpreting Conv-LSTM for Spatio-Temporal Soil Moisture Prediction in China” In _Agriculture (Switzerland)_ 13.5, 2023 DOI: 10.3390/agriculture13050971
  * [275] Ravil I. Mukhamediev et al. “Soil Salinity Estimation for South Kazakhstan Based on SAR Sentinel-1 and Landsat-8,9 OLI Data with Machine Learning Models” In _Remote Sensing_ 15.17, 2023 DOI: 10.3390/rs15174269
  * [276] B. Pradhan et al. “A New Method to Evaluate Gold Mineralisation-Potential Mapping Using Deep Learning and an Explainable Artificial Intelligence (XAI) Model” In _Remote Sensing_ 14.18, 2022 DOI: 10.3390/rs14184486
  * [277] Yanan Zhou et al. “Identification of Soil Texture Classes under Vegetation Cover Based on Sentinel-2 Data with SVM and SHAP Techniques” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 15, 2022, pp. 3758–3770 DOI: 10.1109/JSTARS.2022.3164140
  * [278] I.. Smorkalov “Soil Respiration Variability: Contributions of Space and Time Estimated Using the Random Forest Algorithm” In _Russian Journal of Ecology_ 53.4, 2022, pp. 295–307 DOI: 10.1134/S1067413622040051
  * [279] Z.M. Labe and E.A. Barnes “Predicting Slowdowns in Decadal Climate Warming Trends with Explainable Neural Networks” In _Geophysical Research Letters_ 49.9, 2022 DOI: 10.1029/2022GL098173
  * [280] Yuhong Hu, Chaofan Wu, Michael E. Meadows and Meili Feng “Pixel Level Spatial Variability Modeling Using SHAP Reveals the Relative Importance of Factors Influencing LST” In _Environmental Monitoring and Assessment_ 195.3, 2023, pp. 407 DOI: 10.1007/s10661-023-10950-2
  * [281] Minjun Kim, Dongbeom Kim and Geunhan Kim “Examining the Relationship between Land Use/Land Cover (LULC) and Land Surface Temperature (LST) Using Explainable Artificial Intelligence (XAI) Models: A Case Study of Seoul, South Korea” In _International Journal of Environmental Research and Public Health_ 19.23 MDPI, 2022 DOI: 10.3390/ijerph192315926
  * [282] Pinyang Luo et al. “Understanding the Relationship between 2D/3D Variables and Land Surface Temperature in Plain and Mountainous Cities: Relative Importance and Interaction Effects” In _Building and Environment_ , 2023, pp. 110959 DOI: 10.1016/j.buildenv.2023.110959
  * [283] Yanting Shen et al. “Using GeoAI to Reveal the Contribution of Urban Park Green Space Features to Mitigate the Heat Island Effect” 2, Proceedings of the International Conference on Education and Research in Computer Aided Architectural Design in Europe, 2023, pp. 49–58
  * [284] Zhonghao Li et al. “Automatic Bridge Detection of SAR Images Based on Interpretable Deep Learning Algorithm” 2562, Journal of Physics: Conference Series, 2023 DOI: 10.1088/1742-6596/2562/1/012013
  * [285] Ru Luo et al. “Glassboxing Deep Learning to Enhance Aircraft Detection from Sar Imagery” In _Remote Sensing_ 13.18 MDPI, 2021 DOI: 10.3390/rs13183650
  * [286] Shoulin Yin et al. “G2Grad-CAMRL: An Object Detection and Interpretation Model Based on Gradient-Weighted Class Activation Mapping and Reinforcement Learning in Remote Sensing Images” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , 2023, pp. 1–16 DOI: 10.1109/JSTARS.2023.3241405
  * [287] Peng Li, Cunqian Feng, Xiaowei Hu and Zixiang Tang “SAR-BagNet: An Ante-hoc Interpretable Recognition Model Based on Deep Network for SAR Image” In _Remote Sensing_ 14.9 Multidisciplinary Digital Publishing Institute, 2022, pp. 2150 DOI: 10.3390/rs14092150
  * [288] Bowen Peng and Bo Peng “Clutter-Invariant Regularization for DNN-based SAR Target Recognition” In _2023 6th International Conference on Electronics Technology (ICET)_ , 2023, pp. 1456–1461 DOI: 10.1109/ICET58434.2023.10211978
  * [289] Mingzhe Zhu et al. “LIME-Based Data Selection Method for SAR Images Generation Using GAN” In _Remote Sensing_ 14.1 Multidisciplinary Digital Publishing Institute, 2022, pp. 204 DOI: 10.3390/rs14010204
  * [290] Sijia Feng et al. “PAN: Part Attention Network Integrating Electromagnetic Characteristics for Interpretable SAR Vehicle Target Recognition” In _IEEE Transactions on Geoscience and Remote Sensing_ , 2023, pp. 1–1 DOI: 10.1109/TGRS.2023.3256399
  * [291] Dandan Guo, Bo Chen, Meixi Zheng and Hongwei Liu “SAR Automatic Target Recognition Based on Supervised Deep Variational Autoencoding Model” In _IEEE Transactions on Aerospace and Electronic Systems_ 57.6, 2021, pp. 4313–4328 DOI: 10.1109/TAES.2021.3096868
  * [292] Peng Li et al. “SAR-AD-BagNet: An Interpretable Model for SAR Image Recognition Based on Adversarial Defense” In _IEEE Geoscience and Remote Sensing Letters_ , 2022, pp. 1–1 DOI: 10.1109/LGRS.2022.3230243
  * [293] Min Zhou et al. “Local Attention Networks for Occluded Airplane Detection in Remote Sensing Images” In _IEEE Geoscience and Remote Sensing Letters_ 17.3, 2020, pp. 381–385 DOI: 10.1109/LGRS.2019.2924822
  * [294] H. Kawauchi and T. Fuse “SHAP-Based Interpretable Object Detection Method for Satellite Imagery” In _Remote Sensing_ 14.9, 2022 DOI: 10.3390/rs14091970
  * [295] Mandeep, Husanbir Singh Pannu and Avleen Malhi “Deep Learning-Based Explainable Target Classification for Synthetic Aperture Radar Images” In _2020 13th International Conference on Human System Interaction (HSI)_ , 2020, pp. 34–39 DOI: 10.1109/HSI49210.2020.9142658
  * [296] Amir Hosein Oveis et al. “LIME-Assisted Automatic Target Recognition with SAR Images: Toward Incremental Learning and Explainability” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 16, 2023, pp. 9175–9192 DOI: 10.1109/JSTARS.2023.3318675
  * [297] Sijia Feng et al. “SAR Target Classification Based on Integration of ASC Parts Model and Deep Learning Algorithm” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 14, 2021, pp. 10213–10225 DOI: 10.1109/JSTARS.2021.3116979
  * [298] Masanori Onishi and Takeshi Ise “Explainable Identification and Mapping of Trees Using UAV RGB Image and Deep Learning” In _Scientific Reports_ 11.1 Nature Publishing Group, 2021, pp. 903 DOI: 10.1038/s41598-020-79653-9
  * [299] Yunzhe Jia et al. “Studying and Exploiting the Relationship between Model Accuracy and Explanation Quality” In _Machine Learning and Knowledge Discovery in Databases. Research Track_ Cham: Springer International Publishing, 2021, pp. 699–714
  * [300] T.-A. Nguyen, B. Kellenberger and D. Tuia “Mapping Forest in the Swiss Alps Treeline Ecotone with Explainable Deep Learning” In _Remote Sensing of Environment_ 281, 2022 DOI: 10.1016/j.rse.2022.113217
  * [301] Patricia Arrogante-Funes et al. “Assessment of the Regeneration of Landslides Areas Using Unsupervised and Supervised Methods and Explainable Machine Learning Models” In _Landslides_ , 2023 DOI: 10.1007/s10346-023-02154-z
  * [302] Wantong Li et al. “Widespread Increasing Vegetation Sensitivity to Soil Moisture” In _Nature Communications_ 13.1 Nature Publishing Group, 2022, pp. 3959 DOI: 10.1038/s41467-022-31667-9
  * [303] Chuanpeng Zhao et al. “Toward a Better Understanding of Coastal Salt Marsh Mapping: A Case from China Using Dual-Temporal Images” In _Remote Sensing of Environment_ 295, 2023 DOI: 10.1016/j.rse.2023.113664
  * [304] Chuanpeng Zhao et al. “Identifying Mangroves through Knowledge Extracted from Trained Random Forest Models: An Interpretable Mangrove Mapping Approach (IMMA)” In _ISPRS Journal of Photogrammetry and Remote Sensing_ 201, 2023, pp. 209–225 DOI: 10.1016/j.isprsjprs.2023.05.025
  * [305] Josepha Schiller, Clemens Jänicke, Moritz Reckling and Masahiro Ryo “Higher Crop Diversity in Less Diverse Landscapes”, 2023 DOI: 10.21203/rs.3.rs-3410387/v1
  * [306] M. Müller et al. “Features Predisposing Forest to Bark Beetle Outbreaks and Their Dynamics during Drought” In _Forest Ecology and Management_ 523, 2022, pp. 120480 DOI: 10.1016/j.foreco.2022.120480
  * [307] Kyle A. Hilburn, Imme Ebert-Uphoff and Steven D. Miller “Development and Interpretation of a Neural-Network-Based Synthetic Radar Reflectivity Estimator Using GOES-R Satellite Observations” In _Journal of Applied Meteorology and Climatology_ 60.1 American Meteorological Society, 2020, pp. 3–21 DOI: 10.1175/JAMC-D-20-0084.1
  * [308] Zane K. Martin, Elizabeth A. Barnes and Eric Maloney “Using Simple, Explainable Neural Networks to Predict the Madden-Julian Oscillation” In _Journal of Advances in Modeling Earth Systems_ 14.5, 2022 DOI: 10.1029/2021MS002774
  * [309] Kirsten J. Mayer and Elizabeth A. Barnes “Subseasonal Forecasts of Opportunity Identified by an Explainable Neural Network” In _Geophysical Research Letters_ 48.10, 2021 DOI: 10.1029/2020GL092092
  * [310] Dangfu Yang et al. “Predictor Selection for CNN-based Statistical Downscaling of Monthly Precipitation” In _Advances in Atmospheric Sciences_ , 2023 DOI: 10.1007/s00376-022-2119-x
  * [311] S.A. Upadhyaya, P.-E. Kirstetter, R.J. Kuligowski and M. Searls “Classifying Precipitation from GEO Satellite Observations: Diagnostic Model” In _Quarterly Journal of the Royal Meteorological Society_ 147.739, 2021, pp. 3318–3334 DOI: 10.1002/qj.4130
  * [312] Jacob Mardian, Catherine Champagne, Barrie Bonsal and Aaron Berg “Understanding the Drivers of Drought Onset and Intensification in the Canadian Prairies: Insights from Explainable Artificial Intelligence (XAI)” In _Journal of Hydrometeorology_ -1.aop American Meteorological Society, 2023 DOI: 10.1175/JHM-D-23-0036.1
  * [313] Nafsika Antoniadou et al. “Comparison of Data-Driven Methods for Linking Extreme Precipitation Events to Local and Large-Scale Meteorological Variables” In _Stochastic Environmental Research and Risk Assessment_ 37.11, 2023, pp. 4337–4357 DOI: 10.1007/s00477-023-02511-3
  * [314] E.S. Maddy and S.A. Boukabara “MIIDAPS-AI: An Explainable Machine-Learning Algorithm for Infrared and Microwave Remote Sensing and Data Assimilation Preprocessing - Application to LEO and GEO Sensors” In _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ 14, 2021, pp. 8566–8576 DOI: 10.1109/JSTARS.2021.3104389
  * [315] L. Bergamasco, S. Saha, F. Bovolo and L. Bruzzone “AN EXPLAINABLE CONVOLUTIONAL AUTOENCODER MODEL for UNSUPERVISED CHANGE DETECTION” 43, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives, 2020, pp. 1513–1519 DOI: 10.5194/isprs-archives-XLIII-B2-2020-1513-2020
  * [316] Hyunglok Kim et al. “True Global Error Maps for SMAP, SMOS, and ASCAT Soil Moisture Data Based on Machine Learning and Triple Collocation Analysis” In _Remote Sensing of Environment_ 298, 2023 DOI: 10.1016/j.rse.2023.113776
  * [317] Dae-Seong Lee, Da-Yeong Lee and Young-Seuk Park “Interpretable Machine Learning Approach to Analyze the Effects of Landscape and Meteorological Factors on Mosquito Occurrences in Seoul, South Korea” In _Environmental Science and Pollution Research_ , 2022 DOI: 10.1007/s11356-022-22099-5
  * [318] Felix Wagner et al. “Using Explainable Machine Learning to Understand How Urban Form Shapes Sustainable Mobility” In _Transportation Research Part D: Transport and Environment_ 111, 2022, pp. 103442 DOI: 10.1016/j.trd.2022.103442
  * [319] P. Taconet et al. “Data-Driven and Interpretable Machine-Learning Modeling to Explore the Fine-Scale Environmental Determinants of Malaria Vectors Biting Rates in Rural Burkina Faso” In _Parasites and Vectors_ 14.1, 2021 DOI: 10.1186/s13071-021-04851-x
  * [320] Christoph Molnar et al. “Pitfalls to Avoid When Interpreting Machine Learning Models” In _XXAI: Extending Explainable AI beyond Deep Models and Classifiers, ICML 2020 Workshop_ , 2020
  * [321] Satyapriya Krishna et al. “The Disagreement Problem in Explainable Machine Learning: A Practitioner’s Perspective” arXiv, 2022 DOI: 10.48550/arXiv.2202.01602
  * [322] Cynthia Rudin “Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead” In _Nature Machine Intelligence_ 1.5 Nature Publishing Group, 2019, pp. 206–215 DOI: 10.1038/s42256-019-0048-x
  * [323] Leon Sixt, Maximilian Granz and Tim Landgraf “When Explanations Lie: Why Many Modified BP Attributions Fail” In _37th International Conference on Machine Learning, ICML 2020_ PartF16814 International Machine Learning Society (IMLS), 2020, pp. 8993–9004 DOI: 10.48550/arxiv.1912.09818
  * [324] Antonios Mamalakis, Elizabeth A. Barnes and Imme Ebert-Uphoff “Investigating the Fidelity of Explainable Artificial Intelligence Methods for Applications of Convolutional Neural Networks in Geoscience” In _Artificial Intelligence for the Earth Systems_ 1.4, 2022, pp. e220012 DOI: 10.1175/AIES-D-22-0012.1
  * [325] Philine Bommer et al. “Finding the Right XAI Method – A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science” arXiv, 2023 DOI: 10.48550/arXiv.2303.00652
  * [326] Johannes Haug, Stefan Zürn, Peter El-Jiz and Gjergji Kasneci “On Baselines for Local Feature Attributions” arXiv, 2021 DOI: 10.48550/arXiv.2101.00905
  * [327] Antonios Mamalakis, Elizabeth A. Barnes and Imme Ebert-Uphoff “Carefully Choose the Baseline: Lessons Learned from Applying XAI Attribution Methods for Regression Tasks in Geoscience” In _Artificial Intelligence for the Earth Systems_ 2.1 American Meteorological Society, 2023 DOI: 10.1175/AIES-D-22-0058.1
  * [328] Astrid Bertrand, Rafik Belloum, James R. Eagan and Winston Maxwell “How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review” In _Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’22 New York, NY, USA: Association for Computing Machinery, 2022, pp. 78–91 DOI: 10.1145/3514094.3534164
  * [329] Raymond S. Nickerson “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises” In _Review of General Psychology_ 2.2, 1998, pp. 175–220 DOI: 10.1037/1089-2680.2.2.175
  * [330] Markus Langer et al. “What Do We Want from Explainable Artificial Intelligence (XAI)? – A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research” In _Artificial Intelligence_ 296, 2021, pp. 103473 DOI: 10.1016/j.artint.2021.103473
  * [331] Mohamed Karim Belaid, Eyke Hüllermeier, Maximilian Rabus and Ralf Krestel “Do We Need Another Explainable AI Method? Toward Unifying Post-hoc XAI Evaluation Methods into an Interactive and Multi-dimensional Benchmark” arXiv, 2022 DOI: 10.48550/arXiv.2207.14160
  * [332] Chirag Agarwal et al. “OpenXAI: Towards a Transparent Evaluation of Model Explanations” arXiv, 2023 DOI: 10.48550/arXiv.2206.11104
  * [333] Wei Xu, Marvin J. Dainoff, Liezhong Ge and Zaifeng Gao “Transitioning to Human Interaction with AI Systems: New Challenges and Opportunities for HCI Professionals to Enable Human-Centered AI” In _International Journal of Human–Computer Interaction_ 39.3 Taylor & Francis, 2023, pp. 494–518 DOI: 10.1080/10447318.2022.2041900