longtable

# Statistical inference using machine learning and classical techniques based
on accumulated local effects (ALE)

Chitu Okoli | SKEMA Business School – Université Côte d’Azur, Paris

(2023-10-15)

###### Abstract

Accumulated Local Effects (ALE) is a model-agnostic approach for global
explanations of the results of black-box machine learning (ML) algorithms.
There are at least three challenges with conducting statistical inference
based on ALE: ensuring the reliability of ALE analyses, especially in the
context of small datasets; intuitively characterizing a variable’s overall
effect in ML; and making robust inferences from ML data analysis. In response,
we introduce innovative tools and techniques for statistical inference using
ALE, establishing bootstrapped confidence intervals tailored to dataset size
and introducing ALE effect size measures that intuitively indicate effects on
both the outcome variable scale and a normalized scale. Furthermore, we
demonstrate how to use these tools to draw reliable statistical inferences,
reflecting the flexible patterns ALE adeptly highlights, with implementations
available in the ‘ale’ package in R. This work propels the discourse on ALE
and its applicability in ML and statistical analysis forward, offering
practical solutions to prevailing challenges in the field.

##  1 Introduction

Accumulated Local Effects (ALE) were initially developed as a model-agnostic
approach for global explanations of the results of black-box machine learning
(ML) algorithms (Apley and Zhu 2020). ALE has a key advantage over other
approaches like partial dependency plots (PDP) and SHapley Additive
exPlanations (SHAP): its values represent a clean functional decomposition of
the model (Molnar, Casalicchio and Bischl 2020). As such, ALE values are not
affected by the presence or absence of interactions among variables in a mode.
Moreover, its computation is relatively rapid (Molnar 2022).

Despite the potential of ALE, at least three challenges in interpretable
machine learning (IML) remain unresolved, despite numerous incremental
attempts. Firstly, the reliability of results derived from ALE analyses is
brought into question, particularly when considering the prevalent data-only
bootstrapping approach (Flora 2023; Jumelle, Kuhn-Regnier and Rajaratnam
2020), which, especially in the context of smaller datasets that preclude a
training-test split, potentially risks overfitting and undermines the
generalizability of the findings.

This important issue is primarily a concern for small datasets, so it is
pertinent to ask how small is “small”? From the perspective of this article,
the key issue at stake is that applying the training-test split that is common
in ML is a crucial technique for increasing the generalizability of data
analysis. So, the question becomes focused on, “How small is too small for a
training-test split for ML analysis?” We could consider a general principle
that ML requires at least 200 rows of data for each predictor variable. So,
for example, with five input variables, we would need at least 1,000 rows of
data. But this number refers not to the size of the entire dataset but to the
minimum size of the training subset. So, with an 80-20 split on the full
dataset (that is, 80% training set), we would need at least 1,000 rows for the
training set and another 250 rows for the test set, for a minimum of 1,250
rows. (And if we were to carry out hyperparameter tuning with cross-validation
on that training set, then we would need even more data.) From these
considerations, we suggest that most datasets of less than 2,000 rows are
probably “small”. Indeed, even many datasets that are more than 2,000 rows are
nonetheless “small”.

This consideration is pertinent because ALE is a valuable technique for
visually characterizing the relationships between predictors and outcomes for
any model, not just for large datasets typical in ML, but also for smaller
datasets typical in statistical analysis. It is often inappropriate to
transfer the ALE analysis techniques that assume large datasets to smaller
datasets that need specialized treatment.

A second ongoing challenge concerns how to characterize a variable’s overall
effect. While effect sizes are extensively treated in statistical analysis
based on the general linear model, only recently have ML researchers started
trying to develop model-agnostic measures that can characterize the results of
any ML analysis. These initial attempts, while promising, do not always show
the effects of individual variables (Molnar, Casalicchio and Bischl 2020) and,
when they do, are often not intuitively interpretable (Molnar, Casalicchio and
Bischl 2020; Lötsch and Ultsch 2020; Friedman and Popescu 2008).

Third, even with reliable effect size measures, it is not clear how to make
reliable inferences from data analysis based on ML. Messner (2023) has made an
admirable initial attempt towards hypothesis testing based on ML analysis, but
his framework nonetheless masks some of the fine nuances that ALE hopes to
uncover with its visual portrayal of flexible relationships.

In response to these challenges, the research objective of this article is to
introduce tools and techniques for statistical inference using machine
learning and classical statistical techniques based on ALE. We address
trusting the reliability of analysis results by creating bootstrapped
confidence intervals for ALE using techniques appropriate to the size of the
dataset. To characterize the overall effects of a predictor variable on its
outcome variable, we create a set of ALE effect size measures that intuitively
indicate the effect either on the scale of the outcome variable or on a
normalized scale that is comparable across datasets. However, to avoid
simplistic summaries of overall effects that might hide important details, we
demonstrate how to use these tools to make reliable conclusions of statistical
inference that reflect the flexible patterns that ALE can so capably
highlight.

We conduct all analyses using the R package ale (Okoli 2023), which was
specifically developed to extend ALE with the functionality we describe in
this article. Indeed, this article is largely an explanation of the scientific
background of the ale package.

The rest of this article is organized as follows. In the “Related work”
section, we delve into the existing literature and software implementations of
ALE, exploring aspects such as ALE confidence intervals, bootstrapping, effect
size measures in machine learning, and inference from analysis results, while
also identifying opportunities for improvement in the current methodologies.
Subsequently, “Illustrative datasets and models” introduces two distinct
datasets and corresponding models—a neural network model for diamond prices
and a generalized additive model for mathematics achievement scores—to provide
a practical context for the ensuing analyses. The section on “Bootstrapping of
accumulated local effects” elucidates the methodologies of data-only
bootstrapping, particularly focusing on a large dataset, and introduces model
bootstrapping, with a spotlight on a smaller dataset and its application to
ALE. Moving forward, “ALE effect size measures” introduces and explores novel
effect size measures, both on the scale of the y outcome variable and in a
normalized form, while also discussing the implications and applications of
random variables. In “Statistical inference with ALE,” we navigate through
classical statistical inference, explore ALE data structures, and delve into
bootstrap-based inference with ALE, culminating in a discussion on confidence
regions and random variables. Finally, the “Discussion” section encapsulates
the contributions and practical implications of the methodologies and insights
presented, providing a comprehensive overview and concluding remarks on the
potential trajectories and applications of the enhanced ALE methodologies in
machine learning model interpretation and analysis.

##  2 Related work

In the seminal work of Apley and Zhu (2020), the concept of Accumulated Local
Effects (ALE) was introduced as a global explanation technique for IML. ALE
provides functionality akin to Partial Dependence Plots (PDP), both of which
graphically delineate the relationship between a single input variable X and
the outcome variable Y, thereby illustrating non-linear and flexible variant
relationships. A noteworthy enhancement of ALE over PDP is its resilience to
the interactions between variables. A secondary, yet non-negligible advantage
of utilizing the ALE algorithm is its reduced computational expense, which
serves as a substantial practical incentive for machine learning scientists to
incorporate ALE visualizations throughout their analytical processes.

There have been a few recent extensions to fine-tune the original algorithm.
Gkolemis et al. (2023) present Robust and Heterogeneity-Aware Accumulated
Local Effects (RHALE), a technique that endeavours to address a notable
limitation inherent in the original ALE algorithm’s approach to binning
numeric data according to quantiles. A pivotal challenge resides in the
absence of indicators for the heterogeneity of data within each quartile.
Consequently, the ALE plotted line may not accurately represent the data. To
mitigate this, RHALE considers the standard deviation of data within each
prospective bin when determining the ALE bin boundaries. Gkolemis, Dalamagas
and Diou (2023) present Differential Accumulated Local Effects (DALE) to
address two prominent limitations of the original ALE formulation. Firstly,
they argue that the original ALE does not scale effectively to high-
dimensional data. Secondly, they note that with smaller samples, the original
ALE calculation may not be representative of out-of-distribution sampling.
DALE is an adjusted algorithm for calculating ALE that attempts to enhance
both the scalability and representational accuracy of the technique

Although our present article focuses on the original ALE algorithm, its
findings can probably be readily extrapolated to such extensions. Indeed, the
kinds of extensions in which we are interested have not been documented in
scholarly articles that we could find but are rather found to various extents
in software packages that implement ALE. Thus, our review in this section
largely surveys software implementations in addition to published literature.

###  2.1 Software implementations of ALE

Since the initial ALEPlot reference package in R was released in 2018 (Apley
2018), implementations in various programming languages have translated or
extended the initial concept. ALEPython (Jumelle, Kuhn-Regnier and Rajaratnam
2020) and PyALE (Jomar 2023) in Python have been specifically dedicated to
implementing ALE, sometimes with extensions. Most, however, are more general
IML packages that include ALE among other techniques. These include iml
(Molnar and Schratz 2022) in R; scikit-explain (Flora 2023) and Alibi (Seldon
Technologies 2023) in Python; DALEX (Biecek, Maksymiuk and Baniecki 2023) in R
and Python; and the Interpretation (RapidMiner 2023) extension for RapidMiner,
a Java-based no-code machine learning platform. In addition to these, this
present article introduces ale (Okoli 2023), an R package dedicated to the
implementation and extension of ALE. In particular, ale aims to resolve some
of the issues that we describe subsequently in this section. In Table
LABEL:tbl-ale-packages, we list some key characteristics of these packages,
focusing on features that are pertinent to the subject of this present
article.

Table 1: Software packages that implement ALE |  |  |  |  |  |
---|---|---|---|---|---|---
Primary focus |  Package |  Latest release |  Language |  Confidence intervals |  Bootstrap type |  ALE statistics
ALE |  ALEPlot (Apley 2018) |  2018 |  R |  No |  N/A |  None
ALE |  ALEPython (Jumelle, Kuhn-Regnier and Rajaratnam 2020) |  2020 |  Python |  Monte Carlo |  data-only |  None
IML |  iml (Molnar and Schratz 2022) |  2022 |  R |  No |  N/A |  None
IML |  DALEX (Biecek, Maksymiuk and Baniecki 2023) |  2023 |  R and Python |  No |  N/A |  None
ALE |  PyALE (Jomar 2023) |  2023 |  Python |  T-statistic |  N/A |  None
IML |  Interpretation (RapidMiner 2023) |  2023 |  RapidMiner |  No |  N/A |  None
IML |  Alibi (Seldon Technologies 2023) |  2023 |  Python |  No |  N/A |  None
IML |  scikit-explain (Flora 2023) |  2023 |  Python |  Bootstrap |  data-only |  • Friedman H-statistic for interactions • Interaction strength (IAS) • Main effect complexity (MEC)
ALE |  ale (introduced in this article) (Okoli 2023) |  2023 |  R |  Bootstrap |  data-only and model |  • ALE deviation (ALED) • ALE ranger (ALER) • Normalized ALED (NALED) • Normalized ALER (NALER)

In the following subsections, we discuss publications and software
implementations that develop confidence intervals, bootstrapping, effect size
measures, and inference, almost always with ALE.

###  2.2 ALE confidence intervals and bootstrapping

The initial ALEPlot implementation did not feature confidence intervals, but
several packages have recognized their importance and have thus extended ALE
with this feature, implemented in different ways. The simplest approach is
that employed by PyALE, which constructs intervals based on the t statistic,
using the standard deviation of the ALE values to construct standard errors.
However, despite the other merits of that implementation, we cannot find any
basis to assume that ALE values are distributed according to a t distribution,
or by any other parametric distribution for that matter. Only bootstrap-based
confidence intervals are appropriate for data like ALE whose distribution
cannot be generalized to any predetermined pattern (Tong, Saminathan and Chang
2017).

Although most readers are likely familiar with the bootstrap algorithm, it is
worth repeating it here to highlight the complexities that ALE presents. The
classic bootstrap approach involves creating multiple samples of the original
dataset. Each bootstrap sample draws rows at random from the original dataset,
as many new rows as there are rows in the original dataset. Crucially,
sampling is done with replacement so that in each bootstrap sample, some of
the original rows are repeated a random number of times and some might not
occur at all. The analyst decides how many bootstrap samples they want. With
the bootstrap samples, the desired aggregate statistics are computed, such as
the mean or standard deviation across all bootstrap samples.

For ALE bootstrapping, the aggregation goal across the bootstrap samples is to
calculate the ALE values. The ALE calculation depends first on establishing
ALE bins or intervals for an X predictor variable and then an ALE Y estimate
is calculated for each ALE X interval. What is particularly tricky with
bootstrapping ALE calculations is that the intervals are calculated directly
from the data, so each time the data is scrambled with a bootstrap sample, the
ALE X intervals necessarily change. Even with large datasets, because the
intervals for numeric X predictors are calculated from the quantiles of the
particular sets of values, each bootstrap sample will produce distinct
intervals, which cannot be combined in any way across bootstrap samples. The
problem persists even with categorical X predictors. Whereas with sufficiently
large datasets each bootstrap sample will likely include at least a few rows
that represent each category of the X predictor, with small datasets, it is
often the case that at least some bootstrap samples might not include every
category from the original dataset. In such cases, ALE values cannot be
combined across bootstrap samples that do not share identical intervals.

In their implementations of bootstrapped confidence intervals, the scikit-
explain (Flora 2023) and ale (Okoli 2023) packages resolve this challenge by
using the full dataset to establish the ALE X intervals and then they apply
these fixed intervals in calculating the ALE Y values for each bootstrap
sample. Thus, bootstrap averages and quantiles can be calculated across each
common interval across all bootstrap samples. In general, we call this
standard approach “data-only bootstrapping” because the dataset is
bootstrapped, but not the model itself. This approach might also be called
model-invariant bootstrapping or case resampling.

The ALEPython package (Jumelle, Kuhn-Regnier and Rajaratnam 2020) adopts a
similar approach in their adapted bootstrap confidence intervals that they
call “Monte Carlo”. This is essentially the same as the standard bootstrap
(sampling with replacement) except that instead of constructing samples of the
same size as the original dataset, they only sample a fixed fraction of the
dataset (10% by default, customizable by the user). Other than that detail,
these three packages essentially employ the same approach.

###  2.3 Effect size measures for machine learning

Although effect size measures are widely used in statistical analyses, usually
with smaller datasets, they are not as widely used in machine learning.
However, there are a few notable examples. Friedman and Popescu (2008)
introduced a metric known as the H-statistic, designed to quantify the
intensity of interactions among variables in datasets. It comes in two forms:
a pairwise version that reveals the strength of two-way interactions between
feature pairs, and another that assesses the interaction strength of a single
feature with any other feature in the dataset or model. Lötsch and Ultsch
(2020) have formulated an effect size metric, drawing parallels to Cohen’s D
used in statistical evaluations of group differences. Their innovation,
tailored for the machine learning context, not only considers the central
tendencies of groups but also the variations within and across these groups,
providing a more nuanced measure.

Molnar, Casalicchio and Bischl (2020) delve into interpreting models through a
process they refer to as functional decomposition. This approach involves
dissecting a model’s predictive elements into three distinct components: a
constant element, the main effects of individual variables, and interactions
between variables. For evaluating main effects, their methodology hinges on
ALE, leading to the formulation of a metric termed Main Effect Complexity
(MEC). In essence, MEC gauges the degree to which a variable’s relationship
with the outcome diverges from a linear trend. Regarding the interaction
component, they employ a metric known as Interaction Strength (IAS), which
serves to signify the robustness of a feature’s interaction with others.

Regarding MEC, two distinct calculations exist. Initially, MEC is determined
for each feature individually, assessing the extent to which the relationship
between a feature and the outcome deviates from a linear representation.
Subsequently, an overall MEC is computed as the mean of individual MECs for
all features within the model, with this aggregate value typically being the
primary focus. Conversely, IAS is consistently calculated for the entire set
of features in the model. It essentially measures what remains unaccounted for
when the average main effects of all features are considered. Therefore, IAS
represents a collective metric for the entire model, rather than a measure of
interaction for individual features. The scikit-explain package (Flora 2023)
provides Friedman’s H-statistic, MEC, and IAS effect size measures.

Messner (2023) adapts Cohen’s $f^{2}$ from classical statistical analysis to
quantify effect strength in ML contexts. He diverges from the conventional
version based on $R^{2}$ used in ordinary least squares regression to an
adapted version he calls $f^{2}_{v}$, based on mean squared error (MSE).
$f^{2}_{v}$, a unit-free measure that ranges from 0 with no upper bound, can
be compared across datasets. Additionally, Messner evaluates the monotonicity
of an effect—whether an effect consistently moves in one direction—and its
directionality, utilizing p-values derived from the Mann-Kendall test and
Theil-Sen slope, respectively. These tests determine if the effect is
unidirectional and if it is increasing or decreasing, with the magnitude of
change assessed via $f^{2}_{v}$.

###  2.4 Inference from analysis results

One of the biggest differences between supervised ML and statistical modelling
is the general goal of the analysis. Supervised ML is predominantly concerned
with obtaining the most accurate prediction possible of the outcome variable.
While the predictor variables are necessary to obtain an accurate prediction,
the specific relationship between predictors and outcomes is often considered
incidental. In contrast, statistical modelling is primarily concerned with
reliably describing the relationship between certain predictor variables and
the outcome. While an accurate estimation model of the outcome is important,
it is only secondarily so as an indication of the reliability of the model.
Statistical analysis attempts to infer from the relationships in the data
analyzed patterns that can be generalized to the larger population from which
the data sample is drawn.

IML bridges the gap between the two modes of analysis by using supervised ML
techniques to describe the relationships between predictors and the outcome
even with models that are not intrinsically interpretable. One major goal of
such interpretation is to apply the more flexible panoply of ML methods for
the same goal as statistical inference: reliably inferring relationships
between predictors and the outcome to a population larger than the sample.
However, whereas statistical inference is largely based on the interpretation
of coefficients of models that are variations of the general linear model
(GLM), most ML techniques do not have such coefficients. Thus, some scholars
have developed approaches to statistical inference specifically tailored to
ML.

Sechidis (2015) delves into the realm of hypothesis testing amid the
complexities of semi-supervised data, characterized by its partially labelled
and incomplete nature. His thesis meticulously examines the unique challenges
posed by this data type, particularly when drawing conclusions from hypothesis
tests. His exploration is not confined to statistical significance alone; it
also encompasses a thorough consideration of effect sizes, thereby providing a
more comprehensive understanding of the analytical implications of the data.

Messner (2023) introduces some new measures with the ultimate goal of enabling
hypothesis testing from a model-agnostic approach using machine learning. The
key elements of his framework are:

  * •

Determine the practical importance of effects based on the magnitude of
$f^{2}_{v}$: < 0.02 is trivial; 0.02 to 0.15 is small; 0.15 to 0.35 is medium;
and >= 0.35 is large. Crucially, rather than statistical significance, his
effect-size-based hypothesis testing emphasizes the practical magnitude of
effects.

  * •

Regardless of magnitude, determine if the effects are monotonic, applying the
Mann-Kendall test for statistical significance.

  * •

For monotonic effects, determine if the effect is increasing or decreasing,
applying the Theil-Sen test for statistical significance.

###  2.5 Opportunities for improvement

Despite the various implementations and valuable extensions of the original
ALE concept, we see some ongoing issues that leave room for improvement to
better adapt it as a tool for statistical inference using ML.

First, there is a crucial limitation of the standard data-only approach to
bootstrapping ALE values that we describe above (Flora 2023; Jumelle, Kuhn-
Regnier and Rajaratnam 2020). Ultimately, as an ML technique, ALE values are
usually calculated on a model to apply the explanations obtained to understand
future data. As such, when a model is trained on some training data, the ALE
should be calculated on independent test data to give assurance that the ALE
does not overfit the data. It is appropriate to bootstrap such independent
test data to obtain ALE confidence intervals. However, some datasets are too
small for a training-test split to be feasible, so the model is created on the
entire dataset. In this case, any generalizations that might be drawn from
such analyses strongly risk overfitting the data and there is no test set to
qualify such generalizations.

Second, while it is encouraging to see the development of various model-
agnostic effect size measures, those that exist tend to be limited when the
goal is to practically quantify the effects of variables. The overall model
MEC and the IAS (Molnar, Casalicchio and Bischl 2020) do not show the effects
for individual variables but only indicate generally the average effects
across all variables. While the variable-specific version of the MEC attempts
to address this concern, it mainly indicates the extent to which a variable is
non-linear but it does not clearly quantify other important aspects of its
possible effect. Similarly, the impact score (Lötsch and Ultsch 2020) and
$f^{2}_{v}$ (Messner 2023) indicate the overall effect of a predictor variable
on the outcome but do not shed much light on the nature of the effect. The
tests for monotonicity and slope (Messner 2023) are an important step in the
direction of clarifying the nature of the effect, but by generalizing the
overall trend in one tendency (e.g., monotonically increasing or non-
monotonic), the details of complex relationships might not be sufficiently
clear.

A third concern is the computational practicality of some of the measures. For
example, Friedman’s H is a useful measure for interaction strength, but it is
computationally rather expensive to calculate (Molnar 2022).

In the following sections of the article, we address these outstanding
concerns.

##  3 Illustrative datasets and models

Before we begin ALE analysis, in this section we describe the datasets and the
models that we will use for our illustrations. Because the issues that we
address are handled differently depending on the relative size of the dataset,
we work with two distinct datasets. We develop a neural network model to
analyze a moderately large dataset of diamond prices and then a generalized
additive model (GAM) to analyze a small dataset of mathematics achievement
scores.

###  3.1 Large dataset: neural network model for diamond prices

For the analysis of a relatively large dataset, we use the diamonds dataset,
built-in with the ggplot2 graphics system (Wickham et al. 2023). We cleaned
the original version by removing duplicates (Mayer 2021) and invalid entries
where the length (x), width (y), or depth (z) is 0. Table LABEL:tbl-diamonds-
dataset presents the description of the modified dataset. The outcome variable
that is the focus of our analysis is the price of a diamond, with minimum
$326, median $3,365, and maximum $18,823 in USD.

Table 2: Large dataset: diamond prices |
---|---
Variable |  Description
price |  price in US dollars ($326–$18,823)
carat |  weight of the diamond (0.2–5.01)
cut |  quality of the cut (Fair, Good, Very Good, Premium, Ideal)
color |  diamond color, from D (best) to J (worst)
clarity |  a measurement of how clear the diamond is (I1 (worst), SI2, SI1, VS2, VS1, VVS2, VVS1, IF (best))
x_length |  length in mm (3.73–10.74)
y_width |  width in mm (3.68–58.9)
z_depth |  depth in mm (1.07–31.8)
depth_pct |  total depth percentage = z / mean(x, y) = 2 * z / (x + y) (43–79)
table |  width of top of diamond relative to widest point (43–95)
rand_norm |  a completely random variable (-4.9 to 4.4)

Of particular note is the variable rand_norm. We have added this completely
random variable (with a normal distribution) to demonstrate what randomness
looks like in our analysis. (However, we deliberately selected the specific
random seed of 6 because it highlights some particularly interesting points.)

For machine learning analysis that intends to extrapolate its results to
future data, we must first split the dataset into training and test samples.
The model is developed on the training set and then evaluated on the test set.
So, we split the dataset with an 80-20 split for a training set of 31,758 rows
and a test set of 7,981 rows.

ALE is a model-agnostic IML approach, that is, it works with any kind of ML
model. For this demonstration, we train a neural network to predict diamond
prices, selected to demonstrate the benefits of ALE to a notoriously black-box
model.

###  3.2 Small dataset: generalized additive model for mathematics achievement
scores

We demonstrate ALE statistics on small datasets with a dataset composed and
transformed from the nlme package (Pinheiro et al. 2023). The structure has
160 rows, each of which refers to a school whose students have taken a
mathematics achievement test. Table LABEL:tbl-math-dataset describes the data
based on documentation from the nlme package but, unfortunately, many details
are not quite clear.

Table 3: Small dataset: average mathematics achievement scores for schools |  |
---|---|---
variable |  format |  description
math_avg |  double |  average mathematics achievement scores of all students in the school
size |  double |  the number of students in the school
public |  logical |  TRUE if the school is in the public sector; FALSE if in the Catholic sector
academic_ratio |  double |  the percentage of students on the academic track
female_ratio |  double |  percentage of students in the school that are female
mean_ses |  double |  mean socioeconomic status for the students in the school (measurement is not quite clear)
minority_ratio |  double |  percentage of students that are members of a minority racial group
high_minority |  logical |  TRUE if the school has a high ratio of students of minority racial groups (unclear, but perhaps relative to the location of the school)
discrim |  double |  the “discrimination climate” (perhaps an indication of extent of racial discrimination in the school?)
rand_norm |  double |  a completely random variable

Of particular note again is the variable rand_norm. As with the large dataset,
we have added this completely random variable (with a normal distribution) to
demonstrate what randomness looks like in our analysis. (However, we selected
the specific random seed of 6 because it highlights some particularly
interesting points.)

The outcome variable that is the focus of our analysis is math_avg, the
average mathematics achievement scores of all students in each school, with
minimum 4.24, median 12.9, and maximum 19.7.

Because the samples here are relatively small, we will use general additive
models (GAM) for the modelling (Wood 2023). GAM is an extension of statistical
regression analysis that lets the model fit flexible patterns in the data
instead of being restricted to the best-fitting straight line. It is an ideal
approach for samples that are too small for machine learning because it
provides flexible curves unlike ordinary least squares regression yet will not
overfit excessively as would most machine learning techniques when working
with such small samples (Ross 2019).

##  4 Bootstrapping of accumulated local effects

We now come to the application of ALE to address the issues we highlighted
above in the introduction and related work sections. In this section, we
explore two distinct bootstrapping approaches, each applied to datasets of
varying sizes and characteristics. Initially, we begin with the data-only
bootstrap approach to improve the reliability of ALE plots by providing
confidence intervals that offer more reliable estimates than those derived
from a single ALE calculation. Next, we assess the imperative of model
bootstrapping to mitigate the risk of overfitting in the context of smaller
datasets.

###  4.1 Data-only bootstrapping of a large dataset (diamonds)

Figure 1: Simple ALE plots for neural network model of diamond prices

Figure 1 displays the simple ALE plots for the neural network model of diamond
prices. (We refer to these plots as “simple” because they simply calculate ALE
without confidence intervals.) In this case and all our other analyses on the
diamonds dataset, the ALE is calculated on the test dataset, not the training
set on which the neural network was trained.

A significant feature of the ale package that we use in this article is that
it centres ALE values on the median, unlike the original ALEPlot
implementation (Apley 2018), which centres them on zero. Although the ale
package lets the user centre on zero or even on the mean, the median is
particularly crucial for the statistical inference framework that the
alepackage offers. An essential element of its visualization is the middle
grey band that indicates the median ± 2.5%, that is, the middle 5% of all
price values in the dataset. We call this the “median band”. The idea is that
if any predictor can do no better than influencing price to fall within this
middle median band, then it only has a minimal effect. For an effect to be
considered practically meaningful, its ALE effect should be above or below the
median band. Looking at these simple ALE plots, we could draw the following
initial conclusions:

  * •

The prices increase sharply with the carat of the diamonds until they plateau
at the highest end of the domain just above 2 carat. The rug plot indicates
that there are very few diamonds with a higher carat than that.

  * •

cut does not seem to have much of an effect on prices.

  * •

The H, I, and J colors have lower prices; the other values do not seem to have
much of an effect.

  * •

The I1, S12, and SI1 clarity categories have lower prices while VVS2, VVS1,
and IF have higher prices. VS2 and VS1 do not seem to have much of an effect.

  * •

depth_pct has slightly lower prices at its low end and considerably higher
prices at its high end. However, the rug plot indicates that there are very
few diamonds at either end, so these results are not well supported by data.

  * •

Conversely, table has considerably higher prices at its low end and somewhat
lower prices at its high end, but these results are not well supported by
data.

  * •

x_length barely crosses the median band in either direction, so it does not
seem to have much of a meaningful effect.

  * •

As y_width increases, price sharply increases with it. Although the plot shows
a long plateau, the rug plot indicates that the plateau is predominantly in
regions of high y_width where there are few diamonds.

  * •

As z_depth increases, price steadily decreases with it throughout the range of
z_depth.

  * •

Somewhat surprisingly, although the effect is not so strong as is more marked
at the extremes of its domain, the random variable rand_norm seems to have a
distinctly negative effect on prices. We will pay attention to this variable
as we continue our analyses.

Despite these interesting initial results, it is crucial to bootstrap the ALE
results to ensure that they are reliable, that is, generalizable to data
beyond the sample on which the model was built. So, now we recreate
bootstrapped versions of the ALE data and plots. Specifically, because the ALE
data was computed on a distinct test dataset, we only bootstrap the test
dataset and we recalculate ALE using the same model on each bootstrap sample;
this is data-only bootstrapping.

Figure 2: Bootstrapped ALE plots for neural network model of diamond prices

ALE is a relatively stable IML algorithm (compared to others like PDP), so 100
bootstrap samples should be sufficient for relatively stable results,
especially for model development (Apley and Zhu 2020).

The bootstrapped results in Figure 2 are very similar to single (non-
bootstrapped) ALE results from Figure 1, which attests to the stability of ALE
results when working with moderately large datasets. However, bootstrapping
adds confidence intervals to the plot (we use a 95% interval in our
illustrations), which adds some uncertainty when certain values are close to
the median band as to whether they overlap the band or not. We present tools
to clarify this point below.

###  4.2 Model bootstrapping of a small dataset (math achievement)

Figure 3: Simple ALE plots for GAM of mathematics achievement scores

Turning to the case of small datasets, we again begin with simple ALE plots to
see what results look like without bootstrapping. The ALE plots in Figure 3
suggest that:

  * •

size, academic_ratio, and mean_ses have positive relationships with math
achievement scores. The relationship with size is linear, but those with the
other variables is non-linear.

  * •

female_ratio and minority_ratio seem to have negative relationships with math
achievement scores. The negative relationship of the female_ratio does not
seem to be that strong, though it seems to be driven by the fact that all-male
schools have above-average scores while all-female schools have below-average
scores.

  * •

public schools seem to have lower scores on average while those with
high_minority ratios have higher scores.

  * •

The discrimination climate discrim, though somewhat negative, does not seem to
have much of an effect.

  * •

The random variable rand_norm, though somewhat positive, does not seem to have
much of an effect.

However, before reading too much into these results, we must remember that
results that are not bootstrapped are simply not reliable.

####  4.2.1 Inappropriate data-only bootstrapping of a small dataset

Figure 4: Data-only (inappropriate) bootstrapped ALE plots for GAM of
mathematics achievement scores

To illustrate the implications of the different bootstrap approaches, we will
first carry out a data-only bootstrap as we did with the 100 bootstrap
iterations

With data-only bootstrapping, the results in Figure 4 are very similar to
those without bootstrapping in Figure 3, with only mean_ses displaying wide
confidence bands that suggest that some of the ranges of values might not be
relevant. However, data-only bootstrapping is not the appropriate approach for
small samples. The issue is not directly the size of the dataset but the fact
that the model was trained on the entire dataset rather than on a sample
distinct from that on which the ALE data and plots are created. Thus, for the
results to represent a broader sample than that of the small dataset, we must
not only bootstrap the entire dataset but we must also retrain the entire
model on each of the bootstrapped datasets. Although the problem might not be
so readily evident by examining the inappropriately bootstrapped plot above,
we revisit this issue below when we discuss bootstrap-based inference with ALE
and we see how inappropriate this approach is in handling the random variable
rand_norm.

####  4.2.2 Appropriate full-model bootstrapping of a small dataset

Data-only bootstrapping is not appropriate for small datasets where a model is
trained on the entirety of available data because there is a high risk of
overfitting. Indeed, This is one of the classic cases that the bootstrap was
originally designed to address. In such cases, the entire dataset should be
bootstrapped and the model should be retrained on each bootstrap sample. Thus,
there will be as many models as there are bootstrap samples. Any necessary
calculations (typically overall model statistics and variable coefficients)
are calculated for each model and the averages, quantiles, etc. of these
calculations across model-bootstrap samples are calculated as the bootstrapped
estimates.

We call this approach model bootstrapping because not only is the data
bootstrapped, but the model itself is also bootstrapped. This might also be
called model-dependent bootstrap or resampling with model re-fitting. Whereas
data-only bootstrapping has $n_{it}$ (number of iterations) bootstrap samples
of the data but only one model, model bootstrapping has $n_{it}$ distinct
models, one for each of the $n_{it}$ bootstrap samples. Model bootstrapping is
much slower than data-only bootstrapping because the model has to be retrained
$n_{it}$ times. While model bootstrapping does not mitigate overfitting, it
provides confidence intervals that effectively give more reliable estimates
than naively accepting only the single estimates from a single model on the
entire dataset. It should be considered mandatory for datasets that are too
small to be split further into distinct training and test sets.

Model bootstrapping for ALE is similar to data-only bootstrapping for ALE in
that fixed ALE intervals must be calculated from the full dataset and then
applied to each bootstrap sample, even though in this case each bootstrap
sample has a different model. The essential difference is that a different
model is used to calculate the ALE values each time—the model specific to each
bootstrap sample.

Thus, we apply the appropriate model bootstrapping approach to the math
dataset. Model bootstrapping is particularly slow, even on small datasets,
since the entire process is repeated that many times. However, 100 iterations
should be sufficiently stable for model building.

Figure 5: Model (appropriate) bootstrapped ALE plots for GAM of mathematics
achievement scores

With model bootstrapping, although the general patterns of relationships in
Figure 5 are similar to those of the data-only bootstrapping in Figure 4, the
confidence bands are generally much wider, which indicates that the data-only
bootstrap greatly exaggerated the reliability of the ALE results for this
small sample. We can see that most variables seem to have some sort of mean
effect across various values. However, for statistical inference, our focus
must be on the bootstrap intervals. For an effect to be considered
statistically significant, there should be no overlap between the confidence
regions of a predictor variable and the median band.

For categorical variables (public and high_minority above), the confidence
interval bands for all categories overlap the median band. The confidence
interval bands indicate two useful pieces of information to us. When we
compare them to the median band, their overlap or lack thereof tells us about
the practical significance of the category. When we compare the confidence
bands of one category with those of others, it allows us to assess if the
category has a statistically significant effect that is different from that of
the other categories; this is equivalent to the regular interpretation of
coefficients for GAM and other GLM models. In both cases, the confidence
interval bands of the TRUE and FALSE categories overlap each other, indicating
that there is no statistically significant difference between categories. Each
confidence interval band overlaps the median band, indicating that none of the
effects is practically significant, either.

For numeric variables, the confidence regions overlap the median band for most
of the domains of the predictor variables except for some regions that we will
examine. The extreme points of each variable (except for discrim and
female_ratio) are usually either slightly below or slightly above the median
band, indicating that extreme values have the most extreme effects: math
achievement increases with increasing school size, academic track ratio, and
mean socioeconomic status, whereas it decreases with increasing minority
ratio. The ratio of females and the discrimination climate both overlap the
median band for the entirety of their domains, so any apparent trends are not
supported by the data.

Of particular interest is the random variable rand_norm, whose average ALE
appears to show some sort of pattern. However, we can see that the confidence
intervals overlap the median band for its entire domain. We will return to the
implications of this observation below.

##  5 ALE effect size measures

Now that we can calculate reliable ALE effects by bootstrapping, we proceed to
consider the strengths of the variable effects. In all cases, we focus on the
average of the ALE values calculated from the bootstrap samples. The “average”
here is usually the mean, though the median can also be used.

In general, effect size measures are most frequently used in statistical
analysis, which typically analyzes relatively small datasets. They are also
used to a lesser extent in machine learning with large datasets, typically in
the context of variable importance measures for IML (Molnar 2022). Thus, the
measures that we describe here are relevant to both small and large datasets.
Given their relatively greater relevance for statistical analysis of smaller
datasets, we will demonstrate them in this section almost exclusively using
the mathematics achievement dataset. However, we note that everything we
describe here is fully relevant to large datasets; indeed, we end this section
with a pertinent illustration

###  5.1 ALE effect size plot

Although ALE plots allow rapid and intuitive conclusions for statistical
inference, it is often helpful to have summary numbers that quantify the
average strengths of the effects of a variable. Thus, we have developed a
collection of effect size measures based on ALE tailored for intuitive
interpretation. To understand the intuition underlying the various ALE effect
size measures, even before we explain the measures in detail, it is useful to
first examine the ALE effects plot in Figure 6 that graphically summarizes the
effect sizes of all the variables in the ALE analysis.

Figure 6: ALE effects plot for mathematics achievement scores

This plot requires some explanation:

  * •

The y (vertical) axis displays the x variables, rather than the x-axis. This
is consistent with most effect size plots because they list the full names of
variables. It is more readable to list them as labels on the y-axis than the
other way around.

  * •

The x (horizontal) axis thus displays the y (outcome) variable. But there are
two representations of this same axis, one at the bottom and one at the top.

  * •

On the bottom is a more typical axis of the outcome variable, in our case,
math_avg. It is scaled as expected. In our case, the axis breaks default to
five units each from 5 to 20, evenly spaced. The median of 13 is also
specifically marked.

  * •

On the top, the outcome variable is expressed as percentiles ranging from 0%
(the minimum outcome value in the dataset) to 100% (the maximum). It is
divided into ten deciles of 10% each. Because percentiles are usually not
evenly distributed in a dataset, the decile breaks are not evenly spaced.

  * •

Thus, this plot has two x axes, the lower one in units of the outcome variable
and the upper one in percentiles of the outcome variable. To reduce the
confusion, the major vertical grid lines that are slightly darker align with
the units of the outcome (lower axis) and the minor vertical grid lines that
are slightly lighter align with the percentiles (upper axis).

  * •

The vertical grey band in the middle is the median band, representing the
median ± 2.5% of values.

  * •

The variables on the horizontal axis are sorted by decreasing NALED value
(explained below). NALED is the most universal ALE measure of effect size.

Although it is somewhat confusing to have two axes, the percentiles are a
direct transformation of the raw outcome values. The first two base ALE effect
size measures below are in units of the outcome variable while their
normalized versions are in percentiles of the outcome. Thus, the same plot can
display the two kinds of measures simultaneously. Referring to this plot can
help to understand each of the measures, which we proceed to explain in
detail.

###  5.2 ALE effect size measures on the scale of the y outcome variable

In this subsection, we describe the two base effect size measures that are
scaled on the same unit of measure as the y outcome variable.

However, before we explain these measures in detail, we must reiterate the
timeless reminder that correlation is not causation. So, none of the scores
necessarily means that an x variable _causes_ a certain effect on the y
outcome; we can only say that the ALE effect size measures indicate associated
or related variations between the two variables.

####  5.2.1 ALE range (ALER)

The easiest ALE statistic to understand is the ALE range (ALER), so we begin
there. It is simply the range from the minimum to the maximum of any ale_y
value for that variable. Mathematically, we see this in Equation 1.

$\hypertarget{eq-
aler}{}{\mathrm{ALER}(\mathrm{ale\\_y})=\\{\min(\mathrm{ale\\_y}),\max(\mathrm{ale\\_y})\\}}$
(1)

where $\mathrm{ale\\_y}$ is the vector of ALE y values for a variable.

We note that all the ALE effect size measures are centred on zero so that they
are consistent regardless of whether the ALE plots are centred on zero (as in
the original ALEPlot implementation (Apley 2018)) or on the median (as in the
default with the ale package that we use in this article). Specifically,

  * •

aler_min: minimum of any ale_y value for the variable.

  * •

aler_max: maximum of any ale_y value for the variable.

ALER shows the extreme values of a variable’s effect on the outcome. In the
effects plot above, it is indicated by the extreme ends of the horizontal
lines for each variable. We can access ALE effect size measures through the
ale$stats element of the bootstrap result object, with multiple views. To
focus on all the measures for a specific variable, we can access the
ale$stats$by_term element. In Table 4, we see the effect size measures for the
categorical public.

Table 4: Effect size measures for the categorical public from the math
achievements dataset  statistic estimate conf.low median mean conf.high aled
0.377 0.019 0.339 0.377 0.984 aler_min -0.344 -0.846 -0.320 -0.344 -0.020
aler_max 0.421 0.017 0.369 0.421 1.198 naled 5.106 0.427 4.880 5.106 12.081
naler_min 45.672 38.796 46.586 45.672 49.703 naler_max 56.118 50.297 55.625
56.118 67.828

We see there that public has an ALER of -0.34, 0.42. When we consider that the
median math score in the dataset is 12.9, this ALER indicates that the minimum
of any ALE y value for public (when public == TRUE) is -0.34 below the median.
This is shown at the 12.6 mark in the plot above. The maximum (public ==
FALSE) is 0.42 above the median, shown at the 13.3 point above.

The unit for ALER is the same as the outcome variable; in our case, that is
math_avg ranging from 2 to 20. No matter what the average ALE values might be,
the ALER quickly shows the minimum and maximum effects of any value of the x
variable on the y variable.

In Table 5, we see the ALE effect size measures for the numeric
academic_ratio.

Table 5: Effect size measures for the numeric academic_ratio from the math achievements dataset  statistic | estimate | conf.low | median | mean | conf.high
---|---|---|---|---|---
aled | 0.630 | 0.324 | 0.615 | 0.630 | 1.023
aler_min | -3.660 | -7.770 | -3.577 | -3.660 | -0.509
aler_max | 1.765 | 0.688 | 1.734 | 1.765 | 2.905
naled | 8.139 | 3.802 | 7.913 | 8.139 | 13.151
naler_min | 18.378 | 1.536 | 14.907 | 18.378 | 45.093
naler_max | 74.281 | 58.688 | 74.541 | 74.281 | 88.347

The ALER for academic_ratio is considerably broader with -3.66 below and 1.77
above the median.

####  5.2.2 ALE deviation (ALED)

While the ALE range shows the most extreme effects a variable might have on
the outcome, the ALE deviation indicates its average effect over its full
domain of values. The zero-centred ALE values, it is conceptually similar to
the weighted mean absolute error (MAE) of the ALE y values. Mathematically, we
see this in Equation 2.

$\hypertarget{eq-
aled}{}{\mathrm{ALED}(\mathrm{ale\\_y},\mathrm{ale\\_n})=\frac{\sum_{i=1}^{k}\left|\mathrm{ale\\_y}_{i}\times\mathrm{ale\\_n}_{i}\right|}{\sum_{i=1}^{k}\mathrm{ale\\_n}_{i}}}$
(2)

where $i$ is the index of $k$ ALE x intervals for the variable (for a
categorical variable, this is the number of distinct categories),
$\mathrm{ale\\_y}_{i}$ is the ALE y value for the $i$th ALE x interval, and
$\mathrm{ale\\_n}_{i}$ is the number of rows of data in the $i$th ALE x
interval.

Based on its ALED, we can say that the average effect on math scores of
whether a school is in the public or Catholic sector is 0.38 (again, out of a
range from 2 to 20). In the effects plot above, the ALED is indicated by a
white box bounded by parentheses ( and ). As it is centred on the median, we
can readily see that the average effect of the school sector barely exceeds
the limits of the median band, indicating that it barely exceeds our threshold
of practical relevance. The average effect for the ratio of academic track
students is slightly higher at 0.63. We can see on the plot that it slightly
exceeds the median band on both sides, indicating its slightly stronger
effect. We will comment on the values of other variables when we discuss the
normalized versions of these scores, to which we proceed next.

###  5.3 Normalized ALE effect size measures

Since ALER and ALED scores are scaled on the range of y for a given dataset,
these scores cannot be compared across datasets. Thus, we present normalized
versions of each with intuitive, comparable values. For intuitive
interpretation, we normalize the scores on the minimum, median, and maximum of
any dataset. In principle, we divide the zero-centred y values in a dataset
into two halves: the lower half from the 0th to the 50th percentile (the
median) and the upper half from the 50th to the 100th percentile. (Note that
the median is included in both halves). With zero-centred ALE y values, all
negative and zero values are converted to their percentile score relative to
the lower half of the original y values while all positive ALE y values are
converted to their percentile score relative to the upper half. (Technically,
this percentile assignment is called the empirical cumulative distribution
function (ECDF) of each half.) Each half is then divided by two to scale them
from 0 to 50 so that together they can represent 100 percentiles. (Note: when
a centred ALE y value of exactly 0 occurs, we choose to include the score of
zero ALE y in the lower half because it is analogous to the 50th percentile of
all values, which more intuitively belongs in the lower half of 100
percentiles.) The transformed maximum ALE y is then scaled as a percentile
from 0 to 100%. Its formula is in Equation 3.

$\hypertarget{eq-
norm_ale_y}{}{\mathrm{norm\\_ale\\_y}=100\times\begin{cases}\frac{ECDF_{y_{\geq
0}}(\mathrm{ale\\_y})}{2}&\text{if }\mathrm{ale\\_y}>0\\\ \frac{-ECDF_{y_{\leq
0}}(\mathrm{ale\\_y})}{2}&\text{otherwise}\end{cases}}$ (3)

where - $ECDF_{y_{\geq 0}}$ is the ECDF of the non-negative values in y. -
$-ECDF_{y_{\leq 0}}$ is the ECDF of the negative values in y after they have
been inverted (multiplied by -1).

Of course, the formula could be simplified by multiplying by 50 instead of by
100 and not dividing the ECDFs by 2 each. But we prefer the form we have given
because it is explicit that each ECDF represents only half the percentile
range and that the result is scored to 100 percentiles.

####  5.3.1 Normalized ALER (NALER)

Based on this normalization, we first have the normalized ALER (NALER), which
scales the minimum and maximum ALE y values from 0 to 100%, centred on 50%, as
seen in Equation 4.

$\hypertarget{eq-
naler}{}{\mathrm{NALER}(\mathrm{y,ale\\_y})=\\{\min(\mathrm{norm\\_ale\\_y})+100,\max(\mathrm{norm\\_ale\\_y})+100\\}}$
(4)

where $y$ is the full vector of y values in the original dataset, required to
calculate $\mathrm{norm\\_ale\\_y}$.

ALER shows the extreme values of a variable’s effect on the outcome. In the
effects plot above, it is indicated by the extreme ends of the horizontal
lines for each variable. We see there that public has an ALER of -0.34, 0.42.
When we consider that the median math score in the dataset is 12.9, this ALER
indicates that the minimum of any ALE y value for public (when public == TRUE)
is -0.34 below the median. This is shown at the 12.6 mark in the plot above.
The maximum (public == FALSE) is 0.42 above the median, shown at the 13.3
point above. The ALER for academic_ratio is considerably broader with -3.66
below and 1.77 above the median.

The result of this transformation is that NALER values can be interpreted as
percentiles with respect to the range of y around the median (50%). naler_min
is always less than 50% and naler_max is always greater than 50%. Their
numbers represent the limits of the effect of the x variable with units in
percentile scores of y. In the effects plot above, because the percentile
scale on the top corresponds exactly to the raw scale below, the NALER limits
are represented by exactly the same points as the ALER limits; only the scale
changes. The scale for ALER and ALED is the lower scale of the raw outcomes;
the scale for NALER and NALED is the upper scale of percentiles.

So, with a NALER of 45.67, 56.12, the minimum of any ALE value for public
(public == TRUE) shifts math scores to the 46th percentile of y values whereas
the maximum (public == FALSE) shifts math scores to the 56th percentile.
Academic track ratio has a NALER of 18.38, 74.28, ranging from the 18th to the
74th percentiles of math scores.

####  5.3.2 Normalized ALED (NALED)

The normalization of ALED scores applies the same ALED formula as before but
on the normalized ALE values instead of on the original ALE y values, as seen
in Equation 5.

$\hypertarget{eq-
naled}{}{\mathrm{NALED}(y,\mathrm{ale\\_y},\mathrm{ale\\_n})=\mathrm{ALED}(\mathrm{norm\\_ale\\_y},\mathrm{ale\\_n})}$
(5)

NALED produces a score that ranges from 0 to 100%. It is essentially the ALED
expressed in percentiles, that is, the average effect of a variable over its
full domain of values. So, the NALED of public school status of 5.1 indicates
that its average effect on math scores spans the middle 5.1% of scores.
Academic ratio has an average effect expressed in NALED of 8.1% of scores.

The NALED is particularly helpful in comparing the practical relevance of
variables against our threshold by which we consider that a variable needs to
shift the outcome on average by more than 5% of the median values. This
threshold is the same scale as the NALED. So, we can tell that public school
status with its NALED of 5.1 just barely crosses our threshold.

###  5.4 The median band and random variables

Figure 7: ALE plot for the random variable from mathematics achievement scores

It is particularly striking to focus on the random rand_norm. We reexamine its
ALE plot in Figure 7 and then we examine its ALE effect size measures in Table
6.

Table 6: Effect size measures for the random variable from the math achievements dataset  statistic | estimate | conf.low | median | mean | conf.high
---|---|---|---|---|---
aled | 0.324 | 0.094 | 0.334 | 0.324 | 0.527
aler_min | -1.450 | -4.109 | -1.195 | -1.450 | -0.159
aler_max | 1.424 | 0.253 | 1.417 | 1.424 | 2.699
naled | 4.452 | 1.367 | 4.413 | 4.452 | 7.242
naler_min | 34.315 | 10.562 | 35.405 | 34.315 | 46.250
naler_max | 69.834 | 53.125 | 70.625 | 69.834 | 86.960

rand_norm has a NALED of 4.5. It might be surprising that a purely random
value has any “effect size” to speak of, but statistically, it must have some
numeric value or the other. However, by setting our default value for the
median band at 5%, we effectively exclude rand_norm from serious
consideration. Setting the median band too low at a value like 1% would not
have excluded the random variable, but 5% seems like a suitable balance. Thus,
the effect of a variable like the discrimination climate score (discrim, 4.1)
should probably not be considered practically meaningful.

Figure 8: ALE plot for the random variable from diamond prices

This point is even more striking when we examine the ALE plot of the random
variable from our large diamonds dataset in Figure 8 and then we examine its
ALE effect size measures in Table 7. Here we see that, even for a large
dataset with more values across which to average, a purely random variable can
vary by as much as 2.5%.

Table 7: Effect size measures for the random variable from the diamonds dataset  statistic | estimate | conf.low | mean | median | conf.high
---|---|---|---|---|---
aled | 250.398 | 100.432 | 250.398 | 219.003 | 599.682
aler_min | -517.816 | -1308.046 | -517.816 | -493.426 | -23.858
aler_max | 1206.920 | 1206.920 | 1206.920 | 1206.920 | 1206.920
naled | 2.451 | 0.977 | 2.451 | 2.084 | 6.420
naler_min | 44.418 | 33.861 | 44.418 | 45.026 | 49.739
naler_max | 61.711 | 61.711 | 61.711 | 61.711 | 61.711
Figure 9: ALE effects plot for diamonds prices

The analysis of the diamonds dataset is better contextualized when we examine
its ALE effects plot in Figure 9. Here we see that three out of nine variables
have NALED values less than that of the purely random variable, indicating
that those variables cannot be considered to have any meaningful variation,
despite the appearances of slight patterns seen in their ALE plots above.

On one hand, 5% as a threshold for the median band might seem to be somewhat
arbitrary, inspired by traditional $\alpha$ = 0.05 for statistical
significance and confidence intervals. The “correct” baseline should be a
qualitative question, depending on an analyst’s goals and the context of a
specific study. On the other hand, our initial analyses here show that 5%
seems to be an effective choice for excluding a purely random variable from
consideration, whether for small or large datasets.

##  6 Statistical inference with ALE

Although effect sizes are valuable in summarizing the global effects of each
variable, they mask much nuance, since each variable varies in its effect
along its domain of values. Thus, ALE is particularly powerful in its ability
to make fine-grained inferences of a variable’s effect depending on its
specific value. Even more so than with effect sizes, statistical inference is
primarily a topic considered with relatively smaller datasets. Thus, we
demonstrate the principles in this section only using the mathematics
achievement dataset. That said, everything we describe here is fully relevant
to large datasets analyzed with ML.

###  6.1 Classical statistical inference

Table 8: Bootstrapped coefficents for GAM for mathematics achievement scores  term | estimate | conf.low | mean | median | conf.high | std.error
---|---|---|---|---|---|---
(Intercept) | 12.713 | 11.654 | 12.713 | 12.718 | 13.613 | 0.484
publicTRUE | -0.689 | -2.015 | -0.689 | -0.652 | 0.415 | 0.637
high_minorityTRUE | 1.031 | -0.318 | 1.031 | 1.054 | 2.324 | 0.676
s(size) | 3.536 | 1.000 | 3.536 | 2.503 | 8.385 | 2.603
s(academic_ratio) | 6.085 | 1.000 | 6.085 | 7.323 | 8.756 | 2.810
s(female_ratio) | 4.203 | 1.000 | 4.203 | 3.652 | 8.396 | 2.382
s(mean_ses) | 7.492 | 2.561 | 7.492 | 8.449 | 8.995 | 2.008
s(minority_ratio) | 7.300 | 2.588 | 7.300 | 8.120 | 8.987 | 1.797
s(discrim) | 4.205 | 1.000 | 4.205 | 3.560 | 8.800 | 2.673
s(rand_norm) | 6.796 | 1.000 | 6.796 | 7.777 | 8.767 | 2.394

We begin by briefly reviewing the classical approach to statistical inference.
First, we can see the bootstrapped values of the effects of individual
variables in Table 8. It is beyond the scope of this article to explain in
detail how GAM works (see Ross 2019 for a tutorial). However, for our model
illustration here, the estimates for the parametric variables (the non-numeric
ones in our model) are interpreted as regular statistical regression
coefficients whereas the estimates for the non-parametric smoothed variables
(those whose variable names are encapsulated by the smooth s() function) are
actually estimates for expected degrees of freedom (EDF in GAM). The smooth
function s() lets GAM model these numeric variables as flexible curves that
fit the data better than a straight line. The estimate values for the smooth
variables above are not so straightforward to interpret, but suffice it to say
that they are completely different from regular regression coefficients.

With bootstrap-based confidence intervals, based on the default 95% confidence
intervals, a coefficient is statistically significant if conf.low and
conf.high are both positive or both negative. However, the statistical
significance of the estimate (EDF) of the smooth terms is meaningless here
because EDF cannot go below 1.0. Thus, even the random term s(rand_norm)
appears to be “statistically significant”. Only the values for the non-smooth
(parametric terms) public and high_minority should be considered here. So, we
find that neither of the coefficient estimates of public nor of high_minority
has an effect that is statistically significantly different from zero. (The
intercept is not conceptually meaningful here; it is a statistical artifact.)

This initial analysis highlights two limitations of classical hypothesis-
testing analysis. First, it might work suitably well when we use models that
have traditional linear regression coefficients. But once we use more advanced
models like GAM that flexibly fit the data, we cannot interpret coefficients
meaningfully and so it is not so clear how to reach inferential conclusions.
Second, a basic challenge with models that are based on the general linear
model (including GAM and almost all other statistical analyses) is that their
coefficient significance compares the estimates with the null hypothesis that
there is no effect. However, even if there is an effect, it might not be
practically meaningful. As we will see, ALE-based statistics are explicitly
tailored to emphasize practical implications beyond the notion of “statistical
significance”.

###  6.2 ALE data structures for categorical and numeric variables

To understand how bootstrapped ALE can be used for statistical inference, we
must understand the structure of ALE data. We can begin by examining the
structure for a binary variable with just two categories, public, in Table 9.

Table 9: Structure of ALE data for public (categorical) from math achievement
dataset  ale_x ale_n ale_y ale_y_lo ale_y_hi FALSE 70 13.302 12.192 14.176
TRUE 90 12.613 11.892 13.350

To understand how bootstrapped ALE can be used for statistical inference, we
must understand the structure of ALE data. We can begin by examining the
structure for a binary variable with just two categories, public, in Table 9.
The columns for a categorical variable mean:

  * •

ale_x: the different categories that exist in the categorical variable.

  * •

ale_n: the number of rows for that category in the dataset provided to the
function.

  * •

ale_y: the mean bootstrapped ALE function value calculated for that category.

  * •

ale_y_lo and ale_y_hi: the lower and upper confidence intervals for the
bootstrapped ale_y value.

By default, the ale package centres ALE values on the median of the outcome
variable; in our dataset, the median of all the schools’ average mathematics
achievement scores is 12.9. With ALE centred on the median, the weighted sum
of ALE y values (weighted on ale_n) above the median is approximately equal to
the weighted sum of those below the median. So, in the ALE plots above, when
we consider the number of instances indicated by the rug plots and category
percentages, the average weighted ALE y approximately equals the median.

Table 10: Structure of ALE data for academic_ratio (numeric) from math achievement dataset  ale_x | ale_n | ale_y | ale_y_lo | ale_y_hi
---|---|---|---|---
0.00 | 1 | 9.273 | 4.628 | 12.241
0.05 | 2 | 10.935 | 8.749 | 12.819
0.09 | 1 | 11.738 | 10.693 | 12.776
0.10 | 2 | 11.924 | 10.947 | 13.174
0.13 | 1 | 12.367 | 11.322 | 14.011
0.14 | 2 | 12.246 | 11.150 | 13.478
0.17 | 1 | 12.312 | 11.426 | 13.527
0.18 | 4 | 12.364 | 11.389 | 13.608
0.19 | 3 | 12.409 | 11.521 | 13.563
0.20 | 3 | 12.394 | 11.489 | 13.440
… | … | … | … | …
0.81 | 3 | 13.589 | 12.748 | 14.381
0.82 | 1 | 13.611 | 12.739 | 14.581
0.85 | 2 | 13.703 | 12.690 | 14.736
0.87 | 1 | 13.827 | 12.848 | 14.892
0.90 | 2 | 13.874 | 12.867 | 15.073
0.91 | 1 | 13.874 | 12.956 | 15.048
0.95 | 2 | 14.179 | 13.234 | 15.299
0.96 | 4 | 14.242 | 13.347 | 15.339
0.97 | 3 | 14.330 | 13.393 | 15.373
1.00 | 7 | 14.625 | 13.363 | 16.021

In Table 10, we see the ALE data structure for a numeric variable,
academic_ratio. The columns are the same as with a categorical variable, but
the meaning of ale_x is different since there are no categories. To calculate
ALE for numeric variables, the range of x values is divided into fixed
intervals (such as into 100 percentiles). If the x values have fewer than 100
distinct values in the data, then each distinct value becomes an ale_x
interval. (This is often the case with smaller datasets like ours; here
academic_ratio has only 65 distinct values.) If there are more than 100
distinct values, then the range is divided into 100 percentile groups. So,
ale_x represents each of these x-variable intervals. The other columns mean
the same thing as with categorical variables: ale_n is the number of rows of
data in each ale_x interval and ale_y is the calculated ALE for that ale_x
value.

###  6.3 Inference with ALE based on bootstrapped confidence intervals

Whereas the coefficient table above based on classic statistics indicated this
conclusion for public, it indicated that high_minority had a statistically
significant effect; our ALE analysis indicates that high_minority does not. In
addition, e

With the structure of ALE data clear, we can now proceed to statistical
inference with ALE based on bootstrapped confidence intervals. In a
bootstrapped ALE plot, values within the confidence intervals are
statistically significant; values outside of the median band can be considered
at least somewhat meaningful. Thus, the essence of ALE-based statistical
inference is that only effects that are simultaneously within the confidence
intervals AND outside of the median band should be considered conceptually
meaningful.

Figure 10: ALE plot for academic_ratio from math achievement dataset Table 11: Confidence regions for academic_ratio (numeric) from math achievement dataset  start_x | end_x | x_span | n | n_pct | start_y | end_y | trend | relative_to_mid
---|---|---|---|---|---|---|---|---
0.00 | 0.00 | 0.00 | 1 | 0.006 | 9.273 | 9.273 | 0.000 | below
0.05 | 0.91 | 0.86 | 143 | 0.894 | 10.935 | 13.874 | 0.230 | overlap
0.95 | 1.00 | 0.05 | 16 | 0.100 | 14.179 | 14.625 | 0.599 | above

We can see this, for example, with the plot of academic_ratio in Figure 10.
However, it might not always be easy to tell from a plot which regions are
relevant, so the results of statistical significance are summarized with a
confidence regions table in Table 11. For numeric variables, the confidence
regions summary has one row for each consecutive sequence of x values that
have the same status: all values in the region are below the median band, they
overlap the band, or they are all above the band. Here are the summary
components:

  * •

start_x is the first and end_x is the last x value in the sequence. start_y is
the y value that corresponds to start_x while end_y corresponds to end_x.

  * •

n is the number of data elements in the sequence; n_pct is the percentage of
total data elements out of the total number.

  * •

x_span is the length of x of the sequence that has the same confidence status.
However, so that it may be comparable across variables with different units of
x, x_span is expressed as a percentage of the full domain of x values.

  * •

trend is the average slope from the point (start_x, start_y) to (end_x,
end_y). Because only the start and end points are used to calculate trend, it
does not reflect any ups and downs that might occur between those two points.
Since the various x values in a dataset are on different scales, the scales of
the x and y values in calculating the trend are normalized on a scale of 0 to
1 each so that the trends for all variables are directly comparable. A
positive trend means that, on average, y increases with x; a negative trend
means that, on average, y decreases with x; a zero trend means that y has the
same value at its start and end points–this is always the case if there is
only one point in the indicated sequence.

  * •

relative_to_mid is the key information here. It indicates if all the values in
sequence from start_x to end_x are below, overlapping, or above the median
band:

    * –

below: the higher limit of the confidence interval of ALE y (ale_y_hi) is
below the lower limit of the median band.

    * –

above: the lower limit of the confidence interval of ALE y (ale_y_lo) is above
the higher limit of the median band.

    * –

overlap: neither of the first two conditions holds; that is, the confidence
region from ale_y_lo to ale_y_hi at least partially overlaps the median band.

These results tell us simply that, for academic_ratio, from 0 to 0, ALE is
below the median band from 9.27 to 9.27. From 0.05 to 0.91, ALE overlaps the
median band from 10.9 to 13.9. From 0.95 to 1, ALE is above the median band
from 14.2 to 14.6. Considering the details from Table 11, we can see that
academic_ratio overlaps with the median range for most of its range. Although
the lowest value is nominally below the median band, since this represents
only 1 data point (0.6% of the dataset), we can discount that finding.
However, it is meaningful that when the ratio of academic subjects is 95% or
higher, representing 10% of the schools in the dataset, the average math
achievement scores are above the median. We would consider this a practically
meaningful result supported by analysis.

Figure 11: ALE plot for public from math achievement dataset Table 12: Confidence regions for public (categorical) from math achievement dataset  x | n | n_pct | y | relative_to_mid
---|---|---|---|---
FALSE | 70 | 0.438 | 13.302 | overlap
TRUE | 90 | 0.562 | 12.613 | overlap

Confidence region summary tables are available not only for numeric but also
for categorical variables, as we see with the ALE plot for public (Figure 11)
and its confidence regions summary table in Table 12. Since we have categories
here, there are no start or end positions and there is no trend. We instead
have each x category and its single ALE y value, with the n and n_pct of the
respective category and relative_to_mid as before to indicate whether the
indicated category is below, overlaps with, or is above the median band. These
results tell us that, for public, for FALSE, the ALE of 13.3 overlaps the
median band. For TRUE, the ALE of 12.6 also overlaps the median band. In other
words, there is no statistically (or practically) significant value of public.
The data does not provide evidence to support the claim that public or non-
public schools have significantly different math achievement scores.

###  6.4 Confidence regions and random variables

Figure 12: ALE plot for the random variable from math achievement dataset Table 13: Confidence regions for the full-model (appropriately) bootstrapped random variable from math achievement dataset  start_x | end_x | x_span | n | n_pct | start_y | end_y | trend | relative_to_mid
---|---|---|---|---|---|---|---|---
-2.397 | 2.608 | 1 | 160 | 1 | 11.924 | 12.545 | 0.042 | overlap

Again, our random variable rand_norm is particularly interesting, as we can
see from its ALE plot in Figure 12 and its confidence regions summary table in
Table 13. Despite the apparent pattern, we see that from -2.4 to 2.61, ALE
overlaps the median band from 11.9 to 12.5. So, despite the random highs and
lows in the bootstrap confidence interval, there is no reason to suppose that
the random variable has any effect anywhere in its domain.

Figure 13: ALE plot for the data-only (inappropriately) bootstrapped random variable from math achievement dataset Table 14: Confidence regions for the data-only (inappropriately) bootstrapped random variable from math achievement dataset  statistic estimate conf.low mean median conf.high aled 0.105 0.093 0.105 0.104 0.118 aler_min -0.196 -0.196 -0.196 -0.196 -0.196 aler_max 0.402 0.319 0.402 0.402 0.489 naled 1.552 1.430 1.552 1.534 1.751 naler_min 46.875 46.875 46.875 46.875 46.875 naler_max 54.706 53.750 54.706 55.000 55.000 Table 15: Confidence regions for academic_ratio (numeric) from math achievement dataset  start_x | end_x | x_span | n | n_pct | start_y | end_y | trend | relative_to_mid
---|---|---|---|---|---|---|---|---
-2.397 | -1.952 | 0.089 | 2 | 0.013 | 12.700 | 12.725 | 0.019 | below
-1.722 | 2.015 | 0.747 | 154 | 0.963 | 12.734 | 13.203 | 0.042 | overlap
2.207 | 2.608 | 0.080 | 4 | 0.025 | 13.243 | 13.323 | 0.067 | above

It is important to remember that our analysis of the math achievement dataset
uses model bootstrapping because its dataset is too small to be analyzed with
a train-test split. If, however, we were to use the inappropriate data-only
bootstrapping, as we did above for illustration, we can see the random
variable from that analysis in Figure 13, its ALE effect size measures in
Table 14, and its confidence regions summary table in Table 15.

The data-only bootstrap makes rand_norm appear more reliably irrelevant with a
very low NALED of 1.6%. But we note that the considerably higher NALED of 4.5%
in the model bootstrap probably more accurately reflects the variability of
the random variable. More seriously, the data-only bootstrap, with its
excessively narrow confidence intervals, makes rand_norm appear to have
statistically significant and practically relevant results at its two
extremes, with the inaccurate conclusion that the lowest values are slightly
below the median bar while the highest values are slightly above it. Leaving
aside the fact that these extremes represent only 1.3% and 2.5% of the data
points, respectively, we see that data-only bootstrapping suggests dubious
conclusions whereas the appropriate model bootstrapping leaves no doubt that
rand_norm has no practically relevant effect on math achievement scores.

##  7 Discussion

In the preceding sections, we have navigated through the multifaceted
landscape of ALE, exploring its potential and the innovative extensions
proposed in this study. The contributions we present not only forge a path
towards more nuanced, interpretable, and computationally efficient
applications in ML model interpretation. Through the implementation of full
model bootstrapping, the introduction of intuitive ALE-based effect size
measures, and the innovative concept of confidence regions, this work
endeavours to enhance the robustness and interpretability of ALE analyses.

In this concluding discussion, we shall reflect on the implications,
applications, and potential future trajectories of these contributions,
anchoring our discourse in the context of both the theoretical and practical
domains of ML model interpretation and analysis.

###  7.1 Contributions

The extensions to the implementation of ALE that we have described in the
preceding sections address the opportunities for improvement that we highlight
above in the section on related work.

Whereas data-only bootstrapping of ALE values has already been implemented

First, we implement full model bootstrapping specifically tailored for
calculating ALE rather than just data-only bootstrapping. This allows the
creation of appropriate confidence intervals for the ALE of models that are
trained on all available data, which is typically the case for small datasets.
It is not so surprising that the Python packages that implement bootstrapping
or variations feature just data-only bootstrapping (Flora 2023; Jumelle, Kuhn-
Regnier and Rajaratnam 2020). Python is rarely used to analyze small datasets
that are not amenable to a train-test split; smaller datasets are typically
analyzed with R rather than with Python.

Second, we introduce two new pairs of effect size measures based on ALE
designed for intuitive interpretability. Unlike the overall model MEC and the
IAS (Molnar, Casalicchio and Bischl 2020) that only show the average effects
across all variables, the measures we present here show the effects for
individual variables. The ALE deviation indicates the average dispersion of
ALE values while the sALE range indicates the maximum dispersion. These base
measures are scaled to the Y outcome variable and so are readily interpretable
in terms of the applications to the outcome; they are also directly comparable
across predictor variables. For meaningful comparison of effect sizes across
different datasets and contexts, each of these has a normalized version that
is scaled on a percentile scale relative to the range of possible Y outcome
values.

Third, to address the nuances of non-linear relationships, we go beyond effect
size measures and introduce the notion of confidence regions as an approach to
statistical inference based on ALE confidence intervals. Whereas several ML
effect size measures like the impact score (Lötsch and Ultsch 2020) and
$f^{2}_{v}$ (Messner 2023)—and, indeed, ALED, ALER, NALED, and NALER—only
indicate a general effect, Messner (2023) has made an initial effort in
indicating the complexity of relationships by testing for monotonicity and
average direction. Confidence regions fully embrace the potential complexity
of relationships by not trying to reduce them to a single number. Rather, they
identify areas significantly above or below a median band. For numeric
predictors, confidence regions indicate the trend within these regions to
infer the average slope, somewhat mirroring Messner’s slope measure, but on a
more granular scale that communicates necessary nuances. Monotonicity is less
pertinent from this perspective; understanding the relationship’s intricate
shape, which necessitates graphical interpretation, holds greater significance
than a single numeric summary.

A fourth minor but non-negligible contribution is that, as these measures and
techniques are all calculated directly from the data generated from ALE, they
not only inherit the inherent superior computational efficiency compared to
other processor-intensive effect size measures (Molnar 2022), but they are
computationally negligible to calculate in any analysis that has already
calculated ALE.

###  7.2 Practical Implications

There are several practical implications of our ALE-based effect size measures
and of statistical inference using bootstrapped ALE confidence regions.

Overall effect size measures afford a comprehensive perspective on model
performance, providing a global depiction of the impact of each variable on
the outcome throughout the model. In contrast, due to the non-homogeneous
effect across the entire domain of the X input factors, the ALE confidence
regions delineate specific zones within the domain of X where an input
predictor may be active or inactive.

The confidence regions offer a nuanced context to aid the interpretation of
the overall measures. Specifically, when encountering a particularly broad ALE
range or a substantial ALE range interval, coupled with a more modest ALE
deviation score, the confidence regions clarify which areas of the domain
exhibit atypical effects of that variable.

Overall effect size measures facilitate rapid identification of variables that
most significantly impact the outcomes of a system of interest. Subsequently,
for precise action, the confidence regions assist decision-makers in focusing
on specific values or ranges of interest that might exert undesirable or
desirable effects on the outcome.

The intriguing occurrence of a substantial disparity between overall effects,
as indicated by ALE deviation, and extreme effects, as highlighted by ALE
range, can be dissected with the assistance of confidence regions. These focus
on the origins of such disparities and, instead of being deemed problematic,
can serve as a fountainhead of unexpected insights. Given that ALE averages
effects across intervals, spikes in the ALE range do not invariably represent
the influence of potential spurious outliers; they typically signify the
effects within a particular data range. Such disparities generally merit
closer investigation, often paving the way for unforeseen and valuable
insights.

Generally, a pitfall of any overall effect size measure is that it may
misleadingly convey the data as more homogeneous than it truly is. While some
extensions have been proposed to the original ALE algorithm to account for
such data heterogeneity, the confidence regions adeptly illuminate them by
indicating where effects are homogeneous and where they are heterogeneous.

###  7.3 Conclusion

In this article, we have unveiled novel methodologies and metrics that enhance
the applicability and interpretability of ALE in ML model analysis. Our
contributions span the implementation of a full model bootstrapping approach,
tailored for ALE, which facilitates the creation of apt confidence intervals,
especially pivotal for models trained on smaller datasets. Furthermore, we
introduced two pairs of effect size measures—ALE deviation (ALED) and ALE
range (ALER), along with their normalized counterparts (NALED and
NALER)—designed to provide a nuanced depiction of individual variable effects,
thereby offering a more granular insight compared to existing models. The
introduction of confidence regions, which embrace the complexity of
relationships without reducing them to a singular numeric value, marks a
significant stride towards comprehending the intricacies of non-linear
relationships. Lastly, the computational efficiency of our proposed measures
and techniques, derived directly from ALE data, not only underscores their
practicality but also positions them as computationally viable options in
analyses that have already computed ALE.

As we reflect upon these contributions, we recognize the potential they hold
in bridging theoretical understanding and practical application, thereby
paving the way for more robust, interpretable, and insightful analyses in the
realm of ML model interpretation. Future endeavours may explore further
applications and validations of these methodologies across diverse datasets
and contexts, ensuring their robustness and utility in varied analytical
scenarios.

##  8 Acknowledgments

We used ChatGPT in developing the outline of this article and in drafting much
of the text. The author takes full responsibility for the entirety of the
final revised contents. In addition, we used ChatGPT extensively in developing
the code for the ale package.

##  9 References

reApley, Daniel W. 2018. _ALEPlot: Accumulated Local Effects (ALE) Plots and
Partial Dependence (PD) Plots_. 2018. 1.1.
<https://cran.r-project.org/web/packages/ALEPlot/> Retrieved June 28, 2023.

preApley, Daniel W. and Jingyu Zhu. 2020. Visualizing the Effects of Predictor
Variables in Black Box Supervised Learning Models. _Journal of the Royal
Statistical Society Series B: Statistical Methodology_ 82, 4, 1059–1086.
<https://doi.org/10.1111/rssb.12377> Retrieved June 16, 2023.

preBiecek, Przemyslaw, Szymon Maksymiuk and Hubert Baniecki. 2023. _DALEX:
moDel Agnostic Language for Exploration and eXplanation_. 2023. 2.4.3.
<https://cran.r-project.org/web/packages/DALEX/index.html> Retrieved October
10, 2023.

preFlora, Montgomery. 2023. _Scikit-explain: A user-friendly python package
for computing and plotting machine learning explainability output._ 2023\.
0.1.4. <https://github.com/monte-flora/scikit-explain/> Retrieved October 10,
2023.

preFriedman, Jerome H. and Bogdan E. Popescu. 2008. Predictive Learning via
Rule Ensembles. _The Annals of Applied Statistics_ 2, 3, 916–954.
<https://www.jstor.org/stable/30245114> Retrieved October 12, 2023.

preGkolemis, Vasilis et al. 2023. RHALE: Robust and Heterogeneity-aware
Accumulated Local Effects. <http://arxiv.org/abs/2309.11193> Retrieved October
5, 2023.

preGkolemis, Vasilis, Theodore Dalamagas and Christos Diou. 2023. DALE:
Differential Accumulated Local Effects for efficient and accurate global
explanations. In _Proceedings of The 14th Asian Conference on Machine
Learning_ , 375–390. PMLR.
<https://proceedings.mlr.press/v189/gkolemis23a.html> Retrieved October 5,
2023.

preJomar, Dana. 2023. _PyALE: ALE plots with python_. 2023. 1.1.3.
<https://github.com/DanaJomar/PyALE> Retrieved October 10, 2023.

preJumelle, Maxime, Alexander Kuhn-Regnier and Sanjif Rajaratnam. 2020.
_ALEPython_. 2020. <https://github.com/blent-ai/ALEPython> Retrieved October
10, 2023.

preLötsch, Jörn and Alfred Ultsch. 2020. A non-parametric effect-size measure
capturing changes in central tendency and data distribution shape. _PLOS ONE_
15, 9, e0239623.
<https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0239623>
Retrieved October 12, 2023.

preMayer, Michael. 2021. A Curious Fact on the Diamonds Dataset – Michael’s
and Christian’s Blog. <https://lorentzen.ch/index.php/2021/04/16/a-curious-
fact-on-the-diamonds-dataset/> Retrieved October 3, 2023.

preMessner, Wolfgang. 2023. From black box to clear box: A hypothesis testing
framework for scalar regression problems using deep artificial neural
networks. _Applied Soft Computing_ 146, 110729.
<https://www.sciencedirect.com/science/article/pii/S1568494623007470>
Retrieved October 11, 2023.

preMolnar, Christoph. 2022. _Interpretable Machine Learning_.
<https://christophm.github.io/interpretable-ml-book/> Retrieved January 17,
2019.

preMolnar, Christoph, Giuseppe Casalicchio and Bernd Bischl. 2020. Quantifying
Model Complexity via Functional Decomposition for Better Post-hoc
Interpretability. In P. Cellier and K. Driessens (eds.), _Machine Learning and
Knowledge Discovery in Databases_ , Communications in Computer and Information
Science, 193–204. Cham: Springer International Publishing.

preMolnar, Christoph and Patrick Schratz. 2022. _Iml: Interpretable Machine
Learning_. 2022. 0.11.1. <https://cran.r-project.org/web/packages/iml/>
Retrieved October 10, 2023.

preOkoli, Chitu. 2023. _Ale: Interpretable Machine Learning and Statistical
Inference with Accumulated Local Effects (ALE)_. 2023. 0.2.0.
<https://cran.r-project.org/web/packages/ale/index.html> Retrieved October 10,
2023.

prePinheiro, José et al. 2023. _Nlme: Linear and Nonlinear Mixed Effects
Models_. 2023. 3.1-163. <https://cran.r-project.org/web/packages/nlme/>
Retrieved October 13, 2023.

preRapidMiner. 2023. _Interpretation_. 2023. 0.7.1.
<https://marketplace.rapidminer.com/UpdateServer/faces/product_details.xhtml?productId=rmx_interpretation>
Retrieved October 10, 2023.

preRoss, Noam. 2019. Introduction to Generalized Additive Models · Generalized
Additive Models in R. <https://noamross.github.io/gams-in-r-course/chapter1>
Retrieved October 13, 2023.

preSechidis, Konstantinos. 2015. _Hypothesis testing and feature selection in
semi-supervised data_. PhD thesis. University of Manchester.
<https://www.research.manchester.ac.uk/portal/en/theses/hypothesis-testing-
and-feature-selection-in-semisupervised-
data(97f5f950-f020-4ace-b6cd-49cb2f88c730).html> Retrieved October 12, 2023.

preSeldon Technologies. 2023. _Alibi: Algorithms for monitoring and explaining
machine learning models_. 2023. 0.9.4. <https://github.com/SeldonIO/alibi>
Retrieved October 10, 2023.

preTong, L. I., R. Saminathan and C. W. Chang. 2017. Uncertainty Assessment of
Non-normal Emission Estimates Using Non-parametric Bootstrap Confidence
Intervals. _Journal of Environmental Informatics_ 28, 1, 1, 61–70.
<http://www.jeionline.org/index.php?journal=mys&page=article&op=view&path%5B%5D=201500322>
Retrieved October 11, 2023.

preWickham, Hadley et al. 2023. _Ggplot2: Create Elegant Data Visualisations
Using the Grammar of Graphics_. 2023. 3.4.4.
<https://cran.r-project.org/web/packages/ggplot2/index.html> Retrieved October
13, 2023.

preWood, Simon. 2023. _Mgcv: Mixed GAM Computation Vehicle with Automatic
Smoothness Estimation_. 2023. 1.9-0.
<https://cran.r-project.org/web/packages/mgcv/> Retrieved October 13, 2023.

p