Multiobjective Evolutionary Pruning of Deep Neural Networks with
  Transfer Learning for improving their Performance and Robustness
Evolutionary Computation algorithms have been used to solve optimization
problems in relation with architectural, hyper-parameter or training
configuration, forging the field known today as Neural Architecture Search.
These algorithms have been combined with other techniques such as the pruning
of Neural Networks, which reduces the complexity of the network, and the
Transfer Learning, which lets the import of knowledge from another problem
related to the one at hand. The usage of several criteria to evaluate the
quality of the evolutionary proposals is also a common case, in which the
performance and complexity of the network are the most used criteria. This work
proposes MO-EvoPruneDeepTL, a multi-objective evolutionary pruning algorithm.
MO-EvoPruneDeepTL uses Transfer Learning to adapt the last layers of Deep
Neural Networks, by replacing them with sparse layers evolved by a genetic
algorithm, which guides the evolution based in the performance, complexity and
robustness of the network, being the robustness a great quality indicator for
the evolved models. We carry out different experiments with several datasets to
assess the benefits of our proposal. Results show that our proposal achieves
promising results in all the objectives, and direct relation are presented
among them. The experiments also show that the most influential neurons help us
explain which parts of the input images are the most relevant for the
prediction of the pruned neural network. Lastly, by virtue of the diversity
within the Pareto front of pruning patterns produced by the proposal, it is
shown that an ensemble of differently pruned models improves the overall
performance and robustness of the trained networks.
Javier Poyatos, Daniel Molina, Aitor Martínez, Javier Del Ser, Francisco Herrera
Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness

###### Abstract

Evolutionary Computation algorithms have been used to solve optimization problems in relation with architectural, hyper-parameter or training configuration, forging the field known today as Neural Architecture Search. These algorithms have been combined with other techniques such as the pruning of Neural Networks, which reduces the complexity of the network, and the Transfer Learning, which lets the import of knowledge from another problem related to the one at hand. The usage of several criteria to evaluate the quality of the evolutionary proposals is also a common case, in which the performance and complexity of the network are the most used criteria. This work proposes MO-EvoPruneDeepTL, a multi-objective evolutionary pruning algorithm. MO-EvoPruneDeepTL uses Transfer Learning to adapt the last layers of Deep Neural Networks, by replacing them with sparse layers evolved by a genetic algorithm, which guides the evolution based in the performance, complexity and robustness of the network, being the robustness a great quality indicator for the evolved models. We carry out different experiments with several datasets to assess the benefits of our proposal. Results show that our proposal achieves promising results in all the objectives, and direct relation are presented among them. The experiments also show that the most influential neurons help us explain which parts of the input images are the most relevant for the prediction of the pruned neural network. Lastly, by virtue of the diversity within the Pareto front of pruning patterns produced by the proposal, it is shown that an ensemble of differently pruned models improves the overall performance and robustness of the trained networks.

keywords: Evolutionary Deep Learning, Multi-Objective Algorithms, Pruning, Out of Distribution Detection, Transfer Learning +
Footnote †: journal: Applied Soft Computing

## 1 Introduction

Evolutionary Computation (EC) refers to a family of global optimization algorithms inspired by biological evolution (Back & Schwefel, 1996, May). EC algorithms such as Evolutionary Algorithms (EA) (Back et al., 1997) have been used to solve several complex optimization problems which cannot be analytically solved in polynomial time. In many real-world optimization problems, there is not only one criterion or objective to improve, but several objectives to consider. Multi-Objective Evolutionary Algorithms (MOEAs) are an family of EAs capable of efficiently tackle optimization problems comprising several goals (Deb, 2011).

Structural search and, in particular, Neural Architecture Search (NAS), is one of the non-polynomial problems which has been approached with EAs over the years (Martinez et al., 2021). This problem consists
of looking for neural network configurations that fit better one dataset by optimizing the performance or loss of the network in function of the selected evaluation metric (Stanley and Miikkulainen, 2002). There have been several Neural Networks (NN), and particularly when integrated with Deep Learning (DL) called as Deep Neural Network (DNN), to which NAS has been applied are well-known networks with one or more objectives (Pham et al., 2018; July; Yang et al., 2020; June).

Among other decision variables considered in NAS, this area has also approached the improvement of a NN by optimally pruning their neural connections. Pruning techniques seek to reduce the number of parameters of the network, targeting network architectures with less complexity. Usually this comes at the cost of a lower performance of the network. When a new learning task is present, a manner to compensate the lack of quality of data is the usage of the Transfer Learning (TL), whose most straightforward approximation is the usage of pre-trained network in very large datasets (Krizhevsky et al., 2017) for the extraction of features, followed by a specialization of the last layers of the network. Due to the fact that the number of trainable parameters of these last layers is lower, it is possible to avoid an early overfitting of the network, which can happen if there are few examples for large models. For those cases, the search of optimal pruning patters using evolutionary NAS is done for these last layers (Poyatos et al., 2023).

Robustness is one of the unavoidable requirements to ensure proper performance in scenarios where risks must be controlled and certain guarantees are needed to ensure the proper performance of the models (ISO, 2021a,b). The combination of EAs together with techniques that allow evaluating the robustness of the models paves the way towards the creation of better models for all types of problems. It could be useful to incorporate robustness as a target, but unfortunately, robustness has been rarely considered an objective (Wang et al., 2021). Robustness can be measured in several ways for a DNN model, one being the performance in Out-of-Distribution (OoD) detection problems (Hendrycks and Gimpel, 2016). This problem consists of detecting whether a new test instance queried to the model belongs to the distribution underneath the learning dataset i.e., the In-Distribution (InD) dataset or, instead, it belongs to another different distribution (correspondingly, the Out-Distribution dataset, OoD).

The natural extension of NAS is the development of proposals with several objectives. In this scenario, the MOEAs can take place, as they evolve the networks meanwhile an optimization of several objectives is made (Lu et al., 2022b; Elsken et al., 2019). The MONAS term arises as the union of multi-objective algorithms which are used for NAS problems (MONAS). MONAS algorithms usually rely in several objectives, being a standard objective the performance of the network. The complexity of the network is a common second objective, which can be modeled as the number of parameters pruned form the network, network compression or other alternatives. More sophisticated proposals consider another objective based on the energy consumption or hardware device in use, among others (Chitty-Venkata and Somani, 2022; Wei et al., 2022). The addition of the robustness, with a OoD detection technique applied to the DL model being optimized, as an additional objective unleashes a new vision for the MONAS proposals.

The main hypothesis is the convenience of using a MOEA to evolve the pruning patterns of the fully-connected layers of a neural network via a sparse representation, simultaneously according to the generalization performance of the network, its complexity and the robustness of a OoD detection technique relying on the activation signals inside the network against samples that may or may not belong to the distribution of the training data.

In this work, a novel MOEAs is proposed based on the evolution of the pruning patterns of fully-connected layers, which we hereafter refer to as Multi-Objective Evolutionary Pruning for Deep Transfer Learning (MO-EvoPruneDeepTL). MO-EvoPruneDeepTL uses TL to focus the evolutionary search for pruning patterns on the last layers of the neural network, so as to adapt them to the problem at hand. This work gets inspired by recent work in (Poyatos et al., 2023), in which dense layers are pruned using a configuration that define the active neurons. In the previous work, that configuration evolves using a binary genetic algorithm guided by the performance of the network. In this manuscript, the previous problem is reformulated to optimize the pruning patterns with a MOEA, in which the search is guided by three objectives: performance of the network, its complexity, and the robustness measured by the capability of a OoD detection technique to detect InD and OoD samples. The insight about these objectives is that a most pruned network may reduce its performance and robustness of the OoD detector that works with the activations of this pruned network. For that reason, a minimum number of neurons needs to be active to achieve balanced models with
the performance and robustness. This OoD context falls within the umbrella of the Open-World Learning (OWL) (Parmar et al., 2022; Zhou, 2022). The aim of OWL consist in learning non-controlled environments so that the Artificial Intelligence (AI) agent (in this study the evolved trained models) will become more and more knowledgeable, as the models can be asked with new data. However, OWL is also included in the umbrella of General Purpose Artificial Intelligence (GPAI), which is a relevant and important term nowadays. The basic of GPAI lays in AI generating AI (Clune, 2019; Real et al., 2020, July), because this work uses an EA that learns in base to DL models.

To assess the quality of MO-EvoPruneDeepTL, different experiments have been designed that allow inspecting several aspects of the performance of MO-EvoPruneDeepTL from different perspectives. To that end, the main purpose of the experimental setup is to provide an informed answer to the following research questions (RQ):

* How are the approximated Pareto fronts produced by the proposal in each of the considered datasets?
* Is there any remarkable pruning pattern that appears in all the solutions of the Pareto front?
* Do our models achieve an overall improvement in performance when combined through ensemble modeling?

A general insight about these experiments is the the achievement of optimized networks in these objectives, but also that the evolutionary process gives rise to pruning patterns that maintain relevant neurons with information about the input of the model, and leads to the use of ensembles to further improve modeling performance in terms of generalization and robustness to OoD.

The rest of the article is structured as follows: Section 2 briefly overviews background literature related to the proposal. Section 3 shows the details of the proposed MO-EvoPruneDeepTL model. Section 4 presents the experimental framework designed to thoroughly examine the behavior of MO-EvoPruneDeepTL with respect to the RQ formulated above. In Section 5, we show and discuss in depth the results obtained by MO-EvoPruneDeepTL in the different experiments. Several indicators are presented to show the quality of MO-EvoPruneDeepTL. Finally, Section 6 draws the main conclusions from this study, as well as future research lines stimulated by our findings.

## 2 Related Work

The aim of this section is to make a review of contributions to the literature about the key elements of this study: Neural Architecture Search (Subsection 2.1), Transfer Learning and Pruning of Convolutional Neural Networks (CNN, Subsection 2.2) and OoD detection (Subsection 2.3). The last paragraph of this section resumes the benefits of MO-EvoPruneDeepTL.

### Neural Architecture Search

The design of the NN that best fits for the problem at hand is a challenging task. The search for the best design of the network is also considered as another problem, as it is necessary to find the best architecture that optimally fits the data. In this context, NAS has achieved a great importance in this area. The main purpose of NAS proposals is the search for the best design of the NN to solve the considered problem.

First NAS-based proposals started to emerge in the beginning of this century. NEAT, presented in (Stanley and Miikkulainen, 2002) was a pioneering proposal about how EAs -- specifically, a Genetic Algorithm (GA) - can be used to evolve NNs. They showed that a constructive modeling of the NN with the benefits of the GA can lead to optimized NN topologies. The natural extension of this seminal work allowing for the evolution of DNN was presented years after in (Miikkulainen et al., 2019), in which the authors use a co-evolutionary algorithm based on the co-operation scheme to evolve DNN.

In the last years, more NAS proposals have been developed. One of them is EvoDeep (Martin et al., 2018), in which the authors create an EA with specific operators to create and evolve DL models from scratch. More examples of the importance of NAS come with the next proposals. In (Dufourq and Bassett, 2017, November), authors propose another EA to perform the evolution of NN, similarly to the previous
proposal, but with a difference in relation to the fitness function, which is influenced by the accuracy and complexity of the network. The other example is presented in (Assuncao et al., 2019). In this case, the evolution comes in two different ways: topology and parameters of the Convolutional Neural Networks (CNN). In (Trivedi et al., 2018), the authors propose a GA that evolves the weights of the softmax layer to improve the performance of the NN. Suganuma et al. propose in (Suganuma et al., 2020) a \((1+\lambda)\) evolutionary strategy to evolve DNN. In 2020, (Real et al., 2020, July) presents an advanced technique that automatically searches for the best model, operating from scratch and obtaining a good performance with the problems at hand. The use of NAS has been applied in other areas like the Reinforcement Learning (RL). In that area, there is a great example of NAS (Zoph and Le, 2016). In that work, authors use a recurrent network (RNN) to generate the model descriptions of NN and train this RNN with RL to maximize the expected accuracy of the generated architectures on a validation set.

There are more examples of NAS in the literature like the NAS algorithm which comprises of two surrogate through a supernet, with the objective of improving the gradient descent training efficiency (Lu et al., 2020, August). Another NAS comes in (Lu et al., 2022a), in which the authors propose a pipeline with also a surrogate NAS applied to real-time semantic segmentation. They manage to convert the original NAS task into an ordinary multi-objective optimization problem.

Lastly, there are more advanced techniques of NAS and EA given by (Real et al., 2019, January), in which a new model for evolving a classifier is presented, and by (Real et al., 2020, July), in which the authors propose AutoML-Zero, an evolutionary search to build a model from scratch (with low-level primitives for feature combination and neuron training) which is able to get a great performance over the addressed problem.

The main characteristic of the previous NAS proposals is the evolution of the DL model guided by a single objective, usually the accuracy or another that measures the performance of the network. The following proposals share a common aspect: the evolution of the model is done using more than one objective. This leads to the algorithms in the field of MONAS.

We can find several approaches of MONAS that have been applied to diverse fields with great results. One of them is presented in (Elsken et al., 2019, May). This work propose a MONAS that lets the approximation of the Pareto-front of all the architectures. In relation to medical images area, in (Baldeon-Calisto and Lai-Yuen, 2020) a MONAS that evolves both accuracy and model size is proposed. Moreover, following this research in medical images, in (Baldeon Calisto and Lai-Yuen, 2020), the authors use a MO evolutionary based algorithm that minimizes both the expected segmentation error and number of parameters in the network. Another interesting work is presented in (Calisto and Lai-Yuen, 2020), in which they have created a pipeline for the automatic design of neural architectures while optimizing the network's accuracy and size.

Typically, MONAS evaluate two or three objectives. A common objective is usually the performance of the network. The others objectives are related with the complexity of the network and other empirical and measurable objectives. In (Lu et al., 2021), the authors propose a multi-objective evolutionary algorithm for the design of DNN for image classification, adopting the classification performance and the number of floating-point operations as its objectives. Another example is DeepMaker, (Loni et al., 2020), which is a multi-objective evolutionary approach that considers both the accuracy of the network and its size to evolve robust DNN architectures for embedded devices.

There are some well-known MOEAs in the literature. One of them is NSGA-II. A new version of it has been developed to use it for NAS (Lu et al., 2019, July), called NSGA-Net. This proposal looks for the best architecture through a three-step search based an initialization step, followed by an exploration step that performs the EA operators to create new architectures, and an exploitation step that uses the previous knowledge of all the evaluated architectures.

### Transfer Learning and CNN Pruning

One of the objectives that EAs used for NAS usually aim to optimize is the complexity of the network. NN are structures with a great amount of parameters. These networks are composed of two main parts. The first one extracts the main features of the problem, i.e., learns to distinguish the patterns of the images (when working with image classification) and the second part is responsible to classify these patterns into several classes.
In this context, TL appears as a figure that helps in the learning process when there are few data, i.e., prevents the overfitting when the input examples is not large (Pan and Yang, 2010). TL is a DL mechanism encompassing a broad family of techniques. The most common method of TL with DL is the usage of a previous network structure with pre-trained parameters in a similar problem to the related task, being trained with huge datasets like (Krizhevsky et al., 2017). This fact involves the usage of a DL model with fixed and pre-trained weights in the convolutional layers with a dataset and then add and train several layers to adapt the network to a different classification problem (Khan et al., 2019).

Another technique to reduce the complexity of the networks is pruning. Pruning a CNN model consists of reducing the parameters of the model, but it may lead into a decrease of the performance of the model. An example of the fusion of EAs, DNN and pruning is shown in (Wang et al., 2020), which proposes a novel approach based on a combination of pruning CNN of sparse layers (layer with fewer connections between neurons) guided by a genetic algorithm. The main consequences of this study is the reduction of a great fraction of the network, but at the penalty of a lower generalization performance of the network.

Following the idea of EAs and Dnn, in (Poyatos et al., 2023) the authors propose also propose a combination of sparse layers and a genetic algorithm. They have shown that pruning can be done in a TL scheme with sparse layers and EAs. Their proposal is only guided by the performance of the model, but they also achieve a great reduction in the optimized sparse layers.

### Out of Distribution Detection

Robustness is a term that has been used with related yet different meanings among the literature of the Machine Learning (ML) community. In this work we refer to the model's ability to handle the unknown, to detect whether it has been queried with an example of a not learned distribution, therefore refusing to make the classification it has been trained to do. This is precisely what the Out of Distribution detection framework measures.

In this problem, a model learns to classify instances in the different classes from a training dataset that is sampled from a distribution, namely the In-Distribution. After the process, the model is asked to correctly distinguish between test examples that are drawn equally from either the in-distribution or from a semantically different distribution, the Out-Distribution dataset (Yang et al., 2021). The term semantically different refers to the fact that the classes contained in this foreign distribution are distinct from the ones present in the InD. As ML and DNN model are not natively prepared for this task, an OoD detection technique is wrapped around the model to allow this behavior. Typically, these techniques are based on creating a score for every example processed by the model, such that the score obtained by an OoD instance is significantly different from the one obtained by a InD example. Then, by simply defining a threshold on this score, the model can decide whether an instance is from the in or out distribution.

A great variety of methods exist in the literature, which was started by Hendrycks and Gimpel (2016), where the so-called _baseline_ method was introduced. It relies on the simple observation that ID instances tend to have greater Maximum Softmax Probability, the softmax probability of the predicted class. By simply applying a threshold to this score, they achieved acceptable performance on many classification problems. In Liang et al. (2018, Aprila), this idea was refined by applying temperature scaling to the softmax probabilities, what further separates apart from each other the distributions of the scores of the in- and out- distribution samples probabilities. Authors also implemented an input preprocessing pipeline that enhanced a bit the performance by adding a small quantity of gradient and softmax dependent noise. The paper presented in Lee et al. (2018), instead of using the softmax probabilities, exploits the feature space of the layer right before softmax and assumes that it follows a multivariate Gaussian distribution, enabling the calculation of its mean and variance for every sample. After creating a class-conditional distribution utilizing the training samples, the score for every test sample is the closest Mahalanobis distance between the sample and the calculated Gaussian class-conditional distributions.

The technique proposed in (Hendrycks et al., 2019, May), in contrast to previous works, focuses on modifying model's training by adding a term to the loss function (that depends on the classification of the task, density estimation, etc.), helping the model learn heuristics that will improve the performance of other OoD methods applied afterwards. This new term needs to be trained with OoD data, which can
be obtained by leveraging the large amount of data publicly available on the internet. Authors prove that the learned heuristics for arbitrary OoD datasets generalize well to other unseen OoD data. Thereafter, (Liu et al., 2020) based its detector in what they called the free energy function, that combines concepts of the energy based models with modern neural networks and their capability of assigning a scalar to every instance fed to the model without changing its parametrization. Specifically, the free energy function is based on the logits of the network, and the work empirically demonstrates that OoD instances tend to have higher energy, enabling the distinction between InD and OoD data. In the following work in (Lin et al., 2021, June) exploited the idea that easy OoD samples can be detected by leveraging low-level statistics. On this basis, several intermediate classifiers are trained at different depths and each example is outputted through one of them depending on its complexity. To measure complexity, a function based on the number of bits used to encode the compressed image is harnessed. The OoD scoring function employed is the above presented energy function adapted to the corresponding depth.

Although only a few research contributions are presented in this work, it must be noted that the OoD problem has been widely studied in the literature (Salehi et al., 2021), with proposals ranging from the more complex and well performing ones to the more simple yet effective ones. As the aim of this paper is to show that the robustness in the OoD can be affected when the pruning of the network is done. Therefore, the OoD detection method will be selected to be computationally cheap yet effective, to not add computational complexity to the MOEA.

In this section, a review of the related work from three different perspectives has been presented. Terms like MONAS, MOEA are important as this work presents a new work about these topics. Moreover, it is based on a TL scheme in which an evolutionary pruning of the last layers is done. In the last years, several proposals have been published over these topics, i.e, MOEAs that search for the best architecture attending to one or more objectives and also pruning approaches for CNNs. However, this study introduces a new manner to guide the evolutionary pruning of the models with the usage of a OoD mechanism. MOEvoPruneDeepTL tries to solve the problem of achieving robust models with high performance and least active neurons. This scheme, a MOEA that performs pruning in the last layers (TL paradigm) with three objectives is a new contribution to all this fields at the same time.

## 3 Multi-Objective Evolutionary Pruning for Deep Transfer Learning

This section is devised to explain the details of MO-EvoPruneDeepTL. First, in Subsection 3.1 the formulation to the problem at hand is presented. In Subsection 3.2, we will describe the objectives used to guide the search. Then, in Subsection 3.3, the description of the OoD detector is explained. The DL and network schemes are shown in Subsection 3.4. Finally, in Subsection 3.5, the evolutionary parts of MO-EvoPruneDeepTL are described.

### Problem formulation

This section aims at defining and explaining the mathematical components that circumscribe MO-EvoPruneDeepTL. We explore different concepts needed to fully understand the basics of our study.

We define the concept of dataset. Mathematically, we define a training dataset \(\mathcal{D}\doteq\{(\mathbf{x}_{i},y_{i})\}_{i=1}^{N}\) composed by \(N\) (instance, label) pairs. Such a dataset is split in training, validation and test partitions, such that \(\mathcal{D}=\mathcal{D}_{tr}\cup\mathcal{D}_{val}\cup\mathcal{D}_{test}\) with \(|\mathcal{D}_{tr}|=N_{tr}\).

Another important concept to keep in mind is the model. We define a model \(M_{\theta}\) to represent the relationship between its input \(\mathcal{X}\) and its corresponding output \(y\in\{1,\ldots,Y\}\), where \(Y\) denotes the number of classes present in \(\mathcal{D}\). Learning the parameter values \(\theta^{*}\) that best model this relationship can be accomplished by using a learning algorithm \(\theta^{*}=\text{ALG}(M;\mathcal{D}_{tr})\) that aims to minimize a measure of the difference (_loss_) between the model's output and its ground truth over the training instances in \(\mathcal{D}_{tr}\) (e.g. gradient backpropagation in neural networks). In what follows \(M_{\theta}\) is assumed to be a NN, so that \(\theta\) represent the totality of trainable weights in its neural connections.

In the context of TL for classification tasks, the NN \(M_{\theta}\) is assumed to be composed by a pre-trained feature extractor \(F_{\phi}(\mathbf{x})\) (whose parameters \(\phi\) are kept fixed while ALG(\(\cdot\)) operates), and a dense (i.e.
fully-connected) part \(G_{\theta}(\cdot)\) that maps the output of the feature extractor to the class/label to be predicted. Therefore, after tuning the trainable parameters of the network as \(\theta^{*}=\text{ALG}(G;\mathcal{D}_{tr})\), the class \(\widehat{y}\in\{1,\dots,Y\}\) predicted for an input instance \(\mathbf{x}\) is given by:

\[\widehat{y}=(F\circ G_{\theta^{*}})(\mathbf{x})=G_{\theta^{*}}(F(\mathbf{x})), \tag{1}\]

where \(\circ\) denotes composition of functions. When predictions are issued over the validation partition \(\mathcal{D}_{val}\), a measure of accuracy can be done by simply accounting for the ration of correct predictions to the overall size of the set, i.e. \(\text{ACC}_{val}=(1/N_{val})\cdot\sum_{i\in\mathcal{D}_{val}}\mathbb{I}( \widehat{y}_{i}=y_{i})\), where \(\mathbb{I}(\cdot)\) equals 1 if its argument is true (0 otherwise).

Bearing this notation in mind, pruning can be defined as a binary vector \(\mathbf{p}=\{p_{j}\}_{j=1}^{P}\), where \(P\) denotes the length of the feature vectors extracted by \(F_{\phi}(\mathbf{x})\) for every input instance to the network. As such, \(p_{j}=0\) indicates that neural links that connect the \(j\)-th component of the feature vector to the rest of neurons in the dense part \(G_{\theta}(\cdot)\) of the network are disconnected, causing that all trainable parameters from this disconnected input to the output of the overall model are pruned. Conversely, if \(p_{j}=1\) the \(j\)-th input neuron is connected to the densely connected layers of the neural hierarchy. By extending the previous notation, the training algorithm is redefined to \(\theta^{*}(\mathbf{p})=\text{ALG}(G;\mathcal{D}_{tr},\mathbf{p})\) to account from the fact that the network has been pruned as per \(\mathbf{p}\). This dependence of the trained parameters on the pruned vector propagate to the measure of accuracy over the validation instances, yielding \(\text{ACC}_{val}(\mathbf{p})\). Likewise, a measure of the reduction of the number of trainable parameters can be also computed for a given pruning vector \(\mathbf{p}\) relative to the case when no pruning is performed (i.e., \(\mathbf{p}=\mathbf{1}\doteq\{1\}_{j=1}^{P}\)) as \(\text{MEM}(\mathbf{p})=|\theta(\mathbf{p})|/|\theta(\mathbf{1})|\).

Intuitively, a good pruning strategy should consider the balance between the reduced number of trainable parameters and its impact on the accuracy when addressing the modeling task at hand. Reducing the amount of parameters to be stored has practical benefits in terms of memory space, and can yield a lower inference latency when the trained model is queried.

A third dimension of the network that can be affected by pruning is its capacity to detect Out of Distribution (OoD) instances. A significant fraction of the techniques proposed so far for identifying query samples that deviate from the distribution of training data rely on the network dynamics between neurons while the instance flows through the network. This is the case of ODIN (Liang et al., 2018, Aprila), BASELINE (Hendrycks and Gimpel, 2016) and ENERGY (Liu et al., 2020), among others. To quantify the capability of a pruned network \(M_{\theta^{*}(\mathbf{p})}\) to detect OoD instances, we utilize other datasets \(\boldsymbol{\mathcal{D}}^{\prime}=\{\mathcal{D}_{d}^{\prime}\}_{d=1}^{D_{OoD}}\) different from \(\mathcal{D}\), whose instances \((\mathbf{x}^{\prime},y^{\prime})\) are assumed to be representative of the OoD test instances with which the model can be queried in practice. An OoD detection technique \(T_{OoD}(\mathbf{x};M_{\theta^{*}(\mathbf{p})})\equiv T_{OoD}(\mathbf{x})\) processes the activations triggered by \(\mathbf{x}\) throughout the trained pruned model \(M_{\theta^{*}(\mathbf{p})}\) so as to decide whether \(\mathbf{x}\) is an In-Distribution (\(T_{OoD}(\mathbf{x})=0\)) or an OoD instance (corr. \(T_{OoD}(\mathbf{x})=1\)). This being said, true positive and false negative rates can be computed for \(T(\mathbf{x})\) over the test subset \(\mathcal{D}_{test}\) of \(\mathcal{D}\) and random \(N_{val}/D_{OoD}\)-sized samples drawn from every other dataset \(\mathcal{D}_{d}^{\prime}\), which can be aggregated in a compound performance metric. Among other choices for this purpose, we consider the AUROC measure AUROC(\(\mathbf{p}\)), which measures the ability of \(T(\cdot)\) to discriminate between positive and negative examples. This measure is set dependent on \(\mathbf{p}\) in accordance with previous notation, as \(T(\mathbf{x})\) operates on the neural activations stimulated by \(\mathbf{x}\).

### Objectives of MO-EvoPruneDeepTL

This section introduces the objectives that guide MO-EvoPruneDeepTL during its evolutionary process. We define them using the notation previously commented in Subsection 3.1.

The optimization problem addressed in this work aims to discover the set of Pareto-optimal pruning vectors \(\{\mathbf{p}_{k}^{opt}\}_{k=1}^{K}\) that best balance between three objectives:

1. The modeling performance of the pruned model over dataset \(\mathcal{D}\). This performance is measured with the accuracy over the test dataset (\(\mathcal{D}_{test}\)). It is the percentage of well classified images out of the total set of images.
2. The number of active neurons left after the pruning operation. The number of active neurons corresponds with the remaining active connections after the pruning and evolutionary process.
3. The capability of an OoD detection technique to discriminate between Out-of-Distribution and In-Distribution data by inspecting the activations inside the pruned model.

Mathematically:

\[\{\mathbf{p}_{k}^{opt}\}_{k=1}^{K} =\arg_{\mathbf{p}\in\{0,1\}^{P}}\left[\max\text{ACC}_{val}(\mathbf{ p}),\min\text{MEM}(\mathbf{p}),\max\text{AUROC}(\mathbf{p})\right],\] (2) s.t. \[\mathcal{D}:\text{In-distribution dataset}, \tag{3}\] \[\mathcal{D}^{\prime}_{1},\ldots,\mathcal{D}^{\prime}_{D_{OoD}}: \text{Out-of-distribution datasets},\] (4) \[F_{\phi}(\mathbf{x}):\text{Pre-trained feature extractor},\] (5) \[T(\mathbf{x}):\text{Out-of-distribution detection technique}. \tag{6}\]

### Out of Distribution detector of MO-EvoPruneDeepTL

In the following subsection the technique selected to assess the OoD performance of the pruned models is presented, along with a clarification about the metrics used to measure it.

Due to the fact that every new child of the population in the evolutionary algorithm must have its OoD performance correctly assessed, the chose method should not entail a big computational burden while maintaining a sufficient effectiveness in detecting OoD samples. The technique presented in (Liang et al., 2018, Aprilb), ODIN, fulfills these requirements and is the selected one.

Before explaining ODIN, the already mentioned performance metric used in this study must be clarified, namely the AUROC or _Area Under the Receiver Operation Characteristic curve_. It is a threshold-independent metric for binary classification that can be considered as the probability that the model ranks a random positive example with higher score than a random negative example. Is defined as TPR/FPR, which stand for True Positive Rate and False Positive Rate respectively and can be computed as \(\text{TPR}=\text{TP}/(\text{TP}+\text{FN})\) and \(\text{FPR}=\text{FP}/(\text{FP}+\text{TN})\). Therefore, in order to compute the AUROC, the FPR value for every TPR needs to be calculated. In this work, TP is used to refer to an in-distribution sample correctly classified as such, whereas a TN represents an OoD sample detected correctly by the OoD detector.

The basic principle of ODIN is to use maximum softmax probability with temperature scaling as the OoD score for every sample, defined by the expression

\[f_{ODIN}(\mathbf{x};T)=\max_{i}(S_{i}(\mathbf{x};T))=S_{\hat{y}}(\mathbf{x};T), \tag{7}\]

where \(S_{i}(x;T)\) is the softmax probability of the \(i^{t}h\) class for the input instance \(\mathbf{x}\), scaled by a temperature parameter \(T\in\mathbb{R}^{+}\). This scaled softmax can be calculated as:

\[S_{i}(x;T)=\frac{\exp(h_{i}(\mathbf{x})/T)}{\sum_{j=1}^{N}\exp(h_{i}(\mathbf{x })/T)}. \tag{8}\]

where \(h_{i}(\mathbf{x})\mid i\in\{1,\ldots,Y\}\) are the logits, the values prior to the softmax activation function. Then, and in accordance with notation presented in Subsection 3.1, the OoD detection technique \(T_{OoD}\), ODIN in this case \(T_{ODIN}\), will output a 1 if the instance's score is below a defined threshold, indicating that is considered an out-of-distribution sample, outputting a 0 otherwise:

\[\mathbf{x}\text{ belongs to}\begin{cases}\text{in-distribution}&\text{if }T_{ODIN}( \mathbf{x};T;\lambda)=0\iff f_{ODIN}(\mathbf{x};T)\geq\lambda,\\ \text{out-distribution}&\text{if }T_{ODIN}(\mathbf{x};T;\lambda)=1\iff f_{ODIN}( \mathbf{x};T)<\lambda.\end{cases} \tag{9}\]

It is important to remark that ODIN also uses an input preprocessing pipeline to further improve its performance in the OoD detection problem, but that in this study it will be discarded for the sake of reducing the computational burden of the algorithm.

So, in order to implement ODIN, the below presented steps must be followed. First, the model \(M_{\theta}\) must be trained using the training set \(\mathcal{D}_{tr}\) of the in-distribution dataset \(\mathcal{D}\). Then the logits of the instances of the test set \(\mathcal{D}_{test}\) must be extracted for the sake of calculating the temperature scaled softmax outputs using
the equation (8). The OoD score of each input instance \(f_{ODIN}(\mathbf{x};T)\) will be the maximum of these scaled softmax outputs, i.e., the value corresponding to the predicted class, as expression (7) indicates.

Next, same operation must be repeated with the the out-of-distribution detection set, composed by samples drawn from every other dataset \(\mathcal{D}^{\prime}_{d}\) as indicated in Subsection 3.1. In this manner, we have created two distributions of OoD scores: one for the in-distribution samples of \(\mathcal{D}_{test}\) and other for the out-of-distribution ones. Now, the threshold on the score for each TPR is defined by using the score distribution of test instances and equation (9). The corresponding FPR for each TPR is computed by employing the OoD distribution and the defined threshold, obtaining a set of [TPR, FPR] values that compose the ROC curve. Finally, from this curve the AUROC can be computed, therefore obtaining the desired robustness score for the model \(M_{\theta}\) and in-distribution dataset \(\mathcal{D}\).

In this study, in order to evaluate the robustness of each model, a practical approach is used, which involves the usage of the OoD detector with the other datasets that are not covered in training phase. However, the design of the algorithm accommodates any other dataset as an OoD evaluation dataset.

### Network characteristics of MO-EvoPruneDeepTL

In this subsection, we introduce the characteristics of the used network in MO-EvoPruneDeepTL. In our study, we use the TL paradigm, i.e., the weights of the convolutional phase are imported and fixed from other trained network in a similar task. For that reason, the DL model we use works as a feature extractor. The images pass through the network and it extracts their main features. These features correspond with the neurons which are evolved by the evolutionary components of MO-EvoPruneDeepTL. In Subsection 4.2, we give more details about the feature extractor and its characteristics.

These features are the input for the last layers of our model. The composition of the layer for our models is a single fully-connected layer which is composed of a single layer with 512 neurons, followed by the output layer. The input of the model correspond with extracted features from the network.

The fully-connected layer is converted into a sparse layer, which means that the layer has less active connections. In our study, the chromosome is decoded to an adjacency matrix which defines the connections of the neurons in the layer. MO-EvoPruneDeepTL creates the architecture of the model using this matrix and then, makes the pruning in relation to the connections which represent the input of the neuron.

### Evolutionary components of MO-EvoPruneDeepTL

In this section, we introduce the evolutionary components of MO-EvoPruneDeepTL. It is a MOEA, called Non-Dominated Sorting Genetic Algorithm II (NSGA-II) (Deb et al., 2002). The population of networks is evolved using the common operators from this GA, but, in this case, only two individuals are used for the evolution as parents. As a result, two offspring individuals are produced. Our MO-EvoPruneDeepTL uses a binary encoding strategy, which represents if a neuron is active or not. A neuron is active if its gen is 1 or not active if it is 0. Thanks to this direct encoding approach, each gen determines uniquely a neuron in the decoded network.

The initialization of the chromosomes correspond with a random discrete initialization in \([0,1]\), the selection is done using a binary tournament selection method, whereas the replacement strategy is the dual strategy of ranges of Pareto dominance and crowding distance of NSGA-II. Finally, the crossover and mutation operator are outlined:

**Crossover**: the crossover operator used in MO-EvoPruneDeepTL is the uniform crossover. This operator defines two new individuals from two parents. Mathematically, given these two parents \(\mathbf{p}\) and \(\mathbf{q}\), where \(\mathbf{p}=\{p_{i}\}_{i=1}^{P}\) and \(\mathbf{q}=\{q_{i}\}_{i=1}^{P}\) and their length is \(P\), the resultant offsprings \(\mathbf{p}\)' \(=\{p^{\prime}_{i}\}_{i=1}^{P}\) and \(\mathbf{q}\)' \(=\{q^{\prime}_{i}\}_{i=1}^{P}\) (also with length \(P\)) are generated using these equations:

\[\begin{split} p^{\prime}_{i}&=\left\{\begin{array}{ ll}p_{i}&\text{if }r\leq 0.5\\ q_{i}&\text{otherwise}\end{array}\right.\\ q^{\prime}_{i}&=\left\{\begin{array}{ll}q_{i}&\text{if }r\leq 0.5\\ p_{i}&\text{otherwise}\end{array}\right.\end{split} \tag{10}\]
where \(r\) is the realization of a continuous random variable with support over the range \([0.0,1.0]\). This operator creates two individuals using information of the genes of both parents. Each position \(i\) of the new individual takes the value of the gene of \(\mathbf{p}\) or \(\mathbf{q}\) until the offspring is fully created.

**Mutation**: the mutation performed by MO-EvoPruneDeepTL is the Bit Flip mutation. This operator needs a mutation probability defined by \(mut_{p}\). Thus, for each chromosome, all of its genes can be mutated if the mutation is really performed, which means changing the value of the gene from active to not active or vice versa. The parameter that controls if a gene is flipped or not is \(mut_{p}\).

Next, we give a brief explanation about the process that MO-EvoPruneDeepTL performs. First, we need to know the data required by MO-EvoPruneDeepTL, which is the dataset for training the network and its test dataset, the InD data, and also the OoD data, so that each model can also be tested on it. Lastly, the configuration of the GA and of the network are also required.

Once all the data is gathered, then the evolutionary process takes place. Algorithm 1 shows the pseudocode of MO-EvoPruneDeepTL. The beginning of the process is the standard procedure of initialization and evaluation of the initial population (lines 1 and 2). Then, the evolution is performed. In each generation, the operators are being executed sequentially. The parents are selected using the selection operator (line 4). After that, they generate their offspring using the crossover operator (line 5) which are mutated using the mutation operator (line 6). Then, both children are evaluated to obtain the values of the objectives that guide the evolutionary process. Thus, for each child, its chromosome is decoded into a sparse network (line 8) which is trained using the train set of the InD data (line 9). Then, the information contained in the logits is passed through the OoD detector which determines the robustness of the child using the AUROC metric (line 10). The accuracy is calculated using the test set of the InD (line 11) and the complexity of the network is also achieved using the number of neurons which are active in the child chromosome (line 12). Then, the objectives are retained as part of the information of the child for further generations (line 13).

```
Input : InD dataset, OoD dataset, configuration of the GA and configuration of the network Output : Evolved pruned network
1 Initialization of individuals of the population using the initialization operator;
2 Evaluation of the initial population (see lines 9-14);
3while evaluations \(<\) max_evalsdo
4 Parent selection using binary tournament;
5 Generate offsprings using uniform crossover;
6 Mutation of individuals using the bit flip mutation;
7for each child \(p\) in children populationdo
8 SparseNetwork\({}_{\mathbf{p}}\)\(\leftarrow\) Decodification of child chromosome;
9 SparseTrainedNetwork\({}_{\mathbf{p}}\)\(\leftarrow\) Train SparseNetwork\({}_{\mathbf{p}}\) using the train set of InD data;
10 AurocChild\({}_{\mathbf{p}}\)\(\leftarrow\) Robustness metric of child using OoD data;
11 AccChild\({}_{\mathbf{p}}\)\(\leftarrow\) Accuracy of SparseTrainedNetwork\({}_{\mathbf{p}}\) evaluated in test set of InD data;
12 ComplexChild\({}_{\mathbf{p}}\)\(\leftarrow\) Number of active neurons in SparseNetwork\({}_{\mathbf{p}}\);
13 SolutionVector(AccChild,ComplexChild,AurocChild)\({}_{\mathbf{p}}\);
14 evaluations+=1;
15
16 end for
17 Replacement Strategy;
18
19 end for
```

**Algorithm 1**MO-EvoPruneDeepTL

## 4 Experimental Framework

This section is intended to describe the framework surrounding the experiments conducted in this study. In Subsection 4.1, a detailed description of the datasets is given. Then, Subsection 4.2 shows the values of the parameters and the network setup of MO-EvoPruneDeepTL in the experiments.
### Dataset information

In this study we have selected several datasets which fit in our working environment. These datasets represent a good choice for TL approaches due to their size, as the training and inference times are lower. Thus, these datasets are suitable for problems related with population metaheuristics, since a large number of individuals will be evaluated. We present a brief description of each dataset:

* CATARACT ([dataset]Sungjoon Choi, 2020) is a dataset related with the medical environment. It classifies different types of eye diseases.
* LEAVES ([dataset]Hafiz Tayyab Rauf et al., 2019) is a dataset that is composed of images of different types of leaves, since healthy to unhealthy with different shades of green.
* PAINTING is related to the painting environment ([dataset]Virtual Russian Museum, 2018). This dataset is composed of images which represent different types of paintings.
* PLANTS is dataset which presents a great variety of leaves and plants, which ranges from tomato, or corn plants to other leaves, among others ([dataset] Singh et al., 2020, May).
* RPS ([dataset]Laurence Moroney, 2019) is a dataset whose purpose is to distinguish the gesture of the hands in the popular Rock Paper Scissors game from artificially-created images with different positions and skin colors.
* SRSMAS is based on the marine world whose aim is to classify different coral reef types ([dataset] Gomez-Rios et al., 2019).

Next, we show some examples for several of the above datasets are shown in Fig. 1.

Finally, we highlight the main characteristics in quantitative terms of instances, classes and metrics with non-pruned networks for each dataset. Table 1 show these numbers.

Figure 1: Images of datasets. Left: LEAVES examples. Middle: RPS examples. Right: SRSMAS examples.
### Training and network setup

In this subsection, we describe both the training and network setup of MO-EvoPruneDeepTL. First, we explain how our datasets are split. Then, the network setup is presented. Lastly, we discuss the parameters of MO-EvoPruneDeepTL.

In this study, we use six different datasets in our experiments. We need to split the images of these datasets into a train and test subsets, as the evaluation of MO-EvoPruneDeepTL requires it. We have created a 5-fold cross-validation evaluation environment, meanwhile for the rest of the datasets, their train and test subsets had already been predefined.

Another component of MO-EvoPruneDeepTL is the used network along all the experiments. In our case, we have chosen ResNet-50 as the pre-trained network. This election has been taken to maintain the balance between the number of features, which leads to a higher computational space, and the performance obtained in the TL process. The combinatorial problem can be huge for typical values of feature extractors commonly used in problems where TL is in use. Using this network yields feature vectors \(F_{\phi}(\mathbf{x})\) comprising \(P=2,048\) components, leading to a total of \(2^{2,048}\approx 3.23\cdot 10^{616}\) possible pruning patterns. Furthermore, the evaluation of pruned networks during the search requires repeatedly training over the instances in the test subset can be computationally expensive. Note that, although in our experiments a CNN model is used, the pruning can also be performed with other type or architectures, like Long Short-Term Memory (LSTM) (Wang et al., 2019).

These extracted features are passed through the last layers, which are the layers that are going to be trained. The model with the larger accuracy on the training set is saved. The optimizer of the training environment is the standard SGD. The parameters of MO-EvoPruneDeepTL are shown in Table 2. The maximum number of training epochs is 600, but the training phase stops if no improvement is achieved in ten consecutive rounds. The last important parameter appears in the OoD phase. It is called _Tem\({}_{ODIN}\)_ and it controls how the softmax values are computed using the logits from the Ind and OoD. The value of this parameter has been tested with several values that the authors chosen as best values for it.

The last contribution of this section is the discussion of the parameters of MO-EvoPruneDeepTL. The maximum evaluations of MO-EvoPruneDeepTL is set to 200 and the size of the population of networks for each generation is 30. Table 3 shows the evaluation time for each individual, so that the total time

\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Dataset** & **Image Size** & \(L\) & **\# Instances** & **Accuracy** & **AUROC** \\  & & (**\# classes**) & (**train / test**) & (**No Pruning**) & (**No Pruning**) \\ \hline CATARACT & \((256,256)\) & 4 & 480 / 121 & 0.732 & 0.870 \\ LEAVES & \((256,256)\) & 4 & 476 / 120 & 0.935 & 0.960 \\ PAINTING & \((256,256)\) & 5 & 7721 / 856 & 0.951 & 0.990 \\ PLANTS & \((100,100)\) & 27 & 2340 / 236 & 0.480 & 0.820 \\ RPS & \((300,300)\) & 3 & 2520 / 372 & 0.954 & 0.934 \\ SRSMAS & \((299,299)\) & 14 & 333 / 76 & 0.885 & 0.999 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Datasets used in the experiments.

\begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Maximum Evals & 200 \\ \# Runs & 10 \\ Population size & 30 \\ \(p_{mut}\) & \(\frac{1}{P}\) \\ Batch Size & 32 \\ Tem\({}_{ODIN}\) & 1000 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Parameters of MO-EvoPruneDeepTL.
of execution is the time of the first column multiplied by the number of evaluations. Each OoD detection requires a minute, but in the datasets with the 5-fold cross-validation, this time reaches the five minutes. Moreover, we also indicate the inference time for test and the required time to calculate the AUROC metric in the OoD phase. Those times force us to keep a low number of runs and evaluations to meet a computationally affordable balance between the performance of our models and the high execution times required for our simulations. Moreover, although statistical tests are important to assess the significance of the differences in the results, but due to these limited number of runs, we can not apply them, as large number of runs is required to achieve statistically reliable insights.

The experiments have been carried out using Python 3.6 and a Keras/Tensorflow implementation deployed and running on a Tesla V100-SXM2 GPU.

## 5 Results and Discussion

This section is devised to analyze the behavior of MO-EvoPruneDeepTL. To this end, we define three research questions (RQ) which are going to be answered in the following subsections with diverse experiments over the previous datasets. We will show and analyze several plots to illustrate the benefits of MO-EvoPruneDeepTL. The RQ can be stated as follows:

* How are the approximated Pareto fronts produced by the proposal in each of the considered datasets? The Pareto front can be defined as the set of non-dominated solutions, being chosen as optimal, if no objective can be improved without sacrificing at least one other objective. The problem at hand is approximated using a multi-objective approach. For that reason, we want to check that not only we have promising solutions in the extreme values of the Pareto front, but also to have a wide population of diverse solutions in the whole Pareto. As a consequence of that, to answer this RQ, we will analyze how is the Pareto front for each dataset and if there exists any direct connection between the objectives of the study: accuracy, complexity of the network, and robustness.
* Is there any remarkable pruning pattern that appears in all the solutions of the Pareto front? We compare the pruning patterns of all the models of the Pareto fronts of MO-EvoPruneDeepTL to show if there are some important patterns which are key to identify the most important zones of the input images. We employ a well-known technique called Grad-CAM (Selvaraju et al., 2017, October), which uses the gradient of the classification score with respect to the convolutional features of the network to check which parts of the image are most important for the classification task. Grad-CAM lies in the group of Explainable Artificial Intelligence (XAI) techniques, as it produces details to make easy to understand which neurons are the relevant ones in all the experiments (Barredo Arrieta et al., 2020). These neurons lead to specific pixels or group of them of the original images that are passed through the network.

\begin{table}
\begin{tabular}{c r r r r} \hline \hline Dataset & Total & Evaluation & Training and Inference & OoD Detection \\ \hline CATARACT & 332 min & 1.66 min & 0.66 min & 1 min \\ LEAVES & 1600 min & 8 min & 3 min & 5 min \\ PAINTING & 1700 min & 8.5 min & 7.5min & 1 min \\ PLANTS & 800 min & 4 min & 3 min & 1 min \\ RPS & 900 min & 4.5 min & 3.5 min & 1 min \\ SRSMAS & 1500 min & 7.5 min & 2.5 min & 5 min \\ \hline \hline \end{tabular}
\end{table}
Table 3: Average time in evaluations of MO-EvoPruneDeepTL.
* Do our models achieve an overall improvement in performance when merged through ensemble modelling?
* MO-EvoPruneDeepTL trains a great variety of models which leads to a wide diversity of models in the Pareto front for each dataset. The aim of this RQ is to check whether the diversity of pruning patterns in the Pareto front can be used to improve our DL models through ensemble strategies. Our aim is to check if an ensemble of differently pruned models can yield more accurate predictions, leading to a better overall performance than their compounding models in isolation. Beyond improving the accuracy through ensembles, we will also explore whether ensemble modeling allows obtaining more robust models, so that the number of OoD samples that the network wrongly predicts as InD is lower.

This section is divided in Section 5.1, where we analyze the different Pareto front for each considered dataset in order to answer RQ1. Next, in Section 5.2, we will examine the different pruning patterns of our models. Precisely, we will look for the neurons that appears in most of them, and we will highlight the essential zones of the input images, as this is the key part to answer RQ2. Lastly, we will discuss in Section 5.3 the benefits of the diversity of our models when ensemble modeling is performed, to show if an improvement in terms of accuracy and AUROC is achieved, which the principles lines of the RQ3.

### Answering RQ1: Analyzing the Pareto fronts of MO-EvoPruneDeepTL

The objective of this section is to answer RQ1 by performing a complete analysis of the Pareto fronts of MO-EvoPruneDeepTL. This analysis is to be performed focusing on two important aspects: i) how are the Pareto fronts for each dataset? and ii) how are the projections in each objective for each dataset? Note that for each dataset, the Pareto front shown is the so-called super Pareto front, which consists of taking the solutions of each Pareto for each run and calculating the Pareto front on these solutions. Moreover, it has been included in these Pareto fronts the results for the non-pruned network for each dataset, which are composed of solutions with all the active neurons and the accuracy and AUROC showed in Table 1.

With these graphics we analyze the quality of each Pareto and, particularly, by assessing the full spectrum of solutions that can be achieved in each of the Pareto. Moreover, we are going to study whether there is a direct relationship between any of the objectives we have formulated in the previous sections. In order to develop these plots, we have collected all the solutions of the super Pareto front (called Pareto front from now), selecting 10% of the best solutions for each objective in order to make their projections.

These Pareto front are presented in Figures 2 and 3. We can observe the diversity of pruning patterns produced by MO-EvoPruneDeepTL. Moreover, another insight from these Pareto front comes up when we inspect extreme values of each objective, as they systematically achieve good results in each dataset. Most

Figure 2: Pareto fronts of MO-EvoPruneDeepTL. Left: CATARACT dataset. Middle: RPS dataset. Right: PAINTING dataset.
of the solutions obtain high values of accuracy and robustness meanwhile their remaining active neurons are kept low.

First, we focus on the central part of the 3D projections, in which we visualize the three objectives. Our goal is to detect if there exist some kind of relationship between them. We clearly see that the projection in all the Pareto front takes values to the upper corner in which the three objectives present low values of percentage of active neurons, but high accuracy and AUROC. Moreover, this distribution of the points in both group of figures indicate that there is a tendency of the solutions to that plane in which the number of active neurons is low.

Analyzing the two-dimensional planes, there is not a clear relation between the performance and robustness. Nonetheless, the common point of this projection, namely, the complexity of the network, sheds light to the fact that it can be related with the performance and robustness separately. In both cases, a minimum number of active neurons is needed in order to start achieving good results in each objective. For both objectives, there is a certain range of optimal number of active neurons in which each of them obtains their best values.

The Pareto fronts shown in this section have allowed us to obtain valuable information on the different executions of MO-EvoPruneDeepTL. The configuration of MO-EvoPruneDeepTL has allowed us to obtain a fairly diverse set of solutions, with competitive solutions at the extremes values of the different objectives of the study. A second conclusion drawn from this analysis is the existence of relationship or direct Pareto both the complexity of the network and its performance and the complexity and robustness, but it does not appear to exist between the performance and the robustness.

### Answering RQ2: Remarkable pruning patterns in the Pareto fronts of MO-EvoPruneDeepTL

This RQ aims to analyze if there are certain pruning patterns, along the different trained networks, that allows detecting important regions in the input images to the pruned networks.

In order to answer this RQ, we must discriminate relevant neurons that appear in most of the pruning patterns in the Pareto fronts produced by MO-EvoPruneDeepTL. In doing so, we resort to a XAI technique called GradCAM (Selvaraju et al., 2017, October), which permits to localize the regions within the image that are responsible for a class prediction. Thanks to GradCAM, we can go backwards from the neurons of the solutions and highlight these key pixel regions. For each dataset, we depict several query images, and remark the 10 most relevant neurons as per GradCAM and their distribution among the three objectives. In the following figures, the central sections are relevance heatmaps obtained by GradCAM, remarking the most influential zones of the input images as warmer colors. In addition to the heatmap, we also present two more plots. The first one, in the left top, show as a bar diagram the index of the 10 most relevant neurons which appear as active in most of the solutions of the Pareto front, with their relative frequency. The second one, at the right, shows the distribution of the objectives values for these representative neurons. In this chart, a boxplot is shown for each objective and neuron: from left to right, accuracy, percentage

Figure 3: Pareto fronts of MO-EvoPruneDeepTL. Left: LEAVES dataset. Middle: PLANTS dataset. Right: SRSMAS dataset.
of active neurons and AUROC, respectively. The border of the heatmaps and the color of the bars in the figures are related, so that the reader can match each heatmap to the corresponding neuron and frequency of appearance in the Pareto.

Figure 4 shows the previous information for the CATARACT dataset. In the first one, the barplot, we can see that the least important neuron achieves a 60% of frequency in the Pareto front, i.e., it appears in the 60% of solutions meanwhile the best one has a frequency rate of more than the 80%. The second figure, the boxplot, shows the distribution of the objectives for solutions which have these relevant neurons. These results show that low complexity is presented in these neurons and high accuracy and AUROC. Lastly, we see the heatmaps for this dataset. For the shown images, we can see how these pruning patterns that MO-EvoPruneDeepTL achieves during its evolutionary process. These patterns let us recognize how the network dictate the class for each image thanks to these ten most important neurons.

The next figure, Figure 5 shows the results for the RPS dataset. The bar graph shows that these neurons achieve an appearance in more than the 80% of solutions of the Pareto and the boxplot confirm that the solutions in which these neurons are presented achieve, in most cases, less than 10% of active neurons, accuracies near 90% and AUROC around an 80%. The examples images shown in the heatmaps present the effect of these important neurons. As we have previously noted, the keys to recognize the images are position of the fingers and even the separation among them, as warmer color are presented for them.

The next dataset is PAINTING. Figure 6 shows the set of graphics for this dataset. The relevant neurons for PAINTING achieve a minimum percentage of appearance of 70% among all the solutions in the Pareto front. There is a significant difference between the first relevant neuron and the rest in terms of appearance. These solutions present almost a 20% of active neurons, but also high performance both in accuracy and AUROC, between 90 and 100%. This indicates the great level of uniformity in the robustness for this dataset. These neurons help us to analyze the images of this dataset. The third image presents a woman and, taking a deep look into the heatmaps, we see that the network recognizes the face, and then the outer parts, like the arms and the hair. Another interesting image is the fifth one. Our network is able to recognize the chest and also the arms and the rest of the body extremities.

We continue our analysis of the obtained pruning patterns of MO-EvoPruneDeepTL with the PLANTS dataset, shown in Figure 7. The most important neurons have an appearance rate between 60 and 80% in all the solutions of the Pareto front. Their distribution of objective report us a very low complexity of the network, near the 10% in average, with a good results for this dataset both in accuracy and AUROC. This dataset contains images of leaves and plants of fruit and vegetables and, for that reason, our network focus in the recognition of the shape of these leaves, as it is shown in the three bottom images of the figure.

We continue with this analysis with the LEAVES dataset. In this case, Figure 8 shows the three graphics for this dataset. The first one, which is related with the relevance of the neurons, exhibit two neurons which appear in all the solutions of the Pareto front and another two which present almost a 100% of appearance. For those neurons, the boxplot chart report us similar distributions because, in all the cases, the remaining active neurons are kept low and the accuracy and AUROC are high. Lastly, the images from this dataset show both diseased and healthy leaves. The achieved pruning patterns of MO-EvoPruneDeepTL are able to distinguish the healthy from the diseased leaves (last image versus the third one starting from the top), and then the type of the disease.

The last dataset is SRSMAS, whose charts are presented in Figure 9. The most relevant neurons obtain a minimum of 60% of appearance in all the solutions of the Pareto front, which has been a constant factor in all the datasets. Moreover, the distribution of the objectives for the solutions, in which these neurons are active, shares a common lines: high values both performance and robustness and low complexity of the network. These neurons draw pruning patterns that identify the class for the input images. As an example, in the fourth image it is only necessary to recognize the silhouette of the coral reef, but in the fifth one, the network needs to understand how is the central part of the coral reef and then its extremities.

In these figures, we have seen several datasets in which the difference rate between the most important neurons is close (RPS and PLANTS), but there are other datasets in which these difference is up to 20% between the most and least relevant neurons. Moreover, the distribution of the objectives for each dataset gives us good insights about the uniformity of the performance and robustness in most of the datasets.

The good work done by MO-EvoPruneDeepTL in training the models has made possible to achieve
Figure 4: Bars, boxplots and heatmaps of CATARACT.
Figure 5: Bars, boxplots and heatmaps of RPS.
Figure 6: Bars, boxplots and heatmaps of PAINTING.
Figure 7: Bars, boxplots and heatmaps of PLANTS.
Figure 8: Bars, boxplots and heatmaps of LEAVES.
Figure 9: Bars, boxplots and heatmaps of SRSMAS.
remarkable pruning patterns. These have helped us to decipher not only those neurons that have been key in the whole training and inference process, but also to locate in the input images those groups of influential pixels which have been important to decide the class of each of these images.

### Answering RQ3: Quality of the models through ensemble modeling

This last section is devised to formally answer to RQ3, which is to show if the diversity of the models trained by MO-EvoPruneDeepTL lets us achieve new models granted with the benefits of the ensemble. These benefits are obtaining models with better predictions, and better performance, than any single contributing model, and reducing the dispersion of the predictions and model performance.

In our study, we have selected three of the six datasets to be part of the tests performed to answer this RQ. These datasets are CATARACT, PAINTING and RPS. The objective of the next experiments is to show how the ensemble modeling performs in terms of making better predictions and how it helps to identify less number of images of the OoD samples as InD, which will lead to an increase of the AUROC. In this section, we analyze the behavior of the ensemble by quartiles, in order to give more information about the models with different ranges of accuracy and AUROC.

Before showing the graphs that result from making the ensemble, we explain how to proceed with their reading so that the meaning that emerges from each one of them can be clearly understood. We have selected the quantiles for minimum and maximum individual accuracy/AUROC, starting from 50 until 95. Then, we have divided the graph into 8 zones, each of them follows the minimum and maximum quantile of \((Q_{min},Q_{max})\), where \(Q_{min}=50\%,55\%,60\%,\ldots,85\%\) and \(Q_{max}=60\%,65\%,70\%,\ldots,95\%\). With this division, we have the following intervals \((50\%,60\%),(55\%,65\%),\ldots,(85\%,95\%)\). These intervals show us the value of the accuracy or AUROC of the models which participate in the ensemble. There are three symbols for each of these intervals. The first one is a rectangle which shows the distribution of the values of accuracy/AUROC, then the second symbol is a square, which symbolizes the best result for the measure at hand and, lastly, a star, which represent the accuracy/AUROC of the ensemble.

With this explanation, we can interpret the two graphs that result from making the ensemble. The first one is related to the accuracy of the network. Figure 10 shows this graph. It presents three graphs sorted alphabetically by dataset. The first one corresponds to CATARACT, the second to RPS and the third to PAINTING. Each of them shows, for each interval of quantile the distribution of individual accuracies, the maximum of the distribution and the accuracy of the ensemble.

The overall performance in the three cases is positive since the diversity of the models allows us to find new models that improve the accuracy for each quantile interval, except in the case of RPS where we only have one model in the interval \((60\%,70\%)\). In the RPS case, we have models near the 90% of accuracy and the ensemble produces a new model with almost 96% of it, which is a great result. Moreover, models with higher accuracy (96% or more) achieve close to 100% of accuracy. For RPS (the chart of the right), most of the ensemble models get a 95% of accuracy, meanwhile their individual models are present a lower value in accuracy. As a result, these charts show the benefits of the ensemble modeling for the accuracy objective.

The second part of this section consists of replicating the previous experiment, but for the case of OoD detection in order to check if the AUROC improves when ensemble modeling occurs. In such a case, we will be able to confirm that the new model detects less OoD sample as InD, which makes an improvement in the associated metric.

The interpretation of the set of charts is the same as in the previous case. We have made the ensemble with the models for each interval. The same characteristics are presented in this graphic. It is shown the distribution of individual AUROC values, its maximum and then, marked with a star, the AUROC of the ensemble. The CATARACT case shows an improvement in the AUROC in all the cases but one. For the RPS case (middle chart), the case of \((85\%,95\%)\) achieves almost a 95% of AUROC, meanwhile the individual values get a maximum of 87%. The PAINTING dataset also presents outstanding results. Its minimum AUROC for all the intervals of the ensemble is more than 98.5% and the least value is of individual models is less than 97%. The results obtained from the graph are similar to those obtained for the case of accuracy, since they improve on the individual results in the vast majority of the intervals.

In this section, we have conducted two experiments which involve the ensemble modeling of the trained models by MO-EvoPruneDeepTL. The ensemble has been done taking into account the performance of the
Figure 11: Ensemble modeling of the models trained by MO-EvoPruneDeepTL in terms of OoD detection. Left: CATARACT dataset. Middle: RPS dataset. Right: PAINTING dataset.

Figure 10: Ensemble modeling of the models trained by MO-EvoPruneDeepTL in terms of accuracy. Left: CATARACT dataset. Middle: RPS dataset. Right: PAINTING dataset.
network and the robustness and we have given the liberty to choose the interval of values for each measure. The results drawn from these graphics show that both of the objectives have been improved. Performing a MO search not only provides the user with a wide range of models that balance between the three stated objectives, but it also achieves more diversity among the models in order to ensemble them and achieve even higher performance and robustness.

## 6 Conclusions

This paper has introduced MO-EvoPruneDeepTL, a MONAS model that evolves sparse layers of a DL model which has been instantiated using the TL paradigm. MO-EvoPruneDeepTL uses a MOEA, which evolves these sparse layers, in order to obtain adapted, pruned layers to the problem at hand and making decisions about the neurons that need to be active or inactive.

MO-EvoPruneDeepTL is a model that evolves the extracted features from the pre-trained network in order to train the last layers to tackle the considered problem. Our results draw two conclusions from the Pareto fronts: there exists a great diversity in the solutions and they also establish promising values for the objectives in their extremes values. Moreover, the projections for each objective shed light on the existence of a direct relationships between the complexity of the network and each of the other two objectives (performance and robustness), whereas there is no direct relationship between the latter two. This work falls within the umbrella of OWL because the evolved models are asked about new data, which is the OoD datasets. Moreover, OWL is related with GPAI and, particularly, in this manuscript, the experiments have shown the capability of AI generating AI as the MOEA has learnt from the trained DL models.

The trained models of MO-EvoPruneDeepTL lead to several pruning patterns in which there exist neurons that appear in most of the best solutions of the Pareto front. These patterns help us to recognize the key group of regions of the input images that our models consider the most important ones when assigning the class to the input image at inference time.

The diversity of the models of MO-EvoPruneDeepTL has shown that ensemble modeling is able to increase the overall performance, both in performance of the network and robustness, in most of the quantiles for minimum and maximum considered objective values.

The evolved trained models have shown a great performance with a minimum number of active neurons, but it is also shown the great contribution of the robustness for these models, as each DL model is tested with data that it has not previously seen. Moreover, the objectives of the MOEA has been the performance, complexity and robustness, but other alternatives can be formulated as objectives such as the latency or energy used of the GPU in the inference of the pruned model or the epistemic uncertainty level.

## Acknowledgments

F. Herrera, D. Molina and J. Poyatos are supported by the Andalusian Excellence project P18-FR-4961, the infrastructure project with reference EQC2018-005084-P and the R&D and Innovation project with reference PID2020-119478GB-I00 granted by the Spain's Ministry of Science and Innovation and European Regional Development Fund (ERDF). Aitor Martinez and Javier Del Ser would like to thank the Basque Government for the funding support received through the EMAITEK and ELKARTEK programs, as well as the Consolidated Research Group MATHMODE (IT1456-22) granted by the Department of Education of this institution.
