MoreauGrad: Sparse and Robust Interpretation of Neural Networks via
  Moreau Envelope
Explaining the predictions of deep neural nets has been a topic of great
interest in the computer vision literature. While several gradient-based
interpretation schemes have been proposed to reveal the influential variables
in a neural net's prediction, standard gradient-based interpretation frameworks
have been commonly observed to lack robustness to input perturbations and
flexibility for incorporating prior knowledge of sparsity and group-sparsity
structures. In this work, we propose MoreauGrad as an interpretation scheme
based on the classifier neural net's Moreau envelope. We demonstrate that
MoreauGrad results in a smooth and robust interpretation of a multi-layer
neural network and can be efficiently computed through first-order optimization
methods. Furthermore, we show that MoreauGrad can be naturally combined with
$L_1$-norm regularization techniques to output a sparse or group-sparse
explanation which are prior conditions applicable to a wide range of deep
learning applications. We empirically evaluate the proposed MoreauGrad scheme
on standard computer vision datasets, showing the qualitative and quantitative
success of the MoreauGrad approach in comparison to standard gradient-based
interpretation methods.
Jingwei Zhang, Farzan Farnia
# MoreauGrad: Sparse and Robust Interpretation of Neural Networks via Moreau Envelope

###### Abstract

Explaining the predictions of deep neural nets has been a topic of great interest in the computer vision literature. While several gradient-based interpretation schemes have been proposed to reveal the influential variables in a neural net's prediction, standard gradient-based interpretation frameworks have been commonly observed to lack robustness to input perturbations and flexibility for incorporating prior knowledge of sparsity and group-sparsity structures. In this work, we propose MoreauGrad1 as an interpretation scheme based on the classifier neural net's Moreau envelope. We demonstrate that MoreauGrad results in a smooth and robust interpretation of a multi-layer neural network and can be efficiently computed through first-order optimization methods. Furthermore, we show that MoreauGrad can be naturally combined with \(L_{1}\)-norm regularization techniques to output a sparse or group-sparse explanation which are prior conditions applicable to a wide range of deep learning applications. We empirically evaluate the proposed MoreauGrad scheme on standard computer vision datasets, showing the qualitative and quantitative success of the MoreauGrad approach in comparison to standard gradient-based interpretation methods.

Footnote 1: The paper’s code is available at [https://github.com/buyeah1109/MoreauGrad](https://github.com/buyeah1109/MoreauGrad)

## 1 Introduction

Deep neural networks (DNNs) have achieved state-of-the-art performance in many computer vision problems including image classification [1], object detection [2], and medical image analysis [3]. While they manage to attain super-human scores on standard image and speech recognition tasks, a reliable application of deep learning models to real-world problems requires an interpretation of their predictions to help domain experts understand and investigate the basis of their predictions. Over the past few years, developing and analyzing interpretation schemes that reveal the influential features in a neural network's prediction have attracted great interest in the computer vision community.

A standard approach for interpreting neural nets' predictions is to analyze the gradient of their prediction score function at or around an input data point. Such gradient-based interpretation mechanisms result in a feature saliency map revealing the influential variables that locally affect the neural net's assigned prediction score. Three well-known examples of gradient-based interpretation schemes are the simple gradient [4], integrated gradients [5], and DeepLIFT [6] methods. While the mentioned methods have found many applications in explaining neural nets' predictions, they have been observed to lack robustness to input perturbations and to output a dense noisy saliency map in their application to computer vision datasets [7, 8]. Consequently, these gradient-based explanations can be considerably altered by minor random or adversarial input noise.

A widely-used approach to improve the robustness and sharpness of gradient-based interpretations is SmoothGrad [9] which applies Gaussian smoothing to the mentioned gradient-based interpretation methods. As shown by [9], SmoothGrad can significantly boost the visual quality of a neural net's gradient-based
saliency map. On the other hand, SmoothGrad typically leads to a dense interpretation vector and remains inflexible to incorporate prior knowledge of sparsity and group-sparsity structures. Since a sparse saliency map is an applicable assumption to several image classification problems where a relatively small group of input variables can completely determine the image label, a counterpart of SmoothGrad which can simultaneously achieve sparse and robust interpretation will be useful in computer vision applications.

In this paper, we propose a novel approach, which we call _MoreauGrad_, to achieve a provably smooth gradient-based interpretation with potential sparsity or group-sparsity properties. The proposed Moreau-Grad outputs the gradient of a classifier's Moreau envelope which is a useful optimization tool for enforcing smoothness in a target function. We leverage convex analysis to show that MoreauGrad behaves smoothly around an input sample and therefore provides an alternative optimization-based approach to SmoothGrad for achieving a smoothly-changing saliency map. As a result, we demonstrate that similar to SmoothGrad, MoreauGrad offers robustness to input perturbations, since a norm-bounded perturbation will only lead to a bounded change to the MoreauGrad interpretation.

Next, we show that MoreauGrad can be flexibly combined with \(L_{1}\)-norm-based regularization penalties to output sparse and group-sparse interpretations. Our proposed combinations, Sparse MoreauGrad and Group-Sparse MoreauGrad, take advantage of elastic-net [10] and group-norm [11] penalty terms to enforce sparse and group-sparse saliency maps, respectively. We show that these extensions of MoreauGrad preserve the smoothness and robustness properties of the original MoreauGrad scheme. Therefore, our discussion demonstrates the adaptable nature of MoreauGrad for incorporating prior knowledge of sparsity structures in the output interpretation.

Figure 1: Interpretation of Sparse MoreauGrad (ours) vs. standard gradient-based baselines on an ImageNet sample before and after adding a norm-bounded interpretation adversarial perturbation.
Finally, we present the empirical results of our numerical experiments applying MoreauGrad to standard image recognition datasets and neural net architectures. We compare the numerical performance of MoreauGrad with standard gradient-based interpretation baselines. Our numerical results indicate the satisfactory performance of vanilla and \(L_{1}\)-norm-based MoreauGrad in terms of visual quality and robustness. Figure 1 shows the robustness and sparsity of the Sparse MoreauGrad interpretation applied to an ImageNet sample in comparison to standard gradient-based saliency maps. As this and our other empirical findings suggest, MoreauGrad can outperform standard baselines in terms of the sparsity and robustness properties of the output interpretation. In the following, we summarize the main contributions of this paper:

* Proposing MoreauGrad as an interpretation scheme based on a classifier function's Moreau envelope
* Analyzing the smoothness and robustness properties of MoreauGrad by leveraging convex analysis
* Introducing \(L_{1}\)-regularized Sparse MoreauGrad to obtain an interpretation satisfying prior sparsity conditions
* Providing numerical results supporting MoreauGrad over standard image recognition datasets

## 2 Related Work

**Gradient-based Interpretation.** A large body of related works develop gradient-based interpretation methods. Simonyan et al. [4] propose to calculate the gradient of a classifier's output with respect to an input image. The simple gradient approach in [4] has been improved by several related works. Notably, the method of Integrated Gradients [5] is capable of keeping highly relevant pixels in the saliency map by aggregating gradients of image samples. SmoothGrad [9] removes noise in saliency maps by adding Gaussian-random noise to the input image. The CAM method [12] analyzes the information from global average pooling layer for localization, and Grad-CAM++ [13] improves over Grad-CAM [14] and generates coarse heat-maps with improved multi-object localization. The NormGrad [15] focuses on the weight-based gradient to analyze the contribution of each image region. DeepLIFT [6] uses difference from reference to propagate an attribution signal. However, the mentioned gradient-based methods do not obtain a sparse interpretation, and their proper combination with \(L_{1}\)-regularization to promote sparsity remains highly non-trivial and challenging. On the other hand, our proposed MoreauGrad can be smoothly equipped with \(L_{1}\)-regularization to output sparse interpretations and can further capture group-sparsity structures.

**Mask-based Interpretation.** Mask-based interpretation methods rely on adversarial perturbations to interpret neural nets. Applying a mask which perturbs the neural net input, the importance of input pixels is measured by a masked-based method. This approach to explaining neural nets has been successfully applied in References [16, 17, 18, 19] and has been shown to benefit from dynamic perturbations [20]. More specifically, Dabkowski and Gal [19] introduce a real-time mask-based detection method; Fong and Vedaldi [17] develop a model-agnostic approach with interpretable perturbations; Wagner et al. [16] propose a method that could generate fine-grained visual interpretations. Moreover, Lim et al. [18] leverage local smoothness to enhance their robustness towards samples attacked by PGD [21]. However, [17] and [19] show that perturbation-based interpretation methods are still vulnerable to adversarial perturbations.

We note that the discussed methods depend on optimizing perturbation masks for interpretations, and due to the non-convex nature of neural net loss functions, their interpretation remains sensitive to input perturbations. In contrast, our proposed MoreauGrad can provably smooth the neural net score function, and can adapt to non-convex functions using norm regularization. Hence, MoreauGrad can improve both the sparsity and robustness of the interpretation.

**Robust Interpretation.** The robustness of interpretation methods has been a subject of great interest in the literature. Ghorbani et al. [7] introduce a gradient-based adversarial attack method to alter the neural nets' interpretation. Dombrowski et al. [22] demonstrate that interpretations could be manipulated, and they suggest improving the robustness via smoothing the neural net classifier. Heo et al. [8] propose a manipulation method that is capable of generalizing across datasets. Subramanya et al. [23] create adversarial patches fooling both the classifier and the interpretation.
To improve the robustness, Sparsified-SmoothGrad [24] combines a sparsification technique with Gaussian smoothing to achieve certifiable robustness. The related works [16, 17, 18, 19, 25] discuss the application of adversarial defense methods against classification-based attacks to interpret the prediction of neural net classifiers. We note that these papers' main focus is not on defense schemes against interpretation-based attacks. Specifically, [16] filter gradients internally during backpropogation, and [18] leverage local smoothness to integrate more samples. Unlike the mentioned papers, our work proposes a model-agnostic optimization-based method which is capable of generating simultaneously sparse and robust interpretations.

## 3 Preliminaries

In this section, we review three standard interpretation methods as well as the notation and definitions in the paper.

### Notation and Definitions

In the paper, we use notation \(\mathbf{X}\in\mathbb{R}^{d}\) to denote the feature vector and \(Y\in\{1,\ldots,k\}\) to denote the label of a sample. In addition, \(f_{\mathbf{w}}:\mathbb{R}^{d}\rightarrow\mathbb{R}^{k}\) denotes a neural net classifier with its weights contained in vector \(\mathbf{w}\in\mathcal{W}\) where \(\mathcal{W}\) is the feasible set of the neural net's weights. Here \(f_{\mathbf{w}}\) maps the \(d\)-dimensional input \(\mathbf{x}\) to a \(k\)-dimensional prediction vector containing the likelihood of each of the \(k\) classes in the classification problem. For every class \(c\in\{1,\ldots,k\}\), we use the notation \(f_{\mathbf{w},c}:\mathbb{R}^{d}\rightarrow\mathbb{R}\) to denote the \(c\)-th entry of \(f_{\mathbf{w}}\)'s output which corresponds to class \(c\).

We use \(\|\mathbf{x}\|_{p}\) to denote the \(\ell_{p}\)-norm of input vector \(\mathbf{x}\). Furthermore, we use notation \(\|\mathbf{x}\|_{p,q}\) to denote the \(\ell_{p,q}\)-group-norm of \(\mathbf{x}\) defined in the following equation for given variable subsets \(S_{1},\ldots,S_{t}\subseteq\{1,\ldots,d\}\):

\[\|\mathbf{x}\|_{p,q}=\big{\|}\left[\|\mathbf{x}_{S_{1}}\|_{p},\ldots,\| \mathbf{x}_{S_{t}}\|_{p}\right]\big{\|}_{q} \tag{1}\]

In other words, \(\|\mathbf{x}\|_{p,q}\) is the \(\ell_{q}\)-norm of a vector containing the \(\ell_{p}\)-norms of the subvectors of \(\mathbf{x}\) characterized by index subsets \(S_{1},\ldots,S_{t}\).

### Gradient-based Saliency Maps

In our theoretical and numerical analysis, we consider the following widely-used gradient-based interpretation baselines which apply to a classifier neural net \(f_{\mathbf{w}}\) and predicted class \(c\) for input \(\mathbf{x}\):

1. **Simple Gradient**: The simple gradient interpretation returns the saliency map of a neural net score function's gradient with respect to input \(\mathbf{x}\): \[\mathrm{SG}\big{(}f_{\mathbf{w},c},\mathbf{x}\big{)}\,:=\,\nabla_{\mathbf{x}} f_{\mathbf{w},c}(\mathbf{x}).\] (2) In the applications of the simple gradient approach, \(c\) is commonly chosen as the neural net's predicted label with the maximum prediction score.
2. **Integrated Gradients:** The integrated gradients approach approximates the integral of the neural net's gradient function between a reference point \(\mathbf{x}^{0}\) and the input \(\mathbf{x}\). Using \(m\) intermediate points on the line segment connecting \(\mathbf{x}^{0}\) and \(\mathbf{x}\), the integrated gradient output will be \[\mathrm{IG}\big{(}f_{\mathbf{w},c},\mathbf{x}\big{)}\,:=\,\frac{\Delta\mathbf{ x}}{m}\sum_{i=1}^{m}\nabla_{\mathbf{x}}f_{\mathbf{w},c}\big{(}\mathbf{x}^{0}+ \frac{i}{m}\Delta\mathbf{x}\big{)}.\] (3) In the above \(\Delta\mathbf{x}:=\mathbf{x}-\mathbf{x}^{0}\) denotes the difference between the target and reference points \(\mathbf{x},\mathbf{x}^{0}\).
3. **SmoothGrad:** SmoothGrad considers the averaged simple gradient score over an additive random perturbation \(Z\) drawn according to an isotropic Gaussian distribution \(Z\sim\mathcal{N}(\mathbf{0},\sigma^{2}I_{d})\). In practice, the SmoothGrad interpretation is estimated over a number \(t\) of independently drawn noise vectors \(\mathbf{z}_{1},\ldots,\mathbf{z}_{t}\stackrel{{\text{i.i.d.}}}{{ \sim}}\mathcal{N}(\mathbf{0},\sigma^{2}I_{d})\) according to the zero-mean Gaussian distribution: \[\text{SmoothGrad}\big{(}f_{\mathbf{w},c},\mathbf{x}\big{)}\,:=\mathbb{E}\big{[} \nabla_{\mathbf{x}}f_{\mathbf{w},c}(\mathbf{x}+Z)\big{]}\;\approx\;\frac{1}{t} \sum_{i=1}^{t}\nabla_{\mathbf{x}}f_{\mathbf{w},c}(\mathbf{x}+\mathbf{z}_{i}).\] (4)

## 4 MoreauGrad: An Optimization-based Interpretation Framework

As discussed earlier, smooth classifier functions with a Lipschitz gradient help to obtain a robust explanation of neural nets. Here, we propose an optimization-based smoothing approach based on Moreau-Yosida regularization. To introduce this optimization-based approach, we first define a function's Moreau envelope.

**Definition 1**.: _Given regularization parameter \(\rho>0\), we define the Moreau envelope of a function \(g:\mathbb{R}^{d}\to\mathbb{R}\) as:_

\[g^{\rho}(\mathbf{x})\,:=\,\min_{\widetilde{\mathbf{x}}\in\mathbb{R}^{d}}\;g \big{(}\widetilde{\mathbf{x}}\big{)}+\frac{1}{2\rho}\big{\|}\widetilde{ \mathbf{x}}-\mathbf{x}\big{\|}_{2}^{2}. \tag{5}\]

In the above definition, \(\rho>0\) represents the Moreau-Yosida regularization coefficient. Applying the Moreau envelope, we propose the MoreauGrad interpretation as the gradient of the classifier's Moreau envelope at an input \(\mathbf{x}\).

**Definition 2**.: _Given regularization parameter \(\rho>0\), we define the MoreauGrad interpretation \(\mathrm{MG}_{\rho}:\mathbb{R}^{d}\to\mathbb{R}^{d}\) of a neural net \(f_{\mathbf{w}}\) predicting class \(c\) for input \(\mathbf{x}\) as_

\[\mathrm{MG}_{\rho}(f_{\mathbf{w},c},\mathbf{x})\,:=\,\nabla f_{\mathbf{w},c}^{ \rho}(\mathbf{x}).\]

To compute and analyze the MoreauGrad explanation, we first discuss the optimization-based smoothing enforced by the Moreau envelope. Note that the Moreau envelope is known as an optimization tool to turn non-smooth convex functions (e.g. \(\ell_{1}\)-norm) into smooth functions. Here, we discuss an extension of this result to weakly-convex functions which also apply to non-convex functions.

**Definition 3**.: _A function \(g:\mathbb{R}^{d}\to\mathbb{R}\) is called \(\lambda\)-weakly convex if \(\Phi(\mathbf{x}):=g(\mathbf{x})+\frac{\lambda}{2}\|\mathbf{x}\|_{2}^{2}\) is a convex function, i.e. for every \(\mathbf{x}_{1},\mathbf{x}_{2}\in\mathbb{R}^{d}\) and \(0\leq\alpha\leq 1\) we have:_

\[g\big{(}\alpha\mathbf{x}_{1}+(1-\alpha)\mathbf{x}_{2}\big{)}\;\leq\;\alpha g( \mathbf{x}_{1})+(1-\alpha)g(\mathbf{x}_{2})+\frac{\lambda\alpha(1-\alpha)}{2} \big{\|}\mathbf{x}_{1}-\mathbf{x}_{2}\big{\|}_{2}^{2}.\]

**Theorem 1**.: _Suppose that \(g:\mathbb{R}^{d}\to\mathbb{R}\) is a \(\lambda\)-weakly convex function. Assuming that \(0<\rho<\frac{1}{\lambda}\), the followings hold for the optimization problem of the Moreau envelope \(g^{\rho}\) and the optimal solution \(\widetilde{x}_{\rho}^{*}(\mathbf{x})\) solving the optimization problem:_

1. _The gradients of_ \(g^{\rho}\) _and_ \(g\) _are related as for every_ \(\mathbf{x}\)_:_ \[\nabla g^{\rho}(\mathbf{x})=\nabla g\big{(}\widetilde{x}_{\rho}^{*}(\mathbf{x })\big{)}.\]
2. _The difference_ \(\widetilde{x}_{\rho}^{*}(\mathbf{x})-\mathbf{x}\) _is aligned with_ \(g^{\rho}\)_'s gradient:_ \[\nabla g^{\rho}(\mathbf{x})=\frac{-1}{\rho}\big{(}\,\widetilde{x}_{\rho}^{*}( \mathbf{x})-\mathbf{x}\,\big{)}.\]
3. \(g^{\rho}\) _will be_ \(\max\{\frac{1}{\rho},\frac{\lambda}{1-\rho\lambda}\}\)_-smooth, i.e. for every_ \(\mathbf{x}_{1},\mathbf{x}_{2}\)_:_ \[\big{\|}\nabla g^{\rho}(\mathbf{x}_{1})-\nabla g^{\rho}(\mathbf{x}_{2})\big{\|} _{2}\,\leq\,\frac{1}{\min\big{\{}\rho,\frac{1}{\lambda}-\rho\big{\}}}\big{\|} \mathbf{x}_{1}-\mathbf{x}_{2}\big{\|}_{2}.\]
Proof.: This theorem is known for convex functions. In the Appendix, we provide another proof for the result. 

**Corollary 1**.: _Assume that the prediction score function \(f_{\mathbf{w},c}:\mathbb{R}^{d}\to\mathbb{R}\) is \(\lambda\)-weakly convex. Then, the MoreauGrad interpretation \(\mathrm{MG}_{\rho}\) will remain robust under an \(\epsilon\)-\(\ell_{2}\)-norm bounded perturbation \(\|\mathbf{\delta}\|_{2}\leq\epsilon\) as_

\[\left\|\mathrm{MG}_{\rho}(\mathbf{x}+\mathbf{\delta})-\mathrm{MG}_{\rho}(\mathbf{ x})\right\|_{2}\leq\frac{\epsilon}{\min\bigl{\{}\rho,\frac{1}{\lambda}-\rho \bigr{\}}}.\]

The above results imply that by choosing a small enough coefficient \(\rho\) the Moreau envelope will be a differentiable smooth function. Moreover, the computation of the Moreau envelope will reduce to a convex optimization task that can be solved by standard or accelerated gradient descent with global convergence guarantees. Therefore, one can efficiently compute the MoreauGrad interpretation by solving the optimization problem via the gradient descent algorithm. Algorithm 1 applies gradient descent to compute the solution to the Moreau envelope optimization which according to Theorem 1 yields the MoreauGrad explanation.

As discussed above, MoreauGrad will be provably robust as long as the regularization coefficient will dominate the weakly-convexity degree of the prediction score. In the following proposition, we show this condition can be enforced by applying either Gaussian smoothing.

**Proposition 1**.: _Suppose that \(f_{\mathbf{w},c}\) is \(L\)-Lipschitz, that is for every \(\mathbf{x}_{1},\mathbf{x}_{2}\mid f_{\mathbf{w},c}(\mathbf{x}_{1})-f_{ \mathbf{w},c}(\mathbf{x}_{2})|\leq L\|\mathbf{x}_{2}-\mathbf{x}_{1}\|_{2}\), but could be potentially non-differentiable and non-smooth. Then, \(h_{\mathbf{w},c}(\mathbf{x}):=\mathbb{E}[f_{\mathbf{w},c}(\mathbf{x}+\mathbf{ Z})]\) where \(\mathbf{Z}\sim\mathcal{N}(\mathbf{0},\sigma^{2}I_{d\times d})\) will be \(\frac{L\sqrt{d}}{\sigma}\)-weakly convex._

Proof.: We postpone the proof to the Appendix. 

The above proposition suggests the regularized MoreauGrad which regularizes the neural net function to satisfy the weakly-convex condition through Gaussian smoothing.

## 5 Sparse and Group-Sparse MoreauGrad

To further extend the MoreauGrad approach to output sparsely-structured feature saliency maps, we further include an \(L_{1}\)-norm-based penalty term in the Moreau-Yosida regularization and define the following \(L_{1}\)-norm-based sparse and group-sparse Moreau envelope.

**Definition 4**.: _For a function \(g:\mathbb{R}^{d}\to\mathbb{R}\) and regularization coefficients \(\rho,\eta>0\), we define \(L_{1}\)-Moreau envelope \(g_{L_{1}}^{\rho,\eta}\):_

\[g_{L_{1}}^{\rho,\eta}(\mathbf{x})\,:=\min_{\widetilde{\mathbf{x}}\in\mathbb{R }^{d}}\,g(\widetilde{\mathbf{x}})+\frac{1}{2\rho}\bigl{\|}\widetilde{\mathbf{ x}}-\mathbf{x}\bigr{\|}_{2}^{2}+\eta\bigl{\|}\widetilde{\mathbf{x}}- \mathbf{x}\bigr{\|}_{1}.\]

_We also define \(L_{2,1}\)-Moreau envelope \(g_{L_{2,1}}^{\rho,\eta}\) as_

\[g_{L_{2,1}}^{\rho,\eta}(\mathbf{x})\,:=\,\min_{\widetilde{\mathbf{x}}\in \mathbb{R}^{d}}\,g(\widetilde{\mathbf{x}})+\frac{1}{2\rho}\bigl{\|}\widetilde{ \mathbf{x}}-\mathbf{x}\bigr{\|}_{2}^{2}+\eta\bigl{\|}\widetilde{\mathbf{x}}- \mathbf{x}\bigr{\|}_{2,1}.\]

_In the above, the group norm \(\|\cdot\|_{2,1}\) is defined as \(\|\mathbf{x}\|_{2,1}:=\sum_{i=1}^{t}\|\mathbf{x}_{S_{i}}\|_{2}\) for given subsets \(S_{1},\ldots,S_{t}\subseteq\{1,\ldots,d\}\)._

**Definition 5**.: _Given regularization coefficients \(\rho,\eta>0\), we define the Sparse MoreauGrad (\(\mathrm{S-MG}_{\rho,\eta}\)) and Group-Sparse MoreauGrad (\(\mathrm{GS-MG}_{\rho,\eta}\)) interpretations as_

\[\mathrm{S-MG}_{\rho,\eta}(f_{\mathbf{w},c},\mathbf{x})\,:= \frac{1}{\rho}\bigl{(}\,\widetilde{\mathbf{x}}_{L_{1}}^{*}(\mathbf{x})- \mathbf{x}\,\bigr{)},\] \[\mathrm{GS-MG}_{\rho,\eta}(f_{\mathbf{w},c},\mathbf{x})\,:= \frac{1}{\rho}\bigl{(}\,\widetilde{\mathbf{x}}_{L_{2,1}}^{*}(\mathbf{x})- \mathbf{x}\,\bigr{)},\]

_where \(\widetilde{\mathbf{x}}_{L_{1}}^{*}(\mathbf{x}),\,\widetilde{\mathbf{x}}_{L_{2,1}}^{*}(\mathbf{x})\) denote the optimal solutions to the optimization tasks of \(f_{\mathbf{w},c,L_{1}}^{\rho,\eta}(\mathbf{x}),\,f_{\mathbf{w},c,L_{2,1}}^{ \rho,\eta}(\mathbf{x})\), respectively._
In Theorem 2, we extend the shown results for the standard Moreau envelope to our proposed \(L_{1}\)-norm-based extensions of the Moreau envelope. Here, we use \(\text{ST}_{\alpha}\) and \(\text{GST}_{\alpha}\) to denote sparse and group-sparse soft-thresholding functions defined entry-wise and group-entry-wise as

\[\text{ST}_{\alpha}(\mathbf{x})_{i} :=\begin{cases}0&\text{ if }|x_{i}|\leq\alpha\\ x_{i}-\text{sign}(x_{i})\alpha&\text{ if }|x_{i}|>\alpha,\end{cases}\] \[\text{GST}_{\alpha}(\mathbf{x})_{S_{i}} :=\begin{cases}\mathbf{0}&\text{ if }\|\mathbf{x}_{S_{i}}\|_{2}\leq \alpha\\ \big{(}1-\frac{\alpha}{\|\mathbf{x}_{S_{i}}\|_{2}}\big{)}\mathbf{x}_{S_{i}}& \text{ if }\|\mathbf{x}_{S_{i}}\|_{2}>\alpha.\end{cases}\]

**Theorem 2**.: _Suppose that \(g:\mathbb{R}^{d}\to\mathbb{R}\) is a \(\lambda\)-weakly convex function. Then, assuming that \(0<\rho<\frac{1}{\lambda}\), Theorem 1's parts 1 and 3 will further hold for the sparse Moreau envelope \(g_{L_{1}}^{\rho,\eta}\) and group-sparse Moreau envelope \(g_{L_{2,1}}^{\rho,\eta}\) and their optimization problems' optimal solutions \(\widetilde{\mathbf{x}}_{\rho,\eta,L_{1}}^{*}(\mathbf{x})\) and \(\widetilde{\mathbf{x}}_{\rho,\eta,L_{2,1}}^{*}(\mathbf{x})\). To parallel Theorem 1's part 2 for \(L_{1}\)-Moreau envelope, the followings hold_

\[\text{ST}_{\rho\eta}\big{(}\!-\!\rho\nabla g_{L_{1}}^{\rho,\eta}( \mathbf{x})\big{)} = \widetilde{\mathbf{x}}_{\rho,\eta,L_{1}}^{*}(\mathbf{x})- \mathbf{x},\] \[\text{GST}_{\rho\eta}\big{(}\!-\!\rho\nabla g_{L_{2,1}}^{\rho, \eta}(\mathbf{x})\big{)} = \widetilde{\mathbf{x}}_{\rho,\eta,L_{2,1}}^{*}(\mathbf{x})- \mathbf{x}.\]

Proof.: We defer the proof to the Appendix. 

**Corollary 2**.: _Suppose that the prediction score function \(f_{\mathbf{w},c}\) is \(\lambda\)-weakly convex. Assuming that \(0<\rho<\frac{1}{\lambda}\), the Sparse MoreauGrad \(\text{S-MG}_{\rho,\eta}\) and Group-Sparse MoreauGrad \(\text{GS-MG}_{\rho,\eta}\) interpretations will be robust to every norm-bounded perturbation \(\|\boldsymbol{\delta}\|_{2}\leq\epsilon\) as:_

\[\big{\|}\text{GS-MG}_{\rho,\eta}(\mathbf{x}+\boldsymbol{\delta}) -\text{GS-MG}_{\rho,\eta}(\mathbf{x})\big{\|}_{2} \leq \frac{\epsilon}{\min\big{\{}\rho,\frac{1}{\lambda}-\rho\big{\}}}.\]

To compute the Sparse and Group-Sparse MoreauGrad, we propose applying the proximal gradient descent algorithm as described in Algorithm 1. Note that Algorithm 1 applies the soft-thresholding function as the proximal operator for the \(L_{1}\)-norm function present in Sparse MoreauGrad.

```
Input: data \(\mathbf{x}\), label \(c\), classifier \(f_{\mathbf{w}}\), regulatization coeff. \(\rho\), stepsize \(\gamma\), noise std. parameter \(\sigma\), number of updates \(T\) Initialize\(\mathbf{x}^{(0)}=\mathbf{x}\), for\(t=0,\ldots,T\)do ifRegularized Modethen Draw noise vectors \(\mathbf{z}_{1},\ldots,\mathbf{z}_{m}\sim\mathcal{N}(\mathbf{0},\sigma^{2}I_{d \times d})\) Compute\(\mathbf{g}_{t}=\frac{1}{m}\sum_{i=1}^{m}\nabla f_{\mathbf{w},c}(\mathbf{x}^{(t)}+ \mathbf{z}_{i})\) else Compute\(\mathbf{g}_{t}=\nabla f_{\mathbf{w},c}(\mathbf{x}^{(t)})\) end Update\(\mathbf{x}^{(t+1)}\leftarrow(1-\frac{\gamma}{\rho})\mathbf{x}^{(t)}-\gamma(\mathbf{g}_{t}- \frac{1}{\rho}\mathbf{x})\) ifSparse Modethen Update\(\mathbf{x}^{(t+1)}\leftarrow\text{SoftThreshold}_{\gamma\eta}\big{(}\mathbf{x}^{(t+1)}- \mathbf{x}\big{)}+\mathbf{x}\) end Output\(\text{MG}(\mathbf{x})=\frac{1}{\rho}\big{(}\mathbf{x}^{(T)}-\mathbf{x}\big{)}\)
```

**Algorithm 1** MoreauGrad Interpretation
## 6 Numerical Results

We conduct several numerical experiments to evaluate the performance of the proposed MoreauGrad. Our designed experiments focus on the smoothness, sparsity, and robustness properties of MoreauGrad interpretation maps as well as the feature maps of several standard baselines. In the following, we first describe the numerical setup in our experiments and then present the obtained numerical results on the qualitative and quantitative performance of interpretation methods.

### Experiment Setup

In our numerical evaluation, we use the following standard image datasets: CIFAR-10 [26] consisting of 60,000 labeled samples with 10 different labels (50,000 training samples and 10,000 test samples), and ImageNet-1K [27] including 1.4 million labeled samples with 1,000 labels (10,000 test samples and 1.34 million training samples). For CIFAR-10 experiments, we trained a standard ResNet-18 [28] neural network with the softplus activation. For ImageNet experiments, we used an EfficientNet-b0 network [29] pre-trained on the ImageNet training data. In our experiments, we compared the MoreauGrad schemes with the following baselines: 1) the simple gradient [4], 2) Integrated Gradients [14], 3) DeepLIFT [6], 4) SmoothGrad [9], 5) Sparsified SmoothGrad [24], 6) RelEx [18]. We note that for baseline experiments we adopted the official implementations and conducted the experiments with hyperparameters suggested in their work.

Figure 3: Visualization of Sparse MoreauGrad with various coefficient \(\eta\)’s. \(\eta=0\) is Vanilla MoreauGrad.

Figure 2: Visualization of MoreauGrad with various coefficient \(\rho\)’s. \(\rho=0\) is Simple Gradient.
### Effect of Smoothness and Sparsity Parameters

We ran the numerical experiments for unregularized Vanilla MoreauGrad with multiple smoothness coefficient \(\rho\) values to show the effect of the Moreau envelope's regularization. Figure 2 visualizes the effect of different \(\rho\) on the Vanilla MoreauGrad saliency map. As can be seen in this figure, the saliency map qualitatively improves by increasing the value of \(\rho\) from 0 to 1. Please note that for \(\rho=0\), the MoreauGrad simplifies to the simple gradient interpretation. However, as shown in Theorem 1 the proper performance of Vanilla MoreauGrad requires choosing a properly bounded \(\rho\) value, which is consistent with our observation that when \(\rho\) becomes too large, the Moreau envelope will be computationally difficult to optimize and the quality of interpretation maps could deteriorate to some extent. As numerically verified in both CIFAR-10 and ImageNet experiments, we used the rule of thumb \(\rho=\frac{1}{\sqrt{\mathbb{E}[\|\mathbf{X}\|_{2}]}}\) measured over the empirical training data to set the value of \(\rho\), which is equal to 1 for the normalized samples in our experiments.

Regarding the sparsity hyperparameter \(\eta\) in Sparse and Group-Sparse MoreauGrad experiments, we ran several experimental tests to properly tune the hyperparameter. Note that a greater coefficient \(\eta\) enforces more strict sparsity or group-sparsity in the MoreauGrad interpretation, and the degree of sparsity could be simply adjusted by changing this coefficient \(\eta\). As shown in Figure 3, in our experiments with different \(\eta\) coefficients the interpretation map becomes sparser as we increase the \(L_{1}\)-norm penalty coefficient \(\eta\). Similarly, to achieve a group-sparse interpretation, we used \(L_{2,1}\)-regularization on groups of adjacent pixels as discussed in Definition 4. The effect of the group-sparsity coefficient was similar to the sparse case in our experiments, as fewer pixel groups took non-zero values and the output interpretations showed more structured interpretation maps when choosing a larger coefficient \(\eta\). The results with different group-sparsity hyperparameters are demonstrated in Figure 4.

### Qualitative Comparison of MoreauGrad vs. Gradient-based Baselines

In Figure 5, we illustrate the Sparse, and Group-Sparse MoreauGrad interpretation outputs as well as the saliency maps generated by the gradient-based baselines. The results demonstrate that MoreauGrad generates qualitatively sharp and, in the case of Sparse and Group-Sparse MoreauGrad, sparse interpretation maps. As shown in Figure 5, promoting sparsity in the MoreauGrad interpretation maps has improved the visual quality, and managed to erase the less relevant pixels like the background ones. Additionally, in the case of Group-Sparse MoreauGrad, the maps exhibit both sparsity and connectivity of selected pixels.

Figure 4: Visualization of Group-Sparse MoreauGrad maps with various coefficient \(\eta\)’s.
Figure 5: Qualitative comparison between Sparse, Group-Sparse MoreauGrad and the baselines.
### Robustness

We qualitatively and quantitatively evaluated the robustness of MoreauGrad interpretation. To assess the empirical robustness of interpretation methods, we adopt a \(L_{2}\)-bounded interpretation attack method defined by [24]. Also, for quantifying the empirical robustness, we adopt three robustness metrics. The first metric is the Euclidean distance of the normalized interpretations before and after the attack:

\[D(I(\mathbf{x}),I(\mathbf{x}^{\prime}))=\big{\|}\frac{I(\mathbf{x})}{\|I( \mathbf{x})\|_{2}}-\frac{I(\mathbf{x}^{\prime})}{\|I(\mathbf{x}^{\prime})\|_ {2}}\big{\|}_{2} \tag{6}\]

Note that a larger distance between the normalized maps indicates a smaller similarity and a higher vulnerability of the interpretation method to adversarial attacks.

The second metric is the top-k intersection ratio. This metric is another standard robustness measure used in [7, 24]. This metric measures the ratio of pixels that remain salient after the interpretation attack. A robust interpretation is expected to preserve most of the salient pixels under an attack. The third metric is the structural similarity index measure (SSIM) [30]. A larger SSIM value indicates that the two input maps are more perceptively similar.

Using the above metrics, we compared the MoreauGrad schemes with the baseline methods. As qualitatively shown in Figure 7, using the same attack magnitude, the MoreauGrad interpretations are mostly similar

Figure 6: Quantitative robustness comparison between MoreauGrad and the baselines.
Figure 7: Visualization of robustness against interpretation attacks. The top and bottom rows show original and attacked maps.
before and after the norm-bounded attack. The qualitative robustness of MoreauGrad seems satisfactory compared to the baseline methods. Finally, Figure 6 presents a quantitative comparison of the robustness measures for the baselines and proposed MoreauGrad on CIFAR-10, Tiny-ImageNet, and ImageNet datasets. As shown by these measures, MoreauGrad outperforms the baselines in terms of the robustness metrics.

## 7 Conclusion

In this work, we introduced MoreauGrad as an optimization-based interpretation method for deep neural networks. We demonstrated that MoreauGrad can be flexibly combined with \(L_{1}\)-regularization methods to output sparse and group-sparse interpretations. We further showed that the MoreauGrad output will enjoy robustness against input perturbations. While our analysis focuses on the sparsity and robustness of the MoreauGrad explanation, studying the consistency and transferability of MoreauGrad interpretations is an interesting future direction. Moreover, the application of MoreauGrad to convex and norm-regularized neural nets could be another topic for future study. Finally, our analysis of \(\ell_{1}\)-norm-based Moreau envelope could find independent applications to other deep learning problems.
