Question:
You are a highly experienced, conscientious, and fair academic reviewer, please help me review this paper. The review should be organized into nine sections:
1. Summary: A summary of the paper in 100-150 words.
2. Strengths/Weaknesses/Questions: The Strengths/Weaknesses/Questions of paper, which should be listed in bullet points, with each point supported by specific examples from the article where possible.
3. Soundness/Contribution/Presentation: Rate the paper's Soundness/Contribution/Presentation, and match this score to the corresponding description from the list below and provide the result. The possible scores and their descriptions are:
    1 poor
    2 fair
    3 good
    4 excellent
4. Rating: Give this paper an appropriate rating, match this rating to the corresponding description from the list below and provide the result. The possible Ratings and their descriptions are:
    1 strong reject
    2 reject, significant issues present
    3 reject, not good enough
    4 possibly reject, but has redeeming facets
    5 marginally below the acceptance threshold
    6 marginally above the acceptance threshold
    7 accept, but needs minor improvements
    8 accept, good paper
    9 strong accept, excellent work
    10 strong accept, should be highlighted at the conference
5. Paper Decision: It must include the Decision itself(Accept or Reject) and the reasons for this decision, based on the criteria of originality, methodological soundness, significance of results, and clarity and logic of presentation.

Here is the template for a review format, you must follow this format to output your review result:
**Summary:**
Summary content

**Strengths:**
- Strength 1
- Strength 2
- ...

**Weaknesses:**
- Weakness 1
- Weakness 2
- ...

**Questions:**
- Question 1
- Question 2
- ...

**Soundness:**
Soundness result

**Presentation:**
Presentation result

**Contribution:**
Contribution result

**Rating:**
Rating result

**Paper Decision:**
- Decision: Accept/Reject
- Reasons: reasons content

Please ensure your feedback is objective and constructive. The paper is as follows:

# Volume Feature Rendering for Fast Neural Radiance Field Reconstruction

 Kang Han Wei Xiang1  Lu Yu

School of Computing, Engineering and Mathematical Sciences

La Trobe University

{k.han, w.xiang, l.yu}@latrobe.edu.au

Corresponding author.

Footnote 1: footnotemark:

###### Abstract

Neural radiance fields (NeRFs) are able to synthesize realistic novel views from multi-view images captured from distinct positions and perspectives. In NeRF's rendering pipeline, neural networks are used to represent a scene independently or transform queried learnable feature vector of a point to the expected color or density. With the aid of geometry guides either in the form of occupancy grids or proposal networks, the number of color neural network evaluations can be reduced from hundreds to dozens in the standard volume rendering framework. However, many evaluations of the color neural network are still a bottleneck for fast NeRF reconstruction. This paper revisits volume feature rendering (VFR) for the purpose of fast NeRF reconstruction. The VFR integrates the queried feature vectors of a ray into one feature vector, which is then transformed to the final pixel color by a color neural network. This fundamental change to the standard volume rendering framework requires only one single color neural network evaluation to render a pixel, which substantially lowers the high computational complexity of the rendering framework attributed to a large number of color neural network evaluations. Consequently, we can use a comparably larger color neural network to achieve a better rendering quality while maintaining the same training and rendering time costs. This approach achieves the state-of-the-art rendering quality on both synthetic and real-world datasets while requiring less training time compared with existing methods.

## 1 Introduction

For the task of view synthesis, unobserved views of a scene are synthesized from captured multi-view images. This is a long-standing problem that has been studied for several decades in computer vision and computer graphics. The most popular solution to this problem at present is the neural radiance field (NeRF) [18] since it can render views with high fidelity. NeRF represents the density and color of a spatial point in a given direction using a neural network (NN), typically the multilayer perceptron (MLP). With the predicted densities and colors of sampled points along a ray, the NeRF and its variants [2, 3, 28] use the volume rendering technique to aggregate colors to obtain the final rendered pixel color. As many samples are required to render one pixel, the underlying MLP needs to run many times, leading to high computational complexities for both training and rendering. Alternatively, researchers introduced extra learnable features in the form of 3D grids [25, 17], hash tables [19] and decomposed tensors [6, 12, 7, 23] in addition to MLP's parameters to represent a scene's density and color fields. As querying the feature vector of a sampled point by interpolation is much faster than one MLP evaluation, only a small MLP is used to transform the queried feature vector to the density or color, leading to substantial speed acceleration compared with pure MLP representations.

Furthermore, modeling the density field independently by computational efficient representations can greatly reduce the number of samples in the color field. The density representations could be occupancy grids [19], hash grids with less parameters [16], and small MLPs [3]. These coarse density fields enable importance sampling to make the model focus on non-empty points near the surface. As a result, only dozens of samples in the color field (typically represented by both learnable features and NNs) are needed to render a pixel. For example, Zip-NeRF [4] uses 32 samples after importance sampling through efficient density representations.

Nevertheless, standard volume rendering techniques still need to run a color NN dozens of times to render a single pixel. This is why Instant-NGP [19] uses a small color MLP for fast training and rendering, and why the latest Zip-NeRF [4] is comparably slow when using large color MLPs to achieve a better rendering quality. Although a radiance field can be implemented even without NNs [9], the importance of large color NNs for high-quality rendering is undoubted, considering that the state-of-the-art rendering quality can be achieved by pure MLP representations. The authors of Zip-NeRF [4] and NRFF [12] also highlight the importance of large color NNs in modeling view-dependent effects. Unfortunately, the standard volume rendering technique limits the size of color NNs for the sake of fast training and rendering.

One the other hand, research in generative NeRF models [5; 11; 21] and fast NeRF rendering [13] shows that the feature vectors of samples along a ray can be rendered or integrated first to enable one single evaluation of following NNs. However, it is unclear whether this volume feature rendering (VFR) technique can also be used for fast NeRF reconstruction. Towards this end, this paper revisits the VFR method for the purpose of fast NeRF reconstruction. Our investigation reveals that the VFR can be integrated into the training phase of NeRF models, thereby enabling the utilization of large color neural networks to either enhance rendering quality while maintaining the same training time, or to achieve a similar rendering quality with much faster training.

## 2 Related work

The volume rendering technique integrates the colors of samples along a ray according to their densities [15]. A global density field that is differentiable is very effective in finding the underlying geometry of a scene from multi-view 2D color observations using volume rendering. This capability is one of the essential reasons to NeRF's success. As every point in a scene has a density value, the density field is capable of modeling complex geometry. However, standard volume rendering employed by NeRF and its successors [18; 2; 3; 26; 35; 27; 6; 34] suffer from high computational complexity caused by many NN calls even with importance sampling guided by coarse density fields, as discussed in the preceding section.

Representing the underlying geometry by the signed distance function (SDF) leads to a well-defined surface to to enable one sample-based fast rendering [33; 8; 29; 20; 10; 32]. However, the SDF struggles with modeling complex geometry in the context of neural inverse rendering, e.g., trees and leaves, and this is why the best rendering quality is still achieved by volume representations. In this paper, we revisit the VFR framework that uses density as the geometry representation but shares a similar behavior as the SDF that processes a feature vector by color NNs only once. Thus, the VFR inherits the advantage of volume representation in modeling complex geometry and also has the strength of the SDF in single NN evaluation.

Generative NeRF models such as EG3D [5], StyleNeRF [11] and StyleSDF [21] focus on 3D consistency and fidelity of generated content using generalizable neural networks with NeRF's structure. In comparison, 3D reconstruction from multi-view images using NeRF is a different task from 3D generation. As per-scene optimization is required for NeRF reconstruction, the reconstruction speed is a critical issue in this field, but this problem has not been investigated in the existing generative NeRF works. This paper revisits feature integration in volume rendering for fast reconstruction and demonstrate its effectiveness in the NeRF reconstruction task.

It is noted that feature accumulation is also discussed in the Sparse Neural Radiance Grid (SNeRG) [13] mainly for the purpose of modeling specular effects. However, the diffuse color is still obtained using the standard volume rendering method in [13], which requires many times of MLP evaluations. Besides, the SNeRG needs to train a NeRF first (require days to train) and then bakes the trained NeRF to a sparse grid with specular features. In this work, we demonstrate that this pre-training is not necessary, and the accumulated feature vector can be used to predict the final view-dependentcolor instead of only the specular color. Our VFR method can be directly optimized from scratch instead of baking the optimized features in the SNeRG. As a result, our method requires significantly less training time but achieves a much better rendering quality compared with the SNeRG.

Integrating feature vectors of samples along the ray is natural in transformer models [24; 30]. For example, Suhail _et al._ proposed an epipolar transformer to aggregate features extracted from reference views on epipolar lines [24]. However, the computational cost of transformer is much higher than MLP and that transformer-based method is significantly (8 times) slower than MLP-based Mip-NeRF [2] on the same hardware. In this work, we demonstrate that the VFR's aggregation only involves a weighted combination of feature vectors based on densities such that the VFR is significantly faster than transformer-based feature aggregation.

## 3 Model architecture

### Standard volume rendering

Standard volume rendering integrates color along a cast ray according to density [15]. A high density of a point on the ray indicates a high probability of hitting the surface. The NeRF and its various variants adopt this standard volume rendering technique, where the color and density of a point are typically predicted by a neural network. For a cast ray \(\mathbf{r}(t)=\mathbf{o}+t\mathbf{d}\), where \(\mathbf{o}\) is the camera origin and \(\mathbf{d}\) is the view direction, the rendered color \(C(\mathbf{r})\) using standard volume rendering is:

\[C(\mathbf{r})=\int T(t)\sigma(\mathbf{r}(t))\mathbf{c}(\mathbf{r}(t),\mathbf{ d})\,dt,\,\text{where}\,T(t)=\exp\left(-\int_{0}^{t}\sigma(\mathbf{r}(s))\,ds\right) \tag{1}\]

where \(\sigma(\mathbf{x})\) and \(\mathbf{c}(\mathbf{x},\mathbf{d})\) are the density and color at position \(\mathbf{x}\), repsectively. In practice, the above integral is solved by sampling and integration along the cast ray. After transforming the densities to weights by \(w(t)=T(t)\sigma(\mathbf{r}(t))\), the color will be rendered as follows:

\[C(\mathbf{r})=\sum_{i=1}^{N}w_{i}\text{NeuralNet}\left(F(\mathbf{x}_{i}), \mathbf{d}\right) \tag{2}\]

where \(\mathbf{x}\) is the position of a sample point on the ray and \(F\) is a function to query the corresponding feature vector of \(\mathbf{x}\). Pure MLP-based methods including the NeRF [18], Mip-NeRF [2], Ref-NeRF [28] and Mip-NeRF 360 [3] represent \(F\) by using MLPs. However, this representation imposes a computational burden as \(N\) MLP evaluations are required to render a ray. Alternatively, recent research [19; 25; 6] shows that modeling \(F\) by extra learnable features is significantly faster than pure-MLP based representations, because linear interpolation of learnable features is much more efficient than MLP evaluations. These learnable features are typically organized in the form of the 3D grids, hash tables, and decomposed tensors. With the learnable features, a small NN can achieve the state-of-the-art rendering quality but accelerate both the training and rendering time significantly.

Furthermore, hierarchical importance sampling guided by coarse geometries is employed to greatly reduce the number of samples in (2). This importance sampling can be achieved by modeling the density fields independently [25; 6; 12], occupancy grid [19], and proposal networks [3; 4]. As shown in Fig. 1, importance sampling makes the model focus on valuable samples on the surface. For instance, by using two levels of importance sampling, Zip-NeRF [4] reduces the final number of valuable samples to 32. However, the color NN still needs to run 32 times to deliver the best rendering quality. Although integrating colors predicted by the color NN is in line with the concept in classical volume rendering, the many color NN calls are the main obstacle for fast training and rendering. On the other hand, the importance of large color NNs for realistic rendering especially in modeling the view-dependent effect is highlighted in recent works [3; 12]. Considering the fact that valuable samples on the surface are close in terms of spatial position (see red sampling points in Fig. 1), the queried feature vectors are also very similar as they are interpolated according to their position. As such, evaluating the color NN with similar input feature vectors is not necessary. In the next subsection, we will introduce the volume feature rendering method that integrates these feature vectors to enable a single evaluation of the color NN.

### Volume feature rendering

As shown in Fig. 1, the volume feature rendering method integrates queried feature vectors instead of predicted colors to form a rendered feature vector. A subsequent color NN is then applied to transform the feature vector to the final rendered color. By doing this, the color NN only needs to run once. Specifically, we take the NN out of the integral as follows:

\[C(\mathbf{r})=\text{NeuralNet}\left(\left(\sum_{i=1}^{N}w_{i}F(\mathbf{x}_{i}) \right),\mathbf{d}\right). \tag{3}\]

This fundamental change to the standard volume rendering framework relieves the rendering framework from the high computational cost caused by many NN evaluations. Consequently, we can use a larger network to achieve a better rendering quality while maintaining a similar training time.

### Pilot network

It is reasonable to integrate feature vectors of non-empty samples in the VFR. However, at the beginning of training, the model does not have a coarse geometry to focus on samples on the surface. The VFR will integrate feature vectors of all samples during the early training stage. We found that

Figure 1: Standard volume rendering needs to run NeuralNet many times for all the samples on the surface, while the volume feature rendering only needs to evaluate NeuralNet once.

Figure 2: The VFR fails to converge on the scene of _ship_ from the dataset in [18]. A small pilot network functioning as standard volume rendering by a few training steps in the early training stage resolves this problem.

this feature integration works well for most scenes but fails to converge for some scenes. Such failure is more likely to occur when employing a comparably large NN. The VFR normally converges using the NN with two hidden layers each of 256 neurons, but it does not converge when using four hidden layers. Fig. 2 illustrates a failure case on the scene of _ship_ from the NeRF synthetic dataset [18].

As increasing the size of the NN only has a slight implication to reconstruction and rendering speeds but can lead to a better rendering quality thanks to the VFR framework, we are motivated to use a large NN. We hypothesize that the reason for the aforementioned convergence failure is that the model cannot find the correct geometry due to the feature integration of all samples in the early training stage. Motivated by this insight, we design a pilot network that functions as standard volume rendering to aid the model in finding a coarse geometry in the early training stage. A small pilot network and a few training steps are sufficient to achieve this goal. In this work, we use the pilot network with two hidden layers, each with 64 neurons. The pilot network is only used in the first 300 training steps. After this training, the pilot network will be discarded, and we can use the VFR to train the model normally. The number of pilot training steps is small compared with the total training steps, e.g., 20K steps. Thus, it has a negligible effect on the overall training time. As shown in Fig. 2, the model converges normally with the aid of the pilot network.

### View-dependent effect

We use the spherical harmonics (SH) feature encoding method to encode view direction for modeling view-dependent effects. This approach can be seen as a variant of the rendering equation encoding in [12]. Different from the vanilla SH encoding [19] employed in the literature, we multiply each encoded SH coefficient by a feature vector as shown in Fig. 3. Specifically, the rendered feature vector by the VFR is fed into a spatial MLP to produce two feature vectors: SH feature vector and bottleneck feature vector. The SH feature vector will be split into small feature vectors \(\mathbf{f}\) for SH coefficients, i.e., \(\{\mathbf{f}_{0}^{0}|l=0\}\), \(\{\mathbf{f}_{1}^{-1},\mathbf{f}_{1}^{0},\mathbf{f}_{1}^{1}|l=1\}\), etc. In the SH feature encoding block, the SH feature vector \(\mathbf{f}_{m}^{l}\) will be multiplied with SH coefficient \(Y_{l}^{m}(\mathbf{d})\) with view direction \(\mathbf{d}\) to form one SH feature encoding vector \(\mathbf{e}_{m}^{l}\), written as

\[\mathbf{e}_{l}^{m}=\mathbf{f}_{l}^{m}Y_{l}^{m}(\mathbf{d}) \tag{4}\]

A comprehensive SH encoding is derived by concatenating all encoding vectors as \(\mathbf{E}\) = \(\{\mathbf{e}_{0}^{0},\mathbf{e}_{1}^{-1},\mathbf{e}_{1}^{0}...\}\), which will be further concatenated with the bottleneck feature vector (view direction independent). This concatenated vector is used to predict the final rendered color by a directional MLP. The NN in (3) thus includes the spatial and directional MLPs.

## 4 Implemention details

We implement the proposed VFR using the NerfAcc library [16], which is based on the deep learning framework PyTorch [22]. The learnable features are organized by a multiresolution hash grid (MHG) [19]. This feature representation models the feature function \(F\) in (3) that accepts the input of the position and outputs a feature vector. The color NN contains a spatial MLP and a directional MLP. The spatial MLP has two layers and the directional MLP has four layers, all with 256 hidden neurons. The size of the bottleneck feature is set to 256. We use the GELU [14] instead of the commonly used ReLU [1] activation function, as we found the GELU results in a slightly better quality.

Figure 3: Architecture of the designed neural network for modeling view-dependent effects. Each spherical harmonics basis function is multiplied by a feature vector to encode the input view direction.

The density of a sample \(\mathbf{x}_{i}\) is derived from the queried feature vector \(F(\mathbf{x}_{i})\) by a tiny density mapping layer. On the real-world 360 dataset [3], the mapping layer is a linear transform from the feature vector to the density value. On the NeRF synthetic [18] dataset, we found a tiny network with one hidden layer and 64 neurons yields a slightly better rendering quality. The density mapping layers have negligible impact on the overall training time for both datasets.

We employ a hierarchical importance sampling strategy on both the NeRF synthetic [18] and real-world 360 datasets [3]. The occupancy grid is used for efficient sampling on the NeRF synthetic dataset. For the 360 dataset, we follow the most recent Zip-NeRF [4] that uses two levels of importance sampling. These two density fields are implemented by two small MHGs, where the number of hashing levels is ten and the feature channel in each level is one. The final number of samplings on the radiance field is 32. The models are trained using the Adam optimizer with a learning rate of 0.01 and the default learning rate scheduler in NerfAcc [16]. We use PSNR, SSIM [31] and LPIPS [36] to evaluate the rendered image quality. The running times are measured on one RTX 3090 GPU. Table 1 summarizes other experimental settings in this work.

## 5 Experimental results

### NeRF synthetic dataset

The VFR achieves the best rendering quality but uses the minimum training time compared with state-of-the-art fast methods on the NeRF synthetic dataset. Since the original Instant-NGP [19] uses the alpha channel in the training images to perform background augmentation, we report the results of the Instant-NGP from NerfAcc [16], where such augmentation is not employed for fair comparison. As shown in Table 2, the VFR outperforms Instant-NGP by 2.27 dB and NerfAcc by 1.51 dB, but only uses 3.3 minutes for training. Compared with TensoRF [6], our method only uses 33% of its training time while surpassing TensoRF by 1.48 dB in PSNR. In addition, the VFR achieves the best SSIM and LPIPS on this dataset, which is consistent with the PSNR metric.

This significant improvement in rendering quality stems from the increased size of the NN. The MLP in Instant-NGP and NerfAcc has 4 layers and 64 neurons in each layer. By comparison, we use a six-layer MLP with 256 neurons. The computational cost of our MLP is roughly 24 times higher than MLPs in Instant-NGP and NerfAcc. However, thanks to the VFR framework, our MLP only needs to run one time instead of many times as in the standard volume rendering framework employed in the

\begin{table}
\begin{tabular}{l l r r r r} \hline \hline Dataset & Model & \# MHG & HGH & \#Batch & SH degree & \#SH feature \\ \hline NeRF synthetic[18] & VFR-small & 13M & 16\(\times 2^{19}\times 2\) & \(2^{18}\) points & 4 & 4 \\ Real-world 360[3] & VFR-base & 34M & 32\(\times 2^{19}\times 2\) & \(2^{13}\) rays & 7 & 8 \\ Real-world 360[3] & VFR-large & 134M & 32\(\times 2^{21}\times 2\) & \(2^{13}\) rays & 7 & 8 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Experimental settings on the NeRF synthetic and real-world 360 datasets. #Features represents the number of learnable features in the underlying hash grids, which is estimated by hash grid hyperparameters (HGH), i.e., the number of levels\(\times\) the number of voxels each level\(\times\)the number of feature channels.

\begin{table}
\begin{tabular}{l r r r r} \hline \hline  & PSNR\(\uparrow\) & SSIM\(\uparrow\) & LPIPS\(\downarrow\) & Training time\(\downarrow\) \\ \hline NeRF [18] & 31.69 & 0.953 & 0.050 & hours \\ Mip-NeRF [2] & 33.09 & 0.961 & 0.043 & hours \\ \hline TensoRF [6] & 33.14 & 0.963 & 0.047 & 10 mins \\ Instant-NGP [19] & 32.35 & 0.960 & 0.042 & 4.2 mins \\ NerfAcc [16] & 33.11 & 0.961 & 0.053 & 4.3 mins \\ \hline VFR-small: 6K & 33.02 & 0.960 & 0.055 & 0.97 mins \\ VFR-small: 20K & **34.62** & **0.971** & **0.038** & **3.3 mins** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Rendering quality and training time on the NeRF synthetic dataset. The VFR trained with 6K steps in 0.97 minutes achieves a similar quality compared with existing methods.

compared methods. As can be observed from Table 2, the single color NN evaluation property in the VFR leads to a minimum training time even when we use a much larger color NN. Noticeably, the VFR only needs 6K training steps to reach a similar rendering quality compared with the existing algorithms. Such 6K training steps can be completed within 1 minute on one RTX 3090 GPU, which is less than the 25% of training times of Instant-NGP and NerfAcc, and only 10% of the training time in TensoRF.

Fig. 4 provides a visual comparison of the rendered novel views by NerfAcc (based on Instant-NGP) and the VFR. On the compared scenes, both NerfAcc and the VFR can recover good geometries. However, for the scene of _drums_, obvious visual artefacts appear in the rendered views from NerfAcc while our results do not have such visual artefacts. Besides, the VFR produces more realistic visual effects such as shadow and reflection, as observed on the scenes of _hotdog_. The presented error maps and objective PSNRs in Fig. 4 also demonstrate the advantage of the VFR in terms of rendering quality. While NerfAcc and our method use the same feature representation, i.e., hash grids, we believe the quality advantage of our method stems from the large color NN enabled by the VFR.

### Real-world 360 dataset

We also experiment on seven real-world unbounded scenes from the 360 dataset [3]. The scenes of _flowers_ and _treehill_ in this dataset are not included as they are not publicly available. We use the unbounded ray parameterization method proposed in Mip-NeRF 360 [3], where points are mapped to a sphere if their radius to the scene center is larger than one.

The visual comparison in Fig. 5 shows that the VFR is able to render views in similar quality with Zip-NeRF but render more realistic views than NerfAcc (based on Instant-NGP). The VFR provides a more accurate geometry reconstruction than NerfAcc on the scene of _bicycle_. As can be observed from Fig. 5, the rendering results of NerfAcc contain obvious visual artefacts on the scenes of _bonsai_ and _kitchen_, while the VFR's results are more realistic and closer to the ground truth. The reflective effect in the highlighted patches on the scene of _kitchen_ is also well modeled by our model thanks to our new SH feature encoding method.

As shown in Table 3, the VFR is able to achieve similar rendering quality to the state-of-the-art Zip-NeRF [4] but using the minimum training time. Although Mip-NeRF 360 is able to synthesize novel high-fidelity views, it requires days to train on high-end GPUs because it is based on a pure MLP representation. Using learned features organized in hash grids, Instant-NGP achieves a comparable rendering quality but reduces the training time from days to hours. The most recent Zip-NeRF uses multisampling for high-quality anti-aliasing view rendering at the expense of training time relative to Instant-NGP. The VFR achieves a slightly better rendering quality than Zip-NeRF in terms of the PSNR. However, we achieve this state-of-the-art rendering quality with a significantly less training

Figure 4: Visual comparison of the synthesized novel views on the scenes of (from top to bottom) _drums_ and _hotdog_ in the NeRF synthetic dataset.

time. As a comparison, the VFR-large model only uses 22 minutes to train on one RTX 3090 GPU, while Zip-NeRF's training time is 5.2 hours on the same GPU.

It should be noted that we achieve this high-quality view rendering using an entirely different approach than Zip-NeRF. As Zip-NeRF focuses on anti-aliasing, multisampling is a reasonable solution but leads to a higher computational cost. Besides, the authors of Zip-NeRF found a larger color NN helps in improving the rendering quality. Since Zip-NeRF still employs the standard volume rendering method, a larger color NN will undoubtedly increase the training time. The VFR, however, relieves the rendering framework from the high computational cost caused by many color network evaluations. As only one color network evaluation is required in the VFR, NN evaluation is no longer the computing bottleneck in the rendering framework. As a result, the VFR enables one to achieve a similar rendering quality as Zip-NeRF but with a significantly less training time.

### Ablation study

Table 4 shows the ablation study of the proposed methods. Compared with the commonly used ReLU, we found that GELU activation improves the rendering quality for both the standard volume rendering and the VFR. Larger MLP does increase the quality using the standard volume rendering but at the expense of more training time. For the VFR, when increasing the size of the NN by using more hidden neurons, the rendering quality constantly improves but the training time only slightly

\begin{table}
\begin{tabular}{l r r r r r} \hline \hline  & PSNR\(\uparrow\) & SSIM\(\uparrow\) & LPIPS\(\downarrow\) & \#Features\(\downarrow\) & Training time\(\downarrow\) \\ \hline NeRF [18] & 24.85 & 0.659 & 0.426 & N/A & days \\ Mip-NeRF [2] & 25.12 & 0.672 & 0.414 & N/A & days \\ NeRF++ [35] & 26.21 & 0.729 & 0.348 & N/A & days \\ Mip-NeRF 360 [3] & 29.11 & 0.846 & 0.203 & N/A & days \\ \hline Instant-NGP [19] & 27.06 & 0.796 & 0.265 & 84M & 0.81 hrs \\ Zip-NeRF [4] & 29.82 & **0.874** & **0.170** & 84M & 5.20 hrs \\ NerfAcc [16] & 28.69 & 0.834 & 0.221 & 34M & 11 mins \\ \hline VFR-base: 20K & 29.48 & 0.830 & 0.233 & 34M & 5.7 mins \\ VFR-base: 40K & 29.92 & 0.850 & 0.208 & 34M & 11 mins \\ VFR-large: 20K & 29.51 & 0.846 & 0.252 & 134M & 11 mins \\ VFR-large: 40K & **29.90** & 0.862 & 0.231 & 134M & 22 mins \\ \hline \hline \end{tabular}
\end{table}
Table 3: Rendering quality and training time on the real-world 360 dataset. The scale factor for NerfAcc and the VFR-base models remains constant at four for both outdoor and indoor scenes. For the VFR-large model, the scale factor is adjusted to four for outdoor scenes (_bicycle, garden, stump_) and two for indoor scenes (_bonsai, counter, kitchen, room_) in order to maintain conformity with Zip-NeRF [4].

Figure 5: Visual comparison of synthesized novel views on the scenes of (from top to bottom) _bicycle_, _bonsai_, and _kitchen_ on the real-world 360 dataset [3].

increases. Replacing the SH encoding with the SH feature encoding (W/O SHFE vs Full model) also leads to a better quality with negligible extra training time.

## 6 Limitations

Similar to many other neural rendering methods, the VFR requires per-scene optimization to deliver high-fidelity view rendering. Although the VFR can be optimized in several minutes for one scene, we are still far from realizing real-time reconstruction for 3D video applications. As for the rendering time, our method can render 4\(\sim\)5 frames per second at the resolution of 800\(\times\)800 on the NeRF synthetic dataset, similar to the rendering speed of NerfAcc. On the real-world 360 dataset, the rendering speed is about 1\(\sim\)2 frames per second at the resolution of 1300\(\times\)840.

Elaborate implementation optimization can help in achieving high-quality real-time rendering using this VFR. However, compared with the recent work that specifically focuses on real-time rendering such as BakedSDF [33] and MobileNeRF [8], the rendering speed of the VFR needs to be further improved. BakedSDF and MobileNeRF represent a scene's surface to enable one feature querying and one neural network evaluation, at the expense of some rendering quality decline compared with volume representation. The VFR uses the volume representation with one color NN evaluation to deliver the state-of-the-art rendering quality, but the feature querying still needs to be conducted many times for a cast ray. Considering the fact that the queried feature vectors of non-empty samples are finally integrated into one feature vector using the VFR, we believe the number of feature querying can also be reduced to a small value, e.g., from 32 to 5, to achieve high-quality and real-time rendering simultaneously. An additional prospective constraint of the VFR lies in its potential inefficacy when confronted with semitransparent objects as it integrates feature vectors first and then predicts a single final color. This property is not well reflected in the NeRF synthetic and real-world 360 datasets and requires further investigation in future work.

## 7 Conclusion

We revisited the volume feature rendering method in the context of fast NeRF reconstruction. The VFR integrates the queried feature vectors of non-empty samples on a ray and then transforms the integrated feature vector to the final rendered color using one single color neural network evaluation. We found the VFR cannot guarantee the convergence on some scenes and thus introduced the pilot network to guide the early-stage training to tackle this problem. The VFR is able to achieve the state-of-the-art rendering quality while using significantly less training time. It has a great potential to replace the standard volume rendering in many neural rendering methods based on volume representation, with the benefits of either a better rendering quality by employing a larger color neural network or a significantly reduced training time thanks to one single color neural network evaluation.

\begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline  & & & & \multicolumn{4}{c}{NeRF synthetic (VFR-small)} & \multicolumn{4}{c}{Real-world 360 (VFR-base)} \\ \cline{4-11}  & Activation & MLP & VDE & Time\(\downarrow\) & PSNR\(\uparrow\) & SSIM\(\uparrow\) & LPIPS\(\downarrow\) & Time\(\downarrow\) & PSNR\(\uparrow\) & SSIM\(\uparrow\) & LPIPS\(\downarrow\) \\ \hline Baseline & ReLU & 4\(\times\)64 & SH & 4.4 & 33.11 & 0.9614 & 0.053 & 5.6 & 28.39 & 0.8147 & 0.245 \\ +GELU & GELU & 4\(\times\)64 & SH & 4.4 & 33.43 & 0.9634 & 0.049 & 5.6 & 28.43 & 0.8152 & 0.245 \\ +Medium MLP & GELU & 4\(\times\)128 & SH & 5.6 & 33.92 & 0.9663 & 0.045 & 6.8 & 28.70 & 0.8205 & 0.239 \\ \hline VFR & ReLU & 4\(\times\)64 & SH & 2.9 & 33.17 & 0.9619 & 0.053 & 5.0 & 28.20 & 0.8013 & 0.262 \\ +GELU & GELU & 4\(\times\)64 & SH & 3.0 & 33.61 & 0.9641 & 0.048 & 5.0 & 28.26 & 0.8064 & 0.254 \\ +Medium MLP & GELU & 6\(\times\)64 & SHFE & 3.0 & 34.05 & 0.9678 & 0.043 & 5.2 & 29.06 & 0.8215 & 0.240 \\ +Large MLP & GELU & 6\(\times\)128 & SHFE & 3.0 & 34.33 & 0.9697 & 0.041 & 5.4 & 29.28 & 0.8266 & 0.236 \\ W/O SHFE & GELU & 6\(\times\)256 & SH & 3.2 & 34.46 & 0.9701 & 0.040 & 5.5 & 28.91 & 0.8192 & 0.245 \\ Full model & GELU & 6\(\times\)256 & SHFE & 3.3 & **34.62** & **0.9713** & **0.038** & 5.7 & **29.48** & **0.8301** & **0.233** \\ \hline \hline \end{tabular}
\end{table}
Table 4: Ablation study. SH means spherical harmonics for view direction encoding (VDE) while SHFE stands for SH feature encoding. We use the NerfAcc’s implementation of Instant-NGP as a baseline. MLP’s size is represented by layers\(\times\)neurons. Times are measured in minutes by 20K training steps.

## References

* [1]A. Fred Agarap (2018) Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375. Cited by: SS1.
* [2]J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin-Brualla, and P. P. Srinivasan (2021) MIP-NeRF: a multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5855-5864. Cited by: SS1.
* [3]J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman (2022) MIP-NeRF 360: unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5470-5479. Cited by: SS1.
* [4]J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman (2023) ZIP-NeRF: anti-aliased grid-based neural radiance fields. arXiv preprint arXiv:2304.06706. Cited by: SS1.
* [5]E. R. Chan, C. Z. Lin, M. A. Chan, K. Nagano, B. Pan, S. De Mello, O. Gallo, L. J. Guibas, J. Tremblay, S. Khamis, et al. (2022) Efficient geometry-aware 3D generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16123-16133. Cited by: SS1.
* [6]A. Chen, Z. Xu, A. Geiger, J. Yu, and H. Su (2022) TensoRF: tensorial radiance fields. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXII, pp. 333-350. Cited by: SS1.
* [7]A. Chen, Z. Xu, X. Wei, S. Tang, H. Su, and A. Geiger (2023) Factor fields: a unified framework for neural fields and beyond. arXiv preprint arXiv:2302.01226. Cited by: SS1.
* [8]Z. Chen, T. Funkhouser, P. Hedman, and A. Tagliasacchi (2022) MobileNeRF: exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. arXiv preprint arXiv:2208.00277. Cited by: SS1.
* [9]S. Fridovich-Keil, A. Yu, M. Tancik, Q. Chen, B. Recht, and A. Kanazawa (2022) Plenoxels: radiance fields without neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5501-5510. Cited by: SS1.
* [10]Q. Fu, Q. Xu, Y. S. Ong, and W. Tao (2022) Geo-NeuS: geometry-consistent neural implicit surfaces learning for multi-view reconstruction. Advances in Neural Information Processing Systems35, pp. 3403-3416. Cited by: SS1.
* [11]J. Gu, L. Liu, P. Wang, and C. Theobalt (2021) StyleNeRF: a style-based 3D aware generator for high-resolution image synthesis. In International Conference on Learning Representations, Cited by: SS1.
* [12]K. Han and W. Xiang (2021) Multiscale tensor decomposition and rendering equation encoding for view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4232-4241. Cited by: SS1.
* [13]P. Hedman, P. P. Srinivasan, B. Mildenhall, J. T. Barron, and P. Debevec (2021) Baking neural radiance fields for real-time view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5875-5884. Cited by: SS1.
* [14]D. Hendrycks and K. Gimpel (2016) Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415. Cited by: SS1.
* [15]J. T. Kajiya and B. P. Von Herzen (1984) Ray tracing volume densities. ACM SIGGRAPH18 (3), pp. 165-174. Cited by: SS1.
* [16]R. Li, M. Tancik, and A. Kanazawa (2022) Nerfacc: a general nerf acceleration toolbox. arXiv preprint arXiv:2210.04847. Cited by: SS1.
* [17]L. Liu, J. Gu, K. Z. Lin, T. Chua, and C. Theobalt (2020) Neural sparse voxel fields. Advances in Neural Information Processing Systems33, pp. 15651-15663. Cited by: SS1.

* [18] B Mildenhall, PP Srinivasan, M Tancik, JT Barron, R Ramamoorthi, and R Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. In _European Conference on Computer Vision_, 2020.
* [19] Thomas Muller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. _ACM Transactions on Graphics (ToG)_, 41(4):1-15, 2022.
* [20] Michael Oechsle, Songyou Peng, and Andreas Geiger. Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_, pages 5589-5599, 2021.
* [21] Roy Or-El, Xuan Luo, Mengyi Shan, Eli Shechtman, Jeong Joon Park, and Ira Kemelmacher-Shlizerman. StyleSDF: High-resolution 3d-consistent image and geometry generation. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 13503-13513, 2022.
* [22] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In _Proceedings of Thirty-first Conference on Neural Information Processing Systems Workshop_, 2017.
* [23] Ruizhi Shao, Zerong Zheng, Hanzhang Tu, Boning Liu, Hongwen Zhang, and Yebin Liu. Tensor4D: Efficient neural 4D decomposition for high-fidelity dynamic reconstruction and rendering. _arXiv preprint arXiv:2211.11610_, 2022.
* [24] Mohammed Suhail, Carlos Esteves, Leonid Sigal, and Ameesh Makadia. Light field neural rendering. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 8269-8279, 2022.
* [25] Cheng Sun, Min Sun, and Hwann-Tzong Chen. Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 5459-5469, 2022.
* [26] Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. _Advances in Neural Information Processing Systems_, 33:7537-7547, 2020.
* [27] Matthew Tancik, Vincent Casser, Xinchen Yan, Sabeek Pradhan, Ben Mildenhall, Pratul P Srinivasan, Jonathan T Barron, and Henrik Kretzschmar. Block-NeRF: Scalable large scene neural view synthesis. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_, pages 8248-8258, 2022.
* [28] Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T Barron, and Pratul P Srinivasan. Ref-NeRF: Structured view-dependent appearance for neural radiance fields. In _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_, pages 5481-5490. IEEE, 2022.
* [29] Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. NeuS: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In _Advances in Neural Information Processing Systems_, 2021.
* [30] Yunxiao Wang, Yanjie Li, Peidong Liu, Tao Dai, and Shu-Tao Xia. NeXT: Towards high quality neural radiance fields via multi-skip transformer. In _Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXII_, pages 69-86. Springer, 2022.
* [31] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. _IEEE transactions on image processing_, 13(4):600-612, 2004.
* [32] Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. Volume rendering of neural implicit surfaces. _Advances in Neural Information Processing Systems_, 34:4805-4815, 2021.

* [33] Lior Yariv, Peter Hedman, Christian Reiser, Dor Verbin, Pratul P Srinivasan, Richard Szeliski, Jonathan T Barron, and Ben Mildenhall. BakedSDF: Meshing neural SDFs for real-time view synthesis. _arXiv preprint arXiv:2302.14859_, 2023.
* [34] Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. Plenoctrees for real-time rendering of neural radiance fields. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_, pages 5752-5761, 2021.
* [35] Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun. NeRF++: Analyzing and improving neural radiance fields. _arXiv preprint arXiv:2010.07492_, 2020.
* [36] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In _Proceedings of the IEEE conference on computer vision and pattern recognition_, pages 586-595, 2018.

Response:
## Review

### Summary

The paper presents a novel approach to view synthesis by introducing a method that leverages multi-resolution hash grid features, optimizing them without the need for multiple MLP evaluations on individual points. Instead, it utilizes a single MLP to predict pixel colors from these accumulated features, leading to improved rendering speed and quality. The proposed technique demonstrates superior performance over existing state-of-the-art methods in terms of training time and view synthesis quality on both synthetic and real-world datasets. However, the authors are encouraged to refine claims regarding the novelty and better position their work within the existing literature.

### Strengths

- The proposed system outperforms state-of-the-art methods like Zip-NeRF and Instant-NGP in both training times and view synthesis quality.
- An extensive ablation study is included, clarifying the contributions of various components to overall performance.
- The manuscript is well-organized, making it easy to follow.
- The introduction of a pilot network aids in stabilizing training and improving robustness.
- The paper is generally easy to reproduce and follow, with solid presentation.

### Weaknesses

- Time comparisons are not conducted on the same hardware, which raises concerns about the validity of performance claims.
- The consistency of rendered frames (3D smoothness) is not adequately demonstrated, potentially affecting qualitative results.
- The method's dependence on a pilot network and how it integrates with the rendering process needs clearer explanation.
- Key figures and equations lack clarity, which may confuse readers.
- The paper does not provide video results to demonstrate improvements in view-dependent effects.

### Questions

- Are the time comparisons performed on the same hardware?
- How does the proposed method maintain 3D consistency and smoothness in rendered animations compared to traditional methods?
- What is the exact role and operation of the pilot network in the proposed method?
- Can the authors clarify how the density is predicted and integrated into the rendering process?
- Could the authors explain the details of the spherical harmonic-based deferred rendering technique more precisely?

### Soundness

**Score:** 3

**Description:** 3 = good: The methodology is solid and the results are generally convincing, though some claims require more rigorous justification.

### Presentation

**Score:** 3

**Description:** 3 = good: The paper is well-structured and clear, but some sections need refinement for better clarity and precision.

### Contribution

**Score:** 3

**Description:** 3 = good: While the paper presents a valuable improvement over existing methods, its novelty is somewhat diminished by prior works that have explored similar concepts.

### Rating

**Score:** 6

**Description:** 6 = accept, but needs major improvements: The paper is technically sound and the contributions are beneficial, but significant revisions are necessary to address clarity and novelty concerns.

### Paper Decision

**Decision:** Accept (poster)

**Reasons:** The paper demonstrates solid technical contributions with improvements in speed and quality over existing methods. However, it requires revisions to clarify claims about its novelty and better position it in the context of related works. Incorporating reviewer feedback in the final version will enhance its overall quality.


Question:
You are a highly experienced, conscientious, and fair academic reviewer, please help me review this paper. The review should be organized into nine sections:
1. Summary: A summary of the paper in 100-150 words.
2. Strengths/Weaknesses/Questions: The Strengths/Weaknesses/Questions of paper, which should be listed in bullet points, with each point supported by specific examples from the article where possible.
3. Soundness/Contribution/Presentation: Rate the paper's Soundness/Contribution/Presentation, and match this score to the corresponding description from the list below and provide the result. The possible scores and their descriptions are:
    1 poor
    2 fair
    3 good
    4 excellent
4. Rating: Give this paper an appropriate rating, match this rating to the corresponding description from the list below and provide the result. The possible Ratings and their descriptions are:
    1 strong reject
    2 reject, significant issues present
    3 reject, not good enough
    4 possibly reject, but has redeeming facets
    5 marginally below the acceptance threshold
    6 marginally above the acceptance threshold
    7 accept, but needs minor improvements
    8 accept, good paper
    9 strong accept, excellent work
    10 strong accept, should be highlighted at the conference
5. Paper Decision: It must include the Decision itself(Accept or Reject) and the reasons for this decision, based on the criteria of originality, methodological soundness, significance of results, and clarity and logic of presentation.

Here is the template for a review format, you must follow this format to output your review result:
**Summary:**
Summary content

**Strengths:**
- Strength 1
- Strength 2
- ...

**Weaknesses:**
- Weakness 1
- Weakness 2
- ...

**Questions:**
- Question 1
- Question 2
- ...

**Soundness:**
Soundness result

**Presentation:**
Presentation result

**Contribution:**
Contribution result

**Rating:**
Rating result

**Paper Decision:**
- Decision: Accept/Reject
- Reasons: reasons content

Please ensure your feedback is objective and constructive. The paper is as follows:

# On Formal Feature Attribution and Its Approximation

Anonymous Author(s)

Affiliation

Address

email

###### Abstract

Recent years have witnessed the widespread use of artificial intelligence (AI) algorithms and machine learning (ML) models. Despite their tremendous success, a number of vital problems like ML model brittleness, their fairness, and the lack of interpretability warrant the need for the active developments in explainable artificial intelligence (XAI) and formal ML model verification. The two major lines of work in XAI include _feature selection_ methods, e.g. Anchors, and _feature attribution_ techniques, e.g. LIME and SHAP. Despite their promise, most of the existing feature selection and attribution approaches are susceptible to a range of critical issues, including explanation unsoundness and _out-of-distribution_ sampling. A recent formal approach to XAI (FXAI) although serving as an alternative to the above and free of these issues suffers from a few other limitations. For instance and besides the scalability limitation, the formal approach is unable to tackle the feature attribution problem. Additionally, a formal explanation despite being formally sound is typically quite large, which hampers its applicability in practical settings. Motivated by the above, this paper proposes a way to apply the apparatus of formal XAI to the case of feature attribution based on formal explanation enumeration. Formal feature attribution (FFA) is argued to be advantageous over the existing methods, both formal and non-formal. Given the practical complexity of the problem, the paper then proposes an efficient technique for approximating exact FFA. Finally, it offers experimental evidence of the effectiveness of the proposed approximate FFA in comparison to the existing feature attribution algorithms not only in terms of feature importance and but also in terms of their relative order.

## 1 Introduction

Thanks to the unprecedented fast growth and the tremendous success, Artificial Intelligence (AI) and Machine Learning (ML) have become a universally acclaimed standard in automated decision making causing a major disruption in computing and the use of technology in general [1; 29; 35; 47]. An ever growing range of practical applications of AI and ML, on the one hand, and a number of critical issues observed in modern AI systems (e.g. decision bias [3] and brittleness [64]), on the other hand, gave rise to the quickly advancing area of theory and practice of Explainable AI (XAI).

Numerous methods exist to explain decisions made by what is called black-box ML models [46; 48]. Here, _model-agnostic_ approaches based on random sampling prevail [46], with the most popular being _feature selection_[56] and _feature attribution_[40; 56] approaches. Despite their promise, model-agnostic approaches are susceptible to a range of critical issues, like unsoundness of explanations [21; 24] and _out-of-distribution sampling_[34; 62], which exacerbates the problem of trust in AI.

An alternative to model-agnostic explainers is represented by the methods building on the success of formal reasoning applied to the logical representations of ML models [42; 61]. Aiming to address the limitations of model-agnostic approaches, formal XAI (FXAI) methods themselves suffer from a few downsides, including the lack of scalability and the requirement to build a complete logicalrepresentation of the ML model. Formal explanations also tend to be larger than their model-agnostic counterparts because they do not reason about (unknown) data distribution [65]. Finally and most importantly, FXAI methods have not been applied so far to answer feature attribution questions.

Motivated by the above, we define a novel formal approach to feature attribution, which builds on the success of existing FXAI methods [42]. By exhaustively enumerating all formal explanations, we can give a crisp definition of _formal feature attribution_ (FFA) as the proportion of explanations in which a given feature occurs. We argue that formal feature attribution is hard for the second level of the polynomial hierarchy. Although it can be challenging to compute exact FFA in practice, we show that existing anytime formal explanation enumeration methods can be applied to efficiently approximate FFA. Our experimental results demonstrate the effectiveness of the proposed approach in practice and its advantage over SHAP and LIME given publicly available tabular and image datasets, as well as on a real application of XAI in the domain of Software Engineering [45; 52].

## 2 Background

This section briefly overviews the status quo in XAI and background knowledge the paper builds on.

### Classification Problems

Classification problems consider a set of classes \(\mathcal{K}=\{1,2,\ldots,k\}\)1, and a set of features \(\mathcal{F}=\{1,\ldots,m\}\). The value of each feature \(i\in\mathcal{F}\) is taken from a domain \(\mathbb{D}_{i}\), which can be categorical or ordinal, i.e. integer, real-valued or Boolean. Therefore, the complete feature space is defined as \(\mathbb{F}\triangleq\prod_{i=1}^{m}\mathbb{D}_{i}\). A concrete point in feature space is represented by \(\mathbf{v}=(v_{1},\ldots,v_{m})\in\mathbb{F}\), where each component \(v_{i}\in\mathbb{D}_{i}\) is a constant taken by feature \(i\in\mathcal{F}\). An _instance_ or _example_ is denoted by a specific point \(\mathbf{v}\in\mathbb{F}\) in feature space and its corresponding class \(c\in\mathcal{K}\), i.e. a pair \((\mathbf{v},c)\) represents an instance. Additionally, the notation \(\mathbf{x}=(x_{1},\ldots,x_{m})\) denotes an arbitrary point in feature space, where each component \(x_{i}\) is a variable taking values from its corresponding domain \(\mathbb{D}_{i}\) and representing feature \(i\in\mathcal{F}\). A classifier defines a non-constant classification function \(\kappa:\mathbb{F}\rightarrow\mathcal{K}\).

Footnote 1: Any set of classes \(\{c_{1},\ldots,c_{k}\}\) can always be mapped into the set of the corresponding indices \(\{1,\ldots,k\}\).

Many ways exist to learn classifiers \(\kappa\) given training data, i.e. a collection of labeled instances \((\mathbf{v},c)\), including decision trees [23] and their ensembles [11; 12], decision lists [57], neural networks [35], etc. Hereinafter, this paper considers boosted tree (BT) models trained with the use of XGBoost [12].

**Example 1**.: _Figure 1 shows a BT model trained for a simplified version of the adult dataset [33]. For a data instance \(\mathbf{v}=\{\textit{Education}=\textit{Bachelors},\textit{Status}=\textit{ Separated, Occupation}=\textit{Sales, Relation

Figure 1: Example boosted tree model [12] trained on the well-known _adult_ classification dataset.

Figure 2: Examples of feature attribution reported by LIME and SHAP, as well as both AXp’s (no more AXp’s exist) followed by FFA for the instance \(\mathbf{v}\) shown in Example 1.

ship \(=\) Not-in-family_, \(\text{Sex}=\) Male, _Hours/w \(\leq\) 40_), _the model predicts \(<\)50\(k\) because the sum of the weights in the 3 trees for this instance equals \(-0.4073=(-0.1089-0.2404-0.0580)<0\).

### ML Model Interpretability and Post-Hoc Explanations

Interpretability is generally accepted to be a subjective concept, without a formal definition [39]. One way to measure interpretability is in terms of the succinctness of information provided by an ML model to justify a given prediction. Recent years have witnessed an upsurge in the interest in devising and applying interpretable models in safety-critical applications [48, 58]. An alternative to interpretable models is post-hoc explanation of _black-box_ models, which this paper focuses on.

Numerous methods to compute explanations have been proposed recently [46, 48]. The lion's share of these comprise what is called _model-agnostic_ approaches to explainability [40, 55, 56] of heuristic nature that resort to extensive sampling in the vicinity of an instance being explained in order to "estimate" the behavior of the classifier in this local vicinity of the instance. In this regard, they rely on estimating input data distribution by building on the information about the training data [34]. Depending on the form of explanations model-agnostic approaches offer, they are conventionally classified as _feature selection_ or _feature attribution_ approaches briefly discussed below.

Feature Selection.A feature selection approach identifies subsets of features that are deemed _sufficient_ for a given prediction \(c=\kappa(\mathbf{v})\). As mentioned above, the majority of feature selection approaches are model-agnostic with one prominent example being Anchors [56]. As such, the sufficiency of the selected set of features for a given prediction is determined statistically based on extensive sampling around the instance of interest, by assessing a few measures like _fidelity, precision_, among others. As a result, feature selection explanations given as a set of features \(\omega\subseteq\mathcal{F}\) should be interpreted as the conjunction \(\bigwedge_{i\in\omega}\left(x_{i}=v_{i}\right)\) deemed responsible for prediction \(c=\kappa(\mathbf{v})\), \(\mathbf{v}\in\mathbb{F}\), \(c\in\mathcal{K}\). Due to the statistical nature of these explainers, they are known to suffer from various explanation quality issues [24, 34, 63]. An additional line of work on _formal_ explainability [25, 61] also tackles feature selection while offering guarantees of soundness; these are discussed below.

Feature Attribution.A different view on post-hoc explanations is provided by feature attribution approaches, e.g. LIME [55] and SHAP [40]. Based on random sampling in the neighborhood of the target instance, these approaches attribute responsibility to all model's features by assigning a numeric value \(w_{i}\in\mathbb{R}\) of importance to each feature \(i\in\mathcal{F}\). Given these importance values, the features can then be ranked from most important to least important. As a result, a feature attribution explanation is conventionally provided as a linear form \(\sum_{i\in\mathcal{F}}w_{i}\cdot x_{i}\), which can be also seen as approximating the original black-box explainer \(\kappa\) in the _local_ neighborhood of instance \(\mathbf{v}\in\mathbb{F}\). Among other feature attribution approaches, SHAP [5, 6, 40] is often claimed to stand out as it aims at approximating Shapley values, a powerful concept originating from cooperative games in game theory [60].

Formal Explainability.In this work, we build on formal explainability proposed in earlier work [8, 13, 25, 42, 61]. where explanations are equated with _abductive explanations_ (AXp's). Abductive explanations are _subset-minimal_ sets of features formally proved to suffice to explain an ML prediction given a formal representation of the classifier of interest. Concretely, given an instance \(\mathbf{v}\in\mathbb{F}\) and a prediction \(c=\kappa(\mathbf{v})\), an AXp is a subset-minimal set of features \(\mathcal{X}\subseteq\mathcal{F}\), such that

\[\forall(\mathbf{x}\in\mathbb{F})\cdot\bigwedge_{i\in\mathcal{X}}(x_{i}=v_{i}) \rightarrow(\kappa(\mathbf{x})=c) \tag{1}\]

Abductive explanations are guaranteed to be subset-minimal sets of features proved to satisfy (1). As other feature selection explanations, they answer _why_ a certain prediction was made. An alternate way to explain a model's behavior is to seek an answer _why not_ another prediction was made, or, in other words, _how_ to change the prediction. Explanations answering _why not_ questions are referred to as _contrastive explanations_ (CXp's) [26, 42, 46]. As in prior work, we define a CXp as a subset-minimal set of features that, if allowed to change their values, are _necessary_ to change the prediction of the model. Formally, a CXp for prediction \(c=\kappa(\mathbf{v})\) is a subset-minimal set of features \(\mathcal{Y}\subseteq\mathcal{F}\), such that

\[\exists(\mathbf{x}\in\mathbb{F})\cdot\bigwedge_{i\not\in\mathcal{Y}}(x_{i}=v_{ i})\wedge(\kappa(\mathbf{x})\neq c) \tag{2}\]

Finally, recent work has shown that AXp's and CXp's for a given instance \(\mathbf{v}\in\mathbb{F}\) are related through the _minimal hitting set duality_[26, 54]. The duality implies that each AXp for a prediction \(c=\kappa(\mathbf{v})\)is a _minimal hitting set2_ (MHS) of the set of all CXp's for that prediction, and the other way around: each CXp is an MHS of the set of all AXp's. The explanation enumeration algorithm [26] applied in this paper heavily relies on this duality relation and is inspired by the MARCO algorithm originating from the area of over-constrained systems [36, 37, 53]. A growing body of recent work on formal explanations is represented (but not limited) by [2, 4, 7, 9, 10, 14, 18, 20, 27, 41, 42, 43, 65].

Footnote 2: Given a set of sets \(\mathbb{S}\), a _hitting set_ of \(\mathbb{S}\) is a set \(H\) such that \(\forall S\in\mathbb{S},S\cup H\neq\emptyset\), i.e. \(H\) “hits” every set in \(\mathbb{S}\). A hitting set \(H\) for \(\mathbb{S}\) is _minimal_ if none of its strict subsets is also a hitting set.

**Example 2**.: _In the context of Example 1, feature attribution computed by LIME and SHAP as well as all 2 AXp's are shown in Figure 2. AXp \(\mathcal{X}_{1}\) indicates that specifying Education \(=\) Bachelors and Hours/\(\kappa\leq\) 40 guarantees that any compatible instance is classified as \(<\) 50k independent of the values of other features, e.g. Status and Relationship, since the maximal sum of weights is \(0.0770-0.0200-0.0580=-0.0010<0\) as long as the feature values above are used. Observe that another AXp \(\mathcal{X}_{2}\) for \(\mathbf{v}\) is [Education, Status]. Since both of the two AXp's for \(\mathbf{v}\) consist of two features, it is difficult to judge which one is better without a formal feature importance assessment._

## 3 Why Formal Feature Attribution?

On the one hand, abductive explanations serve as a viable alternative to non-formal feature selection approaches because they (i) guarantee subset-minimality of the selected sets of features and (ii) are computed via formal reasoning over the behavior of the corresponding ML model. Having said that, they suffer from a few issues. First, observe that deciding the validity of (1) requires a formal reasoner to take into account the complete feature space \(\mathbb{F}\), assuming that the features are independent and uniformly distributed [65]. In other words, the reasoner has to check all the combinations of feature values, including those that _never appear in practice_. This makes AXp's being unnecessarily _conservative_ (long), i.e. they may be hard for a human decision maker to interpret. Second, AXp's are not aimed at providing feature attribution. The abundance of various AXp's for a single data instance [25], e.g. see Example 2, exacerbates this issue as it becomes unclear for a user which of the AXp's to use to make an informed decision in a particular situation.

On the other hand, non-formal feature attribution in general is known to be susceptible to out-of-distribution sampling [62, 34] while SHAP is shown to fail to effectively approximate Shapley values [21]. Moreover and quite surprisingly, [21] argued that even the use of exact Shapley values is inadequate as a measure of feature importance. Our results below confirm that both LIME and SHAP often fail to grasp the real feature attribution in a number of practical scenarios.

To address the above limitations, we propose the concept of _formal feature attribution_ (FFA) as defined next. Let us denote the set of all formal abductive explanations for a prediction \(c=\kappa(\mathbf{v})\) by \(\mathbb{A}_{\kappa}(\mathbf{v},c)\). Then formal feature attribution of a feature \(i\in\mathcal{F}\) can be defined as the proportion of abductive explanations where it occurs. More formally,

**Definition 1**:: **(FFA).** The _formal feature attribution_\(\text{ffa}_{\kappa}(i,(\mathbf{v},c))\) of a feature \(i\in\mathcal{F}\) to an instance \((\mathbf{v},c)\) for machine learning model \(\kappa\) is

\[\text{ffa}_{\kappa}(i,(\mathbf{v},c))=\frac{|\{\mathcal{X}\mid\mathcal{X}\in \mathbb{A}_{\kappa}(\mathbf{v},c),i\in\mathcal{X})|}}{|\mathbb{A}_{\kappa}( \mathbf{v},c)|} \tag{3}\]

Formal feature attribution has some nice properties. First, it has a strict and formal definition, i.e. we can, assuming we are able to compute the complete set of AXp's for an instance, exactly define it for all features \(i\in\mathcal{F}\). Second, it is fairly easy to explain to a user of the classification system, even if they are non-expert. Namely, it is the percentage of (formal abductive) explanations that make use of a particular feature \(i\). Third, as we shall see later, even though we may not be able to compute all AXp's exhaustively, we can still get good approximations fast.

**Example 3**.: _Recall Example 2. As there are 2 AXp's for instance \(\mathbf{v}\), the prediction can be attributed to the 3 features with non-zero FFA shown in Figure 1(d). Also, observe how both LIME and SHAP (see Figure 1(a) and Figure 1(b)) assign non-zero attribution to the feature Relationship, which is in fact irrelevant for the prediction, but overlook the highest importance of feature Education._

One criticism of the above definition is that it does not take into account the length of explanations where the feature arises. Arguably if a feature arises in many AXp's of size 2, it should be considered more important than a feature which arises in the same number of AXp's but where each is of size 10. An alternate definition, which tries to take this into account, is the weighted formal feature attribution (WFFA), i.e. the _average_ proportion of AXp's that include feature \(i\in\mathcal{F}\). Formally,

**Definition 2**: **(WFFA).** The _weighted formal feature attribution_\(\text{wfffa}_{\kappa}(i,(\mathbf{v},c))\) of a feature \(i\in\mathcal{F}\) to an instance \((\mathbf{v},c)\) for machine learning model \(\kappa\) is

\[\text{wfffa}_{\kappa}(i,(\mathbf{v},c))=\frac{\sum_{\mathcal{X}\in\mathbb{A}_{ \kappa}(\mathbf{v},c),i\in\mathcal{X}}|\mathcal{X}|^{-1}}{|\mathbb{A}_{\kappa} (\mathbf{v},c)|} \tag{4}\]

Note that these attribution values are not on the same scale although they are convertible:

\[\sum_{i\in\mathcal{F}}\text{fffa}_{\kappa}(i,(\mathbf{v},c))=\frac{\sum_{ \mathcal{X}\in\mathbb{A}_{\kappa}(\mathbf{v},c)}|\mathcal{X}|}{|\mathbb{A}_{ \kappa}(\mathbf{v},c)|}\times\sum_{i\in\mathcal{F}}\text{wffa}_{\kappa}(i,( \mathbf{v},c)).\]

FFA can be related to the problem of _feature relevancy_[22], where a feature is said to be _relevant_ if it belongs to at least one AXp. Indeed, feature \(i\in\mathcal{F}\) is relevant for prediction \(c=\kappa(\mathbf{v})\) if and only if \(\text{ffa}_{\kappa}(i,(\mathbf{v},c))>0\). As a result, the following claim can be made.

**Proposition 1**: _Given a feature \(i\in\mathcal{F}\) and a prediction \(c=\kappa(\mathbf{v})\), deciding whether \(\text{fffa}_{\kappa}(i,(\mathbf{v},c))>\omega\), \(\omega\in(0,1]\), is at least as hard as deciding whether feature \(i\) is relevant for the prediction._

The above result indicates that computing exact FFA values may be expensive in practice. For example and in light of [22], one can conclude that the decision version of the problem is \(\Sigma_{2}^{\text{p}}\)-hard in the case of DNF classifiers.

Similarly and using the relation between FFA and feature relevancy above, we can note that the decision version of the problem is in \(\Sigma_{2}^{\text{p}}\) as long as deciding the validity of (1) is in NP, which in general is the case (unless the problem is simpler, e.g. for decision trees [28]). Namely, the following result is a simple consequence of the membership result for the feature relevance problem [22].

**Proposition 2**: _Deciding whether \(\text{fffa}_{\kappa}(i,(\mathbf{v},c))>\omega\), \(\omega\in(0,1]\), is in \(\Sigma_{2}^{\text{p}}\) if deciding (1) is in NP._

## 4 Approximating Formal Feature Attribution

As the previous section argues and as our experimental results confirm, it may be challenging in practice to compute exact FFA values due to the general complexity of the problem. Although some ML models admit efficient formal encodings and reasoning procedures, effective principal methods for FFA approximation seem necessary. This section proposes one such method.

Normally, formal explanation enumeration is done by exploiting the MHS duality between AXp's and CXp's and the use of MARCO-like [37] algorithms aiming at efficient exploration of minimal hitting sets of either AXp's or CXp's [26, 36, 37, 53]. Depending on the target type of formal explanation, MARCO exhaustively enumerates all such explanations one by one, each time extracting a candidate minimal hitting set and checking if it is a desired explanation. If it is then it is recorded and blocked such that this candidate is never repeated again. Otherwise, a dual explanation is extracted from the subset of features complementary to the candidate [25], gets recorded and blocked so that it is hit by each future candidate. The procedure proceeds until no more hitting sets of the set of dual explanations can be extracted, which signifies that all target explanations are enumerated. Observe that while doing so, MARCO also enumerates all the dual explanations as a kind of "side effect".

One of the properties of MARCO used in our approximation approach is that it is an _anytime_ algorithm, i.e. we can run it for as long as we need to get a sufficient number of explanations. This means we can stop it by using a timeout or upon collecting a certain number of explanations.

The main insight of FFA approximation is as follows. Recall that to compute FFA, we are interested in AXp enumeration. Although intuitively this suggests the use of MARCO targeting AXp's, for the sake of fast and high-quality FFA approximation, we propose to target CXp enumeration with AXp's as dual explanations computed "unintentionally". The reason for this is twofold: (i) we need to get a good FFA approximation as fast as we can and (ii) according to our practical observations, MARCO needs to amass a large number of dual explanations before it can start producing target explanations. This is because the hitting set enumerator is initially "blind" and knows nothing about the featuresit should pay attention to -- it uncovers this information gradually by collecting dual explanations to hit. This way a large number of dual explanations can quickly be enumerated during this initial phase of grasping the search space, essentially "for free". Our experimental results demonstrate the effectiveness of this strategy in terms of monotone convergence of approximate FFA to the exact FFA with the increase of the time limit. A high-level view of the version of MARCO used in our approach targeting CXp enumeration and amassing AXp's as dual explanations is shown in Algorithm 1.

```
1:procedureXpEnum(\(\kappa\), \(\mathbf{v}\), \(c\))
2:\((\mathbb{A},\mathbb{C})\leftarrow(\emptyset,\emptyset)\)\(\triangleright\)Sets of AXp's and CXp's to collect.
3:while true do
4:\(\mathcal{Y}\leftarrow\textsc{MinimalHS}(\mathbb{A},\mathbb{C})\)\(\triangleright\)Get a new MHS of \(\mathbb{A}\) subject to \(\mathbb{C}\).
5:if\(\mathcal{Y}=\bot\)thenbreak\(\triangleright\)Stop if none is computed.
6:if\(\exists(\mathbf{x}\in\mathbb{F}).\bigwedge_{i\not\in\mathcal{Y}}(x_{i}=v_{i}) \land(\kappa(\mathbf{x})\neq c)\)then\(\triangleright\)Check CXp condition (2) for \(\mathcal{Y}\).
7:\(\mathbb{C}\leftarrow\mathbb{C}\cup\{\mathcal{Y}\}\)\(\triangleright\)\(\mathcal{Y}\) appears to be a CXp.
8:else\(\triangleright\)There must be a missing AXp\(\mathcal{X}\subseteq\mathcal{F}\setminus\mathcal{Y}\).
9:\(\mathcal{X}\leftarrow\textsc{ExtractAXp}(\mathcal{F}\setminus\mathcal{Y}, \kappa,\mathbf{v},c)\)\(\triangleright\)Get AXp\(\mathcal{X}\) by iteratively checking (1) [25].
10:\(\mathbb{A}\leftarrow\mathbb{A}\cup\{\mathcal{X}\}\)\(\triangleright\)Collect new AXp\(\mathcal{X}\). return\(\mathbb{A}\), \(\mathbb{C}\)
```

**Algorithm 1** MARCO-like Anytime Explanation Enumeration

## 5 Experimental Evidence

This section assesses the formal feature attribution for gradient boosted trees (BT) [12] on multiple widely used images and tabular datasets, and compares FFA with LIME and SHAP. In addition, it also demonstrates the use of FFA in a real-world scenario of Just-in-Time (JIT) defect prediction, which assists teams in prioritizing their limited resources on high-risk commits or pull requests [52].

Setup and Prototype Implementation.All experiments were performed on an Intel Xeon 8260 CPU running Ubuntu 20.04.2 LTS, with the memory limit of 8 GByte. A prototype of the approach implementing Algorithm 1 and thus producing FFA was developed as a set of Python scripts and builds on [27]. As the FFA and WFFA values turn out to be almost identical (subject to normalization) in our experiments, here we report only FFA. WFFA results can be found in supplementary material.

Datasets and Machine Learning Models.The well-known MNIST dataset [15; 50] of handwritten digits 0-9 is considered, with two concrete binary classification tasks created: 1 vs. 3 and 1 vs. 7. We also consider PneumoniaMNIST [67], a binary classification dataset to distinguish X-ray images of pneumonia from normal cases. To demonstrate extraction of _exact_ FFA values for the above datasets, we also examine their downscaled versions, i.e. reduced from 28 \(\times\) 28 \(\times\) 1 to 10 \(\times\) 1. We also consider 11 tabular datasets often applied in the area of ML explainability and fairness [3; 16; 17; 19; 49; 59]. All the considered datasets are randomly split into 80% training and and 20% test data. For images, 15 test instances are randomly selected in each test set for explanation while all tabular test instances are explained. For all datasets, gradient boosted trees (BTs) are trained by XGBoost [12], where each BT consists of 25 trees of depth 3 per class.3 Finally, we show the use of FFA on 2 JIT defect prediction datasets [52], with 500 instances per dataset chosen for analysis.

Footnote 3: Test accuracy for MNIST digits is 0.99, while it is 0.83 for PneumoniaMNIST. This holds both for the 28 \(\times\) 28 and 10 \(\times\) 10 versions of the datasets. The average accuracy across the 11 selected tabular datasets is 0.80.

### Formal Feature Attribution

In this section, we restrict ourselves to examples where we can compute the _exact_ FFA values for explanations by computing all AXp's. To compare with LIME and SHAP, we take their solutions, replace negative attributions by the positive counterpart (in a sense taking the absolute value) and then normalize the values into \([0,1]\). We then compare these approaches with the computed FFA values, which are also in \([0,1]\). The _error_ is measured as Manhattan distance, i.e. the sum of absolute differences across all features. We also compare feature rankings according to the competitors (again using absolute values for LIME and SHAP) using Kendall's Tau [31] and rank-biased overlap (RBO) [66]metrics.4 Kendall's Tau and RBO are measured on a scale \([-1,1]\) and \([0,1]\), respectively. A higher value in both metrics indicates better agreement or closeness between a ranking and FFA.

Footnote 4: Kendall’s Tau is a correlation coefficient assessing the ordinal association between two ranked lists, offering a measure of similarity in the order of values; on the other hand, RBO is a metric that measures the similarity between two ranked lists, taking into account both the order and the depth of the overlap.

Tabular Data.Figure 3 exemplifies a comparison of FFA, LIME and SHAP on an instance of the Compas dataset [3]. While FFA and LIME agree on the most important feature, "Asian", SHAP gives it very little weight. Neither LIME nor SHAP agree with FFA, though there is clearly some similarity.

Table 1 details the comparison conducted on 11 tabular datasets, including _adult_, _compas_, and _recidivism_ datasets commonly used in XAI. For each dataset, we calculate the metric for each individual instance and then average the outcomes to obtain the final result for that dataset. As can be observed, the errors of LIME's feature attribution across these datasets span from 1.39 to 5.13. SHAP demonstrates similar errors within a range \([1.40,4.76]\). LIME and SHAP also exhibit comparable performance in relation to the two ranking comparison metrics. The values of Kendall's Tau for LIME (resp. SHAP) are between \(-0.36\) and \(0.22\) (resp. \(-0.39\) and \(0.27\)). Regarding the RBO values, LIME exhibits values between 0.39 and 0.68, whereas SHAP demonstrates values ranging from 0.44 to 0.67. Overall, as Table 1 indicates, both LIME and SHAP fail to get close enough to FFA.

10 \(\times\) 10 Digits.We now compare the results on 10 \(\times\) 10 downscaled MNIST digits and PneumoniaMNIST images, where it is feasible to compute all AXp's. Table 2 compares LIME's, SHAP's feature attribution and approximate FFA. Here, we run AXp enumeration for a number of seconds, which is denoted as FFA\({}_{*}\), \(*\in\mathbb{R}^{+}\). The runtime required for each image by LIME and SHAP is less than one second. The results show that the errors of our approximation are small, even after 10 seconds it beats both LIME and SHAP, and decreases as we generate more AXp's. The results for the orderings show again that after 10 seconds, FFA\({}_{*}\) ordering gets closer to the exact FFA than both LIME and SHAP. Observe how LIME is particularly far away from the _exact_ FFA ordering.

Summary._These results make us confident that we can get useful approximations to the exact FFA without exhaustively computing all AXp's while feature attribution determined by LIME and SHAP is quite erroneous and fails to provide a human-decision maker with useful insights, despite being fast.

\begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline
**Dataset** & **adult** & **appendicitis australian** & **cars** & **compas** & **heart-statilog** & **hungarian** & **lending** & **liver-disorder** & **pima** & **redictivism** \\ (\(|\mathcal{F}|\)) & (12) & (7) & (14) & (8) & (11) & (13) & (13) & (9) & (6) & (8) & (15) \\ \hline
**Approach** & & & & & & & & & & & & \\ \hline \hline \multirow{2}{*}{LIME} & 4.48 & 2.25 & 5.13 & 1.53 & 3.28 & 4.48 & 4.56 & 1.39 & 2.39 & 2.72 & 4.73 \\  & 4.47 & 2.01 & 4.49 & 1.40 & 2.67 & 3.71 & 4.14 & 1.44 & 2.28 & 3.00 & 4.76 \\ \hline \multicolumn{10}{c}{**Kendall’s Tau**} \\ \hline LIME & 0.07 & 0.11 & 0.22 & -0.11 & -0.11 & 0.17 & 0.04 & -0.36 & -0.22 & 0.17 & 0.05 \\  & SHAP & 0.03 & 0.12 & 0.27 & -0.10 & -0.10 & 0.17 & 0.20 & -0.39 & -0.21 & 0.07 & 0.12 \\ \multicolumn{10}{c}{**RBO**} \\ \hline LIME & 0.54 & 0.66 & 0.49 & 0.63 & 0.55 & 0.56 & 0.41 & 0.59 & 0.66 & 0.68 & 0.39 \\ SHAP & 0.49 & 0.67 & 0.55 & 0.66 & 0.59 & 0.52 & 0.49 & 0.61 & 0.67 & 0.63 & 0.44 \\ \hline \hline \end{tabular}
\end{table}
Table 1: LIME and SHAP versus FFA on tabular data.

Figure 3: Explanations for an instance of Compas \(\mathbf{v}=\{\text{\#Priors}=3,\text{Score\_factor}=1,\text{Age\_ Above\_ FourtyFive}=0,\text{Age\_ Below\_ TwentyFive}=1,\text{African\_ American}=1,\text{Asian}=0,\text{Hispanic}=0,\text{Native\_ American}=0,\text{Other}=0,\text{Female}=0,\text{Misdemeanor}=1\}\) predicted as Two\(\_\)yr\(\_\)Recidivism \(=\) true.

### Approximating Formal Feature Attribution

Since the problem of formal feature attribution "lives" in \(\Sigma_{2}^{\mathrm{P}}\), it is not surprising that computing FFA may be challenging in practice. Table 2 suggests that our approach gets good FFA approximations even if we only collect AXp's for a short time. Here we compare the fidelity of our approach versus the approximate FFA computed after 2 hours (7200s). Figure 4, 5, and 6 depict feature attributions generated by LIME, SHAP and FFA\({}_{*}\) for the three selected 28 \(\times\) 28 images. The comparison between LIME, SHAP, and the approximate FFA computation is detailed in Table 3. The LIME and SHAP processing time for each image is less than one second. The average findings detailed in Table 3 are consistent with those shown in Table 2. Namely, FFA approximation yields better errors, Kendall's Tau and RBO values, outperforming both LIME, and SHAP after 10 seconds. Furthermore, the results demonstrate that after 10 seconds our approach places feature attributions closer to FFA\({}_{7200}\) compared to both LIME and SHAP hinting on the features that are truly relevant for the prediction.

defect prediction has often been considered a black-box, lacking explainability for practitioners. To tackle this challenge, our proposed approach to generating FFA can be employed, as model-agnostic approaches cannot guarantee to provide accurate feature attribution (see above). We use logistic regression models of [52] based on large-scale open-source Openstack and Qt datasets provided by [45] commonly used for JIT defect prediction [52]. Monotonicity of logistic regression enables us to enumerate explanations using the approach of [44] and so to extract _exact FFA_ for each instance _within a second_. Table 4 details the comparison of FFA, LIME and SHAP in terms of the three considered metrics. As with the outcomes presented in Table 1, Table 2, and Table 3, neither LIME nor SHAP align with formal feature attribution, though there are some similarities between them.

## 6 Limitations

Despite the rigorous guarantees provided by formal feature attribution and high-quality of the result explanations, the following limitations can be identified. First, our approach relies on formal reasoning and thus requires an ML model of interest to admit a representation in some fragments of first-order logic, and the corresponding reasoner to deal with it [42]. Second, the problem complexity impedes immediate and widespread use of FFA and signifies the need to develop effective methods of FFA approximation. Finally, though our experimental evidence suggests that FFA approximations quickly converge to the exact values of FFA, whether or not this holds in general remains an open question.

## 7 Conclusions

Most approaches to XAI are heuristic methods that are susceptible to unsoundness and out-of-distribution sampling. Formal approaches to XAI have so far concentrated on the problem of feature selection, detecting which features are important for justifying a classification decision, and not on feature attribution, where we can understand the weight of a feature in making such a decision. In this paper we define the first formal approach to feature attribution (FFA) we are aware of, using the proportion of abductive explanations in which a feature occurs to weight its importance. We show that we can compute FFA exactly for many classification problems, and when we cannot we can compute effective approximations. Existing heuristic approaches to feature attribution do not agree with FFA. Sometimes they markedly differ, for example, assigning no weight to a feature that appears in (a large number of) explanations, or assigning (large) non-zero weight to a feature that is irrelevant for the prediction. Overall, the paper argues that if we agree that FFA is a correct measure of feature attribution then we need to investigate methods that compute good FFA approximations quickly.

\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Approach**} & \multicolumn{3}{c}{**openstack \((|\mathcal{F}|=13)\)**} & \multicolumn{3}{c}{**qt \((|\mathcal{F}|=16)\)**} \\ \cline{2-7}  & **Error** & **Kendall’s Tau** & **RBO** & **Error** & **Kendall’s Tau** & **RBO** \\ \hline LIME & 4.84 & 0.05 & 0.55 & 5.63 & -0.08 & 0.45 \\ SHAP & 5.08 & 0.00 & 0.53 & 5.22 & -0.13 & 0.44 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Just-in-Time Defect Prediction comparison of FFA versus LIME and SHAP.

Figure 5: 28 \(\times\) 28 MNIST 1 vs. 7. The prediction is digit 7.

Figure 6: 28 \(\times\) 28 PneumoniaMNIST. The prediction is normal.

## References

* [1] ACM. Fathers of the deep learning revolution receive ACM A.M. Turing award. [http://tiny.cc/9plzpz](http://tiny.cc/9plzpz), 2018.
* [2] L. Amgoud and J. Ben-Naim. Axiomatic foundations of explainability. In L. D. Raedt, editor, _IJCAI_, pages 636-642, 2022.
* [3] J. Angwin, J. Larson, S. Mattu, and L. Kirchner. Machine bias. [http://tiny.cc/dd7mjz](http://tiny.cc/dd7mjz), 2016.
* [4] M. Arenas, D. Baez, P. Barcelo, J. Perez, and B. Subercaseaux. Foundations of symbolic languages for model interpretability. In _NeurIPS_, 2021.
* [5] M. Arenas, P. Barcelo, L. E. Bertossi, and M. Monet. The tractability of SHAP-score-based explanations for classification over deterministic and decomposable Boolean circuits. In _AAAI_, pages 6670-6678. AAAI Press, 2021.
* [6] M. Arenas, P. Barcelo, L. E. Bertossi, and M. Monet. On the complexity of SHAP-score-based explanations: Tractability via knowledge compilation and non-approximability results. _CoRR_, abs/2104.08015, 2021.
* [7] M. Arenas, P. Barcelo, M. A. R. Orth, and B. Subercaseaux. On computing probabilistic explanations for decision trees. In _NeurIPS_, 2022.
* [8] G. Audemard, F. Koriche, and P. Marquis. On tractable XAI queries based on compiled representations. In _KR_, pages 838-849, 2020.
* [9] G. Blanc, J. Lange, and L. Tan. Provably efficient, succinct, and precise explanations. In _NeurIPS_, 2021.
* [10] R. Boumazouza, F. C. Alili, B. Mazure, and K. Tabia. ASTERYX: A model-Agnostic SaT-basEd appRoach for SYmbolic and score-based eXplanations. In _CIKM_, pages 120-129, 2021.
* [11] L. Breiman. Random forests. _Mach. Learn._, 45(1):5-32, 2001.
* [12] T. Chen and C. Guestrin. XGBoost: A scalable tree boosting system. In _KDD_, pages 785-794, 2016.
* [13] A. Darwiche and A. Hirth. On the reasons behind decisions. In _ECAI_, pages 712-720, 2020.
* [14] A. Darwiche and P. Marquis. On quantifying literals in Boolean logic and its applications to explainable AI. _J. Artif. Intell. Res._, 72:285-328, 2021.
* [15] L. Deng. The MNIST database of handwritten digit images for machine learning research. _IEEE Signal Processing Magazine_, 29(6):141-142, 2012.
* [16] D. Dua and C. Graff. UCI machine learning repository, 2017. [http://archive.ics.uci.edu/ml](http://archive.ics.uci.edu/ml).
* [17] FairML. Auditing black-box predictive models. [http://tiny.cc/6e7mjz](http://tiny.cc/6e7mjz), 2016.
* [18] J. Ferreira, M. de Sousa Ribeiro, R. Goncalves, and J. Leite. Looking inside the black-box: Logic-based explanations for neural networks. In _KR_, page 432-442, 2022.
* [19] S. Friedler, C. Scheidegger, and S. Venkatasubramanian. On algorithmic fairness, discrimination and disparate impact. [http://fairness.haverford.edu/](http://fairness.haverford.edu/), 2015.
* [20] N. Gorji and S. Rubin. Sufficient reasons for classifier decisions in the presence of domain constraints. In _AAAI_, pages 5660-5667, 2022.
* [21] X. Huang and J. Marques-Silva. The inadequacy of Shapley values for explainability. _CoRR_, abs/2302.08160, 2023.
* [22] X. Huang, M. C. Cooper, A. Morgado, J. Planes, and J. Marques-Silva. Feature necessity & relevancy in ML classifier explanations. In _TACAS (1)_, pages 167-186, 2023.

* Hyafil and Rivest [1976] L. Hyafil and R. L. Rivest. Constructing optimal binary decision trees is NP-complete. _Inf. Process. Lett._, 5(1):15-17, 1976. URL [https://doi.org/10.1016/0020-0190](https://doi.org/10.1016/0020-0190)(76)90095-8.
* Ignatiev [2020] A. Ignatiev. Towards trustable explainable AI. In _IJCAI_, pages 5154-5158, 2020.
* Ignatiev et al. [2019] A. Ignatiev, N. Narodytska, and J. Marques-Silva. Abduction-based explanations for machine learning models. In _AAAI_, pages 1511-1519, 2019.
* Ignatiev et al. [2020] A. Ignatiev, N. Narodytska, N. Asher, and J. Marques-Silva. From contrastive to abductive explanations and back again. In _AI*IA_, pages 335-355, 2020.
* Ignatiev et al. [2022] A. Ignatiev, Y. Izza, P. J. Stuckey, and J. Marques-Silva. Using MaxSAT for efficient explanations of tree ensembles. In _AAAI_, pages 3776-3785, 2022.
* Izza et al. [2022] Y. Izza, A. Ignatiev, and J. Marques-Silva. On tackling explanation redundancy in decision trees. _J. Artif. Intell. Res._, 75:261-321, 2022. URL [https://doi.org/10.1613/jair.1.13575](https://doi.org/10.1613/jair.1.13575).
* Jordan and Mitchell [2015] M. I. Jordan and T. M. Mitchell. Machine learning: Trends, perspectives, and prospects. _Science_, 349(6245):255-260, 2015.
* Kamei et al. [2013] Y. Kamei, E. Shihab, B. Adams, A. E. Hassan, A. Mockus, A. Sinha, and N. Ubayashi. A Large-Scale Empirical Study of Just-In-Time Quality Assurance. _IEEE Transactions on Software Engineering (TSE)_, 39(6):757-773, 2013.
* Kendall [1938] M. G. Kendall. A new measure of rank correlation. _Biometrika_, 30(1/2):81-93, 1938.
* Kim et al. [2007] S. Kim, T. Zimmermann, E. J. Whitehead Jr, and A. Zeller. Predicting Faults from Cached History. In _ICSE_, pages 489-498, 2007.
* Kohavi [1996] R. Kohavi. Scaling up the accuracy of naive-Bayes classifiers: A decision-tree hybrid. In _KDD_, pages 202-207, 1996.
* Lakkaraju and Bastani [2020] H. Lakkaraju and O. Bastani. "How do I fool you?": Manipulating user trust via misleading black box explanations. In _AIES_, pages 79-85, 2020.
* LeCun et al. [2015] Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. _Nature_, 521(7553):436, 2015.
* Liffiton and Malik [2013] M. H. Liffiton and A. Malik. Enumerating infeasibility: Finding multiple MUSes quickly. In _CPAIOR_, pages 160-175, 2013.
* Liffiton et al. [2016] M. H. Liffiton, A. Previti, A. Malik, and J. Marques-Silva. Fast, flexible MUS enumeration. _Constraints An Int. J._, 21(2):223-250, 2016.
* Lin et al. [2021] D. Lin, C. Tantithamthavorn, and A. E. Hassan. The impact of data merging on the interpretation of cross-project just-in-time defect models. _IEEE Transactions on Software Engineering_, 2021.
* Lipton [2018] Z. C. Lipton. The mythos of model interpretability. _Commun. ACM_, 61(10):36-43, 2018.
* Lundberg and Lee [2017] S. M. Lundberg and S. Lee. A unified approach to interpreting model predictions. In _NeurIPS_, pages 4765-4774, 2017.
* Malfa et al. [2021] E. L. Malfa, R. Michelmore, A. M. Zbrzezny, N. Paoletti, and M. Kwiatkowska. On guaranteed optimal robust explanations for NLP models. In _IJCAI_, pages 2658-2665, 2021.
* Marques-Silva and Ignatiev [2022] J. Marques-Silva and A. Ignatiev. Delivering trustworthy AI through formal XAI. In _AAAI_, pages 12342-12350. AAAI Press, 2022.
* Marques-Silva et al. [2020] J. Marques-Silva, T. Gerspacher, M. C. Cooper, A. Ignatiev, and N. Narodytska. Explaining naive Bayes and other linear classifiers with polynomial time and delay. In _NeurIPS_, 2020.
* Marques-Silva et al. [2021] J. Marques-Silva, T. Gerspacher, M. C. Cooper, A. Ignatiev, and N. Narodytska. Explanations for monotonic classifiers. In _ICML_, pages 7469-7479, 2021.
* McIntosh and Kamei [2017] S. McIntosh and Y. Kamei. Are fix-inducing changes a moving target? A longitudinal case study of Just-in-Time defect prediction. _IEEE Transactions on Software Engineering (TSE)_, pages 412-428, 2017.

* Miller [2019] T. Miller. Explanation in artificial intelligence: Insights from the social sciences. _Artif. Intell._, 267:1-38, 2019.
* Mnih et al. [2015] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. Human-level control through deep reinforcement learning. _Nature_, 518(7540):529, 2015.
* Molnar [2020] C. Molnar. _Interpretable Machine Learning_. Leanpub, 2020. [http://tiny.cc/6c76tz](http://tiny.cc/6c76tz).
* Olson et al. [2017] R. S. Olson, W. G. L. Cava, P. Orzechowski, R. J. Urbanowicz, and J. H. Moore. PMLB: a large benchmark suite for machine learning evaluation and comparison. _BioData Min._, 10(1):36:1-36:13, 2017.
* Paszke et al. [2019] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Z. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala. PyTorch: An imperative style, high-performance deep learning library. In _NeurIPS_, pages 8024-8035, 2019.
* Pornprasit and Tantithamthavorn [2021] C. Pornprasit and C. Tantithamthavorn. JITLine: A Simpler, Better, Faster, Finer-grained Just-In-Time Defect Prediction. In _MSR_, pages 369-379, 2021.
* Pornprasit et al. [2021] C. Pornprasit, C. Tantithamthavorn, J. Jiarpakdee, M. Fu, and P. Thongtanunam. PyExplainer: Explaining the predictions of Just-In-Time defect models. In _ASE_, pages 407-418, 2021.
* Previti and Marques-Silva [2013] A. Previti and J. Marques-Silva. Partial MUS enumeration. In _AAAI_. AAAI Press, 2013.
* Reiter [1987] R. Reiter. A theory of diagnosis from first principles. _Artif. Intell._, 32(1):57-95, 1987.
* Ribeiro et al. [2016] M. T. Ribeiro, S. Singh, and C. Guestrin. "Why should I trust you?": Explaining the predictions of any classifier. In _KDD_, pages 1135-1144, 2016.
* Ribeiro et al. [2018] M. T. Ribeiro, S. Singh, and C. Guestrin. Anchors: High-precision model-agnostic explanations. In _AAAI_, pages 1527-1535, 2018.
* Rivest [1987] R. L. Rivest. Learning decision lists. _Mach. Learn._, 2(3):229-246, 1987.
* Rudin [2019] C. Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. _Nat. Mach. Intell._, 1(5):206-215, 2019.
* Schmidt and Witte [1988] P. Schmidt and A. D. Witte. Predicting recidivism in North Carolina, 1978 and 1980. _InterUniversity Consortium for Political and Social Research_, 1988.
* Shapley [1953] L. S. Shapley. A value of \(n\)-person games. _Contributions to the Theory of Games_, 2(28):307-317, 1953.
* Shih et al. [2018] A. Shih, A. Choi, and A. Darwiche. A symbolic approach to explaining Bayesian network classifiers. In _IJCAI_, pages 5103-5111, 2018.
* Slack et al. [2020] D. Slack, S. Hilgard, E. Jia, S. Singh, and H. Lakkaraju. Fooling LIME and SHAP: adversarial attacks on post hoc explanation methods. In _AIES_, pages 180-186, 2020.
* Slack et al. [2021] D. Slack, A. Hilgard, S. Singh, and H. Lakkaraju. Reliable post hoc explanations: Modeling uncertainty in explainability. In _NeurIPS_, pages 9391-9404, 2021.
* Szegedy et al. [2014] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In _ICLR (Poster)_, 2014.
* Waldchen et al. [2021] S. Waldchen, J. MacDonald, S. Hauch, and G. Kutyniok. The computational complexity of understanding binary classifier decisions. _J. Artif. Intell. Res._, 70:351-387, 2021.
* Webber et al. [2010] W. Webber, A. Moffat, and J. Zobel. A similarity measure for indefinite rankings. _ACM Transactions on Information Systems (TOIS)_, 28(4):1-38, 2010.
* Yang et al. [2023] J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni. MedMNIST v2-a large-scale lightweight benchmark for 2D and 3D biomedical image classification. _Scientific Data_, 10(1):41, 2023.

Response:
## Review

### Summary

This paper introduces a novel approach called Formal Feature Attribution (FFA) for explaining predictions of black box models, addressing limitations of existing model-agnostic methods like SHAP and LIME. FFA defines feature attribution as the proportion of minimal explanations in which a feature occurs, leveraging abductive explanations. Although the authors propose an approximation algorithm to compute FFA efficiently, they acknowledge the computational challenges of exact calculations. The empirical validation of FFA is presented through experiments on various datasets, showcasing its potential in the field of Explainable AI.

### Strengths

- The paper is well-written and easy to understand, making it a pleasure to read.
- FFA is rooted in a simple concept, with clear motivations and sensible algorithm design.
- Good empirical performance was demonstrated on various datasets.
- The authors clearly discuss essential limitations and provide a well-structured limitations section.
- The related work is well-referenced and provides a good context for the proposed method.

### Weaknesses

- The rationale for claiming FFA as the 'real' feature attribution is not well-articulated.
- Important theoretical aspects, such as axioms governing FFA, are missing or unclear.
- The experimental results are limited, particularly in the context of comparing FFA with state-of-the-art methods.
- The paper lacks a formal proof for some propositions and does not provide guarantees for the approximation method.
- Some definitions and annotations in the paper are not adequately clarified, making it hard to follow.

### Questions

- What is the motivation behind claiming that FFA provides a better feature attribution than LIME and SHAP?
- How does FFA address the issues of out-of-distribution sampling effectively?
- Could the averaging of AXps lead to loss of important information regarding feature interactions?
- What axioms do FFA explanations adhere to, and how do they compare with those of LIME and SHAP?

### Soundness

**Score:** 2

**Description:** 2 = fair. The theoretical foundations are somewhat lacking, with key aspects such as approximation guarantees and axiomatic analysis not fully developed.

### Presentation

**Score:** 3

**Description:** 3 = good. While the writing is generally clear and organized, some definitions and explanations require better clarity for full comprehension.

### Contribution

**Score:** 2

**Description:** 2 = fair. The paper presents a novel approach, but significant gaps exist in theoretical grounding and empirical evaluation, limiting its impact.

### Rating

**Score:** 5

**Description:** 5 = borderline accept: The paper has technical merit, but due to limited evaluation and clarity issues, it does not meet the standards for acceptance at this time.

### Paper Decision

**Decision:** Reject

**Reasons:** While the paper presents an interesting approach to feature attribution with potential contributions to Explainable AI, the lack of rigorous theoretical justification, insufficient empirical validation, and unclear motivations for its claims hinder its acceptance. The reviewers unanimously felt that the ideas need further development to achieve publication standards.