Learnable Gabor kernels in convolutional neural networks for seismic
  interpretation tasks
The use of convolutional neural networks (CNNs) in seismic interpretation
tasks, like facies classification, has garnered a lot of attention for its high
accuracy. However, its drawback is usually poor generalization when trained
with limited training data pairs, especially for noisy data. Seismic images are
dominated by diverse wavelet textures corresponding to seismic facies with
various petrophysical parameters, which can be suitably represented by Gabor
functions. Inspired by this fact, we propose using learnable Gabor
convolutional kernels in the first layer of a CNN network to improve its
generalization. The modified network combines the interpretability features of
Gabor filters and the reliable learning ability of original CNN. More
importantly, it replaces the pixel nature of conventional CNN filters with a
constrained function form that depends on 5 parameters that are more in line
with seismic signatures. Further, we constrain the angle and wavelength of the
Gabor kernels to certain ranges in the training process based on what we expect
in the seismic images. The experiments on the Netherland F3 dataset show the
effectiveness of the proposed method in a seismic facies classification task,
especially when applied to testing data with lower signal-to-noise ratios.
Besides, we also test this modified CNN using different kernels on
salt$\&$pepper and speckle noise. The results show that we obtain the best
generalization and robustness of the CNN to noise when Gabor kernels are used
in the first layer.
Fu Wang, Tariq Alkhalifah
# Learnable Gabor kernels in convolutional neural networks for seismic interpretation tasks

###### Abstract

The use of convolutional neural networks (CNNs) in seismic interpretation tasks, like facies classification, has garnered a lot of attention for its high accuracy. However, its drawback is usually poor generalization when trained with limited training data pairs, especially for noisy data. Seismic images are dominated by diverse wavelet textures corresponding to seismic facies with various petrophysical parameters, which can be suitably represented by Gabor functions. Inspired by this fact, we propose using learnable Gabor convolutional kernels in the first layer of a CNN network to improve its generalization. The modified network combines the interpretability features of Gabor filters and the reliable learning ability of original CNN. More importantly, it replaces the pixel nature of conventional CNN filters with a constrained function form that depends on 5 parameters that are more in line with seismic signatures. Further, we constrain the angle and wavelength of the Gabor kernels to certain ranges in the training process based on what we expect in the seismic images. The experiments on the Netherland F3 dataset show the effectiveness of the proposed method in a seismic facies classification task, especially when applied to testing data with lower signal-to-noise ratios. Besides, we also test this modified CNN using different kernels on salt\(\&\)pepper and speckle noise. The results show that we obtain the best generalization and robustness of the CNN to noise when Gabor kernels are used in the first layer.

## 1 Introduction

Seismic interpretation is a fundamental step in the seismic exploration value chain, and it is often the most critical step in the decision-making process (Dumay and Fournier, 1988). Within the seismic interpretation tasks, identifying seismic facies from seismic images plays a vital role in hydrocarbon exploration and development. Seismic facies with different petrophysical parameters often induce different seismic responses. The identification of seismic facies can be regarded as a pattern-recognition problem. The traditional manual interpretation of seismic facies is highly dependent on the skills of interpreters and it is also time-consuming. In order to overcome these limitations, multiple seismic attributes with explainable correspondence have been proposed to assist seismic interpretation. Subsequently, some machine learning methods including support vector machine (Zhao et al., 2014; Zhang et al., 2015; Wrona et al., 2018), the random forest (Ao et al., 2019), self-organizing maps (Saraswat and Sen, 2012; Zhao et al., 2017), generative topographic mapping (Roy et al., 2014), and independent component analysis (Lubo-Robles and Marfurt, 2019) use these attributes to classify seismic facies.

Recently, with the rising popularity of deep learning, many have proposed using convolutional neural networks (CNNs) to promote seismic interpretation tasks with reasonable success, like seismic facies classification (Zhao, 2018; Di et al., 2020; Feng et al., 2021; Liu et al., 2020), salt body identification (Waldeland and Solberg, 2017; Shi et al., 2019; Di and AlRegib, 2020), fault detection
[Di et al., 2018; Zhao and Mukhopadhyay, 2018; Wu et al., 2019, 2019, 2019], and horizon tracking [Wu et al., 2019, 2019]. The advantage of CNNs is their strong representation and nonlinear mapping abilities in handling large datasets, which improve the speed and accuracy of seismic facies interpretation. However, CNNs also have some disadvantages. Without enough training data pairs, they can easily overfit, resulting in poor generalization. In other words, the performance on tests and especially inference data could be poor, which is a serious limitation of CNNs. One of the many suitable ways to remedy this weakness is to embed prior information into CNNs. Specifically, for seismic facies classification, the input seismic images mainly include texture features constituted by band-limited wavelets and some additive or correlated noise. Thus, we consider the Gabor filter as a good prior to constraining the CNNs. Seismic data have long been represented and filtered by Gabor functions [Womack and Cruz, 1994]. For machine learning, Daugman [1985] first proposed the Gabor function to model the spatial summation properties (of the receptive fields) of simple cells in the visual cortex. Soon afterward, Gabor filters were widely used in diverse pattern analysis applications and are regarded as an efficient tool in extracting spatial local texture features. Besides, Krizhevsky et al. [2017] demonstrated that after training on real-life images, the first convolution layer of deep CNNs tends to learn Gabor-like filters. Thus, using Gabor filters in the first layer of a CNN for seismic pattern recognition problems could provide more reliable results.

In this work, encouraged by the good interpretability of the Gabor filter and the reliable learning ability of CNNs, we use a Gabor convolution kernel in the first layer of a CNN for seismic facies classification, as an example of interpretation task. Such a modification will replace the pixel focus of the common CNN filters with a constrained function form that is in line with seismic signatures. As a result, the Gabor filter parameters are learnable, and these parameters, like wavelength and angle, provide identifiable characteristics of seismic signals. In fact, each Gabor kernel is controlled by 5 parameters with distinct meanings and can be constrained within the range we expect in the seismic images. Thus, we utilize constraints on the wavelength and angle range of the Gabor filter. We test our method on the Netherland F3 datasets to validate the features gained by replacing the conventional convolution kernel with the Gabor kernel.

Specifically, the main contributions of this paper can be summarized as follows:

* We propose a simple modified CNN with Gabor kernels in the first layer, which embeds the prior seismic texture into neural networks. We regularize certain parameters of Gabor kernels to filter the input based on our expectations of seismic images.
* Our approach provides new insights into incorporating the prior information provided by Gabor functions into CNN for seismic facies classification, and these insights could be easily adapted to other seismic data tasks.
* We evaluate the effectiveness of the CNN with Gabor kernels on seismic images with lower signal-noise ratios, and with added salt\(\&\)pepper and speckle noise, and show the improvements in accuracy compared to conventional CNN.

The rest of the paper is organized as follows: we first introduce the Gabor function. Next, we explain how to replace Conv kernels with Gabor kernels in the first layer of CNN, and how to regularize the parameter of Gabor kernels during the training. Then we show the results of conventional CNN and our modified CNN, and the modified CNN with constrained Gabor kernels. To further validate the generalization and robustness to noise, we also test on salt\(\&\)pepper and speckle noise. Finally, we share out thoughts and summarize our developments in the discussion and conclusion sections.

## 2 Method

### The Gabor function

A two-dimensional Gabor function is defined as a complex sinusoidal plane wave weighted by a Gaussian filter, as follows

\[g(x,y;\lambda,\theta,\phi,\sigma,\gamma)=\exp\left(-\frac{x^{\prime 2}+ \gamma^{2}y^{\prime 2}}{2\sigma^{2}}\right)\exp\left(i\left(2\pi\frac{x^{ \prime}}{\lambda}+\phi\right)\right), \tag{1}\]

where \(x^{\prime}=x\cos\theta+y\sin\theta\) and \(y^{\prime}=-x\sin\theta+y\cos\theta\); \(i\) is the imaginary unit and \(\Theta=\{\lambda,\theta,\phi,\gamma,\sigma\}\) are the parameters controlling the shape of Gabor function. Thereinto, \(\lambda\) represents the wavelength of the sinusoidal plane wave component of the Gabor function; \(\theta\) defines the
orientation (angle) of the plane wave; \(\phi\) is the phase shift; \(\gamma\) represents the ellipticity of the Gaussian support of the Gabor function; \(\sigma\) controls the standard deviation of the Gaussian filter of the Gabor function.

Equation 1 can be rewritten in real numbers form, by splitting it into its real and imaginary parts, with the real part

\[g_{r}(x,y;\lambda,\theta,\phi,\sigma,\gamma)=\exp\left(-\frac{x^{2}+\gamma^{2}y ^{\prime 2}}{2\sigma^{2}}\right)\cos\left(2\pi\frac{x^{\prime}}{\lambda}+\phi \right), \tag{2}\]

and the imaginary part

\[g_{i}(x,y;\lambda,\theta,\phi,\sigma,\gamma)=\exp\left(-\frac{x^{\prime 2}+ \gamma^{2}y^{2}}{2\sigma^{2}}\right)\sin\left(2\pi\frac{x^{\prime}}{\lambda}+ \phi\right), \tag{3}\]

In this work, we use (equation 2) to replace the classic convolution kernel in the first layer of our CNN, as shown in Figure 1.

### Learnable Gabor kernels in CNNs

Traditionally, we specify the Gabor function parameters in advance before we use them as a filter. Gabor filters generated in this way may not be optimal for certain inputs. Besides, it is not an easy task to select suitable parameter values for the Gabor function. So in our work, all the parameters in the Gabor function are potentially learned from the training data in the training process. In other words, we use Gabor convolution kernels that adaptively update their parameters instead of using hand-crafted parameters. Compared with classical convolution kernels in which each pixel is learned independently, the parameters \(\Theta=\{\lambda,\theta,\phi,\gamma,\sigma\}\) in the Gabor kernel constrain the relation between the pixels to produce the Gabor filter. In addition, a Gabor convolution kernel only has 5 parameters, whereas a classical convolution with kernel size \(k\) will have \(k^{2}\) independent pixels to determine. Luckily, the Gabor convolution module can be embedded into any network and we can train the 5 parameters using the gradient descent method as part of the full network for as many CNN channels we need to represent the input.

### Constrained Gabor kernels in first layers

Due to the distinct meaning of the five parameters in the Gabor convolution kernel, we can regularize them to control (filter) the input. For seismic images, two key parameters are the wavelength and direction of the plane wave. In this work, we set a range for these parameters that is in line with what we want to include from the seismic images for facies classification. This range can also help reduce the propagation of noise into our CNN.

Figure 1: The U-Net structure with learnable Gabor kernels in the first layer.
## 3 Numerical examples

We use part of the Netherland F3 dataset (specifically, inlines [300,700] and crosslines [300,1000]), which includes six types of facies [Alaudah et al., 2019] to demonstrate the generalization abilities of our proposed method. The dataset includes six groups of lithoststratigraphic units, from top to bottom consisting of the Upper North Sea group, the Middle North Sea group, the Lower North Sea group, the Rijnland/chalk group, the Scruff group, and the Zechstein group. We uniformly select 21 of the 401 inline profiles as the training dataset, while testing on the rest. To make it more challenging, we add 0.4dB random noise to the training dataset, but add random noise with the same and lower signal-to-noise ratios to the testing dataset. The backbone network used here is UNet with 23 layers. First, we test two conventional UNets with 3\(\times\)3 and 11\(\times\)11 Conv Kernels, and a modified UNet with a Gabor convolutional kernels (11\(\times\)11) without any constraints in the first layer. As for the residual layers, we use 3\(\times\)3 Conv Kernels. Then we test the Gabor kernels with different angles constraints (\(\theta\in[-\pi/6,\pi/6]\), \([-\pi/4,\pi/4]\) and \([-\pi/3,\pi/3]\)) and wavelength constraints (\(\lambda\in[4,+\infty)\), \([8,+\infty)\), \([12,+\infty)\)), respectively. Besides, to further test the generalization of the Gabor kernels, we include salt&pepper and speckle noise in our tests.

Here we use the following metrics: pixel accuracy (PA), class accuracy (CA) for a single class, mean class accuracy (MCA) for all the classes, mean class accuracy (MCA), frequency-weighted intersection over union (FWIU), and mean intersection over union (Mean IU) to evaluate the performance of seismic facies segmentation with various kernels in the first layer. A detailed explanation of these metrics is provided in Appendix.

### The comparison between Gabor kernel and Conv kernel

We first display the training metric curves, including FWIU, MCA, Mean IU, PA, and loss, for the 3\(\times\)3 and 11\(\times\)11 Conv kernels and the 11\(\times\)11 Gabor kernels in Figure 2. Figure 3 shows the corresponding test metric curves for the test data with 0.4dB random noise. During the training, we observe that the FWIU and PA of the CNNs with 3\(\times\)3 and 11\(\times\)11 Conv kernels or the 11\(\times\)11 Gabor kernel are close to each other, though there is an improvement using the Gabor kernel. While in terms of MCA and Mean IU, the CNN with the 11\(\times\)11 Gabor kernel has noticeable improvements compared to the classic conventional CNN, where the CNN with the Gabor kernel converges much faster (at 45 epochs) than those with the Conv kernels, which converge at 90 epochs. Recall that FWIU and PA (see Appendix), unlike the other measure, reflect the overall performance of the classification. Still, the MCA and Mean IU provide average scores for different classes, which means the high score only happens when the accuracy for each class reaches a high level. Thus, metrics demonstrate that the CNN with the Gabor kernel could quickly fit the classes with fewer samples.

At the testing stage, we observe that when the training accuracy reaches the same level, the testing accuracy of the CNN with the Gabor kernel is much better than that of the CNN with Conv kernel. This implies that the CNN with the Gabor kernel has better generalization properties.

Figure 4 is the visualization of the predictions for inline #355 using the trained model after 50 training epochs. It shows that with limited training epochs, the CNN with Conv kernel could not reasonably predict the classes with limited training samples, while the CNN with Gabor kernel could achieve that. Figure 5 is the visualization of the predictions for inline #375 using the trained model after 100 training epochs. For this case, all types of CNN predict well.

To further test the generalization of the CNN with different kernels, we test these types of CNNs on testing images containing -5.6 dB white Gaussian noise, shown in Figure 6. We see that even when dealing with noisy images, whose signal-to-noise level is lower than that of the training data, the Gabor kernels still produce good performance when considering early stopping. It means that the CNN with the Gabor kernel is more robust to the noise.

In this case, we adopt the early stopping and evaluate the CNNs. We show the predictions for inline #355 and #609 in Figures 7 and 8, respectively. The CNN with 3\(\times\)3 Conv kernel is the model trained for 36 epochs, the CNN with 11\(\times\)11 Conv kernel is trained for 32 epochs, and the CNN with 11\(\times\)11 Gabor kernel is trained for 50 epochs. We can again see that the CNN with the Gabor kernel is robust to noise.
Figure 4: The prediction results for inline #355 with 0.4dB Gaussian noise using the trained CNN models after 50 training epochs in Figure 2.

Figure 5: The prediction results for inline #375 with 0.4dB Gaussian noise using the trained CNN models after 50 training epochs in Figure 2.

Figure 6: The testing metric curves for the rest (validation) of the 380 inline profiles with -5.6 dB Gaussian noise, including (a)FWIU, (b)MCA, (c)Mean IU, (d)PA, and (e)loss, for the 3\(\times\)3 and 11\(\times\)11 Conv kernels and the 11\(\times\)11 Gabor kernels.

Figure 3: The testing metric curves for the rest (validation) of the 380 inline profiles with 0.4dB white Gaussian noise, including (a)FWIU, (b)MCA, (c)Mean IU, (d)PA, and (e)loss, for the 3\(\times\)3 and 11\(\times\)11 Conv kernels and the 11\(\times\)11 Gabor kernels.

Figure 7: The prediction results for inline #355 with -5.6 dB Gaussian noise using the trained CNN models with different epochs, for 3\(\times\)3 Conv kernel is 36 epoch, for 11\(\times\)11 Conv kernel is 32 epoch, and for 11\(\times\)11 Gabor kernel is 50 epoch.
### Constraining the Gabor parameters

As we know, the most obvious and important features in seismic imaging results, for interpretation, are textures, angles, and frequencies. As mentioned before, the beauty of the Gabor kernels is that we could control these parameters, as they define the Gabor basis function, to improve the performance of the CNN with Gabor kernels and also to avoid overfitting to some extent. Thus, in this section, we focus on tests where parameters of the Gabor kernel in CNN are constrained within a range. Specifically, we apply constraints on the angles and wavelengths of the Gabor basis function. Figure 10 shows the training metric curves of the CNN with the Gabor kernels under different constraints of angles using the training data containing 0.4 dB white Gaussian noise. The constraints admit slightly faster convergences than that without the constraints. Figure 11 is the corresponding test curves on the testing data containing -5.6 dB Gaussian noise. It is obvious that with the angle constraint, the CNN with Gabor kernels is much better than that without the constraint, avoiding overfitting to some extent. We further show in Figure 11 the predictions on the inline #333 after 50 training epochs. When we constrain the angle \(\theta\in[-\pi/6,\pi/6]\), for the facies denoted by the red color, the CNN with the Gabor kernel has a hard time identifying the class where the slope of the texture is large. As we increase the range of the angles in the Gabor kernel, the CNN could identify the red class with higher accuracy.

Figures 12 shows the training metric curves for the CNN with the Gabor kernels under different constraints applied to the wavelength using the training data containing 0.4 dB Gaussian noise. The CNN with a \([12,\infty]\) wavelength constraint, converges slightly faster. Figure 13 shows the corresponding test metric curves on the data containing -5.6 dB Gaussian noise. It is obvious that with the wavelength constraint, the CNN with Gabor kernels is more stable than that without the constraint and also avoids overfitting to some extent. We further show in Figure 14 the predictions on the inline #465 after 50 training epochs. When the minimum wavelength of the Gabor kernel increases, the predictions for the classes, where the imaging result contains long-wavelength components, are

Figure 8: The prediction results for inline #609 with -5.6 dB Gaussian noise using the trained CNN models with different epochs, for 3\(\times\)3 Conv kernel is 36 epoch, for 11\(\times\)11 Conv kernel is 32 epoch, and for 11\(\times\)11 Gabor kernel is 50 epoch.

Figure 10: The testing metric curves of the CNN with the Gabor kernels under different constraints of angles (no-constraints, \(\theta\in[-\pi/6,\pi/6]\), \([-\pi/4,\pi/4]\) and \([-\pi/3,\pi/3]\)) using the testing data containing -5.6 dB Gaussian noise.

Figure 9: The training metric curves of the CNN with the Gabor kernels under different constraints of angles (no-constraints, \(\theta\in[-\pi/6,\pi/6]\), \([-\pi/4,\pi/4]\) and \([-\pi/3,\pi/3]\)) using the training data containing 0.4 dB Gaussian noise.
improving. The application of constraints on particular Gabor parameters helps the training of the other Gabor parameters, as we reduce the search space. So, the improvements of the CNN with a constrained Gabor kernel are not only the contribution of the constrained parameters but also the other parameters.

Figure 11: The prediction results for inline #333 with -5.6 dB Gaussian noise using the trained CNN models with different angle constraints in the Gabor kernel after 50 epochs.

Figure 14: The prediction results for inline #465 with -5.6 dB white Gaussian noise using the trained CNN models with different wavelength constraints in the Gabor kernel after 50 epochs.

Figure 12: The training metric curves of the CNN with the Gabor kernels under different constraints of wavelengths (no-constraints, \(\lambda\in[4,+\infty)\), \([8,+\infty)\), \([12,+\infty)\)) using the training data containing 0.4 dB Gaussian noise.

Figure 13: The testing metric curves of the CNN with the Gabor kernels under different constraints of wavelengths (no-constraints, \(\lambda\in[4,+\infty)\), \([8,+\infty)\), \([12,+\infty)\)) using the testing data containing -5.6 dB Gaussian noise.
### Test on salt\(\&\)pepper and speckle noises

In this section, we test the robustness of the CNN with different kernels (3\(\times\)3 Conv kernel, 11\(\times\)11 Conv kernel, and 11\(\times\)11 Gabor kernel) on the different types of noise, e.g., salt\(\&\)pepper noise and speckle noise. For different kernels, the CNN models used here are the same as that in Figures 7 and 8. Figures 15 and 16 show the predictions on the imaging results with 30% salt \(\&\) pepper noise for inline #510 and #695. The CNN with the Gabor kernel obviously outperforms others. Table 1 is the quantitative evaluation results on the whole 3D volumes excluding the training profiles. The CNN with the Gabor kernel shows better performance compared to the CNN with Conv kernel.

As for the speckle noise, we add the noise by multiplying the mean noise with the image, where the mean and variance of the mean noise are 0 and 0.49. Then we test the models with different kernels. The model used here are the same as above. Figures 17 and 18 are the predictions for inline #388 and #557. We also show the quantitative evaluation on the whole 3D volume excluding the training profiles in Table 2. The metrics demonstrate that the CNN with the Gabor kernel is robust to the noise even dealing with the noise it has never seen during the training (The salt\(\&\)pepper and speckle noises can be regarded as out-of-distribution noise).
## 4 Discussion

The Gabor kernel layer added to the UNet CNN architecture in the first layer provides a favorable filter for input seismic images. With its five trainable parameters and functional form, it is inherently biased against noise. The Gabor kernel also allows us to constrain some of the Gabor parameters like wavelength and angle to further control the type of data we admit to the convolutional layers. So if certain angle ranges are preferred in the facies classification, we can emphasize these angles in the network by constraining the range. Though the parameters are still learned as part of the training of the network, the learned values will be within the specified range. The network also allows us to fix some of these parameters if needed. If a certain wavelength is preferred to isolate certain classes of facies, we can set that wavelength in the network and it is not learned.

Though we show the effectiveness of having a Gabor layer for a certain network Unet on a certain task, facies classification, we feel that the Gabor layer for seismic inputs is beneficial for any network, convolutional or not, including networks like ResNet and Vision Transformers. We also speculate that the benefits go beyond facies classification tasks to other seismic tasks like salt segmentation and horizon picking and even denoising. The message here is that Gabor functions are optimal basis functions for seismic data, and having them represent input seismic data should only help in the feature extraction of seismic data.

## 5 Conclusions

Here, we proposed a modified UNet with learnable Gabor convolutional kernels for facies classification. Because Gabor functions are suitable for representing seismic wavefields and images, the features extracted by the Gabor convolutional kernels adapt easily to the seismic texture. Meanwhile, since the Gabor parameters have recognizable meanings with respect to direction and wavelength, the CNN performance can be further improved by constraining these parameters based on our expectations of them in seismic images. The experiments on the Netherland F3 dataset show the effectiveness of the proposed method. The test on salt\(\&\)pepper and speckle noises further demonstrates the good generalization and robustness of the CNN with Gabor kernels in the first layer. In many practical exploration applications of deep learning, where we train the CNN model on limited data, adopting Gabor kernels in shallow layers promises high potential in improving the model's generalization.

## 6 Appendix

To evaluate the performance of CNN models with various kernels on the F3 dataset, we use several evaluation metrics. First, we define \(G_{i}\) as the set of pixels that belong to class \(i\), and \(F_{i}\) represents the set of pixels predicted as class \(i\). Then, the set of correctly predicted pixels is denoted by \(G_{i}\cap F_{i}\), and the number of elements in a set is extracted by the operator \(|\cdot|\). Thus, the evaluation metrics can be defined as follows:

* Pixel accuracy (PA) represents the percentage of pixels in the image correctly identified to the right class \[\mathrm{PA}=\frac{\sum_{i}|F_{i}\cap G_{i}|}{\sum_{i}|G_{i}|}\] (4) This metric could reflect the overall accuracy of the classification but may fail for the class with limited testing samples.

\begin{table}
\begin{tabular}{c|c|c|c|c|c|c|c|c} \hline \multirow{2}{*}{Metric Model} & \multirow{2}{*}{PA} & \multirow{2}{*}{MCA} & \multirow{2}{*}{FWIU} & \multicolumn{4}{c}{Class Accuracy} \\  & & & & Upper N.S. & Middle N. S. & Lower N.S. & Rijnland/Chalk & Scruff & Zechstein \\ \hline Conv 3\(\times\)3 & 0.619 & 0.302 & 0.408 & 0.354 & 0.164 & 0.998 & 0.181 & 0.114 & 0.000 \\ \hline Conv 11\(\times\)11 & 0.899 & 0.646 & 0.815 & 0.990 & 0.786 & 0.983 & 0.383 & 0.737 & 0.000 \\ \hline Gabor 11\(\times\)11 & **0.954** & **0.886** & **0.916** & **0.997** & **0.947** & 0.969 & 0.820 & 0.674 & 0.911 \\ \hline \end{tabular}
\end{table}
Table 2: The accuracy of prediction using different kernels with speckle noise
* Class accuracy (CA) represents the prediction accuracy for each class, and \(\mathrm{CA}_{i}\) represents the percentage of pixels correctly classified in a class \(i\) \[\mathrm{CA}_{i}=\frac{|F_{i}\cap G_{i}|}{|G_{i}|}\] (5) \(MCA\) is defined as the average of CA for all classes \[\mathrm{MCA}=\frac{1}{n_{c}}\sum_{i}\mathrm{CA}_{i}=\frac{1}{n_{c}}\sum_{i} \frac{|F_{i}\cap G_{i}|}{|G_{i}|},\] (6) where \(n_{c}\) is the number of classes. The high MAC only happens when the accuracy for each class reaches a high level.
* Intersection over union \((\mathrm{IU}_{i})\) is defined as the number of elements of the intersection of \(G_{i}\) and \(F_{i}\) over the number of elements of their union set \[\mathrm{IU}_{i}=\frac{|F_{i}\cap G_{i}|}{|F_{i}\cup G_{i}|}\] (7) This metric measures the overlap between the ground truth and the predictions, and higher scores mean better. It equals one only in case all pixels were correctly identified to the right class. The further metric averaging IU over all classes is defined as the mean intersection over union (Mean IU) \[\mathrm{Mean\ IU}=\frac{1}{n_{c}}\sum_{i}\mathrm{IU}_{i}=\frac{1}{n_{c}}\sum_{ i}\frac{|F_{i}\cap G_{i}|}{|F_{i}\cup G_{i}|}.\] (8) To avoid the bias of this metric for the imbalanced classes and sensitivity to the certain class (whose samples are few), the weighted version of the metric results in FWIU: \[\mathrm{FWIU}=\frac{1}{\sum_{i}|\mathcal{G}_{i}|}\cdot\sum_{i}|\mathcal{G}_{ i}|\cdot\frac{|\mathcal{F}_{i}\cup\mathcal{G}_{i}|}{|\mathcal{F}_{i}\cap \mathcal{G}_{i}|}.\] (9)

## Acknowledgment

The authors thank KAUST and the DeepWave Consortium sponsors for their support, and Xinquan Huang for useful discussion. We would also like to thank the SWAG group for the collaborative environment.
