Adaptive aggregation of Monte Carlo augmented decomposed filters for
  efficient group-equivariant convolutional neural network
Group-equivariant convolutional neural networks (G-CNN) heavily rely on
parameter sharing to increase CNN's data efficiency and performance. However,
the parameter-sharing strategy greatly increases the computational burden for
each added parameter, which hampers its application to deep neural network
models. In this paper, we address these problems by proposing a
non-parameter-sharing approach for group equivariant neural networks. The
proposed methods adaptively aggregate a diverse range of filters by a weighted
sum of stochastically augmented decomposed filters. We give theoretical proof
about how the continuous group convolution can be approximated by our methods.
Our method applies to both continuous and discrete groups, where the
augmentation is implemented using Monte Carlo sampling and bootstrap
resampling, respectively. We demonstrate that our methods serve as an efficient
extension of standard CNN. Experiments on group equivariance tests show how our
methods can achieve superior performance to parameter-sharing group equivariant
networks. Experiments on image classification and image denoising tasks show
that in certain scenarios, with a suitable set of filter bases, our method
helps improve the performance of standard CNNs and build efficient lightweight
image denoising networks. The code will be available at
https://github.com/ZhaoWenzhao/MCG_CNN.
Wenzhao Zhao, Barbara D. Wichtmann, Steffen Albert, Angelika Maurer, Frank G. Zöllner, Ulrike Attenberger, Jürgen Hesser
Adaptive aggregation of Monte Carlo augmented decomposed filters for efficient group-equivariant convolutional neural network

###### Abstract

Filter-decomposition-based group-equivariant convolutional neural networks (G-CNN) have been demonstrated to increase CNN's data efficiency and contribute to better interpretability and controllability of CNN models. However, so far filter-decomposition-based affine G-CNN methods rely on parameter sharing for achieving high parameter efficiency and suffer from a heavy computational burden. They also use a limited number of transformations and in particular ignore the shear transform in the application. In this paper, we address these problems by emphasizing the importance of the diversity of transformations. We propose a flexible and efficient strategy based on weighted filter-wise Monte Carlo sampling. In addition, we introduce shear equivariant CNN to address the highly sparse representations of natural images. We demonstrate that the proposed methods are intrinsically an efficient generalization of traditional CNNs, and we explain the advantage of bottleneck architectures used in the existing state-of-the-art CNN models such as ResNet, ResNet, and ConvNeXt from the group-equivariant perspective. Experiments on image classification and image denoising tasks show that with a set of suitable filter basis, our methods achieve superior performance to standard CNN with high data efficiency. The code will be available at [https://github.com/ZhaoWenzhao/MCG_CNN](https://github.com/ZhaoWenzhao/MCG_CNN).

 Group equivariance, convolutional neural network, Monte Carlo sampling, filter decomposition.

## I Introduction

Convolutional neural networks (CNNs) belong to one of the most widespread deep neural network architectures in computer vision. Its success originates from its "sliding window" strategy inspired by human vision [13][29], which shows a desirable property of translation equivariance. In recent years, a sheer amount of publications have emerged aiming at developing and applying more advanced group equivariant CNNs to improve CNN's sample efficiency and generalizability [27][19][34]. The concept of group equivariant CNN (GCNN) was first proposed by Cohen and Welling in [6], which exploited a higher degree of weight sharing by increasing the number of convolutional channels with the periodical rotation of the same convolutional kernel. This idea was further extended in [8] by introducing steerable filters which decomposed the convolutional kernel with an orthogonal basis of roto-reflection groups.

Following the work of rotation equivariant CNN, in recent years, there have been a lot of studies based on filter decomposition for exploring scale equivariant CNN [41][40][39][51], and scale-rotation equivariant CNN [14][19]. Attention mechanisms have been introduced in [38][19] to help better identify optimal filter banks and boost equivariance performance. The idea of group equivariance has also been introduced to transformer networks to improve the transformer's data efficiency. Apart from filter decomposition, more recently, the feature alignment has also proven to be helpful for improving CNN's group equivariance against affine image transforms [42].

The existing works for filter-decomposition-based group equivariant CNN all require increasing channel numbers to increase parameter sharing, which brings in a heavy computational burden [27] and hence hampers their practical application to natural images. Due to the computational burden needed for considering one kind of transform equivariance, the existing works of affine G-CNN are limited to transforms such as scaling, rotation, and reflection. So far, further including the shear transform is rarely considered in the conventional framework of affine G-CNN. In this paper, we propose an efficient implementation based on an adaptive aggregation of Monte Carlo augmented decomposed filters. The contribution of this paper is embodied in three aspects:

Our approach does not increase the computation burden and achieves high parameter and data efficiency compared with conventional CNNs.

In addition, thanks to the convenience of weighted Monte Carlo (MC) sampling in implementation, our work is able to consider a more flexible mix of different transforms, we thereby introduce shear transform and demonstrate its potential to improve networks' performance on natural images.

Our methods achieve superior performance to conventional CNN in both image classification and image denoising tasks.

The paper is organized as follows: In the Methods section, we review the general framework of the group-equivariant model and introduce the details of our approach. We show the experimental results and discussions in the Experiments section and conclude the paper in the Conclusion section.
## II Methods

### _The general framework of group-equivariant model_

Borrowing the concepts of [24], we will briefly introduce the definition of group equivariant mapping and group convolution. Although we constrain the discussion to a few transformation groups, the concept can be applied to any type of group and hence group equivariance. In particular, it applies to any dimension of the image space.

#### Ii-A1 Group equivariance

In this paper, we consider a group \(G\) for the affine transformations on 2D images \(\mathbb{R}^{2}\), which can be written as \(G=\mathbb{R}^{2}\rtimes\mathcal{A}\), a semidirect product between the translation group \(\mathbb{R}^{2}\) and another affine transform group \(\mathcal{A}\) (whose group element for 2D images takes the representation of a \(2\times 2\) matrix). Its group product rule is defined as

\[\begin{array}{l}g_{1}\bullet g_{2}=(x_{1},a_{1})\bullet(x_{2},a_{2})\\ =(x_{1}+M(a_{1})x_{2},a_{1}+a_{2}),\end{array} \tag{1}\]

where "\(\bullet\)" denotes the group product operator, \(g_{1}=(x_{1},a_{1})\), \(g_{2}=(x_{2},a_{2})\) with \(x_{1},x_{2}\in\mathbb{R}^{2}\), \(a_{1},a_{2}\in\mathbb{R}^{3}\), and function \(M:\mathbb{R}^{3}\rightarrow\mathcal{A}\). In this paper, we consider the following affine group, in particular, for any \(a=(\alpha,\sigma,s)\) with \(\alpha,\sigma,s\in\mathbb{R}\), \(M(a)=R(\theta)A(\alpha)S(s)\), where

\[S(s)=\begin{bmatrix}1&s\\ 0&1\end{bmatrix}, \tag{2}\]

\[A(\alpha)=\begin{bmatrix}2^{\alpha}&0\\ 0&2^{\alpha}\end{bmatrix}, \tag{3}\]

\[R(\theta)=\begin{bmatrix}\cos\theta&\sin\theta\\ -\sin\theta&\cos\theta\end{bmatrix}. \tag{4}\]

It should be noted that the existing works on affine G-CNN only consider translation, scaling, rotation, and mirror transforms. In this work, shear transform is included to form a more general case and explore its potential for boosting G-CNN's performance on natural images.

For a group element of the affine transformation group \(g\in G\), there is a corresponding group action on an index set \(\mathcal{X}\), i.e., a transformation \(T:G\times\mathcal{X}\rightarrow\mathcal{X}\). And for any \(g_{1},g_{2}\in G\) and \(x\in\mathcal{X}\), we have

\[T(g_{1}\bullet g_{2},x)=T(g_{1},T(g_{2},x)). \tag{5}\]

For any function \(f:\mathcal{X}\rightarrow\mathbb{C}\), we further define \(\mathbb{T}_{g}:f\to f^{\prime}\) where \(f^{\prime}(T(g,x))=f(x)\).

With the concept of group and group actions, we can now define the group equivariant map. Suppose we have a function \(f:\mathcal{X}\to V\) to be the input image or feature map of a neural network layer with \(V\) as a vector space. Let \(L_{V}(\mathcal{X})\) denote the Banach space of functions \(f:\mathcal{X}\to V\). Consider a map \(\phi:L_{V_{1}}(\mathcal{X}_{1})\to L_{V_{2}}(\mathcal{X}_{2})\) between two function spaces \(L_{V_{1}}(\mathcal{X}_{1}):\{f:\mathcal{X}_{1}\to V_{1}\}\) and \(L_{V_{2}}(\mathcal{X}_{2}):\{f:\mathcal{X}_{2}\to V_{2}\}\). For \(g\in G\), we have \(T_{g}\) and \(T_{g}^{\prime}\) to be G actions corresponding to set \(\mathcal{X}_{1}\) and \(\mathcal{X}_{2}\), as well as \(\mathbb{T}_{g}\) and \(\mathbb{T}_{g}^{\prime}\). The map \(\phi\) is group equivariant if and only if

\[\forall g\in G,\phi(\mathbb{T}_{g}(f))=\mathbb{T}_{g}^{\prime}(\phi(f)) \tag{6}\]

#### Ii-A2 Group convolution

A standard convolution of functions \(f\) with \(\psi\colon\mathbb{R}\rightarrow\mathbb{R}\) is a translation-equivariant map, which can be written as

\[(\psi\ast f)(x)=\int\psi(-x+x^{\prime})f(x^{\prime})dx^{\prime}, \tag{7}\]

Group convolution is a generalization of standard convolution by introducing the group operation. The group convolution [24][7][3][19] on a compact group \(G\) at group element \(g\) is written as

\[(\psi\ast f)(g)=\int_{G}\psi(g^{-1}\bullet g^{\prime})f(g^{\prime})d\mu(g^{ \prime}) \tag{8}\]

where \(\mu\) is the Haar measure, and \(f,\psi:G\rightarrow\mathbb{C}\). It should be noted that plain convolution is a special case of group convolution when only the translation group is considered (i.e., \(g^{-1}=-x\); \(g^{\prime}=x^{\prime}\) and the "\(\bullet\)" corresponds to "\(+\)"). [24] proved that the group convolution defined in the equation (8) is a group-equivariant map for affine transform groups.

### _Adaptive aggregation of Monte Carlo augmented decomposed filters_

In a discrete implementation of group convolution, the integral is usually implemented based on the trapezoidal rule [2] using evenly sampled group elements \(g^{\prime}\) in equation (8). For each input feature map channel (when considering many different kinds of affine transforms such as scaling, rotation, and mirror), nested integrals are needed, i.e. one nested integral per transform considered. By this, the approach increases the computation burden exponentially with the number of considered transforms leading to the curse of dimensionality [45]. For example, when we have \(m\) different elements per transform and \(n\) transforms, this amounts to \(m^{n}\) terms to be evaluated.

To improve the flexibility of group convolution for the general affine transform group and avoid the curse of dimensionality, in this work, we propose to approximate the multi-dimensional integral over group operations in the group convolution by MC integration.

#### Ii-B1 Monte Carlo integration

MC integration is known to tackle high-dimensional integration with robust convergence independent of the number of dimensions [45]. We consider for brevity only the standard MC variant, being aware that more efficient schemes such as Quasi-MC have the potential to substantially increase the performance further [5][30].

For multi-dimensional Monte Carlo integral, we have the theorem [36][25][23] as follows,

**Theorem II.1**.: _Let \(\mu_{p}\) be a probabilistic measure on \((\mathbb{R}^{d},\mathcal{B}(\mathbb{R}^{d}))\), i.e., \(\mu_{p}(\mathbb{R}^{d})=1\), and \(\mathcal{B}(\mathbb{R}^{d})\) denotes the Borel algebra on \(\mathbb{R}^{d}\) with \(d\) the number of dimensions. For \(f\in L^{2}(\mathbb{R}^{d},\mathcal{B}(\mathbb{R}^{d}),\mu_{p})\), we define_

\[I(f)=\int_{\mathbb{R}^{d}}f(x)d\mu_{p}(x), \tag{9}\]

_and_

\[Q_{N}(f)=\frac{1}{N}\sum_{i=1}^{N}f(\xi_{i}), \tag{10}\]
_where \((\xi_{i})_{i\in N}\) is an i.i.d sequence of random variables with distributions \(\mu_{p}\). We have \(Q_{N}(f)\to I(f)\) when \(N\to+\infty\). For all \(N\in\mathbb{N}\), there is_

\[(\mathbb{E}\|I(f)-Q_{N}(f)\|^{2})^{1/2}=\sigma(f)/\sqrt{N}, \tag{11}\]

_where \(\sigma^{2}(f)=I(f^{2})-(I(f))^{2}\), and \(\|\cdot\|\) is the \(l^{2}\) norm._

A finite non-zero Haar measure in (8) can be normalized to get a corresponding probabilistic measure \(\mu_{p}\). Therefore, it is theoretically justified to apply MC sampling for the discrete implementation of G-CNN.

#### Ii-B2 Discrete implementation of G-CNN with MC integration

In the discrete implementation, we stochastically sample the group operations including scaling, rotation, and shear transform. This approach allows a more flexible choice of the number of used transformations and decouples the relationship between the number of output channels and the number of categories of considered transformations.

Specifically, when we consider a filter \(W=w\cdot\psi\) with a fixed base filter \(\psi\) and \(w\) the trainable scalar weight, a continuous CNN layer can be written as

\[\begin{array}{l}f_{c_{o}}^{(l+1)}(x)=\sum_{c_{i}}w_{c_{o},c_{i}}^{(l)}(\psi *f_{c_{i}}^{(l)})(x)\\ =\sum_{c_{i}}\int_{\mathbb{R}^{2}}w_{c_{o},c_{i}}^{(l)}\psi(u-x)f_{c_{i}}^{(l) }(u)du\end{array} \tag{12}\]

A corresponding discrete implementation of the convolutional layer1 of \(l\)-th layer is as below

Footnote 1: It should be noted that in this paper, for simplicity, we omit point-wise nonlinearity functions, constant scalar coefficients, and normalization layers in neural networks, which do not affect the group equivariance [24].

\[f_{c_{o}}^{(l+1)}(x)=\sum_{c_{i}}\sum_{u}w_{c_{o},c_{i}}^{(l)}\psi(u-x)f_{c_{i }}^{(l)}(u) \tag{13}\]

where \(x,u\in\mathbb{R}^{2}\), \(\psi(\cdot)\) denotes the spatial convolutional filter function with a domain of translation group \(\mathbb{R}^{2}\), \(c_{i}\in[1,C_{l}]\) and \(c_{o}\in[1,C_{l+1}]\). \(f_{c_{i}}^{(l)}(x)\) is the feature map of the \(l\)-th layer and \(w_{c_{o},c_{i}}^{l}\) is the filter weight for filter of the \(l\)-th layer with output channel \(c_{o}\) and input channel \(c_{i}\).

A continuous affine group equivariant CNN can be written as

\[\begin{array}{l}f_{c_{o}}^{(l+1)}(g)=\sum_{c_{i}}w_{c_{o},c_{i}}^{(l)}(\psi *f_{c_{i}}^{(l)})(g)\\ =\sum_{c_{i}}\int_{G}w_{c_{o},c_{i}}^{(l)}\psi(g^{-1}\bullet g^{\prime})f_{c_{ i}}^{(l)}(g^{\prime})d\mu(g^{\prime})\end{array} \tag{14}\]

For simplicity, in the following part of this paper, we denote \(f(x)\) a function with domain on \(\mathbb{R}^{2}\), and we denote the corresponding function with domain on group \(G\) as \(f(g)=f(x,a)\) with \(x\in\mathbb{R}^{2}\) the spatial position, and \(a\in\mathbb{R}^{3}\) the transform parameter vector for affine transform group.

Let \(g=(x,a)\) and \(g^{\prime}=(u,b)\), we can rewrite the Haar integration in group convolution of the \(l\)-th layer as:

\[\begin{array}{l}f_{c_{o}}^{(l+1)}(x,a)=\sum_{c_{i}}\int_{\mathbb{R}^{3}}\int _{\mathbb{R}^{2}}w_{c_{o},c_{i}}^{(l)}2^{-2\alpha_{a_{a}}}\\ \psi(-x+M(-a)u,-a+b)f_{c_{i}}^{(l)}(u,b)dudb\end{array} \tag{15}\]

where we have the transform parameter vectors \(a=[\alpha_{a},\theta_{a},s_{a}]\), and \(b=[\alpha_{b},\theta_{b},s_{b}]\).

A typical corresponding discrete G-CNN can be written as below:

\[\begin{array}{l}f_{c_{o}}^{(l+1)}(x,a)=\sum_{c_{i}}\sum_{b}\sum_{u}w_{c_{o},c_{i}}^{(l)}2^{-2\alpha_{a_{a}}}\\ \psi(-x+M(a)u,-a+b)f_{c_{i}}^{(l)}(u,b)\end{array} \tag{16}\]

In particular, the sum over the parameter vector \(b\) is a three-layer nested sum corresponding to the nested integrals in the continuous domain, which, as mentioned in previous sections, leads to a heavy computational burden.

The Monte-Carlo integration considers \(a\) and \(b\) as random variables. Suppose their entries \(\alpha=\xi_{\alpha}\), \(\theta=\xi_{\theta}\), and \(s=\tan(\xi_{s})\), where \(\xi_{\alpha}\), \(\xi_{\theta}\) and \(\xi_{s}\) are uniformly distributed in the range of \([\eta_{\alpha}^{1},\eta_{\alpha}^{2})\), \([-\eta_{\theta},\eta_{\theta})\), and \([-\eta_{s},\eta_{s})\), respectively.

Suppose we draw \(N^{\prime}\) samples of \(a\), and \(N\) samples of \(b\), respectively. The nested sum over \(b\) collapses into a one-dimension sum over \(N\) samples for MCG-CNN (Monte Carlo Group-equivariant CNN):

\[\begin{array}{l}f_{c_{o}}^{(l+1)}(x,a_{n})=\sum_{c_{i}}\sum_{n}\sum_{u}w_{c_{o },c_{i}}^{(l)}2^{-2\alpha_{a_{a_{a^{\prime}}}}}\\ \psi(-x+M(-a_{n^{\prime}})u,-a_{n^{\prime}}+b_{n})f_{c_{i}}^{(l)}(u,b_{n}) \end{array} \tag{17}\]

where \(n^{\prime}\in\{1,\ldots,N^{\prime}\}\), and \(n\in\{1,\ldots,N\}\).

#### Ii-B3 Adaptive aggregation of MC-augmented filters

The Monte-Carlo approximation of G-CNN allows a flexible choice of the number of sampling points \(N\) per trainable weight \(w^{(l)}\) independent of the number of dimensions. However, compared with standard CNN, the computational burden of MCG-CNN is still \(N\) times larger. To eliminate the difference in computational burden between MCG-CNN and standard CNN, we propose WMCG-CNN (Weighted Monte Carlo Group-equivariant CNN)2, which reduces the number of transformations per input feature map channel (also per trainable weight) \(N\) to \(1\) and uses filter-weight-wise sampling instead. Specifically, we establish a one-to-one relationship between \(b\), \(c_{o}\) and \(c_{i}\), as well as \(a\) and \(c_{o}\) by using \(c_{o}\) and \(c_{i}\) to index \(a\) and \(b\). Thus we introduce notation \(b_{c_{o},c_{i}}\) and \(a_{c_{o}}\).

Footnote 2: The word “Weighted” in WMCG-CNN is used to emphasize that the number of trainable filter weights becomes transformation-wise in WMCG-CNN, which is thus an adaptive aggregation of augmented filters.

In this way, we yield WMCG-CNN with the equation (17) simplified into:

\[\begin{array}{l}f_{c_{o}}^{(l+1)}(x,a_{c_{o}})=\sum_{c_{i}}\sum_{u}w_{c_{o},c _{i}}^{(l)}2^{-2\alpha_{a_{c_{o}}}}\\ \psi(-x+M(-a_{c_{o}})u,-a_{c_{o}}+b_{c_{o},c_{i}})f_{c_{i}}^{(l)}(u,b_{c_{o},c_{ i}}),\end{array} \tag{18}\]

WMCG-CNN allows us to significantly increase the number of used transformations without increasing the computational burden, which, as shown in the later experiments, helps WMCG-CNN achieve superior performance to traditional discrete G-CNN.

However, due to the changes happening to WMCG-CNN, a question arises, i.e., under which circumstances, the WMCG-CNN can still be analogous to continuous G-CNN as the discrete G-CNN does? Below, we show that random initialization of the trainable weights can help the WMCG-CNN to be analogous to continuous G-CNN.

**Theorem II.2**.: _Let \(f^{(l)}\) be an input feature map of the \(l\)-th layer with the number of channels \(C_{l}\), and for each channel the number of spatial sampling points along vertical direction \(N_{H}\), the number of spatial sampling points along horizontal direction \(N_{W}\). A WMCG-CNN layer is group equivariant when the width of CNN, \(C_{l}\to\infty\), \(N_{H}\to\infty\), \(N_{W}\to\infty\), and
\(\|\int_{\mathbb{R}}wd\mu_{w}(w)\|<+\infty\) with \(\mu_{w}\) a probabilistic measure on \((\mathbb{R},\mathcal{B}(\mathbb{R}))\) for the filter weight \(w\), being a random variable._

Proof.: To prove the theorem, we have two steps: first, we construct a weighted integration function \(I\) and prove it is group equivariant. Then, we show that equation (18) corresponds to the discrete form of \(I\).

1) Given \(g=(x,a_{c_{o}})\) and \(g^{\prime}=(u,b)\), we define the integration on \(\mathbb{R}\times G\) as

\[\begin{array}{l}I(x,a_{c_{o}})\\ =\int_{\mathbb{R}\times G}w\cdot\psi(g^{-1}\bullet g^{\prime})f^{(l)}(g^{ \prime})d\mu(g^{\prime})d\mu_{w}(w)\\ =\int_{\mathbb{R}}\int_{\mathbb{R}^{3}}\int_{\mathbb{R}^{2}}w\psi(-x+M(-a_{c_{ o}})u,-a_{c_{o}}+b)\\ f^{(l)}(u,b)dudbdw\end{array} \tag{19}\]

Since \(\|\int_{\mathbb{R}}wd\mu_{w}(w)\|<+\infty\), we have the constant \(C=\int_{\mathbb{R}}wd(w)\). Thus

\[\begin{array}{l}I(x,a_{c_{o}})=C\cdot\int_{\mathbb{R}^{3}}\int_{\mathbb{R}^{ 2}}\psi(-x+M(-a_{c_{o}})u,-a_{c_{o}}+b)\\ f^{(l)}(u,b)dudb\end{array} \tag{20}\]

which is group equivariant.

2) Let \(q(x,a_{c_{o}},b)=\int_{\mathbb{R}^{2}}\psi(-x+M(-a_{c_{o}})u,-a_{c_{o}}+b)f^{( l)}(u,b)du\), so we have

\[I(x,a_{c_{o}})=\int_{\mathbb{R}}\int_{\mathbb{R}^{3}}wq(x,a_{c_{o}},b)dbdw \tag{21}\]

Now, we consider the transition from continuous to discrete formulations. Since both \(w\) and \(b\) are independently randomly sampled with the samples indexed by \(c_{i}\). According to Theorem II.1, we have

\[\begin{array}{l}I(x,a_{c_{o}})=\lim_{C_{l}\rightarrow\infty}\frac{1}{C_{l}} \sum_{c_{i}}w^{(l)}_{c_{o},c_{i}}q(x,a_{c_{o}},b_{c_{o},c_{i}})\end{array} \tag{22}\]

Since \(u\) is sampled based on the trapezoidal rule, we have

\[\begin{array}{l}q(x,a_{c_{o}},b_{c_{o},c_{i}})\\ =\lim_{N_{H}\rightarrow+\infty}\lim_{N_{H}\rightarrow+\infty}\frac{1}{N_{H}N_ {W}}\sum_{u}2^{-2\alpha_{ac_{o}}}\cdot\\ \psi(-x+M(-a_{c_{o}})u,-a_{c_{o}}+b_{c_{o},c_{i}})f^{(l)}_{c_{i}}(u,b_{c_{o},c _{i}}),\end{array} \tag{23}\]

Meanwhile, we rewrite the corresponding convolution part of WMCG-CNN equation (18) as

\[\begin{array}{l}f^{(l+1)}_{c_{o}}(x,a_{c_{o}})=\frac{1}{C_{l}N_{H}N_{W}}\sum _{c_{i}}\sum_{u}w^{(l)}_{c_{o},c_{i}}2^{-2\alpha_{ac_{o}}}\cdot\\ \psi(-x+M(-a_{c_{o}})u,-a_{c_{o}}+b_{c_{o},c_{i}})f^{(l)}_{c_{i}}(u,b_{c_{o},c _{i}}),\end{array} \tag{24}\]

where \(c_{i}\in\{1,2,\ldots,C_{l}\}\), \(u=(u_{1},u_{2})\) with \(u_{1}\in\{1,2,\ldots,N_{H}\}\) and \(u_{2}\in\{1,2,\ldots,N_{W}\}\). Here we include coefficient \(\frac{1}{C_{l}N_{H}N_{W}}\) so that \(f^{(l+1)}_{c_{o}}\) is the average of the samples.

Therefore, by combining (22) and (23), we have

\[\begin{array}{l}I(x,a_{c_{o}})\\ =\lim_{C_{l}\rightarrow\infty}\lim_{N_{H}\rightarrow+\infty}\lim_{N_{W} \rightarrow+\infty}f^{(l+1)}_{c_{o}}(x,a_{c_{o}})\end{array} \tag{25}\]

The proof is completed. 

As we know, random initialization of trainable weights is a common strategy adopted in most existing state-of-the-art deep learning methods. Theorem II.2 proves that the random weight initialization strategy together with the MC-augmented filters can help raise the CNN to a good starting point before training with an optimization algorithm, which therefore makes it easier for the network to find the optimal solution. This starting point is a network that approximately satisfies convolutional-layer-wise group equivariance. Obviously, a necessary condition of an optimal solution is that in contrast to the approximate convolutional-layer-wise group equivariance, it is at least at the level of the entire neural network that the group equivariance is achieved approximately.

From Theorem II.1, we know that the convergence speed of the Monte Carlo integration is slow. When the number of samples are small, the variance may not be satisfactory. However, with the weight \(w\) as learnable parameters and the samples of transformations fixed, the neural network can learn optimal weight distribution to improve the group equivariance, which will be shown in the later experiments (Fig. 2). Such sampling mechanism is thereby similar to that of importance sampling [16]. The difference is that the weight distribution in WMCG-CNN is not manually designed but is learned by iterative data-driven optimization algorithms for neural networks instead.

#### Ii-B4 Filter decomposition and the relationship to traditional CNN filters

In the previous section, we only consider one basis filter function \(\psi\), to increase the expressiveness of networks, we adopt the filter decomposition approach to build convolution filters by the sum of multiple weighted filter basis. Specifically, we have \(W^{(l)}_{c_{o},c_{i}}(x,a)=\sum_{j}w^{(l)}_{c_{o},c_{i},j}\tilde{\psi}_{j}(x,a)\) with \(\tilde{\psi}_{j}(x,a)\) an orthogonal basis function with \(x\in\mathbb{R}^{2}\) and \(a\in\mathbb{R}^{3}\) the transform parameter vector, \(w^{(l)}_{c_{o},c_{i},j}\) the trainable weights, \(j\in[1,K]\), and \(K\) the chosen number of basis functions. In the proposed WMCG-CNN, according to equation (18) the WMCG-CNN can be written in a similar way to the standard CNN in equation (13) as below:

\[\begin{array}{l}f^{(l+1)}(c_{o},x,a_{c_{o}})=\sum_{c_{i}}\sum_{u}2^{-2\alpha _{ac_{o}}}W^{(l)}_{c_{o},c_{i}}(\\ -x+M(-a_{c_{o}})u,-a_{c_{o}}+b_{c_{o},c_{i}})f^{(l)}_{c_{i}}(u,b_{c_{o},c_{i}}), \end{array} \tag{26}\]

In the practical discrete implementation, we adopt the Fourier-Bessel (FB) basis [37]. As in the previous section, the scaling, rotation, and shear transformations are used to augment the filters. Supposing any filter basis is a matrix of size \(k\times k\), for FB basis, we can have \(k^{2}-1\) non-constant basis and a constant scalar basis at the most.

It should be noted that the choice of basis for filter decomposition can also be flexible. When using the basis consisting of translation-augmented discrete Dirac delta functions, the proposed methods fall back into standard CNN filters.

### _Integrating WMCG-CNN into the existing state-of-the-art CNN architectures_

We see that when \(\tilde{\psi}_{j}\) degenerates to a scalar, i.e. a \(1\times 1\) base filter, the convolution is obviously exactly group equivariant, while on the other hand, the non-scalar filter \(\tilde{\psi}_{j}\) requires a huge number of sampling points to approximate the continuous G-CNN. To leverage the advantage of \(1\times 1\) base filter, one can add \(1\times 1\)-filter-based convolution layers as a secondary adaptive aggregation of features from the output channels of WMCG-CNN. By combining the \(1\times 1\) layer with the \(k\times k\) convolution layer into a single unit or block, the total number of considered transformations is increased from \(C_{l}\) to
(i.e., the number of all the \(k\times k\) filters used in the \(l\)-th layer) with a relatively small increase of parameter number. In addition, the \(1\times 1\) CNN layer also helps enrich the design space for WMCG-CNN, where the use of the small \(1\times 1\) kernel helps achieve high parameter efficiency given the same level of expressiveness and the same number of parameters [18].

Interestingly, the secondary aggregation with cascaded \(1\times 1\) convolutional layer is intrinsically similar to the bottleneck architecture that is adopted in all the state-of-the-art CNNs derived from ResNet [18]. The only difference is that the bottleneck architecture uses one extra \(1\times 1\) convolution layer before the \(k\times k\) convolution layer.

Apart from \(1\times 1\) layers, we also note that the channel grouping convolution technique3 proposed in ResNeXt [46] is also a helpful technique for improving CNN's performance.

Footnote 3: It should be noted that here the channel group is a concept that is different from the transformation group. The channel grouping convolution technique divides the input feature map channels into multiple channel groups of the same width to perform convolution operations separately.

Thanks to the flexibility of the proposed WMCG-CNN, we can easily combine these techniques with the WMCG-CNN. An example is shown in Fig. 1. Similar blocks but with different filter sizes will be used in the later experiments for image denoising.

## III Experiments

We test WMCG-CNN on both image classification and image denoising tasks. The ablation experiments are also conducted in image classification tasks.

### _Performance metrics_

We adopt the following performance metrics: the number of trainable parameters in million (\(10^{6}\)), Params(M); the number of Multiply-Accumulate Operations in giga (\(10^{9}\)), MACs(G); the prediction error in percentage, Error(%); mean prediction error on corrupted validation image datasets in percentage, mCE(%); top 1 accuracy in percentage, top-1 acc.(%); top 5 accuracy in percentage, top-5 acc.(%); peak signal-to-noise ratio in dB, PSNR(dB).

In addition, for the section of the ablation experiments, we define mean group-equivariant error (mGE) according to equation (6):

\[mGE=\mathbb{E}(\|\phi(\mathbb{T}_{g}(f))-\mathbb{T}_{g}^{\prime}(\phi(f))\|) \tag{27}\]

where for each input image, a random affine transformation \(g\in G\) is selected with the shear range \([-0.0625\pi,0.0625\pi)\), the scaling range \([1.0,1.1)\) and rotation angle range \([-0.125\pi,0.125\pi)\).

### _Ablation experiments_

For ablation experiments, we consider a subset of the ImageNet1k dataset. ImageNet1k has \(1.28\) million color images for \(1{,}000\) different classes from WordNet. The validation dataset consists of \(50{,}000\) images. For quick experiments, we extract the first \(40\) classes for ablation experiments (i.e., from class \(n01440764\) to class \(n01677366\)), and thus we denote the corresponding datasets as ImageNet40. We scale all the images to \(224\times 224\) resolution and normalize images in a classic way. The prediction Error (%) is used to measure the classification performance.

We use ResNet18, ResNet50, and ResNeXt50 [46] as the baseline networks. We follow the state-of-the-art robust training methods as in [22]. The neural networks are trained for \(90\) epochs with an initial learning rate of \(0.01\) following a cosine decay schedule. The Pixmix augmentation technique is used with its default setting as in [22]. Pixmix uses affine transformations including translation, rotation, and shear transform as well as other augmentation methods to generate augmented clean images. As for WMCG-CNN, we replace all the hidden non-\(1\times 1\) CNN layers with the proposed WMCG-CNN layers. By default, the size of FB basis is \(5\times 5\), the number of basis per filter is \(9\) (Bessel filter of order from \(0\) to \(8\)), the scaling range is \([1.0,2.0)\), the rotation angle range \([-2\pi,2\pi)\), and the shear transform angle range \([-0.25\pi,0.25\pi)\). For simplicity, we name each version of the tested networks with a series of suffixes. Specifically, "kn" means the filter size is \(n\times n\). In the experiments with shear transforms, we use the suffix "shear-\(n\pi\)" to denote shear transform angle \([-n_{s}\pi,n_{s}\pi)\). We change the value of \(n\) from \(0.00\) to \(0.40\pi\) to test the effect of the shear transform. In particular, \(n_{s}=0.00\) means that there is no shear transform applied. In addition, with ResNet18 as a baseline network, we also tested the conventional scale-equivariant CNN, and the proposed MC scale-equivariant CNN. The suffix "scale-\(n\)" means \(n\) scaling transformation \(\alpha=\{0,1/n,2/n,\ldots,(n-1)/n,1\}\) are used. The suffix "MC-scale-\(n\)" means \(n\) MC-augmented scaling transformations are used. The suffix "MC-affine-\(n\)" means \(n\) affine (scaling-rotation-shear) transformations are used. For the implementation of "scale-\(n\)", "MC-scale-\(n\)", and "MC-affine-\(n\)", we draw \(n\) samples of transformation for the input feature

Fig. 1: Integrating the proposed WMCG-CNN into the classic bottleneck architecture. (a) The example bottleneck block with group convolution using \(3\times 3\) filters; (b) An example of filter composition with MC-augmented basis.
map, and we only have \(1\) sample of transformation for the corresponding output feature map to avoid the computational burden from becoming too heavy. "width1/\(n\)" means that the total widths of output feature maps are reduced to \(1/n\) by decreasing the number of channels per transformation. "nb \(n\)" means \(n\) basis used per filter. "nb1-\(n\)" means only Bessel basis of order \(n\) is used.

Figure (a)a shows the mGE results for the the first hidden CNN layer of ResNet18 and ResNet18-k5-WMCG-shear-0.25\(\pi\) for the first 10-epoch traning on ImageNet dataset. We see that compared with ResNet18, WMCG network starts from a lower mGE but continues to converge smoothly. Figure (b)b shows that the distribution of the learned weights are centered around zero.

Table I shows the results of ablation experiments on ImageNet40, where the results with respect to Params(M) and MACs(G) are also displayed. We see that the shear transform with a suitable range shear angle is helpful for increasing WMCG-CNN's performance. In all the following experiments, we adopt \(n_{s}=0.25\) by default if not explicitly stated.

About the results with different versions of ResNet18-WMCG-k5-nb1, we see the choice of FB basis affects the prediction performance significantly. Low-frequency basis, i.e., Bessel basis of low order, is shown to be more important than high-frequency basis. Therefore, to select a fixed number of basis, we must include the low-order Bessel basis first.

The conventional scale-equivariant CNN architecture ResNet18-k5-scale-4 has a decent prediction error. But the computational burden is extremely high. When we try to reduce the computational burden by decreasing the width of the network to get ResNet18-k5-scale-4-width1/4, the number of trainable parameters is reduced significantly at the same time, which leads to poorer prediction performance. The MCG-CNN also has a heavy computational burden and is superior to its corresponding G-CNN when we use a larger number of transformations and more transformation types (such as ResNet18-k5-MC-affine-16-width1/16).

Among the tested ResNet baseline architectures, the results with ResNet18 give the lowest mean error, which indicates that the deeper models such as ResNet50 and ResNeXt50 suffer from overfitting because the number of classes is reduced from 1k to 40. However, the WMCG-CNN is able to reduce the overfitting consistently for all the considered baseline models. WMCG-CNN versions of ResNet18 yield the best classification performance. Generally, the results on ImageNet40 demonstrate that WMCG-CNN is superior to standard CNN in sample efficiency, helps avoid overfitting, and enables a quicker convergence.

### _Experiments on multiple image classification benchmark datasets_

In this section, we test the proposed method on Cifar-10 [26] and ImageNet1k datasets. The Cifar-10 dataset consists of color images of size \(32\times 32\times 3\) for 10 classes. There are \(50{,}000\) training images and \(10{,}000\) testing images. In addition, we use Cifar10-C and ImageNet1k-C [20] validation datasets to test neural networks' robustness and generalizability against image corruptions, where 15 diverse corruption types [20] are included for both the Cifar10-C and ImageNet1k-C validation datasets. Two kinds of training routine are used: robust training strategies with affine transform augmentation included, and the state-of-the-art fully-training strategy for comparison with ConvNeXt [32].

Fig. 2: (a) The mGE of the first hidden CNN layer of ResNet18 and ResNet18-k5-WMCG-shear-0.25\(\pi\) for the first 10-epoch traning on ImageNet dataset. (b) The histogram of the learned weights for the FB basis of order \(0\) in the first hidden CNN layer of ResNet18-k5-WMCG-shear-0.25\(\pi\).
For experiments on Cifar10 dataset [26], and Cifar10-C [20] datasets, we use ResNeXt29 (\(32\times 4\)) [46] as the baseline network and the Augmix-based [21] robust training strategy. We denote "ResNeXt29-k3-WMCG-nb9" as the network created by replacing the \(3\times 3\) convolution layer with WMCG CNN of the \(3\times 3\) FB basis size and each convolutional filter using \(9\) basis. We denote "ResNeXt29-k3-WMCG-nb1-r2" and "ResNeXt29-k5-WMCG-nb1-r2" as the networks that have a similar WMCG CNN layer but use only one FB basis of size \(3\times 3\) and \(5\times 5\), respectively. And both of the FB basis are of order \(2\). Empirically, only for the experiments with Cifar10 dataset, we use scaling range \([1.0,1.5)\), while in other experiments we keep \([1.0,2.0)\). All the CNN are trained with the same training strategy as in [21]. Specifically, all the networks are trained using an initial learning rate of \(0.1\) and a cosine learning rate schedule. The optimizer uses stochastic gradient descent with Nesterov momentum and a weight decay of \(0.0005\). The input images are first pre-augmented with standard random left-right flipping and cropping, and then the Augmix method [21] is applied with its default settings. Augmix uses affine transformations including translation, rotation, and shear transform as well as other augmentation methods to generate augmented clean images.

As for experiments on ImageNet1k [11], and ImageNet1k-C [20] datasets, we use ResNeXt50 [46] as the baseline network for the Pixmix-based [22] robust training. We denote "ResNeXt50-k5-WMCG-nb9" as the network created by replacing the \(3\times 3\) convolution layer with WMCG-CNN of the \(5\times 5\) FB basis size and each convolutional filter using \(9\) basis. The neural networks are trained with the same strategy in Pixmix [22]. All the neural networks are trained from scratch to compare the sample efficiency and convergence speed of different networks.

In addition, we test our methods with the recently proposed ConvNeXt network model [32] on ImageNet40 and ImageNet1k datasets. We use ConvNeXt-S as the baseline network. We denote "ConvNeXt-S-k7-WMCG-nb49" as the network created by replacing all the \(7\times 7\) convolution layer with WMCG-CNN of the \(7\times 7\) FB basis size and each convolutional filter using \(49\) basis. The training on both datasets is in the same way as described in [32], where the neural networks are trained for 300 epochs using an AdamW optimizer. Similar to [32], the top 1 and top 5 accuracies are considered.

Table II shows all the results for our image classification experiments. We see that under the robust training strategies, the proposed WMCG-CNNs reduce the classification errors on both clean and corrupted datasets while using the same or smaller number of parameters. The augmented FB basis of order \(2\) alone achieves the highest robustness with less number of parameters, which is partly because the FB basis of order \(2\) used in "ResNeXt29-k3-WMCG-nb1-r2" and "ResNeXt29-k5-WMCG-nb1-r2" relatively considers more low-frequency signals than FB basis of higher orders that are included in "ResNeXt29-k3-WMCG-nb9". It is also noted that a large filter size can help increase the classification precision and robustness of neural networks. As for the experiment with ConvNeXt, WMCG-CNN improves ConvNeXt-S on both ImageNet40 and ImageNet1k datasets without increasing the number of parameters as well as computational burden. It is also noted that shear transform is also helpful for performance boost under the \(300\)-epoch fully-training routine.

### _Experiments on image denoising_

Although it has been shown that in certain cases with known noise levels, the traditional algorithms can surpass CNNs in denoising quality [50][49], their processing speed is much slower than CNNs. And blind denoising with unknown noise levels is also a more practical scenario in the application. Thus in this paper, we only test the CNNs' performance on blind denoising tasks.

The experiments are divided into three parts: grayscale synthetic additive Gaussian noisy image denoising, color synthetic additive Gaussian noisy image denoising, and real-world color noisy image denoising (whose image noise is generated in the camera imaging process). For grayscale image denoising, as in [47], the same \(400\)\(180\times 180\) images are used for training. The training images are corrupted by synthetic additive Gaussian noise of noise level (i.e., the standard deviation of noise) \(\sigma\in[0,55]\). \(128\times 3\),\(000\) patches of size \(50\times 50\) are cropped to train the CNN model. For color synthetic noisy image denoising, we follow [43], where the same \(400\) color images are augmented with Bicubic downscaling, counterclockwise rotation, and horizontal flip. As for real-world noisy images, as in [43], the training dataset consists of \(100\)\(512\times 512\) JPEG images collected from five digital cameras Canon 80D, Nikon D800, Canon 600D, Sony A7 II and Canon 5D Mark II with ISO of 800, \(1\),\(600\), \(3\),\(200\), \(6\),\(400\), \(12\),\(800\) and \(25\),\(600\).

Five public test datasets are considered, including the grayscale image datasets Set12 [31], BSD68 [31], the color image datasets CBSD68 [31], Kodak24 [12], and the public real noisy consumer camera image dataset CC [35]. The public CC dataset consists of 15 images that are captured by three different digital cameras: Canon 5D Mark III, Nikon D600, and Nikon D800 with ISO values of \(1\),\(600\), \(3\),\(200\), or \(6\),\(400\). The training images are cropped into \(41\times 41\) patches for training the networks.

We consider one of the most famous denoising CNNs, DnCNN-B [4][47] as the baseline network for experiments on gray-scale image denoising. We build a brand new denoising
network called DnNeXt-B by replacing every plain hidden CNN layer in DnCNN-B with the bottleneck block shown in Fig. 1(b). We further denote "DnNeXt-B-k5-WMCG-nb9" as the network created by replacing the hidden \(3\times 3\) convolution layer in DnNeXt-B with WMCG-CNN of the \(5\times 5\) FB basis size and each convolutional filter decomposed by \(9\) basis. Likewise, "DnNeXt-B-k7-WMCG-nb9" is a corresponding version with FB basis of size \(7\times 7\). To emphasize the efficiency of our approach, we also include another Wavelet-based denoising CNN, MWDCNN [44] for comparison. We test all the CNNs on the standard grayscale image datasets Set12 [31], and BSD68 [31]. The DnCNN, DnNeXt, and DnNeXt-WMCG are trained with the same training strategy as in [47]. We use SGD optimizer with a weight decay of \(0.0001\), and a momentum of \(0.9\). The networks are trained for 50 epochs with a batch size of 128. During the 50 epochs of training, the learning rate decreases exponentially from \(1.0\times 10^{-1}\) to \(1.0\times 10^{-4}\).

Table III shows the denoising results with the metric of peak signal-to-noise ratio (PSNR) on images corrupted by simulated white Gaussian noise of different noise levels. The number of trainable parameters and MACs are also displayed. In particular, for all the calculations of MACs in image-denoising experiments, we assume the input patch size is \(3\times 32\times 32\) for a fair comparison of computational burden, which is different from the actual case. We find that the proposed DnNeXt and DnNeXt-MCG outperform DnCNN and MWDCNN with a much smaller number of learnable parameters. In addition, the proposed DnNeXt-WMCG achieves the highest average PSNR of all CNNs and yields especially higher PSNR on high noise levels. The larger FB basis helps gain a higher PSNR score on high noise levels, yet may cause poor performance on low noise levels.

We consider DudeNet [43], an upgrading of DnCNN as the baseline CNN for the synthetic color noisy image denoising and real camera image denoising experiment. We build a new network DudeNeXt by replacing every plain hidden \(3\times 3\) CNN layer in DudeNet with the bottleneck block shown in Fig. 1(b). We further denote "DudeNeXt-k5-WMCG-nb9" as the network created by replacing the hidden \(3\times 3\) convolution layer in DudeNeXt with WMCG-CNN of the \(5\times 5\) FB basis size and each convolutional filter decomposed by \(9\) basis. We follow the same training strategy as in [43]. We use Adam optimizer with an initial learning rate of \(1.0\times 10^{-3}\) and a batch size of 128. The networks are trained for 70 epochs. During the 70 epochs of training, the learning rate decreases exponentially from \(1.0\times 10^{-3}\) to \(1.0\times 10^{-5}\).We compare our methods with two conventional denoising algorithms CBM3D [10], TID [33], as well as three deep learning methods DnCNN [47], DudeNet [43], and MWDCNN [44].

Table IV shows the average PSNR results on the public CBSD68 and Kodak24 color image datasets. Table V shows the PSNR results on the public CC dataset. On both synthetic and real-world color image denoising experiments, generally, the proposed networks achieve superior performance with respect to the average PSNR.

### _Analysis and discussion_

The ablation experiments on ImageNet40 demonstrate the sample efficiency of WMCG-CNN for all the tested baseline network architectures including ResNet18, ResNet50, and ResNeXt50. We note that the proposed method gives a larger improvement in Error for ResNet18 and ResNet50 than that for ResNeXt50. This is probably because a larger proportion of learnable parameters in ResNeXt50 lies in \(1\times 1\) Conv layers which as shown in the results causes heavy overfitting on the small dataset ImageNet40.

The comparison experiments with discrete G-CNN, MCG-CNN, and WMCG-CNN proves that the diversity of transformations is helpful for performance boost. The introduction of MC sampling allows us to consider any mix of affine transforms. In the experiments on ImageNet40, we see that the additional use of shear transform with a suitable shear range can consistently improve image classification. Meanwhile, a high degree of shear transform can harm the performance, which is because, in discrete implementation, shear transform leads to compressing of information along a certain direction that causes information loss.

The shear-transform-augmented convolutional filters can be considered as an example of the classic continuous shear wavelet [1][17]. The shear wavelet can help achieve a highly sparse representation of multidimensional data [17], which explains the superior performance it brings to the proposed WMCG-CNN. In the future, we may exploit Wavelet theory to further improve our methods.

We also note that in the field of MC integral and stochastic simulation, there are a lot of advanced techniques such as quasi-MC sampling [5], Markov chain MC [9], and multi-level MC [15]. There is a potential that these methods can help improve both MCG-CNN and WMCG-CNN further, and we will study this in future work.

In this work, we do not compare with peer filter-decomposition-based G-CNNs proposed in other papers on multiple benchmark datasets. This is because as far as we know, all the existing filter-decomposition-based G-CNNs are much slower than standard CNN, and require larger GPU memory storage. They are typically tested with small datasets such as MNIST [28]. Due to the high degree of parameter sharing and a large number of channels used, those G-CNN can achieve a good inference accuracy but are usually unsuitable and over-expensive for practical application on natural image datasets.

The results of experiments on robust image classification and image denoising show the generalizability of WMCG-CNN. On Cifar10 and Cifar10-C datasets, we see that with the same filter size and number of trainable parameters, WMCG-CNN outperforms plain CNN in prediction performance on both clean and corrupted datasets. By enlarging the filter size, the robustness of WMCG-CNN is enhanced further. This even allows a much smaller number of trainable parameters to surpass the plain CNNs, which demonstrates WMCG-CNN's high parameter efficiency.

The proposed WMCG-CNN shows higher flexibility and controllability than the conventional CNNs. The use of filter decomposition decouples the relationship between the filter
size and the number of trainable parameters. For a certain convolutional kernel, the corresponding number of trainable parameters can be as small as only 1, or as large as any integer. In addition, we can choose a certain custom design basis as one prefer to control the performance of the network. For example, in the experiment on the Cifar10 dataset, we simply choose a single low-frequency FB basis that is of order 2 and can still can a good result on the Cifar10 dataset with higher robustness.

## IV Conclusion

In this paper, we propose an efficient and flexible implementation of group-equivariant CNN based on filter-wise weighted Monte Carlo sampling, which allows a higher degree of diversity of transformations for a performance boost. The proposed WMCG-CNN is shown to be an efficient generalization of standard CNN. The utility of shear transformation for tasks on natural images is demonstrated. The proposed WMCG-CNN shows superior efficiency on both image classification and image denoising tasks. We can also extend it for other computer vision tasks such as image segmentation and image reconstruction.

## Acknowledgments

This work was supported by the Deutsche Forschungsgemeinschaft (DFG) under grant no. 428149221, by Deutsches Zentrum fur Luft- und Raumfahrt e.V. (DLR), Germany under grant no. 01ZZ2105A and no. 01KD2214, and by Fraunhofer Gesellschaft e.V. under grant no. 017-100240/B7-aneg.
