Lie Group Decompositions for Equivariant Neural Networks
Invariance and equivariance to geometrical transformations have proven to be
very useful inductive biases when training (convolutional) neural network
models, especially in the low-data regime. Much work has focused on the case
where the symmetry group employed is compact or abelian, or both. Recent work
has explored enlarging the class of transformations used to the case of Lie
groups, principally through the use of their Lie algebra, as well as the group
exponential and logarithm maps. The applicability of such methods is limited by
the fact that depending on the group of interest $G$, the exponential map may
not be surjective. Further limitations are encountered when $G$ is neither
compact nor abelian. Using the structure and geometry of Lie groups and their
homogeneous spaces, we present a framework by which it is possible to work with
such groups primarily focusing on the groups $G = \text{GL}^{+}(n, \mathbb{R})$
and $G = \text{SL}(n, \mathbb{R})$, as well as their representation as affine
transformations $\mathbb{R}^{n} \rtimes G$. Invariant integration as well as a
global parametrization is realized by a decomposition into subgroups and
submanifolds which can be handled individually. Under this framework, we show
how convolution kernels can be parametrized to build models equivariant with
respect to affine transformations. We evaluate the robustness and
out-of-distribution generalisation capability of our model on the benchmark
affine-invariant classification task, outperforming previous proposals.
Mircea Mironenco, Patrick Forré
# Lie Group Decompositions for Equivariant Neural Networks

###### Abstract

Invariance and equivariance to geometrical transformations have proven to be very useful inductive biases when training (convolutional) neural network models, especially in the low-data regime. Much work has focused on the case where the symmetry group employed is compact or abelian, or both. Recent work has explored enlarging the class of transformations used to the case of Lie groups, principally through the use of their Lie algebra, as well as the group exponential and logarithm maps. The applicability of such methods to larger transformation groups is limited by the fact that depending on the group of interest \(G\), the exponential map may not be surjective. Further limitations are encountered when \(G\) is neither compact nor abelian. Using the structure and geometry of Lie groups and their homogeneous spaces, we present a framework by which it is possible to work with such groups primarily focusing on the Lie groups \(G=\text{GL}^{+}(n,\mathbb{R})\) and \(G=\text{SL}(n,\mathbb{R})\), as well as their representation as affine transformations \(\mathbb{R}^{n}\rtimes G\). Invariant integration as well as a global parametrization is realized by decomposing the 'larger' groups into subgroups and submanifolds which can be handled individually. Under this framework, we show how convolution kernels can be parametrized to build models equivariant with respect to affine transformations. We evaluate the robustness and out-of-distribution generalisation capability of our model on the standard affine-invariant benchmark classification task, where we outperform all previous equivariant models as well as all Capsule Network proposals.

## 1 Introduction

Symmetry constraints in the form of invariance or equivariance to geometric transformations have shown to be widely applicable inductive biases in the context of deep learning (Bronstein et al., 2021). Group-theoretic methods for imposing such constraints have led to numerous breakthroughs across a variety of data modalities. Convolutional neural networks (LeCun et al., 1995) which make use of translation equivariance while operating on image data have been generalized in several directions. Group-equivariant convolutional neural networks (GCNNs) represent one such generalization. Originally proposed in Cohen & Welling (2016), GCNNs make use of group convolution operators to construct layers that produce representations which transform in a predictable manner whenever the input signal is transformed by an a-priori chosen symmetry group \(G\)1. These models have been shown to exhibit increased generalization capabilities, while being less sensitive to \(G\)-perturbations of the input data. For these reasons, equivariant architectures have been proposed for signals in a variety of domains such as graphs (Han et al., 2022), sets (Zaheer et al., 2017) or point cloud data (Thomas et al., 2018). Constructing equivariant (convolutional) networks generally entails first choosing a group \(G\), a representation for the signal space in which our data lives and a description of the way this space transforms when the group _acts_ on it. Choosing a particular group \(G\) entails making a modelling assumption about the underlying (geometrical) structure of the data that should
be preserved. The group and group action employed also dictate how difficult it is to impose the desired symmetry constraint on the model. In the case of equivariant networks, early work has focused on the case where \(G\) is finite, with subsequent work largely concentrated on the Euclidean group \(\mathds{E}(n)\), and its subgroups \(\text{SE}(n,\mathbb{R})\) or \(\text{SO}(n)\).

Working with continuous groups is much more challenging, and the vast majority of equivariant models focus on the case where the group \(G\) has a set of desirable topological and structural properties, namely \(G\) is either compact or abelian, or both. Recent work (Bekkers, 2019; Finzi et al., 2020) has began to explore the possibility of building equivariant networks for "larger" groups, primarily through their representation as Lie groups - continuous groups with a differentiable structure. This research direction is promising since it allows for the modelling of symmetry groups beyond Euclidean geometry. Affine and projective geometry, and respectively affine and homography transformations are ubiquitous within computer vision, robotics and computer graphics (Zacur et al., 2014). Accounting for a larger degree of geometric variation has the promise of making vision architectures more robust to real-world data shifts. When working with non-compact and non-abelian Lie groups, for which the group exponential map is not surjective, standard tools from harmonic analysis cannot be employed straightforwardly. However, the structure and geometry of these groups can be exploited to build tractable optimization algorithms. Our main contribution is a framework by which it is possible to work with such groups, addressing the non-surjectivity of the exponential map as well as presenting a procedure by which invariant integration with respect to the Haar measure can be done in a principled manner. Rather than decomposing their representation, we choose to decompose the groups themselves, while working with the regular representation. The methodology and tools used are generally applicable to the case where one is working with a Lie group with finitely many connected components, however we restrict our focus to the groups \(\text{GL}^{+}(n,\mathbb{R})\) and \(\text{SL}(n,\mathbb{R})\), and more broadly the family of affine matrix Lie groups \(\mathbb{R}^{n}\rtimes H\), where \(H\) is one of the aforementioned groups. After presenting how one can parameterize functions and perform invariant integration on these groups, we show how an equivariant convolutional network can be constructed while still allowing kernels to be defined in the Lie algebra.

## 2 Related work

Recent proposal for Lie group equivariance (Bekkers, 2019; Finzi et al., 2020) focus on the infinite-dimensional regular representation of a group and rely on the group exponential map to allow convolution kernels to be defined analytically in the Lie algebra of the group. Working with the regular representation entails dealing with an intractable convolution integral over the group, and a (Monte Carlo) numerical integration procedure approximating the integral needs to be employed, which requires sampling group elements with respect to the _Haar_ measure of the group. Unfortunately, the applicability of these methods is limited to Lie groups for which the group exponential map is surjective, which is not the case for the affine group \(\text{Aff}(n)=\mathbb{R}^{n}\rtimes\text{GL}(n,\mathbb{R})\). Furthermore, these methods rely on the fact that for compact and abelian groups sampling with respect to the Haar measure of the group is straightforward, which is not the case for the affine groups of interest. MacDonald et al. (2022) aim to address these limitations, proposing a framework which can be applied to larger Lie groups, while still relying on the group exponential. While very general, their method comes with an exponential increase in memory requirements, as the group elements used for every convolutional layer needs to be sampled and kept in memory before the forward pass.

## 3 Background

Lie groupsA group \(G\) is a set together with an associative binary operation \(G\times G\to G\) which tells us how group elements can be composed to form another. Every element \(g\in G\) has an inverse \(g^{-1}\in G\), and the group contains an identity element \(e\in G\) such that \(gg^{-1}=e\). A Lie group \(G\) is a group as well as a smooth manifold, such that \(\forall g,h\in G\) the group operation \((g,h)\mapsto gh\) and the inversion map \(g\mapsto g^{-1}\) are smooth. Let \(\text{M}_{nn}(\mathbb{R})\coloneqq\text{M}_{n}(\mathbb{R})\) be the vector space of \(n\times n\) real matrices, and \(\text{GL}(n,\mathbb{R})\) the Lie group consisting of all invertible \(n\times n\) matrices. A _linear or matrix_ Lie group refers to a Lie subgroup of \(\text{GL}(n,\mathbb{R})\). \(\text{GL}(n,\mathbb{R})\), the translation group \((\mathbb{R}^{n},+)\) and the family of affine groups \(\mathbb{R}^{n}\rtimes H\), \(H\leq\text{GL}(n,\mathbb{R})\) are our primary interest, with \(H\) usually being one of \(\text{GL}^{+}(n,\mathbb{R})\), \(\text{SL}(n,\mathbb{R})\leq\text{GL}^{+}(n,\mathbb{R})\) or the rotation group \(\text{SO}(n)\).
Continuous group equivarianceEquivariance with respect to the action of a locally compact group \(G\) can be realized by constructing layers using the cross-correlation/convolution operators. We recall that in the continuous setting we model our signals as functions \(f:X\to\mathbb{R}^{K}\) defined on some underlying domain \(X\). For example, images and feature maps can be defined \(K\)-channel functions \(f\in L_{\mu}^{2}(\mathbb{R}^{2},\mathbb{R}^{K})\) which are square-integrable (with respect to the measure \(\mu\)), and which have bounded support in practice, e.g. \(f:[-1,1]^{2}\subseteq\mathbb{R}^{2}\to\mathbb{R}^{K}\). \(\mathcal{L}_{g}\) denotes the left-regular representation of \(G\), encoding the action of \(G\) on function spaces. For any continuous \(f\in C(X)\):

\[[\mathcal{L}_{g}f](x)\coloneqq f(g^{-1}x),\quad\forall g\in G,\;x\in X \tag{1}\]

Every locally compact \(G\) has a left (right) nonzero Radon measure \(\mu_{G}\) such that \(\mu_{G}(gB)=\mu_{G}(B)\) (\(\mu_{G}(Bg)=\mu_{G}(B)\)) for any Borel subset \(B\subseteq G\) and \(g\in G\). \(\mu_{G}\) is called the left (right) _Haar measure_ of \(G\) and it is unique up to multiplicative constants. A canonical example is the case \(G=(\mathbb{R}^{n},+)\) in which case \(\mu_{G}\) is the Lebesgue measure. The Haar measure allows for \(G\)-invariant integration to be realized, and for the group convolution to be defined. The invariance property can be easier to state without Borel sets if we define the functional \(\lambda_{\mu_{G}}:L^{1}(G)\to\mathbb{R}\) given by:

\[\lambda_{\mu_{G}}(f)=\int_{G}f(g)\text{d}\mu_{G}(g),\;\forall f\in L^{1}(G) \tag{2}\]

Then, a left Haar measure respects \(\lambda_{\mu_{G}}(\mathcal{L}_{g}f)=\lambda_{\mu_{G}}(f)\) for any \(g\in G\) and \(f\in L^{1}(G)\). For a group \(G\) acting transitively on locally compact spaces \(X\) and \(Y\) we then seek to construct an operator \(\mathcal{K}:L^{2}(X)\to L^{2}(Y)\) satisfying the _equivariance constraint_\(\mathcal{L}_{g}\circ\mathcal{K}=\mathcal{K}\circ\mathcal{L}_{g}\). Taking \(\mathcal{K}\in\mathcal{B}(L^{2}(X),L^{2}(Y))\) (linear and bounded), there exists a corresponding kernel \(k\in L^{1}(Y\times X)\) such that \(\mathcal{K}\) is an integral operator (Arendt & Bukhvalov, 1994, Theorem 1.3):

\[\mathcal{K}[f](y)=\int_{X}f(x)k(y,x)\text{d}\mu_{X}(x),\;\forall f\in L^{2}(X) \tag{3}\]

We formalize two scenarios, when \(X\) is a homogeneous space of \(G\) (not necessarily a group) and \(Y=G\), and the case where \(X=Y=G\). Focusing on the second case, if \(\text{d}\mu_{X}=\text{d}\mu_{G}\) is the Haar measure on \(G\), the integral operator can be reduced to the standard convolution/cross-correlation. Let \(k:Y\times X\to\mathbb{R}\) be a kernel that is invariant to the left action of \(G\) in both arguments, such that \(k(gx,gy)=k(x,y)\) for any \((x,y)\in Y\times X\) and \(g\in G\). Let \(\mu_{X}\) be a \(G\)-invariant Radon measure on \(X\), and define an operator \(\mathcal{K}\coloneqq C_{k}:L^{p}(X)\to L^{p}(G)\) (\(p\in\{1,2\}\)) such that \(\forall f\in L^{p}(X)\):

\[C_{k}:f\mapsto C_{k}f(y)=\int_{X}f(x)k(x,y)\text{d}\mu_{X}(x),\quad\forall y\in Y \tag{4}\]

\(C_{k}\) is \(G\)-equivariant: \(\mathcal{L}_{g}\circ C_{k}=C_{k}\circ\mathcal{L}_{g},\;\forall g\in G\) (A.2). Since \(X=Y=G\) are homogeneous spaces of \(G\) we can easily define a bi-invariant kernel by projection \(k(x,y)=\tilde{k}(g_{y}^{-1}x)\) (\(\tilde{k}:G\to\mathbb{R}\)) for any \((x,y)\in Y\times X\), where \(y=g_{y}y_{0}\) for some fixed \(y_{0}\). The kernel is bi-invariant:

\[k(hx,hy)=\tilde{k}((hg_{y})^{-1}hx)=\tilde{k}(g_{y}^{-1}h^{-1}hx)=\tilde{k}(g_{ y}^{-1}x)=k(x,y),\quad\forall h\in G \tag{5}\]

For the case \(Y=G\) and \(g_{y}=y\) (\(y_{0}=e\), the identity of \(G\)) this corresponds to a cross-correlation operator. For a convolution operator, we would analogously define \(k(x,y)=\tilde{k}(g_{x}^{-1}y)\) where \(x=g_{x}x_{0}\) for \(x_{0}\in X\) fixed. In this case we conclude that the _essential_ component needed for equivariance of the operator \(C_{k}\) is the \(G\)-invariant measure \(\text{d}\mu_{X}\), which is the Haar measure when \(X=Y=G\). When \(X\) is a homogeneous space of \(G\), but not necessarily \(G\) itself, we have to work with an operator which takes in a signal in \(L^{p}(X)\) and produces a signal \(L^{p}(G)\) on the group. This encompasses the case of the _lifting_ (cross-correlation) layers, which are commonly employed when working with the regular representation of a group (Cohen & Welling, 2016; Kondor & Trivedi, 2018; Bekkers, 2019). The form of the kernel \(k(\cdot)\) in this case can be derived through an equivariance constraint formulation as in Bekkers (2019); Cohen et al. (2019). It can also be shown (A.2) that an equivariant lifting cross-correlation can be defined as an operator \(C_{k}^{\uparrow}\) such that for any \(f\in L^{p}(X)\):

\[C_{k}^{\uparrow}:f\mapsto C_{k}^{\uparrow}f,\quad C_{k}^{\uparrow}f:g\mapsto \int_{X}f(x)k(g^{-1}x)\delta(g^{-1})\text{d}\mu_{X}(x),\;\forall g\in G \tag{6}\]

where \(\delta:G\to\mathbb{R}^{\times}_{\geq 0}\) records the change of variables by the action of \(G\) (see A.2). Group cross-correlation \(C_{k}^{\ast}\coloneqq C_{k}\) and convolution \(C_{k}^{\ast}\) operators will be defined for any \(f\in L^{p}(G)\):

\[C_{k}:f\mapsto C_{k}f,\quad C_{k}f:g\mapsto\int_{G}f(\tilde{g})k(g^{-1}\tilde{g })\text{d}\mu_{G}(\tilde{g}),\;\forall g\in G \tag{7}\]

\[C_{k}^{\ast}:f\mapsto C_{k}^{\ast}f,\quad C_{k}^{\ast}f:g\mapsto\int_{G}f(\tilde{ g})k(\tilde{g}^{-1}g)\text{d}\mu_{G}(\tilde{g}),\;\forall g\in G \tag{8}\]
Lie algebras and the group exponentialThe tangent space at the identity of a Lie group \(G\) is denoted by \(\mathfrak{g}\) and called the Lie algebra of \(G\). A (real) Lie algebra is a vector space (over \(\mathbb{R}\)) equipped with a bilinear map \([\cdot,\cdot]:\mathfrak{g}\times\mathfrak{g}\rightarrow\mathfrak{g}\) called the Lie bracket. For every Lie group we can define the Lie group exponential map \(\text{expm}:\mathfrak{g}\to G\), which is a diffeomorphism locally around \(0\in\mathfrak{g}\). Since we are interested in \(\text{GL}(n,\mathbb{R})\) and its subgroups, we can make things more concrete as follows. \(\text{M}_{n}(\mathbb{R})\) equipped with the matrix commutator \([X,Y]=XY-YX\) for \(X,Y\in\text{M}_{n}(\mathbb{R})\) is a Lie algebra, and more precisely it is the Lie algebra of \(\text{GL}(n,\mathbb{R})\). The notation \(\text{gl}(n,\mathbb{R})=\text{M}_{n}(\mathbb{R})\) is used for this identification. For \(G=\text{GL}(n,\mathbb{R})\) with \(\mathfrak{g}=\text{gl}(n,\mathbb{R})\), the group exponential is the matrix exponential \(\text{expm}:\text{gl}(n,\mathbb{R})\rightarrow\text{GL}(n,\mathbb{R})\), with the power series expression \(X\mapsto e^{X}=\sum_{k=0}^{\infty}\frac{1}{k!}X^{k}\).

Lie algebra parametrizationTo construct an equivariant layer using the Lie algebra of the group, one can define the kernels \(k(\cdot)\) in (7) or (8) as functions which take in Lie algebra elements. This requires a map \(\xi:\mathfrak{g}\to G\) which is (at least locally) a diffeomorphism, with an inverse that can be easily calculated, preferably in closed-form. This allows us to rewrite the kernel \(k:G\rightarrow\mathbb{R}\) as:

\[k(g^{-1}\tilde{g})=k(\xi(\xi^{-1}(g^{-1}\tilde{g})))=\tilde{k}_{\theta}(\xi^{ -1}(g^{-1}\tilde{g})) \tag{9}\]

\(\tilde{k}_{\theta}(\cdot)\) is effectively an approximation of \(k(\cdot)\) of the form \(\tilde{k}_{\theta}\cong k\circ\xi:\mathfrak{g}\rightarrow\mathbb{R}\) with learnable parameters \(\theta\). Using the inverse map \(\xi^{-1}(g^{-1}\tilde{g})\), \(\tilde{k}_{\theta}\) maps the Lie algebra coordinates of the 'offset' group element \(g^{-1}\tilde{g}\) (using the notation from the cross-correlation) to real values corresponding to the evaluation \(k(g^{-1}\tilde{g})\). Our kernels are now maps \(\tilde{k}_{\theta}\circ\xi^{-1}:G\rightarrow\mathbb{R}\), requiring the implementation of \(\xi^{-1}(\cdot)\) and a particular choice for the Lie algebra kernel \(\tilde{k}_{\theta}\). This description encompasses recent proposal for Lie group equivariant layers. In Bekkers (2019) the kernels are implemented by modelling \(\tilde{k}_{\theta}\) via B-splines, while Finzi et al. (2020) choose to parametrize \(\tilde{k}_{\theta}\) as small MLPs. Once \(\tilde{k}_{\theta}\) and \(\xi\) are chosen, we can approximate e.g. the cross-correlation using Monte Carlo integration:

\[\int_{G}f(\tilde{g})\tilde{k}_{\theta}(\xi^{-1}(g^{-1}\tilde{g}))\text{d}\mu _{G}(\tilde{g})\approx\frac{\mu_{G}(G)}{N}\sum_{i=1}^{N}f(\tilde{g}_{i}) \tilde{k}_{\theta}(\xi^{-1}(g^{-1}\tilde{g}_{i})),\;\tilde{g}_{i}\sim\mu_{G} \tag{10}\]

where \(\mu_{G}(G)\) denotes the volume of the integration space \(G\) and \(\tilde{g}_{i}\sim\mu_{G}\) indicates that \(\tilde{g}_{i}\) is sampled (uniformly) with respect to the Haar measure. This allows one to obtain equivariance (in expectation) with respect to \(G\). For compact groups, \(\mu_{G}\) can be normalized such that \(\mu_{G}(G)=1\).

## 4 Lie group decompositions for continuous equivariance

Limitations of the group exponentialOne common choice for \(\xi\) in (9) is the matrix exponential \(\xi:=\text{expm}\). Given a subgroup \(G\leq\text{GL}(n,\mathbb{R})\) for which expm is surjective, the advantage of this parametrization is given by the fact that every element \(g\in G\) can be expressed as \(g=\text{expm}(X)=e^{X}\) for \(X\in\mathfrak{g}\). One then needs to consider if both \(\xi\) and \(\xi^{-1}\) need to be implemented, and whether these maps are available in closed form. If one does employ the group exponential for \(\xi\), the inverse map \(\xi^{-1}\) is given by the matrix logarithm:

\[\xi^{-1}(g^{-1}\tilde{g})=\text{logm}(g^{-1}\tilde{g}),\quad\text{logm}:G \rightarrow\mathfrak{g} \tag{11}\]

Assuming there exist \(X\) and \(Y\) such that \(e^{X}=g^{-1}\) and \(e^{Y}=\tilde{g}\), (11) can be rewritten as \(\text{logm}(g^{-1}\tilde{g})=\text{logm}(e^{X}e^{Y})\). A key optimization underlying this framework is enabled by employing the BCH formula (A.3), which tells us that for abelian Lie groups \(\text{logm}(e^{X}e^{Y})=X+Y\). This simplifies calculations considerably and allows one to work primarily at the level of the Lie algebra, bypassing the need to calculate and sample the kernel inputs \(g^{-1}\tilde{g}\) at the group level. Considering the affine Lie groups \(\mathbb{R}^{n}\rtimes H,\;H\leq\text{GL}(n,\mathbb{R})\), this simplification can be used for example for the abelian groups \(H=\text{SO}(n)\) and \(H=\mathbb{R}^{\times}(n)\times\text{SO}(n)\), consisting of rotations and scaling. Bekkers (2019); Finzi et al. (2020) primarily work with these groups, and choose \(\xi\) and \(\xi^{-1}\) to be the matrix exponential and logarithm, respectively. For the non-abelian Lie groups \(\text{SL}(n,\mathbb{R})\) or \(\text{GL}^{+}(n,\mathbb{R})\) the non-surjectivity of the exponential map limits the applicability of the matrix logarithm outside of a neighborhood around the identity (A.3). The class of equivariant networks that can be implemented with this framework is then firstly limited by the 'parametrization' map \(\xi:\mathfrak{g}\to G\), and its inverse \(\xi^{-1}\), motivating the search for an alternative solution. Another key limitation is that for (10) to realize an equivariant estimator when numerically approximating the convolution/cross-correlation integral, sampling needs to be realised with respect to the Haar measure of the group \(G\).
Techniques for sampling with respect to the Haar measure on the groups \(\text{SO}(n)\) or \(\mathbb{R}^{\times}(n)\times\text{SO}(n)\) are known, and generally reduce to working with uniform measures on Euclidean spaces or unit quaternions in the case of \(\text{SO}(3)\). These concerns are also discussed by MacDonald et al. (2022), whose work is closest to our own. While their approach can be applied to any matrix Lie group, in practice their method requires that all group elements \(\tilde{g}^{-1}g\) be precalculated and kept in memory before the forward pass, greatly limiting the scalability of the method due to its exponential memory requirements. We aim to address these limitations, while not restricting the class of 'Lie algebra kernels' \(k_{\theta}:\mathfrak{g}\to\mathbb{R}\) that can be used. Under our framework one should be able to employ any \(k_{\theta}\) that uses the coordinates of tangent vectors in \(\mathfrak{g}\) expressed in some basis.

### Lie group decomposition theory

We exploit the fact that the groups \(\text{GL}^{+}(n,\mathbb{R})\) and \(\text{SL}(n,\mathbb{R})\) have an underlying product structure that allows them to be decomposed into subgroups and submanifolds which are easier to work with individually. More precisely, \(G\in\{\text{GL}^{+}(n,\mathbb{R}),\text{SL}(n,\mathbb{R})\}\) can be decomposed as a product \(P\times H\), where \(H\leq G\) is the maximal compact subgroup of \(G\) and \(P\subseteq G\) is a submanifold which is diffeomorphic to \(\mathbb{R}^{k}\), for some \(k\geq 0\), and we have a diffeomorphism \(\varphi:P\times H\to G\). It can be shown that if the map \(\varphi\) is chosen correctly the Haar measure \(\mu_{G}\) can be written as the pushforward measure \(\varphi_{*}(\mu_{P}\otimes\mu_{H})\), where \(\mu_{P}\) is a \(G\)-invariant measure on \(P\) and \(\mu_{H}\) is the Haar measure on \(H\).

Factorizing the Haar measureSuppose \(G\) is a locally compact group of interest (e.g. \(\text{GL}^{+}(n,\mathbb{R})\)), with (left) Haar measure \(\mu_{G}\). Assume there exist a set of subspaces or subgroups \(P\subseteq G\), \(K\subseteq G\), such that \(G=PK\), and a homeomorphism \(\varphi:P\times K\to G\). Further assume that \(\mu_{P}\) and \(\mu_{K}\) are (left) \(G\)-invariant Radon measures on the corresponding spaces. We look to express (up to multiplicative coefficients) the Haar measure \(\mu_{G}\) as the pushforward of the product measure \(\mu_{P}\otimes\mu_{K}\) under the map \(\varphi\). This allows for the following change of variables for any \(f\in L^{1}(G)\):

\[\int_{G}f(g)\text{d}\mu_{G}(g)=\int_{P\times K}f(\varphi(p,k))\text{d}(\mu_{P} \otimes\mu_{K})(p,k)=\int_{P}\int_{K}f(\varphi(p,k))\text{d}\mu_{K}(k)\text{ d}\mu_{P}(p) \tag{12}\]

In the context of Monte Carlo simulation this will enable us to produce random samples distributed according the measure \(\mu_{G}\) by sampling on the _independent_ factor spaces \(P\) and \(K\) and constructing a sample on \(P\times K\) and respectively on \(G\) using the map \(\varphi\). The space \(P\) will either be another closed subgroup, or a measurable subset \(P\subseteq G\) that is homeomorphic to the quotient space \(G/K\). In particular, if \(P\) is not a subgroup, we will focus on the case where \(P\) is a homogeneous space of \(G\) with stabilizer \(K\) such that \(P\cong G/K\). When the left and right Haar measure of a group coincide, the group is called _unimodular_. The groups \(\text{GL}^{+}(n,\mathbb{R})\), \(\text{SL}(n,\mathbb{R})\) are unimodular, however this is not true for all affine groups \(\mathbb{R}^{n}\rtimes H\). For groups which are volume-preserving, this is not as much of an issue in practice. However, \(\text{GL}^{+}(n,\mathbb{R})\) is not volume-preserving, and we also desire that our framework be general enough to deal with the non-unimodular case as well. If \(G\) is not unimodular and \(\mu_{G}\) is its left Haar measure, there exists a continuous group homomorphism \(\Delta_{G}:G\to\mathbb{R}_{>0}^{>}\), called the _modular function_ of \(G\), which records the degree to which \(\mu_{G}\) fails to be right-invariant. We now have the tools necessary to record two possible integral decomposition methods.

**Theorem 4.1**.: _(1) Let \(G\) be a locally compact group, \(H\leq G\) a closed subgroup, with left Haar measures \(\mu_{G}\) and \(\mu_{H}\) respectively. There is a \(G\)-invariant Radon measure \(\mu_{G/H}\) on \(G/H\) if and only if \(\left.\Delta_{G}\right|_{H}=\Delta_{H}\). The measure \(\mu_{G/H}\) is unique up to a scalar factor and if suitably normalized:_

\[\int_{G}f(g)\text{d}\mu_{G}(g)=\int_{G/H}\int_{H}f(gh)\text{d}\mu_{H}(h)\text{ d}\mu_{G/H}(gH),\ \forall f\in L^{1}(G) \tag{13}\]

_(2) Let \(P\leq G\), \(K\leq G\) closed subgroups such that \(G=PK\). Assume that \(P\cap K\) is compact, and \(Z_{0}\) denotes the stabilizer of the transitive left action of \(P\times K\) on \(G\) given by \((p,k)\cdot g=pkg^{-1}\), for any \((p,k)\in P\times K\) and \(g\in G\). Let \(G\), \(P\) and \(K\) be \(\sigma\)-compact (which holds for matrix Lie groups), \(\mu_{G}\), \(\mu_{P}\) and \(\mu_{K}\) left Haar measures on \(G\), \(P\), and \(K\) respectively and \(\Delta_{G}|_{K}=\Lambda\) is the modular function of \(G\) restricted to \(K\). Then \(\mu_{G}\) is given by \(\mu_{G}=\pi_{*}(\mu_{P}\otimes\Lambda^{-1}\mu_{K})\), where \(\pi:P\times K\to(P\times K)/Z_{0}\) is the canonical projection. In integral form we have:_

\[\int_{G}f(g)\text{d}\mu_{G}(g)=\int_{P}\int_{K}f(pk)\frac{\Delta_{G}(k)}{\Delta_{ K}(k)}\text{d}\mu_{K}(k)\text{d}\mu_{P}(p),\quad\forall f\in L^{1}(G) \tag{14}\]

Proof.: Folland (2016, Theorem 2.51) and Wijsman (1990, Proposition 7.6.1).
Affine groupsWhile these results are very abstract, when going to the Lie group setting, we can already deal with semi-direct products \(G=N\rtimes H\), where \(N,H\) are subgroups of \(G\), and \(N\) is normal. The modular function on \(N\rtimes H\) is \(\Delta_{N\rtimes H}(n,h)=\Delta_{N}(n)\Delta_{H}(h)\delta(h)^{-1}\)(Kaniuth & Taylor, 2013). The term \(\delta:H\to\mathbb{R}^{\times}_{>0}\) records the effect of the action of \(H\) on \(N\), and it coincides with the term \(\delta(\cdot)\) used in the lifting layer definition (28). Making things concrete, take the affine groups \(G=\mathbb{R}^{n}\rtimes H\), \(H\leq\text{GL}(n,\mathbb{R})\), which are defined under the semi-direct product structure by:

\[G=\mathbb{R}^{n}\rtimes H=\{(x,A)\mid x\in\mathbb{R}^{n},\ A\in H\} \tag{15}\]

\(H\) acts on \(\mathbb{R}^{n}\) by matrix multiplication and for any \((x,A),(y,B)\in G\), the product and inverse are:

\[(x,A)(y,B)=(x+Ay,AB),\quad(x,A)^{-1}=(-A^{-1}x,A^{-1}) \tag{16}\]

Elements of \(\mathbb{R}^{n}\) are concretely represented as column vectors. Viewing \((\mathbb{R}^{n},+)\) as the additive group, we have \(\delta:H\to\mathbb{R}^{\times}_{>0}\) given by \(\delta(A)=|\text{det}(A)|\) for any \(A\in H\). Applying Thm. 4.1, gives:

\[\int_{G}f(g)\;\text{d}\mu_{G}(g)=\int_{H}\int_{\mathbb{R}^{n}}f((x,A))\;\frac{ \text{d}x\text{d}\mu_{H}(A)}{|\text{det}(A)|},\;\forall f\in C_{e}(G) \tag{17}\]

\(f((x,A))\) will be denoted by \(f(x,A)\) going forward. Expressing the cross-correlation \(C_{k}f\) in the product space parametrization we have for \(f\in L^{2}(\mathbb{R}^{n}\rtimes H)\) and \((x,A)\in\mathbb{R}^{n}\rtimes H\):

\[C_{k}f:(x,A)\mapsto\int_{H}\int_{\mathbb{R}^{n}}f(\tilde{x},\tilde{A})k((x,A) ^{-1}(\tilde{x},\tilde{A}))\delta(\tilde{A}^{-1})\text{d}\tilde{x}\text{d}\mu _{H}(\tilde{A}) \tag{18}\]

Manifold splitting via Cartan/Polar decompositionLet \(\text{Sym}(n,\mathbb{R})\) be the vector space of \(n\times n\) real symmetric matrices and \(\text{Pos}(n,\mathbb{R})\) the subset of \(\text{Sym}(n,\mathbb{R})\) of symmetric positive definite (SPD) matrices. Denote by \(\text{SPos}(n,\mathbb{R})\) the subset of \(\text{Pos}(n,\mathbb{R})\) consisting of SPD matrices with unit determinant, and by \(\text{Sym}_{0}(n,\mathbb{R})\) the subspace of \(\text{Sym}(n,\mathbb{R})\) of traceless real symmetric matrices. Any matrix \(A\in\text{GL}(n,\mathbb{R})\) can be uniquely decomposed via the left polar decomposition as \(A=PR\) where \(P\in\text{Pos}(n,\mathbb{R})\) and \(R\in\text{O}(n)\) (A.8). The factors of this decomposition are uniquely determined and we have a bijection \(\text{GL}(n,\mathbb{R})\to\text{Pos}(n,\mathbb{R})\times\text{O}(n)\) given by:

\[A\mapsto(\sqrt{AA^{T}},\sqrt{AA^{T}}^{-1}A),\quad\forall A\in\text{GL}(n, \mathbb{R}) \tag{19}\]

For the reader unfamiliar with Lie group structure theory, the following results can simply be understood in terms of matrix factorizations commonly used in numerical linear algebra. The polar decomposition splits the manifold \(\text{GL}^{+}(n,\mathbb{R})\) into the product \(\text{Pos}(n,\mathbb{R})\times\text{SO}(n)\), and \(\text{SL}(n,\mathbb{R})\) into \(\text{SPos}(n,\mathbb{R})\times\text{SO}(n)\). We use the notation \(G\to M\times H\) to cover both cases. This decomposition can be generalized, as the spaces \(\text{Pos}(n,\mathbb{R})=\text{GL}^{+}(n,\mathbb{R})/\text{SO}(n)\) and \(\text{SPos}(n,\mathbb{R})=\text{SL}(n,\mathbb{R})/\text{SO}(n)\) are actually _symmetric spaces_, and a _Cartan decomposition_ is available in this case (A.8). The Cartan decomposition tells us how to decompose not only at the level of the Lie group, but also at the level of the Lie algebra. In fact, using this decomposition we can also obtain a factorization of the measure on these groups. Let \((G/H,M,\mathfrak{m})\) define our 'Lie group data', corresponding to \((\text{GL}^{+}(n,\mathbb{R})/\text{SO}(n),\text{Pos}(n,\mathbb{R}),\text{Sym} (n,\mathbb{R}))\) or \((\text{SL}(n,\mathbb{R})/\text{SO}(n),\text{SPos}(n,\mathbb{R}),\text{Sym}_{0 }(n,\mathbb{R}))\).

**Theorem 4.2**.: _Let \((G/H,M,\mathfrak{m})\) be as above, and denote by \(\mathfrak{g}\), \(\mathfrak{h}\) the Lie algebras of \(G\) and \(H\)._

1. _The matrix exponential and logarithm are diffeomorphisms between_ \(\mathfrak{m}\) _and_ \(M\)_, respectively. For any_ \(P\in M\) _and_ \(\alpha\in\mathbb{R}\)_, the power map_ \(P\mapsto P^{\alpha}\) _is smooth and can be expressed as:_ \[P^{\alpha}=\text{expm}(\alpha\text{logm}(P)),\quad\forall P\in\text{Pos}(n, \mathbb{R})\] (20)
2. \(G\cong M\times H\) _and_ \(G\cong\mathfrak{m}\times H\)_. We have group-level diffeomorphisms:_ \[\chi:M\times H\to G,\quad\chi(P,R)\mapsto PR\] (21) \[\Phi:\mathfrak{m}\times H\to G,\quad\Phi:(X,R)\mapsto\text{expm}(X)R=e^{X}R\] (22)
3. _The above maps can be inverted in closed-form:_ \[\chi^{-1}:G\to M\times H,\;\chi^{-1}:A\mapsto(\sqrt{AA^{T}},\sqrt{AA^{T}}^{- 1}A)\] (23) \[\Phi^{-1}:G\to\mathfrak{m}\times H,\quad\Phi^{-1}:A\mapsto(\frac{1}{2} \text{logm}(AA^{T}),\text{expm}(-\frac{1}{2}\text{logm}(AA^{T}))A)\] (24)

See (A.9) for proofs and references. At the level of the Lie algebra, we have the decomposition \(\mathfrak{gl}(n,\mathbb{R})=\mathfrak{so}(n)\oplus\text{Sym}(n,\mathbb{R})\). The Lie algebra of \(\text{SL}(n,\mathbb{R})\) is \(\mathfrak{sl}(n,\mathbb{R})=\{X\in\mathfrak{gl}(n,\mathbb{R})\mid\text{tr}(X)=0\}\). It decomposes similarly \(\mathfrak{sl}(n,\mathbb{R})=\mathfrak{so}(n)\oplus\text{Sym}_{0}(n,\mathbb{R})\). Then \(\mathfrak{h}=\mathfrak{so}(n)\) with \(\mathfrak{m}=\text{Sym}(n,\mathbb{R})\) if \(G=\text{GL}^{+}(n,\mathbb{R})\) and \(\mathfrak{m}=\text{Sym}_{0}(n,\mathbb{R})\) if \(G=\text{SL}(n,\mathbb{R})\).
### A parametrization based on the Cartan Decomposition

Consider again the notation \((G/H,M,\mathfrak{m})\) as in Theorem 4.2 (\(G=\text{GL}^{+}(n,\mathbb{R})\) or \(G=\text{SL}(n,\mathbb{R})\)).

Concrete integral decompositionsFrom Theorem 4.2 and the fact that symmetric matrices have a unique square root, we actually have equivalent decompositions for \(A\in G\) as \(A=PR\) or \(A=S^{1/2}R\) for \(S,P\in M\), \(R\in H\) and \(P=S^{1/2}\). For \(\text{GL}(n,\mathbb{R})\), the decomposition \(A=S^{1/2}R\), has a factorization of the Haar measure of \(\text{GL}(n,\mathbb{R})\) as a product of invariant measures on \(\text{Pos}(n,\mathbb{R})\) (shortened \(\text{Pos}(n)\)) and \(\text{O}(n)\). Let \(\mu_{\text{Pos}(n)}\) denote the \(\text{GL}(n,\mathbb{R})\) invariant measure on \(\text{Pos}(n)\).

**Theorem 4.3**.: _Denote \(G=\text{GL}(n,\mathbb{R})\), \(H=\text{O}(n)\), and let \(\mu_{G}\) be the Haar measure on \(G\) and \(\mu_{H}\) the Haar measure on \(H\) normalized by \(\text{\rm Vol}(H)=1\). For \(A\in G\), under the decomposition \(A=S^{1/2}R\), \(S\in\text{Pos}(n)\), \(R\in H\), the measure on \(G\) splits as \(d\mu_{G}(A)=\beta_{n}d\mu_{\text{Pos}(n)}(S)d\mu_{H}(R)\), where \(\beta_{n}=\frac{\text{\rm Vol}(\mathfrak{m}(n))}{2}\) is a normalizing constant. Restricting to \(G=\text{\rm GL}^{+}(n,\mathbb{R})\) and \(H=\text{\rm SO}(n)\) and ignoring constants, we have:_

\[f\mapsto\int_{G}f(A)\text{\rm d}\mu_{G}(A)=\int_{\text{Pos}(n)}\int_{H}f(S^{1/ 2}R)\text{\rm d}\mu_{H}(R)\text{\rm d}\mu_{\text{Pos}(n)}(S),\;\forall f\in C _{c}(G) \tag{25}\]

The Haar measure of \(\text{\rm GL}(n,\mathbb{R})\) is \(d\mu_{\text{GL}(n,\mathbb{R})}(A)=|\text{\rm det}(A)|^{-n}dA\), with \(dA\) the Lebesgue measure on \(\mathbb{R}^{n^{2}}\). We now describe how to sample on the individual factors to obtain \(\text{\rm GL}(n,\mathbb{R})\) samples.

**Theorem 4.4**.: _If a random matrix \(A\in\text{\rm GL}(n,\mathbb{R})\) has a left-\(\text{\rm O}(n)\) invariant density function relative to \(|AA^{T}|^{-n/2}dA\), then \((AA^{T})^{1/2}=S^{1/2}\) and \(R=(AA^{T})^{-1/2}A\) are independent random matrices and \(R\) has a uniform probability distribution on \(\text{\rm O}(n)\). The uniform distribution on \(\text{\rm O}(n)\) will be the normalized Haar measure \(\mu_{\text{\rm O}(n)}\). Conversely, if \(S\in\text{\rm Pos}(n)\) has a density function \(f:\text{\rm Pos}(n)\to\mathbb{R}_{\geq 0}\) relative to \(\mu_{\text{Pos}(n)}\) and \(R\in\text{\rm O}(n)\) is uniformly distributed with respect to the Haar measure \(\mu_{\text{\rm O}(n)}\), then \(A=S^{1/2}R\) has a density function \(\beta_{n}^{-1}f(AA^{T})|\text{\rm det}(A)|^{-n}\) relative to \(dA\)._

Theorems 4.3 and 4.4 are known results that appear in the random matrix theory literature, but have not seen recent application in the context of deep learning. In (A.10) we provide more details and references. Using the decomposition \(A=S^{1/2}R\) invariant integration problems on \(G\) can be transferred to the product space \(M\times H\), and we can express up to normalization the invariant measure \(\mu_{G}\) as \(\varphi_{*}(\mu_{M}\otimes\mu_{H})\). To construct samples \(\{\mathbf{A}_{1},\dots,\mathbf{A}_{n}\}\sim\mu_{G}\) one produces samples \(\{\mathbf{R}_{1},\dots,\mathbf{R}_{n}\}\sim\mu_{H}\) where \(\mu_{H}\) will be the uniform distribution on \(H\), and samples \(\{\mathbf{M}_{1},\dots,\mathbf{M}_{n}\}\sim\mu_{M}\). Then \(\mu_{G}\)-distributed random values are obtained by \(\{\mathbf{A}_{1},\dots,\mathbf{A}_{n}\}=\{\varphi(\mathbf{M}_{1},\mathbf{R}_ {1}),\dots,\varphi(\mathbf{M}_{n},\mathbf{R}_{n})\}\), where again \(\varphi:M\times H\to G\) is given by \(\varphi:(S,R)\mapsto S^{1/2}R\).

Mapping to the Lie algebra and backAny \(A\in G\) can be expressed uniquely as \(A=e^{X}R\) for \(x\in\mathfrak{m}\) and \(R\in H\). Since \(H=\text{\rm SO}(n)\) in both cases, the fact that \(\text{\rm expm}:\mathfrak{so}(n)\to\text{\rm SO}(n)\) is surjective, allows us to write it \(A=e^{X}e^{Y}\), \(Y\in\mathfrak{so}(n)\). The factors \(X\) and \(R=e^{Y}\) are obtained using \(\Phi^{-1}\) (23). Then by taking the principal branch of the matrix logarithm on \(H=\text{\rm SO}(n)\), \(Y=\text{\rm logm}(R)\). A map \(\xi^{-1}:G\to\mathfrak{g}\) as described in (9) and (10) is constructed as \(\xi^{-1}=(\text{\rm id}_{\mathfrak{m}}\times\text{\rm logm})\circ\Phi^{-1}\). More precisely, for any \(A=e^{X}e^{Y}\in G\), using \(\xi^{-1}\) we obtain the tangent vectors \((Y,X)\in\mathfrak{so}(n)\times\mathfrak{m}\) and since \(\mathfrak{g}=\mathfrak{so}(n)\oplus\mathfrak{m}\) we have a unique \(Z=X+Y\in\mathfrak{g}\). Details are given in (A.11).

Define \(\tilde{K}_{\theta}\coloneqq k_{\theta}\circ\xi^{-1}:G\to\mathbb{R}\) as our Lie algebra kernel. A Monte Carlo approximation of a cross-correlation operator \(C_{k}:L^{2}(G)\to L^{2}(G)\) as in (10) will be of the form:

\[C_{k}f:g\mapsto\frac{1}{N}\sum_{i=1}^{N}f(\tilde{g}_{i})\tilde{K}_{\theta}( \tilde{g}_{i}^{-1}g),\;\tilde{g}_{i}\sim\mu_{G},\quad\forall g\in G \tag{26}\]

Layer definitionsEvery element \((x,A)\) of \(\mathbb{R}^{n}\rtimes H\), can be uniquely decomposed as \((x,I)(0,A)\), with \(I\) the \(n\times n\) identity matrix. One can use the fact that \(\mathcal{L}_{(x,A)}=\mathcal{L}_{(x,I)}\mathcal{L}_{(0,A)}\) to write:

\[k((x,A)^{-1}(\tilde{x},\tilde{A}))=\mathcal{L}_{(x,A)}k(\tilde{x},\tilde{A})= \mathcal{L}_{(x,I)}[\mathcal{L}_{(0,A)}k(\tilde{x},\tilde{A})]=\mathcal{L}_{x}[k (A^{-1}\tilde{x},A^{-1}\tilde{A})] \tag{27}\]

An efficient implementation of a convolutional layer can be realised in practice for \(n\in\{2,3\}\) by first obtaining the transformed kernel \(k(A^{-1}\tilde{x},A^{-1}\tilde{A})\) and then applying the translation \(\mathcal{L}_{x}\) using an efficient convolution routine, as done for example in Cohen & Welling (2016); Bekkers (2019).
We can approximate a continuous convolution (Finzi et al., 2020) by sampling the translation factor in a uniform grid of coordinates \(\tilde{x}\sim[-1,1]^{n}\subset\mathbb{R}^{n}\) as the parametrization \(\xi:\mathbb{R}^{n}\times\mathfrak{gl}(n,\mathbb{R})\to G\) is the identity map for the first factor. Our lifting layers are:

\[C_{k}^{\dagger}f:(x,A) \mapsto\int_{\mathbb{R}^{n}}f(\tilde{x})\mathcal{L}_{x}k(A^{-1} \tilde{x})\delta(A^{-1})\text{d}\tilde{x} \tag{28}\] \[\approx\frac{1}{N}\sum_{i=1}^{N}f(\tilde{x}_{i})\mathcal{L}_{x}[k _{\theta}(A^{-1}\tilde{x}_{i})\delta(A^{-1})],\ x_{i}\sim[-1,1]^{n} \tag{29}\]

For the non-lifting layers, starting from (18), denoting \(\text{d}\tilde{A}=\text{d}\mu_{G}(\tilde{A})\) and applying (27) we have:

\[[C_{k}f](x,A)=\int_{\mathbb{R}^{n}}\int_{G}f(\tilde{x},\tilde{A})\mathcal{L}_ {x}k(A^{-1}\tilde{x},A^{-1}\tilde{A})\delta(\tilde{A}^{-1})\text{d}\tilde{x} \text{d}\tilde{A} \tag{30}\]

Using Theorem 4.3, denote the invariant measures \(\mu_{M}\) and \(\mu_{H}\) by \(\text{d}S\) and \(\text{d}R\), we obtain:

\[[C_{k}f](x,A)=\beta_{n}\int_{\mathbb{R}^{n}}\int_{H}\int_{M}f(\tilde{x},S^{1/2 }R)\mathcal{L}_{x}k(A^{-1}\tilde{x},A^{-1}S^{1/2}R)\delta(S^{-1/2})\text{d}S \text{d}R\text{d}\tilde{x} \tag{31}\]

The kernel in (26) is now of the form \(K_{\theta}:\mathbb{R}^{n}\rtimes G\to\mathbb{R}\), giving us:

\[[C_{k}f](x,A)\approx\frac{V}{N}\sum_{i=1}^{N}f(\tilde{x_{i}},S_{i}^{1/2}R_{i })\mathcal{L}_{x}[K_{\theta}(A^{-1}\tilde{x_{i}},\xi^{-1}(A^{-1}S_{i}^{1/2}R_ {i}))\delta(S_{i}^{-1/2})] \tag{32}\]

where \(\tilde{x_{i}}\sim[-1,1]^{n}\), \(S_{i}\times R_{i}\sim(\mu_{M}\otimes\mu_{H})\), and \(R_{i}\) sampled uniformly with respect \(\mu_{H}\). \(V\) records both the volume of the integration space from the MC approximation as well as the constant \(\beta_{n}\).

## 5 Experiments

For all experiments we use a ResNet-style architecture, replacing convolutional layers with cross-correlations that are equivariant (in expectation) with respect to the groups \(\mathbb{R}^{2}\rtimes\text{GL}^{+}(2,\mathbb{R})\) and \(\mathbb{R}^{2}\rtimes\text{SL}(2,\mathbb{R})\). Details regarding the network architecture and training are given in Appendix B.

Affine-transformation invarianceWe evaluate our model on a benchmark affine-invariant image classification task employing the affNIST dataset2. The main works we compare with are the affine-equivariant model of MacDonald et al. (2022) and the Capsule Network of De Sousa Ribeiro et al. (2020) which are state of the art for this task. The experimental setup involves training on the standard set of \(50000\) non-transformed MNIST images (padded to \(40\times 40\)), and evaluating on the affNIST test set, which consists of \(320000\) affine-transformed MNIST images. The model never sees the transformed affNIST images during training, and we do not use any data augmentation techniques. In this case, robustness with respect to the larger groups of the affine family of transformations is needed. For a fair comparison we roughly equalize the number of parameters with the referenced models.

Footnote 2: [http://www.cs.toronto.edu/tijmen/affNIST](http://www.cs.toronto.edu/tijmen/affNIST)

\begin{table}
\begin{tabular}{c c c c c} \hline Model & affNIST Acc. & MNIST Acc. & Parameters & MC. Samples \\ \hline \(\mathbb{R}^{2}\rtimes\text{SL}(2,\mathbb{R})\) & \(97.9(\pm 0.25)\) & \(99.61(\pm 0.1)\) & \(370\)K & \(10\) \\ \hline RU CapsNet (De Sousa Ribeiro et al., 2020) & \(97.69\) & \(99.72\) & \(>580\)K & — \\ \hline \(\mathbb{R}^{2}\rtimes\text{GL}^{+}(2,\mathbb{R})\) & \(97.4(\pm 0.2)\) & \(99.5(\pm 0.1)\) & \(395\)K & \(10\) \\ \hline affConv (MacDonald et al., 2022) & \(95.08\) & \(98.7\) & \(374\)K & \(100\) \\ \hline affine CapsNet (Gu and Tresp, 2020) & \(93.21\) & \(99.23\) & — & — \\ \hline Equivariant CapsNet (Lenssen et al., 2018) & \(89.1\) & \(98.42\) & \(235\)K & — \\ \hline \end{tabular}
\end{table}
Table 1: affNIST classification accuracy, after training on MNIST.
Table 1 reports the average test performance of our model at the final epoch, over five training runs with different initialisations. The results and parameter counts for Gu and Tresp (2020); De Sousa Ribeiro et al. (2020); MacDonald et al. (2022) are taken from (MacDonald et al., 2022, Figure 2) and (De Sousa Ribeiro et al., 2020, Table 3). We observe that our equivariant models are robust and generalize well, with the \(\mathbb{R}^{2}\rtimes\text{SL}(2,\mathbb{R})\) model outperforming all previous equivariant models and Capsule Networks. Note that, compared to MacDonald et al. (2022), our sampling scheme requires \(10\) times less samples to realize an accurate Monte Carlo approximation of the convolution. The \(\mathbb{R}^{2}\rtimes\text{GL}^{+}(2,\mathbb{R})\) model performs slightly worse than the volume-preserving affine group \(\mathbb{R}^{2}\rtimes\text{SL}(2,\mathbb{R})\). This can be explained by considering that the affNIST dataset contains only a small degree of scaling.

Homography transformationsWe further evaluate and report in Table 2 the performance of the same model evaluate on the homNIST dataset of MacDonald et al. (2022). The setup is identical to the affNIST case, with the images now being transformed by random homographies. We observe a similar degree of robustness in this case, again outperforming previous methods applied to this task.

## 6 Conclusion

We have built a framework for constructing equivariant networks when working with matrix Lie groups that are not necessarily compact or abelian. Using the structure theory of semisimple/reductive Lie groups we have shown one possible avenue for constructing invariant/equivariant (convolutional) layers primarily relying on tools which allow us to decompose larger groups into smaller ones. In our preliminary experiments, the robustness and out-of-distribution capabilities of the equivariant models were shown to outperform previous proposals on tasks where the symmetry group of relevance is one of \(\text{GL}^{+}(n,\mathbb{R})\) or \(\text{SL}(n,\mathbb{R})\).

Our contribution is largely theoretical, providing a framework by which equivariance/invariance to complex symmetry groups can be obtained. Further experiments will look to validate the applicability of our method to other data modalities, such as point clouds or molecules, as in Finzi et al. (2020).

While we have primarily focused on convolution operators, we remark that the tools explored here are immediately applicable to closely-related machine learning models which employ Lie groups and their regular representation for invariance/equivariance. For example, the 'LieTransformer' architecture proposed in Hutchinson et al. (2021) opts to replace convolutional layers with self-attention layers, while still using the Lie algebra of the group as a mechanism for incorporating positional information. They face the same challenge in that their parametrization is dependent on the mapping elements back and forth between a chosen Lie group and its Lie algebra, and they require a mechanism for sampling on the desired group. The methods presented here are directly applicable in this case. Future work will explore expanding the class of Lie groups employed by such models using the tools presented here.

Another potential avenue to explore is the applicability of the presented tools to the problem of 'partial' and 'learned' invariance/equivariance (Benton et al., 2020). The sampling mechanism of the product decomposition allows one to specify a probability distribution for the non-orthogonal factor, which could be learned from data.

\begin{table}
\begin{tabular}{c c c} \hline \hline Model & homNIST Acc. & MC. Samples \\ \hline \(\mathbb{R}^{2}\rtimes\text{SL}(2,\mathbb{R})\) & \(98.1(\pm 0.15)\) & \(10\) \\ \hline \(\mathbb{R}^{2}\rtimes\text{GL}^{+}(2,\mathbb{R})\) & \(97.71(\pm 0.1)\) & \(10\) \\ \hline affConv (MacDonald et al., 2022) & \(95.71\) & \(100\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: homNIST classification.