Lie Neurons: Adjoint-Equivariant Neural Networks for Semisimple Lie
  Algebras
This paper proposes an equivariant neural network that takes data in any
semi-simple Lie algebra as input. The corresponding group acts on the Lie
algebra as adjoint operations, making our proposed network adjoint-equivariant.
Our framework generalizes the Vector Neurons, a simple
$\mathrm{SO}(3)$-equivariant network, from 3-D Euclidean space to Lie algebra
spaces, building upon the invariance property of the Killing form. Furthermore,
we propose novel Lie bracket layers and geometric channel mixing layers that
extend the modeling capacity. Experiments are conducted for the
$\mathfrak{so}(3)$, $\mathfrak{sl}(3)$, and $\mathfrak{sp}(4)$ Lie algebras on
various tasks, including fitting equivariant and invariant functions, learning
system dynamics, point cloud registration, and homography-based shape
classification. Our proposed equivariant network shows wide applicability and
competitive performance in various domains.
Tzu-Yuan Lin, Minghan Zhu, Maani Ghaffari
# Lie Neurons: Adjoint-Equivariant Neural Networks for Semisimple Lie Algebras

###### Abstract

This paper proposes an adjoint-equivariant neural network that takes Lie algebra data as input. Various types of equivariant neural networks have been proposed in the literature, which treat the input data as elements in a vector space carrying certain types of transformations. In comparison, we aim to process inputs that are transformations between vector spaces. The change of basis on transformation is described by conjugations, inducing the adjoint-equivariance relationship that our model is designed to capture. Leveraging the invariance property of the Killing form, the proposed network is a general framework that works for arbitrary semisimple Lie algebras. Our network possesses a simple structure that can be viewed as a Lie algebraic generalization of a multi-layer perceptron (MLP). This work extends the application of equivariant feature learning. As an example, we showcase its value in homography modeling using \(\mathfrak{sl}(3)\) Lie algebra.

## 1 Introduction

Respecting the symmetry in data is essential for deep learning models to understand the underlying objects. The fundamental rules of a process and the definition of a concept are typically buried in certain types of symmetries. For example, the conservation of mechanical energy is independent of the choice of inertial reference frame. The internal angles of an equilateral triangle are 60 degrees regardless of its orientation and size. Equivariant networks refer to deep learning models that decouple certain types of transformations from the variance in the data so that the essential characteristics can be better learned from the preserved symmetry. The most well-known example of equivariance in deep learning is the convolutional neural networks (CNNs), which are translation-equivariant, enabling stable image features regardless of the pixel positions in the image plane. CNNs are prevailing, and we usually only call models that extend equivariance beyond translation equivariant networks. Typical extensions include rotation (Cohen et al., 2017) and scale (Worrall and Welling, 2019) equivariance, while more general extensions are also explored (MacDonald et al., 2022).

Compared with non-equivariant models, which impose no structure on the input data, equivariant models regard the input data as elements in a vector space (typically represented as vectors) that carries certain types of transformation (typically represented as matrices). The set of transformations often belongs to a group. A group action on the data is realized through (left) matrix multiplication on the vectors, which can be viewed as a change of basis for the vectors. The equivariance property preserves such structures in both the input and output spaces. This is a general formulation that is true for most existing equivariant models and has gained success in various domains, including but not limited to the modeling of molecules (Thomas et al., 2018), physical systems (Finzi et al., 2020), social networks (Maron et al., 2018), images (Worrall et al., 2017), and point clouds (Zhu et al., 2023).

In this paper, we propose a new type of equivariant model that captures the symmetry in conjugacy classes. _We view the input data of our model not as elements in a vector space but as transformations (maps) between vector spaces, typically represented as matrices for linear groups_. Accordingly, group action on the input data stands for a change of basis on the transformations, which is also known as conjugation. The input data as transformations, when viewed as a continuous symmetry group, form a Lie group. To facilitate the numerical computation, the proposed network takes the
transformations to the Lie algebra. The Lie algebraic approach is particularly attractive as it enables working in vector spaces and exploiting its isometry structure to handle data in typical vector-form data. However, we reiterate that our network models the conjugation equivariance on transformation input or, equivalently, the adjoint equivariance on the corresponding Lie algebra. One way to intuitively understand the relations between existing equivariant networks and our proposed network is illustrated in Figure 1.

Overall, the contributions of our paper are summarized as follows:

* We propose a new adjoint-equivariant network architecture, enabling the processing of input data that represent transformations.
* We develop new network designs using the Killing form and the Lie bracket structure for equivariant activation and invariant layers for Lie algebraic representation learning.
* The proposed network models the equivariance of any semisimple Lie algebras, which relaxes the requirement in previous work Deng et al. (2021) for the Lie group to be compact.
* We open-source our code at: [https://github.com/UMich-CURLY/LieNeurons](https://github.com/UMich-CURLY/LieNeurons).

## 2 Related Work

Equivariant networks enable the model output to change in a predicted way as the input goes through certain transformations. This means that the model, by construction, generalizes over the variations caused by those transformations. Therefore, it reduces the sampling complexity in learning and improves the robustness and transparency facing input variations. Cohen and Welling (2016) initials the idea to generalize equivariance in deep learning models, realizing equivariance to 90-degree rotations in a 2D image plane using group convolution. This method works with discrete transformations by augmenting the input domain with a dimension for the set of transformations. The approach is generalized to other discretized groups in \(SE(2)\), \(SE(3)\), and \(E(3)\)(Hoogeboom et al., 2018; Winkels and Cohen, 2018; Worrall and Brostow, 2018; Chen et al., 2021). Steerable convolution is proposed in Cohen and Welling (2016), leveraging the irreducible representations to remove the need for discretization and facilitate equivariant convolution on continuous groups in the frequency domain (Worrall et al., 2017; Cohen et al., 2017; Weiler et al., 2018). Beyond convolutions, more general equivariant network architectures are proposed, for example, Fuchs et al. (2020); Hutchinson et al. (2021); Chatzipantzis et al. (2022) for transformers and Batzner et al. (2022); Brandstetter et al. (2021) for message passing networks. Vector Neurons (Deng et al., 2021) designs a multi-layer perception (MLP) and graph network that generalize the scalar features to 3D features to realize

Figure 1: Comparison between existing equivariant networks and our work. Red represents the underlying objects to be studied by the models. For example, a commonly studies type of object in existing equivariant networks is shapes. For our work, the studied object is transformations. Given a reference frame, we obtain an observation of the studied object illustrated in yellow. For existing equivariant networks, the inputs are represented as vectors (including tensors). For our work, the inputs are represented as matrices. Change of basis, illustrated in blue, acts on vectors by left multiplication while acting on transformations by conjugation.
\(SO(3)\)-equivariance on spatial data. The above works mainly focus on compact groups, on which more general recipes for building equivariant layers that are not limited to a specific group are also proposed (Kondor and Trivedi, 2018; Cohen et al., 2019; Weiler and Cesa, 2019; Xu et al., 2022; Lang and Weiler, 2020). The extension of equivariance beyond compact groups is also explored. Finzi et al. (2021) constructs MLPs equivariant to arbitrary matrix groups using their finite-dimensional representations. With the Monte Carlo estimator, equivariant convolutions are generalized to matrix groups with surjective exponential maps (Finzi et al., 2020) and all finite-dimensional Lie groups (MacDonald et al., 2022), where Lie algebra is used to parameterize elements in the continuous Lie groups as a lifted domain from the input space.

Our model structure resembles the MLP style of Vector Neurons (Deng et al., 2021), but our work models the equivariance of arbitrary semisimple groups under adjoint actions. Lie algebra is the input space of our network, representing transformation data.

## 3 Preliminaries

In this section, we provide some preliminaries for Lie groups. Specifically, we focus on matrix Lie groups. For more detailed explanations, we refer the readers to Hall (2013); Rossmann (2006); Kirillov (2008).

### Lie Group and Lie Algebra

A Lie group \(\mathcal{G}\) is a smooth manifold whose elements satisfy the group axioms. Because of this, a special vector space naturally arises at the identity of every Lie group named the Lie algebra, denoted \(\mathfrak{g}\). A Lie algebra locally captures the structure of the Lie group. It is possible to move from a Lie group to a Lie algebra around the identity and vice versa. This is achieved by employing the exponential and log maps:

\[\text{Exp}:\quad\mathfrak{g}\to\mathcal{G},\quad X\mapsto\text{ Exp}(X) \tag{1}\] \[\text{Log}:\quad\mathcal{G}\to\mathfrak{g},\quad a\mapsto\text{ Log}(a). \tag{2}\]

For a matrix Lie group, the exponential map is the matrix exponential, and the log map is the matrix logarithm. In addition to the exponential map, every Lie algebra is equipped with an asymmetric binary operator called the Lie bracket:

\[[\cdot,\,]:\quad\mathfrak{g}\times\mathfrak{g}\to\mathfrak{g}. \tag{3}\]

The elements of the Lie algebra have non-trivial structures. However, since the Lie algebra is a vector space, for a Lie algebra of dimension \(m\), we can always represent it using \(\mathbb{R}^{m}\) given appropriate bases. As a result, we introduce two useful functions:

\[\text{Vee}:\quad\mathfrak{g}\to\mathbb{R}^{m},\quad x^{\wedge} \mapsto(x^{\wedge})^{\vee}=\sum_{i=1}^{m}x_{i}e_{i} \tag{4}\] \[\text{Hat}:\quad\mathbb{R}^{m}\to\mathfrak{g},\quad x\mapsto x^{ \wedge}=\sum_{i=1}^{m}x_{i}E_{i}, \tag{5}\]

where \(e_{i}\) are the canonical basis of \(\mathbb{R}^{m}\) and \(E_{i}=(e_{i})^{\wedge}\in\mathfrak{g}\). Using the Hat and Vee maps, we can represent an element of the Lie algebra in a neural network using \(\mathbb{R}^{m}\), while performing structure-preserving operations on \(\mathfrak{g}\).

### Adjoint Representation

Given an element of the Lie algebra \(X\in\mathfrak{g}\) and its corresponding Lie group \(\mathcal{G}\), every \(a\in\mathcal{G}\) defines an automorphism of the Lie algebra \(Ad_{a}:\mathfrak{g}\to\mathfrak{g}\) by

\[Ad_{a}(X)=aXa^{-1}. \tag{6}\]

This is called the adjoint action of the group \(\mathcal{G}\) on the Lie algebra \(\mathfrak{g}\). It represents the change of basis operations on the algebra. Since the adjoint \(Ad_{a}\) is linear, we can find a matrix that maps the \(\mathbb{R}^{m}\) representation of the Lie algebra to another. That is, for every \(Ad_{a}\) and \(X\in\mathfrak{g}\), we have

\[Adm_{a}:\quad\mathbb{R}^{m}\to\mathbb{R}^{m},\quad x\mapsto Adm_{a}x, \tag{7}\]
with \(Adm_{a}\in\mathbb{R}^{m\times m}\), \(x^{\wedge}=X\) and \(Adm_{a}x=(ax^{\wedge}a^{-1})^{\vee}\). This is an important property as it allows us to model the group adjoint action using a matrix multiplication on \(\mathbb{R}^{m}\), which enables the adjoint equivariant layer design in Section 4.

Conversely, if we view the \(Ad\) as a function of a group element \(a\in\mathcal{G}\), it maps the group element to a Lie algebra automorphism:

\[Ad:\mathcal{G}\rightarrow\mathtt{Aut}(\mathfrak{g}),\quad a\mapsto Ad_{a}. \tag{8}\]

This \(Ad\) is called the _adjoint representation_ of the group. Similarly, we can obtain the adjoint representation of the Lie algebra by differentiating the adjoint representation of the group at the identity:

\[ad:\mathfrak{g}\rightarrow\mathtt{Der}(\mathfrak{g}),\quad X\mapsto ad_{X}( \cdot)=[X,\cdot]\,, \tag{9}\]

where \(\mathtt{Der}(\mathfrak{g})\) is the Lie algebra of \(\mathtt{Aut}(\mathfrak{g})\), and \([\cdot,\cdot]\) is the Lie bracket of the Lie algebra. For a matrix group, the Lie bracket is defined by the commutator: \([X,Y]=XY-YX\). It is worth noticing that the Lie bracket is equivariant under the group adjoint action.

### Killing Form

If a Lie algebra \(\mathfrak{g}\) is of finite dimension and associated with a field \(\mathbb{R}\), a symmetric bilinear form called the _Killing form_ is defined as:

\[B(X,Y):\mathfrak{g}\times\mathfrak{g}\rightarrow\mathbb{R},\quad(X,Y) \mapsto\mathrm{tr}(ad_{X}\circ ad_{Y}) \tag{10}\]

**Definition 1**: _A bilinear form \(B(X,Y)\) is said to be non-degenerate iff \(B(X,Y)=0\) for all \(Y\in\mathfrak{g}\) implies \(X=0\)._

**Theorem 1** (Kirillov (2008)): _A Lie algebra is semisimple iff the Killing form is non-degenerate.1_

Footnote 1: This is also known as the _Cartan’s Criterion_.

**Theorem 2** (Kirillov (2008)): _The Killing form is invariant under the group adjoint action \(Ad_{a}\) for all \(a\in\mathcal{G}\), i.e.,_

\[B(Ad_{a}\circ X,Ad_{a}\circ Y)=B(X,Y).\]

If the Lie group is also compact, the Killing form is negative definite, and the inner product naturally arises from the negative of the Killing form.

## 4 Methodology

We present Lie Neurons (LN), a general adjoint-equivariant neural network on Lie algebras. It is greatly inspired by Vector Neurons (VN) (Deng et al., 2021). The Lie Neurons generalize the 3-dimensional \(SO(3)\) equivariant VN to a \(K\)-dimensional network that is equivariant by construction for any semisimple Lie algebra. Different from the VN, the Lie Neurons take elements of a Lie algebra as inputs, and they capture the equivariance of a Lie group under the adjoint action. We will discuss how the Lie Neuron almost specializes to the VN later in Section 4.5. Here, we start by describing the network structure of Lie Neurons.

The standard multilayer perceptron (MLP) networks are constructed with scalar neurons, \(z\in\mathbb{R}\). For each layer, the neurons are concatenated in the feature dimension into a \(C^{(d)}\)-dim vector \(\mathbf{z}\in\mathbb{R}^{C^{(d)}}\), where \(d\) denotes the layer index. Vector Neurons lift the neuron representation from scalars to \(\mathbb{R}^{3}\). For an input composed of a set of \(N\) points (e.g., a point cloud), the features learned from a VN layer are of shape \(\mathbb{R}^{3\times C^{(d)}\times N}\).

The Lie Neurons generalize Vector Neurons. Each Lie Neuron, \(X\in\mathfrak{g}\), is an element of a semisimple Lie algebra. A Lie algebra is a vector space with non-trivial structures. However, using (4), we can express the neuron in \(x=X^{\vee}\in\mathbb{R}^{K}\) with appropriate bases. Similar to Vector Neurons, the features learned from a Lie Neuron layer are \(\mathcal{X}^{(d)}=\{\mathbf{x}_{i}\}_{i=1}^{N}\in\mathbb{R}^{K\times C^{(d)} \times N}\), where \(\mathbf{x}_{i}\in\mathbb{R}^{K\times C^{(d)}}\).

By construction, the LN are equivariant to the group adjoint action (also known as similarity transform in matrix groups). In particular, for a given element \(a\) in a semisimple Lie group \(\mathcal{G}\), we have

\[f(a\mathcal{X}a^{-1};\theta)=af(\mathcal{X};\theta)a^{-1}, \tag{11}\]
where \(f\) is the function defined by an LN model with parameters \(\theta\). 2 The Lie Neurons framework consists of a linear layer, two nonlinear activation layers, a pooling layer, and an invariant layer. We start by discussing the linear layers as follows.

Footnote 2: We slightly abuse the notation here by setting \(a\mathcal{X}a^{-1}=\{\{(a(x_{ij})^{\wedge}a^{-1})^{\vee}\}_{i=1}^{C^{(d)}}\}_{j=1}^ {N}\).

### Linear Layers

Linear layers are the basic building blocks of an MLP. A linear layer has a learnable weight matrix \(\mathbf{W}\in\mathbb{R}^{C\times C^{\prime}}\), which operates on input features \(\mathbf{x}\in\mathbb{R}^{K\times C}\) by left matrix multiplication:

\[\mathbf{x}^{\prime}=f_{\text{LN-Linear}}(\mathbf{x};\mathbf{W})=\mathbf{x} \mathbf{W}\in\mathbb{R}^{K\times C^{\prime}}. \tag{12}\]

Recall (7), if we use the vector representation \(x\in\mathbb{R}^{K}\) for \(X\in\mathfrak{g}\), we can always find a linear adjoint matrix \(Adm_{a}\in\mathbb{R}^{K\times K}\)such that \(Adm_{a}x=aXa^{-1}\). As a result, the adjoint action on the linear layer becomes:

\[\begin{split} f_{\text{LN-Linear}}(Ad_{a}(\mathbf{x});\mathbf{W })&=f_{\text{LN-Linear}}(Adm_{a}\mathbf{x};\mathbf{W})\\ &=Adm_{a}\mathbf{x}\mathbf{W}\in\mathbb{R}^{K\times C^{\prime}} \\ &=Adm_{a}f_{\text{LN-Linear}}(\mathbf{x};\mathbf{W})\\ &=Ad_{a}(f_{\text{LN-Linear}}(\mathbf{x};\mathbf{W})),\end{split} \tag{13}\]

which proves the equivariant property of the linear layer. It is worth mentioning that we ignore the bias term to preserve the equivariance. Lastly, similar to the Vector Neurons, the weights may or may not be shared across the elements \(\mathbf{x}\) of \(\mathcal{X}\).

### Nonlinear Layers

Nonlinear layers enable the neural network to approximate complicated functions. We propose two designs for the equivariant nonlinear layers, LN-ReLU and LN-Bracket.

#### 4.2.1 LN-ReLU: Nonlinearity based on the Killing form

We can use an invariant function to construct an equivariant nonlinear layer. The VN leverages the inner product in a standard vector space, which is invariant to \(SO(3)\), to design a vector ReLU nonlinear layer. We generalize this idea by replacing the inner product with the negative of the Killing form. As described in Section 3, the negative Killing form falls back to the inner product for compact semisimple Lie groups, and it is invariant to the group adjoint action.

For an input \(\mathbf{x}\in\mathbb{R}^{K\times C}\), a Killing form \(B(\cdot,\cdot)\), and a learnable weight \(U\in\mathbb{R}^{C\times C}\), the nonlinear layer \(f_{\text{LN-ReLU}}\) is defined as:

\[f_{\text{LN-ReLU}}(\mathbf{x})=\begin{cases}\mathbf{x},&\text{if }B(x,d)\leq 0\\ \mathbf{x}+B(x,d)d,&\text{otherwise},\end{cases} \tag{14}\]

where \(d=\mathbf{x}U\in\mathbb{R}^{K\times C}\) is the learnable direction. Optionally, we can share the learned direction across channels by setting \(U\in\mathbb{R}^{C\times 1}\).

From Theorem 2, we know the Killing form is invariant under the group adjoint action, and the equivariance of the learned direction is proven in (13). Therefore, the second output of (14) becomes a linear combination of two equivariant quantities. As a result, the nonlinear layer is equivariant to the adjoint action.

We can also construct variants of ReLU, such as the leaky ReLU in the following form:

\[f_{\text{LN-LeakyReLU}}=\alpha\mathbf{x}+(1-\alpha)f_{\text{ReLU}}(\mathbf{x}). \tag{15}\]

#### 4.2.2 LN-Bracket: Nonlinearity based on the Lie bracket

As introduced in Section 3, Lie algebra is a vector space with an extra binary operator called Lie bracket, which is equivariant under group adjoint actions. For a matrix Lie group, the Lie bracket
of its Lie algebra is defined using the commutator: \([X,Y]=XY-YX\). We use this operation to build a novel nonlinear layer.

We use two learnable weight matrices \(U,V\in\mathbb{R}^{C\times C}\) to map the input to different Lie algebra vectors, \(\mathbf{u}=\mathbf{x}U,\mathbf{v}=\mathbf{x}V\). The Lie bracket of \(\mathbf{u}\) and \(\mathbf{v}\) becomes a nonlinear function on the input: \(\mathbf{x}\mapsto[(\mathbf{x}U)^{\wedge},(\mathbf{x}V)^{\wedge}]\). Theoretically, we can directly use it as our nonlinear layer. However, we note that the Lie bracket essentially captures _the failure of matrices to commute_. (Guggenheimer, 2012), and that \([X,X]=0,\forall X\). We find that when using two learnable Lie algebra elements from the same input, the Lie bracket cancels out most of the information and only passes through the residual. As a result, we add a residual path to enhance the information flow, inspired by ResNet (He et al., 2016). The final design of the LN-Bracket layer becomes:

\[f_{\text{LN-Bracket}}(\mathbf{x})=\mathbf{x}+[(\mathbf{x}U)^{\wedge},(\mathbf{ x}V)^{\wedge}]^{\vee}. \tag{16}\]

The nonlinear layer is often combined with a linear layer to form a module. In the rest of the paper, we will use LN-LR to denote an LN-Linear followed by an LN-ReLU, and LN-LB to denote an LN-Linear with an LN-Bracket layer.

### Pooling Layers

Pooling layers provide a means to aggregate global information across the \(N\) observation points within one measurement. This can be done by mean pooling, which is adjoint equivariant. In addition, we also introduce a max pooling layer. For each input \(\mathcal{X}=\{\mathbf{x}_{n}\}_{n=1}^{N}\in\mathbb{R}^{K\times C\times N}\), and a weight matrix \(\mathbf{W}\in\mathbb{R}^{C\times C}\), we learn a set of directions as: \(\mathcal{D}=\{\mathbf{x}_{n}\mathbf{W}\}_{n=1}^{N}\).

We again employ the Killing form, \(B(\cdot,\cdot)\), as the invariant function. For each channel \(c\in C\), we have the max pooling function as \(f_{\text{LN-Max}}(\mathbf{x}_{n})\left[c\right]=\mathbf{x}_{n^{*}}\left[c\right]\), where

\[n^{*}(c)=\operatorname*{arg\,max}_{n}\;B(\mathbf{x}_{n}\left[c\right]\mathbf{W },\mathbf{x}_{n}\left[c\right]). \tag{17}\]

\(\mathbf{x}_{n}\left[c\right]\) denotes the input at channel \(c\). Max pooling reduces the feature set from \(\mathcal{X}\in\mathbb{R}^{K\times C\times N}\) to \(\mathcal{X}\in\mathbb{R}^{K\times C\times 1}\). The layer is equivariant to the adjoint action due to the invariance of \(B(\cdot,\cdot)\).

### Invariant Layers

Equivariant layers allow steerable feature learning. However, some applications demand invariant features Lin et al. (2023); Zheng et al. (2022); Li et al. (2021). We introduce an invariant layer that can be attached to the network when necessary. Given an input \(\mathbf{x}\in\mathbb{R}^{K\times C}\), we have:

\[f_{\text{LN-law}}(\mathbf{x})=B(\mathbf{x},\mathbf{x}), \tag{18}\]

where \(B(\cdot,\cdot)\) is the adjoint-invariant Killing form, and \(f_{\text{LN-law}}(\mathbf{x})\in\mathbb{R}^{1\times C}\).

### Relationship to Vector Neurons

Our method can almost specialize to the Vector Neurons when working with \(\mathfrak{so}(3)\). This is because the linear adjoint matrix \(Adm_{a}\) is exactly the rotation matrix for \(\mathfrak{so}(3)\). Therefore, the group adjoint action becomes a left multiplication on the \(\mathbb{R}^{3}\) representation of the Lie algebra. Moreover, \(SO(3)\) is a compact group. Thus, the negative Killing form of \(\mathfrak{so}(3)\) defines an inner product. We would like to note that we omit the normalization in the ReLU layer because the norm is not well defined when the Killing form is not negative definite. We also do not implement a batch normalization layer. Therefore, the Lie Neurons do not reduce to VN completely when working with \(\mathfrak{so}(3)\). Despite the similarity in appearance, the \(\mathbb{R}^{3}\) vectors are viewed as vectors in Euclidean space subject to rotation matrix multiplication in Vector Neurons, while they are treated as \(\mathfrak{so}(3)\) Lie algebras in our framework, where the rotation matrices stand for conjugation. In addition, we propose a novel Lie bracket nonlinear layer.

## 5 Experiments

The Lie Neurons can be applied to any semisimple Lie algebra with a real matrix representation. This allows us to extend the network to operate on noncompact Lie algebras, such as the special linear Lie algebra \(\mathfrak{sl}(3)\). In this section, we instantiate the LN on \(\mathfrak{sl}(3)\), which can be represented using traceless matrices. The corresponding group, the special linear group \(SL(3)\), can be represented
using matrices with unit determinants. The special linear group has 8 degrees of freedom and can be used to model the homography transformation between images Hua et al. (2020); Zhan et al. (2022).

We perform three different experiments to verify the LN. We first solve a regression problem on an invariant function, where the function maps two \(\mathfrak{sl}(3)\) elements to a real number. In the second experiment, we fit an equivariant function that maps from \(\mathfrak{sl}(3)\) to \(\mathfrak{sl}(3)\). Lastly, we formulate a classification problem, where we classify among three Platonic solids. Across all three experiments, we compare our method with a standard 3-layer MLP by flattening the input to \(\mathbb{R}^{K\times C\times N}\). In addition, we set the feature dimension to 256 for all models.

### Invariant Function Regression

We begin our evaluation with an invariant function fitting experiment. Given \(X,Y\in\mathfrak{sl}(3)\), we ask the network to regress the following function:

\[g(X,Y)=\sin(\operatorname{tr}(XY))+\cos(\operatorname{tr}(YY))-\frac{ \operatorname{tr}(YY)^{3}}{2}+\det(XY)+\exp(\operatorname{tr}(XX)). \tag{19}\]

We randomly generate \(10,000\) training samples and \(10,000\) testing samples. In addition, in order to evaluate the invariance of the learned network, we randomly apply \(500\) group adjoint action to each test sample to generate augmented testing data.

In this task, we experiment with three different modules, LN-LR, LN-LB, and LN-LR + LN-LB, each followed by an LN-Inv and a final linear mapping from the feature dimension to a scalar. For each input, we concatenate \(X\) and \(Y\) in the feature dimension and have \(\mathcal{X}\in\mathbb{R}^{K\times C\times N}=\mathbb{R}^{8\times 2\times 1}\).

We report the Mean Squared Error (MSE) and the invariance error in Table 1. The invariance error \(E_{\text{inv}}\) is defined as:

\[E_{\text{inv}}:=\frac{\sum_{i=1}^{N_{x}}\sum_{j=1}^{N_{a}}f(\mathcal{X}_{i})-f (a_{j}\mathcal{X}_{i}a_{j}^{-1})}{N_{x}N_{a}}, \tag{20}\]

where \(a\in SL(3)\) are the randomly generated adjoint actions, \(N_{x}\) is the number of testing points, and \(N_{a}\) is the number of conjugations. The invariance error measures the adjoint invariance property of the networks.

From the table, we see that the LN outperforms MLP except for LN-LB. When tested on the \(SL(3)\) augmented test set, the performance of the LN remains consistent, while the error from the MLP increases significantly. The results of the invariance error demonstrate that the proposed method is invariant to the adjoint action while the MLP is not. In this experiment, we observe that LN-LR performs well on the invariant task, but the LN-LB alone does not. Nevertheless, if we combine both nonlinearities, the performance remains competitive.

### Equivariant Function Regression

In the second experiment, we ask the network to fit an equivariant function that takes two elements on \(\mathfrak{sl}(3)\) back to itself:

\[h(X,Y)=\left[\left[X,Y\right],Y\right]+\left[Y,X\right]. \tag{21}\]

Similar to the first experiment, we generate \(10,000\) training and test samples, as well as the additional \(500\) adjoint actions on the test set. We again report the MSE on the regular test set. For the adjoint-augmented test set, we map the output back with the inverse adjoint action and compute the

\begin{table}
\begin{tabular}{l|l l l l} \hline \hline  & MLP & LN-LR & LN-LB & LN-LR + LN-LB \\ \hline MSE \(Id/Id\)\(\downarrow\) & 0.143 & **0.103** & 0.558 & 0.115 \\ MSE \(Id/SL(3)\)\(\downarrow\) & 5.566 & **0.103** & 0.558 & 0.115 \\ Invariance Error \(\downarrow\) & 1.359 & 0.002 & \(\mathbf{4.9\times 10^{-5}}\) & \(9.0\times 10^{-4}\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: The mean squared errors and the invariant errors on the invariant function regression task. \(Id/Id\) denotes models trained and evaluated both on non-augmented data. \(Id/SL(3)\) denotes testing with data augmented by \(SL(3)\) adjoint action. \(\downarrow\) means the lower the better.
MSE with the ground truth value. To evaluate the equivariance of the network, we compute the equivariance error \(E_{\text{equiv}}\) as:

\[E_{\text{equiv}}:=\frac{\sum_{i=1}^{N_{x}}\sum_{j=1}^{N_{a}}a_{j}f(\mathcal{X}_{ i})a_{j}^{-1}-f(a_{j}\mathcal{X}_{i}a_{j}^{-1})}{N_{x}N_{a}}. \tag{22}\]

In this experiment, we evaluate LN using 3 different architectures. They are 2 LN-LR, 2 LN-LB, and 2 LN-LR + 2 LN-LB, respectively. Each of them is followed by a regular linear layer to map the feature dimension back to \(1\).

Table 2 lists the results of the equivariant experiment. We see that the MLP performs well on the regular test set but fails to generalize to the augmented data. Moreover, it has a high equivariance error. Our methods, on the other hand, generalize well on the adjoint-augmented data and achieve the lowest errors. The 2 LN-LB model performs the best.

From both the invariant and equivariant experiments, we observe that the LN-LR module works better on invariant tasks, while the LN-LB module performs better on the equivariant ones. We speculate this is because the LN-LR relies on the Killing form, which is an adjoint-invariant function, while the LN-LB leverages the Lie bracket, which is adjoint-equivariant. Nevertheless, if we combine both modules, the network performs favorably on both invariant and equivariant tasks.

### Platonic Solid Classification

Other than the numerical experiments above, we further design an experiment with practical meanings to hint at the real-world implications of the proposed network. The task is to classify polyhedrons from their projection on an image plane. While rotation equivariance naturally emerges for the 3D shape, the rotation equivariance relation is lost in the 2D projection of the 3D polyhedrons. Instead, the projection yields homography relations, which can be modeled using the \(SL(3)\) group Hua et al. (2020); Zhan et al. (2022). When projected onto an image plane, the two neighboring faces of a polyhedron can be described using homography transformations, which are different for each polyhedron type. Therefore, we use the homography transforms among the projected neighboring faces as the input for polyhedron classification.

Without loss of generality, we assume the camera intrinsic matrix \(K\) to be identity. In this case, given a homography matrix \(H\in SL(3)\) that maps one face to another in the image plane, the homography between these two faces becomes \(RHR^{-1}\) when we rotate the camera by \(R\in SO(3)\subset SL(3)\).

In this experiment, we use three types of Platonic solids: a tetrahedron, an octahedron, and an icosahedron. An input data point refers to the homography transforms between the projection of a pair of neighboring faces within one image. Figure 2 visualizes an example of the neighboring face pair for the three Platonic solids. The homographies of all neighboring face pairs form a complete set of data describing a Platonic solid. We use these data to learn a classification model of the three Platonic solids. During training, we fix the camera and object pose. Then, we test with the original pose and with rotated camera poses to verify the equivariance property of our models.

We once again test with LN-LR, LN-LB, and LN-LR+LN-LB. Each of them is followed by an LN-Max layer, an LN-Inv layer, and a final linear mapping.

Table 3 shows the classification accuracy. The LN achieves higher accuracy than the MLP. Since the MLP is not invariant to the adjoint action, its accuracy drops drastically when the camera is rotated. We also notice that the LN-LB performs slightly worse than the other two formulations. This agrees with our previous observations, as the classification tasks rely mostly on invariant features.

\begin{table}
\begin{tabular}{l|c c c c} \hline \hline  & MLP & 2 LN-LR & 2 LN-LB & 2 LN-LR + 2 LN-LB \\ \hline MSE \(Id/Id\downarrow\) & 0.009 & 0.213 & \(\mathbf{9.6\times 10^{-10}}\) & \(2.2\times 10^{-6}\) \\ MSE \(Id/SL(3)\downarrow\) & 2.025 & 0.213 & \(\mathbf{4.5\times 10^{-8}}\) & \(2.2\times 10^{-6}\) \\ Equivariance Error \(\downarrow\) & 0.445 & \(1.0\times 10^{-4}\) & \(\mathbf{6.5\times 10^{-8}}\) & \(7.9\times 10^{-5}\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: The mean squared errors and the equivariant errors in equivariant function regression.
### Ablation Study

We introduce the LN-Bracket layer in Section 4.2.2 and discuss how the residual connection improves the performance. In this subsection, we perform ablation studies on an alternative Lie bracket nonlinear layer design without the residual connection. That is, \(f_{\text{LN-Bracket-N}}(\mathbf{x})=[(\mathbf{x}U)^{\wedge},(\mathbf{x}V^{ \wedge})^{\wedge}]^{\vee}\). We denote this nonlinear layer combined with an LN-Linear as LN-LBN and show the results of this method in Table 4. From the table, we can clearly see the benefits of having the residual connection in the Lie bracket layer.

## 6 Discussion and Limitations

Lie Neurons is a group adjoint equivariant network by construction. It does not require the Lie group to be compact. However, the LN-ReLU layer relies on a non-degenerated Killing form. As a result, the current formulation can only operate on semisimple Lie algebras. Secondly, the Lie Neurons take elements on the Lie algebra as inputs, but most modern sensors return measurements in standard vector spaces. More practical applications of the proposed work are yet to be explored. Lastly, although the Lie Neurons contain only simple layer structures, their scalability is yet to be verified.

## 7 Conclusion

In this paper, we propose an adjoint-equivariant network, Lie Neurons. Compared with existing equivariant models, our proposed framework extends the scope of equivariance by modeling the symmetry of change-of-basis on _transformations_, rather than vectors. Our model is generally applicable to any semisimple Lie groups, compact or non-compact. Our network builds upon simple MLP-style layers. To facilitate the learning of expressive Lie algebraic features, we propose equivariant nonlinear activation functions based on the Killing form and the Lie bracket. We also design an equivariant pooling layer and an invariant layer to extract global equivariant features and invariant features. In the experiments, we verify the equivariance property and the ability to fit equivariant and invariant targets of our model in regression and classification tasks, with \(\mathfrak{sl}(3)\) as an example. We believe the new paradigm of transformation feature learning could open new possibilities in both equivariant modeling and more general deep learning.

\begin{table}
\begin{tabular}{l|l l l l} \hline \hline  & MLP & LN-LR & LN-LB & LN-LR + LN-LB \\ \hline Accuracy \(\uparrow\) & 0.967 & 0.994 & 0.986 & **0.998** \\ Accuracy (Rotated) \(\uparrow\) & 0.385 & 0.994 & 0.979 & **0.997** \\ \hline \hline \end{tabular}
\end{table}
Table 3: The accuracy of the Platonic solid classification task using the inter-face homography transforms in the image plane as inputs. \(\uparrow\) means the higher, the better.

Figure 2: A visualization of the three Platonic solids in our classification task. The yellow and blue colors highlight a neighboring pair of faces, between which the homography transforms in the image plane are taken as input to our models.