Clifford Group Equivariant Neural Networks
We introduce Clifford Group Equivariant Neural Networks: a novel approach for
constructing $\mathrm{O}(n)$- and $\mathrm{E}(n)$-equivariant models. We
identify and study the $\textit{Clifford group}$, a subgroup inside the
Clifford algebra tailored to achieve several favorable properties. Primarily,
the group's action forms an orthogonal automorphism that extends beyond the
typical vector space to the entire Clifford algebra while respecting the
multivector grading. This leads to several non-equivalent subrepresentations
corresponding to the multivector decomposition. Furthermore, we prove that the
action respects not just the vector space structure of the Clifford algebra but
also its multiplicative structure, i.e., the geometric product. These findings
imply that every polynomial in multivectors, An advantage worth mentioning is
that we obtain expressive layers that can elegantly generalize to inner-product
spaces of any dimension. We demonstrate, notably from a single core
implementation, state-of-the-art performance on several distinct tasks,
including a three-dimensional $n$-body experiment, a four-dimensional
Lorentz-equivariant high-energy physics experiment, and a five-dimensional
convex hull experiment.
David Ruhe, Johannes Brandstetter, Patrick Forré
# Clifford Group Equivariant Neural Networks

###### Abstract

We introduce Clifford Group Equivariant Neural Networks: a novel approach for constructing \(\mathrm{E}(n)\)-equivariant networks. We identify and study the _Clifford group_, a subgroup inside the Clifford algebra, whose definition we slightly adjust to achieve several favorable properties. Primarily, the group's action forms an orthogonal automorphism that extends beyond the typical vector space to the entire Clifford algebra while respecting the multivector grading. This leads to several non-equivalent subrepresentations corresponding to the multivector decomposition. Furthermore, we prove that the action respects not just the vector space structure of the Clifford algebra but also its multiplicative structure, i.e., the geometric product. These findings imply that every polynomial in multivectors, including their grade projections, constitutes an equivariant map with respect to the Clifford group, allowing us to parameterize equivariant neural network layers. Notable advantages are that these layers operate directly on a vector basis and elegantly generalize to any dimension. We demonstrate, notably from a single core implementation, state-of-the-art performance on several distinct tasks, including a three-dimensional \(n\)-body experiment, a four-dimensional Lorentz-equivariant high-energy physics experiment, and a five-dimensional convex hull experiment.

## 1 Introduction

Incorporating _group equivariance_ to ensure symmetry constraints in neural networks has been a highly fruitful line of research [19; 89; 20; 59; 86; 88; 30; 14; 87; 16]. Besides translation and permutation equivariance [92; 72], rotation equivariance proved to be vitally important for many graph-structured problems as encountered in, e.g., the natural sciences. Representative works include Tensor Field Networks [79], LieConv [31], and EGNNs [71]. Applications of such methods include modeling the dynamics of complex physical systems or motion trajectories [53; 12]; studying or generating molecules, proteins, and crystals [68; 39; 18; 73; 94; 80; 4]; and point cloud analysis [90; 81]. Note that many of these focus on three-dimensional problems involving rotation, reflection, or translation equivariance by considering, e.g., the groups \(\mathrm{O}(3)\), \(\mathrm{SO}(3)\), or \(\mathrm{E}(3)\).

Such equivariant neural networks can be broadly divided into three categories: approaches that scalarize geometric quantities, methods employing regular group representations, and those utilizing irreducible representations, often of \(\mathrm{O}(3)\)[42]. Scalarization methods operate exclusively on scalar features or manipulate higher-order geometric quantities such as vectors via scalar multiplication [73; 23; 54; 57; 71; 50; 35; 74; 26; 47; 78]. Such methods can be limited by the fact that they do not extract all directional information. Regular representation methods construct equivariant
transformations through an integral over the respective group [19; 59]. For continuous Lie groups, however, this integral is intractable and requires coarse approximation [31; 7]. Methods of the third category employ the irreducible representations of \(\mathrm{O}(3)\) (the Wigner-D matrices) and operate in a _steerable_ spherical harmonics basis [79; 2; 34; 12; 5]. This basis allows a decomposition into type-\(l\) vector subspaces that transform under \(D^{l}\): the type-\(l\) matrix representation of \(\mathrm{O}(3)\)[60; 33]. Using tensor products decomposed using Clebsch-Gordan coefficients (Clebsch-Gordan tensor product), vectors (of different types) interact equivariantly. These tensor products can be parameterized using learnable weights. Key limitations of steerable methods include the necessity for an alternative basis, along with acquiring the Clebsch-Gordan coefficients, which, although they are known for unitary groups of any dimension [51], are not trivial to obtain [1].

We propose _Clifford Group Equivariant Neural Networks_ (CGENNs): an equivariant parameterization of neural networks based on _Clifford algebras_. Inside the algebra, we identify the _Clifford group_ and its action, termed the _adjusted twisted conjugation_, which has several advantageous properties. Unlike classical approaches that represent these groups on their corresponding vector spaces, we carefully extend the action to the entire Clifford algebra. There, it automatically acts as an _orthogonal automorphism_ that respects the multivector grading [11; 70], enabling nontrivial subrepresentations that operate on the algebra subspaces. Furthermore, the adjusted twisted conjugation respects the Clifford algebra's multiplicative structure, specifically the _geometric product_, allowing us to bypass the need for explicit tensor product representations. As a result, we obtain two remarkable properties. First, all polynomials in multivectors generate Clifford group equivariant maps from the Clifford algebra to itself. Additionally, _grade projection_ also is an equivariant operation, allowing for a denser parameterization of such polynomials. We then demonstrate how one can construct parameterizable neural network layers using these properties.

Our method comes with several advantages. First, instead of operating on alternative basis representations such as the spherical harmonics, CGENNs (similarly to scalarization methods) directly transform data in a vector basis. Second, we can extract and transform higher-order features that carry vector-valued information, resulting in a more accurate and nuanced interpretation of the underlying structures compared to scalarization methods. Finally, our method readily generalizes to orthogonal groups regardless of the dimension or metric signature of the space. These advantages are demonstrated on equivariance benchmarks of different dimensionality. Note that specialized tools were developed for many of these tasks, while CGENNs can be applied more generally.

## 2 The Clifford Algebra

Clifford algebras (also known as _geometric algebras_) are powerful mathematical objects with applications in various areas of science and engineering. For a complete formal construction, we

Figure 1: CGENNs (represented with \(\phi\)) are able to operate on multivectors (elements of the Clifford algebra) in an \(\mathrm{E}(n)\)-equivariant way. Specifically, when an action \(\rho(w)\) of the Clifford group, representing an orthogonal transformation such as a rotation, is applied to the data, the model’s representations _corotate_. Multivectors can be decomposed into scalar, vector, bivector, trivector, and even higher-order components. These elements can represent geometric quantities such as areas or volumes. The action \(\rho(w)\) is designed to respect these structures when acting on them.
refer the reader to Appendix C. Let \(V\) be a finite-dimensional vector space over a field \(\mathbb{F}\) equipped with a _quadratic form_\(\mathfrak{q}:V\to\mathbb{F}\). The _Clifford algebra_\(\operatorname{Cl}(V,\mathfrak{q})\) is the unitary, associative, non-commutative algebra generated by \(V\) such that for every \(v\in V\) the relation \(v^{2}=\mathfrak{q}(v)\) holds, i.e., _vectors square to scalars_. This simple property solely generates a unique mathematical theory that underpins many applications. Note that every element \(x\) of the Clifford algebra \(\operatorname{Cl}(V,\mathfrak{q})\) is a linear combination of (formal, non-commutative) products of vectors modulo the condition that every appearing square \(v^{2}\) gets identified with the scalar square \(\mathfrak{q}(v)\): \(x=\sum_{i\in I}c_{i}\cdot v_{i,1}\cdots v_{i,k_{i}}\)1. Here, the index set \(I\) is finite, \(c_{i}\in\mathbb{F}\), \(k\in\mathbb{N}_{0}\), \(v_{i,j}\in V\). The Clifford algebra's associated _bi-linear form_\(\mathfrak{b}(v_{1},v_{2}):=\frac{1}{2}\left(\mathfrak{q}(v_{1}+v_{2})- \mathfrak{q}(v_{1})-\mathfrak{q}(v_{2})\right)\) yields the _fundamental Clifford identity_: \(v_{1}v_{2}+v_{2}v_{1}=2\mathfrak{b}(v_{1},v_{2})\) for \(v_{1},v_{2}\in V\) (Lemma C.3). In this context, the quantity \(v_{1}v_{2}\) represents the _geometric product_, which is aptly named for its ability to compute geometric properties and facilitate various transformations. Note that when \(v_{1},v_{2}\) are orthogonal (e.g., for orthogonal basis vectors), \(\mathfrak{b}(v_{1},v_{2})=0\), in which case \(v_{1}v_{2}=-v_{2}v_{1}\). The dimensionality of the algebra is \(2^{n}\), where \(n:=\dim V\) (Theorem C.15). Let \(e_{1},\ldots,e_{n}\) be an orthogonal basis of \(V\). The tuple \((e_{A})_{A\subseteq[n]},[n]:=\{1,\ldots,n\}\), is an orthogonal basis for \(\operatorname{Cl}(V,\mathfrak{q})\), where for all such \(A\) the product \(e_{A}:=\prod_{i\in A}e_{i}\) is taken in increasing order (Theorem C.26). We see below that we can decompose the algebra into vector subspaces \(\operatorname{Cl}^{(m)}(V,\mathfrak{q}),m=0,\ldots,n\), called _grades_, where \(\dim\operatorname{Cl}^{(m)}(V,\mathfrak{q})=\binom{n}{m}\). Elements of grade \(m=0\) and \(m=1\) are scalars (\(\operatorname{Cl}^{(0)}(V,\mathfrak{q})=\mathbb{F}\)) and vectors (\(\operatorname{Cl}^{(1)}(V,\mathfrak{q})=V\)), respectively, while elements of grade \(m=2\) and \(m=3\) are referred to as _bivectors_ and _trivectors_. Similar terms are used for elements of even higher grade. These higher-order grades can represent (oriented) points, areas, and volumes, as depicted in Figure 1.

Footnote 1: If \(k_{i}=0\) the product of vectors \(v_{i,1}\cdots v_{i,k_{i}}\) is empty, and, in this case, we mean the unit \(1\) in \(\operatorname{Cl}(V,\mathfrak{q})\).

Clifford algebras provide tools that allow for meaningful algebraic representation and manipulation of geometric quantities, including areas and volumes [69; 29; 28]. In addition, they offer generalizations such as extensions of the exterior and Grassmannian algebras, along with the natural inclusion of complex numbers and Hamilton's quaternions [40; 41]. Applications of Clifford algebras can be found in robotics [6; 43], computer graphics [84; 13], signal processing [44; 8], and animation [45; 17]. In the context of machine learning, Clifford algebras have been employed to improve the performance of algorithms in various tasks. For example, [63] learn an equivariant embedding using _geometric neurons_ used for classification tasks. Further, [76] introduce geometric algebra attention networks for point cloud problems in physics, chemistry, and biology. More recently, [11] introduce Clifford neural layers and Clifford Fourier transforms for accelerating solving partial differential equations. [70] continue this direction, strengthening the geometric inductive bias by the introduction of geometric templates. Further, [61; 64; 93] introduce complex-valued and quaternion-valued networks. Finally, [56] define normalizing flows on the group of unit quaternions for sampling molecular crystal structures.

## 3 Theoretical Results

In order to construct equivariant multivector-valued neural networks, we outline our theoretical results. We first introduce the following theorem regarding the multivector grading of the Clifford algebra, which is well-known in case the algebra's metric is non-degenerate. Although the general case, including a potentially degenerate metric, appears to be well-known, we were unable to find a self-contained proof during our studies. Hence, we include it here for completeness.

**Theorem 3.1** (The multivector grading of the Clifford algebra).: _Let \(e_{1},\ldots,e_{n}\) and \(b_{1},\ldots,b_{n}\) be two orthogonal bases of \((V,\mathfrak{q})\). Then the following sub-vector spaces \(\operatorname{Cl}^{(m)}(V,\mathfrak{q})\) of \(\operatorname{Cl}(V,\mathfrak{q})\), \(m=0,\ldots,n\), are independent of the choice of the orthogonal basis, i.e.,_

\[\operatorname{Cl}^{(m)}(V,\mathfrak{q}):=\operatorname{span}\left\{e_{A}\,|\,A \subseteq[n],|A|=m\right\}\stackrel{{!}}{{=}}\operatorname{ span}\left\{b_{A}\,|\,A\subseteq[n],|A|=m\right\}. \tag{1}\]

The proof can be found in Theorem C.27. We now claim that the Clifford algebra \(\operatorname{Cl}(V,\mathfrak{q})\) decomposes into an orthogonal direct sum of the vector subspaces \(\operatorname{Cl}^{(m)}(V,\mathfrak{q})\). To this end, we need to extend the bilinear form \(\mathfrak{b}\) from \(V\) to \(\operatorname{Cl}(V,\mathfrak{q})\). For elements \(x_{1},x_{2},x\in\operatorname{Cl}(V,\mathfrak{q})\), the _extended bilinear form_\(\bar{\mathfrak{b}}\) and the _extended quadratic form_\(\bar{\mathfrak{q}}\) are given via the projection onto the zero-component, where
\(\beta:\operatorname{Cl}(V,\mathfrak{q})\to\operatorname{Cl}(V,\mathfrak{q})\) denotes the _main anti-involution_ of \(\operatorname{Cl}(V,\mathfrak{q})\)2 :

Footnote 2: Recall that any \(x\in\operatorname{Cl}(V,\mathfrak{q})\) can be written as a linear combination of vector products. \(\beta\) is the map that reverses the order: \(\beta\left(\sum_{i\in I}c_{i}\cdot v_{i,1}\cdots v_{i,k_{i}}\right):=\sum_{i\in I }c_{i}\cdot v_{i,k_{i}}\cdots v_{i,1}\). For details, see Definition C.18.

\[\bar{\mathfrak{b}}:\operatorname{Cl}(V,\mathfrak{q})\times\operatorname{Cl}(V,\mathfrak{q})\to\mathbb{F},\hskip 28.452756pt\bar{\mathfrak{b}}(x_{1},x_{2}):=( \beta(x_{1})x_{2})^{(0)},\hskip 28.452756pt\bar{\mathfrak{q}}(x):=\bar{ \mathfrak{b}}(x,x). \tag{2}\]

Note that by construction, both \(\bar{\mathfrak{b}}\) and \(\bar{\mathfrak{q}}\) reduce to their original versions when restricted to \(V\). Using the extended quadratic form, the tuple \((\operatorname{Cl}(V,\mathfrak{q}),\bar{\mathfrak{q}})\) turns into a quadratic vector space in itself. As a corollary (see Corollary C.29), the Clifford algebra has an orthogonal-sum decomposition w.r.t. the extended bilinear form \(\bar{\mathfrak{b}}\):

\[\operatorname{Cl}(V,\mathfrak{q})=\bigoplus_{m=0}^{n}\operatorname{Cl}^{(m)}( V,\mathfrak{q}),\hskip 56.905512pt\dim\operatorname{Cl}^{(m)}(V, \mathfrak{q})=\binom{n}{m}. \tag{3}\]

This result implies that we can always write \(x\in\operatorname{Cl}(V,\mathfrak{q})\) as \(x=x^{(0)}+x^{(1)}+\cdots+x^{(n)}\), where \(x^{(m)}\in\operatorname{Cl}^{(m)}(V,\mathfrak{q})\) denotes the grade-\(m\) part of \(x\). Selecting a grade defines an orthogonal projection:

\[(\_)^{(m)}:\operatorname{Cl}(V,\mathfrak{q})\to\operatorname{Cl}^{(m)}(V, \mathfrak{q}),\hskip 36.135ptx\mapsto x^{(m)},\hskip 36.135ptm=0,\ldots,n. \tag{4}\]

Let us further introduce the notation \(\operatorname{Cl}^{[0]}(V,\mathfrak{q}):=\bigoplus_{m\text{ even}}^{n} \operatorname{Cl}^{(m)}(V,\mathfrak{q})\), whose elements are of _even parity_, and \(\operatorname{Cl}^{[1]}(V,\mathfrak{q}):=\bigoplus_{m\text{ odd}}^{n} \operatorname{Cl}^{(m)}(V,\mathfrak{q})\) for those of _odd parity_. We use \(x=x^{[0]}+x^{[1]}\) to denote the parity decomposition of a multivector.

### The Clifford Group and its Clifford Algebra Representations

Let \(\operatorname{Cl}^{\times}(V,\mathfrak{q})\) denote the group of invertible elements of the Clifford algebra, i.e., the set of those elements \(w\in\operatorname{Cl}(V,\mathfrak{q})\) that have an inverse \(w^{-1}\in\operatorname{Cl}(V,\mathfrak{q})\): \(w^{-1}w=ww^{-1}=1\). For \(w\in\operatorname{Cl}^{\times}(V,\mathfrak{q})\), we then define the _adjusted twisted conjugation_ as follows:

\[\rho(w):\operatorname{Cl}(V,\mathfrak{q})\to\operatorname{Cl}(V,\mathfrak{q}),\hskip 36.135pt\rho(w)(x):=wx^{[0]}w^{-1}+\alpha(w)x^{[1]}w^{-1}, \tag{5}\]

where \(\alpha\) is the _main involution_ of \(\operatorname{Cl}(V,\mathfrak{q})\), which is given by \(\alpha(w):=w^{[0]}-w^{[1]}\). This map \(\rho(w):\operatorname{Cl}(V,\mathfrak{q})\to\operatorname{Cl}(V,\mathfrak{q})\), notably not just from \(V\to V\), will be essential for constructing equivariant neural networks operating on the Clifford algebra. In general, \(\rho\) and similar maps defined in the literature do not always posses the required properties (see Motivation D.1). However, when our \(\rho\) is restricted to a carefully chosen subgroup of \(\operatorname{Cl}^{\times}(V,\mathfrak{q})\), many desirable characteristics emerge. This subgroup will be called the _Clifford group3_ of \(\operatorname{Cl}(V,\mathfrak{q})\) and we define it as:

Footnote 3: We elaborate shortly on the term _Clifford group_ in contrast to similar definitions in Remark D.14.

\[\Gamma^{[\times]}(V,\mathfrak{q}):=\left\{w\in\operatorname{Cl}^{\times}(V, \mathfrak{q})\cap\left(\operatorname{Cl}^{[0]}(V,\mathfrak{q})\cup\operatorname {Cl}^{[1]}(V,\mathfrak{q})\right)\Big{|}\,\forall v\in V.\,\rho(w)(v)\in V \right\}. \tag{6}\]

In words, \(\Gamma^{[\times]}(V,\mathfrak{q})\) contains all invertible (parity) homogeneous elements that preserve vectors (\(m=1\) elements) via \(\rho\). The _special Clifford group_ is defined as \(\Gamma^{[0]}(V,\mathfrak{q}):=\Gamma^{[\times]}(V,\mathfrak{q})\cap \operatorname{Cl}^{[0]}(V,\mathfrak{q})\).

Regarding the adjusted twisted conjugation, \(\rho(w)\) was ensured to reduce to a _reflection_ when restricted to \(V-\) a property that we will conveniently use in the upcoming section. Specifically, when \(w,x\in\operatorname{Cl}^{(1)}(V,\mathfrak{q})=V\), \(w\in\operatorname{Cl}^{\times}(V,\mathfrak{q})\), then \(\rho(w)\) reflects \(x\) in the hyperplane that is normal to \(w\):

\[\rho(w)(x)=-wxw^{-1}\stackrel{{!}}{{=}}x-2\frac{\mathfrak{b}(w,x)}{ \mathfrak{b}(w,w)}w. \tag{7}\]

Next, we collect several advantageous identities of \(\rho\) in the following theorem, which we elaborate on afterwards. For proofs, consider Lemma D.8, Theorem D.10, and Theorem D.28.

**Theorem 3.2**.: _Let \(w_{1},w_{2},w\in\Gamma^{[\times]}(V,\mathfrak{q})\), \(x_{1},x_{2},x\in\operatorname{Cl}(V,\mathfrak{q})\), \(m\in\{0,\ldots,n\}\). \(\rho\) then satisfies:_

1. _Additivity:_ \(\rho(w)(x_{1}+x_{2})=\rho(w)(x_{1})+\rho(w)(x_{2})\)_,_
2. _Multiplicativity:_ \(\rho(w)(x_{1}x_{2})=\rho(w)(x_{1})\rho(w)(x_{2})\)_,_
3. _Invertibility:_ \(\rho(w^{-1})(x)=\rho(w)^{-1}(x)\)_,_
_._
4. _Composition:_ \(\rho(w_{2})\left(\rho(w_{1})(x)\right)=\rho(w_{2}w_{1})(x)\)_,_
5. _Orthogonality:_ \(\bar{\mathfrak{b}}\left(\rho(w)(x_{1}),\rho(w)(x_{2})\right)=\bar{\mathfrak{b}} \left(x_{1},x_{2}\right)\)_._

The first two properties state that \(\rho(w)\) is not only _additive_, but even _multiplicative_ regarding the geometric product. The third states that \(\rho(w)\) is invertible, making it an _algebra automorphism_ of \(\operatorname{Cl}(V,\mathfrak{q})\).

The fourth property then states that \(\rho:\,\Gamma^{[\times]}(V,\mathfrak{q})\to\operatorname{Aut}_{\mathbf{Alg}} (\operatorname{Cl}(V,\mathfrak{q}))\) is also a _group homomorphism_ to the group of all algebra automorphisms. In other words, \(\operatorname{Cl}(V,\mathfrak{q})\) is a linear representation of \(\Gamma^{[\times]}(V,\mathfrak{q})\) and, moreover, it is also an algebra representation of \(\Gamma^{[\times]}(V,\mathfrak{q})\). Finally, the last point shows that each \(\rho(w)\) generates an orthogonal map with respect to the extended bilinear form \(\bar{\mathfrak{b}}\). These properties yield the following results (see also Theorem D.16 and Corollary D.18).

**Corollary 3.3** (All grade projections are Clifford group equivariant).: _For \(w\in\Gamma^{[\times]}(V,\mathfrak{q})\), \(x\in\operatorname{Cl}(V,\mathfrak{q})\) and \(m=0,\dots,n\) we have the following equivariance property:_

\[\rho(w)(x^{(m)})=\left(\rho(w)(x)\right)^{(m)}. \tag{8}\]

_In particular, for \(x\in\operatorname{Cl}^{(m)}(V,\mathfrak{q})\) we also have \(\rho(w)(x)\in\operatorname{Cl}^{(m)}(V,\mathfrak{q})\)._

This implies that the grade projections: \(\operatorname{Cl}(V,\mathfrak{q})\to\operatorname{Cl}^{(m)}(V,\mathfrak{q})\) are \(\Gamma^{[\times]}(V,\mathfrak{q})\)-equivariant maps, and, that each \(\operatorname{Cl}^{(m)}(V,\mathfrak{q})\) constitutes an orthogonal representation of \(\Gamma^{[\times]}(V,\mathfrak{q})\). The latter means that we have group homomorphisms from the Clifford group to the group of all orthogonal invertible linear transformations of \(\operatorname{Cl}^{(m)}(V,\mathfrak{q})\): \(\rho:\,\Gamma^{[\times]}(V,\mathfrak{q})\to\operatorname{O}\left(\operatorname {Cl}^{(m)}(V,\mathfrak{q}),\bar{\mathfrak{q}}\right)\).

**Corollary 3.4** (All polynomials are Clifford group equivariant).: _Let \(F\in\mathbb{F}[T_{1},\dots,T_{\ell}]\) be a polynomial in \(\ell\) variables with coefficients in \(\mathbb{F}\), \(w\in\Gamma^{[\times]}(V,\mathfrak{q})\). Further, consider \(\ell\) elements \(x_{1},\dots,x_{\ell}\in\operatorname{Cl}(V,\mathfrak{q})\). Then we have the following equivariance property:_

\[\rho(w)\left(F(x_{1},\dots,x_{\ell})\right)=F(\rho(w)(x_{1}),\dots,\rho(w)(x_{ \ell})). \tag{9}\]

To prove that \(\rho(w)\) distributes over any polynomial, we use both its additivity and multiplicativity regarding the geometric product.

By learning the coefficients of such polynomials, we have achieved flexible parameterizations of Clifford group equivariant layers for Clifford algebras over quadratic vector spaces of any dimension or metric. We further involve grade projections to achieve denser parameterizations (see Figure 2). More details regarding neural network constructions follow in Section 4.

### Orthogonal Representations

To relate the theory above to the classical orthogonal group, we will restrict \(\rho(w)\) in this section to \(V\) for \(w\in\Gamma^{[\times]}(V,\mathfrak{q})\). Further, we restrict \((V,\mathfrak{q})\) to be _non-degenerate_ as we never consider degenerate quadratic forms in our experiments. In the supplementary material, however, we are more general.

**Theorem 3.5** (The adjusted twisted conjugation acting on \(V\)).: _The range of the Clifford group \(\Gamma^{[\times]}(V,\mathfrak{q})\) under the adjusted twisted conjugation \(\rho\), when restricted to act on \(V\), coincides with the group of all orthogonal (invertible) linear maps \(\operatorname{O}(V,\mathfrak{q})\) of \((V,\mathfrak{q})\). Its kernel is given by \(\ker\rho=\mathbb{F}^{\times}\)._

\[\operatorname{ran}\left(\rho:\,\Gamma^{[\times]}(V,\mathfrak{q})\to \operatorname{GL}(V)\right)=\operatorname{O}(V,\mathfrak{q}). \tag{10}\]

Figure 2: Commutative diagrams expressing Clifford group equivariance with respect to the main operations: polynomials \(F\) (left) and grade projections \((\_)^{(m)}\) (right).
This theorem allows one to provide explicit descriptions of the element of the Clifford group, see Corollary D.26. To show Theorem 3.5, we invoke _Cartan-Dieudonne_ (Theorem B.13), stating that every orthogonal transformation decomposes into compositions of reflections, and recall that \(\rho(w)\) reduces to a reflection when restricted to \(V\), see Theorem D.24. Consequently, **equivariance w.r.t. the Clifford group**\(\Gamma^{[\times]}(V,\mathfrak{q})\), acting on \(\mathrm{Cl}(V,\mathfrak{q})\) via \(\rho\), rules as in Theorem 3.2, **implies equivariance w.r.t. the orthogonal group**\(\mathrm{O}(V,\mathfrak{q})\), acting on \(\mathrm{Cl}(V,\mathfrak{q})\) via \(\rho\), (Remark D.30).

Finally, it is worth noting that our method is also \(\mathrm{Pin}\) and \(\mathrm{Spin}\) group-equivariant4. These groups, sharing properties with the Clifford group, are also often studied in relation to the orthogonal group.

Footnote 4: We discuss in Appendix D.6 a general definition of these groups.

## 4 Methodology

We restrict our layers to use \(\mathbb{F}:=\mathbb{R}\). Our method is most similar to steerable methods such as [12]. However, unlike these works, we do not require an alternative basis representation based on spherical harmonics, nor do we need to worry about Clebsch-Gordan coefficients. Instead, we consider simply a steerable vector basis for \(V\), which then automatically induces a _steerable multivector basis_ for \(\mathrm{Cl}(V,\mathfrak{q})\). By steerability, we mean that this basis can be transformed in a predictable way under an action from the Clifford group, which acts orthogonally on both \(V\) and \(\mathrm{Cl}(V,\mathfrak{q})\) (see Figure 1).

We present layers yielding Clifford group-equivariant optimizable transformations. All the main ideas are based on Corollary 3.3 and Corollary 3.4. It is worth mentioning that the methods presented here form a first exploration of applying our theoretical results, making future optimizations rather likely.

**Linear Layers** Let \(x_{1},\ldots,x_{\ell}\in\mathrm{Cl}(V,\mathfrak{q})\) be a tuple of multivectors expressed in a steerable basis, where \(\ell\) represents the number of input channels. Using the fact that a polynomial restricted to the first order constitutes a linear map, we can construct a linear layer by setting

\[y^{(k)}_{c_{\text{out}}}:=T^{\text{lin}}_{\phi_{c_{\text{out}}}}(x_{1},\ldots, x_{\ell})^{(k)}:=\sum_{c_{\text{in}}=1}^{\ell}\phi_{c_{\text{out}}\text{in}k}\,x ^{(k)}_{c_{\text{in}}}, \tag{11}\]

where \(\phi_{c_{\text{in}}c_{\text{in}}k}\in\mathbb{R}\) are optimizable coefficients and \(c_{\text{in}}\), \(c_{\text{out}}\) denote the input and output channel, respectively. As such, \(T_{\phi}:\mathrm{Cl}(V,\mathfrak{q})^{\ell}\rightarrow\mathrm{Cl}(V,\mathfrak{ q})\) is a linear transformation in each algebra subspace \(k\). Recall that this is possible due to the result that \(\rho(w)\) respects the multivector subspaces. This computes a transformation for the output channel \(c_{\text{out}}\); the map can be repeated (using different sets of parameters) for other output channels, similar to classical neural network linear layers. For \(y^{(0)}_{c_{\text{out}}}\in\mathbb{R}\) (the scalar part of the Clifford algebra), we can additionally learn an invariant bias parameter.

**Geometric Product Layers** A core strength of our method comes from the fact that we can also parameterize interaction terms. In this work, we only consider layers up to second order. Higher-order interactions are indirectly modeled via multiple successive layers. As an example, we take the pair \(x_{1},x_{2}\). Their interaction terms take the form \(\left(x_{1}^{(i)}x_{2}^{(j)}\right)^{(k)},i,j,k=0,\ldots,n\); where we again make use of the fact that \(\rho(w)\) respects grade projections. As such, all the grade-\(k\) terms resulting from the interaction of \(x_{1}\) and \(x_{2}\) are parameterized with

\[P_{\phi}(x_{1},x_{2})^{(k)}:=\sum_{i=0}^{n}\sum_{j=0}^{n}\phi_{ijk}\,\left(x_{ 1}^{(i)}x_{2}^{(j)}\right)^{(k)}, \tag{12}\]

where \(P_{\phi}:\mathrm{Cl}(V,\mathfrak{q})\times\mathrm{Cl}(V,\mathfrak{q}) \rightarrow\mathrm{Cl}(V,\mathfrak{q})\). This means that we get \((n+1)^{3}\) parameters for every geometric product between a pair of multivectors5. Parameterizing and computing all second-order terms amounts to \(\ell^{2}\) such operations, which can be computationally expensive given a reasonable number of channels \(\ell\). Instead, we first apply a linear map to obtain \(y_{1},\ldots,y_{\ell}\in\mathrm{Cl}(V,\mathfrak{q})\). Through this map, the mixing (i.e., the terms that will get multiplied) is learned. That is, we only get \(\ell\) pairs \((x_{1},y_{1}),\ldots,(x_{\ell},y_{\ell})\) from which we then compute \(z^{(k)}_{c_{\text{out}}}:=P_{\phi_{c_{\text{out}}}}(x_{c_{\text{in}}},y_{c_{ \text{in}}})^{(k)}\). Note that here we have \(c_{\text{in}}=c_{\text{out}}\), i.e., the number of channels does not change. Hence, we refer to this layer as
the _element-wise_ geometric product layer. We can obtain a more expressive (yet more expensive) parameterization by linearly combining such products by computing

\[z_{c_{\text{out}}}^{(k)}:=T_{\phi_{c_{\text{out}}}}^{\text{prod}}(x_{1},\ldots,x_ {\ell},y_{1},\ldots,y_{\ell})^{(k)}:=\sum_{c_{\text{in}}=1}^{\ell}P_{\phi_{c_{ \text{out}}}c_{\text{in}}}(x_{c_{\text{in}}},y_{c_{\text{in}}})^{(k)}, \tag{13}\]

which we call the _fully-connected_ geometric product layer. Computational feasibility and experimental verification should determine which parameterization is preferred.

**Normalization and Nonlinearities** Since our layers involve quadratic and potentially higher-order interaction terms, we need to ensure numerical stability. In order to do so, we use a normalization operating on each multivector subspace before computing geometric products by putting

\[x^{(m)}\mapsto\frac{x^{(m)}}{\sigma(a_{m})\,\left(\bar{\mathfrak{q}}(x^{(m)})- 1\right)+1}, \tag{14}\]

where \(x^{(m)}\in\mathrm{Cl}^{(m)}(V,\mathfrak{q})\). Here, \(\sigma\) denotes the logistic sigmoid function, and \(a_{m}\in\mathbb{R}\) is a learned scalar. The denominator interpolates between \(1\) and the quadratic form \(\bar{\mathfrak{q}}\left(x^{(m)}\right)\), normalizing the magnitude of \(x^{(m)}\). This ensures that the geometric products do not cause numerical instabilities without losing information about the magnitude of \(x^{(m)}\), where a learned scalar interpolates between both regimes. Note that by Theorem 3.2, \(\bar{\mathfrak{q}}(x^{(m)})\) is invariant under the action of the Clifford group, rendering Equation (14) an equivariant map.

Next, we use the layer-wide normalization scheme proposed by [70], which, since it is also based on the extended quadratic form, is also equivariant with respect to the adjusted twisted conjugation.

Regarding nonlinearities, first note that the geometric product layers are already nonlinear. However, we still require activation functions after linear layers. To this end, we use a slightly adjusted version of the units proposed by [70]. Since the scalar subspace \(\mathrm{Cl}^{(0)}(V,\mathfrak{q})\) is always invariant with respect to the adjusted twisted conjugation, we can apply \(x^{(m)}\mapsto\mathrm{ReLU}\left(x^{(m)}\right)\) when \(m=0\) and \(x^{(m)}\mapsto\sigma_{\phi}\left(\bar{\mathfrak{q}}\left(x^{(m)}\right)\right) x^{(m)}\) otherwise. We can replace ReLU with any common scalar activation function. Here, \(\sigma_{\phi}\) represents a potentially parameterized nonlinear function. Usually, however, we restrict it to be the sigmoid function. Since we modify \(x^{(m)}\) with an invariant scalar quantity, we retain equivariance. Such gating activations are commonly used in the equivariance literature [88; 36].

## 5 Experiments

Here, we show that CGENNs excel across challenging tasks, attaining top performance in several unique contexts that typically require specialized tools.

### Estimating Volumetric Quantities

\(\mathrm{O}(3)\) **Experiment: Signed Volumes** This task highlights the fact that equivariant architectures based on scalarization are not able to extract some essential geometric properties from input data. In a synthetic setting, we simulate a dataset consisting of random three-dimensional tetrahedra. A main advantage of our method is that it can extract covariant quantities including (among others) _signed volumes_, which we demonstrate in this task. Signed volumes are geometrically significant because they capture the orientation of geometric objects in multidimensional spaces. For instance, in computer graphics, they can determine whether a 3D object is facing towards or away from the camera, enabling proper rendering and improving the overall visual experience. The loss function in this experiment is the mean-squared error between the signed volume and its true value. Note that we are predicting a _covariant_ quantity under \(\mathrm{O}(3)\) transformations. As such, scalarization methods (which rely on invariant quantities) cannot extract this feature from the data. This is illustrated in the left part of Figure 3. We compare against a standard multilayer perceptron (MLP), an MLP version of the \(\mathrm{E}(n)\)-GNN [71] which uses neural networks to update positions with scalar multiplication, _Vector Neurons_ (VN) [26], and _Geometric Vector Perceptrons_ (GVP) [50]. We see that the scalarization methods fail to access covariant features, which are necessary for this task, as evidenced by their test loss not improving even with more available data. The multilayer perceptron, although a universal
approximator, lacks the correct inductive biases. Our model, however, has the correct inductive biases (e.g., the equivariance property) and can also access the signed volume.

\(\mathrm{O}(5)\) **Experiment: Convex Hulls** We go a step further and consider a _five-dimensional_ space, showcasing our models' ability to generalize to high dimensions. We also make the task more challenging by including more points and estimating the volume of the convex hull generated by these points - a task that requires sophisticated algorithms in the classical case. Note that some points may live inside the hull and do not contribute to the volume. We use the same network architectures as before (but now embedded in a five-dimensional space) and present the results in Figure 4. We report the error bars for CGENNs, representing three times the standard deviation of the results of eight runs with varying seeds. Volume (unsigned) is an invariant quantity, enabling the baseline methods to approximate its value. However, we still see that CGENNs outperform the other methods, the only exception being the low-data regime of only 256 available data points. We attribute this to our method being more flexible by considering more than only invariant features, making it slightly more prone to overfitting. To mitigate this issue, future work could explore regularization techniques or other methods to reduce overfitting in low-data scenarios.

### \(\mathrm{O}(5)\) Experiment: Regression

We compare against the methods presented by [32] who propose an \(\mathrm{O}(5)\)-invariant regression problem. The task is to estimate the function \(f(x_{1},x_{2}):=\sin(\|x_{1}\|)-\|x_{2}\|^{3}/2+\frac{x_{1}^{\top}x_{2}}{\|x_ {1}\|\|x_{2}\|}\), where the five-dimensional vectors \(x_{1},x_{2}\) are sampled from a standard Gaussian distribution in order to simulate train, test, and validation datasets. The results are shown in Figure 4. We see that we significantly outperform the MLP and MLP+Aug baselines representing a vanilla MLP and an MLP trained with data augmentation, respectively. Moreover, we also demonstrate an outstanding performance in contrast with the \(\mathrm{O}(5)\) and \(\mathrm{SO}(5)\) EMLP architecture of [32].

### \(\mathrm{E}(3)\) Experiment: \(n\)-Body System

The \(n\)-body experiment [53] serves as a benchmark for assessing the performance of equivariant (graph) neural networks in simulating physical systems [42]. In this experiment, the dynamics of \(n=5\) charged particles in a three-dimensional space are simulated. Given the initial positions and velocities of these particles, the task is to accurately estimate their positions after 1 000 timesteps. To address this challenge, we construct a graph neural network (GNN) using the Clifford equivariant layers introduced in the previous section. The input to the network consists of the mean-subtracted positions of the particles (to achieve translation equivariance) and their velocities. Additionally, we include the invariant charges as part of
the input and their products as edge attributes. We compare against the steerable SE(3)-Transformers [34], Tensor Field Networks [79], and SEGNN [12]. Scalarization baselines include Radial Field [57] and EGNN [71]. Finally, NMP [37] is not an \(\mathrm{E}(3)\)-equivariant method. The number of parameters in our model is maintained similar to the EGNN baseline to ensure a fair comparison.

Results of our experiment are presented in Table 1, where we also present for CGENN three times the standard deviation of three identical runs with different seeds. Our approach significantly outperforms earlier methods and is considerably better than [12], thereby surpassing the baselines. This experiment again demonstrates the advantage of leveraging covariant information in addition to scalar quantities, as it allows for a more accurate representation of the underlying physics and leads to better predictions.

### \(\mathrm{O}(1,3)\) Experiment: Top Tagging

Jet tagging in collider physics is a technique used to identify and categorize high-energy jets produced in particle collisions, as measured by, e.g., CERN's ATLAS detector [15]. By combining information from various parts of the detector, it is possible to trace back the origin of these jets and classify them. Reconstruction algorithms rely on the recovery of the jet's energy, momentum, and other properties [55; 21]. The current experiment seeks to tag jets arising from the heaviest particles of the standard model: the "top quarks" [49]. A jet tag should be invariant with respect to the reference frame in which the jet is observed, whereas the frames themselves change under _Lorentz boosts_ due to the relativistic nature of the particles. A Lorentz boost is a transformation that relates the space and time coordinates of an event as seen from two inertial reference frames that are moving relative to each other at a constant velocity. A defining characteristic of these transformations is that they preserve the _Minkowski metric_, which is given by \(\gamma(ct,x,y,z):=(ct)^{2}-x^{2}-y^{2}-z^{2}\). The set of all such transformations is captured by the orthogonal group \(\mathrm{O}(1,3)\); therefore, our method is fully compatible with modeling this problem.

We evaluate our model on a top tagging benchmark published by [52]. It contains 1.2M training entries, 400k validation entries, and 400k testing entries. For each jet, the energy-momentum \(4\)-vectors are available for up to \(200\) constituent particles, making this a much larger-scale experiment than the ones presented earlier. The baselines include ResNeXt [91], P-CNN [22], PFN [58], ParticleNet [67], LGN [10], EGNN [71], and the more recent LorentzNet [38]. Among these, LGN is a steerable method, whereas EGNN and LorentzNet are scalarization methods. The other methods are not Lorentz-equivariant. Among the performance metrics, there are classification accuracy, Area Under the Receiver Operating Characteristic Curve (AUC), and the background rejection rate \(1/\epsilon_{B}\) at signal efficiencies of \(\epsilon_{S}=0.3\) and \(\epsilon_{S}=0.5\), where \(\epsilon_{B}\) and \(\epsilon_{S}\) are the false positive and true positive rates, respectively. We observe that LorentzNet, a method that uses invariant quantities, is an extremely competitive baseline that was optimized for this task. Despite this, CGENNs are able to match its performance while maintaining the same core implementation.

## 6 Conclusion

We presented a novel approach for constructing \(\mathrm{E}(n)\)-equivariant neural networks based on Clifford algebras. After establishing the required theoretical results, we proposed parameterizations of

\begin{table}
\begin{tabular}{l c c c c} \hline \hline Model & Accuracy (\(\uparrow\)) & AUC (\(\uparrow\)) & \begin{tabular}{c} \(1/\epsilon_{B}\) (\(\uparrow\)) \\ (\(\epsilon_{S}=0.5\)) \\ \end{tabular} & 
\begin{tabular}{c} \(1/\epsilon_{B}\) (\(\uparrow\)) \\ (\(\epsilon_{S}=0.3\)) \\ \end{tabular} \\ \hline ResNeXt [91] & \(0.936\) & \(0.9837\) & \(302\) & \(1147\) \\ P-CNN [22] & \(0.930\) & \(0.9803\) & \(201\) & \(759\) \\ PFN [58] & \(0.932\) & \(0.9819\) & \(247\) & \(888\) \\ ParticleNet [67] & \(0.940\) & \(0.9858\) & \(397\) & \(1615\) \\ EGNN [71] & \(0.922\) & \(0.9760\) & \(148\) & \(540\) \\ LGN [10] & \(0.929\) & \(0.9640\) & \(124\) & \(435\) \\ LorentzNet [38] & \(\mathbf{0.942}\) & \(\mathbf{0.9868}\) & \(\mathbf{498}\) & \(\mathbf{2195}\) \\ \hline CGENN & \(\mathbf{0.942}\) & \(\mathbf{0.9869}\) & \(\mathbf{500}\) & \(\mathbf{2172}\) \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance comparison between our proposed method and alternative algorithms on the top tagging experiment. We present the accuracy, Area Under the Receiver Operating Characteristic Curve (AUC), and background rejection \(1/\epsilon_{B}\) and at signal efficiencies of \(\epsilon_{S}=0.3\) and \(\epsilon_{S}=0.5\).
nonlinear multivector-valued maps that exhibit notable versatility and applicability across several scenarios varying in dimension. This was achieved by the core insight that polynomials in multivectors are \(\mathrm{O}(n)\)-equivariant functions. Theoretical results were empirically substantiated in three distinct experiments, outperforming or matching baselines that were specifically designed for these tasks.

**Limitations** CGENNs induce a (non-prohibitive) degree of computational overhead similar to other steerable methods. On the plus side, we believe that improved code implementations such as custom GPU kernels or alternative parameterizations to the current ones can significantly alleviate this issue, potentially also resulting in improved performances on benchmark datasets. This work provides solid theoretical and experimental foundations for such developments.
