A new perspective on building efficient and expressive 3D equivariant
  graph neural networks
Geometric deep learning enables the encoding of physical symmetries in
modeling 3D objects. Despite rapid progress in encoding 3D symmetries into
Graph Neural Networks (GNNs), a comprehensive evaluation of the expressiveness
of these networks through a local-to-global analysis lacks today. In this
paper, we propose a local hierarchy of 3D isomorphism to evaluate the
expressive power of equivariant GNNs and investigate the process of
representing global geometric information from local patches. Our work leads to
two crucial modules for designing expressive and efficient geometric GNNs;
namely local substructure encoding (LSE) and frame transition encoding (FTE).
To demonstrate the applicability of our theory, we propose LEFTNet which
effectively implements these modules and achieves state-of-the-art performance
on both scalar-valued and vector-valued molecular property prediction tasks. We
further point out the design space for future developments of equivariant graph
neural networks. Our codes are available at
\url{https://github.com/yuanqidu/LeftNet}.
Weitao Du, Yuanqi Du, Limei Wang, Dieqiao Feng, Guifeng Wang, Shuiwang Ji, Carla Gomes, Zhi-Ming Ma
# A new perspective on building efficient and expressive 3D equivariant graph neural networks

###### Abstract

Geometric deep learning enables the encoding of physical symmetries in modeling 3D objects. Despite rapid progress in encoding 3D symmetries into Graph Neural Networks (GNNs), a comprehensive evaluation of the expressiveness of these networks through a local-to-global analysis lacks today. In this paper, we propose a local hierarchy of 3D isomorphism to evaluate the expressive power of equivariant GNNs and investigate the process of representing global geometric information from local patches. Our work leads to two crucial modules for designing expressive and efficient geometric GNNs; namely local substructure encoding (**LSE**) and frame transition encoding (**FTE**). To demonstrate the applicability of our theory, we propose LEFTNet which effectively implements these modules and achieves state-of-the-art performance on both scalar-valued and vector-valued molecular property prediction tasks. We further point out the design space for future developments of equivariant graph neural networks. Our codes are available at [https://github.com/yuanqdu/LeftNet](https://github.com/yuanqdu/LeftNet).

## 1 Introduction

The success of many deep neural networks can be attributed to their ability to respect physical symmetry, such as Convolutional Neural Networks (CNNs) [1] and Graph Neural Networks (GNNs) [2]. Specifically, CNNs encode translation equivariance, which is essential for tasks such as object detection. Similarly, GNNs encode permutation equivariance, which ensures that the node ordering does not affect the output node representations., by aggregating neighboring messages. Modeling 3D objects, such as point clouds and molecules, is a fundamental problem with numerous applications, including robotics [3], molecular simulation [4; 5], and drug discovery [6; 7; 8; 9; 10]. Different from 2D pictures and graphs that only possess the translation [1] and permutation [2] symmetry, 3D objects intrinsically encode the complex \(SE(3)/E(3)\) symmetry [11], which makes their modeling a nontrivial task in the machine learning community.

To tackle this challenge, several approaches have been proposed to effectively encode 3D rotation and translation equivariance in the deep neural network architectures, such as TFN [12], EGNN [13], and SphereNet [14]. TFN leverages spherical harmonics to represent and update tensors equivariantly, while EGNN processes geometric information through vector update. On the other hand, SphereNet is invariant by encoding scalars like distances and angles. Despite rapid progress has
been made on the empirical side, it's still unclear what 3D geometric information can equivariant graph neural networks capture and how the geometric information is integrated during the message passing process [15; 16; 17]. This type of analysis is crucial in designing expressive and efficient 3D GNNs, as it's usually a trade-off between encoding enough geometric information and preserving relatively low computation complexity. Put aside the \(SE(3)/E(3)\) symmetry, this problem is also crucial in analysing ordinary GNNs. For example, 1-hop based message passing graph neural networks [18] are computationally efficient while suffering from expressiveness bottlenecks (comparing with subgraph GNNs [19; 20]). On the other hand, finding a better trade-off for 3D GNNs is more challenging, since we must ensure that the message updating and aggregating process respects the \(SE(3)/E(3)\) symmetry.

In this paper, we attempt to discover better trade-offs between computational efficiency and expressiveness power for 3D GNNs by studying two specific questions: 1. What is the expressive power of invariant scalars in encoding 3D geometric patterns? 2. Is equivariance really necessarily for 3D GNNs? The first question relates to the design of node-wise geometric messages, and the second question relates to the design of equivariant (or invariant) aggregation. To tackle these two problems, we take a local-to-global approach. More precisely, we first define three types of 3D isomorphism to characterize local 3D structures: tree, triangular, and subgraph isomorphism, following a local hierarchy. As we will discuss in the related works section, our local hierarchy lies between the 1-hop and 2-hop geometric isomorphism defined in [21]. Then, we can measure the expressiveness power of 3D GNNs by their ability of differentiating non-isomorphic 3D structures in a similar way as the geometric WL tests in [21]. Under this theoretical framework, we summarize one essential ingredient for building expressive geometric messages on each node: local 3D substructure encoding (**LSE**), which allows an invariant realization. To answer the second question, we analyze whether local invariant features are sufficient for expressing global geometries by message aggregation, and it turns out that frame transition encoding (**FTE**) is crucial during the local to global process. Although **FTE** can be realized by invariant scalars, we further demonstrate that introducing equivariant messaging passing is more efficient. By connecting **LSE** and **FTE** modules, we are able to present a modular overview of 3D GNNs designs.

In realization of our theoretical findings, we propose LEFTNet that efficiently implements **LSE** and **FTE** (with equivariant tensor update) without sacrificing expressiveness. Empirical experiments on real-world scenarios, predicting scalar-valued property (e.g. energy) and vector-valued property (e.g. force) for molecules, demonstrate the effectiveness of LEFTNet.

## 2 Preliminary

In this section, we provide an overview of the mathematical foundations of \(E(3)\) and \(SE(3)\) symmetry, which is essential in modeling 3D data. We also summarize the message passing graph neural network framework, which enables the realization of \(E(3)/SE(3)\) equivariant models.

**Euclidean Symmetry.** Our target is to incorporate Euclidean symmetry to ordinary permutation-invariant graph neural networks. The formal way of describing Euclidean symmetry is the group \(E(3)=O(3)\rtimes T(3)\), where \(O(3)\) corresponds to reflections (parity transformations) and rotations. For tasks that are anti-symmetric under reflections (e.g. chirality), we consider the subgroup \(SE(3)=SO(3)\rtimes T(3)\), where \(SO(3)\) is the group of rotations. We will use \(SE(3)\) in the rest of the paper for brevity except when it's necessary to emphasize reflections.

**Equivariance.** A tensor-valued function \(f(\mathbf{x})\) is said to be **equivariant** with respect to \(SE(3)\) if for any translation or rotation \(g\in SE(3)\) acting on \(\mathbf{x}\in\mathbf{R}^{3}\), we have

\[f(g\mathbf{x})=\mathcal{M}(g)f(\mathbf{x}),\]

where \(\mathcal{M}(\cdot)\) is a matrix representation of \(SE(3)\) acting on tensors. See Appendix A for a general definition of tensor fields. In this paper, we will use **bold** letters to represent an equivariant tensor, e.g., \(\mathbf{x}\) as a position vector. It is worth noting that when \(f(\mathbf{x})\in\mathbf{R}^{1}\) and \(\mathcal{M}(g)\equiv 1\) (the constant group representation), the equivariant function \(f(\mathbf{x})\) is also called an **invariant** scalar function.

**Scalarization.** Scalarization is a general technique that originated from differential geometry for realizing covariant operations on tensors [22]. Our method will apply a simple version of scalarization in \(\mathbf{R}^{3}\) to transform equivariant quantities. At the heart of its realization is the notion of equivariant
orthonormal frames, which consist of three orthonormal equivariant vectors:

\[\mathcal{F}:=(\mathbf{e}_{1},\mathbf{e}_{2},\mathbf{e}_{3}).\]

Based on \(\mathcal{F}\), we can build orthonormal equivariant frames for higher order tensors by taking tensor products \(\otimes\), see Eq. 19 in Appendix. By taking the inner product between \(\mathcal{F}\) and a given equivariant vector (tensor) \(\mathbf{x}\), we get a tuple of invariant scalars (see [23] for a proof):

\[\mathbf{x}\rightarrow\tilde{x}:=(\mathbf{x}\cdot\mathbf{e}_{1},\mathbf{x} \cdot\mathbf{e}_{2},\mathbf{x}\cdot\mathbf{e}_{3}), \tag{1}\]

and \(\tilde{x}\) can be seen as the'scalarized' coordinates of \(\mathbf{x}\).

**Tensorization.** Tensorization, on the other hand, is the'reverse' process of scalarization. Given a tuple of scalars: \((x_{1},x_{2},x_{3})\), tensorization creates an equivariant vector (tensor) out of \(\mathcal{F}\):

\[(x_{1},x_{2},x_{3})\xrightarrow{\text{Pairing}}\mathbf{x}:=x_{1}\mathbf{e} _{1}+x_{2}\mathbf{e}_{2}+x_{3}\mathbf{e}_{3}. \tag{2}\]

The same procedure is extended to higher order cases, see Eq. 20 in Appendix.

**Message Passing Scheme for Geometric Graphs.** A geometric graph \(G\) is represented by \(G=(V,E)\). Here, \(v_{i}\in V\) denotes the set of nodes (vertices, atoms), and \(e_{ij}\in E\) denotes the set of edges. For brevity, the edge feature attached on \(e_{ij}\) is also denoted by \(e_{ij}\). Let \(\mathbf{X}=(\mathbf{x}_{1},\ldots,\mathbf{x}_{n})\in\mathbf{R}^{n\times 3}\) be the 3D point cloud of all nodes' equivariant positions, which determines the 3D geometric structure of \(G\).

A common machine learning tool for modeling graph-structured data is the Message Passing Neural Network (MPNN) [15]. A typical 1-hop MPNN framework consists of two phases: (1) message passing; (2) readout. Let \(h^{l}_{i},h^{l}_{j}\) denote the \(l\)-th layer's node features of source \(i\) and target \(j\) that also depend on the 3D positions \((\mathbf{x}_{i},\mathbf{x}_{j})\), then the aggregated message is

\[m^{l}_{i}=\bigoplus_{j\in\mathcal{N}(i)}m_{ij}(h^{l}(\mathbf{x}_{i}),h^{l}( \mathbf{x}_{j}),e^{l}_{ij}), \tag{3}\]

and \(\bigoplus_{j\in\mathcal{N}(i)}\) is any permutation-invariant pooling operation between the 1-hop neighbors of \(i\). We also include the edge features \(e^{l}_{ij}\) into the message passing phase for completeness. 3D **equivariant** MPNNs (3D GNNs for short) require the message \(m_{i}\) to be equivariant with respect to the geometric graph. That is, for an arbitrary edge \(e_{ij}\):

\[m_{ij}(h^{l}(g\mathbf{x}_{i}),h^{l}(g\mathbf{x}_{j}))=\mathcal{M}(g)m_{ij}(h^ {l}(\mathbf{x}_{i}),h^{l}(\mathbf{x}_{j})), \tag{4}\]

where \(g\in SE(3)\) is acting on the whole geometric graph simultaneously: \((\mathbf{x}_{1},\ldots,\mathbf{x}_{n})\rightarrow(g\mathbf{x}_{1},\ldots,g \mathbf{x}_{n})\). For example, the invariant model ComENet [24] satisfies Eq. 4 by setting \(\mathcal{M}(g)\equiv 1\), and MACE [25] realized Eq. 4 for nonconstant irreducible group representations \(\mathcal{M}(g)\) through spherical harmonics and Clebsch-Gordan coefficients.

## 3 A Local Hierarchy of 3D Isomorphism

As presented in Section 2, defining expressive messages is an essential component for building powerful 3D GNNs. In this section, we develop a fine-grained characterization of local 3D structures and build its connection with the expressiveness of 3D GNNs.

Figure 1: (a) \(\mathbf{S}_{i}\) and \(\mathbf{S}_{j}^{\text{(a)}}\) share the same tree structure (edge lengths are identical), but they are not triangular isomorphic (different dihedral angles); (b) \(\mathbf{S}_{i}\) and \(\mathbf{S}_{j}\) are triangular isomorphic but not subgraph isomorphic (the relative distance between the two triangles is different).
Since the celebrated work [26], a popular expressiveness test for permutation invariant graph neural networks is the 1-WL graph isomorphism test [27], and Wijesinghe and Wang [28] has shown that the 1-WL test is equivalent to the ability to discriminate the **local** subtree-isomorphism. It motivates us to develop a novel (local) 3D isomorphism for testing the expressive power of 3D GNNs. However, this task is nontrivial, since most of the previous settings for graph isomorphism are only applicable to 2D topological features. For 3D geometric shapes, we should take the \(SE(3)\) symmetry into account. Formally, two 3D geometric graphs \(\mathbf{X},\mathbf{Y}\) are defined to be **globally** isomorphic, if there exists \(g\in SE(3)\) such that

\[\mathbf{Y}=g\mathbf{X}. \tag{5}\]

In other words, \(\mathbf{X}\) and \(\mathbf{Y}\) are essentially the same, if they can be transformed into each other through a series of rotations and translations. Inspired by Wijesinghe and Wang [28], now we introduce a novel hierarchy of \(SE(3)\) equivariant local isomorphism to measure the local similarity of 3D structures.

Let \(\mathbf{S}_{i}\) denote the 3D subgraph (and the associated node features) of node \(i\), which contains all edges in \(E\) if the end points are one-hop neighbors of \(i\). For each edge \(e_{ij}\in E\), the mutual 3D substructure \(\mathbf{S}_{i-j}\) is defined by the intersection of \(\mathbf{S}_{i}\) and \(\mathbf{S}_{j}\): \(\mathbf{S}_{i-j}=\mathbf{S}_{i}\cap\mathbf{S}_{j}\).

Given two local subgraphs \(\mathbf{S}_{i}\) and \(\mathbf{S}_{j}\) that correspond to two nodes \(i\) and \(j\), we say \(\mathbf{S}_{i}\) is {-tree, -triangular, -subgraph} isometric to \(\mathbf{S}_{j}\), if there exists a bijective function \(f:\mathbf{S}_{i}\rightarrow\mathbf{S}_{j}\) such that \(h_{f(u)}=h_{u}\) for every node \(u\in\mathbf{S}_{i}\), and the following conditions hold respectively:

* **Tree Isometric:** If there exists a collection of group elements \(g_{iu}\in SE(3)\), such that \((\mathbf{x}_{f(u)},\mathbf{x}_{f(i)})=(g_{iu}\mathbf{x}_{u},g_{iu}\mathbf{x} _{i})\) for each edge \(e_{iu}\in\mathbf{S}_{i}\);
* **Triangular Isometric:** If there exists a collection of group elements \(g_{iu}\in SE(3)\), such that the corresponding mutual 3D substructures satisfy: \(\mathbf{S}_{f(u)-f(i)}=g_{iu}\mathbf{S}_{u-i}\) for each edge \(e_{iu}\in\mathbf{S}_{i-j}\);
* **Subgraph Isometric:** for any two adjacent nodes \(u,v\in\mathbf{S}_{i}\), \(f(u)\) and \(f(v)\) are also adjacent in \(\mathbf{S}_{j}\), and there exist a single group element \(g_{i}\in SE(3)\) such that \(g_{i}\mathbf{S}_{i}=\mathbf{S}_{j}\).

Note that tree isomorphism only considers edges around a central node, which is of a tree shape. On the other hand, the mutual 3D substructure can be decomposed into a bunch of triangles (since it's contained in adjacent node triplets), which explains the name of triangular isomorphism.

In fact, the three isomorphisms form a hierarchy from micro to macro, in the sense that the following implication relation holds:

**Subgraph Isometric \(\Rightarrow\) Triangular Isometric \(\Rightarrow\) Tree Isometric**

This is an obvious fact from the above definitions. To deduce the reverse implication relation, we provide a visualized example. Figure 1 shows two examples of local 3D structures: 1. the first one shares the same tree structure, but is not triangular-isomorphic; 2. the second one is triangular-isomorphic but not subgraph-isomorphic. In conclusion, the following diagram holds:

**Tree Isometric \(\not\Rightarrow\) Triangular Isometric \(\not\Rightarrow\) Subgraph Isometric**

One way to formally connect the expressiveness power of a geometric GNN with their ability of differentiating geometric subgraphs is to define geometric WL tests, the reader can consult [21]. In this paper, we take an intuitive approach based on our nested 3D hierarchy. That is, if two 3D GNN algorithms A and B can differentiate all non-isomorphic local 3D shapes of tree (triangular) level, while A can differentiate at least two more 3D geometries which are non-isomorphic at triangular(subgraph) level than B, then we claim that algorithm A's expressiveness power is more powerful than B.

Since tree isomorphism is determined by the one-hop Euclidean distance between neighbors, distinguishing local tree structures is relatively simple for ordinary 3D equivariant GNNs. For example, the standard baseline SchNet [29] is one instance of Eq. 3 by setting \(e_{ij}^{t}=\textbf{RBF}(d(\mathbf{x}_{i},\mathbf{x}_{j}))\), where \(\textbf{RBF}(\cdot)\) is a set of radial basis functions. Although it is powerful enough for testing tree non-isomorphism (assuming that \(\textbf{RBF}(\cdot)\) is injective), we prove in Appendix B that SchNet cannot distinguish non-isomorphic structures at the triangular level.

On the other hand, Wijesinghe and Wang [28] has shown that by leveraging the topological information extracted from local overlapping subgraphs, we can enhance the expressive power of GNNs
to go beyond 2D sub-tree isomorphism. In our setting, the natural analogue of the overlapping subgraphs is exactly the mutual 3D substructures. Now we demonstrate how to merge the information from 3D substructures to the message passing framework (3). Given an \(SE(3)\)-**invariant** encoder \(\phi\), define the 3D structure weights \(A_{ij}:=\phi(\mathbf{S}_{i-j})\) for each edge \(e_{ij}\in E\). Then, the message passing framework (3) is generalized to:

\[m_{i}^{l}=\bigoplus_{j\in\mathcal{N}(i)}m_{ij}(h^{l}(\mathbf{x}_{i}),h^{l}( \mathbf{x}_{j}),A_{ij}h^{l}(\mathbf{x}_{j}),e_{ij}^{l}). \tag{6}\]

Formula 6 is an efficient realization of enhancing 3D GNNs by injecting the mutual 3D substructures. However, a crucial question remains to be answered: _Can the generalized message passing framework boost the expressive power of 3D GNNs?_ Under certain conditions, the following theorem provides an affirmative answer:

**Theorem 3.1**.: _Suppose \(\phi\) is a a universal SE(3)-invariant approximator of functions with respect to the mutual 3d structures \(\mathbf{S}_{i-j}\), then the collection of weights \(\{\{A_{ij}\}_{e_{ij}\in E}\}\) is able to differentiate local structures beyond tree isomorphism. Moreover, with additional injectivity assumptions (see Eq. 14), 3D GNNs based on the enhanced message passing framework 6 map at least two distinct local 3D subgraphs with isometric local tree structures to different representations._

This theorem confirms that the enhanced 3D GNN (formula 6) is more expressive than the SchNet baseline, at least in testing local non-isomorphic geometric graphs. The complete proof is left in Appendix B. The existence of such local invariant encoder \(\phi\) is also proved by explicit construction. Note that there are other different perspectives on characterizing 3D structures, we will also briefly discuss them in Appendix B.

## 4 From Local to Global: The Missing Pieces

In the last section, we introduced a geometric local isomorphism hierarchy for testing the expressive power of 3D GNNs. Furthermore, we motivated adding a SE(3)-**invariant** encoder to improve the expressive power of one-hop 3D GNNs by scalarizing not only pairwise distances but also their mutual 3D structures in Theorem 3.1. However, to build a powerful 3D GNN, it remains to be analyzed how a 3D GNN acquires higher order (beyond 1-hop neighbors) information by accumulating local messages. A natural question arises: _are invariant features enough for representing **global** geometric information?_

To formally formulate this problem, we consider a two-hop aggregation case. From figure 2, the central atom \(a\) is connected with atoms \(b\) and \(c\). Except for the common neighbor \(a\), other atoms that connect to \(b\) and \(c\) form two 3D clusters, denoted by **B**, **C**. Suppose the ground-truth interaction potential of **B** and **C** imposed on atom \(a\) is described by a tensor-valued function \(f_{a}(\textbf{B},\textbf{C})\). Since **B** and **C** are both beyond the 1-hop neighborhood of \(a\), the information of \(f_{a}(\textbf{B},\textbf{C})\) can only be acquired after two steps of message passing: 1. atoms \(b\) and \(c\) aggregate message separately from **B** and **C**; 2. the central atom \(a\) receives the aggregated message (which contains information of **B** and **C**) from its neighbors \(b\) and \(c\).

Let \(S_{\textbf{B}}\) (\(S_{\textbf{C}}\)) denote the collection of all invariant scalars created by **B** (**C**). For example, \(S_{\textbf{B}}\) contains all relative distances and angles within the 3D structure **B**. Then, the following theorem holds:

**Theorem 4.1**.: _Not all types of invariant interaction \(f_{a}(\textbf{B},\textbf{C})\) can be expressed by inputting the union of two sets \(S_{\textbf{B}}\) and \(S_{\textbf{C}}\). In other words, there exists \(E(3)\) invariant function \(f_{a}(\textbf{B},\textbf{C})\), such that it cannot be expressed as functions of \(S_{\textbf{B}}\) and \(S_{\textbf{C}}\): \(f_{a}(\textbf{B},\textbf{C})\neq\rho(S_{\textbf{B}},S_{\textbf{C}})\) for an arbitrary invariant function \(\rho\)._

Figure 2: Illustrations of different local frames and their transition.
This theorem in essence tells us that naively aggregating 'local' scalar information from different clusters is not enough to approximate 'global' interactions, even if we only consider simple **invariant** interaction tasks. Different from the last section, where the local expressiveness is measured by the ability of classifying geometric shapes, we built regression functions that depend strictly more than the combination of local invariant scalars.

Intuitively, the proof is based on the fact that all scalars in \(S_{\textbf{B}}\) (\(S_{\textbf{C}}\)) can be expressed through equivariant frames separately determined by **B** (**C**). However, the transition matrix between these two frames is not encoded in the aggregation, which causes **information loss** when aggregating geometric features from two sub-clusters. More importantly, the proof also revealed the missing information that causes the expressiveness gap: Frame Transition (FT).

**Frame Transition (FT).** Formally, two orthonormal frames \((e_{1}^{i},e_{2}^{i},e_{3}^{i})\) and \((e_{1}^{j},e_{2}^{j},e_{3}^{j})\) are connected by an orthogonal matrix \(R_{ij}\in SO(3)\):

\[(\textbf{e}_{1}^{i},\textbf{e}_{2}^{i},\textbf{e}_{3}^{i})=R_{ij}(\textbf{e}_ {1}^{j},\textbf{e}_{2}^{j},\textbf{e}_{3}^{j}). \tag{7}\]

Moreover, it is easy to check that when \((\textbf{e}_{1}^{i},\textbf{e}_{2}^{i},\textbf{e}_{3}^{i})\) and \((\textbf{e}_{1}^{j},\textbf{e}_{2}^{j},\textbf{e}_{3}^{j})\) are equivariant frames, all elements of \(R_{xy}\) are invariant scalars. Suppose \(i\) and \(j\) represent indexes of two connected atoms in a geometric graph, then the fundamental torsion angle \(\tau_{ij}\) appeared in ComeNet [24] is just one element of \(R_{ij}\) (see Appendix C).

Towards filling this expressiveness gap, we can straightforwardly inject all invariant pairwise frame transition matrices (**FT**) into the model. Nevertheless, it imposes expensive computational cost when the number of local clusters is large (\(O(k^{2})\) pairs of **FT** for each node). Therefore, compared with pure invariant approaches, a more efficient way is to introduce equivariant tensor features for each node \(i\), denoted by \(\textbf{m}_{i}\). By directly maintaining the equivariant frames in \(\textbf{m}_{i}\), we show in Appendix C that **FT** is easily derived through equivariant message passing.

**Equivariant Message Passing.** Similarly with the standard one-hop message passing scheme 3, the aggregated tensor message \(\textbf{m}_{i}\) from the \(l-1\) layer to the \(l\) layer can be written as: \(\textbf{m}_{i}^{l-1}=\sum_{j\in N(i)}\textbf{m}_{j}^{l-1}\). Since summation does not break the symmetry rule, it is obvious that \(\textbf{m}_{i}^{l-1}\) are still equivariant tensors. However, the nontrivial part lies in the design of the equivariant update function \(\phi\):

\[\textbf{m}_{i}^{l}=\phi(\textbf{m}_{i}^{l-1}). \tag{8}\]

A good \(\phi\) should have enough expressive power while preserving \(SE(3)\) equivariance. Here, we propose a novel way of updating scalar and tensor messages by performing node-wise scalarization and tensorization blocks (the **FTE** module of Figure 3). From the perspective of Eq. 4, \(\textbf{m}(\textbf{x}_{u})\) is transformed equivariantly as:

\[\textbf{m}(g\textbf{x}_{u})=\sum_{i=0}^{l}\mathcal{M}^{i}(g)\textbf{m}_{i}(g \textbf{x}_{u}),\ \ g\in SE(3). \tag{9}\]

Here, \(\textbf{m}(\textbf{x}_{u})\) is decomposed to \((\textbf{m}_{0}(\textbf{x}_{u}),\ldots,\textbf{m}_{l}(\textbf{x}_{u}))\) according to different tensor types, and \(\{\mathcal{M}^{i}(g)\}_{i=0}^{l}\) is a collection of different \(SE(3)\)**tensor representations** (see the precise definition in Appendix A).

To illustrate the benefit of aggregating equivariant messages from local patches, we study a simple case. Let \(f_{a}(\textbf{B},\textbf{C})=\textbf{h}_{B}\cdot\textbf{h}_{C}\) be an invariant function of **B** and **C** (see Fig. 2), then \(f_{a}\) can be calculated by a direction composition of scalar messages and equivariant vector messages: \(f_{a}(\textbf{B},\textbf{C})=\frac{1}{2}[\left\|\textbf{m}_{a}\right\|^{2}- \left\|\textbf{h}_{B}\right\|^{2}-\left\|\textbf{h}_{C}\right\|^{2}]\), where \(\textbf{m}_{a}=\textbf{h}_{B}+\textbf{h}_{C}\) is an equivariant vector. Note that \(\textbf{m}_{a}\) follows the local equivariant aggregation formula 8, and the other vectors' norm \(\left\|\textbf{h}_{B}\right\|\) and \(\left\|\textbf{h}_{C}\right\|\) are obtained through local scalarization on atoms \(b\) and \(c\). As a comparison, it's worth mentioning that \(f_{a}(\textbf{B},\textbf{C})\) can also be expressed by local scalarization with the additional transition matrix data \(R_{BC}\) defined by Eq. 7. Let \(\tilde{h}_{B}\) and \(\tilde{h}_{C}\) be the scalarized coordinates with respect to two local equivariant frames \(\mathcal{F}_{B}\) and \(\mathcal{F}_{C}\). Then \(f_{a}(\textbf{B},\textbf{C})=\frac{1}{2}\left[\left\|R_{BC}^{-1}\tilde{h}_{B} +\tilde{h}_{C}\right\|^{2}-\left\|\tilde{h}_{B}\right\|^{2}-\left\|\tilde{h}_{C }\right\|^{2}\right].\) However, it requires adding the rotation matrix \(R_{BC}\) for each \((\textbf{B},\textbf{C})\) pair, which is computationally expensive compared to directly implementing equivariant tensor updates.
## 5 Building an Efficient and Expressive Equivariant 3D GNN

We propose to leverage the full power of **LSE** and **FTE** along with a powerful **tensor update** module to push the limit of efficient and expressive 3D equivariant GNNs design.

**LSE Instantiation.** We propose to apply edge-wise equivariant frames to encode the local 3D structures \(\mathbf{S}_{i-j}\). By definition, \(\mathbf{S}_{i-j}\) contains edge \(e_{ij}\), nodes \(i\) and \(j\), and their common neighbors. We use the equivariant frame \(\mathcal{F}_{ij}\) built on \(e_{ij}\) (see the precise formula in Appendix D) to scalarize \(\mathbf{S}_{i-j}\). After scalarization (1), the equivariant coordinates of all nodes in \(\mathbf{S}_{i-j}\) are transformed into invariant coordinates: \(\{\mathbf{x}_{k}\rightarrow\tilde{x}_{k}\text{ for }\mathbf{x}_{k}\in\mathbf{S}_{i-j}\}\). To encode these scalars sufficiently, we first weight each \(\tilde{x}_{k}\) by the **RBF** distance embedding: \(\tilde{x}_{k}\rightarrow\textbf{RBF}(\|\mathbf{x}_{k}\|)\odot\text{MLP}( \tilde{x}_{k})\) for each \(\mathbf{x}_{k}\in\mathbf{S}_{i-j}\). Note that to preserve the permutation symmetry, the MLP is shared among the nodes. Finally, the 3D structure weight \(A_{ij}\) is obtained by the average pooling of all node features.

**FTE Instantiation.** We propose to introduce equivariant tensor message passing and update function for encoding local **FT** information. At initialization, let \(\textbf{NF}^{l}(\mathbf{x}_{i},\mathbf{x}_{j})\) denote the embedded tensor-valued edge feature between \(i\) and \(j\). We split it into two parts: 1. the scalar part \(\text{SF}^{l}(\mathbf{x}_{i},\mathbf{x}_{j})\) for aggregating invariant messages; 2. the higher order tensor part \(\textbf{TF}^{l}(\mathbf{x}_{i},\mathbf{x}_{j})\) for aggregating tensor messages. To transform \(\textbf{TF}^{l}(\mathbf{x}_{i},\mathbf{x}_{j})\), we turn to the equivariant frame \(\mathcal{F}_{ij}\) once again. After scalarization by \(\mathcal{F}_{ij}\), \(\textbf{TF}^{l}(\mathbf{x}_{i},\mathbf{x}_{j})\) becomes a tuple of scalars \(\widetilde{\text{TF}}^{l}(\mathbf{x}_{i},\mathbf{x}_{j})\), which is then transformed by MLP. Finally, we output arbitrary tensor messages through equivariant tensorization 20:

\[\widetilde{\text{TF}}^{l}(\mathbf{x}_{i},\mathbf{x}_{j})\xrightarrow[\mathcal{ F}_{ij}]{\textbf{Tensorize}}\textbf{NF}^{l+1}(\mathbf{x}_{i},\mathbf{x}_{j}).\]

Further details are provided in Appendix D. As we have discussed earlier, the node-wise tensor update function \(\phi\) in Eq. 8 is also one of the guarantees for a powerful **FTE**. As a comparison, \(\phi\) is usually a standard MLP for updating node features in 2D GNNs, which is a **universal approximator** of invariant functions. Previous works [13; 30] updated equivariant features by taking linear combinations and calculating the invariant norm of tensors, which may suffer from information loss. Then a natural question arises: _Can we design an equivariant universal approximator for tensor update?_ We answer this question by introducing a novel node-wise frame. Consider node \(i\) with its position \(\mathbf{x}_{i}\), let \(\bar{\mathbf{x}}_{i}:=\frac{1}{N}\sum_{\mathbf{x}_{j}\in N(\mathbf{x}_{i})} \mathbf{x}_{j}\) be the center of mass around \(\mathbf{x}_{i}\)'s neighborhood. Then the orthonormal equivariant frame \(\mathcal{F}_{i}:=(\mathbf{e}_{1}^{i},\mathbf{e}_{2}^{i},\mathbf{e}_{3}^{i})\) with respect to \(\mathbf{x}_{i}\) is defined by

\[(\frac{\mathbf{x}_{i}-\bar{\mathbf{x}}_{i}}{\|\mathbf{x}_{i}-\bar{\mathbf{x}} _{i}\|},\frac{\bar{\mathbf{x}}_{i}\times\mathbf{x}_{i}}{\|\bar{\mathbf{x}}_{i} \times\mathbf{x}_{i}\|},\frac{\mathbf{x}_{i}-\bar{\mathbf{x}}_{i}}{\|\mathbf{ x}_{i}-\bar{\mathbf{x}}_{i}\|}\times\frac{\bar{\mathbf{x}}_{i}\times\mathbf{x}_{i}}{\| \bar{\mathbf{x}}_{i}\times\mathbf{x}_{i}\|}). \tag{10}\]

Finally, we realize a powerful \(\phi\) by the following theorem:

**Theorem 5.1**.: _Equipped with an equivariant frame \(\mathcal{F}_{i}\) for each node \(i\), the equivariant function \(\phi\) defined by the following composition is a universal approximator of tensor transformations: \(\phi:\textbf{Scalarization}\rightarrow\textbf{MLP}\rightarrow\textbf{ Tensorization}\)._

Figure 3: Illustrations of our modular framework for building equivariant GNNs and the realization of LEFTNet. Each interaction block contains **LSE** to encode local 3D structures, equivariant message passing to update both invariant (unbold letters, e.g. \(h_{i}\)) and equivariant (**bold** letter, e.g. \(\mathbf{h}_{i}\)) features, and **FTE** to encode frame transition. \(\mathbf{S}_{i-j}\) is the local 3D structure of each edge \(e_{ij}\). \(\mathcal{F}_{ij}\) and \(\mathcal{F}_{i}\) are the equivariant frames for each edge \(e_{ij}\) and node \(i\). \(\odot\) indicates element-wise multiplication, and \(\|\) indicates concatenation. Note that we do not include \(\mathbf{e}_{ij}\) in the figure since, practically, they are generated based on \(\mathbf{h}_{i}\) and \(\mathbf{h}_{j}\).
The proof is left in Appendix D.

**LEFTNet.** An overview of our \(\{\textbf{LSE},\textbf{FTE}\}\) enhanced efficient graph neural network (LEFTNet) is depicted in Figure 3. LEFTNet receives as input a collection of node embeddings \(\{v^{0}_{1},\dots,v^{0}_{N}\}\), which contain the atom types and 3D positions for each node: \(v^{0}_{i}=(z_{i},\textbf{x}_{i})\), where \(i\in\{1,\dots,N\}\). For each edge \(e_{ij}\in E\), we denote the associated equivariant features consisting of tensors by \(\textbf{e}_{ij}\). During each messaging passing layer, the **LSE** module outputs the scalar weight coefficients \(A_{ij}\) as enhanced invariant edge feature and feed into the interaction module. Moreover, scalarization and tensorization as two essential blocks are used in the equivariant update module that fulfills the function of **FTE**. The permutation equivariance of a geometric graph is automatically guaranteed for any message passing architecture, we provide a complete proof of SE(3)-equivariance for LEFTNet in Appendix D.

**SE(3) vs E(3) Equivariance.** Besides explicitly fitting the \(SE(3)\) invariant molecular geometry probability distribution, modeling the energy surface of a molecule system is also a crucial task for molecule property prediction. However, the Hamiltonian energy function \(E\) of a molecule is invariant under refection transformation: \(\textbf{Energy}(\textbf{X})=\textbf{Energy}(R\textbf{X})\), for arbitrary reflection transformation \(R\in E(3)\). In summary, there exist two different inductive biases for modeling 3D data: **(1)** SE(3) equivariance, e.g. chirality could turn a therapeutic drug to a killing toxin; **(2)** E(3) equivariance, e.g. energy remains the same under reflections.

Since we implement \(SE(3)\) equivariant frames in LEFTNet, our algorithm is naturally \(SE(3)\) equivariant (and reflection anti-equivariant). However, our method is **flexible** to implement \(E(3)\) equivariant tasks as well. For \(E(3)\) equivariance, we can either replace our frames to \(E(3)\) equivariant frames, or modify the scalarization block by taking the absolute value: \(\textbf{x}\rightarrow\tilde{x}:=\underbrace{(\textbf{x}\cdot e_{1},\textbf{x} \cdot e_{2},\textbf{x}\cdot e_{3})}_{SE(3)}\rightarrow\underbrace{(\textbf{x} \cdot e_{1},|\textbf{x}\cdot e_{2}|,\textbf{x}\cdot e_{3})}_{E(3)}\). Intuitively, since the second vector \(e_{2}\) is a pseudo-vector, projections of any equivariant vectors along the \(e_{2}\) direction are not \(E(3)\) invariant until taking the absolute value.

**Efficiency.** To analyze the efficiency of LEFTNet, suppose 3D graph \(G\) has \(n\) vertices, and its average node degree is \(k\). Our algorithm consists of three phases: 1. Building equivariant frames and performing local scalarization; 2. Equivariant message passing; 3. Updating node-wise tensor features through scalarization and tensorization. Let \(l\) be the number of layers, then the computational complexity for each of our three phases are: 1. \(O(nk)\) for computing the frame and local (1-hop) 3D features; 2. \(O(nkl)\) for 1-hop neighborhood message aggregation; 3. \(O(nl)\) for node-wise tensorization and feature update.

## 6 Related Work

In light of the discussions in Section 3 and 4, we summarize two necessary ingredients for building expressive equivariant 3D GNNs: (1) local 3D substructure encodings (**LSE**), such that the local

\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Method** & **Symmetry** & **LSE** & **FTE** & **Complexity** \\ \hline SchNet [29] & E(3)-invariant & ✗ & ✗ & \(O(nk)\) \\ EGNN [13] & E(3)-equivariant & ✗ & ✓\({}^{*}\) & \(O(nk)\) \\ GVP-GNN [30] & E(3)-equivariant & ✗ & ✓ & \(O(nk)\) \\ ColTNet [23] & SE(3)-equivariant & ✗ & ✗ & \(O(nk)\) \\ PairNN [31] & E(3)-equivariant & ✗ & ✓ & \(O(nk)\) \\ ComENet [24] & SE(3)-invariant & ✓ & ✓\({}^{*}\) & \(O(nk)\) \\ TFN [12] & SE(3)/E(3)-equivariant & ✗ & ✓ & \(O(nk)\) \\ Equiformer [32] & SE(3)/E(3)-equivariant & ✗ & ✓ & \(O(nk)\) \\ SphereNet [14] & SE(3)-invariant & ✓\({}^{*}\) & ✓\({}^{*}\) & \(O(nk^{2})\) \\ GemNet [33] & SE(3)-invariant & ✓\({}^{*}\) & ✓\({}^{*}\) & \(O(nk^{3})\) \\ \hline LEFTNet (Ours) & SE(3)/E(3)-equivariant & ✓ & ✓ & \(O(nk)\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Categorization of representative geometric GNN algorithms. \({}^{*}\) denotes partially satisfying the requirement.
message is aware of different local 3D structures; (2) frame transition encodings (**FTE**), such that the 3D GNN is aware of the equivariant coordinate transformation between different local patches.

We review the previous 3D GNNs following this framework and summarize the results in Table 1. For a fair comparison, we also list the computational complexity as it is often a trade-off of expressiveness (see the detailed analysis at the end of the next section). For **LSE**, SphereNet [14] and GemNet [33] (implicitly) encode the local 3D substructures by introducing a computation-intensive 2-hop edge-based update. For **FTE**, most 3D GNNs with equivariant vector update are able to express the local frame transitions (**FT**). While EGNN [13] is an exception, because it only updates the position vector (i.e. one channel), which is insufficient to express the whole **FT**. In other words, whether the update function \(\phi\) of (8) is powerful also affects the **FT** encoding. Except for equivariant update methods, models that encode torsion angle information also partially express **FTE** as illustrated in Appendix C. However, there is a trade-off between the efficiency and expressiveness in terms of number of hops considered for message passing.

Different from our invariant realization of **LSE**, Batatia et al. [25] builds its framework by constructing complete equivariant polynomial basis with the help of spherical harmonics and tensor product, where the monomial variables depend on different nodes (bodies). On the other hand, we realize the function of **LSE** and **FTE** through the edgewise scalarization \(A_{ij}\) and the equivariant message passing (see Fig. 3).

Recently, Joshi et al. [21] propose a **geometric k-WL test** (GWL) to measure the expressiveness power of geometric GNN algorithms. On a high level, our tree isomorphism is equivalent to the 1-hop geometric isomorphism as proposed in GWL, and the fine-grained triangular isomorphism lies between the 1-hop and 2-hop geometric isomorphism as proposed in GWL. From the model design point of view, our realization of **LSE** is through local scalarization, whose expressiveness is guaranteed by the Kolmogorov representation theorem (see [34]) and the universal approximator property of MLP. Moreover, the key concepts of measuring the expresive power in [21] are the body order and tensor order, which originate from classical inter-atomic potential theories and are of the equivariance nature. On the other hand, we discover the **FTE** as the'missing' bridge connecting local invariant scalars and global geometric expressiveness, which (together with **LSE** on mutual 3D substructures) also reveals why the 1-hop scalarization implemented in ClofNet [23] is insufficient.

## 7 Experiments

We test the performance of LEFTNet on both scalar value (e.g. energy) and vector value (e.g. forces) prediction tasks. The scalar value prediction experiment is conducted on the QM9 dataset [35] which includes \(134k\) small molecules with quantum property annotations; the vector value prediction experiment is conducted on the MD17 dataset [36] and the Revised MD17(rMD17) dataset [37] which includes the energies and forces of molecules. We compare our LEFTNet with a list of state-of-the-art equivariant (invariant) graph neural networks including SphereNet [14], PaiNN [31],

\begin{table}
\begin{tabular}{l c c c c c c c c c c c c} \hline \hline Task & \(\alpha\) & \(\Delta\varepsilon\) & \(\varepsilon_{\text{HOMO}}\) & \(\varepsilon_{\text{LUMO}}\) & \(\mu\) & \(C_{r}\) & \(G\) & \(H\) & \(R^{2}\) & \(U\) & \(U_{0}\) & ZPVE \\ Units & bohr\({}^{3}\) & meV & meV & meV & D & cal/mol K & meV & meV & bohr\({}^{3}\) & meV & meV & meV \\ \hline NMP &.092 & 69 & 43 & 38 &.030 &.040 & 19 & 17 &.180 & 20 & 20 & 1.50 \\ Comparant &.085 & 61 & 34 & 38 &.038 &.026 & 20 & 21 &.961 & 21 & 22 & 2.03 \\ Lieconv &.084 & 49 & 30 & 25 &.032 &.038 & 22 & 24 &.800 & 19 & 19 & 2.28 \\ TFN &.223 & 58 & 40 & 38 &.064 &.101 & - & - & - & - & - & - \\ SE3)-Tr. &.142 & 53 & 35 & 33 &.051 &.054 & - & - & - & - & - & - \\ EGNN &.071 & 48 & 29 & 25 & 0.029 &.031 & 12 & 12 & **.106** & 12 & 11 & 1.55 \\ SEGNN &.060 & 42 & 24 & 21 &.023 &.031 & 15 & 16 &.660 & 13 & 15 & 1.62 \\ ClofNet &.063 & 53 & 33 & 25 &.040 &.027 & 9 & 9 &.610 & 9 & **8** & **1.23** \\ EQGAT &.063 & 44 & 26 & 22 &.014 &.027 & 12 & 13 &.257 & 13 & 13 & 1.50 \\ Equformer &.056 & **33** & **17** & **16** &.014 &.025 & 10 & 10 & 10 &.227 & 11 & 10 & 1.32 \\ LEFTNet (ours) & **.048** & 40 & 24 & 18 & **.012** & **.023** & **7** & **6** &.109 & **7** & **6** & 1.33 \\ \hline Schnet &.235 & 63 & 41 & 34 &.033 &.033 &.033 & 14 & 14 &.073 & 19 & 14 & 1.70 \\ DimeNet++ &.044 & 33 & 25 & 20 &.030 &.023 & 8 & 7 &.331 & 6 & 6 & 1.21 \\ SphereNet &.046 & **32** & **23** & **18** &.026 & **.021** & 8 & 6 &.292 & 7 & **6** & **1.12** \\ ClofNet &.053 & 49 & 33 & 25 &.038 &.026 & 9 & 8 &.425 & 8 & 8 & 1.59 \\ PaiNN &.045 & 46 & 28 & 20 &.012 &.024 & 7 & **6** & **.066** & 6 & 6 & 1.28 \\ LEFTNet (ours) &.039 & 39 & **23** & **18** & **.011** &.022 & **6** & **5** &.094 & **5** & **5** & 1.19 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Mean Absolute Error for the molecular property prediction benchmark on QM9 dataset. (The best results are **bolded** and the second best are underlined.)
Equiformer [32], GemNet [33], etc [29; 38; 12; 39; 40; 13; 15; 41; 42; 43; 44; 45; 46; 47].The results on rMD17 and ablation studies are listed in Appendix E.

### QM9 - Scalar-valued Property Prediction

The QM9 dataset is a widely used dataset for predicting molecular properties. However, existing models are trained on different data splits. Specifically, Cormorant [40], EGNN [13], etc., use 100k, 18k, and 13k molecules for training, validation, and testing, while DimeNet [38], SphereNet [14], etc., split the data into 110k, 10k, and 11k. For a fair comparison with all baseline methods, we conduct experiments using both data splits. Experimental results are listed in Table 2. For the first data split, LEFTNet is the best on 7 out of the 12 properties and improves previous SOTA results by 20% on average. In addition, LEFTNet is the second best on 4 out of the other 5 tasks. Consistently, LEFTNet is the best or second best on 10 out of the 12 properties for the second split. These experimental results on both splits validate the effectiveness of LEFTNet on scalar-valued property prediction tasks. The ablation study in Appendix E shows that both **LSE** and **FTE** contribute to the final performance.

### MD17 - Vector-valued Property Prediction

We evaluate the ability of LEFTNet to predict forces on the MD17 dataset. Following existing studies [29; 38; 14], we train a separate model for each of the 8 molecules. Both training and validation sets contain 1000 samples, and the rest are used for testing. Note that all baseline methods are trained on a joint loss of energies and forces, but different methods use different weights of force over energy (WoFE). For example, SchNet [29] sets WoEF as 100, while GemNet [33] uses a weight of 1000. For a fair comparison with existing studies, we conduct experiments on two widely used weights of 100 and 1000 following Liu et al. [14]. The results are summarized in Table 3. We can observe that when WoFE is 100, LEFTNet outperforms all baseline methods on 7 of the 8 molecules and improves previous SOTA results by 16% on average. In addition, LEFTNet can outperform all baseline methods on 6 of the 8 molecules when WoFE is 1000. These experimental results on MD17 demonstrate the performance of LEFTNet on vector-valued property prediction tasks. The ablation study in Appendix E also demonstrates that both **LSE** and **FTE** are important to the final results.

## 8 Limitation and Future Work

In this paper, we seek a general recipe for building 3D geometric graph deep learning algorithms. Considering common prior of 2D graphs, such as permutation symmetry, has been incorporated in off-the-shelf graph neural networks, we mainly focus on the \(E(3)\) and \(SE(3)\) symmetry specific to 3D geometric graphs. Despite our framework being general for modeling geometric objects, we only conducted experiments on commonly used molecular datasets. It's worth exploring datasets in other domains in the future.

To elucidate the future design space of equivariant GNNs, we propose two directions that are worth exploring. Firstly, our current algorithms consider fixed equivariant frames for performing aggregation and node updates. Inspired by the high body-order ACE approach [48] (for modeling atom-centered potentials), it is worth investigating in the future if equivariant frames that relate to many body (e.g., the PCA frame in [49]) can boost the performance of our algorithm. For example, to

\begin{table}
\begin{tabular}{l c c c c c c|c c c c|c c} \hline \hline \multicolumn{11}{c}{} & \multicolumn{4}{c}{WoFE=100} & \multicolumn{4}{c}{WoFE=1000} & \multicolumn{4}{c}{Others} \\ \hline Molecule & sGDML & SchNet & DimeNet & SphereNet & SpoxyNet & LEFTNet & SphereNet & GemNet & LEFTNet & PaNN & NewtonNet \\ \hline Appirin & 0.68 & 1.35 & 0.499 & 0.430 & 0.258 & **0.210** & 0.209 & 0.217 & **0.196** & 0.371 & 0.348 \\ Benzene & 0.20 & 0.31 & 0.187 & 0.178 & – & **0.145** & 0.147 & 0.145 & **0.142** & – & – & – \\ Ethanol & 0.33 & 0.39 & 0.230 & 0.208 & **0.094** & 0.118 & 0.091 & **0.086** & 0.099 & 0.230 & 0.264 \\ Malonaldehyde & 0.41 & 0.66 & 0.383 & 0.340 & 0.167 & **0.159** & 0.172 & 0.155 & **0.142** & 0.319 & 0.323 \\ Naphthalene & 0.11 & 0.58 & 0.215 & 0.178 & 0.089 & **0.063** & 0.048 & 0.051 & **0.044** & 0.083 & 0.084 \\ Salicylic acid & 0.28 & 0.85 & 0.374 & 0.360 & 0.180 & **0.141** & **0.113** & 0.125 & 0.117 & 0.209 & 0.197 \\ Toluene & 0.14 & 0.57 & 0.210 & 0.155 & 0.087 & **0.070** & 0.054 & 0.060 & **0.049** & 0.102 & 0.088 \\ Uracil & 0.24 & 0.56 & 0.301 & 0.267 & 0.119 & **0.117** & 0.106 & 0.097 & **0.085** & 0.140 & 0.149 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Mean Absolute Error for per-atom forces prediction (kcal/mol Å) on MD17 dataset. Baseline results are taken from the original papers (with unit conversions if needed). All models are trained on energies and forces, and WoFE is the weight of force over energy in loss functions. The best results are **bolded**.
build the A-basis proposed in Puny et al. [49], we can replace our message aggregation Eq. 8 from summation to tensor product, which is also a valid pooling operation. Another direction is to explore geometric mesh graphs on manifolds \(M\), where the local frame is defined on the tangent space of each point: \(\mathcal{F}(x)\in T_{x}M\). Since our scalarization technique (crucial for realizing **LSE** in LEFT-Net) originates from differential geometry on frame bundles [22], it is reasonable to expect that our framework also works for manifold data [50, 51].