# Polarization constraints on the X-ray corona in Seyfert Galaxies:
MCG-05-23-16

A. Marinucci,1 F. Muleri,2 M. Dovčiak,3 S. Bianchi,4 F. Marin,5 G. Matt,4 F.
Ursini,4 R. Middei,6,7 H. L. Marshall,8 L. Baldini,9,10 T. Barnouin,5 N.
Cavero Rodriguez,11 A. De Rosa,2 L. Di Gesu,1 D. Harper,11 A. Ingram,12 V.
Karas,3 H. Krawczynski,11 G. Madejski,13 C. Panagiotou,8 P. O. Petrucci,14 J.
Podgorny,5,3,15 S. Puccetti,6 F. Tombesi,16,17,18 A. Veledina,19,20,21 W.
Zhang,22 I. Agudo,23 L. A. Antonelli,7,6 M. Bachetti,24 W. H. Baumgartner,25
R. Bellazzini,9 S. D. Bongiorno,25 R. Bonino,26,27 A. Brez,9 N.
Bucciantini,28,29,30 F. Capitanio,2 S. Castellano,9 E. Cavazzuti,1 S.
Ciprini,17,6 E. Costa,2 E. Del Monte,2 N. Di Lalla,13 A. Di Marco,2 I.
Donnarumma,1 V. Doroshenko,31,21 S. R. Ehlert,25 T. Enoto,32 Y. Evangelista,2
S. Fabiani,2 R. Ferrazzoli,2 J. A. Garcia,33 S. Gunji,34 K. Hayashida,35 J.
Heyl,36 W. Iwakiri,37 S. G. Jorstad,38,39 T. Kitaguchi,32 J. J.
Kolodziejczak,25 F. La Monaca,2 L. Latronico,26 I. Liodakis,40 S. Maldera,26
A. Manfreda,9 A. P. Marscher,38 I. Mitsuishi,41 T. Mizuno,42 C.-Y. Ng,43 S. L.
O’Dell,25 N. Omodei,13 C. Oppedisano,26 A. Papitto,7 G. G. Pavlov,44 A. L.
Peirson,13 M. Perri,6,7 M. Pesce-Rollins,9 M. Pilia,24 A. Possenti,24 J.
Poutanen,19,21 B. D. Ramsey,25 J. Rankin,2 A. Ratheesh,2 R. W. Romani,13 C.
Sgrò,9 P. Slane,45 P. Soffitta,2 G. Spandre,9 T. Tamagawa,32 F. Tavecchio,46
R. Taverna,47 Y. Tawara,41 A. F. Tennant,25 N. E. Thomas,25 A. Trois,24 S. S.
Tsygankov,19,21 R. Turolla,47,48 J. Vink,49 M. C. Weisskopf,25 K. Wu,48 F.
Xie,50,2 S. Zane48 E-mail<EMAIL_ADDRESS>(AM)

(Accepted XXX. Received YYY; in original form ZZZ)

###### Abstract

We report on the first observation of a radio-quiet Active Galactic Nucleus
(AGN) using polarized X-rays: the Seyfert 1.9 galaxy MCG-05-23-16. This source
was pointed with the Imaging X-ray Polarimetry Explorer (IXPE) starting on May
14, 2022 for a net observing time of 486 ks, simultaneously with XMM-Newton
(58 ks) and NuSTAR (83 ks). A polarization degree smaller than $\Pi<4.7\%$ (at
the 99% c.l.) is derived in the 2-8 keV energy range, where emission is
dominated by the primary component ascribed to the hot corona. The broad-band
spectrum, inferred from a simultaneous fit to the IXPE, NuSTAR, and XMM-Newton
data, is well reproduced by a power law with photon index $\Gamma=1.85\pm
0.01$ and a high-energy cutoff $E_{\rm C}=120\pm 15$ keV. A comparison with
Monte Carlo simulations shows that a lamp-post and a conical geometry of the
corona are consistent with the observed upper limit, a slab geometry is
allowed only if the inclination angle of the system is less than 50∘.

###### keywords:

Galaxies: active - Galaxies: Seyfert - Individual: MCG-05-23-16

††pubyear: 2022††pagerange: Polarization constraints on the X-ray corona in
Seyfert Galaxies: MCG-05-23-16–Polarization constraints on the X-ray corona in
Seyfert Galaxies: MCG-05-23-16

##  1 Introduction

It is now widely accepted that the primary X-ray emission of Seyfert galaxies
is produced by multiple up-scattering events of cool photons by hot electrons:
the Comptonization process (Sunyaev & Titarchuk, 1980; Zdziarski et al.,
2000). However, the energy supply of this medium and the conditions leading to
a formation of the hot plasma close to the black hole are debated. The
physical picture of plasma fuelling through the gravitational energy
transformation greatly depends on the geometry and size of the hot medium. In
one scenario, the energy dissipation (and electron heating) is distributed
over a large volume, with characteristic sizes $\sim 10-100\ R_{\rm g}$ (where
$R_{g}=GM/c^{2}$ is the gravitational radius, $G$ is the gravitational
constant, $M$ is black hole mass and $c$ is the speed of light). Early studies
considered the so-called slab-corona model, where the hot medium was assumed
to be distributed above the cold accretion disc (Haardt & Maraschi, 1991,
1993), and presumably energized by some disc instability, likely of magnetic
origin (Merloni, 2003). On the other hand, in a lamp-post geometry, the
primary X-ray emission is assumed to be coming from a compact source ($\sim
1-10\ R_{\rm g}$), located on the accretion disc axis (Fabian et al., 2017b)
and could be associated with an aborted jet (Ghisellini et al., 2004).

Spectroscopic analyses have in principle the capability to constrain the
coronal geometry but, even the best available observations provided by NuSTAR,
while good enough to measure the physical coronal parameters like the optical
depth and the temperature, are not able to distinguish statistically among
different geometries (Tortosa et al., 2018; Middei et al., 2019).

A very promising and powerful tool to assess the coronal geometry is
reverberation mapping of the corona-disc system (Uttley et al., 2014, and
references therein). In fact, the disc response to the corona illumination
depends also on the geometry of the latter (Wilkins et al., 2016). However, to
fully exploit this technique, observations with the next generation X-ray
observatories such as eXTP and Athena are required (De Rosa et al., 2019;
Dovciak et al., 2013), even if very long XMM-$Newton$ observations can already
deliver some results (Fabian et al., 2017a).

X-ray polarization provides an independent tool to constrain the coronal
geometry. Polarization, in fact, is extremely sensitive to the geometry of the
emitting matter and of the photon field (Schnittman & Krolik, 2010;
Beheshtipour et al., 2017; Tamborra et al., 2018; Zhang et al., 2019). With
the aim to constrain its coronal geometry, the Imaging X-ray Polarimetry
Explorer (IXPE: Weisskopf et al., 2016) observed the bright Seyfert galaxy
MCG-05-23-16.

MCG-05-23-16 is a nearby ($z$=0.0085 or 36 Mpc, Wegner et al. 2003) Seyfert
1.9 galaxy with broad emission lines in the near-infrared (Goodrich et al.,
1994). It has been extensively observed in X-rays (Weaver et al., 1997;
Balestra et al., 2004; Braito et al., 2007; Reeves et al., 2007; Perola et
al., 2002; Beckmann et al., 2008; Molina et al., 2013), showing a moderate
cold absorption ($N_{H}\sim$1022 cm-2). Recently, NuSTAR observations were
able to constrain the high energy cutoff ($E_{\rm C}\sim 100-160$ keV,
variable on a time scale of $\sim$100 ks) and therefore the coronal physical
parameters kTe and $\tau$ (Baloković et al., 2015; Zoghbi et al., 2017). X-ray
reverberation features have also been detected with XMM-Newton in this source
(Zoghbi et al., 2013; Kara et al., 2016).
In the optical and near-infrared wavelengths (0.4$-$2.2$\mu$m) the source
exhibits a low continuum linear polarization degree (1$-$2%) and a polarized
flux density which increases with wavelength, a possible sign of Compton
scattering or a different non-thermal component at work (Brindle et al.,
1990).

With a 2$-$10 keV flux of (7$-$10)$\times 10^{-11}$ erg cm-2 s-1 (Mattson &
Weaver, 2004), MCG-05-23-16 is one of the brightest Seyfert galaxies, only
moderately variable on both short and long time-scales, with a relatively
simple spectrum (no significant absorption in the IXPE band) and well measured
coronal parameters. It is therefore the ideal target to search for
polarization signatures of the coronal geometry in radio-quiet AGN. The paper
is organized as follows: in Sect. 2 we discuss the data reduction procedure,
while in Sect. 3 we present the spectro-polarimetric analyses. Our results are
then discussed and summarized in Sect. 4 and 5.

##  2 Observations and data reduction

IXPE (Weisskopf et al., 2022) observed MCG-05-23-16 starting on May 14, 2022
with its three Detector Units (DU), for a net exposure time of 47 ks. The
pointing of the source started again on May 21, for additional 439 ks. Cleaned
level 2 event files were produced and calibrated using standard filtering
criteria with the dedicated ftools tasks and the latest calibration files
available in the IXPE calibration database (CALDB 20211118). $I$, $Q$, $U$
Stokes background spectra were extracted from source-free circular regions
with a radius of 100 arcsec. Extraction radii for the $I$ Stokes spectra of
the source were computed via an iterative process which leads to the
maximization of the Signal-to-Noise Ratio (SNR) in the 2-8 keV energy band,
similar to the approach described in Piconcelli et al. (2004). We therefore
adopted circular regions centered on the source with radii of 62 arcsec, 57
arcsec and 62 arcsec for DU1, DU2 and DU3, respectively. The net exposure
times are 485.7 ks and the same extraction radii were then applied to the $Q$
and $U$ Stokes spectra. We used a constant energy binning of 0.2 keV for $Q$,
$U$ Stokes spectra and required a SNR higher than 5 in each spectral channel,
in the intensity spectra. $I$, $Q$, $U$ Stokes spectra from the three DUs are
always fitted independently in the following, but we will plot them together
using the setp group command in Xspec, for the sake of visual clarity.
Background represents the 2.0%, 1.8% and 2.1% of the total DU1, DU2 and DU3
$I$ spectra, respectively. The summed background subtracted light curves show
an average count rate $C_{\rm 2-8\ keV}$=0.525$\pm$0.002 cts s-1 with a level
of variability of $\sim$20$-$30%, in the range 0.33-0.79 cts s-1.

XMM-Newton started its observation on May 21, 2022 for 83 ks of elapsed time
with the EPIC CCD cameras: the pn (Strüder et al., 2001) and the two MOS
(Turner et al., 2001), operated in small window and medium filter mode. Data
from the MOS detectors are not included in our analysis due to pile-up. The
data from the pn camera show no significant pile-up as indicated by the
epatplot output. The extraction radii and the optimal time cuts for flaring
particle background were computed with SAS 20 (Gabriel et al., 2004) with the
same SNR maximization procedure reported above. The resulting optimal
extraction radii for the source and the background spectra are 40 and 50
arcsec, respectively. The net exposure time for the pn time-averaged spectrum
is 58.1 ks.

NuSTAR (Harrison et al., 2013) observed MCG-05-23-16 simultaneously to XMM-
Newton, with its two coaligned X-ray telescopes with corresponding Focal Plane
Module A (FPMA) and B (FPMB). The total elapsed time is 171.4 ks. The Level 1
data products were processed with the NuSTAR Data Analysis Software
(NuSTARDAS) package (v. 2.1.2). Cleaned event files (level 2 data products)
were produced and calibrated using standard filtering criteria with the
nupipeline task and the latest calibration files available in the NuSTAR
calibration database (CALDB 20220510). Extraction radii for the source and
background spectra were $40$ arcsec and 60 arcsec, FPMA spectra were binned in
order not to over-sample the instrumental resolution more than a factor of 2.5
and to have a SNR greater than 5 in each spectral channel, the same energy
binning was then applied to the FPMB spectra. The net observing times for the
FPMA and the FPMB data sets are 83.4 ks and 83 ks, respectively.
We adopt the cosmological parameters $H_{0}=70$ km s-1 Mpc-1,
$\Omega_{\Lambda}=0.73$ and $\Omega_{\rm m}=0.27$, i.e. the default ones in
Xspec 12.12.1 (Arnaud, 1996). Errors correspond to the 90% confidence level
for one interesting parameter ($\Delta\chi^{2}=2.7$), if not stated otherwise.

Figure 1: Normalized $U/I$ and $Q/I$ Stokes parameters are shown, calculated
using the full 2-8 keV IXPE band. Uncertainties are reported at the 68% c.l.

Figure 2: Left panel: The simultaneous EPIC pn and the grouped NuSTAR FPMA and
FPMB spectra of MCG-05-23-16 divided by the relative effective area are shown
with residuals. The best fitting model is shown as a solid grey line and the
different components as dashed lines. Right panel: IXPE $I$ (grey circles),
$Q$ (green circles) and $U$ (red empty circles) grouped Stokes spectra are
shown with residuals, along with the corresponding best fitting models. Note
the different scales on the $y$-axes for $I$ and $Q$/$U$ data.

##  3 Data analysis

###  3.1 IXPE polarimetric analysis

We start investigating the polarized signal from MCG-05-23-16 by analyzing its
polarization cubes, which are the simplest data structures holding
polarization information. They can be created using ixpeobssim (version
26.3.3: Baldini et al., 2022). This applies the Kislat et al. (2015) formalism
to a user-defined set of events to compute the Stokes parameters, the Minimum
Detectable Polarization (MDP: Elsner et al., 2012), the polarization degree,
the polarization angle, and the associated uncertainties. In our case, we
created one polarization cube for each DU and then one combining the three,
using the whole 2$-$8 keV band. Fig. 1 shows the normalized $U/I$ and $Q/I$
Stokes parameters, data counts from background regions have been subtracted.
Using the polarization cube from the three combined DUs, we retrieve a
MDP=4.3%, a polarization degree $\Pi=(2.1\pm 1.4)\%$ and angle $\Psi=(47\pm
19)^{\circ}$ (using 68% c.l. on one single parameter). We do not constrain any
energy dependence of the polarization properties.

Parameter | Best fitting values
---|---
cutoff power law
NH (cm-2) | $(1.35\pm 0.05)\times 10^{22}$
$\Gamma$ | $1.85\pm 0.01$
$E_{\rm C}$ (keV) | $120\pm 15$
$N$ | $(2.60\pm 0.05)\times 10^{-2}$
$\Pi$ (%) | $2.2\pm 1.7$
$\Psi$ (∘) | $50\pm 25$
xillver
$R$ | $0.30\pm 0.05$
$N$ | $2.14_{-0.1}^{+0.4}\times 10^{-2}$
$\Pi$ (%) | -
$\Psi$ (∘) | -
kerrdisk
$\theta$ (∘) | $48_{-8}^{+12}$
$N$ | $(3.9\pm_{-0.5}^{+0.8})\times 10^{-5}$
$\Pi$ (%) | -
$\Psi$ (∘) | -
$\chi^{2}$/dof | 1250/1169
F2-10 (erg cm-2 s-1) | $(7.45\pm 0.05)\times 10^{-11}$
L2-10 (erg s-1) | $(1.20\pm 0.02)\times 10^{43}$
cross-calibrations
constants | gain
$C_{\rm pn-DU1}$ | $1.09_{-0.01}^{+0.02}$ | $\alpha_{\rm DU1}$ | $0.953\pm 0.009$
$C_{\rm pn-DU2}$ | $1.06\pm 0.02$ | $\beta_{\rm DU1}$ | $0.07\pm 0.03$
$C_{\rm pn-DU3}$ | $0.97\pm 0.02$ | $\alpha_{\rm DU2}$ | $0.963_{-0.009}^{+0.007}$
$C_{\rm pn-FPMA}$ | $1.39\pm 0.01$ | $\beta_{\rm DU2}$ | $0.04\pm 0.03$
$C_{\rm pn-FPMB}$ | $1.43\pm 0.01$ | $\alpha_{\rm DU3}$ | $0.951_{-0.007}^{+0.009}$
|  | $\beta_{\rm DU3}$ | $0.07_{-0.04}^{+0.03}$
Table 1:  Best fit parameters from the joint fit. Normalization units are in
photons keV-1 cm-2 s-1. $R$ is the reflection fraction measured as the ratio
between the Compton reflection and the primary component fluxes between 20 and
40 keV. The 2$-$10 keV flux is retrieved from the EPIC-pn data.

###  3.2 XMM-Newton, NuSTAR and IXPE spectro-polarimetric analysis

We started modelling the simultaneous 2$-$10 keV XMM-Newton and 3$-$79 keV
NuSTAR spectra of MCG-05-23-16 with a model composed of an absorbed cutoff
power law (zTBabs $\times$ cutoffpl in Xspec) and a Compton reflection
component (xillver: García et al., 2013). The former reproduces the primary
continuum of the source while the latter takes into account reflection off
neutral, distant material. Galactic absorption is modeled with TBabs, using a
column density N${}_{\rm H}=7.8\times 10^{20}$ cm-2 (HI4PI Collaboration et
al., 2016) and multiplicative constants take into account cross-calibration
uncertainties between the FPMA, the FPMB and EPIC pn. The photon index and
cutoff energy of the reflection continuum is linked to the one of the primary
continuum, iron abundance is fixed to the solar one and the inclination angle
to $\theta=30^{\circ}$. The resulting $\chi^{2}$/dof is good (785/628) but
some residuals appear at $\sim$6 keV111The inferred energy of the Fe K$\alpha$
line in the pn spectrum is not consistent with 6.4 keV and we therefore added
a vashift component in the model. We retrieve $v=2230_{-550}^{+510}$ km s-1.
Since this effect is not found in the MOS spectra we conclude that is likely
due to calibration issues in the pn.. This could be indicative of a second
iron K$\alpha$ component, smeared by relativistic effects in the inner regions
of the accretion disc. Indeed, when compared with old XMM data, residuals are
perfectly consistent with the ones presented in Braito et al. (2007). A
further spectral component is therefore included: (kerrdisk: Brenneman &
Reynolds, 2006). The black hole spin is fixed to $a=0.998$, the emissivity to
$\epsilon(r)=r^{-3}$, the rest-frame energy of the emission line at 6.4 keV
and the inner radius of the disc to R${}_{\rm in}=37$ Rg (as reported in the
simultaneous XMM+Suzaku analysis: Reeves et al., 2007). We obtain a best fit
$\chi^{2}$/dof=683/625 (Fig. 2) and an inclination angle
$\theta=48_{-8}^{+12}\degr$. A more detailed analysis of the complete data set
will be presented in a forthcoming paper.

We then included IXPE $I$ Stokes spectra to the XMM and NuSTAR fit. We
followed the formalism discussed in Strohmayer (2017) and used the weighted
analysis method presented in Di Marco et al. (2022) (parameter stokes=Neff in
xselect). We obtain a $\chi^{2}$/dof=1378/1000 due to presence of large
residuals at the low and high energies in the IXPE $I$ spectra. This has
already been observed in other bright sources and can be likely explained in
terms of calibration issues (Taverna et al., 2022; Krawczynski et al., 2022).
We therefore modified the response files gains in the $I$ spectra (using gain
fit command) and obtained a $\chi^{2}$/dof=1055/994. We then included the $Q$
and $U$ Stokes spectra and linked their gain parameters to the ones of the $I$
spectra. Cross-calibration constants are included and the three spectral
components of the model are convolved with the polarization model polconst.
Two parameters can be then be inferred for each spectral component: the
polarization degree $\Pi$ and angle $\Psi$, both constant functions of the
energy. The polarization degree and angle associated to the nuclear continuum
are left free to vary while the ones associated to the other spectral
components are fixed to $\Pi=0\%$ and $\Psi=0^{\circ}$. We will verify a
posteriori that the fit is insensitive to these values. We retrieve the photon
index $\Gamma=1.85\pm 0.01$ and the cutoff energy $E_{\rm C}=120\pm 15$ keV,
in agreement with previously reported values (Baloković et al., 2015; Zoghbi
et al., 2017). The best spectro-polarimetric joint fit provides the
polarization degree $\Pi$=$(2.2\pm 1.7)$% and $\Psi$=$(50\pm 25)^{\circ}$ for
the nuclear power law. Therefore, at the 99% c.l. (for one single parameter of
interest, $\Delta\chi^{2}=6.63$) we only get an upper limit $\Pi=4.7\%$. The
contour plots between these two parameters are shown in Fig. 3. If the cutoff
power law is substituted by the Comptonization model compps (Poutanen &
Svensson, 1996) an electron temperature kTe=25$\pm 2$ keV and optical depth
$\tau=1.27\pm 0.08$ are retrieved, assuming a slab geometry
($\chi^{2}$/dof=1248/1169).
On theoretical grounds, the Compton reflection continuum is expected to be
moderately polarized but the iron K$\alpha$ fluorescence emission line arising
from the same scattering material is not (as shown in Goosmann & Matt, 2011;
Marin, 2018). We also note that scattering from distant AGN components do not
significantly impact the measured polarization due to their low contribution
in this energy band (Marin et al., 2018). However, we also tried to leave the
polarization degree of the xillver and kerrdisk components free to vary in the
fit. No statistically significant improvement is found
($\chi^{2}$/dof=1250/1169) and the fit is insensitive to these two parameters.

##  4 Discussion

Several radio-loud AGN have been observed in the first IXPE months of
operations and a highly significant polarized signal has been measured in two
blazars so far, due to synchrotron emission in the jet. On the other hand,
polarized X-rays produced in radio quiet sources are thought to be produced by
the inverse Compton mechanism, occurring within tens of gravitational radii
from the central black hole. Different geometries of the scattering medium,
depending on the degree of asymmetry, will result in a different degree of
polarization.
MCG-05-23-16 is the first radio quiet AGN observed by IXPE and the nuclear
power law component, ascribed to the hot corona, contributes to the 94$\pm$3%
of the 2-8 keV flux. We found that the best spectro-polarimetric fit of the
simultaneous IXPE-NuSTAR-XMM data provides only, at the 99% c.l., an upper
limit $\Pi=4.7\%$. Three different geometries for the hot corona in this AGN
have been recently explored in Ursini et al. (2022), with the Comptonization
code monk (Zhang et al., 2019). Fig. 3 shows the results with input coronal
temperatures kTe=25 keV and different Thomson optical depths $\tau$, which
correspond to the spectral shape of the continuum reported in Tab. 1. The
three coronal geometries are: a slab sandwitching the accretion disc, a
spherical lamp-post on the symmetry axis, and a truncated cone in outflow to
describe the base of a failed jet. The geometrical parameters for the
simulations shown in Fig. 3 are as follows. The slab corona fully covers a
truncated disc with an inner radius of 30 $R_{g}$. The spherical lamp-post has
a radius of 10 $R_{g}$ and a height above the disc of 30 $R_{g}$. As for the
truncated cone, the distance between the lower base and the disc and the
vertical thickness are both of 20 $R_{g}$, while the semi-aperture is 30∘, and
the outflow velocity is $0.3c$. Among these geometries, the slab corona is the
only one that gives a polarization vector perpendicular to the disc plane, the
other two predicting it to be parallel to the accretion disc. We note that the
polarization angle for the three geometries does not significantly vary for
different input kTe and $\tau$ values, while the absolute value of the
polarization degree can change. However, this does not impact much on the
relative differences between the three geometries.

Figure 3: Top-panel: contour plot between the polarization degree $\Pi$ and
angle $\Psi$ associated to the primary power law component. Purple, pink and
orange shaded regions indicate 68%, 90% and 99% confidence levels for two
parameters of interest, respectively. Bottom-panel: Monte Carlo simulations
performed with the Comptonization code monk. The input coronal parameters kTe,
$\tau$ correspond to the spectral shape reported in Tab. 1 and the
polarization angle is reported with respect to the accretion disc plane.

VLA observations at 8.4 GHz of the source showed a possible elongation in
PA$\simeq 169^{\circ}$ east of north (Mundell et al., 2009) while Hubble Space
Telescope WFPC2 images revealed [O iii] emission in PA$\simeq 40^{\circ}$
(Ferruit et al., 2000). The 90% c.l. contour plot shows that the polarization
angle is roughly aligned with the [O iii] emitting region, which likely traces
the Narrow Line Region (NLR) in this object. Assuming that the accretion disc
is perpendicular to the NLR, a slab corona geometry would produce such a
polarization angle, similar to what observed in the X-ray binary Cyg X-1
(Krawczynski et al., 2022). Of course, given the low significance of this
result, it must be taken as no more than suggestive, needing more data for a
confirmation. Highly inclined slab geometries (with $\theta\gtrsim
50^{\circ}$) can be ruled out and our estimate of $\theta=48_{-8}^{+12}\degr$,
from the broad iron K$\alpha$ profile, does not provide further constraints on
the hot coronal geometry.

##  5 Conclusions

The launch of IXPE, on December 9, 2021, opened a new observing window on the
study of supermassive black holes and MCG-05-23-16 is the perfect candidate to
investigate the hot corona close to the supermassive black hole.
Simultaneously observed with IXPE, XMM-Newton and NuSTAR in May 2022, the
source showed a low level of neutral absorption along the line of sight
(N${}_{\rm H}=1.35\pm 0.05\times 10^{22}$ cm-2), a modest level of Compton
reflection ($R=0.30\pm 0.05$) and broad relativistic iron K$\alpha$ emission
line. This translates into a power-law continuum which largely dominates the
total flux in the 2$-$8 keV IXPE band ($F_{\rm pow}/F_{\rm tot}=94\pm 3\%$)
and when it is convolved with a constant polarization model we obtain a 99%
c.l. upper limit to the polarization fraction of 4.7%. This result is
consistent with a spherical ("lamp-post") and a conical geometry of the
corona, while for a slab corona (with kTe=25 keV and $\tau=1.35$) it implies
an inclination angle less than 50∘.

## Acknowledgements

The Imaging X ray Polarimetry Explorer (IXPE) is a joint US and Italian
mission. The US contribution is supported by the National Aeronautics and
Space Administration (NASA) and led and managed by its Marshall Space Flight
Center (MSFC), with industry partner Ball Aerospace (contract NNM15AA18C). The
Italian contribution is supported by the Italian Space Agency (ASI) through
contract ASI-OHBI-2017-12-I.0, agreements ASI-INAF-2017-12-H0 and ASI-
INFN-2017.13-H0, and its Space Science Data Center (SSDC), and by the Istituto
Nazionale di Astrofisica (INAF) and the Istituto Nazionale di Fisica Nucleare
(INFN) in Italy. This research used data products provided by the IXPE Team
(MSFC, SSDC, INAF, and INFN) and distributed with additional software tools by
the High-Energy Astrophysics Science Archive Research Center (HEASARC), at
NASA Goddard Space Flight Center (GSFC). Part of the French contribution is
supported by the Scientific Research National Center (CNRS) and the French
spatial agency (CNES).

## References

  * Arnaud (1996) Arnaud K. A., 1996, in ASP Conf. Ser. 101: Astronomical Data Analysis Software and Systems V. p. 17
  * Baldini et al. (2022) Baldini L., Bucciantini N., Di Lalla N., Ehlert S. R., Manfreda A., Omodei N., Pesce-Rollins M., Sgrò C., 2022, arXiv e-prints, p. arXiv:2203.06384
  * Balestra et al. (2004) Balestra I., Bianchi S., Matt G., 2004, A&A, 415, 437
  * Baloković et al. (2015) Baloković M., et al., 2015, ApJ, 800, 62
  * Beckmann et al. (2008) Beckmann V., Courvoisier T. J. L., Gehrels N., Lubiński P., Malzac J., Petrucci P. O., Shrader C. R., Soldi S., 2008, A&A, 492, 93
  * Beheshtipour et al. (2017) Beheshtipour B., Krawczynski H., Malzac J., 2017, ApJ, 850, 14
  * Braito et al. (2007) Braito V., et al., 2007, ApJ, 670, 978
  * Brenneman & Reynolds (2006) Brenneman L. W., Reynolds C. S., 2006, ApJ, 652, 1028
  * Brindle et al. (1990) Brindle C., Hough J. H., Bailey J. A., Axon D. J., Ward M. J., Sparks W. B., McLean I. S., 1990, MNRAS, 244, 604
  * De Rosa et al. (2019) De Rosa A., et al., 2019, Science China Physics, Mechanics, and Astronomy, 62, 29504
  * Di Marco et al. (2022) Di Marco A., et al., 2022, AJ, 163, 170
  * Dovciak et al. (2013) Dovciak M., et al., 2013, arXiv e-prints, p. arXiv:1306.2331
  * Elsner et al. (2012) Elsner R. F., O’Dell S. L., Weisskopf M. C., 2012, in Space Telescopes and Instrumentation 2012: Ultraviolet to Gamma Ray. p. 84434N (arXiv:1208.0610), doi:10.1117/12.924889
  * Fabian et al. (2017a) Fabian A. C., Alston W. N., Cackett E. M., Kara E., Uttley P., Wilkins D. R., 2017a, Astronomische Nachrichten, 338, 269
  * Fabian et al. (2017b) Fabian A. C., Lohfink A., Belmont R., Malzac J., Coppi P., 2017b, MNRAS, 467, 2566
  * Ferruit et al. (2000) Ferruit P., Wilson A. S., Mulchaey J., 2000, ApJS, 128, 139
  * Gabriel et al. (2004) Gabriel C., et al., 2004, in F. Ochsenbein, M. G. Allen, & D. Egret ed., Astronomical Society of the Pacific Conference Series Vol. 314, Astronomical Data Analysis Software and Systems (ADASS) XIII. pp 759–+
  * García et al. (2013) García J., Dauser T., Reynolds C. S., Kallman T. R., McClintock J. E., Wilms J., Eikmann W., 2013, ApJ, 768, 146
  * Ghisellini et al. (2004) Ghisellini G., Haardt F., Matt G., 2004, A&A, 413, 535
  * Goodrich et al. (1994) Goodrich R. W., Veilleux S., Hill G. J., 1994, ApJ, 422, 521
  * Goosmann & Matt (2011) Goosmann R. W., Matt G., 2011, MNRAS, 415, 3119
  * HI4PI Collaboration et al. (2016) HI4PI Collaboration et al., 2016, A&A, 594, A116
  * Haardt & Maraschi (1991) Haardt F., Maraschi L., 1991, ApJ, 380, L51
  * Haardt & Maraschi (1993) Haardt F., Maraschi L., 1993, ApJ, 413, 507
  * Harrison et al. (2013) Harrison F. A., et al., 2013, ApJ, 770, 103
  * Kara et al. (2016) Kara E., Alston W. N., Fabian A. C., Cackett E. M., Uttley P., Reynolds C. S., Zoghbi A., 2016, MNRAS, 462, 511
  * Kislat et al. (2015) Kislat F., Clark B., Beilicke M., Krawczynski H., 2015, Astroparticle Physics, 68, 45
  * Krawczynski et al. (2022) Krawczynski H., et al., 2022, arXiv e-prints, p. arXiv:2206.09972
  * Marin (2018) Marin F., 2018, A&A, 615, A171
  * Marin et al. (2018) Marin F., Dovčiak M., Kammoun E. S., 2018, MNRAS, 478, 950
  * Mattson & Weaver (2004) Mattson B. J., Weaver K. A., 2004, ApJ, 601, 771
  * Merloni (2003) Merloni A., 2003, MNRAS, 341, 1051
  * Middei et al. (2019) Middei R., Bianchi S., Marinucci A., Matt G., Petrucci P. O., Tamborra F., Tortosa A., 2019, A&A, 630, A131
  * Molina et al. (2013) Molina M., Bassani L., Malizia A., Stephen J. B., Bird A. J., Bazzano A., Ubertini P., 2013, MNRAS, 433, 1687
  * Mundell et al. (2009) Mundell C. G., Ferruit P., Nagar N., Wilson A. S., 2009, ApJ, 703, 802
  * Perola et al. (2002) Perola G. C., Matt G., Cappi M., Fiore F., Guainazzi M., Maraschi L., Petrucci P. O., Piro L., 2002, A&A, 389, 802
  * Piconcelli et al. (2004) Piconcelli E., Jimenez-Bailón E., Guainazzi M., Schartel N., Rodríguez-Pascual P. M., Santos-Lleó M., 2004, MNRAS, 351, 161
  * Poutanen & Svensson (1996) Poutanen J., Svensson R., 1996, ApJ, 470, 249
  * Reeves et al. (2007) Reeves J. N., et al., 2007, PASJ, 59, 301
  * Schnittman & Krolik (2010) Schnittman J. D., Krolik J. H., 2010, ApJ, 712, 908
  * Strohmayer (2017) Strohmayer T. E., 2017, ApJ, 838, 72
  * Strüder et al. (2001) Strüder L., et al., 2001, A&A, 365, L18
  * Sunyaev & Titarchuk (1980) Sunyaev R. A., Titarchuk L. G., 1980, A&A, 86, 121
  * Tamborra et al. (2018) Tamborra F., Matt G., Bianchi S., Dovčiak M., 2018, A&A, 619, A105
  * Taverna et al. (2022) Taverna R., et al., 2022, arXiv e-prints, p. arXiv:2205.08898
  * Tortosa et al. (2018) Tortosa A., Bianchi S., Marinucci A., Matt G., Petrucci P. O., 2018, A&A, 614, A37
  * Turner et al. (2001) Turner M. J. L., et al., 2001, A&A, 365, L27
  * Ursini et al. (2022) Ursini F., Matt G., Bianchi S., Marinucci A., Dovčiak M., Zhang W., 2022, MNRAS, 510, 3674
  * Uttley et al. (2014) Uttley P., Cackett E. M., Fabian A. C., Kara E., Wilkins D. R., 2014, A&ARv, 22, 72
  * Weaver et al. (1997) Weaver K. A., Yaqoob T., Mushotzky R. F., Nousek J., Hayashi I., Koyama K., 1997, ApJ, 474, 675
  * Wegner et al. (2003) Wegner G., et al., 2003, AJ, 126, 2268
  * Weisskopf et al. (2016) Weisskopf M. C., et al., 2016, Results in Physics, 6, 1179
  * Weisskopf et al. (2022) Weisskopf M. C., et al., 2022, Journal of Astronomical Telescopes, Instruments, and Systems, 8, 026002
  * Wilkins et al. (2016) Wilkins D. R., Cackett E. M., Fabian A. C., Reynolds C. S., 2016, MNRAS, 458, 200
  * Zdziarski et al. (2000) Zdziarski A. A., Poutanen J., Johnson W. N., 2000, ApJ, 542, 703
  * Zhang et al. (2019) Zhang W., Dovčiak M., Bursa M., 2019, ApJ, 875, 148
  * Zoghbi et al. (2013) Zoghbi A., Reynolds C., Cackett E. M., Miniutti G., Kara E., Fabian A. C., 2013, ApJ, 767, 121
  * Zoghbi et al. (2017) Zoghbi A., et al., 2017, ApJ, 836, 2

Affiliations:
1ASI - Agenzia Spaziale Italiana, Via del Politecnico snc, 00133 Roma, Italy
2INAF Istituto di Astrofisica e Planetologia Spaziali, Via del Fosso del
Cavaliere 100, 00133 Roma, Italy
3Astronomical Institute of the Czech Academy of Sciences, Boční II 1401/1,
14100 Praha 4, Czech Republic
4Dipartimento di Matematica e Fisica, Università degli Studi Roma Tre, Via
della Vasca Navale 84, 00146 Roma, Italy
5Université de Strasbourg, CNRS, Observatoire Astronomique de Strasbourg, UMR
7550, 67000 Strasbourg, France
6Space Science Data Center, Agenzia Spaziale Italiana, Via del Politecnico
snc, 00133 Roma, Italy
7INAF Osservatorio Astronomico di Roma, Via Frascati 33, 00078 Monte Porzio
Catone (RM), Italy
8MIT Kavli Institute for Astrophysics and Space Research, Massachusetts
Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, USA
9Istituto Nazionale di Fisica Nucleare, Sezione di Pisa, Largo B. Pontecorvo
3, 56127 Pisa, Italy
10Dipartimento di Fisica, Università di Pisa, Largo B. Pontecorvo 3, 56127
Pisa, Italy
11Physics Department and McDonnell Center for the Space Sciences, Washington
University in St. Louis, St. Louis, MO 63130, USA
12School of Mathematics, Statistics, and Physics, Newcastle University,
Newcastle upon Tyne NE1 7RU, UK
13Department of Physics and Kavli Institute for Particle Astrophysics and
Cosmology, Stanford University, Stanford, California 94305, USA
14Université Grenoble Alpes, CNRS, IPAG, 38000 Grenoble, France
15Astronomical Institute, Charles University, V Holešovičkách 2, CZ-18000
Prague, Czech Republic
16Dipartimento di Fisica, Università degli Studi di Roma "Tor Vergata", Via
della Ricerca Scientifica 1, 00133 Roma, Italy
17Istituto Nazionale di Fisica Nucleare, Sezione di Roma "Tor Vergata", Via
della Ricerca Scientifica 1, 00133 Roma, Italy
18Department of Astronomy, University of Maryland, College Park, Maryland
20742, USA
19Department of Physics and Astronomy, 20014 University of Turku, Finland
20Nordita, KTH Royal Institute of Technology and Stockholm University, Hannes
Alfvéns väg 12, SE-10691 Stockholm, Sweden
21Space Research Institute of the Russian Academy of Sciences, Profsoyuznaya
Str. 84/32, Moscow 117997, Russia
22National Astronomical Observatories, Chinese Academy of Sciences, 20A Datun
Road, Beijing 100101, China
23Instituto de Astrofísica de Andalucía—CSIC, Glorieta de la Astronomía s/n,
18008 Granada, Spain
24INAF Osservatorio Astronomico di Cagliari, Via della Scienza 5, 09047
Selargius (CA), Italy
25NASA Marshall Space Flight Center, Huntsville, AL 35812, USA
26Istituto Nazionale di Fisica Nucleare, Sezione di Torino, Via Pietro Giuria
1, 10125 Torino, Italy
27Dipartimento di Fisica, Università degli Studi di Torino, Via Pietro Giuria
1, 10125 Torino, Italy
28INAF Osservatorio Astrofisico di Arcetri, Largo Enrico Fermi 5, 50125
Firenze, Italy
29Dipartimento di Fisica e Astronomia, Università degli Studi di Firenze, Via
Sansone 1, 50019 Sesto Fiorentino (FI), Italy
30Istituto Nazionale di Fisica Nucleare, Sezione di Firenze, Via Sansone 1,
50019 Sesto Fiorentino (FI), Italy
31Institut für Astronomie und Astrophysik, Universität Tübingen, Sand 1, 72076
Tübingen, Germany
32RIKEN Cluster for Pioneering Research, 2-1 Hirosawa, Wako, Saitama 351-0198,
Japan
33California Institute of Technology, Pasadena, CA 91125, USA
34Yamagata University,1-4-12 Kojirakawa-machi, Yamagata-shi 990-8560, Japan
35Osaka University, 1-1 Yamadaoka, Suita, Osaka 565-0871, Japan
36University of British Columbia, Vancouver, BC V6T 1Z4, Canada
37Department of Physics, Faculty of Science and Engineering, Chuo University,
1-13-27 Kasuga, Bunkyo-ku, Tokyo 112-8551, Japan
38Institute for Astrophysical Research, Boston University, 725 Commonwealth
Avenue, Boston, MA 02215, USA
39Department of Astrophysics, St. Petersburg State University, Universitetsky
pr. 28, Petrodvoretz, 198504 St. Petersburg, Russia
40Finnish Centre for Astronomy with ESO, 20014 University of Turku, Finland
41Graduate School of Science, Division of Particle and Astrophysical Science,
Nagoya University, Furo-cho, Chikusa-ku, Nagoya, Aichi 464-8602, Japan
42Hiroshima Astrophysical Science Center, Hiroshima University, 1-3-1
Kagamiyama, Higashi-Hiroshima, Hiroshima 739-8526, Japan
43Department of Physics, The University of Hong Kong, Pokfulam, Hong Kong
44Department of Astronomy and Astrophysics, Pennsylvania State University,
University Park, PA 16802, USA
45Center for Astrophysics | Harvard & Smithsonian, 60 Garden St, Cambridge, MA 02138, USA
46INAF Osservatorio Astronomico di Brera, Via E. Bianchi 46, 23807 Merate
(LC), Italy
47Dipartimento di Fisica e Astronomia, Università degli Studi di Padova, Via
Marzolo 8, 35131 Padova, Italy
48Mullard Space Science Laboratory, University College London, Holmbury St
Mary, Dorking, Surrey RH5 6NT, UK
49Anton Pannekoek Institute for Astronomy & GRAPPA, University of Amsterdam,
Science Park 904, 1098 XH Amsterdam, The Netherlands
50Guangxi Key Laboratory for Relativistic Astrophysics, School of Physical
Science and Technology, Guangxi University, Nanning 530004, China


contents=  Submitted to the Proceedings of the US Community Study on the
Future of Particle Physics (Snowmass 2021) ,
scale=1,placement=top,opacity=1,color=black,position=3.25in,1.2in

#  Symmetry Group Equivariant Architectures for Physics
A Snowmass 2022 White Paper

Alexander Bogatskiy  Flatiron Institute  Sanmay Ganguly  ICEPP, University of
Tokyo  Thomas Kipf  Google Research  Risi Kondor  University of Chicago  David
W. Miller Contact Editor<EMAIL_ADDRESS>University of Chicago
Daniel Murnane  Lawrence Berkeley National Laboratory  Jan T. Offermann
University of Chicago  Mariel Pettee Contact Editor<EMAIL_ADDRESS>Lawrence
Berkeley National Laboratory  Phiala Shanahan  Massachusetts Institute of
Technology  Chase Shimmin  Yale University  Savannah Thais  Princeton
University

###### Abstract

Physical theories grounded in mathematical symmetries are an essential
component of our understanding of a wide range of properties of the universe.
Similarly, in the domain of machine learning, an awareness of symmetries such
as rotation or permutation invariance has driven impressive performance
breakthroughs in computer vision, natural language processing, and other
important applications. In this report, we argue that both the physics
community and the broader machine learning community have much to understand
and potentially to gain from a deeper investment in research concerning
symmetry group equivariant machine learning architectures. For some
applications, the introduction of symmetries into the fundamental structural
design can yield models that are more economical (i.e. contain fewer, but more
expressive, learned parameters), interpretable (i.e. more explainable or
directly mappable to physical quantities), and/or trainable (i.e. more
efficient in both data and computational requirements). We discuss various
figures of merit for evaluating these models as well as some potential
benefits and limitations of these methods for a variety of physics
applications. Research and investment into these approaches will lay the
foundation for future architectures that are potentially more robust under new
computational paradigms and will provide a richer description of the physical
systems to which they are applied.

###### Contents

  1. 1 Executive Summary
  2. 2 Introduction
  3. 3 Equivariance in Machine Learning
  4. 4 Potential Benefits of Equivariant Models
  5. 5 Reconsidering Evaluation Metrics
  6. 6 Soft Equivariance
  7. 7 Discussion
  8. 8 Conclusions

##  1 Executive Summary

This White Paper highlights the potentially significant scientific and
practical benefits of developing machine learning architectures for current
and future physics applications that are inherently organized around symmetry
considerations and structures.

  * •

Symmetries underlie countless physical phenomena throughout the universe
across all length and energy scales. Mathematical descriptions of these
symmetries that we observe in nature have been invaluable in inspiring some of
the most successful physics theories to date.

  * •

Incorporating some kinds of symmetries (rotational, translational, sequential,
temporal, etc.) into machine learning architectures has already resulted in
significant performance benefits for models such as convolutional neural
networks (CNNs), recurrent neural networks (RNNs), graph neural networks
(GNNs), and transformers.

  * •

Despite the reliance of theoretical physics as a discipline on a variety of
symmetry groups, the explicit inclusion of these symmetries into models
designed for physics datasets is rare.

  * •

The potential benefits to physics applications with respect to non-equivariant
models range from model performance (e.g. better generalizability) to resource
efficiency during training (e.g. achieving strong performance with a highly
reduced training dataset and/or model parameters) to interpretability (e.g.
potentially mappable to meaningful physical observables).

  * •

The investment of time and funding into the construction of equivariant
architectures for physically-motivated symmetry groups will have far-reaching
benefits for applications across fundamental research in science and industry.
The development of symmetry group equivariant machine learning architectures
has substantial potential to aid in future physics discoveries, but only if we
recognize its importance by dedicating the necessary resources to support it.

##  2 Introduction

Nearly all areas of science and industry are witnessing a renaissance in the
development, application, and impacts of machine learning (ML) applied to both
old and new problems. Sophisticated and extremely diverse sets of algorithms
based on a wide array of ML architectures and design principles are breathing
new life into problems ranging from image recognition [1] to silicon sensor
readout [2], from intelligent molecular design [3] to parsing the landscape of
string theories [4] and accelerating theory calculations [5], and from
distinguishing quarks from gluons [6] to controlling telescopes searching the
cosmos [7]. As the computational and architectural landscape of ML continues
to evolve, it is essential to also reassess the fundamental mathematical
structures that underlie these approaches, especially when they are applied to
the modeling of physical systems.

One of the driving principles that has often guided the emergence of new
directions in theoretical physics and placed constraints on the methods used
to test them is symmetry. Global and gauge symmetries, continuous and discrete
symmetries, internal and external symmetries: the existence of any one of
these can place constraints on the dynamics of a system, require conservation
of quantities and quantum numbers, or reduce the dimensionality of the phase
space in which the system evolves. However, it is rare to see the explicit
inclusion of these symmetries into the ML algorithms used to model the
dynamics of particles or the evolution of the universe, for example, despite
the fact that symmetries are an inherent and often defining characteristic of
these systems and processes.

In this White Paper we highlight the potentially significant scientific and
practical benefits of developing machine learning architectures for current
and future physics applications that are inherently organized around symmetry
considerations and structures. Building these customized architectures for
physics use cases will require special prioritization and investment from our
community, as this pioneering work often falls outside of the scope of
traditional research grants. Given the strong motivations supporting the
potential of equivariant models to contribute to our scientific discoveries,
we cannot afford to miss out on the opportunity to deepen our understanding of
the relationship between physics and machine learning through the lens of
fundamental symmetries.

##  3 Equivariance in Machine Learning

Most problems in physics involve structured data that have some inherent
compatibility with symmetries: Euclidean vectors, Minkowski vectors,
indistinguishable particles, and so on. Similarly, the underlying processes
generating the data of interest, such as particle collisions or a Monte Carlo
simulation, possess definite symmetry properties. These problems may require
or at least benefit from models that construct intermediate representations
and perform computations that reflect the symmetry in question. Following this
approach, elegant architectures can be informed by these principles, and the
“building blocks” of such architectures may be limited to those allowed by the
imposed symmetries. Beyond imbuing the network design with properties that
mirror the physical system that it describes, this may – perhaps counter-
intuitively – improve out-of-distribution generalization, interpretability,
and uncertainty quantification, while also allowing for certain
simplifications of the model itself. Carefully and appropriately implemented,
this may therefore be a highly sought-after property in neural network design.

Mathematically, symmetries are usually described by groups. We can
characterize the relationship between a function (such as a neural network
layer) and a symmetry group by considering its equivariance properties. A map
$f:X\rightarrow Y$ is said to be equivariant w.r.t. the actions $\rho:G\times
X\to X$ and $\rho^{\prime}:G\times Y\to Y$ of a group $G$ on $X$ and $Y$ if

$f(\rho_{g}(x))=\rho_{g}^{\prime}\left(f(x)\right)\,$ (1)

for all $x\in X$ and $g\in G$. Here, the notation is
$\rho_{g}(x)\equiv\rho(g,x)$. Most commonly we are concerned with the case
where $X$ and $Y$ are vector spaces and $\rho_{g}$ and $\rho^{\prime}_{g}$ are
linear maps, in which case $\\{\rho_{g}\\}$ and $\\{\rho^{\prime}_{g}\\}$ are
so-called _linear representations_ of $G$. Intuitively, Equation 1 tells us
that for a given transformation of the input, the output of an equivariant
function transforms in a definite way that preserves the group
structure.111This is inherent in the definition of the group representation:
$\rho_{g}\rho_{h}\cdot f(x)=\rho_{gh}^{\prime}\cdot f(x)$. Note that in the
special case where $\rho_{g}$ is the identity for all $g\in G$, $f$ is said to
be invariant with respect to $G$. A diagram illustrating the distinction
between invariance and equivariance can be found in Figure 1.

Figure 1: An illustration of the differences between symmetry group invariance
and equivariance for the example case of identifying a handwritten letter in
an image. Here, $f:X\rightarrow Y$ is a map between vector spaces $X$ and $Y$.
$\rho_{g}(x)\equiv\rho(g,x)$ is an action of a group $G$ on $X$ and
$\rho^{\prime}_{g}(y)\equiv\rho^{\prime}(g,y)$ is an action of a group $G$ on
$Y$. The _invariant_ model (left) will output the same result on both the
original and translated images, while the _equivariant_ model (right) will
transform the translated image in a way that reflects the underlying symmetry
group. More formally, this means that the map $f$ is equivariant with respect
to the actions $\rho:G\times X\to X$ and $\rho^{\prime}:G\times Y\to Y$ if
$f(\rho_{g}(x))=\rho_{g}^{\prime}\left(f(x)\right)$ for all $x\in X$ and $g\in
G$.

At first, it may seem that invariance would be the most interesting form of
equivariance. For example, consider the problem of recognizing a letter at an
arbitrary location within an image. The data contains some continuous
variation due to a translation symmetry, yet it is desirable that an image
recognition algorithm meant to identify that letter – also referred to as a
classifier – be able to do so accurately and regardless of the letter’s
position. In this case, the classification algorithm should be invariant with
respect the group of translations. However, the overwhelming success of
Convolutional Neural Networks (CNNs) [8, 1] in image processing serves to
illustrate that even when an invariant is desired for the final output, it is
beneficial to construct more general equivariant intermediate representations.
CNN layers are not invariant, but rather preserve discrete translation
symmetries at each layer; if the input image is shifted by a certain number of
pixels, the output of the CNN layer (ignoring boundary effects) is shifted by
a corresponding amount [9, 10]. In a typical CNN, an invariant representation
is typically formed only in the last few layers of the network.

Intuitively, the reason CNNs are so effective on image data is because they
are able to learn specific features, such as edges or textures, that can be
matched at any location on the input image. This economizes parameters, since
the CNN need not learn the same features over and over at every location. This
also improves generalization, since a given feature can be detected even if it
appears at a location that was not observed in the training dataset.

This motivates us to consider whether new architectures can have a similar
impact by leveraging symmetries relevant to problems in physics, just as CNNs
did by exploiting the natural translation symmetry of image data, and doing so
by constructing equivariant latent representations of the inputs at every
layer of the network. In the context of problems with exact symmetries, it
therefore makes sense to consider architectures that are fully made up from
only equivariant operations. To this end, we must design “building blocks”
with equivariance properties corresponding to the symmetries of interest. Note
that various deviations from this approach cannot be dismissed, as will be
discussed in Section 6.

The overall viability of equivariant machine learning can be motivated by
mathematical results such as universality theorems. The most general form of
equivariant linear layers with respect to the action of any compact group was
derived in [11]. In [12] it was shown that for a wide class of Lie Groups, all
equivariant mappings can be expressed in terms of a finite basis of invariants
and equivariant tensors and easily approximated with conventional feed-forward
neural networks. Further, in [13, 14] it was shown that a fairly generic
equivariant architecture based on tensor products is sufficient to implicitly
generate the relevant finite basis. Thus, equivariance does not impose any
real constraints on the overall design of neural networks, other than
requiring all node-level operations to be equivariant. In particular,
recurrent, adversarial, generative, etc. architectures can be constructed out
of the equivariant “building blocks”, easily mimicking traditional
architectures.

These general ideas have already led to the development of multiple
equivariant architectures for sets (permutation invariance) [15, 16], graphs
(graph isomorphisms and permutation equivariance) [17], 3D data (spatial
rotations) [18, 19], homogeneous spaces of Lie groups such as the two-
dimensional sphere [20, 21], and even gauge equivariant systems [22, 23]. In
fact, the success of many of the more traditional architectures like CNNs and
Recurrent Neural Networks (RNN) can be largely attributed to their
equivariance properties [1].

Symmetries play a central role in any area of physics [24], and as such
physics may provide one of the widest variety of symmetry groups relevant in
computational problems. In particular, high-energy particle physics involves
symmetry groups ranging from $\mathrm{U}(1)$, $\operatorname{SU}(2)$ and
$\operatorname{SU}(3)$ to the orthochronous Lorentz group
$\operatorname{SO}^{+}(1,3)$, and even more exotic ones like $\mathrm{E}_{8}$
and conformal Lie algebras. Architectures that respect these symmetries may be
able to provide more sensible and tractable models with parameters that could
be directly interpreted in the context of known physical models, similar to
attention mapping studies for CNNs used in image recognition.

These ideas are general and may be applied in any field of computational
physics. In this White Paper we focus on a particular subset of applications
in particle physics where the data typically contain the energy-momentum
4-vectors of particles produced in collision events at high-energy particle
accelerators, or by simulation software used to model the collision events. As
4-vectors, these data naturally support an action of the Lorentz group,
therefore architectures that are Lorentz-invariant or Lorentz-equivariant are
a major step towards understanding the ways in which machine learning can
solve physical problems with such inputs. The first successful application was
for the Lorentz-invariant task of top-tagging [14], where a competitive
performance was reached despite a dramatic decrease in the number of
parameters. We are confident that in the near future these applications will
be extended to complex regression tasks and tasks with vector targets.

##  4 Potential Benefits of Equivariant Models

The success of neural networks is often attributed to the massive
computational capacities of modern computers, allowing one to practically
optimize models with billions of parameters. Mathematical statements like the
Universal Approximation Theorem [25], while not guaranteeing much in practice,
make it plausible that an arbitrarily complex system can be modeled and
learned even by a simple fully-connected neural network. Nevertheless, it is
likely that such “uninformed” architectures which make no reference to the
underlying structure of the system will not yield efficient nor possibly even
optimal models even if they can, in principle, provide a high accuracy of
predictions. This section elaborates on the potential benefits to consider in
the construction of more “informed” architectures. We also discuss why a focus
merely on the accuracy of predictions can be detrimental to machine learning
in scientific contexts, and how it can be re-balanced for more physics-
oriented model development.

As discussed in Section 3, many architectures are successful precisely because
they take advantage of some intrinsic properties of the corresponding system
under study – for instance, spatial translation invariance of image
recognition, ‘time’ translation equivariance in text, and permutation
equivariance in graph networks. Recently, generalizations of the symmetry
group of the convolution operation [26] as well as message passing graph
neural networks [27] show that the complexity of the learning task can be
significantly improved with fewer parameters. Symmetry equivariant transformer
networks can surpass the performance of the corresponding convolutional
networks [28], and symmetry equivariant normalizing flows can produce better
generated outputs than non-equivariant normalizing flows [29]. It has been
demonstrated that, in general, the symmetry equivariant networks can estimate
conserved quantities associated with the dynamics better, compared to a
general fully connected network [26, 30]. Some use cases such as accelerating
first-principles theoretical physics calculations suggest that building in
symmetries might be required for guarantees of exactness, or to properly
encode the theory. For the application of generative models to accelerate
lattice gauge field theory calculations, incorporating the high-dimensional
gauge symmetry exactly was found to be essential to train models [22, 23]. We
may conclude that the value of such architectures lies not just in the quality
of the predictions they produce, but also in a number of other properties that
make them better models, as opposed to simply better predictors:

  * •

Model size/complexity. With the symmetries of the underlying problem built-in,
the model size is constrained with respect to one that does not respect these
symmetries. For a given number of parameters, the symmetric model may use
fewer parameters more efficiently [31]. Lower-dimensional models may also be
desirable if they are to be used as physical models, but model size is also
crucial in many low-latency applications. Fewer parameters may also result in
a less complex loss landscape and thus more efficient optimization and
training dynamics. Section 6 expands on this topic with a discussion of
approximate and partial symmetries.

  * •

Interpretability. A model constrained by the symmetry group(s) of the
underlying problem may be more interpretable than a general one – not only for
likely having fewer parameters, but also because these parameters will
represent physically meaningful observables. In the context of CNNs, this fact
is used when interpreting convolutional kernels as translation-invariant
visual features. In a more general context, the parameters of a symmetric
model can be made themselves invariant, and therefore potentially direct
representations of measurable quantities. Additionally, by comparing
equivariant and non-equivariant architectures applied to the same problem, one
may be able to more easily probe the learning dynamics of the non-equivariant
model and characterize what non-symmetric information is being learned.

  * •

Sample efficiency. In contrast to the common way of training a generic
unconstrained model to respect symmetries, i.e., by augmenting the dataset
with a multitude of symmetry transformations, an equivariant model can
potentially achieve the same performance with a dramatically reduced training
dataset. For example, training a fully-connected network for image recognition
may require many modified copies of the input data for the network to “learn"
a translational or rotational symmetry, whereas a CNN requires fewer such
samples. Improvements in sample efficiency have been shown to range from
factors of a few to thousands [30, 32, 33].

  * •

Generalizability. If an equivariant model can learn an entire orbit (manifold
spanned by the action of the symmetry group) from a single sample, then,
similarly, any new input from the same orbit will produce an output entirely
determined by equivariance. In essence, symmetry allows the network to learn
exact reduced representations, and dimension reduction is a known method of
enhancing generalization []. Additionally, diverse training paradigms can be
applied to a single equivariant model architecture in order to extend the
generalizability of the model depending on a variety of end use cases. For
instance, if training efficiency is a priority, one could include symmetry-
breaking parameters initially during the training that subsequently decay,
thereby allowing a more efficient optimization pathway through nonphysical
parts of model space. Alternatively, for systems with approximate symmetries,
one could start with an exact equivariance requirement that is then loosened
later in training.

  * •

Faithfulness to physical laws. Finally, since equivariant architectures
implement physical symmetries as hard constraints “baked into” the structure
of the network itself, they can guarantee that whatever is learned by the
model is not going to violate known laws of nature. This is particularly
important in, e.g., quantum chemistry applications, where an incorrectly
learned interaction term could for example violate the law of conservation of
angular momentum.

While equivariant models are not yet commonplace in fundamental physics and
astrophysics, other fields such as computational chemistry, protein folding,
and more have seen clear benefits from these architectures – with one recent
and notable example being the success of AlphaFold [34] in protein structure
predictions.

Studying the above qualities of a model is likely to benefit both ML
practitioners and physics phenomenologists. For example, there is a challenge
in explaining the difference in performance between completely unconstrained
networks (e.g., ParticleNet [35]) and highly expert-engineered networks (e.g.,
the learnable linear basis described in [36]). Constructing equivariant (or
partially equivariant) models can help interpolate between these two edge
cases, and understand which features are important in the task. This is
clearly related to the qualities of interpretability and generalizability:
presumably overparameterized networks are, to some degree, exploiting
underlying symmetries of the system they are trained upon. However, if
networks are not encouraged to generalize according to the available
transformation properties then they will not explore or exploit these
symmetries well. As shown in Figure 2, jet tagging networks with Lorentz
equivariance are robust to unseen boosting regimes, whereas non-equivariant
designs may be sensitive to these transformations. The consequences of this
sensitivity are also clearly shown in [32] (Figure 3 of that work), where the
jet tagging performance of unconstrained networks diminishes for instance as
input data is Lorentz-boosted. For models where Lorentz equivariance is
enforced, generalization is almost perfect. It remains to be studied whether
partially equivariant networks [37] maintain this generalizability.

Figure 2: The accuracy of various neural networks in performing top quark
tagging, as a function of a Lorentz boost applied to the testing data
reference frame (parametrized with $\beta=v/c$). The simple, 4-layer fully-
connected network is not robust against Lorentz boosts, so the classification
accuracy suffers as the testing data has a hidden Lorentz boost applied. While
this can be potentially mitigated by careful training data augmentation – in
this case by applying random Lorentz boosts to training events – an
equivariant architecture like the Lorentz Group Network [14] can provide for
robustness without the need for any special training procedure. Decreased
accuracy at relatively high boosts for LGN is due to numerical precision
limits of that particular design. Network training and testing was performed
using the top-tagging reference dataset [38], and each curve is averaged over
three separately trained network instances.

There is an open question of whether an unconstrained network can learn the
symmetries of a physical system by being exposed to all possible
transformations in the space during training. Certainly data augmentation with
relevant symmetries can lead to improved performance [39], although there
remains to be done generalizability and interpretability studies of whether
this augmentation is leading to a genuine learning of the underlying
symmetries. It may be possible to use generative models to discover symmetries
on various physics datasets [40]. An additional promising direction may be to
recover symmetries by learning some latent representation of a physical system
(for example as a Structured World Model [41]) and study the symmetry
structure in this space [42].

##  5 Reconsidering Evaluation Metrics

The merits of a scientific method cannot be argued without first establishing
common metrics by which to judge such methods. Traditionally, machine learning
models are evaluated and compared based on simple performance metrics such as
accuracy, error, true/false positive rates, receiver operating characteristic
(ROC), and, to a lesser extent, computational efficiency. These metrics are
indeed good at evaluating the raw performance of a statistical model: they
judge a machine learning model by ability to reproduce or extract certain
features of the data. However, in many scientific applications, and especially
in physics, a good description of a system needs to do more than merely
reproduce observations. A good physical model of a system should make accurate
predictions while also satisfying several qualitative criteria. For example,
it should minimize the number of free parameters needed to express the
complexity of the system; these parameters should be chosen in a way that
clarifies their physical meaning as much as possible; the values of the
model’s parameters can be learned from a dataset that is smaller than that
needed for a generic model with many parameters; the model is generalizable
and makes accurate predictions even for experiments that have never been
conducted – in other words, all of the qualities that equivariance is designed
to enhance. These properties can act as a guide for extending a purely
statistical model to a more comprehensive physical description of a system.

In light of this discussion, we propose that the community focus on developing
and implementing a more holistic approach to evaluating and comparing machine
learning models in scientific applications. Here we list only a few common
ways of quantifying the above qualities of neural networks.

  * •

Quantifying model size. Model size (and/or capacity) is fairly trivial to
report, but is still often under-emphasized in the literature. The number of
parameters is the simplest metric of model size, but others metrics such as
memory usage can also be considered. Lowering the number of parameters,
assuming performance doesn’t suffer, brings one closer to a viable physical
model. The connection between model size and ease-of-training is non-trivial.
While large unconstrained models need to learn symmetries themselves, they are
also more likely to pick a "winning ticket", according to the Lottery Ticket
Hypothesis [43]. This suggests that it is useful to consider the effect of
model size in symmetry-constrained networks separately in training and
inference.

  * •

Quantifying inference efficiency. A typical use-case for physics-informed ML
is to miniaturize a model for inference on-the-edge in some experiment set-up.
We suggest that some measure of power-to-weight ratio may be useful when
surveying the wide variety of ML architectures and features applied to a
physics task. In [37] one such measure is proposed: an “ant factor” that
represents the performance of a network relative to its size. Here
“performance” and “size” should be defined by the scientific field as is most
relevant. An example of this ant factor (based on background rejection rate)
is given in Figure 3 as a function of the number of equivariant channels vs.
non-equivariant channels. This sort of exploration could guide a community to
an architecture most appropriate to its hardware budget or latency
constraints.

  * •

Quantifying sample efficiency. Sample efficiency is commonly illustrated via
learning curves. Depending on the dataset, equivariant neural networks can
achieve equal performance with only a fraction of the training data, and such
comparisons should ideally be reported in publications.

  * •

Quantifying variance/bias tradeoff. Closely related to the above, since
equivariant networks operate in a hypothesis space of reduced size, they are
less likely to overfit the data, specifically they are prevented from
overfitting the data in a way that would violate the underlying group
symmetries.

  * •

Quantifying generalizability and interpretability. These two properties are
much more difficult to quantify, and it is a topic ongoing research in the
machine learning community. Transfer learning, the process of training a
network in some region of phase space and gauging its performance in another,
can be used to demonstrate generalization. Methods such as network dissection
[44], originally suited for CNNs, can be extended to arbitrary equivariant
architectures to provide metrics of interpretability. Notably, this may be
easier to do in scientific applications where the tasks are typically less
complex than in general purpose image recognition, for example.

All this is not to say that pure model performance is unimportant, or that it
needs to be sacrificed in favor of other properties. In fact, equivariant
architectures already provide state-of-the-art predictions in scientific
contexts, as seen, for example, in chemistry and high energy physics [45, 32],
and we believe that further research of equivariant architectures will only
increase the performance gap between them and generic networks applied to
physics problems. Nevertheless, the value of these models is not only in their
raw performance, and we hope that this paper will help convince other
scientists in the community that a broader approach to network design that is
less focused on short-term performance gains can facilitate bridge-building
between machine learning and physics.

Figure 3: An example of quantifying model efficiency as a function of its
symmetry-constrained properties. Unconstrained learning is along the x-axis,
Lorentz-constrained learning is along the y-axis, and lighter colors show
greater efficiency (as captured by the “ant factor”, a power-to-weight ratio)

##  6 Soft Equivariance

Exact symmetries do not exhaust the possibilities for symmetry-informed models
and problems. Many real-world problems deal only with approximate symmetries
or combine symmetric and non-symmetric types of data. This issue comes up
already in typical applications of CNNs for image recognition: the
convolutional layers of CNNs provide translation-equivariant latent
representations, but the network output emerges from a final, non-equivariant
stage of the network (say, a dense layer). Moreover, images are always finite,
and edge effects are a very well known issue for CNNs. The translational
symmetry is realized only with respect to a subset of the full symmetry group.
In machine learning, the relevant distinction is between soft and hard
inductive biases. Up to this point, we have focused on the hard inductive bias
introduced by a rigidly equivariant architecture. Here we briefly discuss the
possibility of softer versions of that constraint. We can distinguish several
types of soft or broken symmetry:

  * •

Infinitesimal symmetry. Finite transformations may not always be a symmetry
for a problem, but their infinitesimal versions might. For example, in image
recognition and neuroscience, learning the infinitesimal generators of
transformations is common. Mathematically, such models work with
representations of Lie algebras as opposed to groups.

  * •

Partial symmetry. By this we mean symmetry with respect to a subset of the
full group of transformations (for example translations only up to a finite
distance). In this case, approaches based on linear representation theory can
still be effective because most of the structure of representations is
determined just by the local neighborhood of the identity in the group.
Including non-equivariant sub-networks also becomes reasonable.

  * •

Approximate symmetry. In many problems the symmetry transformations aren’t
expected to work in an exact manner at all – this could be due to the presence
of noise, interactions between different parts of the data, or even
fundamental physical effects such as symmetry breaking. In these cases it is
possible to relax the symmetry constraint gradually by allowing limited data
mixing between different irreducible representations that wouldn’t interact
under exact symmetry.

One well-known example of a synthesis between equivariant and non-equivariant
methods is AlphaFold [34], whose key component is an equivariant transformer
[46]. Examples of models using partial equivariance for physics tasks include
GNNs for jet tagging [37] and residual networks for dynamical systems [47].

##  7 Discussion

When developing a neural network approach for any task in physics, one should
consider the specifics of the task when determining what kind of architecture
to use: how the data is represented, which symmetry group(s) govern it, and
what kind of information one is trying to extract. Below are a few practical
points to keep in mind when considering the development and use of equivariant
networks.

  * •

Development and training time: At present, the main deterrent to a more
widespread use of equivariant architectures is the time investment they
require for model design, development, and training – which includes the time
possibly needed for the technical implementation of complicated mathematical
symmetry group structures, as well as additional computing time: During the
training process, the forward and backward propagation steps may be less
optimized than those of simpler models and not benefit from optimized tensor
operations as easily, thus significantly extending the required training time.
However, equivariant models generally require significantly smaller training
datasets than simpler models, potentially offsetting the training
inefficiencies introduced by the more complex mathematical computations during
training. Furthermore, the equivariant “building blocks” of such architectures
are typically highly reusable, so it can be the case that the time-consuming
early development for a given symmetry may not need to be repeated.
Additionally, the development of equivariant models for physics tasks opens
potential avenues for collaboration with the broader ML community as many
existing architectures for enforcing Euclidean equivariance may be easily
modified to accommodate more exotic symmetries.

  * •

High performance computing: For the full exploitation of equivariant networks
– and more generally machine learning for physics – it will be important for
physics-specific equivariant architectures to be optimized for deployment on
exascale computing hardware. This will require some investment and recognition
on the HPC side that not all applications of ML to physics will use “out-of-
the-box” designs like CNNs, and so hardware benchmarking and acceptance tests
on limited default architectures may not meet all of our community needs.

  * •

Data dimensionality: One should consider the dimensionality of the sample data
used for training – or more precisely, the dimensionality of the data versus
that of its symmetry group. While the use of equivariant networks can improve
sample efficiency, the added computational cost may outweigh these benefits,
especially for low-dimensional symmetry groups. For example, in [23] it was
found that imposing gauge symmetry significantly improved model efficiency,
but imposing a residual hypercubic symmetry was expensive and did not improve
performance. In addition, network training dynamics can be complicated when
the training path is restricted to lie in symmetry-respecting space, and in
some cases it may be more efficient to allow the network’s optimisation
trajectory to move through non-physical model space.

  * •

Towards a general-use toolkit: There has been significant effort spent in
transferring physics-informed ML ideas from the computer vision community to
the physics community (and indeed proving that these ideas are even
applicable). As pointed out above, for a particular use-case, components of an
equivariant architecture may be re-usable and transferable. However, these
deep explorations of a particular symmetry should run in parallel with an
attempt to generalize equivariance tools across symmetry groups, scientific
fields, and core architectures (CNNs, GNNs, RNNs, transformers, etc.). Most
open-source libraries for implementing equivariance are currently highly
specialized in this regard. This is clearly not a trivial task, with several
significant obstacles. Group calculations are usually performed symbolically
(for example in Sympy [48]), then converted to convolutions or numerical
formulae by hand. Libraries for this are specialized to particular families of
symmetries and it would be a Herculean task to generalize these across
multiple families of symmetry. A second obstacle that a general toolkit would
face is that many of the highly optimized algorithms that enable unconstrained
ML are non-trivial within an equivariant architecture. For example, nearest-
neighbor algorithms are the backbone of many graph neural network approaches,
and within Euclidean space there are dozens of libraries available. However,
libraries for performing neighbor searches in curved or non-positive-definite
space (e.g. spacetime) are essentially non-existent. Some headway may be made
across these obstacles by use of approximations of symmetries, or low-order
truncations of group operations (for example, by the relatively simple
convolutions suggested in [27]).

##  8 Conclusions

Symmetries underlie countless physical phenomena throughout the universe
across all length and energy scales. This White Paper puts forth the arguments
for incorporating knowledge of these symmetries and their equivariance
properties into the algorithms and architectures that scientists use to
investigate and describe numerous physical systems, with a particular emphasis
on the potential for impacts in fundamental particle physics. Numerous
advantages and potential limitations of such an approach are discussed, along
with proposals for developing and implementing a more holistic approach to
evaluating and comparing machine learning models in scientific applications.
The investment of time and funding into the construction of equivariant
architectures for physically-motivated symmetry groups is likely to have far-
reaching benefits for applications across fundamental research in science and
industry. The development of symmetry group equivariant machine learning
architectures has substantial potential to aid in future physics discoveries,
but only if we recognize its importance by dedicating the necessary resources
to support it.

## References

  * [1] W. Rawat and Z. Wang, _Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review_ , _Neural Computation_ 29 (2017) 2352.
  * [2] ATLAS Collaboration, _A neural network clustering algorithm for the ATLAS silicon pixel detector_ , _JINST_ 9 (2014) P09009 [1406.7690].
  * [3] G. Schneider and P. Wrede, _Artificial neural networks for computer-based molecular design_ , _Prog. Biophys_ 70 (1998) 175.
  * [4] J. Carifio, J. Halverson, D. Krioukov and B.D. Nelson, _Machine Learning in the String Landscape_ , _JHEP_ 09 (2017) 157 [1707.00655].
  * [5] M.S. Albergo, G. Kanwar and P.E. Shanahan, _Flow-based generative models for Markov chain Monte Carlo in lattice field theory_ , _Phys. Rev. D_ 100 (2019) 034515 [1904.12072].
  * [6] P.T. Komiske, E.M. Metodiev and M.D. Schwartz, _Deep learning in color: towards automated quark/gluon jet discrimination_ , _JHEP_ 01 (2017) 110 [1612.01551].
  * [7] D.G. Sandler, T.K. Barrett, D.A. Palmer, R.Q. Fugate and W.J. Wild, _Use of a neural network to control an adaptive optics system for an astronomical telescope_ , _Nature_ 351 (1991) 300.
  * [8] Y. LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard et al., _Handwritten digit recognition with a back-propagation network_ , _Advances in Neural Information Processing Systems_ 2 (1989) .
  * [9] T. Cohen and M. Welling, _Group Equivariant Convolutional Networks_ , _Proceedings of the 33rd International Conference on Machine Learning, ICML 2016_ 48 (2016) 2990 [1602.07576].
  * [10] S. Dieleman, J.D. Fauw and K. Kavukcuoglu, _Exploiting cyclic symmetry in convolutional neural networks_ , [1602.02660].
  * [11] R. Kondor and S. Trivedi, _On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups_ , _Proceedings of the 35th International Conference on Machine Learning, ICML 2018_ 80 (2018) 2747.
  * [12] D. Yarotsky, _Universal approximations of invariant maps by neural networks_ , _CoRR_ (2018)  [1804.10306].
  * [13] R. Kondor, _N-body networks: a covariant hierarchical neural network architecture for learning atomic potentials_ , [1803.01588].
  * [14] A. Bogatskiy, B. Anderson, J.T. Offermann, M. Roussi, D.W. Miller and R. Kondor, _Lorentz group equivariant neural network for particle physics_ , _Proceedings of the 37th International Conference on Machine Learning, ICML 2020_ (2020)  [2006.04780].
  * [15] M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R.R. Salakhutdinov and A.J. Smola, _Deep Sets_ , _Advances in Neural Information Processing Systems 30_ (2017) 3391.
  * [16] M.J. Dolan and A. Ore, _Equivariant energy flow networks for jet tagging_ , _Physical Review D_ 103 (2021)  [2012.00964].
  * [17] H. Maron, H. Ben-Hamu, N. Shamir and Y. Lipman, _Invariant and equivariant graph networks_ , _International Conference on Learning Representations, ICLR 2019_ (2019)  [1812.09902].
  * [18] F. Monti, D. Boscaini, J. Masci, E. Rodolà, J. Svoboda and M.M. Bronstein, _Geometric deep learning on graphs and manifolds using mixture model cnns_ , _IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017_ (2017) 5425.
  * [19] N. Thomas, T. Smidt, S.M. Kearnes, L. Yang, L. Li, K. Kohlhoff et al., _Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds_ , _NeurIPS 2018 Workshop on Molecules and Materials_ (2018)  [1802.08219].
  * [20] T.S. Cohen, M. Geiger, J. Köhler and M. Welling, _Spherical CNNs_ , in _International Conference on Learning Representations_ , 2018, https://openreview.net/forum?id=Hkbd5xZRb.
  * [21] R. Kondor, Z. Lin and S. Trivedi, _Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network_ , in _NeurIPS 2018_ , S. Bengio, H.M. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett, eds., pp. 10138–10147, 2018, http://papers.nips.cc/paper/8215-clebschgordan-nets-a-fully-fourier-space-spherical-convolutional-neural-network.
  * [22] G. Kanwar, M.S. Albergo, D. Boyda, K. Cranmer, D.C. Hackett, S. Racanière et al., _Equivariant flow-based sampling for lattice gauge theory_ , [2003.06413].
  * [23] D. Boyda, G. Kanwar, S. Racanière, D.J. Rezende, M.S. Albergo, K. Cranmer et al., _Sampling using $SU(N)$ gauge equivariant flows_, [2008.05456].
  * [24] T. Frankel, _The geometry of physics_ , Cambridge University Press, Cambridge, second ed. (2004).
  * [25] K. Hornik, M. Stinchcombe and H. White, _Multilayer feedforward networks are universal approximators_ , _Neural Netw_ 2 (1989) 359.
  * [26] M. Finzi, S. Stanton, P. Izmailov and A.G. Wilson, _Generalizing convolutional neural networks for equivariance to lie groups on arbitrary continuous data_ , _Proceedings of the 37th International Conference on Machine Learning, ICML 2020_ 119 (2020) 3165 [2002.12880].
  * [27] V.G. Satorras, E. Hoogeboom and M. Welling, _E(n) equivariant graph neural networks_ , _Proceedings of the 38th International Conference on Machine Learning, ICML 2021_ (2021)  [2102.09844].
  * [28] M. Hutchinson, C.L. Lan, S. Zaidi, E. Dupont, Y.W. Teh and H. Kim, _Lietransformer: Equivariant self-attention for lie groups_ , [2012.10885].
  * [29] V.G. Satorras, E. Hoogeboom, F.B. Fuchs, I. Posner and M. Welling, _E(n) equivariant normalizing flows for molecule generation in 3d_ , _CoRR_ abs/2105.09016 (2021)  [2105.09016].
  * [30] S. Batzner, A. Musaelian, L. Sun, M. Geiger, J.P. Mailoa, M. Kornbluth et al., _E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials_ , [2101.03164].
  * [31] T.S. Cohen and M. Welling, _Steerable CNNs_ , in _International Conference on Learning Representations_ , 2017, https://openreview.net/forum?id=rJQKYt5ll.
  * [32] S. Gong, Q. Meng, J. Zhang, H. Qu, C. Li, S. Qian et al., _An efficient Lorentz equivariant graph neural network for jet tagging_ , [2201.08187].
  * [33] M. Dax, S.R. Green, J. Gair, M. Deistler, B. Schölkopf and J.H. Macke, _Group equivariant neural posterior estimation_ , 2021.
  * [34] J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger et al., _Highly accurate protein structure prediction with AlphaFold_ , _Nature_ 596 (2021) 583.
  * [35] H. Qu and L. Gouskos, _Jet tagging via particle clouds_ , _Physical Review D_ 101 (2020) .
  * [36] P.T. Komiske, E.M. Metodiev and J. Thaler, _Energy flow polynomials: a complete linear basis for jet substructure_ , _Journal of High Energy Physics_ 2018 (2018) .
  * [37] D. Murnane, S. Thais and J. Wong, _Semi-Equivariant GNN Architectures for Jet Tagging_ , [2202.06941].
  * [38] G. Kasieczka, T. Plehn, J. Thompson and M. Russel, _Top quark tagging reference dataset_ , _Zenodo_ (2019) .
  * [39] B.M. Dillon, G. Kasieczka, H. Olischlager, T. Plehn, P. Sorrenson and L. Vogel, _Symmetries, safety, and self-supervision_ , [2108.04253].
  * [40] K. Desai, B. Nachman and J. Thaler, _SymmetryGAN: Symmetry discovery with deep learning_ , [2112.05722].
  * [41] T. Kipf, E. van der Pol and M. Welling, _Contrastive learning of structured world models_ , [1911.12247].
  * [42] J.Y. Park, O. Biza, L. Zhao, J.-W. van de Meent and R. Walters, _Learning symmetric representations for equivariant world models_ , .
  * [43] J. Frankle and M. Carbin, _The lottery ticket hypothesis: Finding sparse, trainable neural networks_ , [1803.03635].
  * [44] D. Bau, B. Zhou, A. Khosla, A. Oliva and A. Torralba, _Network dissection: Quantifying interpretability of deep visual representations_ , _Proceedings of the IEEE conference on computer vision and pattern recognition_ (2017) 6541.
  * [45] B.M. Anderson, T. Hy and R. Kondor, _Cormorant: Covariant Molecular Neural Networks_ , in _NeurIPS 2019_ , H.M. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alché-Buc, E.B. Fox and R. Garnett, eds., pp. 14510–14519, 2019, http://papers.nips.cc/paper/9596-cormorant-covariant-molecular-neural-networks.
  * [46] F.B. Fuchs, D.E. Worrall, V. Fischer and M. Welling, _SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks_ , _arXiv_ (2020)  [2006.10503].
  * [47] M. Finzi, G. Benton and A.G. Wilson, _Residual pathway priors for soft equivariance constraints_ , [2112.01388].
  * [48] A. Meurer, C.P. Smith, M. Paprocki, O. Čertík, S.B. Kirpichev, M. Rocklin et al., _Sympy: symbolic computing in python_ , _PeerJ Computer Science_ 3 (2017) e103.