Addentum to Class. Quant. Grav. 24 (2007), 1683-1686 [1]

Caltech # 2021-043

Brandeis # BRX-TH 6695

Higher spin $m\not=0$ excitations on curved backgrounds and cosmological
supergravity

S. Deser1,2 and M. Henneaux3,4

1Walter Burke Institute for Theoretical Physics Caltech, Pasadena CA 91125,
USA;

2Physics Department, Brandeis university, Waltham MA 02454, USA;

3Physique Théorique et Mathématique & International Solvay Institutes,

Université Libre de Bruxelles, Campus Plaine C.P. 231, B-1050 Bruxelles,
Belgium;

4Collège de France, 11 place Marcelin Berthelot, 75005 Paris, France

Some time ago, we showed that a weak (linear) massless spin 2 wave could only
propagate on a Ricci-flat or Ricci-constant background: it must necessarily be
a perturbation of General Relativity. This was just the continuation of the
higher spin chain: massless $s=3/2$ also required Ricci-flatness (which is the
basis of supergravity) while $s>2$ needs Riemann flatness. This note re-
analyzes the problem in a perhaps more physical way: by considering massless
and – only apparently – massive small excitations and showing how their
parameters relate to the cosmological constant. We will thus prove, in a
simple physical way, the necessity, as well as the sufficiency, of the known
supergravity results.

##  1 Introduction

In [1], we showed that massless linear $s=2$ excitations, of a very general a
priori form, could only propagate on a Ricci –, or equivalently Einstein –,
flat or constant space: they had to be effectively perturbations of the
general relativity (GR) background. This result fits neatly into a well-known
sequence: massive or massless $s<3/2$ moves in any space, $s=3/2$ only in a
Ricci-flat (if massless) or “tuned” constant (if massive) one (whence
supergravity (SUGRA) in the coupling to gravity), while all higher than $s=2$
fields require a Riemann-flat background, hence no “hypergravities” with
$s=5/2$ or higher [2]. [Even if $s=5/2$ can live on a fixed, non-dynamical,
background such as AdS, that is as unsatisfactory as living in flat space, for
the same reason.]

A deeper physics explanation for the above result is that spacetime in GR is
not a fixed arena, flat or otherwise, but a dynamical system with its own
action. When added to that of spin $2$ $\sim\int DhDh$, the resulting new
equations read $G_{\mu\nu}(g)=T_{\mu\nu}(h)$. Given the linear $\Box
h+\cdots=0$ equation, the now obvious result is that the total system is
equivalent to pure GR, but with action $\sim\int R(g+h)$, to quadratic
$h$-order. Without this, the spin $2$ field would just be a test field, with
negligible $T_{\mu\nu}$, hence of negligible interest.

A more involved, but similar, story holds for $s=3/2$ and SUGRA rather than
just GR. The combined action of the massless spin $3/2$ field plus GR is
consistent because – and only because – the “Bianchi identities” of the $3/2$
equations, $\sim(Ricci)D\psi_{\mu}=0$ hold using $Ricci\sim T_{\mu\nu}(\psi)$
and the rather involved cubic Fierz identities. Underlying this is of course
the local supersymmetry of the combined system. Otherwise, the spin $3/2$
field would have to be a dull test-field, with a weak $T_{\mu\nu}$ of no
effect as well! [Of course the Einstein Bianchi identities always work, since
they merely express local coordinate invariance.]

In [1], we also considered the cosmological case, where identical arguments
apply to prove that the background must be Ricci-constant. The purpose of this
note is to provide physical insight into the derivation of [1], by emphasizing
how the “mass” of the linear $s=2$ excitations is required to connect to the
cosmological constant for consistency.

##  2 Cosmological excitations

The basic point is that Cosmological General Relativity (CGR) involves a new,
massive, parameter $\Lambda$, so consistent excitations on it must do so as
well, simply by now being (apparently) massive. This lesson was learned in
SUGRA, where the $s=3/2$ excitation had to become massive to allow the more
general SUGRA with cosmological term [3], the upshot being that $s=3/2$ was
actually apparently massive as a result of being in a tuned AdS space as
explained in [4].

We thus let the linear $s=2$ wave have an apparent $m^{2}$ of dimension
$\Lambda$. In the notations and implicitly assumed $(+++-)$ signature of [1],
this means adding to the Pauli-Fierz action

$I_{2}=\int d^{4}xh^{\mu\nu}\theta_{\mu\nu\alpha\beta}h^{\alpha\beta}$ (1)

(where $\theta$ is the second-order Hermitian operator yielding the gauge-
invariant spin $2$ field equation $\Box h_{\mu\nu}+\cdots=0$) the term

$I_{C}=-\frac{m^{2}}{4}\int
d^{4}x\left(h_{\mu\nu}h^{\mu\nu}-\frac{1}{2}h^{2}\right)(-g)^{-\frac{1}{2}}$
(2)

where proper background covariance is manifest, but where we do not assume a
priori any relationship between $m^{2}$ and $\Lambda$ (contrary to what was
done in [1]). Note that the field $h_{\mu\nu}$ carries density weight one,
hence the factors $(-g)^{-\frac{1}{2}}$ in the action [5].

However, enforcing gauge invariance of the spin $2$ field under
$(-g)^{-\frac{1}{2}}\delta
h_{\mu\nu}=D_{\mu}\xi_{\nu}+D_{\nu}\xi_{\mu}-g_{\mu\nu}D^{\alpha}\xi_{\alpha}$
yields, by a trivial extension of the calculation leading to Eq. (3) of [1]
that the background must be Ricci-constant with $\Lambda=-m^{2}$. Indeed, the
variation of the cosmological piece $I_{C}$ of the action adds to the
variation of $I_{2}[h]$

$\delta I_{2}[h]=\int
d^{4}x\xi^{\mu}\left[R_{\mu\sigma}D_{\nu}h^{\sigma\nu}+\frac{1}{2}(D_{\alpha}R_{\mu\beta}+D_{\beta}R_{\mu\beta}-D_{\mu}R_{\alpha\beta})h^{\alpha\beta}\right](-g)^{-\frac{1}{2}}$
(3)

the term

$\delta I_{C}[h]=m^{2}\int
d^{4}x\xi_{\mu}D_{\nu}h^{\mu\nu}(-g)^{-\frac{1}{2}}$ (4)

imposing through $\delta I_{2}[h]+\delta I_{C}[h]=0$ the background equation

$R_{\mu\nu}=-m^{2}g_{\mu\nu}$ (5)

equivalent to

$G_{\mu\nu}+\Lambda g_{\mu\nu}=0$ (6)

with

$\Lambda=-m^{2}$ (7)

(from which $D_{\alpha}R_{\mu\nu}=0$ follows automatically).

We thus see that the background must be indeed Ricci-constant with
$\Lambda=-m^{2}$, so that the total system is equivalent to a pure CGR one
with $g_{\mu\nu}+h_{\mu\nu}$, to quadratic $h$ order. Anti-de Sitter space
corresponds to $m^{2}>0$, i.e., real mass, while de Sitter space corresponds
to imaginary mass.

Of course, as shown in [4], the excitation’s “mass” effectively vanishes as a
result of gauge invariance. For $s=2$ alone, either sign of $m^{2}$ is
allowed, because while dS GR is also effectively massless, it is finite,
involving a geometrical horizon. Real $m$ requires AdS just as in the $s=3/2$
case where $m\sim\sqrt{-\Lambda}$ corresponds to $\Lambda<0$, i.e., anti-
deSitter (AdS) gravity in our conventions. Hence the uniqueness of AdS, vs
absence of dS-SUGRA, corresponding to SO(3,2 ) vs SO(4,1) representations in
group language, an alternate explanation. We stress, however, that the
derivation of (7) given here does not require supersymmetry.

##  3 Conclusions

In summary then, we first rederived our original claim in [1] that massless
s=2 excitations on Einstein-flat/constant spaces in GR are consistent if and
only if they are perturbations of the latter, $g_{\mu\nu}\rightarrow
g_{\mu\nu}+h_{\mu\nu}$ by a physical argument treating those excitations
purely dynamically, without assuming any a priori relationship between the
cosmological constant and the mass of the spin $2$ excitation. More centrally,
we explained in a dynamical sense, how CGR and SUGRA arise from the
corresponding (apparently massive) linear $s=2$ and $3/2$ excitations through
the derived relation $\Lambda=-m^{2}$ in the same way as did their massless
counterparts, that is, this condition is not merely sufficient but necessary.

## Acknowledgments

The work of SD was supported by the U.S.Department of Energy, Office of
Science, Office of High Energy Physics under award number de-sc0011632. The
work of MH was partially supported by FNRS-Belgium (conventions FRFC
PDRT.1025.14 and IISN 4.4503.15) and by funds from the Solvay Family.

## References

  * [1] S. Deser and M. Henneaux, “A Note on spin two fields in curved backgrounds,” Class. Quant. Grav. 24 (2007), 1683-1686 doi:10.1088/0264-9381/24/6/N01 [arXiv:gr-qc/0611157 [gr-qc]].
  * [2] C. Aragone and S. Deser, “Consistency Problems of Hypergravity,” Phys. Lett. B 86 (1979), 161-163 doi:10.1016/0370-2693(79)90808-6
  * [3] P. K. Townsend, “Cosmological Constant in Supergravity,” Phys. Rev. D 15 (1977), 2802-2804 doi:10.1103/PhysRevD.15.2802
  * [4] S. Deser and B. Zumino, “Broken Supersymmetry and Supergravity,” Phys. Rev. Lett. 38 (1977), 1433-1436 doi:10.1103/PhysRevLett.38.1433
  * [5] S. Deser, “Selfinteraction and gauge invariance,” Gen. Rel. Grav. 1 (1970), 9-18 doi:10.1007/BF00759198 [arXiv:gr-qc/0411023 [gr-qc]].

##  4 Comment added

Maxwell as the geodesic to derive GR

S. Deser

Abstract

I prove on the basis of a few qualitative observed properties of Maxwell-
gravity interaction that gravity is necessarily GR. Furthermore, because of
universality of GR, all matter that interacts with it must therefore also be
included.

###  4.1 Introduction

The observational ingredients are well-known: The gravitational field bends
light, hence not a scalar. It is macroscopic, hence not spin 1/2 or 3/2.
Attractive, so not s=1, leaving spin 2 (all higher spin fields are
inconsistent). The theoretical ingredient is a new proof [1] that such a field
can only propagate on a Ricci-/Einstein- flat, or constant if the excitation
is (apparently) massive, background. The range must be infinite because any
finite amount makes the spin 2 acausal [2]. Let us recapitulate the history:
The initial attempt to derive GR was Kraichnan’s [3], but we recently noted
that it was unsatisfactory [4]. Gupta’s later attempt was pure handwaving;
Feynman’s also incomplete - he did not sum the infinite series, but assumed
covariance which is equivalent to guaranteeing GR _ab initio_. Weinberg [5]
did derive the equations in the sense of the interaction representation, but
then the nonlinearity is hidden in the interaction term and the equations are
not explicitly visible. The first correct explicit derivation was a one-line
one [6] relying on the first order (Palatini) form of GR as a cubic theory,
followed by a derivation in an arbitrary background [7] and finally one based
on tree-level QM [8].

###  4.2 Proof

We begin with the source of the spin-2 field, namely the conserved matter
stress tensor, strictly speaking that of all matter in the universe, just as
Maxwell requires ALL charged currents in order to have conserved sources (even
though Maxwell is conformal but the rest of the matter is not. What counts
here is conservation, not trace). Now formally covariantize the stress tensor
- this is always possible, and has NO physical content: it just introduces
generalized coordinates. Its conservation is now replaced by covariant
conservation, still for free. But then the linear spin 2 field’s equation on
the left side must also be covariantized for consistency. At this point we
merely invoke the result of [1]: a spin 2 linear field must necessarily evolve
on a Ricci-/Einstein- flat background, of which it is - importantly - a
perturbation. (It could be Riemann-flat as well, but in that case would not be
“dynamical enough”, i.e., would have to be a source-free linear field, which
is where we began, pre-coupling!) So the coupled system can no longer be just
covariant in flat space with a fictitious metric, but must live in a Riemann
one, subject to the Einstein equations: the metric is now dynamical. Had we
allowed the excitation a “mass” term, the result for the left side would
instead be a perturbation of cosmological GR with constant proportional to
$m^{2}$ [1]. So the only consistent alternative to GR is source-free linear
$s=2$ in flat space, dull and unobservable, and no gravity.

###  4.3 Acknowledgements

I thank Marc Henneaux, my collaborator on [1], of which this note is an
application. This work was supported by the U.S. Department of Energy, Office
of Science, Office of High Energy Physics, Award Number de-sc0011632.

## References

  * [1] S. Deser and M. Henneaux.  ”Higher spin $m\not=0$ excitations on curved backgrounds and cosmological supergravity.” arXiv 2112.09870, Class Quant Grav doi./10.1088/1361-6382/ac6057.
  * [2] S. Deser, and A. Waldron.  ”Acausality of massive gravity.” Physical Review Letters 110.11 (2013): 111101.
  * [3] R. Kraichnan.  ”Special-Relativistic Derivation of Generally Covariant Gravitation Theory.” Phys Rev. 98 1118 (1955).
  * [4] S. Deser.  ”Higher curvature gravities, unlike GR, cannot be bootstrapped from their (usual) linearizations.” Gen. Rel. Grav. 49 149(2017), arXiv 1705.08938.
  * [5] S. Weinberg.  ”Photons and Gravitons in Perturbation Theory: Derivation of Maxwell’s and Einstein’s Equations.” Phys. Rev. 138 B988 (1965).
  * [6] S. Deser.  ”Self-interaction and Gauge Invariance.” Gen Rel Grav 1 9(1970), arXiv 0411023; ibid 42 631(2010), arXiv 0010.2975.
  * [7] S. Deser.  ”Gravity from self-interaction in a curved background.” Class Quant Grav 4 L99(1987).
  * [8] D.G. Boulware and S. Deser.  ”Classical general relativity derived from quantum gravity.” Ann Phys 89 193(1975).


# A Relational Macrostate Theory Guides Artificial Intelligence to Learn Macro
and Design Micro

Yanbo Zhang  School of Earth and Space Exploration, Arizona State University,
Tempe AZ USA  Beyond Center for Fundamental Concepts in Science, Arizona State
University, Tempe AZ USA  Sara Imari Walker  School of Earth and Space
Exploration, Arizona State University, Tempe AZ USA  Beyond Center for
Fundamental Concepts in Science, Arizona State University, Tempe AZ USA

###### Abstract

The high-dimesionality, non-linearity and emergent properties of complex
systems pose a challenge to identifying general laws in the same manner that
has been so successful in simpler physical systems. In Anderson’s seminal work
on why “more is different” he pointed to how emergent, macroscale patterns
break symmetries of the underlying microscale laws. Yet, less recognized is
that these large-scale, emergent patterns must also retain some symmetries of
the microscale rules. Here we introduce a new, relational macrostate theory
(RMT) that defines macrostates in terms of symmetries between two mutually
predictive observations, and develop a machine learning architecture,
MacroNet, that identifies macrostates. Using this framework, we show how
macrostates can be identifed across systems ranging in complexity from the
simplicity of the simple harmonic oscillator to the much more complex spatial
patterning characteristic of Turing instabilities. Furthermore, we show how
our framework can be used for the inverse design of microstates consistent
with a given macroscopic property – in Turing patterns this allows us to
design underlying rule with a given specification of spatial patterning, and
to identify which rule parameters most control these patterns. By
demonstrating a general theory for how macroscopic properties emerge from
conservation of symmetries in the mapping between observations, we provide a
machine learning framework that allows a unified approach to identifying
macrostates in systems from the simple to complex, and allows the design of
new examples consistent with a given macroscopic property.

_K_ eywords Emergence $\cdot$ macrostates $\cdot$ physics $\cdot$ complex
systems $\cdot$ machine learning

##  1 Introduction

Among the most important concepts in physics is that of symmetry, and how
symmetry-breaking at the microscale can give rise to macroscale behaviors.
This deep connection was made clearest in the work of Noether [1] , where she
showed that for differentiable systems with conservative forces, every
symmetry comes with a corresponding conversation law that describes macroscale
behavior. An example is how time translation symmetry gives rise to the
conservation of energy: simple harmonic oscillators conserve energy because,
in the absence of friction, you will observe the same oscillation if starting
a clock at the first cycle as at the thousandth – the behavior is time
invariant. Thus, Noether’s theorem provided a means to relate laws – namely,
regularities that are conserved (e.g., energy conservation) – to symmetries in
the underlying physical system (e.g., time). Physics has been incredibly
successful at discovering laws in this manner. However, so far, finding
similar ‘law-like’ behaviors for complex systems, such as biological and
technological ones, has proved much more challenging because of their high-
dimensionality, non-linear behavior, and emergent properties. Yet, the very
concept of emergence provides a clue that such regularities should exist, even
for complex systems. In Anderson’s seminal work on why “more is different”
[2], he pointed to how symmetry-breaking also plays a prominent role in
emergence: macroscale behaviors do not necessarily share all the same
symmetries as the microscale laws or rules that give rise to them. While some
of the symmetries are clearly lost, this also leaves open the possibility
large-scale patterns that emerge will still retain other symmetries of the
microscale rules. In addition to the rule-behavior mapping, there are other
mappings unique to complex systems such as genotype-phenotype maps, text-image
maps, etc., where symmetries may lead to conserved properties. The challenge
to identifying general laws for complex systems then reduces to identifying
which symmetries are preserved during the mapping – in general this is
challenging because of their high dimensionality, suggesting that machine
learning might be an approach that can aid in identifying conservation laws in
these systems, if we can identify macrostates and the symmetries they retain
from the microscale.

There have been several efforts focused on identifying macrostates associated
with the emergent regularities found in complex systems [3, 4, 5]. Notably,
Shalizi and Moore proposed causal state theory [6], which defines macrostates
based on the _relations between microstates_. Here, two microstates are
equivalent (belong to the same macrostate) if the future microstates
distributions are the same (Figure 1A). Thus, the conserved symmetry is one
pertaining to the prediction of future states. This, however, can exclude some
well-defined macrostates in physics. For example, given a simple harmonic
oscillator, two distinct microstates $u_{1}=(p_{1},x_{1})$ and
$u_{2}=(p_{2},x_{2})$, where $p_{1}$ and $p_{2}$ are two observations of
momentum and $x_{1}$ and $x_{2}$ there corresponding position, can have the
same energy macrostate. However, their future microstate distributions will be
different if $u_{1}$ and $u_{2}$ are not close to each other, say if,
$u_{1}=-u_{2}$. The conserved symmetry of Shalizi and Moore is therefore
violated because this system does not retain predictability of future
microstate distributions at the macroscale (because the macrostate of energy
is related to the time translation symmetry, not the symmetry associated to
predictability of future states).

Figure 1: Comparison between causal state theory, causal emergence theory and
relational macrostate theories (A) In causal state theory, two microstates are
equivalent (belong to the same macrostate) if their future microstate
distributions are the same. (B) Causal emergence theory identifies macrostates
where the past-future mappings are deterministic and non-degenerate at the
macro scale, such that the macrostates can be distinguished from one another
in the past (non-degeneracy) and the future (determinacy). Both causal state
theory and causal emergence theory define macrostates in terms of temporal
relations within a system of interest, as denoted by the square shape
underlying the mapping. (C) In the relational macrostate theory we propose
here, two microstates are equivalent if they relate to the same microstate
distributions, which can be generalized to any type of relation, including
past-future, rule-pattern, genotype-phenotype, etc. – the square and disk
shapes denote that this is sufficiently general to apply to maps that exist
across different spaces.

If a proposed theory to define macrostates is not sufficiently general to
include simple physical examples like the harmonic oscillator, it is unlikely
to apply universally to complex systems. Indeed, Shalizi and Moore were not
looking for a general theory of macrostates, but instead focused on the
specific property of predictability of complex systems. Another approach was
more recently proposed in causal emergence theory [7, 8, 9], which likewise
has a specific goal in mind – to describe causal relations at the macroscale.
Here, instead of using the properties of microstates, macrostates are defined
based on the _relations between macrostates_ by maximizing effective
information at the macroscale. Effective information is the mutual information
between two variables, under intervention to set one of them to maximum
entropy (e.g., a uniform distribution over macrostates). Causal emergence
occurs when the past and future of different macrostates are distinguishable
(Figure 1B). Thus, the symmetry of distinguishing past and future leads to a
conservation of distinguishability of macrostates.

Both causal state theory and causal emergence theory define macrostates in
terms of _temporal_ relations between past and future. However, not all
regularities we might want to associate to laws involve time. For instance, to
get the macrostates of mass, force, and acceleration, physicists of past
generations needed to study the relations between two objects rather than
between points in time (past and future). This suggests that to develop a
general theory of macrostates, these must be defined based on general
relations between two observations (Figure 1C) that retain some of the
symmetries of the underlying sets of observations – those observations could
be objects, or points in time, as physics has already treated. But they can
also be any other observation we can make with a measuring device, including
more “complex” examples like genotype-phenotype maps, or word co-occurrence in
language, or rule-behavior mapping necessary to describe patterning in
biological form. As we will show, the theory of macrostates we propose is
sufficiently general to extend to the symmetry of the microscale level rules
(or laws), which allow us to identify sets of microscale rules that yield a
given emergent, macroscale behavior [2].

When studying the history of the laws of physics, it is important to identify
why the most successful laws have worked so well. Newton’s laws of motion work
because there is a macroscale property called mass, which quantifies the
amount of matter in each object, that reduces the description of the motion of
high dimensional objects to a single measurable scalar quantity (mass) and its
translation in $x,y,z$ coordinates. For complex systems it is not so obvious
what the necessary dimensionality reduction will be that allows identifying
law-like behavior, and it may vary from system to system. Of note, Newton’s
laws cannot be developed in a world where mass can only be defined and
measured in a few countable objects and is undefined or unmeasurable in
others. In the current work, we show how artificial neural networks,
themselves a complex system, can break the barrier of complexity [10, 11, 12]
to identify macrostates based on symmetries in complex systems. Existing
machine learning methods such as contrastive learning [13], contrastive
predictive coding [14], and word2vec [15] have applied similar ideas to find
lower dimensional representations for microstates by relations. However, these
contrastive methods either require large numbers of negative samples that
increases the cost of training, or only learning embeddings instead of
functional mappings. Moreover, these methods are only useful for downstream
tasks, which use the embedding trained by contrastive learning. Although we
describe things at the macroscale, the world still runs on microscale
features. This means we not only need to map microstates to macrostates, but
that we need to provide an inverse path that samples microstate from a given
macrostate. By developing the macrostate theory on general relations, and
introducing invertibility, we propose a machine learning architecture,
MacroNet, that can learn macrostates and design microstates.

In fact, a key feature of learning is demonstrating use cases of the knowledge
learned. Therefore, to demonstrate how MacroNet is indeed learning the
macrostates across of examples of simple physical systems and complex systems,
we also use it to design new examples. There has been a flurry of recent work
by scientists attempting to engineer AI scientists, and in particular AI
physicists, that can learn the laws of nature from data with minimal
supervision. Examples include: AI Feynman, which learns symbolic expressions
[16]; AI Poincare that can learn conservation laws [17]; and Sir Isaac, an
inference algorithm that can learn dynamical laws [18]. Yet, science as done
by scientists goes further than solely extracting laws from data – humans also
implement that understanding in the real world. For example, in the case of
Newton’s laws of motion, our knowledge of them has allowed us to engineer a
range of systems, such as the design of airbags, racecars, airplanes,
helicopters and even optimization of athlete performance, etc. Thus, we view
the next advancement beyond artificial intelligence that can learn the rules
by which data behave is AI that can also use that knowledge to design new
examples of systems that will behave by the rules identified. A critical
aspect of designing new examples of systems is identifying macrostate
variables that reduce high-dimensional data to a few variables that capture
the salient features. The invertibility of MacroNet not only allows the design
of microstates sampled from an identified macrostate, but also provides a low-
cost way to replace negative samplings in contrastive learning.

In what follows, we first introduce our mathematical framework for defining
macrostates in terms of relations defined by symmetries in the data. Then, we
propose a machine learning framework to find macrostates under the definition.
For experiments, we first demonstrate the workflow of the framework by
implementing it for linear dynamical systems. They are simple enough to
demonstrate key concepts, but also exhibit rich behaviors. Then, we introduce
the simple harmonic oscillator as a special case where macrostates are defined
based on temporal relations, which demonstrates how our framework can extract
familiar invariant macrostates (conserved properties associated to symmetries)
from physics, such as energy. Finally, we turn to a real complex system, the
macroscale Turing patterns that arise in diffusion reaction systems. We show
how machine learning finds the macrostates associated with the emergent
patterning in these systems, and then how this can be used to design
microstates consistent with a target macroscale pattern.

##  2 Theory and Method

###  2.1 The relational macrostate theory that generalizable to the study of
complex systems using neural networks

Figure 2: Macrostates are defined by symmetries that define relations between
ensembles of microstates. The rectangle and disks represent the space of
microstates. And the points and links represent the observed microstate pairs
$(u_{i},v_{i})$. The color illustrates the macrostates that represent the
underlying symmetries in the relations between microstates. (A) shows an
optimal solution. (B) shows a trivial but legal solution, which coarse grains
all microstates to the same macrostate. This kind of coarse graining is not
informative since the mutual information of macrostates is zero. (C) shows an
inconsistent solution.

By definition, a macrostate is an ensemble corresponding to an equivalence
class of microstates. Given a mapping $\varphi_{u}$ that maps microstates to
macrostates, two microstates $u$ and $u^{\prime}$ belong to the same
equivalence class if $\varphi_{u}(u)=\varphi_{u}(u^{\prime})$, that is if the
microstates have the same behavior (macrostate) under the operation of the
map. In this way, macrostates are also the parameters to describe
distributions of microstates. This feature is a key reason why machine
learning may be an optimal way to identify macrostates, particularly in cases
of many-to-many mappings such as those that occur in rule-behavior maps, or
under prediction with noise, both of which are characteristic of complex
systems.

Here, we implement a formalism based on using relations arising due to
symmetries to define macrostates. Consider two microstates $u\in U$ and $v\in
V$ as two random variables. Their _micro-to-micro relation_ can be
mathematically represented as a joint distribution $P(u,v)$. The $u$ and $v$
can be mapped to macrostates $\alpha$ and $\beta$ respectively by
$\varphi_{u}$ and $\varphi_{v}$. So, we can also define _micro-to-macro
relation_ by the joint distribution $P(\alpha,v)$ and $P(u,\beta)$. For a
given microstate $u_{i}$ (or $v_{i}$), its micro-to-macro relation can be
represented as a conditional distribution $\Pr(\beta|u_{i})$ (or
$\Pr(\alpha|v_{i})$). Then, we can define macrostates in the most (relational)
general case as:

######  Definition 1.

Two pairs of microstates $u_{i}$ and $u_{j}$ (and $v_{i}$ and $v_{j}$) belong
to the same macrostate if and only if they have the same _micro-to-macro
relation_ :

$\displaystyle u_{i}\sim u_{j}$
$\displaystyle\iff\Pr(\beta|u_{i})=\Pr(\beta|u_{j})\text{ and}$ (1)
$\displaystyle v_{i}\sim v_{j}$
$\displaystyle\iff\Pr(\alpha|v_{i})=\Pr(\alpha|v_{j})$ (2)

Note, this defines an equivalence class of symmetries where $u_{i}\sim u_{j}$
and $v_{i}\sim v_{j}$ (where $\sim$ indicates “is equivalent to” under the
symmetry operation). Thus, as in Noether’s theorem (and in Anderson’s
formalization of emergence) we see that the definition of a macrostate entails
simultaneously defining a class of symmetry operations, although here our
definition is sufficiently general that the system of interest need not
necessarily be continuously differentiable (as in the case of Noether’s
theorem).

The definition can be approached by solving $\varphi_{u}(u)=\varphi_{v}(v)$
(see SI). This equation will be part of the loss function in the specified
machine learning task of MacroNet. Since the macrostate of $U$ is defined by
the macrostate of $V$, and vice versa, the solutions are not computed in a
straightforward way, but must be calculated in relation to one another. As
such, there can exist some inconsistent solutions. Figure 2A shows a
consistent solution, however, Figure 2B shows an inconsistent solution. The
points in red circles are classified into two macrostates, while they both
have the same micro-to-macro relation. Not all consistent solutions are
useful. If all microstates are mapped to the same macrostate, it still follows
the definition, but this is a trivial solution and not informative, see Figure
2C. In addition to definition 1, we therefore require an information criterion
to specify “good macrostates”. We do so by specifying a given dimension of
macrostates, and then maximizing the mutual information
$I\left(\varphi(v);\varphi(u)\right)$ at the macroscopic level, where $(u,v)$
is sampled from $P(u,v)$. As a comparison, the effective information (EI) in
causal emergence theory also uses the mutual information concept, but it has
notable differences in how it is implemented beyond the fact that the theory
presented here was designed for a machine learning implementation and causal
emergence was not. In a discrete macrostate space, to quantify the causal
effect, the EI re-assigns the marginal distribution of the macrostates with a
uniform distribution. We do not make this requirement since we are not
focusing on causal relations. Moreover, in a continuous macrostate space, the
causal relation between macrostates may not make much sense because of the
large number of different macrostates.

###  2.2 A self-supervised generative model for finding macrostates from
observations

Figure 3: Neural network architecture of MacroNet. (A) During the training
process, the two invertible neural networks are optimized to map two types of
microstates to the same macrostates. These microstates can correspond to past
and future states of the same dynamical system, dynamical rules or parameters
and observed behavior, or any other pair of (sets of) variables (B) The
conditional sampling and designing process. First, we can manually make an
example microstate of type $V$. We compute the macrostate. We can sample the
microstates in $U$ or $V$ that have this macrostate. (C) When doing coarse-
graining, parts of the output are abandoned to reduce dimensionality. The
abandoned variables are still trained to follow an independent standard normal
distribution. This independence makes it easy to do conditional sampling
because we can sample the abandoned variables easily. (D) A typical invertible
neural network is RealNVP, which has a specially designed structure that
guarantees invertibility. The log-determinate of the Jacobian is also easy to
compute for this type of neural network.

In the above formalization, a macrostate in $U$ is defined by macrostates in
$V$ (i.e., macrostates are defined only in terms of their relations to other
macrostates). This relational definition necessitates that we optimize the
macrostate mapping iteratively to find an optimal solution. Thus, to implement
the relational macrostates theory, we propose a self-supervised generative
model for finding macrostates from observations (Figure 3A).

Our definition of macrostates can be achieved by optimizing macrostates to
predict other macrostates. Here we use $\varphi_{u}$ and $\varphi_{v}$ to
represent the coarse graining performed by the neural networks on $U$ and $V$
respectively. We have the prediction loss:

$\mathcal{L}_{P}=\mathbb{E}_{(u,v)\sim
P(u,v)}\left|\varphi_{u}(u)-\varphi_{v}(v)\right|^{2},$ (3)

where $(u,v)$ are pairs of microstates sampled from the training data. The
ideal solution for $\varphi$ is
$\varphi_{u}(u)\approx_{\sigma}\varphi_{v}(v)$, meaning the macrostate of $u$
can be predicted by the macrostate of $v$ with error of $\sigma$, and vice
versa. However, we need an additional term to avoid trivial solutions such as
a low dimensional manifold or constant. To do this, we add a distribution
loss, $\mathcal{L}_{D}=\mathcal{L}_{D_{u}}+\mathcal{L}_{D_{v}}$, where:

$\displaystyle\mathcal{L}_{D_{u}}$
$\displaystyle=\log{P_{\text{normal}}\left(\varphi_{u}(u)\right)}-\log\left|{\det\frac{\partial\varphi_{u}(u)}{\partial
u}}\right|,$ (4) $\displaystyle\mathcal{L}_{D_{v}}$
$\displaystyle=\log{P_{\text{normal}}\left(\varphi_{v}(v)\right)}-\log\left|{\det\frac{\partial\varphi_{v}(v)}{\partial
v}}\right|.$ (5)

The distribution loss is be minimized when the outputs follow independent
normal distributions. We train the neural networks by combining the two loss
functions:

$\mathcal{L}=\mathcal{L}_{P}+\gamma\mathcal{L}_{D},$ (6)

where $\gamma$ is the hyperparameter balancing the two loss terms. Combining
these two terms, we can approach the mutual information criterion. Directly
computing $\mathcal{L}_{D}$ can be very expensive since it requires computing
the Jacobian. However, since we want to do sampling, invertible neural
networks (INNs)111We developed an invertible neural network python package –
INNLab, available on GitHub: https://github.com/ELIFE-ASU/INNLab can help. The
INNs are not only designed to be invertible, but also designed to easily
compute the log-determinate of the Jacobian. The INNs will have the same
output dimension as the input, so we abandon part of the dimensions (see SI
S.3.c). For example, if we want to map an 8-dimensional vector to two-
dimensional macrostate, the INNs will still give an 8-dimensional vector as a
result, but we only take the first two variables as the macrostate for
training. The abandoned six variables, however, still have been trained to
follow independent normal distributions so we can do conditional inverse
sampling.

Given an example microstate $v^{\prime}$, suppose we want to find other
microstates in $V$ space with the same macrostate as $v^{\prime}$. We can use
$\varphi_{v}(v^{\prime})$ to compute the macrostate $\beta$ of the example
$v^{\prime}$. Then, we can invert the neural network to sample microstates
$v_{s}$ that have the same macrostates. This conditional sampling allows
identifying the symmetry of macrostates and enables the design of microstates
by sampling from a given target macrostate once the network is trained on
other examples with the same macroscale behavior (Figure 3B). This kind of
sampling can enable the design of complex systems: the identified macrostates
are not given by humans, but instead, computed from examples by neural
networks. This process makes it possible to design complex systems without
needing to first classify their behavior.

##  3 Results

In what follows we consider three explicit examples of the application of
MacroNet. The first is a linear dynamical system, which allows us to
demonstrate the key features of our workflow with a system that allows easily
demonstrating key concepts, via the identification of a rotational symmetry
and design of microstates consistent with this behavior. The second example is
a simple harmonic oscillator (SHO), where we demonstrate MacroNet can identify
a familiar symmetry and its corresponding macrostate in physics – time
translation invariance and energy – by showing that our workflow can identify
equal energy surfaces for the SHO. The final example is Turing patterns, where
we show the utility of MacroNet in solving the inverse problem of mapping
macro-to-micro in a complex system.

###  3.1 Linear dynamical systems

Figure 4: Training neural networks to find macrostates of linear dynamical
systems. (A) the related pairs are parameters and trajectories. (B) by
choosing an example trajectory, we can sample microstates of parameters or
trajectories with the same macrostate. (C) Given the example trajectory (red
dots), we can compute its macrostate. Then, we can sample parameters that have
the sample macrostate. Using the sampled parameters, we can plot the
trajectories generated by the sampled parameter (blue lines). (D) Using the
same macrostate, we can sample an ensemble of trajectories that have the same
macrostate.

We start with an experiment analyzing a linear dynamical system because these
have many-to-many mappings. This allows us to demonstrate the workflow of
identifying macrostates based on symmetries and then designing microstates
from the identified macrostates. Here we choose a two-dimensional linear
dynamical system whose dynamics are given by

$\frac{\text{d}\vec{x}}{\text{d}t}=M\vec{x},$ (7)

where $x$ is the independent variable, and $M$ is a $2\times 2$ matrix that
includes the parameters that specify the dynamics of the system. Given a
matrix $M$ and an initial state $x_{0}$, we can generate a sequence of
observed states by computing $x_{t+1}=x_{t}+Mx_{t}\delta t$. The trajectory
will be $T=[x_{1},x_{2},\ldots,x_{n}]$ in the two-dimensional space, where
$n=8$ and $\delta t=1/n$. Here we choose $n=8$ because it is large enough to
show the pattern of trajectories and not too large to slow the training. In
this example, the micro-to-micro relations are represented by parameter-
trajectory pairs, i.e., $(u,v)=(M,T)$. Note, in contrast to more standard
approaches to studying dynamical systems, we are here not trying to find a
macrostate by coarse graining the trajectory of states (which would depend on
some variety of time symmetry, see introduction). Instead, we are coarse-
graining to a macrostate that provides a map from parameters to observed
trajectories that will enable us to automatically generate new parameter-
trajectory pairs that were not generated by running Eq 7.

We note the many-to-many mapping here means: 1) given one parameter, different
initial states will lead to different trajectories. 2) sampling different
parameters may lead to the same or similar trajectories. We use two neural
networks to learn the macroscale relation between parameters and trajectories:
one uses $\varphi_{u}$ to map the 4-d parameter matrix to a 2-d macrostate,
and the other uses $\varphi_{v}$ to map the 16-d trajectory to a 2-d
macrostate (Figure 4B), where we optimize to reduce the mutual information
between the identified macrostates in both cases (Figure 4A).

After training, we can use the learned macrostates to design microstates. In
Figure 4B, Given an example trajectory $T_{e}$, we can compute its macrostate
$\beta=\varphi_{v}(T_{e})$. The neural network $\varphi_{u}^{-1}$ samples
parameters that can generate trajectories for the example microstate (Figure
4C). The sampled parameters follow a conditional distribution $P(M|\beta)$,
where $M$ is the parameter matrix. In Figure 4C, we show how, given an anti-
clockwise rotating trajectory, the parameters sampled all lead to anti-
clockwise trajectories. By this process, we can design parameters of a system
to mimic the behavior of any example, even without needing to translate the
language describing the behavior to be human-interpretable. This ability has
broad applicability for the design and control of complex systems, where
simple mathematical descriptions have defied human scientists. Even when we do
not know or have access to how we could describe a behavior, the neural
network can still sample parameters to allow design of new examples through
self-supervised learning.

So far, we have demonstrated sampling parameters for the matrix $M$, based on
a specified macrostate (rotating anti-clockwise). We showed how the sampled
parameters allow constructing new example trajectories using the sampled
matrix $M$ in Eq 7 with the desired macroscale behavior. We can also sample
trajectories directly, via a sampling process where we specify the target
macrostate and then use the inverse sampling to recover trajectories. These
sampled trajectories follow the distribution of
$P\left(T\middle|\beta\right)$, where $T$ is the trajectory microstate. Figure
4D show that the sampled trajectories all follow the same behavior, exhibiting
anti-clockwise rotation, just as with the example trajectory. It is worth
noting that here we never had to implement Eq 7 to generate the designed
trajectories, but they were sampled directly from identification of the
macrostate. We also did not give the neural network any concept of “rotate” or
“clockwise”: the neural network discovered this symmetry on its own, as one
that is relevant to how the parameters of the matrix $M$ map to observed
trajectories. This experiment gives a simple example of how a neural network
architecture like MacroNet can aid in identifying genotype-phenotype maps,
where we genotypes play the role of parameters and phenotypes the role of
trajectories.

###  3.2 Simple harmonic oscillators

Although we define macrostates on identifying symmetries underlying general
relations, time relations are still of particular interest because of their
long history in physics and their relationship to energy. Here, we demonstrate
how MacroNet can automatically identify the symmetry of time translation
invariance associated to energy, using a simple harmonic oscillator (SHO) as a
case study. The Hamiltonian of SHOs is:

$\mathcal{H}=\frac{p^{2}}{2m}+\frac{1}{2}kx^{2}.$ (8)

In this experiment, we let $m=1$ and $k=1$ for all cases. The micro-to-micro
relation is a temporal relation, represented by pairs of
$\left(x_{0},p_{0}\right)$ and $\left(x_{\tau},p_{\tau}\right)$, where $x_{0}$
and $p_{0}$ are the initial position and momentum and $\tau$ is uniformly
sampled time interval $(0,2\pi)$ (see Figure 5A). Since we are trying to find
a time invariant quantity, the mapping function $\varphi_{u}$ and
$\varphi_{v}$ should not be different. So, we force the two neural networks
$\varphi_{u}$ and $\varphi_{v}$ to share the same weights.

Figure 5: With a simple harmonic oscillator, we train a neural network to find
invariant quantities as a special case of macrostates. (A) the $(u,v)$ pairs
are sampled from simulations, where $u=(x_{0},p_{0})$ (the black dots) and
$v=(x_{\tau},p_{\tau})$. The $\tau$ is sampled from a uniform distribution
$\mathcal{U}(0,2\pi)$. The white dots in the yellow region show a sampling
example of $v$. Due to the randomness of $\tau$, it is impossible for accurate
prediction at microstate. (B) the neural network learns energy as the
invariant quantity. The $x$-axis is the energy of microstates computed by the
physical theory of SHOs discovered by humans, and the $y$-axis is the
macrostate discovered by the neural network. They show a monotonical relation,
which implies the successful identification of energy by the neural network.
(C) conditional sampling microstates from $P((x,p)|\varphi(x,p)=\alpha_{i})$,
where the $\alpha_{i}$ are the given macrostates. The results approximate
equal energy surfaces, denoted by the dashed circles. Note that the noise in
the sampling is a side effect of the noisy kernel trick we use here (see
appendix). The background color also shows the learned macrostate mapping as a
field.

Figure 5 shows our training results. When we require the neural network to
learn a one-dimensional invariant as a macrostate, the macrostate is exactly a
function of energy (Figure 5B). Figure 5C shows samplings from macrostates to
microstates. The same color represents microstates sampled from the same
macrostate. The sampling shows how the neural network has identified three
concentric circles, which correspond to the equal energy surfaces of the SHOs
(Figure 5C), where the equation $p^{2}+x^{2}=H$ represents a circle with a
radius of $\sqrt{H}$. Note that the uncertainty of $\tau$ makes it impossible
to accurately predict the future microstates. In fact, the optimal prediction
at any microstate will be zero if we optimize the MSE loss. However, using
MacroNet, we can still predict the future macrostates and sample microstates
from them. This is an example of how predictions at the microscale can fail,
and how macrostates can help solving many-to-many mapping problems, such that
predictions are still possible.

###  3.3 Turing patterns

Finally, we applied the same method on a complex system: Turing patterns.
Here, we are using the Gray-Scott Model [19], a 2-d space that has two kinds
of components, $a$ and $b$, which might, for example, correspond to two
different kinds of chemical species. The $a$ and $b$ are two scalar fields
corresponding to concentration of the two species. Their dynamics can be
described by the differential equation:

$\begin{split}\frac{\partial a}{\partial t}&=D_{a}\nabla^{2}a-ab^{2}+F(1-a)\\\
\frac{\partial b}{\partial t}&=D_{b}\nabla^{2}b+ab^{2}-(F+k)b\end{split}$ (9)

where $D_{a}$, $D_{b}$, $F$ and $k$ are four positive constants \- these four
parameters determine the behavior of the system. This model can generate a set
of complex patterns, see Figure 6A. By finding macrostates mapping patterns to
parameters, we can then in turn design related systems by specifying
parameters that will yield user-specified patterns. Here, $u$ is the parameter
vector, $u=(D_{a},D_{b},F,k)$. And $v$ is the generated pattern, represented
by $64\times 64$ images, $v=\left(a^{(64\times 64)},b^{(64\times 64)}\right)$.

Figure 6: Experiments on Turing patterns. (A) By giving an example pattern, we
can compute its macrostate by $\varphi_{v}$. The patterns are colorized for
distinguishing different experiments. (B) Then, we can sample an ensemble of
corresponding parameters from the macrostate by $\varphi_{u}^{-1}$. The points
with the sample color are sampled from the same macrostate computed from the
corresponding example. The dots with black boundary are the parameters that
used to generates the example patterns. (C) Using the sampled parameters, we
can generate Turing patterns with sampled initial states. The generated
patterns show similar macroscopic shape as the corresponding examples.

We trained the neural network to map parameters and patterns to each other at
macroscale (such that these will share the same macrostate). Figure 6 shows
the sampling based on the specified patterns. By giving an example pattern $v$
(Figure 6A), we can sample parameters
$u^{\prime}=\varphi_{u}^{-1}\left(\varphi_{v}(v),z\right)$ with the same
macrostate as $v$ (Figure 6B). As Figure 6C shows, the sampled rules (set of
four parameters) will generate similar patterns as the example patterns. This
experiment shows that our method can design complex systems by sampling
parameters that will generate patterns exhibiting the same macrostate as the
example behavior. That is, MacroNet can solve the inverse problem of going
from pattern to parameters.

The microstate ensembles associated with macrostates can also be directly
discovered by this approach. Figure 6B shows the distribution of parameters
sampled from different macrostates. The sample points with the same color are
considered as equivalent to each other under the mapping $\varphi$, which
takes the microstate to a macrostate. Parameters in the same equivalence class
(sharing the same symmetry) will therefore lead to patterns that have the same
macrostates, so we can sample any parameters along these equivalence curves
and generate Turing patterns with the user-specified behavior.

And additional feature is that observing the sampled parameters can also tell
us the importance of different parameters for specifying a target macroscale
behavior. For example, as shown in Figure 6B, different macrostates have
similar sampling on $D_{a}$. However, on $(F,k)$, different macrostates sample
different parameters. This indicates that $F,k$ will have stronger effect on
differences in macroscale behavior than $D_{a}$. This has implications for
specifying control parameters in designing complex systems. An example of
interest is in pattern formation in regeneration [20], where a framework like
MacroNet could identify the patterns controlling specific features of shape.

##  4 Discussion

Since Anderson published the seminal paper, _More is Different_ , it has been
increasingly recognized that complex systems displaying emergent behaviors do
not necessarily share the same symmetries as their micro-rules [21]. That is,
we know the mapping from a micro-rule to a large-scale system does not
preserve all the symmetries of the micro-rule, due to symmetry breaking and
perturbations from the environment. In some sense, this is the very definition
of “emergence”. However, we might expect some symmetries to be retained such
that micro-rules share at least a subset of their symmetries with any
macroscale emergent behavior. Indeed, this is what we see in the experiments
presented in this work. Each macrovariable can represent a type of symmetry:
for instance, the energy of a simple harmonic oscillator represents how all
states with the same energy are symmetric in time to others with that energy.
In a more complex case, the macrostates of Turing patterns contain the
information that is invariant under the mapping from parameter to pattern,
even under external perturbations. The parameters that having the same
macrostate are symmetric to each other because they all generate the patterns
with the same macrostate. By finding the macrostates via the mutual
information shared between ensembles of microstates, we can align the
symmetries shared by the two sets of microvariables. This is a general
framework for identifying macrostates as maps conserving the symmetries of
systems: hence, while given “more is different” is true in most cases, we can
still find examples of macrovariables that behave as “more is same” because
they will retain underlying symmetries present at the microscale.

The process of finding macrostates can be considered as a prediction problem:
that is, it is one of finding predictable variables of two related
observations. There are no such variables if two observations have zero mutual
information. Thus, if two observations have none-zero mutual information, we
can use macrovariables (ensembles of microstates) to connect the two
observations. In this way, one can consider macrostates as the instantiated
mutual information mapping observations of one system to another (or a system
to itself at a different point in time).

Across our experiments, we showed how macrostates can emerge from identifying
predictive relations between two sets of observations. The parameter-
trajectory relation leads to the macrostate of rotation and direction. The
temporal relation between past and future leads to the macrostate energy in
the simple harmonic oscillator. In the more complex case of Turing patterns,
macrostates arise from parameter-pattern relationships. Thus, by adopting this
relationalism idea, we can establish an approach targeting an ambitious
question in the complex systems field: is it possible find general laws of
complex systems? To address this question, one key task is to find a set of
universal macrostates that can be found in most complex systems. And hence the
laws of the universal macrostates can be considered as the general laws of
complex systems. The method proposed in this work makes an initial step for
this target – by finding macrostates from relations, the macrostates can be
used on both sides of the relations (although they may be interpreted
differently on either side of the relation). For instance, in the Turing
pattern case, the macrostates are not only the macrostates of patterns, but
also the macrostates of parameters. For future work, to find more universal
macrostates, the framework may be extended from second-order relationship to
higher-order relationships. Applying this method more generally to complex
systems may reveal there are indeed universal general laws, or it may reveal
that no map can apply to all systems – that is, that the laws of complex
systems are unique to specific classes of system. In either case, the
framework we have presented here, which offers an automated means for
identifying general laws via symmetries in complex systems, offers new
opportunities for asking and answering such questions.

## Acknowledgement

We would like to acknowledge Dr. Cole Mathis, Dr. Daniel Czegel, Dr. Douglas
G. Moore and Dr. Enrico Borriello from the Walker lab at Arizona State
University for suggestions and discussions. We thank Prof. Yi-Zhuang You from
UCSD for his valuable suggestions on machine learning, Prof. Yunbo Lu from
Tongji University for his support on computational resources, and Prof. Leroy
Cronin from University of Glasgow for research support. This project was
supported by a grant from the John Templeton Foundation.

## Author Contributions

Y.Z. and S.I.W. developed the theory and designed the formalism. Y.Z.
implemented the MacroNet machine learning architecture and performed the
analyses. Y.Z. and S.I.W. wrote the manuscript.

## References

  * [1] Emmy Noether.  Invariante variations probleme, nachr. ges. wiss.  Gottingen Math.-Phys. Kl., pages 235–257, 1918.
  * [2] Philip W Anderson.  More is different: broken symmetry and the nature of the hierarchical structure of science.  Science, 177(4047):393–396, 1972.
  * [3] Boyuan Chen, Kuang Huang, Sunand Raghupathi, Ishaan Chandratreya, Qiang Du, and Hod Lipson.  Discovering state variables hidden in experimental data.  arXiv preprint arXiv:2112.10755, 2021.
  * [4] Thomas Kipf, Elise Van der Pol, and Max Welling.  Contrastive learning of structured world models.  arXiv preprint arXiv:1911.12247, 2019.
  * [5] Yu Sun, Gaoke Hu, Yongwen Zhang, Bo Lu, Zhenghui Lu, Jingfang Fan, Xiaoteng Li, Qimin Deng, and Xiaosong Chen.  Eigen microstates and their evolutions in complex systems.  Communications in Theoretical Physics, 73(6):065603, 2021.
  * [6] Cosma Rohilla Shalizi and Cristopher Moore.  What is a macrostate? subjective observations and objective dynamics.  arXiv preprint cond-mat/0303625, 2003.
  * [7] Erik P Hoel.  When the map is better than the territory.  Entropy, 19(5):188, 2017.
  * [8] Pavel Chvykov and Erik Hoel.  Causal geometry.  Entropy, 23(1):24, 2020.
  * [9] Renzo Comolatti and Erik Hoel.  Causal emergence is widespread across measures of causation.  arXiv preprint arXiv:2202.01854, 2022.
  * [10] John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al.  Highly accurate protein structure prediction with alphafold.  Nature, 596(7873):583–589, 2021.
  * [11] Jaideep Pathak, Brian Hunt, Michelle Girvan, Zhixin Lu, and Edward Ott.  Model-free prediction of large spatiotemporally chaotic systems from data: A reservoir computing approach.  Physical review letters, 120(2):024102, 2018.
  * [12] Alireza Seif, Mohammad Hafezi, and Christopher Jarzynski.  Machine learning the thermodynamic arrow of time.  Nature Physics, 17(1):105–113, 2021.
  * [13] Xinlei Chen and Kaiming He.  Exploring simple siamese representation learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15750–15758, 2021.
  * [14] Aaron van den Oord, Yazhe Li, and Oriol Vinyals.  Representation learning with contrastive predictive coding.  arXiv preprint arXiv:1807.03748, 2018.
  * [15] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean.  Efficient estimation of word representations in vector space.  arXiv preprint arXiv:1301.3781, 2013.
  * [16] Silviu-Marian Udrescu and Max Tegmark.  Ai feynman: A physics-inspired method for symbolic regression.  Science Advances, 6(16):eaay2631, 2020.
  * [17] Ziming Liu and Max Tegmark.  Machine learning conservation laws from trajectories.  Physical Review Letters, 126(18):180604, 2021.
  * [18] Bryan C Daniels and Ilya Nemenman.  Automated adaptive inference of phenomenological dynamical models.  Nature communications, 6(1):1–8, 2015.
  * [19] Peter Gray and Stephen K Scott.  Autocatalytic reactions in the isothermal, continuous stirred tank reactor: Oscillations and instabilities in the system a+ 2b→ 3b; b→ c.  Chemical Engineering Science, 39(6):1087–1097, 1984.
  * [20] Michael Levin.  Endogenous bioelectrical networks store non-genetic patterning information during development and regeneration.  The Journal of physiology, 592(11):2295–2305, 2014.
  * [21] Steven Strogatz, Sara Walker, Julia M Yeomans, Corina Tarnita, Elsa Arcaute, Manlio De Domenico, Oriol Artime, and Kwang-Il Goh.  Fifty years of ‘more is different’.  Nature Reviews Physics, 4(8):508–510, 2022.