# Efficiently Evolving Swarm Behaviors Using Grammatical Evolution With PPA-
style Behavior Trees

Aadesh Neupane & Michael A. Goodrich
Department of Computer Science
Brigham Young University
Provo, UT 84602, USA
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Evolving swarm behaviors with artificial agents is computationally expensive
and challenging. Because reward structures are often sparse in swarm problems,
only a few simulations among hundreds evolve successful swarm behaviors.
Additionally, swarm evolutionary algorithms typically rely on ad hoc fitness
structures, and novel fitness functions need to be designed for each swarm
task. This paper evolves swarm behaviors by systematically combining
Postcondition-Precondition-Action (PPA) canonical Behavior Trees (BT) with a
Grammatical Evolution. The PPA structure replaces ad hoc reward structures
with systematic postcondition checks, which allows a common grammar to learn
solutions to different tasks using only environmental cues and BT feedback.
The static performance of learned behaviors is poor because no agent learns
all necessary subtasks, but performance while evolving is excellent because
agents can quickly change behaviors in new contexts. The evolving algorithm
succeeded in 75% of learning trials for both foraging and nest maintenance
tasks, an eight-fold improvement over prior work.

##  1 Introduction

Bio-inspired models have produced efficient algorithms in various domains
(Karaboga & Basturk, 2007; Dorigo et al., 2006). The potential benefits of
bio-inspired algorithms are limited by the cumbersome task of observing animal
behavior and creating mathematical models to describe both individual and
collective behaviors (Sumpter & Pratt, 2003; Gordon, 2010). A promising
alternative is to design novel swarm behaviors using evolutionary algorithms.

Evolving swarm behaviors in non-episodic setting requires group-level
objectives and rewards to be divided into individual objectives and rewards,
which is a form of the credit assignment problem (CAP) (Sutton, 1984).
Existing swarm evolution algorithms address CAP by designing ad hoc fitness
functions that give the artificial agents sufficient feedback to solve a
particular task (Ferrante et al., 2013; Neupane et al., 2018). Unfortunately,
designing an ad hoc fitness function for each task requires expert knowledge,
and is subject to human biases (Nelson et al., 2009). Also, evolving swarm
behaviors require the designer to choose and aggregate various controllers,
evolutionary algorithms, and fitness functions. In many cases, only a few
combinations are viable to evolve collective behaviors, and the success rate
is low among those viable combinations.

A multi-agent Grammatical Evolution (GE) (O’Neill & Ryan, 2001) algorithm
called GEESE evolved colony-level foraging behaviors, outperforming
conventional GE and hand-coded solutions on a foraging task (Neupane et al.,
2018). GEESE’s controllers were Finite State Machines (FSMs) and the group
fitness was the total food collected, but GEESE could only evolve foraging
behaviors. GEESE-BT (Neupane & Goodrich, 2019b) used a more expressive grammar
that could evolve both foraging and cooperative transport behaviors. GEESE-
BT’s controllers were behavior trees (BTs), and GEESE-BT used ad hoc fitness
functions from the theory of intrinsic motivators. Unfortunately, the ad hoc
reward structures were not sufficient to evolve successful behaviors with high
success rate and required a new fitness function for each new task.

This paper presents the BeTr -GEESE (“better geese”) algorithm, which produces
successful swarm behaviors while learning even though no single agent has all
necessary behaviors at any given point during the evolution. BeTr -GEESE
improves GEESE-BT in three significant ways: (i) primitive behaviors use
Postcondition-Precondition-Action (PPA) structures (Sprague et al., 2018);
(ii) the grammar used PPA-style BT programs; and (iii) the task-specific ad
hoc rewards were replaced with BT execution node status, enabling direct
feedback from PPA nodes. The evolving algorithm succeeds in 75% of learning
trials for two tasks (foraging and nest maintenance), an eight-fold
improvement over GEESE-BT.

Two limitations are noteworthy. First, swarm success requires agents to
performing ongoing evolution. This is problematic if agents experience
catastrophic failures while learning, such as when a physical robot crashes in
the world. Second, the algorithm is only applied to multiagent tasks that are
divisible and additive tasks, meaning that the swarm’s task can be broken into
subtasks achievable by individual programs (Steiner, 1972).

##  2 Related Work

Early work in evolving robot controllers include (a) staged evolution of a
complex motor pattern generator to control a walking robot (Lewis et al.,
1992) and (b) evolving neural-network-based control architectures for visually
guiding robots (Cli et al., 1993). Neural networks can directly map robot
inputs to outputs (Lewis, 1996; Trianni et al., 2003), but tend to be data-
hungry and not transparent (Fan et al., 2021). Kriesel et al. (2008) applied
evolution to swarm systems and demonstrated that simple evolved individuals
can produce effective swarm behaviors.

Evolutionary robotics algorithms differ from each other in the choice of the
evolutionary algorithm and the choice of the agent’s controller. Individual
robot controllers have been designed using Neural Networks (NN), Finite state
machines (FSM) and hierarchical FSMs (HFSM) (Petrovic, 2008; Pintér-Bartha et
al., 2012; König et al., 2009; Brooks, 1986; Valmari, 1996). Behavior Trees
(BT) often provide more readable, scalable, modular, and reactive structures
than HFSMs and NNs (Colledanchise & Ögren, 2017).

Duarte et al. (2016) evolved swarm behaviors for physical robots and
demonstrated scalability, flexibility, and resilience; results were shown for
swarm-level dispersion, homing, clustering, and monitoring. GE and state
machine-type controllers have been combined to evolve swarm behaviors (Neupane
et al., 2018; Neupane & Goodrich, 2019a; Ferrante et al., 2013). Even though
GE exploits prior knowledge in the form of grammars to learn better solutions
much faster than conventional genetic programs (O’Neill & Ryan, 2001), results
depend on the design of ad hoc reward structures.

The lateral genetic transfer of controllers in BeTr -GEESE is bio-inspired.
Prior work claims that complex behaviors rarely evolve solely from crossover
and mutation alone but require endosymbiosis or horizontal transfer (Jablonka
& Lamb, 2014; Lane, 2015; Quammen, 2018). With the horizontal transfer,
evolving single agents with sparse and delayed reward is more computationally
efficient than evolving complex controllers (Lee, 1999; Engebråten et al.,
2018).

PPA structures have been widely used (Fikes & Nilsson, 1971; Knoblock, 1995).
Sprague et al. (2018) used PPA structures to construct a modular, versatile,
and robust control architecture for mission-critical autonomous underwater
vehicles. The PPA structure ensured that the execution of BT followed goal
fulfillment priorities. Colledanchise et al. (2019) introduced a standard
backward chaining algorithm to create a PPA structure automatically, and
showed that the structure could skip actions that were already executed and
only plan when the postconditions were not satisfied. Ögren (2020) proved
convergence guarantees for a particular PPA-BT structure, and Parashar et al.
(2021) presented a PPA layered strategy to transform a mission into
decomposable tasks. BT-based collective behaviors have been used in non-
evolutionary-based optimization. Kucking et al. (2018) used BTs to perform
foraging and aggregation. Kuckling et al. (2021) extended that work with a set
of behavioral modules within a predefined BT structure. The algorithm
optimized swarm behaviors for foraging and marker aggregation tasks when
agents could communicate.

##  3 GEESE-BT Overview

GEESE (Neupane et al., 2018) and GEESE-BT (Neupane & Goodrich, 2019b) are
multi-agent grammatical evolution (GE) algorithms that are similar to standard
GE in terms of initialization, genetic operators, and genotype-to-phenotype
mapping. Each agent is initialized with a fixed-length array of binary strings
called the genome. The genome is then grouped into fixed consecutive blocks
called a codon. Each codon block can be converted into a codon value. The
conversion of the genome to a program with the help of BNF grammar is the
genotype-to-phenotype mapping. The max-tree-depth parameter controls the
number of unexpanded non-terminals in the current parse tree.

GEESE agents uses three evolution steps: sense, act, and update. During the
sense phase, agents exchange genome information with any (nearby) agents in
the field of view. The willingness to transfer their gene is controlled by the
INTERACTION_PROB parameter. During the act phase, an agent queries its storage
pool to determine whether the pool size exceeds STORAGE_THRESHOLD parameter.
Higher thresholds indicate that interactions with more agents are needed
before evolving. Like prior GE work O’Neill & Ryan (2001), if the threshold is
exceeded, agents apply genetic operations to the gene pool in the order: a)
selection, b) crossover, and c) mutation. Selection samples a subset of
genotypes based on the fitness value to form a new population. Crossover
combines two genomes to produce a new genome. Mutation randomly flips the bits
of the genome. During the update phase, an agent replaces its current genotype
with a new geneotype if there is a new genotype with higher fitness. Each
evolution/learning time-step, all agents sense, act, and update.

Parameters | GEESE | GEESE-BT
---|---|---
Storage Threshold | NA | 7
Interaction Probability | 0.8 | 0.85
Parent-Selection | Tournament | Fitness + Truncation
Elite-size | 1 | N/A
Mutation Probability | 0.01 | 0.01
Crossover Probability | 0.9 | 0.9
Crossover | variable_onepoint | variable_onepoint
Genome-Selection | Tournament | Diversity
Number of Agents | 100 | 100
Evolution Steps | 284 | 12000
Max Tree Depth | 10 | 10
Table 1: Evolution parameters used by GEESE and GEEST-BT.

GEESE-BT (Neupane & Goodrich, 2019b) replaced the state-machine controllers in
GEESE with Behavior Trees (BT). Paraphrasing Neupane & Goodrich (2019b) for
context, a BT is a directed rooted tree with internal control flow nodes and
leaf execution nodes (Colledanchise & Ögren, 2018). BT execution starts at the
root node by generating ticks at a fixed frequency. After each tick, a node
returns running, meaning that processing is ongoing, success, meaning that the
node’s objective is achieved, or failure, meaning neither running nor success.
Control nodes include selectors, which act as logical or, parallel and
sequence nodes, which act as logical and, and condition nodes, which act as
logical if.

##  4 BeTr-GEESE

BeTr-GEESE shares the core evolutionary stages (sense, act, and update) and
genetic operations as GEESE and GEEST-BT, but has three significant
improvements: a) PPA-based primitive behaviors, b) PPA-style BNF grammars, and
c) a BT-induced fitness function. Since each of these changes impact swarm
performance, each is experimentally evaluated in Sections 4.4-4.6,
respectively. BeTr-GEESE uses the same parameters as GEESE-BT (Table 1) to
ensure unbiased comparison.

###  4.1 Agent Lifecycle

The BeTr-GEESE algorithm runs for a particular number of steps, which is
defined before the start of the simulation111BeTr-GEESE learning is not
episodic like some GE algorithms; there are no terminal states and replay..
$n$ agents are initialized in a grid environment as described in Section 4.2,
and a random genome of length 100 is created. The genome is transformed to a
BT controller by the genotype-to-phenotype mapping process using the swarm BNF
grammar described in appendix B. Each agent interacts with the environment
using the BT controllers and updates its fitness value using diversity fitness
and an overall fitness function $\mathbf{A}_{t}$ described in Section 4.6.
(Diversity fitness is the total number of unique behaviors nodes divided by
the total behaviors defined in the grammar.) Then the agents independently
perform the sense, act, and update methods described in Section 3.

###  4.2 Simulation Parameters

A 100x100 grid environment was used with a hub of radius ten at the origin.
The population had 100 learning agents. Agents moved with speed of 2 units per
time step. Task performance is measured in terms of food or debris moved at
the end of 12,000 learning steps. 12,000 learning steps was adopted from
GEESE-BT (Neupane & Goodrich, 2019b), and from empirical analysis it was
observed that for all three task reported in GEESE-BT obtained best
performance when the simulation was run for at least 12,000 steps.

The swarm simulation environment was created using the Mesa agent-based
modeling framework (Kazil et al., 2020). The simulator lacks the high fidelity
physic engine, making it faster to run experiments with a large number of
agents but greatly simplifies the robot-environment interactions. BT
controllers for the agents were created using Behavior Tree framework py_trees
(Stonier & Staniaszek, 2021) and PonyGE2 (Fenton et al., 2017) was used to
implement the Grammatical Evolution algorithms GEESE, GEESE-BT, and BeTr-
GEESE. All experiments were performed on a machine with an i9 CPU, 64 GB RAM
running 16 parallel threads.

###  4.3 Evaluation Metrics

BeTr -GEESE is evaluated using two swarm tasks, Foraging and Nest Maintenance.
The Foraging task requires agents to retrieve food from a source to a hub. A
single foraging site of radius ten with multiple “food” objects is randomly
placed at 30 units from the hub. The total food units is set to equal agent
population. Task performance is the percentage of food at the hub.

The Nest Maintenance task requires agents to move debris near the hub to a
place outside a fixed boundary. Multiple “debris” objects are placed within
ten units radius of the hub. The desired boundary is set at 30 units radius
away from the origin. The total debris units at the hub is equal to the agent
population. Task performance is the percentage of debris that is outside the
boundary.

The simulation is labelled learning when the agents are evolving controllers
where the behaviors of the agent changes with time. The simulation is labelled
test when the agent controllers from a learning simulation are transferred to
a new simulation environment and the agents controllers remains static and do
not evolve. For all the experiments reported in this paper, test simulation
does not differ drastically but only in the positions of sites and obstacles
placed randomly at the runtime. Foraging is deemed successful if more than
$80\%$ food is collected, and nest maintenance is successful if more than
$80\%$ debris is removed. Success rate is defined as the ratio of the number
of successful trials to the total number of simulations. Learning efficiency
is defined as foraging or maintenance percentage at the end of a learning
simulation.

###  4.4 Adding PPA to Primitive Behaviors

Figure 1(b) illustrates how GEESE-BT requires a precondition (IsCarryable) to
be satisfied and the task (Carry) successfully performed. The and operator is
implemented as a sequence BT node ($\rightarrow$). BeTr -GEESE uses the PPA
structure in Figure 1(a). The root selector node (represented as the “?”)
checks the postcondition (left branch, AlreadyCarrying) and calls the sequence
node in the right child only if the postcondition is not met. The right child
checks preconditions and implements the action. The postcondition ensures that
the planner does not need to re-execute when the goal has been met. The PPA
structure shown is for the CompositeSingleCarry behavior; see Appendix A.

Figure 1: BTs for CompositeSingleCarry for a) PPA-style and b) nominal trees.

The green and blue boxes in Figures 2(a)-2(b) compare learning efficiency
(over a range of fitness function conditions, described below) between the
GEESE-BT primitive behaviors (PB) and the PPA-based primitive behaviors (BeTr
-PB) for the two tasks. The PPA structure improves performance across all
fitness function conditions and for both tasks.

(a) Foraging experiments.

(b) Nest-maintenance experiments.

Figure 2: Learning efficiency, measured by the percentage of food/debris
transported to/from the hub with respect to variations in primitive behaviors,
grammar, and fitness function.

###  4.5 Adding PPA to the Grammar

Appendix B presents the grammar used by BeTr -GEESE, which is modified from
the GEEST-BT grammar to use the PPA structure. The blue and yellow boxes in
the leftmost and middle groups of Figures 2(a)-2(b) both use PPA-based PBs,
but the yellow boxes use PPA-based grammars, respectively. Changing the
grammar to use PPA structures further improves learning efficiency for both
tasks.

###  4.6 Replacing ad hoc Fitness with BT Status

Essential Fitness Elements. There are various choices of the fitness function
in evolutionary robotics based on the type of controllers and amount of priory
task knowledge the designer has (Nelson et al., 2009). GE requires a diverse
genetic population, and both foraging and nest maintenance require exploring
the world. Thus, two fitness elements are required: diversity (type I) and
exploration (type II). The phenotype is an agent program learned from the
grammar. For GEESE-BT and BeTr -GEESE, the program is a behavior tree.
Diversity fitness is denoted by $D:\\{{\rm
Phenotypes}\\}\rightarrow\mathbb{R}$, promotes diversity in the genotype, and
is defined as the total number of unique behavior nodes in the BT divided by
the total possible behaviors defined in the grammar. Exploration fitness is
denoted by $E:\\{{\rm Locations}\\}\longrightarrow\mathbf{R}$, promotes
visiting new locations, and is defined as the number of unique world locations
visited by the agent.

Ad hoc Fitness Elements. GEESE-BT uses ad hoc fitness functions. Prospective
fitness (type III) rewards “intrinsic” actions like picking up, carrying, or
dropping objects. Task-specific fitness (type IV) uses hand-tuned fitness
functions designed to reward collective actions: total food at the hub and the
total debris away from the hub, respectively.

BT Feedback Fitness. The BT feedback fitness (type V) is denoted by $B:\\{{\rm
BehaviorTreeStatus}\\}\longrightarrow\mathbf{R}$ and is defined as the sum of
potstcondition, constraint, and selector node rewards. When a postcondition
node status is success, a subjectively chosen reward of $+1$ indicates that
some potentially useful condition in the world holds. A subjectively chosen
reward of $-2$ occurs when a constraint node status is failure. A subjectively
chosen reward of $+1$ is returned when the root selector node status success,
indicating that some sub-task has been accomplished somewhere in the BT. Based
on Nelson et al. (2009), BT feedback (type V) incorporates a low level of
prior knowledge, whereas ad hoc fitness (type III & type IV) incorporates high
or excessive prior knowledge. BT feedback is superior to ad hoc fitness
because it requires less prior knowledge, is general for all swarm tasks, and
remains the same while the task changes.

Blending Fitness Types. GEESE-BT blends types I–IV to produce overall fitness.
Since types III and IV are ad hoc, overall fitness is ad hoc. By contrast,
BeTr -GEESE rewards behaviors that promote genetic diversity, promote world
exploration, observe or accomplish subtasks, or avoid constraint violations.
Let $\mathbf{A}_{t}$ denote an agent’s fitness, defined as the exponential
blend $\mathbf{A}_{t}=\beta(\mathbf{A}_{t-1})+(D+E_{t}+B_{t})$, with $\beta$
empirically set to $0.9$. Historical blending overcomes the temporal sparsity
of diversity, exploration, and BT status.

Fitness Results. The leftmost and middle group in Figures 2(a)-2(b) use only
diversity or diversity plus exploration fitness. Performance improves with
exploration fitness. The rightmost group in the figures uses diversity,
exploration, and the ad hoc fitness for GEESE-BT (green) and for GEESE with
PPA structures (blue). The yellow box in the rightmost group uses diversity
and exploration, and replaces the ad hoc and hand-tuned rewards with BT
status. The rightmost group shows that replacing ad hoc fitness with BT status
yields substantial improvement. Out of 64 simulation runs, BeTr-GEESE succeed
75% of the time, whereas GEESE-BT succeeded only 9.3%.

##  5 Performance of Fixed Programs

BeTr -GEESE agents succeed while evolving, but they perform poorly once
evolution stops (fixed agents). This section presents results for various
mixtures of 100 fixed agents.

###  5.1 Homogeneous Populations of Best-Performing Agents

Create a homogeneous population by selecting the fittest agent at the end of
the evolution and forming a population with 100 copies of that agent.
Homogeneous populations of evolved BeTr -GEESE and GEESE-BT agents fail on
both tasks. The performance of homogeneous populations is poor because of the
types of programs evolved. The foraging task requires at least four PPA sub-
trees: explore the environment and find the site, carry the food, bring the
food back to the hub, and drop the food at the hub. Among 100 independent
evolution experiments, 3235 different BeTr -GEESE programs were evolved, 97.8%
had just one PPA sub-tree, and 2% had two PPA sub-trees. These static programs
are not capable of solving the problem by themselves. By contrast, GEESE-BT
more frequently produced agents with all four necessary subtrees but still
failed, indicating heterogeneous controllers are needed.

###  5.2 Heterogeneous Populations of High-Performing Agents

The fitness of an individual agent can be deceiving because the agent might be
fit only when other agents in a heterogeneous population are performing
necessary supporting tasks (Page, 2010). Form a heterogeneous population by
sorting the agents at the end of evolution by their fitness value, identifying
the top n% of the agents, and then cloning those agents to create 100 agents.
As with the homogeneous agents, the performance of a heterogeneous population
of BeTr -GEESE agents was terrible; less than 1% of the food available is
brought to the hub so the results are not shown in the figure. Each agent is
capable of only doing one subtask, and that means that agents cannot both pick
up and drop objects. By contrast, Figure 3 shows that heterogenous blends of
GEESE-BT agents often succeed, especially on the nest-maintenance task,
precisely because they can learn more complicated programs. However, as more
types of agents are added to the population, interference occurs, presumably
because less fit agents perform only partial tasks and thus prevent top-
performing agents from fully performing all tasks.

Figure 3: Population quality for populations created by sampling the top $n\%$
of agents for GEESE-BT and remixing BeTr -GEESE agents.

###  5.3 Heterogeneous Populations of Blended Agents

The better performance of the more complicated GEESE-BT programs suggests a
way to blend the modular BeTr -GEESE programs so that they are more
complicated as follows. First, sort agents most fit to least fit. Second,
select the top $n\%$ of the agents. Each of these agents typically has a BT
that performs only one subtask. Third, or these agents together by forming a
root BT node with a parallel node and then adding agents as children to this
root node. The parallel control node was chosen as it loosely acts as a
logical or, meaning that the blended agent allows its modular BeTr -GEESE
programs (sub-trees) to contribute to the behavior of the agent. For each
agent, the order of the sub-trees is randomized so that different agents try
execute subtasks in different orders.

Figure 3 shows performance of heterogeneous populations of BeTr -GEESE agents
formed using this blended approach. Heterogeneous populations of blended
GEESE-BT agents are not shown because they perform no better than the
heterogeneous populations formed in Section 5.2. The performance of blended
BeTr -GEESE populations slowly increases and peaks at 0.5 and then decreases
gradually for both tasks. Blending works precisely because the resulting
agents capture pieces of agents capable of successfully performing needed
subtasks.

##  6 Conclusion and Future Work

The PPA-structures used in BeTr -GEESE cause agents to learn simple programs
capable of performing required subtasks. The evolving population of BeTr
-GEESE agents succeed because mutation, crossover, and lateral transfer cause
the agents to change between simple programs while learning, effectually “time
multiplexing” between the simple programs. Time-multiplexing allows agents to
adapt to different circumstances in their learning environment, effectually
negating the need to rely on static strategies. GEESE-BT’s success while
learning is low because it is difficult to learn complicated programs even
with the ad hoc reward structures designed to promote efficient learning.
Heterogeneous populations of fixed BeTr -GEESE agents perform poorly because
each agent learns only a portion of the entire task. The success of the
heterogeneous population of can be improved by blending the simple agents
together, but performance is still not satisfactory.

Future work should explore BeTr -GEESE agents could rapidly adapt to changing
conditions, provided that the grammar has a sufficiently rich set of execution
nodes and primitive behaviors. Additionally, future work should explore
genetic algorithm hyper-parameter settings that might enable BeTr -GEESE to
evolve complex, resilient behaviors more efficiently.

##  Appendix A Primitive Behaviors

Spatial Behaviors. The CompositeDrop behavior enables agent to drop the item
it is currently carrying at the current location, and the CompositeSingleCarry
allows an agent to pickup an object at its current location. The MoveTowards
primitive behavior allows the agent to move a unit step towards a particular
object in the environment, and MoveAway moves the agent a unit step away from
an object. Explore moves the agent a unit step in a random direction. Obstacle
avoidance in MoveTowards, MoveAway, and Explore use the simple bug following
algorithm from Lumelsky & Stepanov (1987), which checks for obstacles or traps
in its line-of-sight. If an object is detected, the agent moves one unit
distance parallel to the object’s surface.

##  Appendix B PPA Grammar

¡root¿ ::= ¡sequence¿ — ¡selector¿

¡sequence¿ ::= [Sequence]¡ppa¿[/Sequence]— [Sequence]¡root¿¡root¿[/Sequence]
[Sequence]¡sequence¿¡root¿[/Sequence]

¡selector¿ ::= [Selector]¡ppa¿[/Selector]—[Selector]¡root¿¡root¿[/Selector]
[Selector]¡selector¿¡root¿[/Selector]

¡ppa¿ ::= [Selector]¡postconditions¿¡ppasequence¿[/Selector]

¡postconditions¿ ::= ¡SuccessNode¿ — ¡ppa¿
—[Sequence]¡postcondition¿[/Sequence]

¡postcondition¿ ::= ¡postcondition¿[PostCnd]¡postconditiont¿
[/PostCnd]—[PostCnd]¡postconditiont¿[/PostCnd]

¡postconditiont¿ ::= NeighbourObjects_¡objects¿—…— IsVisitedBefore_¡sobjects¿

¡ppasequence¿ ::= [Sequence]¡preconditions¿[Act]¡action¿
[/Act][/Sequence]—[Sequence]¡constraints¿[Act]¡action¿[/Act][/Sequence]—
[Sequence]¡preconditions¿¡constraints¿ [Act]¡action¿[/Act][/Sequence]

¡preconditions¿ ::= [Sequence]¡precondition¿[/Sequence]

¡precondition¿ ::= ¡precondition¿[PreCnd]¡preconditiont¿
[/PreCnd]— [PreCnd]¡preconditiont¿[/PreCnd]

¡preconditiont¿ ::= IsDropable_¡sobjects¿— … — IsCarrying_¡dobjects¿

¡constraints¿ ::= [Sequence]¡constraint¿[/Sequence]

¡constraint¿ ::= ¡constraint¿[Cnstr]¡constraintt¿[/Cnstr]—
[Cnstr]¡constraintt¿[/Cnstr]

¡constraintt¿ ::= CanMove —…— IsDropable_¡sobjects¿

¡action¿ ::= MoveAway_¡sobjects¿—…—Explore

¡objects¿ ::= ¡sobjects¿—¡dobjects¿

¡sobjects¿ ::= Hub—Sites

¡oobjects¿ ::= Obstacles—Trap

¡dobjects¿ ::= Food—Debris

¡SuccessNode¿ :: = [PostCnd]DummyNode[/PostCnd]

Productions 1–4 define control nodes as part of a hierarchical set of PPA-
structured sub-trees. Productions 5–7 define postconditions, which can be
dummy nodes, an embedded PPA sub-tree, or condition nodes. Production rule 20,
the DummyNode, is a condition node that always returns Success. Production 8
defines the right sub-tree of a PPA structure that sequentially combined
preconditions and actions. Productions 9–11, define preconditions using
condition nodes. Productions 12–14, define constraints using condition nodes.
Production 15 calls the BT-based subtrees in which the primitive behaviors of
the agent are defined. Productions 16–18 define static elements in the swarm
environment, specifically a hub, sites, obstacles, and traps, all of which
have fixed sizes. Production 19 defines movable objects, food and debris, in
the environment. A hub is where agents originate, a site is a source of food
for the agents, an obstacle obstructs the path of the agent and is not
passable, a trap kills the agent when it comes in contact, food is an object
to be carred to the hub, and debris is an object to be removed from the hub.
All execution nodes have self-explanatory names.

## References

  * Brooks (1986) R. Brooks.  A robust layered control system for a mobile robot.  _IEEE Journal on Robotics and Automation_ , 2(1):14–23, 1986.
  * Cli et al. (1993) D. Cli, P. Husbands, and I. Harvey.  Evolving visually guided robots.  In _From Animals to Animats 2. Proc. of the 2nd Intl. Conf. on Simulation of Adaptive Behavior_ , pp. 374–383. MIT Press, 1993.
  * Colledanchise & Ögren (2017) M. Colledanchise and P. Ögren.  How behavior trees modularize hybrid control systems and generalize sequential behavior compositions, the subsumption architecture, and decision trees.  _IEEE Transactions on Robotics_ , 33(2):372–389, 2017.
  * Colledanchise & Ögren (2018) M. Colledanchise and P. Ögren.  Behavior trees in robotics and al: An introduction.  2018\.
  * Colledanchise et al. (2019) M. Colledanchise, D. Almeida, and P. Ögren.  Towards blended reactive planning and acting using behavior trees.  In _2019 ICRA_ , pp. 8839–8845. IEEE, 2019.
  * Dorigo et al. (2006) M. Dorigo, M. Birattari, and T. Stutzle.  Ant colony optimization.  _IEEE Computational Intelligence Magazine_ , 1(4):28–39, 2006.
  * Duarte et al. (2016) M. Duarte, V. Costa, J. Gomes, T. Rodrigues, F. Silva, S. M. Oliveira, and A. L. Christensen.  Evolution of collective behaviors for a real swarm of aquatic surface robots.  _PloS One_ , 11(3):e0151834, 2016.
  * Engebråten et al. (2018) Sondre A Engebråten, Jonas Moen, Oleg Yakimenko, and Kyrre Glette.  Evolving a repertoire of controllers for a multi-function swarm.  In _International Conference on the Applications of Evolutionary Computation_ , pp. 734–749. Springer, 2018.
  * Fan et al. (2021) F. Fan, J. Xiong, M. Li, and G. Wang.  On interpretability of artificial neural networks: A survey.  _IEEE Transactions on Radiation and Plasma Medical Sciences_ , 2021\.
  * Fenton et al. (2017) Michael Fenton, James McDermott, David Fagan, Stefan Forstenlechner, Erik Hemberg, and Michael O’Neill.  Ponyge2: Grammatical evolution in python.  In _Proceedings of the Genetic and Evolutionary Computation Conference Companion_ , pp. 1194–1201, 2017.
  * Ferrante et al. (2013) E. Ferrante, E. Duéñez-Guzmán, A. E. Turgut, and T. Wenseleers.  Geswarm: Grammatical evolution for the automatic synthesis of collective behaviors in swarm robotics.  In _Proc. of the 15th annual GECCO conference_ , pp. 17–24. ACM, 2013.
  * Fikes & Nilsson (1971) R. E. Fikes and N. J. Nilsson.  Strips: A new approach to the application of theorem proving to problem solving.  _Artificial intelligence_ , 2(3-4):189–208, 1971\.
  * Gordon (2010) D. M. Gordon.  _Ant encounters: interaction networks and colony behavior_ , volume 1.  Princeton University Press, 2010.
  * Jablonka & Lamb (2014) Eva Jablonka and Marion J Lamb.  _Evolution in four dimensions, revised edition: Genetic, epigenetic, behavioral, and symbolic variation in the history of life_.  MIT press, 2014.
  * Karaboga & Basturk (2007) D. Karaboga and B. Basturk.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (abc) algorithm.  _Journal of global optimization_ , 39(3):459–471, 2007.
  * Kazil et al. (2020) Jackie Kazil, David Masad, and Andrew Crooks.  Utilizing python for agent-based modeling: The mesa framework.  In Robert Thomson, Halil Bisgin, Christopher Dancy, Ayaz Hyder, and Muhammad Hussain (eds.), _Social, Cultural, and Behavioral Modeling_ , pp. 308–317, Cham, 2020. Springer International Publishing.  ISBN 978-3-030-61255-9.
  * Knoblock (1995) C. A. Knoblock.  Planning, executing, sensing, and replanning for information gathering.  Technical report, University of Southern California, 1995.
  * König et al. (2009) L. König, S. Mostaghim, and H. Schmeck.  Decentralized evolution of robotic behavior using finite state machines.  _International Journal of Intelligent Computing and Cybernetics_ , 2(4):695–723, 2009.
  * Kriesel et al. (2008) D. M. Kriesel, E. Cheung, M. Sitti, and H. Lipson.  Beanbag robotics: Robotic swarms with 1-dof units.  In _ANTS 2008_ , pp. 267–274. Springer, 2008.
  * Kucking et al. (2018) J. Kucking, A. Ligot, D. Bozhinoski, et al.  Behavior trees as a control architecture in the automatic design of robot swarms.  In _ANTS 2018_. IEEE, 2018.
  * Kuckling et al. (2021) J. Kuckling, Vincent Van P., and M. Birattari.  Automatic modular design of behavior trees for robot swarms with communication capabilites.  In _EvoApplications_ , pp. 130–145, 2021.
  * Lane (2015) Nick Lane.  _The vital question: Energy, evolution, and the origins of complex life_.  WW Norton & Company, 2015.
  * Lee (1999) Wei-Po Lee.  Evolving complex robot behaviors.  _Information Sciences_ , 121(1-2):1–25, 1999\.
  * Lewis (1996) F. L. Lewis.  Neural network control of robot manipulators.  _IEEE Expert_ , 11(3):64–75, 1996.
  * Lewis et al. (1992) M. A. Lewis, A. H. Fagg, and A. Solidum.  Genetic programming approach to the construction of a neural network for control of a walking robot.  In _Robotics and Automation, 1992. Proceedings., 1992 IEEE Intl. Conf. on_ , pp. 2618–2623. IEEE, 1992.
  * Lumelsky & Stepanov (1987) V. J. Lumelsky and A. A. Stepanov.  Path-planning strategies for a point mobile automaton moving amidst unknown obstacles of arbitrary shape.  _Algorithmica_ , 2(1):403–430, 1987.
  * Nelson et al. (2009) Andrew L Nelson, Gregory J Barlow, and Lefteris Doitsidis.  Fitness functions in evolutionary robotics: A survey and analysis.  _Robotics and Autonomous Systems_ , 57(4):345–370, 2009.
  * Neupane & Goodrich (2019a) A. Neupane and M. A. Goodrich.  Designing emergent swarm behaviors using behavior trees and grammatical evolution.  In _Proc. of the 18th AAMAS conference_ , pp. 2138–2140, 2019a.
  * Neupane & Goodrich (2019b) A. Neupane and M. A. Goodrich.  Learning swarm behaviors using grammatical evolution and behavior trees.  In _IJCAI_ , pp. 513–520, 2019b.
  * Neupane et al. (2018) A. Neupane, M. A. Goodrich, and E. G. Mercer.  Geese: grammatical evolution algorithm for evolution of swarm behaviors.  In _Proc. of the 20th annual GECCO conference_ , pp. 999–1006, 2018.
  * Ögren (2020) P. Ögren.  Convergence analysis of hybrid control systems in the form of backward chained behavior trees.  _IEEE Robotics and Automation Letters_ , 5(4):6073–6080, 2020.
  * O’Neill & Ryan (2001) M. O’Neill and C. Ryan.  Grammatical evolution.  _IEEE Transactions on Evolutionary Computation_ , 5(4):349–358, 2001.
  * Page (2010) S. E. Page.  _Diversity and Complexity_.  Princeton University Press, 2010.
  * Parashar et al. (2021) P. Parashar, A. Naik, J. Hu, and H. I Christensen.  Meta-modeling of assembly contingencies and planning for repair.  _arXiv preprint arXiv:2103.07544_ , 2021.
  * Petrovic (2008) Pavel Petrovic.  Evolving behavior coordination for mobile robots using distributed finite-state automata.  In _Frontiers in evolutionary robotics_. InTech, 2008.
  * Pintér-Bartha et al. (2012) A. Pintér-Bartha, A. Sobe, and W. Elmenreich.  Towards the light—comparing evolved neural network controllers and finite state machine controllers.  In _Proc. of the 10th Workshop on WISES_ , pp. 83–87. IEEE, 2012\.
  * Quammen (2018) David Quammen.  _The tangled tree: A radical new history of life_.  Simon and Schuster, 2018.
  * Sprague et al. (2018) C. I. Sprague, Ö. Özkahraman, A. Munafo, R. Marlow, et al.  Improving the modularity of auv control systems using behaviour trees.  In _2018 IEEE/OES Autonomous Underwater Vehicle Workshop (AUV)_ , pp. 1–6. IEEE, 2018.
  * Steiner (1972) Ivan Dale Steiner.  _Group process and productivity_.  Academic press, 1972.
  * Stonier & Staniaszek (2021) Daniel Stonier and Michal Staniaszek.  Behavior Tree implementation in Pyton, 12 2021.  URL https://github.com/splintered-reality/py_trees/.
  * Sumpter & Pratt (2003) D. Sumpter and S. Pratt.  A modelling framework for understanding social insect foraging.  _Behavioral Ecology and Sociobiology_ , 53(3):131–144, 2003.
  * Sutton (1984) R. S. Sutton.  _Temporal credit assignment in reinforcement learning_.  PhD thesis, University of Massachusetts Amherst, 1984.
  * Trianni et al. (2003) Vito Trianni, Roderich Groß, Thomas H Labella, Erol Şahin, and Marco Dorigo.  Evolving aggregation behaviors in a swarm of robots.  In _European Conference on Artificial Life_ , pp. 865–874. Springer, 2003.
  * Valmari (1996) A. Valmari.  The state explosion problem.  In _Advanced Course on Petri Nets_ , pp. 429–528. Springer, 1996\.


m}\cos(qA_{\tau})+\cdots\right)$ (VI.12)

(If we move away from very massive limit, then we can still expand the
effective action in harmonics of $\cos(qA_{\tau})$, in which case you can view
this as the leading term.) The 3d symmetry currents that we previously
identified can now be understood in terms of these low-energy fields as:

$J^{ij}={\epsilon}^{ijk}\partial_{k}A_{\tau}\qquad
j^{i}={\epsilon}^{ijk}\partial_{j}A_{k}$ (VI.13)

Note the somewhat surprising fact that in the phase just described, the
$U(1)^{(0)}_{m}$ is spontaneously broken. As we first saw back in the
motivational Section I.1, it is possible to dualize the 3d vector field into a
compact scalar ${\theta}$ in (I.2), in terms of which the action for $A_{i}$
becomes that of a massless scalar action for ${\theta}$. ${\theta}$ is the
Goldstone mode of the symmetry breaking of $U(1)^{(0)}_{m}$.

In the application to plasmas, this captures the fact that the magnetic field
is unscreened. If you imagine probing the finite temperature medium with two
probe magnetic monopoles, they will feel a Coulomb law attraction, mediated by
a gapless long-range force. This force arises from the Goldstone mode.

Note that due to the coupling to the finite-temperature charged matter,
$A_{\tau}$ is gapped out – and therefore probing the system with two probe
electric charges will not result in a long-range force. This is precisely the
physics of Debye screening: the electric field is screened by free charges in
the medium.

The situation can be summarized in the table below; the phase we have just
described is what is normally called plasma202020It is somehow soothing after
all these years to find that this sophisticated pattern of higher-form
symmetry breaking formally justifies the zeal with which my high-school
science textbook distinguished between the “plasma” phase and conventional
liquids and gases. It also allows one to formally distinguish between things
like $\vec{D}$ vs $\vec{E}$ and $\vec{B}$ vs $\vec{H}$ and so on; the reader
who wants to exorcise these possibly unpleasant memories using the
sledgehammer of 1-form symmetry is directed to Appendix B of Das:2023nwl .

| $U(1)^{(1)}_{m}$ | $U(1)^{(0)}_{m}$
---|---|---
Plasma (normal phase) | Unbroken | Spontaneously broken
Superconducting | Unbroken | Unbroken

Here we have also included the pattern of symmetry breaking for the
superconducting phase at finite $T$. This is essentially the same as in Table
1 at zero temperature.

Note that the finite-temperature phase transition between the superconducting
and plasma phase at finite temperature is just the breaking of an ordinary
$U(1)$ 0-form symmetry in the 3 (non-compact) directions. In general this is
the XY universality class. The application to finite-temperature
superconductors is called inverted XY criticality Peskin:1977kp ;
dasgupta1981phase ; kiometzis1994critical ; herbut1996critical  as you will
notice the interesting fact that the plasma (high temperature) phase has the
symmetry spontaneously broken. This counterintuitive behavior arises
ultimately from the origin of the finite temperature 0-form symmetry from the
reduction of a 1-form symmetry on the thermal circle. The consequences of this
pattern of symmetry breaking for a higher-form description of hydrodynamics
was described in Armas:2018zbe ; Armas:2018atq . Discussion in language
similar to that above can be found in Iqbal:2020lrt ; Das:2023vls .

###  VI.2 Finite temperature: magnetohydrodynamics

Let us now think about the real-time dynamics of the finite-temperature QED
plasma. This is the area that is the domain of hydrodynamics.

####  VI.2.1 Lightning review of ordinary hydrodynamics

Let us briefly review the philosophy of hydrodynamics: it is an amazing fact
that essentially any interacting system (quantum or otherwise) at finite
temperature basically behaves at long distances and late times as a fluid,
whose structure is dictated by the global symmetries and conserved charges of
the system landau1987fluid .

To understand this, note that the excitations of most of the degrees of
freedom in a fluid will decay away with a characteristic time-scale $\tau$
that is set by some microscopic scale (e.g. the mean free path
$\tau\sim\frac{\ell_{p}}{v}$, or the temperature $\tau\sim T^{-1}$, etc.).
However, conserved quantities cannot vanish arbitrarily fast: instead they
must “spread out”, and the speed of this process depends on the size of the
excitation. Thus if one creates a lump of charge with some size $L$, the speed
of its decay will decay as some inverse power of $L$ ($\tau\sim L^{-2}$ for
the most generic diffusive processes). At late times only the conserved
charges are left.

In the modern formulation of hydrodynamics, this philosophy has been promoted
to an algorithm: given a structure of global symmetries, one can turn a crank
and a hydrodynamic theory pops out Kovtun:2012rj , which can be expressed in
terms of coarse grained variables.

In the simplest case of a relativistic case with a conserved stress tensor

$\partial_{\mu}T^{\mu\nu}=0$ (VI.14)

the fluid degrees of freedom are a normalized fluid velocity $u^{\mu}(x)$,
$u^{2}=-1$, and a scalar temperature field $T(x)$. The microscopic stress
tensor can be expressed in terms of these coarse-grained variables as

$T^{\mu\nu}=({\epsilon}+p)u^{\mu}u^{\nu}+pg^{\mu\nu}$ (VI.15)

Here ${\epsilon}(T)$ and $p(T)$ are functions of the temperature through the
equation of state of the fluid, and the conservation of the stress tensor
becomes a relativistic Navier-Stokes equation for $u^{\mu}$. If we further
consider the case of hydrodynamics with an ordinary $0$-form global symmetry
then we also add a degree of freedom $n(x)$ for the local charge density and
write

$T^{\mu\nu}=({\epsilon}+p)u^{\mu}u^{\nu}+pg^{\mu\nu}\qquad j^{\mu}=nu^{\mu}$
(VI.16)

The conservation equation $\partial_{\mu}j^{\mu}=0$ provides the required
dynamical equation of motion for $n(x)$.

Finally, we stress that hydrodynamics is an effective theory, i.e. the
expressions above can be systematically improved order by order in
derivatives. Very important physics appears at first order in derivatives,
which we discuss in the higher-form context below.

####  VI.2.2 Magnetohydrodynamics

Let us now apply this formalism to studying QED (with charged matter) at
finite temperature. The hydrodynamic description of this model is normally
called relativistic magnetohydrodynamics; it is conventionally studied by
coupling the gauged electric charge current in the formalism above to
Maxwell’s equations in some form (see bellan2008fundamentals  for a textbook
treatment and Hernandez:2017mch  for a modern discussion).

We will not do this – instead we will go straight to the true 1-form global
symmetry.

The conserved quantities are the regular stress-energy tensor $T^{\mu\nu}$ and
the magnetic 2-form212121In this section we use a different normalization for
the current than in (IV.2), to be closer to the fluid dynamics literature.
current $J^{\mu\nu}=\frac{1}{2}{\epsilon}^{\mu\nu\rho\sigma}F_{\rho\sigma}$.
We now want to realize these objects in terms of some coarse-grained
variables. This can be done by turning the hydrodynamic crank, but generalized
to 1-form symmetries Grozdanov:2016tdf . The idea of casting MHD as a fluid of
strings predates the formal technology of 1-form symmetries, and was first
discussed in Schubring:2014iwa .

Here we discuss only the results, referring you to Grozdanov:2016tdf  for the
derivations. At zeroth order in derivatives we have:

$T^{\mu\nu}_{(0)}=({\epsilon}+p)u^{\mu}u^{\nu}+pg^{\mu\nu}-\mu\rho
h^{\mu}h^{\nu}\qquad J^{\mu\nu}_{(0)}=2\rho u^{[\mu}h^{\nu]}$ (VI.17)

Here $u^{\mu}$ is the regular timelike fluid velocity. $h^{\mu}$ is a
spacelike vector which indicates the direction of the magnetic field; these
satisfy

$u^{2}=-1\qquad h^{2}=1\qquad u\cdot h=0$ (VI.18)

${\epsilon}$ and $p$ are the energy density and pressure as usual. From the
expression for $J^{\mu\nu}$ we see that $\rho$ is the density of magnetic
flux. Finally $\mu$ is a scalar which can be thought of as the chemical
potential that is conjugate to $\rho$. There are two independent scalar
quantities, which can conveniently be chosen to be $T$ and $\mu$, and there
are the regular thermodynamic constraints between the scalars:

${\epsilon}+p=Ts+\mu\rho\qquad dp=sdT+\rho d\mu$ (VI.19)

where $s$ is the entropy density.

Note the term in $-\mu\rho h^{\mu}h^{\nu}$ – this can be understood as the
tension arising from the magnetic field lines.

The conservation equations $\partial_{\mu}T^{\mu\nu}=0$ and
$\partial_{\mu}J^{\mu\nu}=0$ now provide us with dynamics for the fluid
degrees of freedom, which can be thought of as a kind of Navier-Stokes
equation of $u^{\mu}$ and an equation telling us how the magnetic field
$(\rho,h^{\mu})$ evolves. Note that we never need to use Maxwell’s equations –
the equations close all by themselves, though one can reverse the logic and
interpret the magnetic equation of motion as a kind of Maxwell’s equation.

We can now go to higher order in derivatives; the above is simply the first
term in an expansion:

$T^{\mu\nu}=T^{\mu\nu}_{(0)}+T^{\mu\nu}_{(1)}+\cdots\qquad
J^{\mu\nu}=J^{\mu\nu}_{(0)}+J^{\mu\nu}_{(1)}+\cdots$ (VI.20)

Here for reasons of brevity we record only the first-order correction to
$J^{\mu\nu}$, which takes the form:

$J^{\mu\nu}_{(1)}=2m^{[\mu}h^{\nu]}+s^{\mu\nu}$ (VI.21)

where we have

$m^{\mu}=-2r_{\perp}T\Delta^{\mu\beta}h^{\nu}\partial_{\beta}\left(\frac{h_{\nu]}\mu}{T}\right)\qquad
s^{\mu\nu}=-2\mu
r_{\parallel}\Delta^{\mu\rho}\Delta^{\nu\sigma}\partial_{[\rho}h_{\sigma]}$
(VI.22)

We have introduced the projector
$\Delta^{{\alpha}\beta}=g^{{\alpha}\beta}+u^{{\alpha}\beta}-h^{{\alpha}\beta}$,
which annihilates both $u$ and $h$ and thus projects onto the subspace
perpendicular to the magnetic field lines. We have introduced two new
parameters $r_{\perp}$ and $r_{\parallel}$; these can be understood as
resistivities that are parallel and perpendicular to the magnetic field lines.

To determine the resistivities, we compute them from Kubo formulas, which
relate dissipative transport coefficients to retarded Green’s functions of the
current operator $J^{\mu\nu}$ in thermal equilibrium. If we place the
background magnetic field in the $z$ direction, these Kubo formulas are:

$r_{\parallel}=\lim_{\omega\to 0}\frac{G^{xy,xy}_{JJ}(\omega)}{-i\omega}\qquad
r_{\perp}=\lim_{\omega\to 0}\frac{G^{xz,xz}_{JJ}(\omega)}{-i\omega}$ (VI.23)

In (VI.22) there is a particular pattern of derivatives that is quite
constrained; this arises from demanding consistency of the second law of
thermodynamics, which guarantees that entropy always locally increases. Indeed
once one works out the consequences of these terms, we find that there is
diffusion of magnetic field lines. In the simplest case where we set the
background magnetic field to zero, the two resistivities agree
$r_{\perp}=r_{\parallel}=r$ and the dispersion relation takes the form
Hofman:2017vwr :

$\omega=-i\frac{r}{\chi}k^{2}\qquad\chi=\frac{d\rho}{d\mu}$ (VI.24)

where $\chi$ is the susceptibility of the 1-form charge.

This formalism is pleasing because it uses principles only of global
symmetries and effective theory, and thus can be systematically improved and
(in principle) applied also to situations where the normal textbook
description of MHD breaks down, for example in situations of strong
electromagnetic correlations. In principle such an approach might have real-
world applications. The close connection to the global symmetry structure also
allows the technology of holography to be brought to bear, an approach which
has been very fruitful in understanding the structure of conventional
hydrodynamics (see e.g. Son:2007vk  for a review). A sampling of work in
developing the formalism and potential applications of such an approach
includes Armas:2023tyx ; Das:2022fho ; Davison:2022vqh ; Vardhan:2022wxz ;
Armas:2022wvb ; Poovuttikul:2021fdi ; Armas:2019sbe ; Gralla:2018kif ;
Glorioso:2018kcp ; Grozdanov:2018fic ; Grozdanov:2018ewh ; Grozdanov:2017kyl .
One concrete application is to the understanding of real-time lattice
experiments, where lattice effects mean that the effective electromagnetic
coupling $e$ can be of order $1$. See e.g. Das:2023nwl  for a recent numerical
investigation which verified the dispersion relation (VI.24), and where this
EFT approach allowed for an understanding of previous unexplained lattice
results.

Conclusions: In these lectures we had a whirlwind tour of several topics in
generalized global symmetries. We have made no attempt at being complete and
have left out many fascinating topics. A partial list of such omissions
includes:

  * •

Our discussion used very little mathematical formalism and thus did not at all
address the construction of representation theory for non-invertible and
higher-form global symmetries, a program which is currently undergoing rapid
progress (see e.g. Bhardwaj:2022yxj ; Bartsch:2023pzl ; Bartsch:2023wvv ;
Bhardwaj:2023wzd ; Bhardwaj:2023ayw ).

  * •

We only briefly discussed ’t Hooft anomalies for conventional symmetries as a
gateway to ABJ anomalies and their interpretation in terms of non-investible
symmetries. There is clearly much more to say here: a review of anomalies in
the conventional 0-form case can be found in Harvey:2005it . Higher form
symmetries can themselves have anomalies; indeed the electric and magnetic
1-form symmetries in free QED have a mixed anomaly, and a similar anomaly is
present in any phase that spontaneously breaks a continuous $U(1)$ 0-form
symmetry Delacretaz:2019brr . Anomalies in higher-form symmetries can be used
fruitfully to constrain the phase structure of gauge theories, see e.g.
Gaiotto:2017yup ; Gaiotto:2017tne ; Gomis:2017ixy  for foundational early
work. The formalism is reviewed in Bhardwaj:2023kri  and a review of
applications can be found in Brennan:2023mmt .

  * •

Higher groups provide a mechanism by which symmetries of different forms can
mix with each other and fuse into a larger structure; they are characterized
by structure constants which are invariant under RG and can also be used to
understand phases quantum field theory. Foundational work includes
Bhardwaj:2017xup ; Benini:2018reh ; Cordova:2018cvg , and developments are
reviewed in Bhardwaj:2023kri .

Nevertheless, a concrete lesson is that thinking about global symmetries as
defining topological objects results in many insights that are both profound
and useful. A more philosophical lesson is that many interesting systems in
nature involve the existence of extended objects (magnetic field lines, flux
tubes, domain walls, etc.), and the community is currently in the process of
developing a formalism that allows us to reformulate quantum field theory in
terms of the dynamics of these objects, rather than in terms of local fields
(such as gauge fields) who represent extended objects indirectly (through
gauge redundancies).

It remains to be seen where this line of reasoning will lead. It is left as a
final exercise for the readers of these notes to figure this out.

Exercise: Figure this out.

Acknowledgments: I want to thank the students and organizers of the TPI School
at Jena for their enthusiastic participation and insightful questions. I am
also lucky to have many wonderful collaborators and friends who have taught me
most of the things I’ve discussed here. I want to particularly thank John
McGreevy for many delightful conversations about The Correctness of Landau.
I’m also very grateful to Aleksey Cherman, Arpit Das, Iñaki Garcia Etxebarria,
Adrien Florio, Saso Grozdanov, Diego Hofman, Theo Jacobson, Napat Poovuttikul,
Tin Sulejmanpasic and David Tong. I am supported in part by the STFC under
grant number ST/T000708/1.

## References

  * (1) D. Gaiotto, A. Kapustin, N. Seiberg, and B. Willett, “Generalized Global Symmetries,” JHEP 02 (2015) 172, arXiv:1412.5148 [hep-th].
  * (2) C. Cordova, T. T. Dumitrescu, K. Intriligator, and S.-H. Shao, “Snowmass White Paper: Generalized Symmetries in Quantum Field Theory and Beyond,” in Snowmass 2021.  5, 2022.  arXiv:2205.09545 [hep-th].
  * (3) T. D. Brennan and S. Hong, “Introduction to Generalized Global Symmetries in QFT and Particle Physics,” arXiv:2306.00912 [hep-ph].
  * (4) P. R. S. Gomes, “An introduction to higher-form symmetries,” SciPost Phys. Lect. Notes 74 (2023) 1, arXiv:2303.01817 [hep-th].
  * (5) S.-H. Shao, “What’s Done Cannot Be Undone: TASI Lectures on Non-Invertible Symmetries,” arXiv:2308.00747 [hep-th].
  * (6) S. Schafer-Nameki, “ICTP lectures on (non-)invertible generalized symmetries,” Phys. Rept. 1063 (2024) 1–55, arXiv:2305.18296 [hep-th].
  * (7) L. Bhardwaj, L. E. Bottini, L. Fraser-Taliente, L. Gladden, D. S. W. Gould, A. Platschorre, and H. Tillim, “Lectures on generalized symmetries,” Phys. Rept. 1051 (2024) 1–87, arXiv:2307.07547 [hep-th].
  * (8) J. McGreevy, “Generalized Symmetries in Condensed Matter,” Ann. Rev. Condensed Matter Phys. 14 (2023) 57–82, arXiv:2204.03045 [cond-mat.str-el].
  * (9) M. E. Peskin, “Mandelstam ’t Hooft Duality in Abelian Lattice Models,” Annals Phys. 113 (1978) 122.
  * (10) C. Dasgupta and B. Halperin, “Phase transition in a lattice model of superconductivity,” Physical Review Letters 47 no. 21, (1981) 1556.
  * (11) D. Tong, “Gauge theory,” Lecture notes, DAMTP Cambridge 10 (2018) 8.
  * (12) J. M. Maldacena, “The Large N limit of superconformal field theories and supergravity,” Int. J. Theor. Phys. 38 (1999) 1113–1133, arXiv:hep-th/9711200 [hep-th]. [Adv. Theor. Math. Phys.2,231(1998)].
  * (13) W. Ketterle, “Experimental studies of bose-einstein condensation,” Phys. Today 52 no. 12, (1999) 30–35.
  * (14) K. Hinterbichler, D. M. Hofman, A. Joyce, and G. Mathys, “Gravity as a gapless phase and biform symmetries,” JHEP 02 (2023) 151, arXiv:2205.12272 [hep-th].
  * (15) V. Benedetti, H. Casini, and J. M. Magan, “Generalized symmetries of the graviton,” JHEP 05 (2022) 045, arXiv:2111.12089 [hep-th].
  * (16) C. Cheung, M. Derda, J.-H. Kim, V. Nevoa, I. Rothstein, and N. Shah, “Generalized Symmetry in Dynamical Gravity,” arXiv:2403.01837 [hep-th].
  * (17) D. Aloni, E. García-Valdecasas, M. Reece, and M. Suzuki, “Spontaneously Broken $(-1)$-Form U(1) Symmetries,” arXiv:2402.00117 [hep-th].
  * (18) N. Iqbal and J. McGreevy, “Mean string field theory: Landau-Ginzburg theory for 1-form symmetries,” SciPost Phys. 13 (2022) 114, arXiv:2106.12610 [hep-th].
  * (19) A. Kapustin, “Wilson-’t Hooft operators in four-dimensional gauge theories and S-duality,” Phys. Rev. D74 (2006) 025005, arXiv:hep-th/0501015 [hep-th].
  * (20) A. Kovner and B. Rosenstein, “New look at QED in four-dimensions: The Photon as a Goldstone boson and the topological interpretation of electric charge,” Phys. Rev. D 49 (1994) 5571–5581, arXiv:hep-th/9210154.
  * (21) J. Goldstone, A. Salam, and S. Weinberg, “Broken Symmetries,” Phys. Rev. 127 (1962) 965–970.
  * (22) D. M. Hofman and N. Iqbal, “Goldstone modes and photonization for higher form symmetries,” SciPost Phys. 6 no. 1, (2019) 006, arXiv:1802.09512 [hep-th].
  * (23) E. Lake, “Higher-form symmetries and spontaneous symmetry breaking,” arXiv:1802.07747 [hep-th].
  * (24) Y. Hidaka, Y. Hirono, and R. Yokokura, “Counting Nambu-Goldstone Modes of Higher-Form Global Symmetries,” Phys. Rev. Lett. 126 no. 7, (2021) 071601, arXiv:2007.15901 [hep-th].
  * (25) O. Aharony, N. Seiberg, and Y. Tachikawa, “Reading between the lines of four-dimensional gauge theories,” JHEP 08 (2013) 115, arXiv:1305.0318 [hep-th].
  * (26) D. Tong, “Line Operators in the Standard Model,” JHEP 07 (2017) 104, arXiv:1705.01853 [hep-th].
  * (27) S. Gukov and E. Witten, “Gauge Theory, Ramification, And The Geometric Langlands Program,” arXiv:hep-th/0612073.
  * (28) S. Gukov and E. Witten, “Rigid Surface Operators,” Adv. Theor. Math. Phys. 14 no. 1, (2010) 87–178, arXiv:0804.1561 [hep-th].
  * (29) B. Heidenreich, J. McNamara, M. Montero, M. Reece, T. Rudelius, and I. Valenzuela, “Non-invertible global symmetries and completeness of the spectrum,” JHEP 09 (2021) 203, arXiv:2104.07036 [hep-th].
  * (30) J. Polchinski, “Monopoles, duality, and string theory,” Int. J. Mod. Phys. A 19S1 (2004) 145–156, arXiv:hep-th/0304042.
  * (31) A. M. Polyakov, “Compact Gauge Fields and the Infrared Catastrophe,” Phys. Lett. B 59 (1975) 82–84.
  * (32) V. Berezinskii, “Destruction of long-range order in one-dimensional and two-dimensional systems having a continuous symmetry group i. classical systems,” Sov. Phys. JETP 32 no. 3, (1971) 493–500.
  * (33) J. M. Kosterlitz and D. J. Thouless, “Ordering, metastability and phase transitions in two-dimensional systems,” in Basic Notions Of Condensed Matter Physics, pp. 493–515.  CRC Press, 2018.
  * (34) T. Sulejmanpasic and C. Gattringer, “Abelian gauge theories on the lattice: $\theta$-Terms and compact gauge theory with(out) monopoles,” Nucl. Phys. B 943 (2019) 114616, arXiv:1901.02637 [hep-lat].
  * (35) M. Nguyen, Y. Tanizaki, and M. Ünsal, “Noninvertible 1-form symmetry and Casimir scaling in 2D Yang-Mills theory,” Phys. Rev. D 104 no. 6, (2021) 065003, arXiv:2104.01824 [hep-th].
  * (36) F. J. Wegner, “Duality in generalized ising models and phase transitions without local order parameters,” Journal of Mathematical Physics 12 no. 10, (1971) 2259–2272. http://scitation.aip.org/content/aip/journal/jmp/12/10/10.1063/1.1665530.
  * (37) J. Xu, A. M. Ferrenberg, and D. P. Landau, “92 Years of the Ising Model: A High Resolution Monte Carlo Study,” J. Phys. Conf. Ser. 1012 no. 1, (2018) 012002.
  * (38) S. El-Showk, M. F. Paulos, D. Poland, S. Rychkov, D. Simmons-Duffin, and A. Vichi, “Solving the 3D Ising Model with the Conformal Bootstrap,” Phys. Rev. D 86 (2012) 025022, arXiv:1203.6064 [hep-th].
  * (39) D. Poland, S. Rychkov, and A. Vichi, “The Conformal Bootstrap: Theory, Numerical Techniques, and Applications,” Rev. Mod. Phys. 91 (2019) 015002, arXiv:1805.04405 [hep-th].
  * (40) A. M. Somoza, P. Serna, and A. Nahum, “Self-Dual Criticality in Three-Dimensional Z2 Gauge Theory with Matter,” Phys. Rev. X 11 no. 4, (2021) 041008, arXiv:2012.15845 [cond-mat.stat-mech].
  * (41) E. Fradkin and S. H. Shenker, “Phase diagrams of lattice gauge theories with higgs fields,” Phys. Rev. D 19 (Jun, 1979) 3682–3697. https://link.aps.org/doi/10.1103/PhysRevD.19.3682.
  * (42) T. T. Dumitrescu and P.-S. Hsin, “Higgs-Confinement Transitions in QCD from Symmetry Protected Topological Phases,” arXiv:2312.16898 [hep-th].
  * (43) A. Cherman, T. Jacobson, S. Sen, and L. G. Yaffe, “Line operators, vortex statistics, and Higgs versus confinement dynamics,” JHEP 06 (2024) 200, arXiv:2401.17489 [hep-th].
  * (44) G. T. Horowitz, “Exactly soluble diffeomorphism invariant theories,” Communications in Mathematical Physics 125 (1989) 417–437.
  * (45) T. Banks and N. Seiberg, “Symmetries and Strings in Field Theory and Gravity,” Phys. Rev. D83 (2011) 084019, arXiv:1011.5120 [hep-th].
  * (46) J. M. Maldacena, G. W. Moore, and N. Seiberg, “D-brane charges in five-brane backgrounds,” JHEP 10 (2001) 005, arXiv:hep-th/0108152 [hep-th].
  * (47) E. Witten, “SL(2,Z) action on three-dimensional conformal field theories with Abelian symmetry,” in From Fields to Strings: Circumnavigating Theoretical Physics: A Conference in Tribute to Ian Kogan, pp. 1173–1200.  7, 2003.  arXiv:hep-th/0307041.
  * (48) X.-G. Wen, “Topological orders in rigid states,” International Journal of Modern Physics B 4 no. 02, (1990) 239–271.
  * (49) X.-G. Wen, Quantum field theory of many-body systems: From the origin of sound to an origin of light and electrons.  Oxford university press, 2004.
  * (50) A. Y. Kitaev, “Fault tolerant quantum computation by anyons,” Annals Phys. 303 (2003) 2–30, arXiv:quant-ph/9707021.
  * (51) A. Kitaev and J. Preskill, “Topological entanglement entropy,” Phys. Rev. Lett. 96 (2006) 110404, arXiv:hep-th/0510092.
  * (52) M. Levin and X.-G. Wen, “Detecting topological order in a ground state wave function,” Phys. Rev. Lett. 96 (Mar, 2006) 110405. https://link.aps.org/doi/10.1103/PhysRevLett.96.110405.
  * (53) T. Grover, A. M. Turner, and A. Vishwanath, “Entanglement Entropy of Gapped Phases and Topological Order in Three dimensions,” Phys. Rev. B 84 (2011) 195120, arXiv:1108.4038 [cond-mat.str-el].
  * (54) J. R. Fliss and S. Vitouladitis, “Entanglement in BF theory II: Edge-modes,” arXiv:2310.18391 [hep-th].
  * (55) D. Tong, “Lectures on the Quantum Hall Effect,”  6, 2016.  arXiv:1606.06687 [hep-th].
  * (56) E. Verlinde, “Fusion rules and modular transformations in 2d conformal field theory,” Nuclear Physics B 300 (1988) 360–376.
  * (57) J. Frohlich, J. Fuchs, I. Runkel, and C. Schweigert, “Kramers-Wannier duality from conformal defects,” Phys. Rev. Lett. 93 (2004) 070601, arXiv:cond-mat/0404051.
  * (58) C.-M. Chang, Y.-H. Lin, S.-H. Shao, Y. Wang, and X. Yin, “Topological Defect Lines and Renormalization Group Flows in Two Dimensions,” JHEP 01 (2019) 026, arXiv:1802.04445 [hep-th].
  * (59) S. L. Adler, “Axial vector vertex in spinor electrodynamics,” Phys. Rev. 177 (1969) 2426–2438.
  * (60) J. S. Bell and R. Jackiw, “A PCAC puzzle: $\pi^{0}\to\gamma\gamma$ in the $\sigma$ model,” Nuovo Cim. A 60 (1969) 47–61.
  * (61) Y. Choi, H. T. Lam, and S.-H. Shao, “Noninvertible Global Symmetries in the Standard Model,” Phys. Rev. Lett. 129 no. 16, (2022) 161601, arXiv:2205.05086 [hep-th].
  * (62) C. Cordova and K. Ohmori, “Noninvertible Chiral Symmetry and Exponential Hierarchies,” Phys. Rev. X 13 no. 1, (2023) 011034, arXiv:2205.06243 [hep-th].
  * (63) K. Roumpedakis, S. Seifnashri, and S.-H. Shao, “Higher Gauging and Non-invertible Condensation Defects,” Commun. Math. Phys. 401 no. 3, (2023) 3043–3107, arXiv:2204.02407 [hep-th].
  * (64) Y. Choi, C. Cordova, P.-S. Hsin, H. T. Lam, and S.-H. Shao, “Non-invertible Condensation, Duality, and Triality Defects in 3+1 Dimensions,” Commun. Math. Phys. 402 no. 1, (2023) 489–542, arXiv:2204.09025 [hep-th].
  * (65) A. Cherman, T. Jacobson, and M. Neuzil, “1-form symmetry versus large N QCD,” JHEP 02 (2023) 192, arXiv:2209.00027 [hep-th].
  * (66) E. Witten, “Dyons of Charge e theta/2 pi,” Phys. Lett. B 86 (1979) 283–287.
  * (67) I. n. García Etxebarria and N. Iqbal, “A Goldstone theorem for continuous non-invertible symmetries,” JHEP 09 (2023) 145, arXiv:2211.09570 [hep-th].
  * (68) A. Karasik, “On anomalies and gauging of U(1) non-invertible symmetries in 4d QED,” SciPost Phys. 15 no. 1, (2023) 002, arXiv:2211.05802 [hep-th].
  * (69) J. I. Kapusta and C. Gale, Finite-temperature field theory: Principles and applications.  Cambridge university press, 2007.
  * (70) M. Le Bellac, Thermal field theory.  Cambridge university press, 2000.
  * (71) A. M. Polyakov, “Quark Confinement and Topology of Gauge Groups,” Nucl. Phys. B 120 (1977) 429–458.
  * (72) A. Das, A. Florio, N. Iqbal, and N. Poovuttikul, “Higher-form symmetry and chiral transport in real-time lattice $U(1)$ gauge theory,” arXiv:2309.14438 [hep-th].
  * (73) M. Kiometzis, H. Kleinert, and A. M. Schakel, “Critical exponents of the superconducting phase transition,” Physical review letters 73 no. 14, (1994) 1975.
  * (74) I. F. Herbut and Z. Tešanović, “Critical fluctuations in superconductors and the magnetic field penetration depth,” Physical Review Letters 76 no. 24, (1996) 4588.
  * (75) J. Armas and A. Jain, “One-form superfluids & magnetohydrodynamics,” arXiv:1811.04913 [hep-th].
  * (76) J. Armas and A. Jain, “Magnetohydrodynamics as superfluidity,” Phys. Rev. Lett. 122 no. 14, (2019) 141603, arXiv:1808.01939 [hep-th].
  * (77) N. Iqbal and N. Poovuttikul, “2-group global symmetries, hydrodynamics and holography,” SciPost Phys. 15 no. 2, (2023) 063, arXiv:2010.00320 [hep-th].
  * (78) A. Das, Generalised symmetries, anomalous magnetohydrodynamics and holography.  PhD thesis, Durham U., 8, 2023.
  * (79) L. Landau and E. Lifshitz, “Fluid mechanics. 1987,” Course of Theoretical Physics (1987) .
  * (80) P. Kovtun, “Lectures on hydrodynamic fluctuations in relativistic theories,” J. Phys. A 45 (2012) 473001, arXiv:1205.5040 [hep-th].
  * (81) P. M. Bellan, Fundamentals of plasma physics.  Cambridge university press, 2008.
  * (82) J. Hernandez and P. Kovtun, “Relativistic magnetohydrodynamics,” JHEP 05 (2017) 001, arXiv:1703.08757 [hep-th].
  * (83) S. Grozdanov, D. M. Hofman, and N. Iqbal, “Generalized global symmetries and dissipative magnetohydrodynamics,” Phys. Rev. D95 no. 9, (2017) 096003, arXiv:1610.07392 [hep-th].
  * (84) D. Schubring, “Dissipative String Fluids,” Phys. Rev. D 91 no. 4, (2015) 043518, arXiv:1412.3135 [hep-th].
  * (85) D. M. Hofman and N. Iqbal, “Generalized global symmetries and holography,” SciPost Phys. 4 (2018) 005, arXiv:1707.08577 [hep-th].
  * (86) D. T. Son and A. O. Starinets, “Viscosity, Black Holes, and Quantum Field Theory,” Ann. Rev. Nucl. Part. Sci. 57 (2007) 95–118, arXiv:0704.0240 [hep-th].
  * (87) J. Armas and A. Jain, “Approximate higher-form symmetries, topological defects, and dynamical phase transitions,” Phys. Rev. D 109 no. 4, (2024) 045019, arXiv:2301.09628 [hep-th].
  * (88) A. Das, N. Iqbal, and N. Poovuttikul, “Towards an effective action for chiral magnetohydrodynamics,” arXiv:2212.09787 [hep-th].
  * (89) R. A. Davison, B. Goutéraux, and E. Mefford, “Zero sound and higher-form symmetries in compressible holographic phases,” JHEP 12 (2023) 040, arXiv:2210.14802 [hep-th].
  * (90) S. Vardhan, S. Grozdanov, S. Leutheusser, and H. Liu, “A new formulation of strong-field magnetohydrodynamics for neutron stars,” arXiv:2207.01636 [astro-ph.HE].
  * (91) J. Armas and F. Camilloni, “A stable and causal model of magnetohydrodynamics,” JCAP 10 (2022) 039, arXiv:2201.06847 [hep-th].
  * (92) N. Poovuttikul and A. Rajagopal, “Operator lifetime and the force-free electrodynamic limit of magnetised holographic plasma,” JHEP 09 (2021) 091, arXiv:2101.12540 [hep-th].
  * (93) J. Armas and A. Jain, “Viscoelastic hydrodynamics and holography,” JHEP 01 (2020) 126, arXiv:1908.01175 [hep-th].
  * (94) S. E. Gralla and N. Iqbal, “Effective Field Theory of Force-Free Electrodynamics,” Phys. Rev. D99 no. 10, (2019) 105004, arXiv:1811.07438 [hep-th].
  * (95) P. Glorioso and D. T. Son, “Effective field theory of magnetohydrodynamics from generalized global symmetries,” arXiv:1811.04879 [hep-th].
  * (96) S. Grozdanov, A. Lucas, and N. Poovuttikul, “Holography and hydrodynamics with weakly broken symmetries,” arXiv:1810.10016 [hep-th].
  * (97) S. Grozdanov and N. Poovuttikul, “Generalized global symmetries in states with dynamical defects: The case of the transverse sound in field theory and holography,” Phys. Rev. D97 no. 10, (2018) 106005, arXiv:1801.03199 [hep-th].
  * (98) S. Grozdanov and N. Poovuttikul, “Generalised global symmetries in holography: magnetohydrodynamic waves in a strongly interacting plasma,” JHEP 04 (2019) 141, arXiv:1707.04182 [hep-th].
  * (99) L. Bhardwaj, L. E. Bottini, S. Schafer-Nameki, and A. Tiwari, “Non-invertible higher-categorical symmetries,” SciPost Phys. 14 no. 1, (2023) 007, arXiv:2204.06564 [hep-th].
  * (100) T. Bartsch, M. Bullimore, and A. Grigoletto, “Higher representations for extended operators,” arXiv:2304.03789 [hep-th].
  * (101) T. Bartsch, M. Bullimore, and A. Grigoletto, “Representation theory for categorical symmetries,” arXiv:2305.17165 [hep-th].
  * (102) L. Bhardwaj and S. Schafer-Nameki, “Generalized charges, part I: Invertible symmetries and higher representations,” SciPost Phys. 16 no. 4, (2024) 093, arXiv:2304.02660 [hep-th].
  * (103) L. Bhardwaj and S. Schafer-Nameki, “Generalized Charges, Part II: Non-Invertible Symmetries and the Symmetry TFT,” arXiv:2305.17159 [hep-th].
  * (104) J. A. Harvey, “TASI 2003 lectures on anomalies,”  9, 2005.  arXiv:hep-th/0509097.
  * (105) L. V. Delacrétaz, D. M. Hofman, and G. Mathys, “Superfluids as Higher-form Anomalies,” SciPost Phys. 8 (2020) 047, arXiv:1908.06977 [hep-th].
  * (106) D. Gaiotto, A. Kapustin, Z. Komargodski, and N. Seiberg, “Theta, Time Reversal, and Temperature,” JHEP 05 (2017) 091, arXiv:1703.00501 [hep-th].
  * (107) D. Gaiotto, Z. Komargodski, and N. Seiberg, “Time-reversal breaking in QCD4, walls, and dualities in 2 + 1 dimensions,” JHEP 01 (2018) 110, arXiv:1708.06806 [hep-th].
  * (108) J. Gomis, Z. Komargodski, and N. Seiberg, “Phases Of Adjoint QCD3 And Dualities,” SciPost Phys. 5 no. 1, (2018) 007, arXiv:1710.03258 [hep-th].
  * (109) L. Bhardwaj and Y. Tachikawa, “On finite symmetries and their gauging in two dimensions,” JHEP 03 (2018) 189, arXiv:1704.02330 [hep-th].
  * (110) F. Benini, C. Córdova, and P.-S. Hsin, “On 2-Group Global Symmetries and their Anomalies,” JHEP 03 (2019) 118, arXiv:1803.09336 [hep-th].
  * (111) C. Cordova, T. T. Dumitrescu, and K. Intriligator, “Exploring 2-Group Global Symmetries,” arXiv:1802.04790 [hep-th].