Porosities of the sets of attractors

By

Paweł Klinga and Adam Kwela

###### Abstract

This paper is another attempt to measure the difference between the family
$A[0,1]$ of attractors for iterated function systems acting on $[0,1]$ and a
broader family, the set $A_{w}[0,1]$ of attractors for weak iterated function
systems acting on $[0,1]$.

It is known that both $A[0,1]$ and $A_{w}[0,1]$ are meager subsets of the
hyperspace $K([0,1])$ (of all compact subsets of $[0,1]$ equipped in the
Hausdorff metric). Actually, $A[0,1]$ is even $\sigma$-lower porous while the
question about $\sigma$-lower porosity of $A_{w}[0,1]$ is still open.

We prove that $A[0,1]$ is not $\sigma$-strongly porous in $K([0,1])$.
Moreover, we show that $A_{w}[0,1]\setminus A[0,1]$ is dense in $K([0,1])$.
††2010 Mathematics Subject Classification: Primary: 28A80. Secondary: 26A18.
††Key words and phrases: attractors, iterated function systems, weak iterated
function systems, Banach fractals, upper porosity, $\sigma$-upper porosity,
lower porosity, $\sigma$-lower porosity, strong porosity, $\sigma$-strong
porosity

##  1 Introduction

We are mainly interested in two families of compact subsets of the space
$[0,1]^{d}$, where $d\in\mathbb{N}$:

  * •

the set $A[0,1]^{d}$ of attractors for iterated function systems acting on
$[0,1]^{d}$,

  * •

the set $A_{w}[0,1]^{d}$ of attractors for weak iterated function systems
acting on $[0,1]^{d}$.

We will consider both $A[0,1]^{d}$ and $A_{w}[0,1]^{d}$ as subsets of the
hyperspace $K([0,1]^{d})$ of all compact subsets of $[0,1]^{d}$ equipped in
the Hausdorff metric.

This approach is present in literature. In [1, Theorem 3.9] (see also [2]) E.
D’Aniello and T.H. Steele proved that the family $A[0,1]^{d}$ is a meager
subset of $K([0,1]^{d})$, for every $d\in\mathbb{N}$ (see also [12, Theorem 5]
where this fact is proved, however the statement of [12, Theorem 5] is false).
In [3, Theorem 4.1] and [8, Theorem 4.1], independently, it is shown that the
same holds for the family $A_{w}[0,1]^{d}$. Moreover, the result about
attractors for classical iterated function systems has been strengthened in
[8, Theorem 3.1] by showing that $A[0,1]^{d}$ is even smaller – it is a
$\sigma$-lower porous subset of $K([0,1]^{d})$. The question about
$\sigma$-lower porosity of $A_{w}[0,1]^{d}$ is still open (see [8, Question
4.4]).

There are still many other open problems concerning comparison of families
$A[0,1]^{d}$ and $A_{w}[0,1]^{d}$. For instance, $A[0,1]^{d}$ is $F_{\sigma}$,
while the exact descriptive complexity of $A_{w}[0,1]^{d}$ is unknown
(recently in [6] it only has been shown that it is not $F_{\sigma\delta}$).
This paper is another attempt to measure the difference between the families
$A[0,1]^{d}$ and $A_{w}[0,1]^{d}$.

In Section 2 we provide basic definitions needed in our considerations.
Section 3 includes some basic preliminary results. For instance, we show that
the set $A_{w}[0,1]^{d}\setminus A[0,1]^{d}$ cannot be nowhere dense itself as
it is dense in $K([0,1]^{d})$, and argue that $\sigma$-strong porosity is an
appropriate notion for our purposes (as the collection of all finite subsets
of $[0,1]^{d}$ is $\sigma$-strongly porous). In Section 4 we prove our main
result, that $A[0,1]$ is not $\sigma$-strongly porous.

##  2 Basic notions and definitions

###  2.1 Hyperspace of compact subsets

Let $(X,d)$ be a compact metric space. By $K(X)$ we denote the family of all
compact subsets of $X$. We will consider $K(X)$ as a topological space
equipped with the Hausdorff metric $d_{H}$, which is given by

$d_{H}(A,B)=\min\left\\{r\geq 0:A\subseteq\widetilde{B}_{r}\text{ and
}B\subseteq\widetilde{A}_{r}\right\\},$

where for $C\in K(X)$ and $r\geq 0$ we write

$\widetilde{C}_{r}=\bigcup_{c\in C}\left\\{x\in X:d(x,c)\leq r\right\\}.$

For $C\in K(X)$ and $r>0$ by $B_{H}(C,r)$ we will denote the ball in $K(X)$ of
radius $r$ centered at $C$. Throughout the paper we will assume that
$X=[0,1]^{d}$. For convenience in this case we will write $K[0,1]^{d}$ instead
of $K([0,1]^{d})$. Moreover, if $d=1$ then we will write $K[0,1]$ instead of
$K[0,1]^{1}$.

For $r\in\mathbb{R}$, $x=(x_{1},\ldots,x_{d})\in[0,1]^{d}$ and $A\in
K[0,1]^{d}$ we will write

$r\cdot A=\\{(ry_{1},\ldots,ry_{d}):(y_{1},\ldots,y_{d})\in A\\},$
$x+A=\\{(x_{1}+y_{1},\ldots,x_{d}+y_{d}):(y_{1},\ldots,y_{d})\in A\\}.$

Note that both $rA$ and $x+A$ are compact subsets of $\mathbb{R}^{d}$. Thus,
$rA\in K[0,1]^{d}$ and $x+A\in K[0,1]^{d}$ provided that
$rA\subseteq[0,1]^{d}$ and $x+A\subseteq[0,1]^{d}$.

###  2.2 Iterated function systems

By contraction on $X$ we understand a function $f\colon X\to X$ such that:

$\exists_{L\in[0,1)}\ \forall_{x,y\in X}\ d(f(x),f(y))\leqslant L\cdot
d(x,y).$

The smallest $L$ for which this inequality holds is called a Lipschitz
constant of $f$ and denoted $Lip(f)$.

A function $f\colon X\to X$ is a weak contraction if for every pair of
distinct points $x,y\in X$ it is true that $d(f(x),f(y))<d(x,y)$.

Every finite collection of contractions on $X$ will be called an iterated
function system (IFS, in short) acting on $X$. Similarly, a finite collection
of weak contractions on $X$ will be called a weak iterated function system
(wIFS, in short) acting on $X$.

###  2.3 Attractors

Let $\\{s_{1},\dots,s_{k}\\}$ be an IFS acting on $X$. By $\mathcal{S}\colon
K(X)\to K(X)$ we denote the Hutchinson operator for $\\{s_{1},\dots,s_{k}\\}$,
i.e.

$\mathcal{S}(A)=\bigcup_{i=1}^{k}s_{i}(A).$

If $(X,d)$ is complete, then so is $(K(X),d_{H})$. Therefore, by applying the
Banach fixed-point theorem, the equation $\mathcal{S}(A)=A$ has a unique
compact solution. Such set is called the attractor for the iterated function
system $\\{s_{1},\dots,s_{k}\\}$.

Analogously, for a wIFS $\\{s_{1},\dots,s_{k}\\}$ acting on $X$, a compact set
$A\in K(X)$ satisfying $\bigcup_{i=1}^{k}s_{i}(A)=A$ is called a weak IFS
attractor for the system $\\{s_{1},\dots,s_{k}\\}$. It is shown in [4] that
for every weak iterated function system there exists a weak IFS attractor
provided that $X$ is a compact space.

The form of the Hutchinson operator imposes that attractors are self-similar
(at least in some sense), so they are often used to describe fractals.
Clearly, an IFS attractor is a weak IFS attractor. However, the reverse
inclusion is not true by [11].

By $A[0,1]^{d}$ and $A_{w}[0,1]^{d}$ we will denote the sets of all attractors
for IFS and wIFS, respectively, acting on $[0,1]^{d}$. If $d=1$ then we will
write $A[0,1]$ and $A_{w}[0,1]$ instead of $A[0,1]^{1}$ and $A_{w}[0,1]^{1}$,
respectively.

###  2.4 Porosities

Fix a metric space $(X,d)$ and denote by $B(z,\delta)$ a ball in $(X,d)$ of
radius $\delta>0$ centered at $z\in X$. For any subset $A$ of $X$ and any
point $x\in X$ we define

$\overline{p}(A,x)=\limsup_{R\to 0^{+}}\frac{\sup\\{r\geq 0:B(y,r)\subseteq
B(x,R)\setminus A\text{ for some }y\in X\\}}{R},$
$\underline{p}(A,x)=\liminf_{R\to 0^{+}}\frac{\sup\\{r\geq 0:B(y,r)\subseteq
B(x,R)\setminus A\text{ for some }y\in X\\}}{R}.$

Observe that $\overline{p}(A,x)=1$ whenever $x\notin\overline{A}$ and
$0\leq\overline{p}(A,x)\leq\frac{1}{2}$ in the opposite case (see [10, Remark
1.2.(i)]).

We say that $A$ is:

  * •

lower porous if $\underline{p}(A,x)>0$ for every $x\in X$,

  * •

upper porous if $\overline{p}(A,x)>0$ for every $x\in X$,

  * •

strongly porous if $\overline{p}(A,x)\geq\frac{1}{2}$ for every $x\in X$
(equivalently, $\overline{p}(A,x)=\frac{1}{2}$ for every $x\in\overline{X}$),

  * •

$\sigma$-lower porous ($\sigma$-upper porous, $\sigma$-strongly porous) if it
is a countable union of lower porous (upper porous, strongly porous) sets.

Clearly, each lower porous set as well as each strongly porous set is upper
porous. What is more, since the condition defining upper porosity is a
strengthening of nowhere density, each $\sigma$-upper porous set is meager.

In this known that on the real line there are upper porous sets that are not
$\sigma$-lower porous. In fact, there is even a closed strongly porous set
which is not $\sigma$-lower porous ([9, Corollary 4.1.(ii)]). On the other
hand, there exist countable upper porous sets which are not strongly porous
(see [10, Remark 1.2(iv) and Example 4.6]).

We will use a different, but equivalent (thanks to [10, Remark 1.2.(ii)]),
definition of strong porosity: $A$ is strongly porous, if for each $x\in X$
there are two sequences $(x_{n})\subseteq X$ and $(r_{n})\subseteq(0,+\infty)$
such that:

  * •

$\lim_{n}x_{n}=x$,

  * •

$\lim_{n}\frac{r_{n}}{d(x,x_{n})}=1$,

  * •

$B(x_{n},r_{n})\cap A=\emptyset$ for all $n$.

In [8, Theorem 3.1] it is shown that the set $A[0,1]^{d}$ is $\sigma$-lower
porous. The question whether $A_{w}[0,1]^{d}$ is $\sigma$-lower porous is
still open (see [8, Question 4.4]).

For more on porosities see for instance [9], [10] or [13].

##  3 Preliminary results

We start this section with a result enabling to transfer some results about
attractors in $K[0,1]$ to arbitrary dimension $d\in\mathbb{N}$. However the
proof can be found in [7], we repeat it for the sake of completeness.

######  Lemma 3.1 ([7, Lemma 2.2]).

For each $d\in\mathbb{N}$ the following hold:

  * •

$A\in A[0,1]$ if and only if $A\times\\{0\\}^{d-1}\in A[0,1]^{d}$,

  * •

$A\in A_{w}[0,1]$ if and only if $A\times\\{0\\}^{d-1}\in A_{w}[0,1]^{d}$.

###### Proof.

Observe that if $f:[0,1]^{d}\to[0,1]^{d}$ is a (weak) contraction then so is
$f\upharpoonright[0,1]\times\\{0\\}^{d-1}$. On the other hand, for each (weak)
contraction $f:[0,1]\to[0,1]$ the map $g:[0,1]^{d}\to[0,1]^{d}$ given by
$g(x_{1},\ldots,x_{d})=(f(x_{1}),0,\ldots,0)$ is a (weak) contraction as well.
∎

Recall that $A[0,1]^{d}$ is dense in $K[0,1]^{d}$, for every $d\in\mathbb{N}$,
as it contains all nonempty finite sets. In particular, $A[0,1]^{d}$ cannot be
itself nowhere dense. The following observation implies that the same is true
for the set $A_{w}[0,1]^{d}\setminus A[0,1]^{d}$.

######  Proposition 3.2.

The set $A_{w}[0,1]^{d}\setminus A[0,1]^{d}$ is dense in $K[0,1]^{d}$, for
every $d\in\mathbb{N}$.

###### Proof.

Before the main part of the proof, we need to perform a construction in
$[0,1]$, which will be needed later. This part is a simple modification of [7,
Proposition 3.1], however we repeat the reasoning for the sake of
completeness.

Let $f:[0,1]\to[0,1]$ be the weak contraction given by $f(x)=x-x^{2}$ for all
$x\in[0,1]$. Define a sequence of intervals by $I_{1}=[0,\frac{1}{2}]$ and
$I_{n+1}=[0,f(\max I_{n})]$. Inductively pick points $x_{n}$ for
$n\in\mathbb{N}$ in such a way that for each $n\in\mathbb{N}$ we have:

  * (a)

$x_{n}\in I_{1}$,

  * (b)

$x_{n+1}<x_{n}$,

  * (c)

$x_{n}-x_{n+1}>x_{n+1}-x_{n+2}$,

  * (d)

$\\{x_{i}:i\in\mathbb{N}\\}\cap(I_{n}\setminus I_{n+1})$ is finite,

  * (e)

$k_{n}>n+n\cdot(\sum_{i=1}^{n-1}k_{i})$, where
$k_{n}=|\\{x_{i}:i\in\mathbb{N}\\}\cap(I_{n}\setminus I_{n+1})|$.

Note that item (d) guarantees that $\lim_{n}x_{n}=0$. Put
$X=\\{0\\}\cup\\{x_{n}:n\in\mathbb{N}\\}$.

Observe that $X\in A_{w}[0,1]$. Indeed, let $g:[0,1]\to[0,1]$ be the weak
contraction such that $g(x_{n})=x_{n+1}$ for all $n\in\mathbb{N}$ (such $g$
exists by items (b) and (c)). As $\lim_{n}\max I_{n}=0$, the unique fixed
point of $g$ is $0$. Thus, if $h:[0,1]\to[0,1]$ is the function constantly
equal to $x_{1}$ then $X=g[X]\cup h[X]$.

We are ready for the main part of the proof. As the family of all finite
nonempty sets is dense in $K[0,1]^{d}$, it suffices to show that for every
finite nonempty set $F\subseteq[0,1]^{d}$ and every $\delta>0$ there is an
attractor in $B_{H}(F,\delta)\cap(A_{w}[0,1]^{d}\setminus A[0,1]^{d})$.

Let $F$ be a finite nonempty subset of $[0,1]^{d}$ and $\delta>0$ be such that
$2\delta<\min\\{d(x,y):x,y\in F,x\neq y\\}$. Fix any $w\in F$ and consider the
set $Y=F\cup(w+\delta\cdot(X\times\\{0\\}^{d-1}))$. Clearly, $Y\in
B_{H}(F,\delta)$. By Lemma 3.1, $w+\delta\cdot(X\times\\{0\\}^{d-1})\in
A_{w}[0,1]^{d}$. Hence, $Y\in A_{w}[0,1]^{d}$ as
$Y\setminus(w+\delta\cdot(X\times\\{0\\}^{d-1}))$ is finite.

To finish the proof, we will show that $Y\notin A[0,1]^{d}$. Assume towards
contradiction that $f_{1},\ldots,f_{k}:[0,1]^{d}\to[0,1]^{d}$ are standard
contractions such that $Y=\bigcup_{i=1}^{k}f_{i}[Y]$. Denote
$X^{\prime}=w+\delta\cdot(X\times\\{0\\}^{d-1})$ and
$I^{\prime}_{n}=w+\delta\cdot(I_{n}\times\\{0\\}^{d-1})$ for all
$n\in\mathbb{N}$. Find $n\in\mathbb{N}$ such that:

  * (i)

$n>k\cdot|F|$ (here $|F|$ denotes the cardinality of the finite set $F$),

  * (ii)

if $f_{i}(w)\neq w$ then $f_{i}[X^{\prime}]\cap X^{\prime}\cap
I^{\prime}_{n}=\emptyset$ (notice that $f_{i}(w)\neq w$ implies
$f_{i}[X^{\prime}]\cap X^{\prime}$ being finite, by $\lim_{n}x_{n}=0$),

  * (iii)

$\max_{i\leq k}Lip(f_{i})\cdot|I_{n}|<|I_{n+1}|$ (recall that for $f=x(1-x)$
and any $L\in(0,1)$ we have $|f(x)-f(0)|=|x|\cdot|1-x|>L|x-0|$ for all
$x\in(0,1-L)$).

We will justify that $X^{\prime}\cap I^{\prime}_{n}\setminus I^{\prime}_{n+1}$
cannot be covered by $\bigcup_{i\leq k}f_{i}[Y]$. If $f_{i}(w)\neq w$ then
$f_{i}[X^{\prime}]\cap X^{\prime}\cap I^{\prime}_{n}=\emptyset$ by item (ii).
If $f_{i}(w)=w$ then item (iii) implies that $f_{i}[I^{\prime}_{n}]\subseteq
I^{\prime}_{n+1}$. Therefore,

$(X^{\prime}\cap I^{\prime}_{n}\setminus I^{\prime}_{n+1})\cap\bigcup_{i\leq
k}f_{i}[Y]=(X^{\prime}\cap I^{\prime}_{n}\setminus
I^{\prime}_{n+1})\cap\bigcup_{i\leq k}f_{i}[F\cup(X^{\prime}\cap
I^{\prime}_{1}\setminus I^{\prime}_{n})].$

Using $|X^{\prime}\cap I^{\prime}_{1}\setminus
I^{\prime}_{n}|=\sum_{i=1}^{n-1}k_{i}$, we see that

$\left|\bigcup_{i\leq k}f_{i}[F\cup(X^{\prime}\cap I^{\prime}_{1}\setminus
I^{\prime}_{n})]\right|\leq k\cdot\left(|F|+\sum_{i=1}^{n-1}k_{i}\right)<$
$<n+n\cdot\sum_{i=1}^{n-1}k_{i}<k_{n}=|X^{\prime}\cap(I^{\prime}_{n}\setminus
I^{\prime}_{n+1})|.$

Thus, the proof is finished. ∎

We end with showing that $\sigma$-strong-porosity is an appropriate notion for
our purposes – settling whether $A[0,1]^{d}$ is $\sigma$-strongly-porous will
require full depth of $A[0,1]^{d}$ (not only the information that all finite
nonempty sets are in $A[0,1]^{d}$).

######  Proposition 3.3.

The collection of all finite nonempty subsets of $[0,1]^{d}$ is
$\sigma$-strongly-porous in $K[0,1]^{d}$, for every $d\in\mathbb{N}$.

###### Proof.

Observe that the family of all finite nonempty subsets of $[0,1]^{d}$ is equal
to $\mathcal{P}(D)\cup\bigcup_{k\in\omega}\mathcal{F}_{k}$, where

$D=\left\\{(x_{1},\ldots,x_{d})\in[0,1]^{d}:\forall_{i\leq d}\ x_{i}=0\text{
or }x_{i}=1\right\\},$
$\mathcal{F}_{k}=\\{F\subseteq[0,1]^{d}:|F|=k,F\setminus D\neq\emptyset\\}.$

Clearly, $\mathcal{P}(D)$ is strongly porous as a finite set. Thus it suffices
to show that $\mathcal{F}_{k}$ is strongly porous for each $k\in\mathbb{N}$.

Fix $k\in\mathbb{N}$, $F\in\mathcal{F}_{k}$ and any $x\in F\setminus D$. Find
three sequences $(y_{n}),(z_{n})\subseteq[0,1]^{d}$ and
$(r_{n})\subseteq(0,1)$ such that:

  * •

$r_{n}=d(y_{n},x)=d(z_{n},x)$ for all $n$,

  * •

$y_{n}\neq z_{n}$ for all $n$,

  * •

$B(y_{n},r_{n})\cap B(z_{n},r_{n})=\emptyset$ for all $n$,

  * •

$\lim_{n}r_{n}=0$.

This is possible as $x\notin D$.

Put $G_{n}=(F\setminus\\{x\\})\cup\\{y_{n},z_{n}\\}$. As
$d_{H}(G_{n},F)=r_{n}\to 0$, the sequence $(G_{n})$ tends to $F$ (in
$K[0,1]^{d}$) and $\lim_{n}\frac{r_{n}}{d_{H}(G_{n},F)}=1$. To conclude the
proof, we need to justify that
$B_{H}(G_{n},r_{n})\cap\mathcal{F}_{k}=\emptyset$.

Let $H\in B_{H}(G_{n},r_{n})$. Then $H$ has to intersect each of the balls
$B(y_{n},r_{n})$, $B(z_{n},r_{n})$ and $B(w,r_{n})$ for $w\in
F\setminus\\{x\\}$. As those balls are pairwise disjoint, $H$ has to have at
least $2+k-1=k+1$ elements. Hence, $H\notin\mathcal{F}_{k}$ and the proof is
finished. ∎

##  4 Attractors for IFSs

######  Theorem 4.1.

The set $A[0,1]$ is not $\sigma$-strongly porous in $K[0,1]$.

###### Proof.

We will prove the following.

$\exists_{E\in A[0,1]}\>\exists_{\varepsilon>0}\>\forall_{Y\in
B_{H}(E,\varepsilon)}\>\>B_{H}(Y,d_{H}(Y,E)\cdot 0.99)\cap
A[0,1]\neq\emptyset.$

From this, the negation of the definition of $\sigma$-strong porosity will
follow.

The idea of the proof is to find a correspondence between $A[0,1]$ and the set
$[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$ of all sequences with all terms
belonging to the interval $[-\frac{3}{2},\frac{3}{2}]$, then observe that in
any decomposition of $[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$ into countably
many pieces, one will be ”large” (i.e. dense in some set
$U\subseteq[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$ open in the product
topology), pick any attractor $E$ corresponding to an element of $U$ and show
that for any $Y\in B_{H}(E,\varepsilon)$ there is an open $V\subseteq U$ such
that the attractor corresponding to any element of $V$ is in
$B_{H}(Y,d_{H}(Y,E)\cdot 0.99)$.

The correspondence will be as follows: we will match
$(x_{n})\in[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$ (i.e. a sequence of reals)
with a Cantor-like attractor. By a Cantor-like attractor we mean an attractor
generated by two contractions: each of them will be ”based” on the following
functions:

$f_{1}(x)=\frac{1}{10}x+\frac{2}{10},\>\>f_{2}(x)=\frac{1}{10}x+\frac{7}{10}.$

Therefore, in each iteration a new image will be a ”tenth” of a size of the
previous image. However, also, every image of an interval can be moved by
$\frac{3}{2}$ of the tenth of the interval to the left or to the right (for
instance, in the case of the first iteration: the left-hand image will be a
subset of $[\frac{1}{20},\frac{9}{20}]$ and the right-hand image will be a
subset of $[\frac{11}{20},\frac{19}{20}]$ – note that the distance between
them is at least $\frac{1}{10}$, so at the end we will still get an attractor
for some IFS). The endpoints of those images will be determined by the
sequence $(x_{n})$. For instance, a sequence constantly equal to 0 will ”code”
exactly the functions $f_{1},f_{2}$. A sequence constantly equal to
$-\frac{3}{2}$ will result in functions

$\frac{1}{10}x+\frac{1}{20},\frac{1}{10}x+\frac{11}{20}$

and a sequence constantly equal to $\frac{3}{2}$ will result in functions

$\frac{1}{10}x+\frac{7}{20},\frac{1}{10}x+\frac{17}{20}.$

And obviously, all the ”in-between” shifts of intervals will be coded by terms
between $-\frac{3}{2}$ and $\frac{3}{2}$. Below we present details.

Let the correspondence between $(x_{n})$ and $(f_{1},f_{2})$ be the following:
for each $i\in\mathbb{N}$ let $x_{i}$ be such that:

  * $x_{1}:\>\>\>\>$

$f_{1}(0)=\frac{2}{10}+\frac{1}{10}\cdot x_{1}$

  * $x_{2}:\>\>\>\>$

$f_{2}(0)=\frac{7}{10}+\frac{1}{10}\cdot x_{2}$

  * $x_{3}:\>\>\>\>$

$f_{1}(f_{1}(0))=f_{1}(0)+\frac{2}{100}+\frac{1}{100}\cdot x_{3}$

  * $x_{4}:\>\>\>\>$

$f_{2}(f_{1}(0))=f_{1}(0)+\frac{7}{100}+\frac{1}{100}\cdot x_{4}$

  * $x_{5}:\>\>\>\>$

$f_{1}(f_{2}(0))=f_{2}(0)+\frac{2}{100}+\frac{1}{100}\cdot x_{5}$

  * $x_{6}:\>\>\>\>$

$f_{2}(f_{2}(0))=f_{2}(0)+\frac{7}{100}+\frac{1}{100}\cdot x_{6}$

  * $x_{7}:\>\>\>\>$

$f_{1}(f_{1}(f_{1}(0)))=f_{1}(f_{1}(0))+\frac{2}{1000}+\frac{1}{1000}\cdot
x_{7}$

and so on.

Suppose to the contrary that $A[0,1]$ is $\sigma$-strongly porous. Then so is
the family $\mathcal{F}$ of attractors corresponding (in the above way) to
sequences in $[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$. In particular,
$\mathcal{F}=\bigcup_{m\in\mathbb{N}}P_{m}$ where each $P_{m}$ is a strongly
porous subset of $K[0,1]$. The sets $P_{m}$ define a cover of
$[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$ by countably many pieces. Since
$[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$ is a complete metric space, by the
Baire category theorem there is $m\in\mathbb{N}$ and an open set $I_{1}\times
I_{2}\times\dots\times
I_{k}\times[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}\subseteq[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$
(where $I_{1},\ldots,I_{k}$ are open intervals in
$[-\frac{3}{2},\frac{3}{2}]$) such that the sequences corresponding to
attractors from $P_{m}$ are dense in $I_{1}\times I_{2}\times\dots\times
I_{k}\times[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$. For every $i\leq k$ put
$x_{i}$ as a middle of an interval $I_{i}$. Then the sequence
$(x_{1},x_{2},\dots,x_{k},0,0,\dots)\in[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$
determines an attractor $E\in K[0,1]$.

Let us take $\varepsilon$ such that it is half the length of an interval that
is one level ”below” $k$-th interval in the construction of the attractor.
More formally, since $x_{1}$ and $x_{2}$ are responsible for the positions of
intervals of length $\frac{1}{10}$, $x_{3},x_{4},x_{5}$ and $x_{6}$ are
responsible for the positions of intervals of length $\frac{1}{100}$ etc.,
$\varepsilon=\frac{1}{2\cdot 10^{n}}$, where $n\in\mathbb{N}$ is such that
$\sum_{i=1}^{n-2}2^{i}<k\leq\sum_{i=1}^{n-1}2^{i}$.

Now fix $Y\in B_{H}(E,\varepsilon)$ and to shorten the notation put
$\delta=d_{H}(Y,E)$. Obviously, $\delta<\varepsilon=\frac{1}{2\cdot 10^{n}}$.
Thus, there exists $j\geq n$ such that
$\delta\in[\frac{1}{2}\cdot\frac{1}{10^{j+1}},\frac{1}{2}\cdot\frac{1}{10^{j}})$.
Let $J_{1},\ldots,J_{2^{j+2}}$ be the intervals of length $\frac{1}{10^{j+2}}$
from the construction of the attractor $E$.

Observe that:

  * •

$\widetilde{Y}_{\delta}\supseteq E$ (since $\delta=d_{H}(Y,E)$),

  * •

$\delta-\frac{99}{100}\delta=\frac{\delta}{100}<\frac{1}{2\cdot 10^{j+2}}$,

  * •

the maximal possible shift of each $J_{i}$ is equal to

$\frac{3}{2\cdot 10^{j+2}}=\frac{1}{10^{j+2}}+\frac{1}{2\cdot
10^{j+2}}>|J_{i}|+\frac{\delta}{100},$

  * •

the set $\widetilde{Y}_{\frac{99}{100}\delta}$ is a union of closed intervals
each of which has length at least
$2\frac{99}{100}\delta\geq\frac{99}{10^{j+3}}>\frac{1}{10^{j+2}}=|J_{i}|$.

Thus, for each $i\leq 2^{j+2}$ the set
$U_{i}\subseteq[-\frac{3}{2},\frac{3}{2}]$ equal to the interior of

$\left\\{x\in\left[-\frac{3}{2},\frac{3}{2}\right]:J_{i}+x\subseteq\widetilde{Y}_{\frac{99}{100}\delta}\right\\}$

is nonempty. Hence, there is an open subset of $I_{1}\times
I_{2}\times\dots\times I_{k}\times[-\frac{3}{2},\frac{3}{2}]^{\mathbb{N}}$
such that each attractor corresponding to an element of that open set is a
subset of $\widetilde{Y}_{\frac{99}{100}\delta}$.

Actually, by the last item above, we can even conclude that the set
$U_{i}\setminus[-10^{j}\delta,10^{j}\delta]$ is nonempty for each $i\leq
2^{j+2}$. Indeed, the interval $[-10^{j}\delta,10^{j}\delta]$ is responsible
for shifts by at most $\frac{\delta}{100}<\frac{1}{2}\frac{1}{10^{j+2}}$ (as
each $x\in[-\frac{3}{2},\frac{3}{2}]$ corresponds to a shift of $J_{i}$ by
$\frac{x}{10^{j+2}}$). Thus, it suffices to observe that
$|J_{i}|=\frac{1}{10^{j+2}}$,
$|[-\frac{1}{100}\delta,\frac{1}{100}\delta]|<\frac{1}{10^{j+2}}$ and
$2\frac{99}{100}\delta>2\frac{1}{10^{j+2}}$.

Now, for each $i\leq 2^{j+2}$ let $V_{i}\subseteq U_{i}$ be open and such
that:

  * •

if $i$ is odd (i.e., the closest interval $J_{i^{\prime}}$ to $J_{i}$ is on
the right-hand side of $J_{i}$) and
$U_{i}\cap[-\frac{3}{2},-10^{j}\delta)\neq\emptyset$ then
$V_{i}=U_{i}\cap[-\frac{3}{2},-10^{j}\delta)$,

  * •

if $i$ is even (i.e., the closest interval $J_{i^{\prime}}$ to $J_{i}$ is on
the left-hand side of $J_{i}$) and
$U_{i}\cap(10^{j}\delta,\frac{3}{2}]\neq\emptyset$ then
$V_{i}=U_{i}\cap(10^{j}\delta,\frac{3}{2}]$,

  * •

$V_{i}=U_{i}\setminus[-10^{j}\delta,10^{j}\delta]$ in all other cases.

Let $H\in P_{m}$ be an attractor corresponding to any sequence in

$I_{1}\times I_{2}\times\dots\times
I_{k}\times\left[-\frac{3}{2},\frac{3}{2}\right]^{\sum_{i=1}^{j+1}2^{i}-k}\times
V_{1}\times\ldots\times
V_{2^{j+2}}\times\left[-\frac{3}{2},\frac{3}{2}\right]^{\mathbb{N}}.$

Then $H\subseteq\widetilde{Y}_{\frac{99}{100}\delta}$. Moreover,
$Y\subseteq\widetilde{H}_{\frac{99}{100}\delta}$. Indeed, fix $y\in Y$ and let
$e\in E$ be such that $d(y,e)$ is minimal possible (i.e., $d(y,e)=d(y,E)$).
Without loss of generality we may assume that $e\in J_{i}$ for some odd $i\leq
2^{j+2}$ (the case of even $i$ is similar). Let $z$ be the point lying exactly
in the middle between $J_{i}$ and $J_{i+1}$. Denote also by $e^{\prime}$ the
point corresponding to $e$ in $H$. Then $s=d(e,e^{\prime})$ is the shift of
$J_{i}$. There are three possibilities:

  * •

$y\leq e^{\prime}$ and $J_{i}$ was shifted to the left (i.e., towards $y$). In
this case

$d(y,e^{\prime})=d(y,e)-s<\delta-\frac{\delta}{100}=\frac{99}{100}\delta$

as the shift was by more than $\frac{\delta}{100}$.

  * •

$J_{i}$ was shifted to the left, but $y>e^{\prime}$. In this case the most
extreme possibility is $y=z$. However, we have:

$d(z,e)\leq|J_{i}|+\frac{1}{2}\text{dist}(J_{i},J_{i+1})=\frac{1}{10^{j+2}}+\frac{2}{10^{j+2}}=\frac{3}{10^{j+2}}.$

Since $s\leq\frac{3}{2}\cdot\frac{1}{10^{j+2}}$, we get that

$d(z,e^{\prime})=d(z,e)+s\leq\frac{3}{10^{j+2}}+\frac{3}{2}\cdot\frac{1}{10^{j+2}}<\frac{9}{2}\cdot\frac{1}{10^{j+2}}<\frac{99}{100}\frac{1}{2}\frac{1}{10^{j+1}}\leq\frac{99}{100}\delta.$

  * •

$J_{i}$ was shifted to the right. We will show that this implies that there is
no $y\in Y$ such that $d(y,e)=d(y,E)$ for $e\in J_{i}$, i.e., this case is
impossible. Indeed, if $y>z$ then $d(y,J_{i+1})<d(y,J_{i})$. On the other
hand, if $y<\min J_{i}-\delta$ then $d(y,J_{i-1})<d(y,J_{i})$ (as
$y\in\bigcup_{i^{\prime}\leq 2^{j+2}}\widetilde{(J_{i^{\prime}})}_{\delta}$).
Finally, if $\min J_{i}-\delta\leq y\leq z$ then it would be possible to shift
$J_{i}$ by some $\frac{\delta}{100}<t<\frac{3}{2}\frac{1}{10^{j+2}}$ to the
left in such a way that $t+J_{i}\subseteq B(y,\frac{99}{100}\delta)$ as
$2\frac{99}{100}\delta>|J_{i}|$ and

$d(z,\min
J_{i})+\frac{\delta}{100}\leq\frac{3}{10^{j+2}}+\frac{1}{2}\frac{1}{100}\frac{1}{10^{j}}<\frac{99}{100}\frac{1}{2}\frac{1}{10^{j+1}}\leq\frac{99}{100}\delta,$
$d(\min J_{i}-\delta,\max
J_{i})-\frac{3}{2}\frac{1}{10^{j+2}}=\delta+|J_{i}|-\frac{3}{2}\frac{1}{10^{j+2}}=\delta-\frac{1}{2}\frac{1}{10^{j+2}}<\frac{99}{100}\delta,$

(recall that $\delta<\frac{1}{2}\frac{1}{10^{j}}$).

This finishes the proof. ∎

Knowing the status of $\sigma$-strong porosity of $A[0,1]$, we ask the same
question for the difference between this family and a broader one, namely the
family of weak attractors.

######  Question 4.2.

Is the set $A_{w}[0,1]\setminus A[0,1]$ $\sigma$-strongly-porous in $K[0,1]$?

## References

  * [1] E. D’Aniello, T. H. Steele,  Attractors for iterated function schemes on $[0,1]^{N}$ are exceptional,  Journal of Mathematical Analysis and Applications, 424: 534-541, 2015.
  * [2] E. D’Aniello, T. H. Steele,  Attractors for iterated function systems,  Journal of Fractal Geometry, 3: 95-117, 2016.
  * [3] E. D’Aniello, T. H. Steele,  Attractors for classes of iterated function systems,  European Journal of Mathematics, 5: 116-137, 2019.
  * [4] M. Edelstein,  On fixed and periodic points under contractive mappings,  Journal of London Mathematical Society s1-37, 1: 74-79, 1962.
  * [5] A.S. Kechris,  Classical Descriptive Set Theory,  Springer, New York 1998.
  * [6] P. Klinga and A. Kwela,  Borel complexity of the family of attractors for weak IFSs,  submitted.
  * [7] P. Klinga and A. Kwela,  IFSs and weak IFSs of size n and families of their attractors,  submitted.
  * [8] P. Klinga, A. Kwela and M. Staniszewski,  Size of the set of attractors for iterated function systems,  Chaos, Solitons and Fractals, 128: 104—107, 2019.
  * [9] M. Koc,  Upper porous sets which are not $\sigma$-lower porous,  Real Analysis Exchange, 35: 21-30, 2009/2010.
  * [10] M. E. Mera, M. Morán, D. Preiss, and L. Zajíček,  Porosity, $\sigma$-porosity and measures,  Nonlinearity, 16: 247-255, 2003.
  * [11] M. Nowak, M. Fernández-Martínez,  Counterexamples for IFS-attractors,  Chaos Solitons & Fractals, 89: 316-321, 2016.
  * [12] L. L. Stachó and L. I. Szabó,  A note on invariant sets of iterated function systems,  Acta Mathematica Hungarica, 119: 159-164, 2008.
  * [13] L. Zajíček,  On $\sigma$-porous sets in abstract spaces,  Abstract and Applied Analysis, vol. 2005, no. 5: 509-534, 2005.


# Spin glass systems as collective active inference

Conor Heins<EMAIL_ADDRESS>Brennan Klein  VERSES Research Labs, Los
Angeles, California, USA  Network Science Institute, Northeastern University,
Boston, Massachusetts, USA  Laboratory for the Modeling of Biological and
Socio-Technical Systems,
Northeastern University, Boston, Massachusetts, USA  Daphne Demekas  Network
Science Institute, Northeastern University, Boston, Massachusetts, USA
Birkbeck Department of Film, Media and Cultural Studies, London, UK
Miguel Aguilera  Sussex AI Group, Department of Informatics, University of
Sussex, Brighton, UK  Sackler Centre for Consciousness Science, University of
Sussex, Brighton, UK  Christopher Buckley  VERSES Research Labs, Los Angeles,
California, USA  Sussex AI Group, Department of Informatics, University of
Sussex, Brighton, UK  Sackler Centre for Consciousness Science, University of
Sussex, Brighton, UK

###### Abstract

An open question in the study of emergent behaviour in multi-agent Bayesian
systems is the relationship, if any, between individual and collective
inference. In this paper we explore the correspondence between generative
models that exist at two distinct scales, using spin glass models as a sandbox
system to investigate this question. We show that the collective dynamics of a
specific type of active inference agent is equivalent to sampling from the
stationary distribution of a spin glass system. A collective of specifically-
designed active inference agents can thus be described as implementing a form
of sampling-based inference (namely, from a Boltzmann machine) at the higher
level. However, this equivalence is very fragile, breaking upon simple
modifications to the generative models of the individual agents or the nature
of their interactions. We discuss the implications of this correspondence and
its fragility for the study of multiscale systems composed of Bayesian agents.

##  1 Introduction

Emergent phenomena in multi-agent systems are central to the study of self-
organizing, complex systems, yet the relationship between individual
properties and group-level phenomena remains opaque and lacks formal
explanation. One principled approach to understanding such phenomena is
offered by the Free Energy Principle and so-called ‘multiscale active
inference’ [1, 2]. Proponents of this multiscale approach propose that groups
of individually-Bayesian agents necessarily entail an emergent, higher-order
Bayesian agent — in other words, systems are ‘agents all the way down’ by
definition [3, 4, 2, 5, 6]. However, to date, there has been little
theoretical or modeling work aimed at investigating whether this proposition
is true in general or even demonstrating an existence proof in a specific
system.

In this work we investigate this proposal by building a network of active
inference agents that collectively implement a spin glass system. Spin glasses
are a well-studied model class with a long history in statistical physics and
equilibrium thermodynamics [38, 8]. In the context of machine learning and
computational neuroscience, spin glass systems can be tied to models of
Bayesian inference and associative memory, particularly in the form of
Hopfield networks and undirected graphical models like Boltzmann machines [9,
10]. Boltzmann machines are a canonical example of an energy-based model in
machine learning — they are defined by a global energy function and are
analytically equivalent to a particular sort of spin glass model. The
Boltzmann machine can straightforwardly be shown to be an inferential model by
conditioning on the states of particular spins and sampling from the posterior
distribution over the remaining spins’ states [11, 12]. In doing so, Boltzmann
machines and spin glass systems can be described as performing Bayesian
inference about the latent causes (spin configurations) of the ‘data’
(conditioned spins).

In this paper, we set out to investigate whether an inference machine (in our
case, a Boltzmann machine) that exists at a ‘higher-level’, can be
hierarchically decomposed into an ensemble of agents collectively performing
active inference at a ‘lower level.’ We show a simple but rigorous equivalence
between collective active inference and spin glass systems, providing the
first steps for future quantitative study into the relationship between
individual and collective generative models. We show that a group of active
inference agents, equipped with a simple but very specific generative model,
collectively sample from the stationary distribution of a spin glass system at
the higher scale. This can be connected to a particular form of sampling known
as Glauber dynamics [38, 13]. When we further condition on the states of
particular agents, then the system can be interpreted as collectively
performing a form of sampling-based posterior inference over the
configurations of the unconditioned agents, i.e., Boltzmann machine inference.

This paper is structured as follows: first, we specify the generative model
that each spin site in a multi-agent spin glass system is equipped with,
noting that the single agents are constructed explicitly such that their
interactions at the lower-level lead to a spin glass system at the higher
level. Then, we establish the equivalence between this multi-agent active
inference process and Glauber dynamics, a scheme that samples from the
stationary distribution of a spin glass system. We then generalize this result
to sampling-based inference in Boltzmann machines by relaxing the homogeneous
parameterization of each agent’s generative models. We draw exact equivalences
between the precisions of each agent’s generative model and the parameters of
a higher-level Boltzmann machine. We conclude by noting the fragility of the
equivalence between multi-agent active inference and sampling from a
collective spin glass system, and discuss the implications of our results for
the study of emergent Bayesian inference in multiscale systems.

##  2 Generative model for a single spin

We begin by constructing a generative model for a single Bayesian agent, which
we imagine as a single spin in a collective of similar agents. From the
perspective of this single ‘focal agent’ or spin, this generative model
describes the agent’s internal model of how the local environment generates
its sensory data. Throughout this section we will take the perspective of this
single focal agent, keeping in mind that any given agent is embedded in a
larger system of other agents.

###  2.1 States and observations

We begin by specifying the state-space of observations and hidden states that
characterize the focal agent’s ‘world.’ The focal agent’s observations or
sensations are comprised of a collection of binary spin states
$\tilde{\sigma}=\\{\sigma_{j}:j\in M\\}$ where $\sigma_{j}=\pm 1$ and $M$ is
the set of other spins that the focal agent has direct access to. In other
words, the agent directly observes the spin states of neighbouring agents (but
not its own).

The focal agent assumes these observed spin states $\tilde{\sigma}$ all depend
on a single, binary latent variable $z$ — the ‘hidden spin state’, which could
also be interpreted as a coarse-grained ‘average spin’ of its neighbours.
Having specified observations $\tilde{\sigma}$ and latent states $z$, the full
generative model can be written as a joint distribution over observations and
the hidden spin state, $P(\tilde{\sigma},z)$. This in turn factorizes into a
set of distributions that describe the likelihood over observations, given the
latent state $P(\tilde{\sigma}|z)$, and prior beliefs about the latent state
$P(z)$:

$\displaystyle P(\tilde{\sigma},z)=P(\tilde{\sigma}|z)P(z)$

We parameterize the likelihood and prior distributions as Bernoulli
distributions (expressed in a convenient exponential form):

$\displaystyle P(\tilde{\sigma}|z;\gamma)$ $\displaystyle=\prod_{j\in
M}\frac{\exp(\gamma\sigma_{j}z)}{2\cosh(\gamma z)}$ Likelihood

$\displaystyle P(z;\zeta)$ $\displaystyle=\frac{\exp(\zeta z)}{2\cosh(\zeta
z)}$ Prior

The likelihood factorizes into a product of independent Bernoulli
distributions over each neighbouring spin $\sigma_{j}$. The full likelihood is
parameterized with a single sensory precision parameter $\gamma$ whose
magnitude captures the focal agent’s assumption about how reliably
neighbouring spin states $\sigma_{j}$ indicate the identity of the latent
state $z$. A positive $\gamma$ indicates that $\sigma_{j}$ lends evidence to
$z$ being aligned with $\sigma_{j}$, whereas a negative $\gamma$ means that
$\sigma_{j}$ lends evidence to $z$ being opposite to $\sigma_{j}$.

The prior over $z$ is also a Bernoulli distribution, parameterized by a
precision $\zeta$ that acts as a ‘bias’ in the focal agent’s prior belief
about the value of $z$. When $\zeta>0$, the focal agent believes the ‘UP’
($z=+1$) state is more likely a priori, whereas $\zeta<0$ indicates that the
agent believes that $z=-1$ is more likely, with the magnitude of $\zeta$
reflecting the strength or confidence of this prior belief.

Figure 1: Schematic illustration of individual and collective dynamics. (A)
Example of a system of 16 spin sites connected via a 2-D lattice, each in a
state of $\sigma\in\\{-1,+1\\}$ (green down-arrow or yellow up-arrow above),
with a focal agent and its spin, $\sigma_{i}=-1$, highlighted in blue. (B)
Generative model of a single spin. (C) The posterior belief over $z=+1$ as a
function of the spin difference $\Delta\sigma$. Left: The steepness of the
function is tuned by $\gamma$ ($\zeta=0.0$ shown). Right: The horizontal shift
depends on $\zeta$ ($\gamma=0.5$ shown).

###  2.2 Bayesian inference of hidden states

Having specified a generative model, we now consider (from the perspective of
a focal agent) the problem of estimating $z$, given observations
$\tilde{\sigma}=\\{\sigma_{j}:j\in M\\}$ and generative model parameters
$\gamma,\zeta$. This is the problem of Bayesian inference, specifically the
calculation of the posterior distribution over $z$. The conjugate-exponential
form of the generative model means that the Bayesian posterior can be
calculated exactly, and has a Bernoulli form that depends on $\tilde{\sigma}$
and $z$:

$\displaystyle
P(z|\tilde{\sigma};\gamma,\zeta)=\frac{P(\tilde{\sigma},z;\gamma,\zeta)}{P(\tilde{\sigma};\gamma,\zeta)}=\frac{\exp\left(z(\zeta+\gamma\sum_{j}\sigma_{j})\right)}{2\cosh\left(\zeta+\gamma\sum_{j}\sigma_{j}\right)}$
(1)

If we fix the hidden state $z$ to a particular value (e.g. $z=+1$), then we
arrive at a simple expression for the posterior probability that the hidden
spin state is in the ‘UP’ state, given the observations and the generative
model parameters $\gamma,\zeta$. The posterior belief expressed as the sum of
sensory input $\sum_{j}\sigma_{j}$ assumes a logistic or sigmoid form.
Hereafter we refer to the sum of observed spin states as the ‘spin difference’
$\Delta\sigma=\sum_{j}\sigma_{j}$, since this sum is equivalent to the
difference in the number of positive ($\sigma_{j}=+1$) and negative
($\sigma_{j}=-1$) spins. Intuitively, the steepness and horizontal shift of
this logistic function are determined by the likelihood and prior precisions:

$\displaystyle
P(z=+1|\tilde{\sigma},\gamma,\zeta)=\frac{1}{1+\exp\left(-2(\zeta+\gamma\Delta\sigma)\right)}$
(2)

Figure 1C shows the effect of varying the likelihood and prior precisions on
the posterior belief over $z$ as a function of $\Delta\sigma$. We can also
express the posterior as a Bernoulli distribution using the more common form,
with parameter $\phi_{z}$:

$\displaystyle P(z|\tilde{\sigma},\gamma,\zeta;\phi_{z})$
$\displaystyle=(1-\phi_{z})^{1-\frac{z+1}{2}}\phi_{z}^{\frac{z+1}{2}}$
$\displaystyle\phi_{z}$
$\displaystyle=\frac{1}{1+\exp(-2(\zeta+\gamma\Delta\sigma))}$ (3)

We now have a simple update rule that expresses how a focal agent updates its
beliefs about $z$ in response to observed spins $\tilde{\sigma}$. This sigmoid
belief update has a clear, intuitive relationship to the parameters of the
focal agent’s generative model, with $\gamma$ encoding the sensitivity of the
belief to small changes in $\Delta\sigma$ and $\zeta$ encoding a ‘bias’ that
skews the belief towards either $-1$ or $+1$. In the next section, we connect
the generation of spins themselves to an active inference process, that
leverages the Bayesian estimation problem of the current section to determine
a focal agent’s inference of its own spin state.

###  2.3 Active inference of spin states

Having addressed the issue of Bayesian inference or state estimation, we can
now specify a mechanism by which agents generate their own spin states. These
generated spin states will then serve as observations for the neighbours to
whom the focal agent is connected. This turns into a problem of belief-guided
action selection or decision-making. To enable agents to sample spin states as
a function of their beliefs, we supplement each agent’s current generative
model with an extra random variable that corresponds to control states, and
some forward model of how those control states determine observations. We use
active inference to optimize a posterior belief over these control states [14,
32, 34]; an agent can then act to change its spin state by sampling from this
posterior. By equipping each agent with a particular type of forward model of
how its actions impact observations, we can formally tie the collective
dynamics of active inference agents with this generative model to a sampling
scheme from a spin glass model. Appendix B walks through the steps needed to
add a representation of control states into the generative model introduced in
the previous section, and perform active inference with respect to this
augmented generative model.

Active inference agents entertain posterior beliefs not only about the hidden
states of the world, but also about how their own actions affect the world.
Posterior beliefs about actions are denoted $Q(u;\phi_{u})$, where $u$ is a
random variable corresponding to actions and $\phi_{u}$ are the parameters of
this belief. As opposed to the analytic posterior over hidden states $z$,
$Q(u;\phi_{u})$ is an approximate posterior, optimized using variational
Bayesian inference [33]. In our focal agent’s simple action model, control
states have the same support as hidden states, i.e. $u=\pm 1$. The value of
$u$ represents a possible spin action to take (‘UP’ vs. ‘DOWN’). We
parameterize $Q(u;\phi_{u})$ as a Bernoulli distribution with parameter
$\phi_{u}$, which itself encodes the probability of taking the ‘UP’ ($+1$)
action:

$\displaystyle
Q(u_{t};\phi_{u})=(1-\phi_{z})^{1-\frac{u_{t}+1}{2}}\phi_{z}^{\frac{u_{t}+1}{2}}$
(4)

When we equip our spin glass agents with a particular (predictive) generative
model, we can show that the approximate posterior over control states
simplifies to the state posterior (see Appendix B for details), and an agent
can generate a spin state by simply sampling from the posterior over actions:

$\displaystyle Q(u;\phi_{u})$ $\displaystyle\approx
P(z|\tilde{\sigma},\gamma,\zeta;\phi_{z})$ $\displaystyle\phi_{u}$
$\displaystyle\approx\phi_{z}:0\leq\phi_{z}\leq 1$

$\displaystyle\sigma$ $\displaystyle\sim Q(u;\phi_{u})$ $\displaystyle\sim
Q(z;\phi_{z})\triangleq P(z|\tilde{\sigma};\gamma,\zeta)$ (5)

We now have an active inference agent that 1) calculates a posterior belief
$P(z|\tilde{\sigma},\gamma,\zeta;\phi_{z})$ about the latent state $z$ in
response to the observed spins of other agents and 2) generates a spin of its
own by sampling from this belief, which ends up being identical to the
posterior over actions. Intuitively, each agent just broadcasts its beliefs
about the latent cause of its social observations, by sampling from its
posterior over this hidden (average) state. Another way of looking at this is
that each agent emits actions that maximize the accuracy of its beliefs (i.e.,
minimize variational free energy), under the prior assumption it is the author
of its sensations, which, implicitly, are shared with other agents. Note that
the choice to sample from the posterior over actions (as opposed to e.g.
taking the maximum) renders this action-selection scheme a form of probability
matching [36, 19].

###  2.4 Completing the loop

Given this sampling scheme for generating actions, we can simulate a
collective active inference process by equating the actions of one agent to
the observations of another. Specifically, each focal agent’s spin action
becomes an observation ($\sigma_{j}$ for some $j$) for all the other agents
that it (the focal agent) is connected to. Next, we will show how the dynamics
of multi-agent active inference is analogous to a particular algorithm for
sampling from the stationary distribution of a spin glass model, known as
Glauber dynamics [38]. We then examine the fragility of this equivalence by
exploring a number of simple modifications that break it.

##  3 Equivalence to Glauber dynamics

Spin glass models are formally described in terms of a global energy function
over states of the system. The global energy is related to the stationary
probability distribution of the system through a Gibbs law or Boltzmann
distribution:

$\displaystyle p^{*}(\tilde{\sigma})$ $\displaystyle=\frac{1}{Z}\exp(-\beta
E(\tilde{\sigma}))$ (6)

where the stationary density $p^{*}$ and energy function $E$ are defined over
spin configurations, where a configuration is a particular setting of each of
the $N$ spins that comprise the system:
$\tilde{\sigma}=\\{\sigma_{i}\\}_{i=1}^{N}:\sigma_{i}\pm 1$. The partition
function $Z$ is a normalizing constant that ensures $p^{*}(\tilde{\sigma})$
integrates to $1.0$, and $\beta$ plays the role of an inverse temperature that
can be used to arbitrarily rescale the Gibbs measure. In the case of the Ising
model, this energy function is a Hamiltonian that can be expressed as a sum of
pairwise interaction terms and an external drive or bias (often analogized to
an external magnetic field):

$\displaystyle E(\tilde{\sigma})$ $\displaystyle=-\sum_{\langle
i,j\rangle}\sigma_{i}J_{ij}\sigma_{j}-\sum_{i}h_{i}\sigma_{i}$ (7)

where $J_{ij}$ specifies a (symmetric) coupling between spin sites $i$ and $j$
and $h_{i}$ specifies an external forcing or bias term for site $i$. The
bracket notation $\langle i,j\rangle$ denotes a sum over pairs. In numerical
studies of the Ising model, one is typically interested in generating samples
from this stationary density. One scheme for doing so is known as Glauber
dynamics, where each spin $\sigma_{i}$ of the system is updated using the
following stochastic update rule:

$\displaystyle\sigma_{i}$ $\displaystyle\sim P(\sigma_{i}=(-1,+1))$
$\displaystyle P(\sigma_{i}=+1)$
$\displaystyle=\frac{1}{1+\exp(-\beta\Delta_{i}E)}$ $\displaystyle\Delta_{i}E$
$\displaystyle=E_{\sigma_{i}=DOWN}-E_{\sigma_{i}=UP}=2\left(\sum_{j\in
M_{i}}J_{ij}\sigma_{j}+h_{i}\right)$ (8)

where $\Delta_{i}E$ represents the difference in the energy between
configurations where $\sigma_{i}=-1$ and those where $\sigma_{i}=+1$. In other
words, the probability of unit $i$ flipping to $+1$ is proportional to the
degree to which the global energy $E$ would decrease as a result of the flip.
If units are updated sequentially (also known as ‘asynchronous updates’), then
given sufficient time Glauber dynamics are guaranteed to sample from the
stationary density in Equation (6) [38].

The probability of agent $i$ taking action $\sigma_{i}=+1$ is given by the
action sampling rule in Equation (2.3) of the previous section. We can thus
write the probability of taking a particular action in terms of the posterior
over latent states $z$, by plugging in the posterior belief over $z=+1$ (given
in Equation (2)) into Equation (2.3):

$\displaystyle P(\sigma_{i}=+1)$
$\displaystyle=\frac{1}{1+\exp(-2(\zeta_{i}+\gamma_{i}\Delta_{i}\sigma))}$ (9)

where we now index $\zeta,\gamma,\Delta\sigma$ by $i$ to indicate that these
are the generative model parameters and observations of agent $i$. The
identical forms shared by Equations (3) and (9) allow us to directly relate
the parameters of individual generative models to the local energy difference
$\Delta_{i}E$ and the global energy of the system.

$\displaystyle\Delta_{i}E\propto J\sum_{j\in
M_{i}}\sigma_{j}+h_{i}=\gamma\Delta_{i}\sigma+\zeta_{i}\implies
J=\gamma,\hskip 5.69054pth_{i}=\zeta_{i}$

where we assume all agents share the same likelihood precision
$\gamma_{i}=\gamma^{*}:\forall i$, which is equivalent to forcing all
couplings to be identical $J_{ij}=J:\forall i,j$. Individual active inference
agents in this multi-agent dynamic thus behave as if they are sampling spin
states in order to minimize a global energy function defined over spin
configurations, which in the case of spin glass systems like the Ising model,
can be computed using local observations (the spins of one’s neighbours) and
model parameters $\gamma,\zeta$. Going the other direction, one can sample
from the stationary distribution of spin glass system by simulating a
collective of active inference agents with an appropriately parameterized
generative model.

However, the equivalence between Glauber sampling from the stationary
distribution of an Ising model and collective active inference breaks down
when agents update their actions in parallel or synchronously, rather than
asynchronously. In particular, under parallel action updates, the system
samples from a stationary distribution with a different energy function than
that from Equation (7). See Appendix C for derivations on the relationship
between the schedule of action updates (synchronous vs. asynchronous) and the
resulting stationary density of the system.

##  4 Equivalence to inference in Boltzmann machines

Connecting the collective dynamics of these specialized active inference
agents to inference in Boltzmann machines is straightforward. We now equip
each agent’s generative model with a vector of sensory precisions
$\tilde{\gamma}=\\{\gamma_{j}:j\in M\\}$ that act as different reliabilities
assigned to different neighbours’ spin observations. The new factorized
likelihood can be written:

$\displaystyle P(\tilde{\sigma}|z;\tilde{\gamma})=\prod_{j\in
M}\frac{\exp(\gamma_{j}\sigma_{j}z)}{2\cosh(\gamma_{j}z)}$ (10)

We can then write the posterior over $z$ as a function of observations and
generative model parameters $\\{\tilde{\sigma},\zeta,\tilde{\gamma}\\}$. By
fixing $z$ to the value $+1$, we obtain again a logistic expression for the
posterior probability $P(z=+1|\tilde{\sigma},\tilde{\gamma},\zeta)$ that is
nearly identical to the original Equation (2):

$\displaystyle
P(z=+1|\tilde{\sigma},\tilde{\gamma},\zeta)=\frac{1}{1+\exp(-2(\zeta+\sum_{j\in
M}\gamma_{j}\sigma_{j}))}$ (11)

A Boltzmann machine is a special variant of a spin glass model, defined by a
global energy that in turn defines the stationary probability assigned to each
of the system’s configurations. The Boltzmann energy $E_{B}$, as with the
classical spin glass energy, is defined over configurations of the system’s
binary units (also known as nodes or neurons)
$\tilde{x}=\\{x_{i}\\}_{i=1}^{N}$.

In the context of inference, it is common to partition the system’s units into
‘visible units’ and ‘hidden units’, $\tilde{x}=\\{\tilde{v},\tilde{h}\\}$,
with the following energy function:

$\displaystyle E_{B}(\tilde{v},\tilde{h})$
$\displaystyle=-\frac{1}{2}(\tilde{v}^{\top}\mathbf{W}_{vv}\tilde{v}+\tilde{h}^{\top}\mathbf{W}_{hh}\tilde{h}+\tilde{v}^{\top}\mathbf{W}_{vh}\tilde{h})-\tilde{\theta}^{\top}_{v}\tilde{v}-\tilde{\theta}^{\top}_{h}\tilde{h}$
(12)

where $\mathbf{W}_{vv},\mathbf{W}_{hh},\mathbf{W}_{vh}$ are weight matrices
with symmetric couplings between units with no ‘self-edges’ ($W_{ii}=0:\forall
i$) that mediate dependencies both within and between the two subsets of units
$\tilde{v},\tilde{h}$; and $\tilde{\theta}_{v},\tilde{\theta}_{h}$ are vectors
of unit-specific biases or thresholds. The Bayesian inference interpretation
of Boltzmann machines considers the conditional probability distribution over
$\tilde{h}$, given some fixed values of $\tilde{v}$. The ‘clamping’ of visible
nodes to some data vector $\tilde{v}=\tilde{d}$ can simply be absorbed into
the biases $\tilde{\theta}_{v}$, such that samples from the posterior
$P(\tilde{h}|\tilde{v}=\tilde{d})$ are analogous to sampling from the joint
distribution $P(\tilde{h},\tilde{v})$ where the biases of the visible nodes
are adjusted to reflect this clamping. Sampling from this model can be
achieved with Glauber dynamics, since the model is a spin glass system with
heterogeneous (but symmetric) couplings. We can therefore write the single
unit ‘ON’ probability as follows, now in terms of weights $\mathbf{W}$ and
thresholds $\tilde{\theta}$:

$\displaystyle P(x_{i}=+1)$
$\displaystyle=\frac{1}{1+\exp(-\Delta_{i}E_{B})}=\frac{1}{1+\exp(-\sum_{j}W_{ij}x_{j}-\theta_{i})}$
(13)

where the interaction term that comprises the local energy difference
$\Delta_{i}E_{B}$ is equivalent to a dot-product between the
$i\textsuperscript{th}$ row of $\mathbf{W}$ and vector of activities
$\tilde{x}$. It is thus straightforward to relate the weights and biases of a
Boltzmann machine to the sensory and prior precisions of each agent’s
generative model. In particular, the weight connecting unit $j$ to unit $i$ in
a Boltzmann machine $W_{ij}$ is linearly related to the precision that agent
$i$ associates to spin observations coming from agent $j$:
$W_{ij}=2\gamma_{(i,j)}$ where the subscript $(i,j)$ refers to agent $i$’s
likelihood model over observations emitted by agent $j$. If agents $i$ and $j$
are not connected, then $W_{ij}=\gamma_{(i,j)}=0$. The bias of the $i$th unit
is also straightforwardly related to agent $i$’s prior precision via
$\theta_{i}=2\zeta_{i}$.

We have seen how sampling from the posterior distribution of a Boltzmann
machine with fixed visible nodes $\tilde{v}$ is equivalent to the collective
dynamics of a specific multi-agent active inference scheme. We have thus shown
a carefully-constructed system, in which a form of sampling-based Bayesian
inference at one scale emerges from a process of collective active inference
at a lower scale.

##  5 Discussion

Although the equivalences we have shown are exact, there are numerous
assumptions that, when broken, violate the equivalence between the multi-agent
dynamics defined at the lower level, and the higher-level sampling dynamics of
the spin glass system.

The energy-based models we have studied (Ising models, Boltzmann machines) are
all undirected graphical models: this means that the global energy function is
defined by symmetric interaction terms across pairs of spins:
$J_{ij}=J_{ji}$111$\mathbf{W}=\mathbf{W}^{\top}$ for the Boltzmann machine,
respectively. In order to meet this requirement at the level of the individual
agents, one must force the precisions that a pair of agents assign to one
another to be identical: $\gamma_{(i,j)}=\gamma_{(j,i)}$. This constraint also
underpins the equilibrium nature of classical spin glass systems where
detailed balance conditions are met, i.e., the system is in thermodynamic
equilibrium. In natural complex systems (especially biological ones), these
detailed balance conditions are often broken, and the systems operate far from
equilibrium [20, 21, 22, 23, 24, 25, 26, 27]. This may manifest in the case of
realistic multi-agent dynamics in the form of asymmetric beliefs about
reliability of social signals that agents assign to one another [28].

Another fragility of the multiscale equivalence is the structure of the
individual generative model, which relies on very specific assumptions about
how hidden states $z$ relate to observations. Without this generative model at
the single-agent level, there is no equivalence between the collective active
inference process and a spin glass model — the model’s dynamics could become
more complex (and potentially analytically intractable), because the posterior
update is not guaranteed to be a simple logistic function of the sum of
neighbouring spins. This could be easily shown by changing the single agent’s
likelihood model to include a separate latent variable for each neighbour’s
spin state222This is analogous to the approach taken in [28], where each agent
had beliefs about the belief state of each of its neighbours, or if the total
likelihood over neighbouring spin observations did not factorize into a
product of neighbour-specific likelihoods, but had some more complex
statistical structure.

Finally, the convergence of Glauber dynamics to samples from the joint density
over spin configurations depends on the temporal schedule of action updating;
namely, spins have to be updated sequentially or asynchronously, rather than
in parallel (see Appendix C for details) in order to guarantee sampling from
the stationary distribution in Equation (6). If agents act in parallel, then
the stationary distribution of spin states is different than that given by the
classical spin glass Hamiltonian. In other words, depending on the relative
timescales of collective action, agents either will or will not minimize the
global energy function that their actions appear to local minimize (i.e.
actions that minimize the local energy difference $\Delta_{i}E$).

##  6 Conclusion

In this work we demonstrate an exact equivalence between a collective active
inference process and sampling from the stationary density of a spin glass
system. Furthermore, we connect the system’s collective dynamics to sampling-
based inference in energy-based models like Boltzmann machines. In particular,
when we constrain certain agents in the network to be ‘visible nodes’ and fix
their actions, then the whole system samples from the posterior distribution
over spin configurations, given the actions of the ‘visible’ agents. Despite
these exact relationships, we also note the fragility of the equivalence,
which relies on very particular assumptions. These include the symmetry of the
precisions that pairs of agents assign to each other, the temporal scheduling
of the action updates, and the specific generative model used by the agents.
It remains to be seen, whether when these assumptions are broken, an
inferential or ‘agentive’ interpretation still obtains at higher scales, and
if so, whether the form of the ‘collective’ generative model can be
analytically related to the individual generative models as it was in the
present, equilibrium case.

Our results have important implications for the overall agenda of multiscale
active inference, and the quest to uncover the quantitative relationship
between generative models operating at distinct scales in complex systems. In
the system presented in the current work, we show that active inference agents
may collectively achieve sampling-based inference at a distinct, higher level
under particular conditions. Despite the apparent consistency at the two
scales, our result actually conflicts with claims made in the multiscale
active inference literature, that posits that systems hierarchically decompose
into nested levels of active inference agents [2, 3, 5, 1, 4] — in other
words, that systems are inherently active inference processes ‘all the way
down.’ Note that in our system, there are only active inference agents
operating at the lower level — the higher level is not an active inference
agent, but is better described as a passive agent that performs hidden state-
estimation or inference by sampling from a posterior belief over spin
configurations. The agenda of the present work also resonates with ongoing
research into the necessary and sufficient conditions for generic complex
systems to be considered ‘agentive’ or exhibit inferential capacities [29, 30,
31].

Our results suggest that multiscale inference interpretations of complex
systems do not necessarily emerge in any system. We nevertheless hope that the
simple equilibrium case we presented here may serve as a launching pad for
future studies into whether inference interpretations can be rescued at the
higher scale in cases when the fragile assumptions at the single-agent level
are broken.

## Additional information

#### Acknowledgements

The authors thank Alex Kiefer, Beren Millidge, Dalton Sakthivadivel, Magnus
Koudahl, Dimitrije Markovic and Maxwell Ramstead for their feedback and
comments throughout the writing of the manuscript.

#### Funding information

C.H. is supported by the U.S. Office of Naval Research (N00014-19-1-2556).
C.H., B.K., & D.D. acknowledge the support of a grant from the John Templeton
Foundation (61780). The opinions expressed in this publication are those of
the author(s) and do not necessarily reflect the views of the John Templeton
Foundation.

## References

  * [1] Karl Friston  “A free energy principle for a particular physics”  In _arXiv_ , 2019  DOI: 10.48550/arXiv.1906.10184
  * [2] Maxwell J.D. Ramstead, Paul B. Badcock and Karl Friston  “Answering Schrödinger’s question: A free-energy formulation”  In _Physics of Life Reviews_ 24, 2018, pp. 1–16  DOI: 10.1016/j.plrev.2017.09.001
  * [3] Michael Kirchhoff et al.  “The Markov blankets of life: Autonomy, active inference and the free energy principle”  In _Journal of the Royal Society Interface_ 15, 2018  DOI: 10.1098/rsif.2017.0792
  * [4] Ensor Rafael Palacios et al.  “On Markov blankets and hierarchical self-organisation”  In _Journal of Theoretical Biology_ 486, 2020, pp. 110089  DOI: 10.1016/j.jtbi.2019.110089
  * [5] Maxwell J.D. Ramstead, Axel Constant, Paul B. Badcock and Karl Friston  “Variational ecology and the physics of sentient systems”  In _Physics of Life Reviews_ 31, 2019, pp. 188–205  DOI: 10.1016/j.plrev.2018.12.002
  * [6] Casper Hesp et al.  “A multi-scale view of the emergent complexity of life: A free-energy proposal”  In _Evolution, Development and Complexity_ Springer, 2019, pp. 195–227  DOI: 10.1007/978-3-030-00075-2_7
  * [7] Roy J. Glauber  “Time-dependent statistics of the Ising model”  In _Journal of Mathematical Physics_ 4.2, 1963, pp. 294–307  DOI: 10.1063/1.1703954
  * [8] Stephen G. Brush  “History of the Lenz-Ising model”  In _Reviews of Modern Physics_ 39.4, 1967, pp. 883  DOI: 10.1103/RevModPhys.39.883
  * [9] Max Welling and Yee Whye Teh  “Approximate inference in Boltzmann machines”  In _Artificial Intelligence_ 143.1, 2003, pp. 19–50  DOI: 10.1016/S0004-3702(02)00361-2
  * [10] John J. Hopfield  “Neural networks and physical systems with emergent collective computational abilities”  In _Proceedings of the National Academy of Sciences_ 79.8, 1982, pp. 2554–2558  DOI: 10.1073/pnas.79.8.2554
  * [11] Geoffrey E. Hinton and Terrence J. Sejnowski  “Optimal perceptual inference”  In _Proceedings of the IEEE conference on Computer Vision and Pattern Recognition_ 448, 1983, pp. 448–453  URL: http://www.cs.toronto.edu/~hinton/absps/optimal.pdf
  * [12] Geoffrey E. Hinton and Terrence J. Sejnowski  “Learning and relearning in Boltzmann machines”  In _Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1: Foundations_ 1 MIT Press, 1986, pp. 282–317  URL: https://dl.acm.org/doi/10.5555/104279.104291
  * [13] Jean-Charles Walter and Gerard T. Barkema  “An introduction to Monte Carlo methods”  In _Physica A_ 418, 2015, pp. 78–87  DOI: 10.1016/j.physa.2014.06.014
  * [14] Karl Friston, Jean Daunizeau and Stefan J. Kiebel  “Reinforcement learning or active inference?”  In _PloS One_ 4.7, 2009, pp. e6421  DOI: 10.1371/journal.pone.0006421
  * [15] Karl Friston et al.  “Active inference and epistemic value”  In _Cognitive Neuroscience_ 6.4, 2015, pp. 187–214  DOI: 10.1080/17588928.2015.1020053
  * [16] Karl Friston et al.  “Active inference: A process theory”  In _Neural Computation_ 29.1, 2017, pp. 1–49  DOI: 10.1162/NECO_a_00912
  * [17] David M. Blei, Alp Kucukelbir and Jon D. McAuliffe  “Variational inference: A review for statisticians”  In _Journal of the American Statistical Association_ 112.518, 2017, pp. 859–877  DOI: 10.1080/01621459.2017.1285773
  * [18] David R. Shanks, Richard J. Tunney and John D. McCarthy  “A re-examination of probability matching and rational choice”  In _Journal of Behavioral Decision Making_ 15.3, 2002, pp. 233–250  DOI: 10.1002/bdm.413
  * [19] Alfonso Pérez-Escudero and Gonzalo Polavieja  “Collective animal behavior from Bayesian estimation and probability matching”  In _PLOS Computational Biology_ 7.11, 2011, pp. 1–14  DOI: 10.1371/journal.pcbi.1002282
  * [20] Chulan Kwon and Ping Ao  “Nonequilibrium steady state of a stochastic system driven by a nonlinear drift force”  In _Physical Review E_ 84.6, 2011, pp. 061106  DOI: 10.1103/PhysRevE.84.061106
  * [21] Han Yan et al.  “Nonequilibrium landscape theory of neural networks”  In _Proceedings of the National Academy of Sciences_ 110.45, 2013, pp. E4185–E4194  DOI: 10.1073/pnas.1310692110
  * [22] Yian Ma et al.  “Potential function in a continuous dissipative chaotic system: Decomposition scheme and role of strange attractor”  In _International Journal of Bifurcation and Chaos_ 24.02, 2014, pp. 1450015  DOI: 10.1142/S0218127414500151
  * [23] Ana P. Millán, Joaquín J. Torres and Ginestra Bianconi  “Explosive higher-order Kuramoto dynamics on simplicial complexes”  In _Physical Review Letters_ 124.21, 2020, pp. 218301  DOI: 10.1103/PhysRevLett.124.218301
  * [24] Karl Friston et al.  “Stochastic chaos and markov blankets”  In _Entropy_ 23.9, 2021, pp. 1220  DOI: 10.3390/e23091220
  * [25] Miguel Aguilera, S. Moosavi and Hideaki Shimazaki  “A unifying framework for mean-field theories of asymmetric kinetic Ising systems”  In _Nature Communications_ 12.1, 2021, pp. 1–12  DOI: 10.1038/s41467-021-20890-5
  * [26] Christopher W. Lynn et al.  “Broken detailed balance and entropy production in the human brain”  In _Proceedings of the National Academy of Sciences_ 118.47, 2021, pp. e2109889118  DOI: 10.1073/pnas.2109889118
  * [27] Miguel Aguilera, Masanao Igarashi and Hideaki Shimazaki  “Nonequilibrium thermodynamics of the asymmetric Sherrington-Kirkpatrick model”  In _arXiv_ , 2022  DOI: 10.48550/arXiv.2205.09886
  * [28] Mahault Albarracin, Daphne Demekas, Maxwell J.D. Ramstead and Conor Heins  “Epistemic communities under active inference”  In _Entropy_ 24.4, 2022, pp. 476  DOI: 10.3390/e24040476
  * [29] Nathaniel Virgo, Martin Biehl and Simon McGregor  “Interpreting dynamical systems as Bayesian reasoners”  In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_ , 2021, pp. 726–762  Springer  DOI: 10.1007/978-3-030-93736-2_52
  * [30] David Krakauer et al.  “The information theory of individuality”  In _Theory in Biosciences_ 139.2, 2020, pp. 209–223  DOI: 10.1007/s12064-020-00313-7
  * [31] Peaks Krafft, Erez Shmueli, Thomas L. Griffiths and Joshua B. Tenenbaum  “Bayesian collective learning emerges from heuristic social learning”  In _Cognition_ 212, 2021, pp. 104469  DOI: 10.1016/j.cognition.2020.104469

## References

  * [32] Karl Friston et al.  “Active inference and epistemic value”  In _Cognitive Neuroscience_ 6.4, 2015, pp. 187–214  DOI: 10.1080/17588928.2015.1020053
  * [33] David M. Blei, Alp Kucukelbir and Jon D. McAuliffe  “Variational inference: A review for statisticians”  In _Journal of the American Statistical Association_ 112.518, 2017, pp. 859–877  DOI: 10.1080/01621459.2017.1285773
  * [34] Karl Friston et al.  “Active inference: A process theory”  In _Neural Computation_ 29.1, 2017, pp. 1–49  DOI: 10.1162/NECO_a_00912
  * [35] Nir Vulkan  “An economist’s perspective on probability matching”  In _Journal of Economic Surveys_ 14.1, 2000, pp. 101–118  DOI: 10.1111/1467-6419.00106
  * [36] David R. Shanks, Richard J. Tunney and John D. McCarthy  “A re-examination of probability matching and rational choice”  In _Journal of Behavioral Decision Making_ 15.3, 2002, pp. 233–250  DOI: 10.1002/bdm.413
  * [37] Wolfgang Gaissmaier and Lael J. Schooler  “The smart potential behind probability matching”  In _Cognition_ 109.3, 2008, pp. 416–422  DOI: 10.1016/j.cognition.2008.09.007
  * [38] Roy J. Glauber  “Time-dependent statistics of the Ising model”  In _Journal of Mathematical Physics_ 4.2, 1963, pp. 294–307  DOI: 10.1063/1.1703954

## Appendix: Spin glass systems as collective active inference

##  Appendix A Bayesian inference for a single agent

In this appendix we derive the exact Bayesian inference update for the
posterior over the latent state $z$, taking the perspective of a single agent.
We begin by rehearsing the component likelihood and prior distributions of the
generative model in more detail.

###  A.1 Likelihood

The likelihood model relates the hidden state $z$ to the observed spin state
of a particular neighbour $\sigma_{j}$ as an exponential distribution
parameterized by a sensory precision parameter $\gamma$:

$\displaystyle
P(\sigma_{j}|z;\gamma)=\frac{\exp(\gamma\sigma_{j}z)}{2\cosh(\gamma z)}$ (A.1)

The sign and magnitude of $\gamma$ determines the nature of the expected
mapping between hidden states $z$ and neighbouring spin observations
$\sigma_{j}$. For $\gamma>0$, then the observed spin is expected to reflect
the latent state $z$, and with $\gamma<0$, then the observed spin is expected
to be opposite to the latent state $z$. The magnitude of $\gamma$ then
determines how deterministic this mapping is.

Equation (A.1) can alternatively be seen as a collection of two conditional
Bernoulli distributions over $\sigma_{j}$, one for each setting of $z$. This
can be visualized as a symmetric matrix mapping from the two settings of $z$
(the columns, corresponding to $z=-1,+1$) to the values of $\sigma_{j}$ (the
rows $\sigma_{j}=-1,+1$):

$\displaystyle P(\sigma_{j}|z;\gamma)$
$\displaystyle=\begin{bmatrix}\dfrac{1}{1+\exp(-2\gamma)}&\dfrac{1}{1+\exp(2\gamma)}\\\\[5.40608pt]
\dfrac{1}{1+\exp(2\gamma)}&\dfrac{1}{1+\exp(-2\gamma)}\end{bmatrix}$ (A.2)

where this mapping approaches the identity matrix as $\gamma\to\infty$.

Now we can move onto writing down the likelihood over the observed spins of
multiple neighbours: $\tilde{\sigma}=\\{\sigma_{j}:j\in M\\}$ where $M$
denotes the set of the focal agent’s neighbours. We build in a conditional
independence assumption into the focal agent’s generative model, whereby the
full likelihood model over all observed spins factorizes across the agent’s
neighbours. This means we can write the likelihood as a product of the single-
neighbour likelihoods shown in Equation (A.1):

$\displaystyle P(\tilde{\sigma}|z;\gamma)$ $\displaystyle=\prod_{j\in
M}\frac{\exp(\gamma\sigma_{j}z)}{2\cosh(\gamma z)}$
$\displaystyle=\exp\left(z\gamma\sum_{j\in M}\sigma_{j}-K\log(2\cosh(\gamma
z))\right)$ (A.3)

where $K$ is the number of the focal agent’s neighbours (i.e. the size of the
set $M$). We can easily generalize this likelihood to heterogeneous precisions
by instead parameterizing it with a precision vector
$\tilde{\gamma}=\\{\gamma_{j}:j\in M\\}$ that assigns a different precision to
observations coming from each of the focal agent’s neighbours:

$\displaystyle P(\tilde{\sigma}|z;\tilde{\gamma})$ $\displaystyle=\prod_{j\in
M}\frac{\exp(\gamma_{j}\sigma_{j}z)}{2\cosh(\gamma_{j}z)}$
$\displaystyle=\exp\left(z\sum_{j\in M}\gamma_{j}\sigma_{j}-\sum_{j\in
M}\log(2\cosh(\gamma_{j}z))\right)$ (A.4)

###  A.2 Prior

We parameterize the focal agent’s prior beliefs about the latent spin state
$z$ as a simple Bernoulli distribution, and similarly to the likelihood model,
we will express it as an exponential function parameterized by a ‘prior
precision’ parameter $\zeta$:

$\displaystyle P(z;\zeta)$ $\displaystyle=\frac{\exp(\zeta z)}{2\cosh(\zeta
z)}=\exp(\zeta z-\log(2\cosh(\zeta)))$

As with the sensory precision $\gamma$, the prior precision also scales the
strength of the focal agent’s prior belief that the spin state $z$ is
$+1$.333Note that $\cosh(\zeta z)$ can be re-written $\cosh(\zeta)$ when
$z\in\\{-1,+1\\}$ due to the symmetry of the hyperbolic cosine function around
the origin.

###  A.3 Bayesian inference of hidden states

Now we ask the question: how would a focal agent (i.e., the agent that
occupies a single lattice site) optimally compute a belief over $z$, that is
most consistent with a set of observed spins $\tilde{\sigma}$? This is a
problem of Bayesian inference, which can be expressed as calculation of the
posterior distribution over $z$ via Bayes Rule:

$\displaystyle
P(z|\tilde{\sigma};\gamma,\zeta)=\frac{P(\tilde{\sigma},z;\gamma,\zeta)}{P(\tilde{\sigma};\gamma,\zeta)}$
(A.5)

Since we are dealing with a conjugate exponential model 444The Bernoulli prior
is conjugate to the likelihood model, which can also be described of as a set
of conditional Bernoulli distributions., we can derive an analytic form for
the posterior: $P(z|\tilde{\sigma},\gamma,\zeta)$:

$\displaystyle
P(z|\tilde{\sigma};\gamma,\zeta)=\frac{\exp\left(z(\zeta+\gamma\sum_{j}\sigma_{j})\right)}{2\cosh\left(\zeta+\gamma\sum_{j}\sigma_{j}\right)}$
(A.6)

where the sum over neighbouring spins $j$ only includes the neighbours of the
focal agent, i.e, $\sum_{j\in M}\sigma_{j}$. If we fix the hidden state $z$ to
a particular value (e.g. $z=+1$), then we arrive at a simple expression for
the posterior probability that the hidden spin state is in the ‘UP’ state,
given the observations and the generative model parameters $\gamma,\zeta$.
This probability reduces to a logistic or sigmoid function of sensory input,
which is simply the sum of neighbouring spin values
$\Delta\sigma=\sum_{j}\sigma_{j}$. This can also be seen as the ‘spin
difference’, or the number of neighbouring spins that are in the ‘UP’
position, minus those that are in the ‘DOWN’ position. The steepness and
horizontal shift of this logistic function are intuitively given by likelihood
and prior precisions, respectively:

$\displaystyle P(z=+1|\tilde{\sigma},\gamma,\zeta)$
$\displaystyle=\frac{\exp(\zeta+\gamma\Delta\sigma)}{\exp(\zeta+\gamma\Delta\sigma)+\exp(-(\zeta+\gamma\Delta\sigma))}$
$\displaystyle=\left(1+\frac{\exp(-(\zeta+\gamma\Delta\sigma))}{\exp(\zeta+\gamma\Delta\sigma)}\right)^{-1}$
$\displaystyle=\frac{1}{1+\exp(-2(\zeta+\gamma\Delta\sigma))}$ (A.7)

The denominator in the first line of (A.7) follows from the identity
$\cosh(x)=\frac{1}{2}(\exp(x)+\exp(-x))$.

##  Appendix B Active inference derivations

In this section we provide the additional derivations needed to equip each
agent with the ability to infer a posterior over control states and sample
from this posterior to generate actions. This achieved through the framework
of active inference.

Active inference casts the selection of control states or actions as an
inference problem, whereby actions $u$ are sampled or drawn from posterior
belief about controllable hidden states. The posterior over actions is
computed as the softmax transform of a quantity called the expected free
energy [32]. This is the critical objective function for actions that enables
active inference agents to plan actions into the future, since the expected
free energy scores the utility of the anticipated consequences of actions.

###  B.1 Predictive generative model

We begin by writing a so-called ‘predictive’ generative model that crucially
includes probability distributions over the agent’s own control states
$u\in\\{-1,+1\\}$ and how those control states relate to future (anticipated)
observations. In other words, we consider a generative model over two
timesteps: the current timestep $t$ and one timestep in the future, $t+1$.
This will endow our agents with a shallow form of ‘planning’, where they
choose actions in order to maximize some (pseudo-) reward function defined
with respect to expected outcomes. This can be expressed as follows:

$\displaystyle
P(\tilde{\sigma}_{t},z_{t},u_{t},\mathcal{O}_{t+1};\gamma,\zeta)=\tilde{P}(\mathcal{O}_{t+1}|z_{t},u_{t},\tilde{\sigma}_{t})P(\tilde{\sigma}_{t},z_{t},u_{t};\gamma,\zeta)$
(B.8)

where the generative model at the second timestep
$\tilde{P}(\mathcal{O}_{t+1}|z_{t},u_{t},\tilde{\sigma}_{t})$ we hereafter
refer to as the ‘predictive’ generative model, defined over a single future
timestep.

Active inference consists in sampling a belief from the posterior distribution
over control states $u$ — this sampled control state or action is then fixed
to be the spin state of the agent under consideration. Thus the action of one
agent is fed in as the observations for those spin sites that it’s connected
to. In order to imbue active inference agents with a sense of goal-
directedness or purpose, we encode a prior distribution over actions $P(u)$
that is proportional to the negative of the expected free energy, via the
softmax relationship:

$\displaystyle P(u)=\frac{\exp(-\mathbf{G}(u))}{\sum_{u}\exp(-\mathbf{G}(u))}$
(B.9)

Crucially, the expected free energy of an action $\mathbf{G}$ is a function of
outcomes expected under a particular control state $u$, where beliefs about
future outcomes are ‘biased‘ by prior beliefs about encountering particular
states of affairs. In order to optimistically ‘bend’ these future beliefs
towards certain outcomes, and thus make some actions more probable than
others, we supplement the predictive generative model $\tilde{P}$ with a
binary ‘optimality’ variable $\mathcal{O}\pm 1$ that the agent has an inherent
belief that it will observe. This is encoded via a ‘goal prior’ or preference
vector, which is a Bernoulli distribution over seeing a particular value of
$\mathcal{O}$ with some precision parameter $\omega$:

$\displaystyle\tilde{P}(\mathcal{O}_{t+1};\omega)$
$\displaystyle=\frac{\exp(\omega\mathcal{O})}{2\cosh(\omega\mathcal{O})}$
(B.10)

Hereafter we assume an infinitely high precision, i.e. $\omega\to\infty$. This
renders the preference an ‘all-or-nothing’ distribution over observing the
optimality variable being in the ‘positive’ state $\mathcal{O}=+1$:

$\displaystyle=\begin{bmatrix}\tilde{P}(\mathcal{O}_{t+1}=-1)\\\
\tilde{P}(\mathcal{O}_{t+1}=+1)\end{bmatrix}=\begin{bmatrix}0.0\\\
1.0\end{bmatrix}$ (B.11)

To allow an agent the ability to predict the relationship between their
actions and expected observations, it’s important to include an additional
likelihood distribution, what we might call the ‘forward model’ of actions
$P(\mathcal{O}_{t+1}|z_{t},u_{t};\xi)$. This additional likelihood encodes the
focal agent’s assumptions about the relationship between hidden states,
actions, and the (expected) optimality variable. By encoding a deterministic
conditional dependence relationship into this likelihood, we motivate the
agent (via the expected free energy) to honestly signal its own estimate of
the hidden state via its spin action $u$. To achieve this, we explicitly
design this likelihood to have the following structure, wherein the optimality
variable is only expected to take its ‘desired value’ of $\mathcal{O}=+1$ when
$z=u$. This can be written as a set of conditional Bernoulli distributions
over $\mathcal{O}$, and each of which jointly depends on $z$ and $u$ and is
parameterized by a (infinitely high) precision $\xi$:

$\displaystyle P(\mathcal{O}_{t+1}|z_{t},u_{t};\xi)$
$\displaystyle=\frac{\exp(\xi\mathcal{O}_{t+1}z_{t}u_{t})}{2\cosh(\xi
z_{t}u_{t})}$ (B.12)

When we assume $\xi\to\infty$, then we arrive at a form for this likelihood
which can be alternatively expressed as a set of Bernoulli distributions that
conjunctively depend on $z$ and $u$, and can be visualized as follows:

$\displaystyle P(\mathcal{O}_{t+1}|z_{t},u_{t}=-1)$
$\displaystyle=\begin{bmatrix}0&1\\\ 1&0\end{bmatrix}$ $\displaystyle
P(\mathcal{O}_{t+1}|z_{t},u_{t}=+1)$ $\displaystyle=\begin{bmatrix}1&0\\\
0&1\end{bmatrix}$

where the columns of the matrices above correspond to settings of
$z\in\\{-1,+1\\}$. Therefore, the agent only expects to see $\mathcal{O}=+1$
(the desired outcome) when the value of the hidden state and the value of the
control variable are equal, i.e. $z=u$; otherwise $\mathcal{O}=-1$ is
expected. For the purposes of the present study, we assume both the optimality
prior $\tilde{P}(\mathcal{O};\omega)$ and the optimality variable likelihood
$P(\mathcal{O}|z,u;\xi)$ are parameterized by infinitely high precisions
$\omega=\xi=\infty$, and hereafter will exclude them when referring to these
distributions for notational convenience.

Having specified these addition priors and likelihoods, we can write down the
new (predictive) generative model as follows:

$\displaystyle\tilde{P}(\mathcal{O}_{t+1},u_{t},z_{t})=P(\mathcal{O}_{t+1}|z_{t},u_{t})P(u_{t})\tilde{P}(\mathcal{O}_{t+1})P(z_{t})$
(B.14)

###  B.2 Active inference

Under active inference, both state estimation and action are consequences of
the optimization of an approximate posterior belief over hidden states and
actions $Q(z,u;\phi)$. This approximate posterior is optimized in order to
minimize a variational free energy (or alternatively maximize an evidence
lower bound). This is the critical concept for a type of approximate Bayesian
inference known as variational Bayesian inference [33]. This can be described
as finding the optimal set of variational parameters $\phi$ that minimizes the
following quantity:

$\displaystyle\phi^{*}$
$\displaystyle=\underset{\phi}{\arg\min}\,\,\mathcal{F}$
$\displaystyle=\mathbb{E}_{Q}[\ln
Q(z_{t},u_{t};\phi)-\ln\tilde{P}(\tilde{\sigma}_{t},z_{t},u_{t},\mathcal{O}_{t+1};\gamma,\zeta)]$
(B.15)

In practice, because of the factorization of the generative model into a
generative model of the current and future timesteps, we can split state-
estimation and action inference into two separate optimization procedures. To
do this we also need to factorize the posterior as follows:

$\displaystyle Q(z,u;\phi)=Q(z;\phi_{z})Q(u;\phi_{u})$ (B.16)

where we have also separated the variational parameters
$\phi=\\{\phi_{z},\phi_{u}\\}$ into those that parameterize the belief about
hidden states $\phi_{z}$, and those that parameterize the belief about actions
$\phi_{u}$.

When considering state-estimation (i.e. optimization of $Q(z_{t};\phi_{z})$),
we only have to consider the generative model of the current timestep
$P(\tilde{\sigma}_{t},z_{t};\gamma,\zeta)$. The optimal posterior parameters
$\phi^{*}_{z}$ are found as the minimum of the variational free energy, re-
written using only those terms that depend on $\phi_{z}$:

$\displaystyle\phi^{*}_{z}$
$\displaystyle=\underset{\phi_{z}}{\arg\min}\,\,\mathcal{F}(\phi_{z})$
$\displaystyle\mathcal{F}(\phi_{z})$
$\displaystyle=\mathbb{E}_{Q(z_{t};\phi_{z})}[\ln Q(z_{t};\phi_{z})-\ln
P(\tilde{\sigma}_{t},z_{t};\gamma,\zeta)]$ (B.17)

To solve this, we also need to decide on a parameterization of the approximate
posterior over hidden states $z_{t}$. We parameterize $Q(z_{t};\phi_{z})$ as a
Bernoulli distribution with parameter $\phi_{z}$, that can be interpreted as
the posterior probability that $z_{t}$ is in the ‘UP’ ($+1$) state:

$\displaystyle
Q(z_{t};\phi_{z})=(1-\phi_{z})^{1-\frac{z_{t}+1}{2}}\phi_{z}^{\frac{z_{t}+1}{2}}$
(B.18)

By minimizing the variational free energy with respect to $\phi_{z}$, we can
obtain an expression for the optimal posterior $Q(z;\phi^{*}_{z})$ that sits
at the variational free energy minimum. Due to the exponential and conjugate
form of the generative model, $Q(z_{t};\phi_{z})$ is the exact posterior and
thus variational inference reduces to exact Bayesian inference. This means we
can simply re-use the posterior update equation of Equation (A.7) to yield an
analytic expression for $\phi^{*}_{z}$:

$\displaystyle\phi^{*}_{z}=\frac{1}{1+\exp\left(-2(\zeta+\gamma\Delta\sigma)\right)}$
(B.19)

When considering inference of actions, we now consider the generative model of
the future timestep, which crucially depends on the current control state
$u_{t}$ and the optimality variable $\mathcal{O}_{t+1}$. We can then write the
variational problem as finding the setting of $\phi_{u}$ that minimizes the
variational free energy, now re-written in terms of its dependence on
$\phi_{u}$:

$\displaystyle\phi^{*}_{u}$
$\displaystyle=\underset{\phi_{u}}{\arg\min}\,\,\mathcal{F}(\phi_{u})$
$\displaystyle\mathcal{F}(\phi_{u})$
$\displaystyle=\mathbb{E}_{Q(u_{t};\phi_{u})}[\ln
Q(u_{t};\phi_{u})-\ln\tilde{P}(\mathcal{O}_{t+1},u_{t},z_{t})]$ (B.20)

As we did for the posterior over hidden states, we need to decide on a
parameterization for the posterior over actions $Q(u_{t};\phi_{u})$; we also
parameterize this as a Bernoulli distribution with parameter $\phi_{u}$ that
represents the probability of taking the ‘UP’ ($+1$) action:

$\displaystyle
Q(u_{t};\phi_{u})=(1-\phi_{z})^{1-\frac{u_{t}+1}{2}}\phi_{z}^{\frac{u_{t}+1}{2}}$
(B.21)

From Equation (B.20) it follows that the optimal $\phi_{u}$ is that which
minimizes the Kullback-Leibler divergence between the approximate posterior
$Q(u_{t};\phi_{u})$ and the prior $P(u_{t})$, which is a softmax function of
the expected free energy of actions $\mathbf{G}(u_{t})$. In this particular
generative model, the expected free energy can be written as a single term
that scores the ‘expected utility’ of each action [32, 34]:

$\displaystyle\mathbf{G}(u_{t})$
$\displaystyle=-\mathbb{E}_{Q(\mathcal{O}_{t+1}|u_{t})}[\ln\tilde{P}(\mathcal{O}_{t+1})]$
(B.22)

To compute this, we need to compute the ‘variational marginal’ over
$\mathcal{O}_{t+1}$, denoted $Q(\mathcal{O}_{t+1}|u_{t})$:

$\displaystyle Q(\mathcal{O}_{t+1}|u_{t})$
$\displaystyle=\mathbb{E}_{Q(z_{t};\phi^{*}_{z})}[P(\mathcal{O}_{t+1}|z_{t},u_{t})]$
(B.23)

We can simplify the expression for $Q(\mathcal{O}_{t+1}|u_{t})$ when we take
advantage of the Bernoulli-parameterization of the posterior over hidden
states $Q(z;\phi^{*}_{z})$. This allows us to then write the variational
marginals, conditioned on different actions as a matrix, with one column for
each setting of $u_{t}$:

$\displaystyle Q(\mathcal{O}_{t+1}|u_{t})$
$\displaystyle=\begin{bmatrix}\phi^{*}_{z}&1-\phi^{*}_{z}\\\
1-\phi^{*}_{z}&\phi^{*}_{z}\end{bmatrix}$ (B.24)

The expected utility (and thus the negative expected free energy) is then
computed as the dot-product of each column of the matrix expressed in Equation
(B.24) with the log of the prior preferences $\tilde{P}(\mathcal{O}_{t+1})$:

$\displaystyle\mathbb{E}_{Q(\mathcal{O}_{t+1}|u_{t})}[\ln\tilde{P}(\mathcal{O}_{t+1})]$
$\displaystyle=\begin{bmatrix}-\infty\phi^{*}_{z}\\\
-\infty(1-\phi^{*}_{z})\end{bmatrix}$ $\displaystyle\implies\mathbf{G}(u_{t})$
$\displaystyle=\begin{bmatrix}\infty\phi^{*}_{z}\\\
\infty(1-\phi^{*}_{z})\end{bmatrix}$ (B.25)

Because the probability of an action is proportional to its negative expected
free energy, this allows us to write the Bernoulli parameter $\phi^{*}_{u}$ of
the posterior over actions directly in terms of the parameter of the state
posterior:

$\displaystyle\phi^{*}_{u}$
$\displaystyle=\frac{1}{1+\exp(\beta(\infty(1-\phi^{*}_{z})))}$
$\displaystyle=\frac{1}{1+C\exp(-\phi^{*}_{z})))}$ (B.26)

The inverse temperature parameter $\beta$ is an arbitrary re-scaling factor
that can be used to linearize the sigmoid function in (B.26) over the range
$[0,1]$ such that

$\displaystyle\phi^{*}_{u}$ $\displaystyle\approx\phi^{*}_{z}$ (B.27)

Note that the equivalence relation in Equation (B.27) is only possible due to
the infinite precisions $\omega$ and $\xi$ of the likelihood and prior
distributions over the ‘optimality’ variable
$P(\mathcal{O}_{t+1}|u_{t},z_{t})$ and $\tilde{P}(\mathcal{O}_{t+1})$, and
from an appropriately re-scaled $\beta$ parameter that linearizes the sigmoid
relationship in Equation (B.26).

###  B.3 Action sampling as probability matching

Now that we have an expression for the parameter $\phi^{*}_{u}$ of the
posterior over control states $Q(u_{t};\phi^{*}_{u})$, an agent can generate a
spin state by simply sampling from this posterior over actions:

$\displaystyle\sigma$ $\displaystyle\sim Q(u_{t};\phi^{*}_{u})$
$\displaystyle\sim Q(z_{t};\phi^{*}_{z})\triangleq
P(z_{t}|\tilde{\sigma};\gamma,\zeta)$ (B.28)

In short, each agent samples its spin state from a posterior belief over the
state of the latent variable $z_{t}$, rendering their action-selection a type
of probability matching [35, 36, 37], whereby actions (whether to spin ‘UP’ or
‘DOWN’) are proportional to the probability they are assigned in the agent’s
posterior belief. Each agent’s sampled spin state also serves as an
observation ($\sigma_{j}$ for some $j$) for the other agents that the focal
agent is a neighbour of. This collective active inference scheme corresponds
to a particular form of sampling from the stationary distribution of a spin
glass model known as Glauber dynamics [38]. Crucially, however, the temporal
scheduling of the action-updating across the group determines which stationary
distribution the system samples from. We explore this distinction in the next
section.

##  Appendix C Temporal scheduling of action sampling

In this appendix we examine how the stationary distribution from which the
collective active inference system samples depends on the order in which
actions are updated across all agents in the network. First, we consider the
case of synchronous action updates (all agents update their actions in
parallel and only observe the- spin states of their neighbours from the last
timestep), and show how this system samples from a different stationary
distribution than the one defined by the standard Ising energy provided in
Equation (7). We then derive the more ‘classical’ case of asynchronous
updates, where agents update their spins one at a time, and show how in this
case the system converges to the standard statioanry distribution of the Ising
model. This Appendix thus explains one of the ‘fragilities’ mentioned in the
main text, that threaten the unique equivalence between local active inference
dynamics and a unique interpretation at the global level in terms of
inference.

We denote some agent’s spin using $\sigma_{i}$ and its set of neighbours as
$M_{i}$. The local sum of spins or spin difference $\sum_{j\in M}\sigma_{j}$
for agent $i$ we denote $\Delta_{i}\sigma=\sum_{j\in M_{i}}\sigma_{j}$.

###  C.1 Synchronous updates

To derive the stationary distribution in case of synchronous updates, we can
take advantage of the following detailed balance relation, which obtains in
the case of systems at thermodynamic equilibrium:

$\displaystyle\frac{P(\tilde{\sigma})}{P(\tilde{\sigma}^{\prime})}$
$\displaystyle=\frac{P(\tilde{\sigma}|\tilde{\sigma}^{\prime})}{P(\tilde{\sigma}^{\prime}|\tilde{\sigma})}$
$\displaystyle\implies P(\tilde{\sigma})$
$\displaystyle=\frac{P(\tilde{\sigma}|\tilde{\sigma}^{\prime})P(\tilde{\sigma}^{\prime})}{P(\tilde{\sigma}^{\prime}|\tilde{\sigma})}$
(C.29)

where $\tilde{\sigma}$ and $\tilde{\sigma}^{\prime}$ are spin configurations
at two adjacent times $\tau$ and $\tau+1$. In the case of synchronous updates
(all spins are sampled simultaneously, given the spins at the last timestep),
then the spin action of each agent $\sigma_{i}^{\prime}$ at time $\tau+1$ is
conditionally independent of all other spins, given the vector of spins
$\tilde{\sigma}$ at the previous timestep $\tau$. We can therefore expand the
‘forward’ transition distribution $P(\tilde{\sigma}^{\prime}|\tilde{\sigma})$
as a product over the action posteriors of each agent:

$\displaystyle P(\tilde{\sigma}^{\prime}|\tilde{\sigma})$
$\displaystyle=P(\sigma_{1}|\tilde{\sigma})P(\sigma_{1}|\tilde{\sigma})...P(\sigma_{N}|\tilde{\sigma})$
$\displaystyle=\prod_{i}Q(u_{t};\phi^{*}_{u,i})$
$\displaystyle=\prod_{i}\frac{\exp\left(\sigma^{\prime}_{i}\left(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j}\right)\right)}{2\cosh\left(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j}\right)}$
$\displaystyle=\exp\left(\sum_{i}\sigma^{\prime}_{i}(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j})-\sum_{i}\log\left(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j})\right)\right)$ (C.30)

Note we have replaced each latent variable in the posterior $z$ with the
agent’s own spin state $\sigma_{i}$, because there is a one-to-one mapping
between the posterior over $z_{t}$ and the posterior over actions
$\sigma_{i}$.

The reverse transition distribution, yielding the probability of transitioning
from configuration $\tilde{\sigma}^{\prime}\to\tilde{\sigma}$ is the same
expression as for the forward transition, except that $\sigma_{i}^{\prime}$
and $\sigma_{i}$ are swapped:

$\displaystyle P(\tilde{\sigma}|\tilde{\sigma}^{\prime})$
$\displaystyle=\exp\left(\sum_{i}\sigma_{i}(\zeta+\gamma\sum_{j\in
M_{i}}\sigma^{\prime}_{j})-\sum_{i}\log\left(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma^{\prime}_{j})\right)\right)$ (C.31)

The detailed balance equation in (C.29) then tells us that the stationary
probability distribution over $\tilde{\sigma}$ is proportional to the ratio of
the backwards transition to the forwards transition:

$\displaystyle\frac{P(\tilde{\sigma})}{P(\tilde{\sigma^{\prime}})}$
$\displaystyle=\frac{\exp\left(\sum_{i}\sigma_{i}(\zeta+\gamma\sum_{j\in
M_{i}}\sigma^{\prime}_{j})-\sum_{i}\log\left(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma^{\prime}_{j})\right)\right)}{\exp\left(\sum_{i}\sigma^{\prime}_{i}(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j})-\sum_{i}\log\left(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j})\right)\right)}$
$\displaystyle=\frac{\exp\left(\zeta\sum_{i}\sigma_{i}+\gamma\sum_{\langle
i,j\rangle}\sigma_{i}\sigma_{j}^{\prime}\right)\exp\left(-\sum_{i}\log\left(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j}^{\prime}\right)\right)}{\exp\left(\zeta\sum_{i}\sigma_{i}^{\prime}+\gamma\sum_{\langle
i,j\rangle}\sigma_{i}^{\prime}\sigma_{j}\right)\exp\left(-\sum_{i}\log\left(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j}\right)\right)}$
$\displaystyle=\frac{\exp\left(\zeta\sum_{i}\sigma_{i}+\sum_{i}\log\left(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j}\right)\right)}{\exp\left(\zeta\sum_{i}\sigma_{i}^{\prime}+\sum_{i}\log\left(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j}^{\prime}\right)\right)}$ (C.32)

Therefore, we can write down the stationary distribution in the case of
synchronous updates as an exponential term normalized by a partition function:

$\displaystyle P(\tilde{\sigma})$
$\displaystyle=Z^{-1}\exp\left(\zeta\sum_{i}\sigma_{i}+\sum_{i}\log\left(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j})\right)\right)$ $\displaystyle Z$
$\displaystyle=\sum_{\tilde{\sigma}}\exp\left(\zeta\sum_{i}\sigma_{i}+\sum_{i}\log\left(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j})\right)\right)$ (C.33)

Note that the action update for an individual agent can still be written in
terms of the local energy difference $\Delta_{i}E$, where the energy is
defined using the standard Hamiltonian function given by Equation (7) in the
main text. However, due to the temporal sampling of each agent’s action with
respect to the others, the system collectively sample from a system with a
different energy function and Gibbs measure, given by Equation (C.1). This
energy function is therefore nonlinear and can be written:

$\displaystyle E_{sync}(\tilde{\sigma})$
$\displaystyle=-\zeta\sum_{i}\sigma-\sum_{i}\log(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j}))$ (C.34)

###  C.2 Asynchronous updates

Now we treat the case where agents update their agents one-by-one or
asynchronously. This means that at each timestep only one agent is updated,
and that particular agent uses the spin states of all the other agents at the
last timestep as inputs for its posterior inference.

We can write down the forward transition as follows, using the notation
$\sigma_{\setminus i}$ to denote all the spins except for $\sigma_{i}$:

$\displaystyle p(\sigma_{i}^{\prime},\tilde{\sigma}_{\setminus
i}|\tilde{\sigma})$
$\displaystyle=\frac{\exp(\sigma_{i}^{\prime}(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j}))}{2\cosh(\zeta+\gamma\sum_{j\in M_{i}}\sigma_{j})}$ (C.35)

which indicates that only agent $i$ is updated at the current timestep. The
detailed balance condition implies that

$\displaystyle p(\sigma_{i}^{\prime},\tilde{\sigma}_{\setminus
i}|\tilde{\sigma})p(\tilde{\sigma})=p(\tilde{\sigma}|\sigma_{i}^{\prime},\tilde{\sigma}_{\setminus
i})p(\sigma_{i}^{\prime},\tilde{\sigma}_{\setminus i})$ (C.36)

Then

$\displaystyle\frac{p(\tilde{\sigma})}{p(\sigma_{i}^{\prime},\tilde{\sigma}_{\setminus
i})}=$
$\displaystyle\frac{p(\tilde{\sigma}|\sigma_{i}^{\prime},\tilde{\sigma}_{\setminus
i})}{p(\sigma_{i}^{\prime},\tilde{\sigma}_{\setminus
i}|\tilde{\sigma})}=\frac{\exp(\sigma_{i}(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j})-\log(2\cosh(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j})))}{\exp(\sigma_{i}^{\prime}(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j})-\log(2\cosh(\zeta+\gamma\sum_{j\in M_{i}}\sigma_{j})))}$
(C.37) $\displaystyle=$
$\displaystyle\frac{\exp(\sigma_{i}(\zeta+\gamma\sum_{j\in
M_{i}}\sigma_{j}))}{\exp(\sigma_{i}^{\prime}(\zeta+\gamma\sum_{j\in
M_{i}i}\sigma_{j}))}$ (C.38)

By repeating this operation for every agent (i.e. $N-1$ more times), then we
arrive at:

$\displaystyle\frac{p(\tilde{\sigma})}{p(\tilde{\sigma}^{\prime})}=\frac{p(\tilde{\sigma})}{p(\sigma_{i}^{\prime},\tilde{\sigma}_{\setminus
i})}\frac{p(\sigma_{i}^{\prime},\tilde{\sigma}_{\setminus
i})}{p(\sigma_{i}^{\prime},\sigma_{j}^{\prime},\tilde{\sigma}_{\setminus
i,j})}\ldots\frac{p(\tilde{\sigma}^{\prime}_{\setminus
i},\sigma_{i})}{p(\tilde{\sigma}^{\prime})}=$
$\displaystyle\frac{\exp(\zeta\sum_{i}\sigma_{i}+\gamma\sum_{i<j}\sigma_{i}\sigma_{j})}{\exp(\zeta\sum_{i}\sigma^{\prime}_{i}+\gamma\sum_{i<j}\sigma^{\prime}_{i}\sigma^{\prime}_{j})}$
(C.39)

We can therefore write the marginal distributions $p(\tilde{\sigma})$ as
proportional to the numerator of the last term in Equation (C.39)555Note that
because of assumption that the system is at thermal equilibrium, the same
reasoning could be applied to write the distribution over
$p(\tilde{\sigma}^{\prime})$ in terms of the denominator of Equation (C.39):

$\displaystyle p(\tilde{\sigma})$
$\displaystyle\propto\exp(\zeta\sum_{i}\sigma_{i}+\gamma\sum_{\langle
i,j\rangle}\sigma_{i}\sigma_{j})$ $\displaystyle\implies p(x)=$ $\displaystyle
Z^{-1}\exp(\zeta\sum_{i}\sigma_{i}+\gamma\sum_{\langle
i,j\rangle}\sigma_{i}\sigma_{j})$ (C.40)

We thus recover the original stationary distribution with the standard, linear
energy function as given by Equation (7) in the main text, written now in
terms of generative model parameters $\gamma,\zeta$ instead of the standard
‘couplings’ and ‘biases’ $J,h$:

$\displaystyle E_{async}(\tilde{\sigma})$ $\displaystyle=-\gamma\sum_{\langle
i,j\rangle}\sigma_{i}\sigma_{j}-\zeta\sum_{i}\sigma_{i}$ (C.41)