# Hausdorff dimension of fermions on a random lattice

Mattia Varrone<EMAIL_ADDRESS>Department of Applied Mathematics and
Theoretical Physics, University of Cambridge  William E. V. Barker
<EMAIL_ADDRESS>Astrophysics Group, Cavendish Laboratory, JJ Thomson Avenue,
Cambridge CB3 0HE, UK  Kavli Institute for Cosmology, Madingley Road,
Cambridge CB3 0HA, UK

###### Abstract

Geometric properties of lattice quantum gravity in two dimensions are studied
numerically via Monte Carlo on Euclidean Dynamical Triangulations. A new
computational method is proposed to simulate gravity coupled with fermions,
which allows the study of interacting theories on a lattice, such as non-
Riemannian gravity models. This was tested on Majorana spinors, where we
obtained a Hausdorff dimension $d_{W}=4.22\pm 0.03$, consistent with the
bounds from the literature ${4.19<d_{H}<4.21}$.

##  1 Introduction

Lattices are a natural setting to study strongly interacting quantum field
theories, and gravity is no exception. Indeed, discrete systems with finite
size have a well-defined path integral, and in theories with Euclidean
signature, every field configuration is associated with a Boltzmann factor
carrying a probability interpretation. In this paper, we discretize 2D
Euclidean gravity on spherical topology coupled with fermions and we generate
an ensemble of possible geometries via dynamical triangulation, using Markov
Chain Monte Carlo (MCMC) techniques, as prescribed in [1, 2]. The resulting
manifold, called a simplicial manifold, reproduces the critical exponents of
Liouville quantum gravity when the system is coupled with conformal matter
[3]. The methods used here will be founded in the triangulated fermion
construction of Burda, Bogacz, Jurkiewicz, Krzywicki, Petersen and Petersson
(BBJKPP) in their seminal work [4, 5, 6, 7, 8, 9, 10]. We will use a similar
constructions to simulate free Majorana spinors and measure their effect on
the Hausdorff dimension of the manifold. In the past, this was always achieved
by mapping the fermionic system to a proxy Ising model [6, 7] (indeed, both
Majorana spinors and critical the Ising model have central charge
$c=\frac{1}{2}$). Instead, we decided to compute the spinorial action directly
by calculating the determinant of the Dirac–Wilson operator, as suggested in
[7]. This has the advantage of allowing for more general interactions between
gravity and matter, such as through non-Riemannian gauge fields or higher-
point fermionic vertices. Using this method, we compute a value of the
Hausdorff dimension $d_{W}=4.22\pm 0.03$ which differs from the previous value
computed by Bogacz and Burda in [7] $d_{W}=2.87$, while favouring the
conjectural relations between the Hausdorff dimension and central charge
proposed by Watabiki in [11], yielding $d_{W}\approx 4.2122$, and it is
consistent with the bounds rigorously derived by Gwynne $4.1892<d_{W}<4.2156$
[12].
The paper content is organized as follows: firstly, we introduce the essential
ingredients to represent fermions on EDT. Secondly, we discuss how Majorana
spinors affect the path integral and give different weights to triangulations.
Then our measurements for the Hausdorff dimension are presented and compared
to the literature. Finally, generalizations of the model are proposed.

##  2 Local frames and spinor transport

To represent fermions on a lattice we require local frames. On a spherical
triangulation, this is achieved simply by defining a right-handed basis of
orthonormal vectors $e_{i1}$ and $e_{i2}$ at the centre of every triangle $i$.
We will use the convention proposed in [6], as presented in figure 1. To
complete the spin structure, we also need a way to map the frames of
neighbouring triangles: a parallel transport. For triangles $i$ and $j$, we
define the unit vector $n_{ji}$ pointing in direction $i\to j$, and the angles
$\phi_{ji}$ and $\phi_{ij}$, measured clockwise respectively from the frames
$e_{i1}$ and $e_{j1}$ to $n_{ji}$.

Figure 1: The local frames associated with two neighbouring triangles, $i$ and
$j$, are represented by orthonormal basis vectors $e_{1}$ and $e_{2}$. The
connecting vector $n_{ij}$ is also shown. Since the triangles are equilateral,
we can deduce from the figure that: $\phi_{ji}=\pi$,
$\phi_{ij}=\frac{5\pi}{3}$, and $\Delta\phi_{ij}=\frac{5\pi}{3}$

.

For propaedeutic purposes, let us first consider the vector parallel transport
from triangle $i$ to $j$ with relative angle $\Delta\phi_{ij}$, which is
generated by the element of the $\mathfrak{so}(2)$ Lie algebra $\epsilon$, the
standard 2D antisymmetric tensor

$U_{ij}\equiv
e^{\epsilon\Delta\phi_{ij}}=\begin{pmatrix}\cos{\Delta\phi_{ij}}&\sin{\Delta\phi_{ij}}\\\
\\\ -\sin{\Delta\phi_{ij}}&\cos{\Delta\phi_{ij}}\end{pmatrix}.$ (1)

Following Bogacz’s prescription, we choose a representation of fermions such
that the spinor parallel transport operator $i\to j$, $\>\mathcal{U}_{ij}$,
satisfying $\mathcal{U}_{ij}^{2}=U_{ij}$. Hence, we obtain

$\mathcal{U}_{ij}\equiv
s_{ij}\>e^{\frac{\epsilon\Delta\phi_{ij}}{2}}=s_{ij}\begin{pmatrix}\cos{\frac{\Delta\phi_{ij}}{2}}&\sin{\frac{\Delta\phi_{ij}}{2}}\\\
\\\
-\sin{\frac{\Delta\phi_{ij}}{2}}&\cos{\frac{\Delta\phi_{ij}}{2}}\end{pmatrix},$
(2)

where $\Delta\phi_{ij}\equiv\phi_{ij}-\phi_{ji}+\pi$ and the factors $s_{ij}$,
taking values $+1$ or $-1$, are called sign flags. Moreover, the parallel
transport from $i\to j$ and then $j\to i$ leaves spinors unchanged, we must
obtain the identity $\mathbbm{1}$, hence

$\mathcal{U}_{ij}\mathcal{U}_{ji}=s_{ij}s_{ji}\>e^{\frac{\Delta\phi_{ij}+\Delta\phi_{ji}}{2}\epsilon}=s_{ij}s_{ji}\>e^{\pi\epsilon}=-s_{ij}s_{ji}\mathbbm{1},$
(3)

implying

$s_{ij}s_{ji}=-1\Leftrightarrow s_{ij}=-s_{ji}.$ (4)

Unsurprisingly, spinor parallel transport has the effect of rotating field
components by half the amount in comparison to vector parallel transport. The
ambiguity in the value of the sign flags is resolved by enforcing the physical
consistency of spinor transport. Indeed, the trace of the parallel transport
along an elementary counter-clockwise $n$-loop around a vertex $P$ over a set
of triangles $\\{i_{1},i_{2},...,i_{n},i_{1}\\}$ (in lattice gauge theory,
this is referred to as a plaquette) can be related to the curvature at $P$,
characterized by a deficit angle $\Delta_{P}$. In particular, one obtains:

$\Pi_{P}\equiv\frac{1}{2}\text{Tr}\;\mathcal{U}_{i_{n}i_{1}}...\mathcal{U}_{i_{2}i_{3}}\mathcal{U}_{i_{1}i_{2}}=S_{P}\cos{\frac{\Delta_{P}}{2}}.$
(5)

We recall that the deficit angle is defined to be the difference between a
full circle $2\pi$ and the total angle obtained by summing the angles having
$P$ as a vertex, so for a $n$-plaquette on an equilateral triangulation we
have $\Delta_{P}=2\pi-\frac{n\pi}{3}$. The term $S_{P}$ can take values $+1$
or $-1$, depending on all elementary transports on the plaquette. On physical
grounds, as explained in [6], we require the following for all elementary
loops:

$S_{P}=+1,\qquad\forall P.$ (6)

Conditions (4) and (6) determine the sign flags uniquely, and they must be
preserved throughout the evolution of the system. We will show in section 2.2
how to construct a suitable initial sign flag configuration and how to update
it as the system evolves.
This representation is compatible with the Majorana representation of the
gamma matrices

$\gamma_{1}=\sigma_{3}\equiv\begin{pmatrix}1&0\\\
0&-1\end{pmatrix},\qquad\gamma_{2}=\sigma_{1}\equiv\begin{pmatrix}0&1\\\
1&0\end{pmatrix}.$ (7)

It is useful to recall how the $c=1/2$ conformal field theory emerges from
among the Majorana components $\Psi_{\alpha}$ in (17), for which we can write
$\Psi^{\mathrm{T}}=(\Psi_{1},\Psi_{2})$. In the continuum, and with the real
Clifford basis chosen in (7), we hope to recover the free Majorana Lagrangian
in Cartesian coordinates

$L=\frac{1}{2}\bar{\Psi}\not{\partial}\Psi=\frac{1}{2}i\Psi^{\mathrm{T}}\sigma_{2}\left(\sigma_{3}\partial_{x}+\sigma_{1}\partial_{y}\right)\Psi.$
(8)

Using the complexified coordinates $z\equiv x+iy$ and $\bar{z}\equiv x-iy$
with $\partial_{z}\equiv\frac{1}{2}\left(\partial_{x}-i\partial_{y}\right)$
and
$\partial_{\bar{z}}\equiv\frac{1}{2}\left(\partial_{x}+i\partial_{y}\right)$,
we can obtain from (8) the Lagrangian in the component form

$\displaystyle L=$ $\displaystyle\
\frac{i}{2}\left(\Psi_{1}+i\Psi_{2}\right)\partial_{z}\left(\Psi_{1}+i\Psi_{2}\right)$
(9) $\displaystyle\
+\frac{i}{2}\left(i\Psi_{1}+\Psi_{2}\right)\partial_{\bar{z}}\left(i\Psi_{1}+\Psi_{2}\right).$

The terms in (9) can be identified with a new pair of Grassmann numbers,
related to the original variables by a transformation with unit determinant,
which does not affect the functional measure in the path integral. In terms of
these new variables the field equations become

$\partial_{z}\left(\Psi_{1}+i\Psi_{2}\right)/\sqrt{2}=0,\quad\partial_{\bar{z}}\left(i\Psi_{1}+\Psi_{2}\right)/\sqrt{2}=0,$
(10)

and these act to enforce antiholomorphic and holomorphic solutions.

###  2.1 General spin-connections

As a side note, we mention that our work is motivated partly by the aim of
introducing more general gravitational connections on triangulations,
particularly of non-Riemannian character. For this purpose, we note that in 2D
the most general spin connection components $\omega_{abc}$ in a local frame
are given by [13]

$\omega_{abc}=\epsilon_{ab}\Omega_{c}.$ (11)

Moreover, if we allow the spin connection to be torsionful, we can decompose
it into the Ricci rotation coefficients $\Gamma_{abc}\equiv\epsilon_{ab}A_{c}$
and the contorsion tensor $K_{abc}\equiv\epsilon_{ab}\mathcal{A}_{c}$ [14], as

$\omega_{abc}=\Gamma_{abc}+K_{abc}=\epsilon_{ab}(A_{c}+\mathcal{A}_{c}).$ (12)

If we consider parallel transport of vector fields, we have a vector
connection proportional to the 2D rotation generator $\epsilon$, given by
${\omega}$${}_{c}=(A_{c}+\mathcal{A}_{c})\epsilon$. Therefore, the parallel
transport operator on the triangulation between two triangles $i$ and $j$, in
the direction $n_{ji}$, reads

$U_{ij}^{(\omega)}=e^{\epsilon\int_{i}^{j}dx(n_{ji})^{c}(A_{c}+\mathcal{A}_{c})}=e^{(\Delta\phi_{ij}+\theta_{ij})\epsilon}.$
(13)

Accordingly, the Ricci rotation coefficients determine the rotation angle
$\Delta\phi_{ij}$ for vector field components in the local frame basis, and
the contorsion contributes an additional angle $\theta_{ij}$. For the
transport of spinors, we obtain an analogous formula:

$\mathcal{U}_{ij}^{(\omega)}=s_{ij}\>e^{\frac{\Delta\phi_{ij}+\theta_{ij}}{2}\epsilon}=s_{ij}\begin{pmatrix}\cos{\frac{\Delta\phi_{ij}+\theta_{ij}}{2}}&\sin{\frac{\Delta\phi_{ij}+\theta_{ij}}{2}}\\\
\\\
-\sin{\frac{\Delta\phi_{ij}+\theta_{ij}}{2}}&\cos{\frac{\Delta\phi_{ij}+\theta_{ij}}{2}}\end{pmatrix}.$
(14)

Therefore, the presence of "torsion" requires us to keep track of additional
variables $\theta_{ij}$ linking neighbouring triangles. These introduce
additional degrees of freedom in our description of physics, and imply that we
have gauged the group $SO(2)$.

###  2.2 Fixing sign flags

If we discretize euclidean 2D space as a spherical triangulation with a fixed
number of triangles $N$, all possible geometries can be generated from MCMC
with a finite number of flip moves (also known as $(2,2)$-moves) as described
by Ambjørn in [15] and Budd in [1]. The effect of such moves is shown
schematically in figure 2.

Figure 2: A flip move changes the connectivity of triangles from configuration
I to II. Labels $1$ and $2$ identify the triangles before and after the
transformation, while latin letters indicate their edges.

To simulate the effect of fermions on spacetime, the quantities encoding the
spin structure also need to evolve consistently. In the rest of this section,
we will outline an algorithm to update sign flags $s_{ij}$ as flip moves are
performed, while preserving the validity of conditions (4) and (6). In
addition, we will construct a consistent initial configuration of triangles
and sign flags, which will serve as a starting point for MCMC simulations. For
later convenience, we will label parallel transporters and sign flags by the
edges. For example, referring to configuration I in figure 2, the sign flag
associated with the link $1\to 2$ will be relabelled using the index of the
connecting edge in the starting triangle: $s_{12}\rightarrow s_{i}$.
Now, assume that we start from a valid configuration of spin flags on a
certain triangulation, and that we perform a flip move, affecting the
connectivity of two triangles, labelled as in figure 2. This is a local
change, and therefore it only affects the validity of the consistency
conditions for edges and vertices of triangles $1$ and $2$. We will proceed by
first fixing condition (4), with the following assignment of sign flags, in
the specified order:

$\displaystyle\begin{split}s^{\text{new}}_{i}\to&\ -s^{\text{old}}_{n},\\\
s^{\text{new}}_{k}\to&\ -s^{\text{old}}_{m},\\\ s^{\text{new}}_{m}\to&\
-s^{\text{new}}_{k},\\\ s^{\text{new}}_{l}\to&\ -s^{\text{old}}_{i},\\\
s^{\text{new}}_{j}\to&\ -s^{\text{new}}_{l}.\\\ \end{split}$ (15)

Next, we must ensure that condition (6) is met by all the modified plaquettes,
which are those corresponding to the vertices $P,Q,R,S$ as shown in figure 3.
We devised the following algorithm: starting from vertex P, we compute the
trace of the plaquette $\Pi_{P}$ and the deficit angle $\Delta_{P}$. From
these, the sign $S_{P}$ can be deduced using equation (5). Now, since the
parallel transporter $\mathcal{U}_{i}$ contributes to $\Pi_{P}$, we can update
the sign flags

$\displaystyle\begin{split}s_{i}\to s_{i}\cdot S_{P}\\\ s_{n}\to-
s_{i},\end{split}$ (16)

where the second equation ensures that condition (4) remains true.

Figure 3: After a flip move is performed, we can restore the consistency
condition of the flag configuration by calculating the plaquettes
corresponding to vertices, $P,Q,R$ and sequentially updating the sign flags of
edges $i,j,k$.

We can repeat these steps for vertices $Q$ and $R$, and modify $s_{j},s_{k}$
accordingly, since these flags do not affect the plaquettes previously fixed.
Finally, a combinatoric argument layed out from Burda in [4] guarantees that
if all but one elementary loop in a triangulation of a sphere are known to
satisfy conditions (4) and (6), then also the last loop satisfies them, hence
we are done.

Now we are only left to show that a suitable triangulation with a consistent
sign flag configuration exists. Let us consider the explicit construction
given by the fan triangulation shown in figure 4.

Figure 4: A fan triangulation composed of $N$ triangles is shown (the edge
indexing convention can be found in [1]). In green, we denoted edges carrying
a positive sign flag $s=+1$, while flags $s=-1$ correspond to red edges. The
direction of frame vectors $e_{1}$ is shown by cyan lines. This unambiguously
identifies the spin structure of our triangulation. Labels $P$ and $Q$ denote
the polar and equatorial vertices respectively.

Here, we labeled with green the edges carrying a positive sign flag $s=+1$,
while flags $s=-1$ correspond to red edges. The direction of frame vectors
$e_{1}$ is shown by cyan lines. This unambiguously identifies the spin
structure of our triangulation. We will prove that the fan satisfies the
consistency conditions whenever we choose a triangulation of size $N$, with
$N\equiv 2\;(\text{mod}4)$. We begin by noticing that we can classify vertices
into only two types: equatorial vertices, denoted by $Q$, are shared by two
triangles, and two polar vertices, denoted by $P$, shared by all $N$
triangles. By the translation symmetry of the fan triangulation, we deduce
that plaquettes associated with vertices Q share the same sign $S_{Q}$. South
and north pole are also equivalent due to mirror symmetry.
Let us calculate $S_{Q}$ for the vertex connected to the triangle with edges
$0,1,2$. From formula (5) and the property of rotation matrices we know the
plaquette’s trace is given by
$\Pi_{Q}=s_{1}s_{4}\cos{\frac{\Delta\phi_{1}+\Delta\phi_{4}}{2}}=-\cos{\frac{\Delta\phi_{1}+\Delta\phi_{4}}{2}}$.
The spinor transport from edge $1$ to $5$ gives is associated to
$\frac{\Delta\phi_{1}}{2}=\frac{1}{2}(\frac{5\pi}{3}-\frac{\pi}{3}+\pi)=\frac{7\pi}{6}$,
and transport from edge $4$ to $2$ similarly gives
$\frac{\Delta\phi_{1}}{2}=\frac{7\pi}{6}$, hence
$\Pi_{Q}=-\cos(\frac{7\pi}{3})=-\frac{1}{2}$. On the other hand, the angle
deficit is $\Delta_{Q}=\frac{4\pi}{3}$, thus
$\cos{\frac{\Delta_{Q}}{2}}=-\frac{1}{2}=\Pi_{Q}$. Therefore, $S_{Q}=+1$.
Let us now check the same for the polar vertex. Due to the repeating pattern
of the plaquette, we only need to compute:
$\frac{\Delta\phi_{2}}{2}=\frac{1}{2}(\frac{\pi}{3}-\frac{5\pi}{3}+\pi)=-\frac{\pi}{6}$
and $\frac{\Delta\phi_{3}}{2}=\frac{1}{2}(\pi-\pi+\pi)=\frac{\pi}{2}$. Since
there are $\frac{N}{2}$ tuples of triangles contributing the same parallel
transport, we get
$\Pi_{P}=(-1)^{\frac{N}{2}}\cos{\frac{N}{2}(\frac{\Delta\phi_{2}+\Delta\phi_{3}}{2})}=-(-1)^{\frac{N}{2}}\cos{\frac{N\pi}{6}}$.
The deficit angle can be calculated easily as
$\Delta_{P}=2\pi-\frac{N\pi}{3}$, thus
$\cos{\frac{\Delta_{P}}{2}}=-\cos{\frac{N\pi}{6}}$. Finally, these quantities
satisfy the consistency relations if $N\equiv 2\;(\text{mod}4)$, and this
concludes the proof.

##  3 Majorana spinors and path integral

Let us now discuss how Majorana spinors are coupled with gravity on the
lattice. On a given triangulation $T$, fermionic fields are characterized by
degrees of freedom $\Psi_{i}$ and $\bar{\Psi}_{i}$ sitting at the centres of
triangles $i$. A Majorana spinor $\Psi$ carries Grassmann components
$\Psi_{\alpha}$, denoted by a spinor index $\alpha=1,2$ while the components
of its conjugate spinor $\bar{\Psi}$ are represented with an upper index
$\Psi^{\beta}$ and they are related via the Majorana condition

$\Psi^{\beta}=\epsilon^{\beta\alpha}\Psi_{\alpha}\;\leftrightarrow\;\Psi_{\alpha}=\Psi^{\beta}\epsilon_{\beta\alpha}.$
(17)

The condition above implies that $\bar{\Psi}$ belongs to the dual space with
respect to $\Psi$, and the antisymmetric tensor relates the two via charge
conjugation. With the definition of parallel transport (2) and our choices of
representation (7) and (17), we have now all the ingredients to build an
action for free fermions coupled to a triangulation $T$. This reads

$S_{T}=-K\sum_{\langle
ij\rangle}\bar{\Psi}_{i}H_{ij}\Psi_{j}+\frac{1}{2}\sum_{i}\bar{\Psi}_{i}\Psi_{i}.$
(18)

The first contribution is a regularised kinetic term, called hopping term,
with $K$ a constant and
$H_{ij}=\frac{1}{2}(1+n_{ij}^{a}\>\gamma_{a})\>\mathcal{U}_{ij}$. The second
mass term is a counter-term introduced in order to cancel the effective mass
induced by the presence of a lattice, and tune the systems to criticality.
Conformal symmetry is essential to reproduce the behaviour expected from
Liouville theory, and this is achieved when $K\approx 0.3746$ [7]. We can
rewrite the action in a more compact form by employing the Dirac-Wilson
operator $D_{ij}$. Making spinor indices explicit

$\displaystyle\begin{split}S_{T}=\sum_{i,j}\bar{\Psi}_{i}D_{ij}\Psi_{j}=\sum_{i,j}{\Psi}_{i}^{\alpha}[D_{ij}]^{\;\;\beta}_{\alpha}\Psi_{j\beta},\quad\text{where}\\\
[D_{ij}]^{\;\;\beta}_{\alpha}=\frac{1}{2}\delta_{ij}\delta^{\;\;\beta}_{\alpha}-KP_{ij}[H_{ij}]^{\;\;\beta}_{\alpha}.\end{split}$
(19)

Here, $P_{ij}=1$ if $i$ and $j$ are neighbours on the triangulation $T$, and
$0$ otherwise. Note that the summation convention holds for spinor indices.
Using Majorana relation (17), we can express the action using independent
degrees of freedom

$S_{T}=\sum_{i,j}\Psi_{i\alpha}[D_{ij}]^{\alpha\beta}\Psi_{j\beta}\quad\rightarrow\quad[D_{ij}]^{\alpha\beta}=\frac{1}{2}\delta_{ij}\epsilon^{\alpha\beta}-KP_{ij}\epsilon^{\alpha\gamma}[H_{ij}]^{\;\;\beta}_{\gamma}$
(20)

Now, in order to make the indexing of this quantity clearer we repackage pairs
of triangle and spinor indices into a single index, denoted by capital latin
letters: $(i,\alpha)\rightarrow A:=2(i-1)+\alpha$. Note that this map is $1$
to $1$ since $\alpha=2-(A\;(\text{mod}\;2))$ and $i=1+\frac{A-\alpha}{2}$.
Therefore, this induces a bijection for spinorial quantities:

$\displaystyle\begin{split}\Psi_{i\alpha}\rightarrow&\ \Psi_{A},\\\
[D_{ij}]^{\alpha\beta}\rightarrow&\ D_{AB},\end{split}$ (21)

where $A$ and $B$ are integers from $\\{1,2,...2N\\}$, where $N$ is the size
of the triangulation. The Dirac-Wilson operator is therefore represented as an
antisymmetric $2N\times 2N$ matrix, and we are now ready to write the path
integral of the theory as the sum over possible triangulations and Grassmann
field configurations

$Z=\sum_{T}Z_{T}=\sum_{T}\int\prod_{A}d\Psi_{A}e^{-\sum_{A,B}\Psi_{A}D_{AB}\Psi_{B}}.$
(22)

Here $Z_{T}$ represents the fermionic path integral for a given triangulation
$T$ and corresponds to the relative weight of the related geometry. This can
be easily evaluated as the Pfaffian of the Dirac operator (ref)

$Z_{T}=\text{Pf}\>D_{AB}(T)=\sqrt{\text{Det}D_{AB}(T)}$ (23)

Where we introduced the argument $T$ to make explicit the dependence of the
triangulation. Note that in (23) we implicitly used the positivity of the
determinant, which was proved by Burda in [4]. In practice, when simulating
the theory with MCMC, e.g. through the Metropolis–Hastings algorithm [1], one
computes the log of the determinant, since a direct computation of $D_{AB}$ is
prone to cause floating-point overflow. This motivates the definition of a
triangulation effective action

$S_{T}^{\\{eff\\}}=-\ln{Z_{T}}=-\frac{1}{2}\text{logDet}D_{AB}(T).$ (24)

##  4 Hausdorff dimension

To test our method, we measured the scaling behaviour of the distance profile
$\rho_{T}(r)$, a well-studied geometric observable. In this section we will
provide a brief description of its main properties, Let us first introduce a
concept of distance on a triangulation: the graph distance $d_{T}$ between two
vertices $x$ and $y$ is defined to be the number of edges on the shortest path
connecting $x$ and $y$. However, one should note that this form of distance
converges, for large triangulations, to the geodesic distance, a more natural
concept on smooth manifolds [1]. Now we can define the distance profile

$\rho_{T}(r)\equiv\frac{1}{\mathcal{N}}\sum_{x,y}\mathbbm{1}_{(d_{T}(x,y)=r)}$
(25)

Where $\mathcal{N}=\frac{N+4}{2}$is the number of vertices in a triangulation
of size $N$. For an infinite system, the distance profile, averaged over all
triangulations, has the scaling behaviour [16]

$\lim_{N\to\infty}\mathbb{E}[\rho_{T}(r)]\propto r^{d_{W}}$ (26)

where the critical exponent $d_{W}$ is called the Hausdorff dimension, and it
encodes the increase in connectivity with distance on a manifold. For finite
triangulations, the distance profile can only have support up to some maximum
distance and therefore peaks at some value. This behaviour is characterized by
the finite-size scaling relation [15, 17]

$\max_{r}\mathbb{E}[\rho_{T}(r)]\propto N^{1-\frac{1}{d_{W}}}.$ (27)

##  5 Results

We computed the Hausdorff dimension $d^{\text{spinor}}_{W}=4.22\pm 0.03$ for a
system of free spinors using the methods outlined in the previous sections, by
fitting $d_{W}$ in the formula (27), employing triangulations of sizes ranging
from $80$ to $500$ triangles. Similarly, we also computed, the Hausdorff
dimension of the critical Ising model, yielding the same result
$d^{\text{Ising}}_{W}=4.21\pm 0.03$. This is to be expected since both systems
have a central charge $c=\frac{1}{2}$.
Our values disagree with the previous estimates from Bogacz in [7], i.e.
$d^{B}_{W}=2.87$. An important remark is that the Hausdorff dimension in [7]
was obtained with a different method, i.e. by simulating an Ising model tuned
to be dynamically equivalent to spinors. Therefore, our fit for the Ising
model, in line with further supports the thesis. Our measurements are
consistent with the conjectural relation between the Hausdorff dimension and
central charge proposed by Watabiki in [11], yielding $d_{W}\approx 4.2122$,
and it is consistent with the bounds rigorously derived by Gwynne
$4.1892<d_{W}<4.2156$ [12], and with the state of the art numerical results
from Budd, Ambjørn and Barkley [18, 19]. As a practical note, we point out
that computational complexity for the MCMC simulation of free fermions as we
detailed scales as $N^{3}$, with $N$ the size of the triangulation. This is
owed to the necessity to compute the determinant of the Dirac–Wilson operator,
and it poses a limit to its viability for lattices of size greater than
$O(1000)$. For reference, in order to estimate the Hausdorff dimension of the
free spinor model on lattices of size 500, we required approximately four
hours of CPU time for a common laptop.

##  6 Outlook for non-Riemannian models

To illustrate the greater flexibility of our fermion implementation, based on
the Dirac-Wilson operator, we study in this section a slightly more elaborate
interacting model, where a Dirac spinor $\psi$ is coupled with gravity through
a torsionful connection for spinors
$\>\omega_{a}=\frac{1}{2}(\Gamma_{a}+K_{a})\epsilon$, with contorsion
components $K_{a}$, described by the following matter lagrangian

$\mathcal{L}=\bar{\psi}\cancel{D}\psi+\frac{1}{2}\mu
K_{a}K^{a}=\bar{\psi}\gamma^{a}(e_{a}^{\nu}\partial_{\nu}+\frac{\Gamma_{a}}{2}\epsilon+\frac{K_{a}}{2}\epsilon)\psi+\frac{1}{2}\mu
K_{a}K^{a}$ (28)

where $e_{a}^{\nu}$ is the zweibein field, relating the vector coordinate
basis to a local frame basis [14]. This describes a Poincare gauge theory with
non-dynamical torsion, as extensively discussed in Blagojevic’s work [14]. The
equation of motion for the contorsion reads

$\mu K^{a}=\frac{1}{2}\bar{\psi}\gamma^{a}\epsilon\psi.$ (29)

Since this equation provides an algebraic constraint for $K_{a}$, we expect
that the theory

$\mathcal{L^{\prime}}=\mathcal{L}-g\bar{\psi}\gamma^{a}\epsilon\psi\>\bar{\psi}\gamma_{a}\epsilon\psi$
(30)

reproduces the dynamics of a free Dirac spinor, and therefore becomes
conformal, when

$g=g_{crit}\equiv\frac{1}{8\mu}.$ (31)

As a consequence, we could study critical $\mathcal{L^{\prime}}$ theory on EDT
with the following path integral associated to a given triangulation $T$

$Z_{T}=\int\prod_{i}d\bar{\psi}d\psi e^{-S_{T}}.$ (32)

The action $S_{T}$ on EDT can be determined through a similar discretization
scheme as the one we used for equation (18). This time, the action also
contains a four-point interaction term for the spinors, so we can write it,
schematically, as

$S_{T}=S_{K}+\sum_{i,j}\bar{\psi}_{i}D_{ij}\psi_{j}-\sum_{i}g\bar{\psi}_{i}\gamma^{a}\epsilon\psi_{i}\>\bar{\psi}_{i}\gamma_{a}\epsilon\psi_{i}$
(33)

where we define $S_{K}$ to be the contribution to the action that only depends
on the contorsion, and $D_{ij}$ is an analogous term to the Dirac–Wilson
operator in (19), this time also accounting for torsion effects. The path
integral (32) can then be computed as a series in powers of $g$ and simulated
with MCMC in a similar way as with free fermions.

##  7 Conclusion

In this paper we proposed a new method for MCMC simulations of spinors on 2D
euclidean dynamical triangulations, by computing the determinant of the Dirac-
Wilson operator. This was tested numerically by fitting the Hausdorff
dimension ${d^{\text{spinor}}_{W}=4.22\pm 0.03}$, which agrees with the
current theoretical bounds from [12] and disproves the previous measurements
in [7]. Finally, we showed how our technique can be generalized to study
interacting theories and non-Riemannian features of gravity on the lattice.

## Acknowledgements

We are grateful for vital conversations with Zdzisław Burda and the useful
correspondence with Timothy Budd, that provided key inputs for our work. This
work was performed using resources provided by the Cambridge Service for Data
Driven Discovery (CSD3) operated by the University of Cambridge Research
Computing Service (www.csd3.cam.ac.uk), provided by Dell EMC and Intel using
Tier-2 funding from the Engineering and Physical Sciences Research Council
(capital grant EP/T022159/1), DiRAC funding from the Science and Technology
Facilities Council (www.dirac.ac.uk), and ERC funding granted to Will Handley.

W.E.V.B. is grateful for the kind hospitality of Leiden University and the
Lorentz Institute, and the support of Girton College, Cambridge.

## References

  * [1] Timothy Budd.  Monte carlo techniques.  2022\.
  * [2] François David.  Planar diagrams, two-dimensional lattice gravity and surface models.  Nuclear Physics B, 257:45–58, 1985.
  * [3] Vadim G Knizhnik, Alexander M Polyakov, and Alexander B Zamolodchikov.  Fractal structure of 2d—quantum gravity.  Modern Physics Letters A, 3(08):819–826, 1988.
  * [4] Z Burda, J Jurkiewicz, and A Krzywicki.  Wilson fermions on a randomly triangulated manifold.  Physical Review D, 60(10):105029, 1999.
  * [5] Z Burda, J Jurkiewicz, and A Krzywicki.  Fermions on random lattices.  arXiv preprint hep-lat/9907013, 1999.
  * [6] L Bogacz, Z Burda, J Jurkiewicz, A Krzywicki, C Petersen, and Bengt Petersson.  Dirac operator and ising model on a compact 2d random lattice.  arXiv preprint hep-lat/0110063, 2001.
  * [7] L Bogacz, Z Burda, C Petersen, and Bengt Petersson.  Spectrum of the dirac operator coupled to two-dimensional quantum gravity.  Nuclear Physics B, 630(1-2):339–358, 2002.
  * [8] L Bogacz, Z Burda, and J Jurkiewicz.  Fermions in 2d lorentzian quantum gravity.  arXiv preprint hep-lat/0306033, 2003.
  * [9] Sven Bilke, Z Burda, A Krzywicki, Bengt Petersson, J Tabaczek, and G Thorleifsson.  4d simplicial quantum gravity interacting with gauge matter fields.  Physics Letters B, 418(3-4):266–272, 1998.
  * [10] Z. Burda and J. Jurkiewicz.  The Ising Model on a Random Lattice With a Coordination Number Equal 3\.  Acta Phys. Polon. B, 20:949, 1989.
  * [11] Yoshiyuki Watabiki.  Analytic study of fractal structure of quantized surface in two-dimensional quantum gravity.  Progress of Theoretical Physics Supplement, 114:1–17, 1993.
  * [12] Ewain Gwynne and Joshua Pfeffer.  Bounds for distances and geodesic dimension in liouville first passage percolation.  Electronic Communications in Probability, 24:1–12, 2019.
  * [13] M Blagojevic, M Vasilic, and T Vukasinac.  Asymptotic symmetry and conservation laws in the two-dimensional poincaré gauge theory of gravity.  Classical and Quantum Gravity, 13(11):3003, 1996.
  * [14] Milutin Blagojevic.  Gravitation and gauge symmetries.  CRC Press, 2001.
  * [15] Jan Ambjørn, Jerzy Jurkiewicz, and Renate Loll.  Lorentzian and euclidean quantum gravity—analytical and numerical results.  M-Theory and Quantum Geometry, pages 381–450, 2000.
  * [16] Timothy Budd.  Lessons from the Mathematics of Two-Dimensional Euclidean Quantum Gravity.  12 2022.
  * [17] JF LG.  M, g. scaling limits of random planar maps with large faces.  Ann. Probab.,().
  * [18] J Ambjørn and Timothy Budd.  The toroidal hausdorff dimension of 2d euclidean quantum gravity.  Physics Letters B, 724(4-5):328–332, 2013.
  * [19] Jerome Barkley and Timothy Budd.  Precision measurements of hausdorff dimensions in two-dimensional quantum gravity.  Classical and Quantum Gravity, 36(24):244001, 2019.


# Do the Angles of a Triangle Add up to 180$\degree$? – Introducing Non-
Euclidean Geometry

Hanne Kekkonen
EEMCS, Delft University of Technology, Netherlands<EMAIL_ADDRESS>
###### Abstract

How can we convince students, who have mainly learned to follow given
mathematical rules, that mathematics can also be fascinating, creative, and
beautiful? In this paper I discuss different ways of introducing non-Euclidean
geometry to students and the general public using different physical models,
including chalksphere, crocheted hyperbolic surfaces, curved folding, and
polygon tilings. Spherical geometry offers a simple yet surprising
introduction to the topic, whereas hyperbolic geometry is an entirely new and
exciting concept to most. Non-Euclidean geometry demonstrates how crafts and
art can be used to make complex mathematical concepts more accessible, and how
mathematics itself can be beautiful, not just useful.

## Do the Angles of a Triangle Add up to 180$\degree$?

One of the fundamental truths every child learns in school is that the angles
of a triangle add up to 180$\degree$. By the time you graduate from high
school, you have probably seen at least three proofs (likely without
mathematical rigour) of this fact, starting from cutting or folding a paper
triangle to show that when the angles meet in one point, the result is a
straight angle. But is it really true for every triangle you could possibly
draw? Posing this question to students forces them to think outside the box in
which they have been encouraged to stay throughout their whole mathematics
education. To help them, one can elaborate: is it really a property of the
triangle or perhaps a property of the sheet of paper or chalkboard you drew
the triangle on?

Figure 1: A chalksphere.

Even though we usually draw and write on flat surfaces, we live on a globe.
So, imagine that Alice and Bob are stationed at the north pole and Alice
travels directly to the equator. Next, Bob starts his journey at a 90$\degree$
angle to Alice, travels directly to the equator and then follows the equator
to Alice. All lines of longitude meet the equator at a 90$\degree$ angle,
which means that the created triangle has three right angles. Since we are
familiar with the idea of a globe, this example is easy to understand, even
for a younger audience, but still offers a surprise for most. You can then
ponder with the students; what do straight lines on spheres look like? The
simplest way of seeing that straight lines are described by the great circles,
that divide the sphere in two equal parts, is to consider a straight line
segment as the shortest way between two points. Some other differences between
the Euclidean geometry, which we learn in school, and the spherical geometry
are that there are no parallel lines, and that the circumference of any circle
is less than $2\pi r$. A chalksphere or globe offers a great tool for the
students to explore spherical geometry, see Figure 1111All the models and
pictures presented in this article are by the author..

The next step is to guide the students to think about the difference between a
sphere and a sheet of paper, and help them arrive at the conclusion that,
while paper is flat, a sphere curves. In mathematics this is called curvature.
If you draw a tilde $\sim$ you notice that one part of the curve looks like a
hill while another part looks like a valley. To distinguish hills from valleys
we can assign one of them with positive and the other one with negative
curvature. The above example describes one dimensional curvature but how does
this translate to two dimensions? If you place your finger on a ball you
notice that the ball curves away from it in every direction. If you did the
same inside the ball you would notice that there the ball curves towards your
hand in every direction. This is why we say that spheres have positive
curvature. Comparing an American football and a perfectly round ball we notice
that the first one is more curved at the tips than middle, while the second
one looks the same everywhere. This means that spheres have constant positive
curvature.

## Hyperbolic Surfaces

We have now discussed surfaces with zero and positive curvature, which might
lead one to wonder: does a surface with negative curvature exist and how would
it look? To attain negative curvature the surface would need to curve as a
hill in one direction and as a valley in the other, creating a saddle shape

Figure 2: Three crocheted hyperbolic surfaces (a) and a hyperbolic chalk
surface (b).

(or a Pringles potato chip). If the curvature is constant we call such a
surface hyperbolic. Another way of demonstrating the difference between
spheres, flat surfaces and hyperbolic planes is their size. If you try to
flatten a half sphere you end up either ripping or stretching it since it
covers less area than the plane. Conversely, hyperbolic surfaces are larger
than the plane and to flatten one you have to fold it.

Introducing spherical geometry is simple since we are familiar with spheres,
but how to introduce hyperbolic surfaces to students? The earliest physical
models of hyperbolic surfaces were made by taping identical paper annuli
together, see e.g. [5]. These models are rather fragile and hence would have
fairly limited lifespan in the hands of excited members of the audience.
Inspired by these paper models, Daina Taimina came up with the idea of
crocheting hyperbolic planes to help her differential geometry students [3,
5]. These models are durable and allow students to explore hyperbolic geometry
by playing and folding the surfaces. In Figure 2 (a) three differences between
hyperbolic geometry and Euclidean geometry are demonstrated: the angles of a
triangle add up to less than 180$\degree$, a line has infinitely many parallel
lines through a given point and the circumference of a circle is more than
$2\pi r$. I used a crocheted hyperbolic surface as a starting point for
creating a hyperbolic chalk surface, shown in Figure 2 (b). Crocheted models
are not only useful and durable, they are also beautiful. In addition to
Taimina’s work, a large art installation curated by Margaret and Christine
Wertheim called Hyperbolic Crochet Coral Reef, has been exhibited all around
the world.

A second model, that can be used to explain negative curvature, is created
using a technique called curved folding. Here we start by cutting an annulus
out of paper and drawing concentric circles on it. These circles are then
carefully folded with an alternating pattern of mountains and valleys. Paper
is an elastic

Figure 3: When a flat paper ring is folded along concentric circles the paper
shapes itself into a natural equilibrium form (left). By combining several
curved rings one can create paper sculptures.

material and tries to stay flat unless creased, in which case a plastic
deformation occurs. To balance the two forces (staying flat between folds
while folded at the creases) the above model automatically folds into a curved
3d-shape as shown in Figure 3 on left. Notice how the circumference of a
creased circle stays the same while it moves closer to the origin, making the
radius smaller. The mathematical behaviour (or even existence) of these models
is still an open question [1, 2]. The earliest references to curved folding
sculptures are from 1927 with a student’s work at the Bauhaus. Since then, the
technique has fascinated several artists and mathematicians including Thoki
Yenn, Kunihiko Kasahara, David Huffman, and Erik and Martin Demaine. Even
though the curvature is not necessarily constant, these models offer a
different way of understanding negative curvature and allow students to get
creative.

##  Introducing Curvature with Tiling Models

Another way that can be used to introduce curvature is based on approximating
surfaces by tessellations. The honeycomb pattern found in nature consists of
hexagons laid next to each other creating a flat surface. Notice that the
internal angles of a hexagon are $120\degree$, which means that when three of
them meet in a vertex they add up to $360\degree$, creating a flat surface.
Now, if you start building a surface using a pentagon instead of a hexagon,

Figure 4: Polygon tiling models offer a great way of exploring curvature.

and surround it with five hexagons the surface is too small to lie flat and
starts to curve. The internal angles of a pentagon are $108\degree$, so the
angles at a vertex with two hexagons and one pentagon add up to
$348\degree<360\degree$. If continued, this pattern yields a traditional
soccer ball approximation of a sphere, shown in Figure 4. To create an
approximation of a hyperbolic plane, which is too large to lie flat, we start
with a heptagon (internal angles $128\nicefrac{{4}}{{7}}\degree$) and surround
it with seven hexagons (the angles at a vertex add up to
$368\nicefrac{{4}}{{7}}\degree>360\degree$). This method allows people to
quickly build a paper model of the hyperbolic plane. It also has the advantage
that, while the model is curved, each polygon is made of flat paper and hence
it is possible to draw straight lines on it. The model is excellent for more
hands-on events allowing students to explore hyperbolic geometry by drawing
lines on the models, and seeing how parallel lines diverge and the angles of a
triangle add up to less than 180$\degree$. Frank Sottile describes more
activities with the “hyperbolic football” model explained above [4].

##  Summary and Conclusions

Extracurricular mathematics activities usually consist of mathematical puzzles
or problems that can be solved by creative use of logic and high school
mathematics. Non-Euclidean geometry, on the other hand, introduces an entirely
new concept to the audience. It can be introduced without any formulas and
gives the students a peek into university level mathematics, which can be very
far from the image most people have of mathematics. Spherical geometry is easy
to understand, yet offers a couple of surprises, since most people have only
encountered Euclidean geometry in school. After introducing the concept of
curvature, and the fact that spheres have positive curvature while flat
surfaces have zero curvature, one might start to wonder how a surface with
negative curvature would look like. Non-Euclidean geometry offers an excellent
opportunity for combining mathematics with crafts and art, and hence breaking
some preconceptions of both of them. Students can make their own chalkspheres
and study how map projections will always distort some areas and force
straight lines to look curved. The need for physical models becomes even more
obvious when moving to surfaces with negative curvature. The crochet models
offer a tactile way of exploring hyperbolic geometry, while curved folding
provides a great method for creating paper sculptures. Polygon tiling models
allow for studying hyperbolic geometry by drawing on the created hyperbolic
plane and more ambitious students can experiment with different hyperbolic
tilings to see how they affect the outcome.

I gave several outreach talks and hands-on workshops in 2019-2020 at the
University of Cambridge (UK) introducing non-Euclidean geometry to high school
teachers and students of different ages, and organised a stall at the Maths
Public Open Day at the Cambridge Science Festival. Most of the audience in
these events were astonished about how they had never come to think of the
fact that triangles on spheres break the “$180\degree$ rule” (and several of
them seemed rather eager to inform their geometry teacher of this newly found
information). They were also fascinated by the crochet and paper models of the
hyperbolic plane, and hyperbolic geometry offered a couple of surprises even
to many mathematicians. Seeing different models of the hyperbolic plane helped
people to realise how the originally rather abstract seeming concept of
negative curvature can be observed all around us from Pringles to kale leaves
and sea slugs.

## References

  * [1] E.D. Demaine, M.L. Demaine, V. Hart, G.N. Price and T. Tachi. “(Non)Existence of Pleated Folds: How Paper Folds Between Creases.” Graphs and Combinatorics vol. 27, 377–397, 2011.
  * [2] M. A. Dias and C. D. Santangelo. “The shape and mechanics of curved-fold origami structures.” Europhysics Letters vol. 100, no. 5, 2012.
  * [3] D. W. Henderson, and D. Taimina. “Crocheting the hyperbolic plane.” Mathematical Intelligencer, vol. 23, no. 2, 17-27, 2001.
  * [4] F. Sottile. https://www.math.tamu.edu/~frank.sottile/research/stories/hyperbolic˙football/index.html
  * [5] D. Taimina. Crocheting Adventures with Hyperbolic Planes. AK Peters/CRC Press, 2009.


11institutetext: Utrecht University, Utrecht, the Netherlands,
11email<EMAIL_ADDRESS><EMAIL_ADDRESS>
# Using Reinforcement Learning to Optimize Responses in Care Processes: A Case
Study on Aggression Incidents

Bart J. Verhoef 11 Xixi Lu 11

###### Abstract

Previous studies have used prescriptive process monitoring to find actionable
policies in business processes and conducted case studies in similar domains,
such as the loan application process and the traffic fine process. However,
care processes tend to be more dynamic and complex. For example, at any stage
of a care process, a multitude of actions is possible. In this paper, we
follow the reinforcement approach and train a Markov decision process using
event data from a care process. The goal was to find optimal policies for
staff members when clients are displaying any type of aggressive behavior. We
used the reinforcement learning algorithms Q-learning and SARSA to find
optimal policies. Results showed that the policies derived from these
algorithms are similar to the most frequent actions currently used but provide
the staff members with a few more options in certain situations.

###### Keywords:

prescriptive process mining, reinforcement learning, Markov decision process,
process optimization, process mining

##  1 Introduction

_Prescriptive_ process monitoring focuses on analyzing process execution data
to not only predict the future behavior of a process but also provide
actionable recommendations or interventions to optimize the process
fahrenkrog2021 ; DBLP:conf/bpm/MetzgerKP20 ; DBLP:conf/caise/BozorgiDRPST23 .
It goes beyond _descriptive_ or _predictive process monitoring_ by actively
suggesting specific actions or decisions for improving process performance,
compliance, or efficiency. Considering the decision points in business
processes, the ability to offer specific guidance to users regarding optimal
actions is crucial, as it can lead to improved decision-making and efficiency.

One prominent approach is to use reinforcement learning, which learns online
by interacting with an environment to adapt and improve its recommendations
over time. The environments can be learned and built using the historical
execution traces and the feedback they received. While reinforcement learning
methods have been applied in business processes, healthcare processes exhibit
distinct characteristics and present new challenges for these techniques
DBLP:journals/jbi/Munoz-GamaMFJSH22 , such as dynamic workflows, diverse
stakeholders, and patient safety considerations. In particular, patients may
exhibit very diverse statuses, and a wide range of actions is possible at any
stage. Moreover, each patient may react differently to these actions. These
challenges may cause RL methods not to converge or not be able to improve the
current policies. In such dynamic settings, it is worth investigating the
validity and effectiveness of the RL approaches.

In this paper, we focus on the healthcare domain, and in particular, the
process of actions and responses in the aggression incidents by clients with
intellectual impairments in community care facilities hensel_lunsky_dewa_2013
. Being in aggressive situations can have a severe impact on staff members
since there is a mediation effect between experiencing aggressive behavior
from clients and burnout through fear of assault mills_rose_2011 . This means
that experiencing aggressive behavior leads to fear of assault, which in turn
leads to burnout. It also has a negative impact on the clients themselves
because aggressive behavior can lead to more aggressive behavior
bushman_bonacci_pedersen_vasquez_miller_2005 . Therefore, learning the optimal
way to act during aggression incidents helps de-escalate the incidents and
reduce negative impact.

Previous studies have analyzed the aggression incidents of such clients within
Dutch residential care facilities using a process mining approach agg_behav
or proposing to mine potential causal patterns DBLP:conf/bpm/KoornLLR20 ;
KOORN2022102035 ; DBLP:conf/icpm/KoornLLMVR22 . This meant that insights into
the use of actions and their effects could be made visible to show which
actions had a negative and which actions had a positive outcome in each
situation. However, this approach can only provide recommendations for a
single incident and does not take consecutive incidents and their consequences
into account.

In this paper, we investigate the use of prescriptive process monitoring,
inspired by learntoact , particularly reinforcement learning techniques, for
this healthcare process, in which the optimal policies of the best possible
action in a given situation (or state) can be determined. First, we train a
_Markov Decision Process_ (MDP) from the aggression incident log
KOORN2022102035 . Second, we apply reinforcement learning techniques, aiming
to find optimal policies for staff members to minimize aggressive incidents by
clients with intellectual impairments. We use the model-free, value-based
control algorithms: Q-learning and SARSA. The reason for choosing these
methods, rather than the Monte Carlo methods used in learntoact , stems from
their practical advantage of achieving earlier convergence on stochastic
processes sutton_richard .

The structure of the paper is as follows. Section 2 discusses the related
work. Then we explain the methods in Section 3, including the description of
the data set and the design of the environment. Section 4 presents the
results, and Section 5 discusses the results. Section 6 concludes the paper.

##  2 Related work

Research in prescriptive process monitoring has been done in the recent couple
of years, mainly with a focus on business processes. Fahrenkrog-Petersen et
al. fahrenkrog2021  used it to make a framework that parameterized a cost
model to assess the cost–benefit trade-off of generating alarms. Bozorgi et
al. bozorgi  researched it in the context of reducing the cycle time of
general supply chain processes. Both use supervised learning methods instead
of reinforcement learning methods and predict a threshold value that, when
exceeded, recommends an action. The algorithms themselves do not make a
recommendation; only predictions are made, and based on the predictions, a
user-defined action is recommended.

Weinzierl et al. weinzierl2020predictive  also made this remark and proposed
an alternative approach to prescriptive process monitoring in which there is a
learning and a recommendation phase, in which the recommendation gives the
next best action to take. Branchi et al. learntoact  used prescriptive process
monitoring with Monte Carlo methods to determine the best actions to lend out
loans and ensure most traffic fines are paid. The Monte Carlo methods are
valid algorithms, although TD methods such as Q-learning and SARSA tend to
converge earlier on stochastic processes in practice sutton_richard . In this
paper, we use Q-learning and SARSA to find optimal policies.

##  3 Methodology

This section describes the methods used in the research. First, we describe
the data set. We then explain the preprocessing steps and the way the
environment is built. Finally, we discuss the evaluation measures used.

###  3.1 Data set

The data set is from a Dutch residential care organization with several
facilities. The event data contains 21,384 reported aggression incidents from
1,115 clients with intellectual impairments. The data has been anonymized for
privacy reasons. The reported incidents were reported by staff members between
the 1st of January 2015 and the 31st of December 2017. The event data includes
attributes such as the date of the incident, pseudonym client ID, the type of
aggression, the countermeasure that the staff took, and the type of persons
involved (such as family, staff members, and other clients). A simplified
example of the event data is listed in Table 1.

Table 1: A snippet of the incident data where the last column describes the countermeasures taken by staff members to stop the aggression. Pseudonym client | Date of incident | Aggression type | Involved | Measures
---|---|---|---|---
ab45 | 05/01/2016 | va | family | talk to client
ab45 | 06/01/2016 | pp | client | none
lz12 | 06/01/2015 | sib | unknown | seclusion
lz12 | 18/01/2015 | po | client | none

In the event data, four types of aggression are reported, which are _verbal
aggression_ (va), _physical aggression against people_ (pp), _physical
aggression against objects_ (po), and _self-injurious behavior_ (sib). Eight
distinct countermeasures are reported by the staff members: talk to the
client, held with force, no measure taken, seclusion, send to another room,
distract client, terminate contact, and starting preventive measures.

###  3.2 Data cleaning and preprocessing

To use reinforcement learning with this dataset, we preprocess the data. We
follow the same steps as in KOORN2022102035 . First, we add the type of next
aggression incident as an attribute of the current event, in order to create
tuples of three which contain the type of current aggression, the
countermeasures taken by a staff member, and the type of next aggression. The
aim is to use the aggression types as the _states_ a client is in and use the
countermeasures as _actions_. Such a triplet describes a transition from one
state to the next state after taking an action.

In the second step, we group incidents into _episodes_. According to a
behavioral expert at the care organization KOORN2022102035 , an _episode_ is a
sequence of incidents by the same client that occurred after each other, where
the time between incidents is less than or equal to nine days. Following this
domain knowledge, we segment the sequences of incidents into episodes. When
two consecutive incidents $e_{i}$ and $e_{i+1}$ of a client are more than nine
days apart, we insert a _Tau_ after $e_{i}$ as the final state of an episode.
The incident $e_{i+1}$ is the start state of the next episode. An overview of
the approach is shown in Figure 1.

Figure 1: Preprocessing pipeline used to get enriched and clean data

We assign each episode a unique ID. The episodes that do not end in a _Tau_
state are considered incomplete and, therefore, filtered. We obtained a total
of 8,800 episodes after this filter, consisting of 19,848 incidents. In
addition, the episodes where the incidents miss the values on the measures
column are removed; these are incidents in which the staff member did not
report the measures they had taken. Applying this filter reduced the number of
episodes to 8,013, consisting of 15,464 incidents. Finally, we decided to
remove the most infrequent action, ‘preventive measures started’ due to its
ambiguity and to reduce the search space. Any episode that contains this
action was removed, resulting in 14,676 incidents and 7,812 episodes for
training the final MDP. In Table 2, a simplified example of the preprocessed
log is listed.

Table 2: A simplified example of the preprocessed event data

Pseudonym client | Aggression type | Measures | Next aggression type | Episode Id
---|---|---|---|---
ab45 | va | talk to client | pp | 1
ab45 | pp | none | Tau | 1
lz12 | sib | secluded | Tau | 2
lz12 | po | none | Tau | 3

###  3.3 Building the environment

Now that the data is cleaned and preprocessed, we use it to build a finite
MDP. For this, we need the five-tuple consisting of the states, actions,
transition probabilities, rewards, and discount factor sutton_richard . The
discount factor is a hyperparameter that can be tuned; therefore, we later
perform hyperparameter tuning to determine the discount factor for the agent.

We describe the MDP using the standard formalization in sutton_richard  as
follows:

  * •

$\mathcal{S}$ = {va, po, sib, pp, Tau}, i.e., the set of states;

  * •

$\mathcal{A}$ = $\\{$talk to the client, no measure taken, seclusion, holding
with force, send to another room, distract client, terminate contact$\\}$,
i.e., the set of actions;

  * •

$\mathcal{P}$, which is the probability of going from one state to the next
based on the action. This is determined using the following function

$P(s,a,s^{\prime})=\frac{\textrm{Number of times }a\textrm{ leads to
}s^{\prime}}{\textrm{Number of times }a\textrm{ is chosen in state }s}$ (1)

  * •

$\mathcal{R}:\mathcal{A}\times\mathcal{S}\rightarrow\mathbb{Z}$, which is the
reward function. We defined the reward function based on the literature in
assessing the severity of the action and the state soas-r . The reward
(penalty) for each individual action or state is listed in Table 3.

Table 3: The rewards (penalty) assigned for each action or state, based on the severity of the action and the state soas-r . When an agent takes an action and ends in the follow-up state, the combination of the action and state is used to compute the reward. Action or state | Reward
---|---
Tau | 1
Verbal Aggression (va) | 0
Physical Aggression against objects (po) | -1
Self-injurious behavior (sib) | -3
Physical Aggression against people (pp) | -4
Client distracted, Contact terminated, Send to other room | -1
Hold with force, Seclusion | -2
Other actions | 0

Another design choice has been made regarding the calculation of the
transition probabilities. In the data set, multiple actions could be filled in
at each incident. For this paper, a decision was made to consider only the
most frequent action as the transition from one state to the next, in order to
limit the number of possible actions and avoid having too many infrequent
actions. Also, the reward function was designed based on the severity of the
action and the state as indicated in the existing literature in aggression
soas-r . The simple reward function was designed on purpose such that the
results can be more easily communicated to the experts. A subgraph of the
environment can be seen in Figure 2.

Figure 2: A subgraph of the MDP, depicting the current state of self-injurious
behavior (SIB), a sample of actions that can be chosen, and a sample of
transitions. $P$ is the probability of going to that state, and $R$ is the
reward associated with that action and next state.

###  3.4 Training the agents

We used the following parameters in the tuning: the learning rate,
$\alpha\in[0,1]$, the discount factor, $\gamma\in[0,1]$, and the amount of
exploration, $\epsilon\in[0,1]$, which have an impact on the training of the
agents and therefore the results. The best parameters are obtained
experimentally by hyperparameter tuning using the best average reward of 100
runs as the goal, each consisting of 2000 episodes. The search spaces are
$\alpha\in[0.1,0.2,0.3,0.4,0.5]$, $\gamma\in[0.2,0.4,0.6,0.8,1.0]$ and
$\epsilon\in[0.1,0.2,0.3,0.4,0.5]$. First $\gamma$ was obtained while keeping
$\alpha$ and $\epsilon$ on 0.1. After this $\epsilon$ was obtained using the
optimal $\gamma$ value and $\alpha=0.1$ and finally $\alpha$ was obtained
using the optimal $\gamma$ and $\epsilon$ value. Each parameter has been used
for ten different runs to get a fair average. The hyperparameter values for
both Q-learning and SARSA are $\alpha=0.2$, $\gamma=0.2$ and $\epsilon=0.1$.

###  3.5 Evaluation of policies

We evaluate the agents both quantitatively, by comparing the average rewards,
and qualitatively, by discussing the policies. For the quantitative
evaluation, we compute the average reward for the best-trained agent using
Q-learning and the best-trained agent using SARSA. These are then compared
with the average reward of taking random actions and the average reward with
the current policy. The current policy has been derived as the most frequent
action taken in a state. The current policy is “talking to the client” when
they display verbal aggression (va), physical aggression against people (pp),
and physical aggression against objects (po). For the state of self-injurious
behavior (sib) “no action” is the most frequently used action. For the
qualitative evaluation, we discuss the results by looking at the most frequent
variants for each agent and comparing these variants with the ones of the
current policy.

##  4 Results

In this section, we first present the results regarding the rewards. Next, we
discuss the results qualitatively, presenting the optimal policy and the
variants. We used two baselines to compare the results: (1) using random
actions and (2) taking the most frequent action at each state as the policy
for the agent. The data set is shared under the NDA and thus unavailable. The
code and the MDPs used in this paper are online available111
https://git.science.uu.nl/6601723/ppm-aggressive-incidents, which can be used
to reproduce the results.

###  4.1 Quantitative results

In this section, the average reward per policy is described and evaluated. It
is listed in Table 4.

Table 4: Average reward per policy based on 10,000 runs, each with 100 episodes. Policy | Average reward
---|---
Random | -3.783
Most frequent action | -1.105
Q-learning | -1.127
SARSA | -1.168

We run each policy for 10,000 runs, each consisting of 100 episodes, resulting
in 1,000,000 episodes total. To test if the differences between the policies
are significant, we performed a one-way ANOVA with the data from SARSA,
Q-Learning, and the current policy. The one-way ANOVA was done using the Scipy
library from Python 3, specifically the stats.f_oneway function. The p-value
was $7.719e-26$, which is smaller than 0.05, therefore, we can reject the null
hypothesis that the groups have the same mean, meaning there is a significant
difference between them, but we do not know between which. Therefore, we use a
least significant difference test as a posthoc test using the
pairwise_tukeyhsd from the Python 3 library statsmodels. This made three
comparisons, it tested Q-learning against the current policy, tested SARSA
against the current policy, and tested Q-learning against SARSA. It rejected
all three null hypotheses, meaning the average reward per algorithm
significantly differs from one another.

###  4.2 Qualitative results

This section describes the qualitative results where we show the derived
policies and the most common variants of episodes per agent. The derived
policies can be found in Table 5, where the action taken at each state for
each policy can be found.

Table 5: Derived policies for Q-learning and SARSA together with the most frequent actions taken on the 10,000 runs, each with 100 episodes. Policy | VA | SIB | PP | PO
---|---|---|---|---
Most frequent action | talk with client | no measure | talk with client | talk with client
Q-learning | talk with client | no measure | talk with client | no measure
SARSA | no measure | no measure | talk with client | talk with client

The five most common variants with their frequencies for each of the agents
can be found in Tables 6, 7, and 8. In the tables, each variant is a distinct
episode of tuples, where the first element of the tuple is the _current state_
, the second element is the _action_ taken, and the last element is the _next
state_ after the action. If the state is _Tau_ , the episode is ended;
otherwise, another action is taken.

Table 6: Five most common

Path | Frequency
---|---
(va, Talk with client, Tau) | 14454
(sib, No measure, ’Tau’) | 13987
(po, Talk with client, Tau) | 13100
(pp, Talk with client, Tau) | 12769
(pp, Talk with client, pp)(pp, Talk with client, Tau) | 4454

Table 7: Five most common variants when using the policy derived by
Q-learning.

Path | Frequency
---|---
(va, Talk with client, Tau) | 14417
(sib, No measure, Tau) | 14294
(po, No measure, Tau) | 12974
(pp, Talk with client, Tau) | 12866
(pp, Talk with client, pp)(pp, Talk with client, Tau) | 4526

Table 8: Five most common variants when using the policy derived by SARSA.

Path | Frequency
---|---
(va, No measure, Tau) | 14926
(sib, No measure, Tau) | 14025
(pp, Talk with client, Tau) | 13079
(po, Talk with client, Tau) | 12971
(sib, No measure, sib)(sib, No measure, Tau) | 4313

In the tables, it can be seen that the four most frequent variants of episodes
end in one action for all policies. For each state doing that action leads
immediately to Tau regardless of the policy. Also, most of the episodes ended
when only one action had been taken. When we take a closer look at the current
policy, the Q-learning policy, and the SARSA policy, we see that most variants
are the same with only two differences: (1) in the verbal aggression state
(va), “no measure” action is suggested by the Q-learning; (2) in the self-
injury-behavior state (sib), “no measure” action is suggested by SARSA.

###  4.3 Additional analysis

Due to the results from taking all episodes, we decided to do an additional
analysis on a subset of the data. We kept the episodes that had a length of
more than or equal to three incidents and performed the same experiment as we
did on the whole dataset. This subset contained 6687 incidents over 1360
episodes. Taking only the episodes longer than or equal to three incidents, we
focus on the clients who display more severe behavior, which are the ones we
want to help reduce in the first place. We again used Q-learning and SARSA as
described above and compared them to taking random actions and the current
policy, which in this case was “talk with client” in every state. The
hyperparameter tuning was done the same as described in Section 3.4, resulting
in the best performing Q-learning agent and best performing SARSA agent both
with $\alpha=0.1$, $\gamma=0.2$ and $\epsilon=0.1$. In the remaining parts of
the additional analysis, we present the quantitative and qualitative results.

####  4.3.1 Additional analysis quantitative results

In this section, the average reward per policy is shown and can be found in
Table 9.

Table 9: Average reward per policy based on 10000 runs along 100 episodes Policy | Average reward
---|---
Random | -11.925
Most frequent action | -7.342
Q-learning | -7.266
SARSA | -7.275

The same statistical tests as on the whole dataset were done. The p-value of
the one-way ANOVA was $2.917e-12$, which is smaller than 0.05, therefore, we
can reject the null hypothesis. We use a least significant difference test as
a posthoc test. This made three comparisons. It rejected two out of three null
hypotheses. With a p-adj value of 0.6833, it did not reject the hypothesis
that the rewards from Q-Learning and SARSA had the same mean. When taking a
look at Table 10, we can see that they have learned exactly the same policy,
so this result was expected.

####  4.3.2 Additional analysis qualitative results

We list the derived policies with the current policy and compare the most
common variants taken between a random baseline, the most frequent actions
taken, and the two policies derived by the agents.

The derived policies can be found in Table 10, where the action taken at each
state for each policy can be found. The Q-learning and SARSA agent learned
that “talking to a client” was the best option when the state is verbal
aggression (va), physical aggression against people (pp), or physical
aggression against objects (po), and “no measure” when the state is self-
injury behavior (sib).

Table 10: Derived policies for Q-learning and SARSA together with the most
frequent actions taken on a subset of the data set only containing episodes
$\geq 3$

Policy | VA | SIB | PP | PO
---|---|---|---|---
Most frequent action | talk with client | talk with client | talk with client | talk with client
Q-learning | talk with client | no measure | talk with client | talk with client
SARSA | talk with client | no measure | talk with client | talk with client

The five most common variants with their frequencies for the current policy
and the RL agents can be found in Tables 11 and 12. Both the Q-learning agent
and SARSA agent learned the same policy. One noticeable difference between the
frequent episodes of the policies of the RL agents and the most frequent
policy is that the second most frequent episode of self-injurious behavior is
added in Tables 11 and 12.

Table 11: Five most common variants of episodes when using the most frequent
actions for 1,000,000 episodes, plus the most common variant when considering
SIB.

Path | Frequency
---|---
(va, Talk with client, Tau) | 5052
(sib, Talk with client, Tau) | 4772
(pp, Talk with client, Tau) | 4520
(po, Talk with client, Tau) | 4488
(pp, Talk with client, pp)(pp, Talk with client, Tau) | 2617
(sib, Talk with client, sib)(sib, Talk with client, Tau) | 1776

Table 12: Five most common variants by Q-learning for 1,000,000 episodes, plus
the most common variant when considering SIB.

Path | Frequency
---|---
(va, Talk with client, Tau) | 5207
(sib, No measure, Tau) | 4901
(po, Talk with client, Tau) | 4661
(pp, Talk with client, Tau) | 4620
(pp, Talk with client, pp)(pp, Talk with client, Tau) | 2749
(sib, No measure, sib)(sib, No measure, Tau) | 2720

In the tables, it can be seen that the four most frequent variants end in one
action for all policies. For each state, performing that action leads
immediately to Tau regardless of the policy. This time only a fifth of all
episodes lead immediately to Tau, for all policies except the random one. The
random agent only has episodes ending with only one incident in its top ten,
but this can be explained by their frequency, which is relatively much lower
than the other policies (976 vs. 5052). When we take a closer look at the
current policy and the RL agents’ learned policy, the variants differ
significantly in the frequencies of the self-injurious behavior (sib) state.
The frequencies of the single-incident episodes for this state are similar
between the RL agents and the current policy (4901 vs. 4772). When the
episodes consist of two incidents that concern the self-injurious behavior
(sib) state, the frequency of such episodes is much higher in the RL agents
learned policy than the current policy (2720 vs. 1776), meaning that “no
measure” results faster to Tau than “Talk with client” in this case.

##  5 Discussion

The results indicate that the current policy and the RL-derived policies reach
similar conclusions. The current policy performs slightly better than the RL
agents when considering all episodes, but the RL agents provide staff members
with additional options without having a significant negative impact on
rewards. When considering the selected subset of the episodes, the RL agents
slightly outperform the current policy, offering an alternative choice.

In both cases, the staff member can choose to talk to the client or take no
action. Although the RL algorithms exhibit slight variations in performance
compared to the current policy, the policies derived do not significantly
differ. This alignment is reasonable considering the reward function used,
which penalizes all actions except “no measure” and “talk with client”. These
options align with the least disruptive impact on both the client and staff
member, as indicated by previous studies.

However, it is important to note that the models may oversimplify the real
situation, and further factors such as location, time, and individuals
involved have not been included. Collecting relevant data and consulting
behavioral experts could enhance future research in this field. For example,
it is possible to learn the time distribution until the next incident and use
this in the reward function.

Additionally, practical relevance should be acknowledged, as staff members
face challenges in assessing situations and may need to use force in certain
cases. Future research may aim to provide insights tailored to specific
clients or client groups. Combining reinforcement learning and process mining
in prescriptive process monitoring shows promise but requires careful
consideration of data availability and exploration limitations.

##  6 Conclusion

This paper presents the application of reinforcement learning (RL) to optimize
response policies in healthcare processes, specifically addressing aggressive
incidents in care settings. The research aims to investigate the validity of
RL in healthcare and the ability to find optimal response policies for staff
members towards such incidents. The results have shown that RL algorithms can
find such an optimal policy, which consists of taking no measures or talking
with the client depending on the state. The policies are very similar to the
current policy, i.e., the most frequent action taken by staff members.

Despite the simple MDP, the results do show that prescriptive process
monitoring can be used in the healthcare domain. Interestingly, it may be more
beneficial to use the techniques in more complex situations, rather than the
simple situation. However, further research is necessary to validate this
finding.

For future work, one may refine the environment by extending the MDP with more
refined states and actions. Future research should be multidisciplinary, where
such an environment can be more elaborately built based on experts in the
field of aggressive behavior and staff members who work daily with clients.
Results can then also be validated by the experts or staff to help them make
better decisions and therefore their input is crucial.

#### Acknowledgement

This research was supported by the NWO TACTICS project (628.011.004) and Lunet
in the Netherlands. We would like to thank the experts from the Lunet for
their assistance. We also thank Dr. Shihan Wang and Dr. Ronald Poppe for the
invaluable discussions.

## References

  * (1) Fahrenkrog-Petersen, S.A., Tax, N., Teinemaa, I., Dumas, M., Leoni, M.d., Maggi, F.M., Weidlich, M.:  Fire now, fire later: Alarm-based systems for prescriptive process monitoring.  Knowledge and Information Systems 64(2) (2021) 559–587
  * (2) Metzger, A., Kley, T., Palm, A.:  Triggering proactive business process adaptations via online reinforcement learning.  In: BPM. Volume 12168 of Lecture Notes in Computer Science., Springer (2020) 273–290
  * (3) Bozorgi, Z.D., Dumas, M., Rosa, M.L., Polyvyanyy, A., Shoush, M., Teinemaa, I.:  Learning when to treat business processes: Prescriptive process monitoring with causal inference and reinforcement learning.  In: CAiSE. Volume 13901 of Lecture Notes in Computer Science., Springer (2023) 364–380
  * (4) Munoz-Gama, J., et al.:  Process mining for healthcare: Characteristics and challenges.  J. Biomed. Informatics 127 (2022) 103994
  * (5) Hensel, J.M., Lunsky, Y., Dewa, C.S.:  Staff perception of aggressive behaviour in community services for adults with intellectual disabilities.  Community Mental Health Journal 50(6) (2013) 743–751
  * (6) Mills, S., Rose, J.:  The relationship between challenging behaviour, burnout and cognitive variables in staff working with people who have intellectual disabilities.  Journal of Intellectual Disability Research 55(9) (2011) 844–857
  * (7) Bushman, B.J., Bonacci, A.M., Pedersen, W.C., Vasquez, E.A., Miller, N.:  Chewing on it can chew you up: Effects of rumination on triggered displaced aggression.  Journal of Personality and Social Psychology 88(6) (2005) 969–983
  * (8) Koorn, J.J., Lu, X., Leopold, H., Reijers, H.A.:  Towards understanding aggressive behavior in residential care facilities using process mining.  In Guizzardi, G., Gailly, F., Suzana Pitangueira Maciel, R., eds.: Advances in Conceptual Modeling, Cham, Springer International Publishing (2019) 135–145
  * (9) Koorn, J.J., Lu, X., Leopold, H., Reijers, H.A.:  Looking for meaning: Discovering action-response-effect patterns in business processes.  In: BPM. Volume 12168 of Lecture Notes in Computer Science., Springer (2020) 167–183
  * (10) Koorn, J.J., Lu, X., Leopold, H., Reijers, H.A.:  From action to response to effect: Mining statistical relations in work processes.  Information Systems 109 (2022) 102035
  * (11) Koorn, J.J., Lu, X., Leopold, H., Martin, N., Verboven, S., Reijers, H.A.:  Mining statistical relations for better decision making in healthcare processes.  In: ICPM, IEEE (2022) 32–39
  * (12) Branchi, S., Di Francescomarino, C., Ghidini, C., Massimo, D., Ricci, F., Ronzani, M.:  Learning to act: a reinforcement learning approach to recommend the best next activities (2022)
  * (13) Sutton, R.S., Barto, A.G.:  Reinforcement Learning: An Introduction.  A Bradford Book, Cambridge, MA, USA (2018)
  * (14) Bozorgi, Z.D., Teinemaa, I., Dumas, M., Rosa, M.L., Polyvyanyy, A.:  Prescriptive process monitoring for cost-aware cycle time reduction.  In: 2021 3rd International Conference on Process Mining (ICPM). (2021) 96–103
  * (15) Weinzierl, S., Zilker, S., Stierle, M., Matzner, M., Park, G.:  From predictive to prescriptive process monitoring: Recommending the next best actions instead of calculating the next most likely events.  In: Wirtschaftsinformatik (Zentrale Tracks). (2020) 364–368
  * (16) Nijman, H.L., Muris, P., Merckelbach, H.L., Palmstierna, T., Wistedt, B., Vos, A., van Rixtel, A., Allertz, W.:  The staff observation aggression scale–revised (soas-r).  Aggressive Behavior 25(3) (1999) 197–209