# Epidemic models with digital and manual contact tracing

Dongni Zhang  Department of Mathematics, Stockholm University, 106 91
Stockholm, Sweden.  Tom Britton  Department of Mathematics, Stockholm
University, 106 91 Stockholm, Sweden.

###### Abstract

We consider the Markovian SIR epidemic model but where individuals either
recover naturally or are diagnosed, the latter implying isolation and subject
to being contact traced. More specifically, this paper is concerned with two
types of contact tracing: the classical manual contact tracing and the more
recent digital contact tracing based on individuals using a tracing app, and
investigates the preventive effect of each, as well as their combined
preventive effect. The initial phase of an outbreak is approximated by a (two-
type) branching process relying on a large community and under the simplifying
assumption that contact tracing happens without delay. The ”types” in the
branching process are not individuals but rather ”to-be-traced components”. It
is shown that the fraction $\pi$ of app-users for the digital contact tracing
needs to be bigger than the fraction $p$ of successfully contact traced
individuals in the manual contact tracing for the same preventive effect.
Further, the preventive effect of combining the two contact tracing methods is
shown to be bigger than the product of each of the two preventive effects.

##  1 Introduction

One of the main reasons for modelling epidemics is to understand the effect of
different preventive measures, such as vaccination and non-pharmaceutical
interventions (NPI). During the recent Covid-19 pandemic, research about the
effect of various NPIs, including social distancing, case isolation, contact
tracing, lockdown, etc., has received much attention. Some related papers
(e.g., [ferguson_report_2020, flaxman_estimating_2020,
longini_containing_2005]) have been highly influential on public health
policies.

This paper focuses on the preventive measure: ”contact tracing”. Contact
tracing is traditionally performed by public health agency officers who
interview diagnosed individuals and then call the named contacts to advise
them to self-quarantine and to test, which is often referred as the manual
contact tracing. This type of contact tracing has been successful in reducing
transmission in many epidemics like Ebola [ebola_ct], SARS [lloyd-
smith_curtailing_2003], Influenza [agarwal_modeling_2012], and Measles
[liu_role_2015]. However, in the present paper, we also investigate a
different type of contact tracing: digital contact tracing. In the case of
Covid-19, [ferretti_quantifying_2020] suggested that manual contact tracing is
not rapid enough to control the Covid-19 epidemic, and the new idea of
developing a ”contact tracing app” was invented: once an app-user is
diagnosed, a warning message will be instantaneously sent out to all the app-
users who recently have been nearby for a sufficient duration with the
confirmed case. Such notified contacts are then advised to test and quarantine
themselves. Several contact tracing apps have been developed and released
(e.g., in the UK [ferretti_quantifying_2020, wymant_epidemiological_2021], in
the Netherlands [app_netherlands]).

There are several papers analysing digital contact tracing (e.g.,
[pollmann_impact_2020, kretzschmar_isolation_2020,
kucharski_effectiveness_2020, barrat_effect_2021]) focusing on simulation-
based epidemic models, and the results of [rizi2022epidemic] are based on a
combination of simulations and mean-field analysis of the related percolation
problem. Several papers on digital contact tracing analysing Covid-19 (e.g.,
[cencetti_digital_2021, ferretti_quantifying_2020]) are based on the
mathematical model first introduced by [fraser_factors_2004] in terms of
recursive equations for analyzing the timing of infectiousness and the
appearance of symptoms. In particular, the results from
[ferretti_quantifying_2020] showed that the Covid-19 epidemic was unlikely to
be contained only by manual tracing.

Modelling contact tracing is mathematically challenging and does not easily
lend itself to being analyzed using differential equations. Our focus in this
paper is on the beginning of an outbreak before a massive fraction have been
infected. For this reason, we apply stochastic models, which are also suitable
because the outcome of contact tracing itself is random rather than
deterministic. The early phase of an epidemic without preventive measures can
often be approximated by a suitable branching process (e.g.,
[ball_strong_1995]) using the underlying model assumption stipulating that
once an individual has been infected, it behaves (transmits) independently of
its infector. When contact tracing is introduced, this independence breaks
down and other methods need to be developed. However, in the last decades
there has been some progress in more rigorous analyses of epidemic models with
manual contact tracing, e.g., [muller_contact_2000, ball_threshold_2011,
barlow_branching_2020].

In [zhang_analysing_2021] we considered a Markovian SIR epidemic model with
manual contact tracing by assuming that once individuals are diagnosed, they
perform manual tracing without delay, thus reaching each contact with
probability $p$. The model was analyzed by means of a branching process where
the ”individuals” of the branching processes corresponded to ”to-be-traced”
components, which grew and decreased over time until they eventually
disappeared.

In the present paper, we first analyse a similar model as in
[zhang_analysing_2021] but now with digital contact tracing by assuming that a
fraction $\pi$ of individuals installs a contact tracing app (and follows the
advice if notified). Once an app-user is diagnosed, all its app-using contacts
are notified, with the result that they isolate and test themselves without
delay. For those who test positive, their app-using contacts are also
notified, and so on. Furthermore, for both manual and digital contact tracing,
we assume that traced individuals who have recovered naturally are also
identified and are subject to contact tracing.

Using large population approximations, we approximate the early epidemic
process with digital tracing by a two-type branching process where type-1
”individuals” are app-using components (in terms of app-users) and type-2 are
non-app-users. We then analyse the model having both types of contact tracing
in place, i.e., reaching a fraction $p$ of all contacts using manual contact
tracing and additionally all app-using contacts if the tested individual is an
app-user (making up a fraction $\pi$ in the community). The limiting process
in this case is shown to be a two-type branching process, where both the
type-1 and type-2 ”individuals” are the to-be-traced components, but type-1
starts with an app-user and type-2 with a non-app-user.

One important conclusion from the analysis is that the combined effect is
better than if the two types of contact tracing act independently. More
precisely, let $r_{M}$ be the reduction (of the basic reproduction number)
from manual contact tracing, $r_{D}$ from digital contact tracing, and
$r_{DM}$ be the combined reduction. If the manual and digital tracing acts
independently, then we would have $1-r_{DM}=(1-r_{M})(1-r_{D})$, but in fact,
our analysis shows that $r_{DM}>1-(1-r_{M})(1-r_{D}).$

In Section 2, we define the digital contact tracing model as well as the model
having both manual and digital contact tracing. In Section 3, we present and
prove the main result approximating the initial phase of the epidemic in a
large population (Section 3.1 for digital tracing only, Section 3.2 for both
types of tracing). Then, in Section 4 we illustrate our results numerically,
compare the effect of manual and digital contact tracing and investigate their
combined effect. Finally, we summarize our conclusions and discuss potential
improvements of the present models in Section 5.

##  2 Epidemic models with contact tracing

###  2.1 The epidemic model

First, we consider a Markovian SIR (Susceptible $\rightarrow$ Infectious
$\rightarrow$ Recovered) epidemic spreading in a closed and homogeneous mixing
population with fixed size $n$. Initially, there is one infectious individual,
and the rest are susceptible. An infectious individual has infectious contacts
with each susceptible individual randomly in time according to independent
Poisson processes with rate $\beta/n$, so $\beta$ is the overall rate of
contacts. Only contacts with susceptibles result in infection. An infectious
individual recovers naturally at rate $\gamma.$

Additionally, we assume that infectious individuals are tested positive and
are immediately isolated (we refer to this as ”diagnosed” in our paper) at
rate $\delta.$ In conclusion, infectious individuals could stop spreading
disease either from natural recovery (rate $\gamma$) or diagnosis (rate
$\delta$). The infectious periods are assumed to be independent of each other
and the Poisson processes.

For this epidemic model (with testing but without contact tracing), each
infectious period follows the exponential distribution with intensity
$(\gamma+\delta).$ Hence, it is straightforward to compute the mean number of
secondary infections produced by one single infective before recovery or
diagnosis:

$R_{0}=\frac{\beta}{\gamma+\delta}.$ (1)

Somewhat incorrectly, we refer to this as our basic reproduction number
$R_{0}$ in this paper, even though $R_{0}$ equals $\beta/\gamma$ when there is
no testing.

###  2.2 The epidemic model with digital contact tracing

We add digital contact tracing to the epidemic model in Section 2.1 by
assuming that a fraction $\pi$ of individuals use the tracing app (and follow
the recommendations), and that individuals mix uniformly irrespective of using
the app or not.

Once an infectious app-user is diagnosed, all app-using contacts of the
infective will be notified, with the result that they test and isolate
themselves without delay if infectious. The contacts who have been infected,
including also those who have recovered, are then also assumed to trigger
digital tracing among their contacts according to the same procedure, and so
on.

###  2.3 The epidemic model with digital and manual contact tracing

Finally, we describe the epidemic model with both types of contact tracing and
call this the combined model. First, a community fraction $\pi$ are app-users
and follow the instructions. If such an app-user is diagnosed, then all of its
app-using contacts (on average being a fraction $\pi$ of all contacts) will be
traced. To this, we now add the manual contact tracing [zhang_analysing_2021]
by assuming that once an infectious individual is diagnosed, app-user or not,
each of its contacts will be reached by manual tracing independently with
probability $p$. It means that for diagnosed app-users, _all_ app-using
contacts will be traced, and each non-app-using contact will be traced with
probability $p$. For diagnosed non-app-users, each contact, app-user or not,
will be traced with probability $p$. Like before, all traced individuals are
tested and isolated without delay if testing positive. Traced individuals who
test positive, including also those who have recovered, are then contact
traced in the same manner without delay, and so on.

In the case of $\pi=0,$ this model corresponds to our earlier model with
manual tracing only in [zhang_analysing_2021], and if $p=0$ it corresponds to
the model with digital tracing only in Section 2.2. The model parameters are
summarized in Table 2.1.

Table 2.1: Table of Model Parameters Parameter | Notation
---|---
size of population | $n$
transmission rate | $\beta$
rate of natural recovery | $\gamma$
rate of diagnosis | $\delta$
community fraction using the contact tracing app | $\pi$
(and following the advice) |
probability that contact is manually traced successfully | $p$
or naturally recovered contact manually |

##  3 Approximation of the initial phase in a large community

When the population size $n$ is large, it is very unlikely that infectious
contacts will be with already infected people. In fact, the first contact with
someone who has already been infected will happen when around $\sqrt{n}$
individuals have been infected (see, e.g., [andersson_stochastic_2000] page
25). Consequently, at the beginning of the epidemic in a large community,
individuals will infect approximately independently of each other while
infectious, which can be made precise using coupling methods (see
[ball_strong_1995]). However, also in the limit as $n\to\infty$ infectious
individuals will not behave completely independently since an infectious
individual may be contact traced by its infector or one of its infectees.

As a consequence, when contact tracing is introduced, the limiting process of
infectious individuals will not be a branching process as is usually the case
for epidemic models. For this reason, we consider ”to-be-traced” components as
”macro-individuals” since these components will behave independently between
generations. In the following sections, we first describe the limiting
branching processes for the model with digital tracing and then for the
combined model.

###  3.1 The epidemic model with digital contact tracing

When digital tracing is in place, the population comprises app-users and non-
app-users. A given infectious app-user has the possibility of being traced and
hence stops being infectious when one of its app-using contacts is diagnosed.
It implies that infectious app-users no longer infect independently of each
other in our model.

Instead of describing the limiting process in terms of the actual individuals,
we consider to-be-traced components as ”macro-individuals”: since contact
tracing will never happen between such components, these components behave
independently of each other and may hence be approximated by a branching
process (see [zhang_analysing_2021]). Here we focus on the so-called app-using
components (to-be-traced components in terms of app-users). Such app-using
components will start with one infectious app-user infected by a non-app user
(we call this the root of the component) but can later grow to include the
app-users infected by the root and app-users they infected, and so on.
Eventually, the whole app-using component will die out (i.e., stop spreading
the infection) either by someone being diagnosed (when immediately the whole
component gets traced and diagnosed) or if all individuals in the component
have recovered naturally. While the component has infectious individuals, the
component can also infect non-app-users. Non-app-users can, in turn, infect
other non-app-users as well as app-users, and in the latter case, new app-
using components are created.

In Figure 3.1, we show how the two types of ”individuals”: app-using
components (surrounded by dashed lines) and non-app-users (square-shaped
nodes) grow and eventually die out. We set the app-user $A1$ to be the initial
case. While the app-using component $C^{(app)}_{1}$ evolves (new app-users
getting infected or existing ones recovering), the component $C^{(app)}_{1}$
infects two non-app-users $N1$ and $N2.$ The non-app-user $N1$ infects one
non-app-user and one new app-using component $C^{(app)}_{2}$ with root $A2$,
whereas $N2$ infects two other non-app-users. Once the app-user $A2$ is
diagnosed, the whole app-using component $C^{(app)}_{2}$ will die out.

Figure 3.1: Example of the dynamics of app-using components and non-app-users
for the model with digital contact tracing: the circle nodes are app-users and
the squares non-app-users. The white, black, and grey nodes are regarded as
”infectious”, ”diagnosed” and ”naturally recovered”, respectively. The
rectangular region surrounded by the dashed line symbolizes the app-using
component, and the area is filled with diagonal lines when the whole component
is diagnosed and removed.

In conclusion, we consider a two-type branching process
$E_{D}(\beta,\gamma,\delta,\pi)$ with the type-1 individuals being app-using
components and the type-2 being non-app-users, where a new birth corresponds
to a new infection in the epidemic. Then with the coupling argument (see
[ball_strong_1995, andersson_stochastic_2000, britton_stochastic_2010]), it
can be easily proved that the epidemic process in terms of the app-using
components and non-app-users converges to this two-type branching process for
large $n$.

####  3.1.1 Properties of the limiting branching process

We have seen that during the early stage of an outbreak and in the large
population, the epidemic process with digital tracing can be approximated by
the two-type branching process $E_{D}(\beta,\gamma,\delta,\pi)$.

In this section, we derive the component reproduction number $R_{D}$ for the
model with digital tracing as the largest eigenvalue of the mean offspring
matrix $M$ of $E_{D}(\beta,\gamma,\delta,\pi)$ (see e.g.,
[athreya_branching_1972, becker_multi_type_1990]) with

$M=\begin{pmatrix}m_{11}&m_{12}\\\ m_{21}&m_{22}\end{pmatrix},$ (2)

where $m_{ij}$ is the number of secondary infections of type $j$ produced by a
single infected individual of type $i$ on average, for $i,j=1,2.$ And hence
the component reproduction number $R_{D}$ is given by

$R_{D}=\frac{m_{11}+m_{22}}{2}+\sqrt{\frac{(m_{11}+m_{22})^{2}}{4}-m_{11}m_{22}+m_{12}m_{21}}.$
(3)

Next, we derive the expressions for the elements $m_{ij}$ as follows. We note
that the dynamics of app-using components (type 1) works similarly to the ”to-
be-reported components” studied in [zhang_analysing_2021], except that here an
app-using component can not infect a new component, because infected app-users
will belong to the same component. Consequently, we have that

$m_{11}=0.$ (4)

The group of non-app-users will never be reached by digital tracing and thus
spread the infection simply like in a general epidemic without any contact
tracing. That is, each infectious non-app-user infects non-app-users at rate
$\beta(1-\pi)$ and app-users at rate $\beta\pi$, where the infectious periods
of non-app-users are independent and identically exponentially distributed
with intensity ($\delta+\gamma$). Hence, the number of app-using component
(type 1) and non-app-users (type 2) infected by one typical non-app-user are
geometrically distributed with mean

$m_{21}=\frac{\beta\pi}{\delta+\gamma},$ (5)

and

$m_{22}=\frac{\beta(1-\pi)}{\delta+\gamma}$ (6)

respectively.

Finally, it remains to compute $m_{12}.$ Following the idea in
[zhang_analysing_2021], let $Z^{(a)}$ be the total number of non-app-users
born by an app-using component, then we have

$m_{12}=\mathbb{E}[Z^{(a)}]=\mathbb{E}\big{[}\sum_{i=1}^{N^{(a)}_{c}}X^{(na)}_{i}\big{]},$

where $X^{(na)}_{i}$ denotes the number of non-app-users infected between the
$i-1$’th and the $i$’th jump, and $N^{(a)}_{c}$ denotes the number of jumps
that an app-using component makes before dying out and hence stopping
infecting.

For the distribution of $X^{(na)}_{i}$ and $N^{(a)}_{c}$, we briefly state the
idea as follows (see [zhang_analysing_2021] for details). If currently there
are $k$ infectious app-users in an app-using component, the size of this
component increases by 1 when the component infects an app-user, hence at rate
$k\beta\pi$; decreases by 1 if there is a natural recovery (rate $k\gamma$);
and drops to 0 if an app-user is diagnosed which happens at rate $k\delta$.
Thus we consider the process of app-using components as a simple random walk
(new infection/ natural recovery) with ”killing events” (diagnosis) added.
$N^{(a)}_{c}$ is the number of jumps until the component has no infectives,
something which can happen either if all infectious app-user have recovered
naturally or if an infectious app-user is diagnosed.

Furthermore, if currently there are $k$ infectives in the component, the time
until the next event occurs is exponentially distributed with intensity
$k(\beta\pi+\gamma+\delta).$ During this time, such component infects the non-
app-users at $k\beta(1-\pi)$. It follows that the numbers $X^{(na)}_{i}$ of
infections of non-app-users between jumps are identically and independently
distributed (geometric distributed with mean
${\beta(1-\pi)}/({\beta\pi+\gamma+\delta})$) and independent of the current
size of the component.

Consequently, we derive the mean number $m_{12}$ of non-app-users (type 2)
infected by an app-using component (type 1) given by

$m_{12}=\mathbb{E}[X^{(na)}]\mathbb{E}[N^{(a)}_{c}],$ (7)

with

$\mathbb{E}[X^{(na)}]=\frac{\beta(1-\pi)}{\beta\pi+\gamma+\delta},$ (8)

and

$\mathbb{E}[N^{(a)}_{c}]=1+\sum_{k=1}^{\infty}\mathbf{P}(N^{(a)}_{c}>k),$ (9)

where for $k\geq 1,$ the distribution of $N^{(a)}_{c}$ is given by

$\mathbf{P}(N^{(a)}_{c}>k)=\bigg{(}1-\sum_{j=1}^{\left\lceil{k/2}\right\rceil}\frac{1}{2j-1}\binom{2j-1}{j}{\bigg{(}\frac{\beta\pi}{\gamma+\beta\pi}\bigg{)}}^{j-1}{\bigg{(}\frac{\gamma}{\gamma+\beta\pi}\bigg{)}}^{j}\bigg{)}\cdot\bigg{(}\frac{\beta\pi+\gamma}{\beta\pi+\gamma+\delta}\bigg{)}^{k}.$

In conclusion, we obtain with Equation (3) that the component reproduction
number $R_{D}$ for the model with digital tracing only is given by

$R_{D}=\frac{m_{22}}{2}+\sqrt{\frac{{m_{22}}^{2}}{4}+m_{12}m_{21}},$ (10)

with the elements $m_{ij}$ given by Equations (5)-(7). Using the theory of
branching processes (see e.g. [ball_final_1995, andersson_stochastic_2000]),
we collect our findings in the following proposition:

######  Proposition 3.1.

Consider a sequence of epidemic processes with digital tracing
$E^{(n)}_{D}(\beta,\gamma,\delta,\pi)$, for population size $n$, and starting
with one initial infective. Then on any finite time interval,
$E^{(n)}_{D}(\beta,\gamma,\delta,\pi)$ converges in distribution to the two-
type branching process $E_{D}(\beta,\gamma,\delta,\pi)$ as $n\to\infty.$

Let $Z_{\infty}$ denote the number of individuals ever born in the branching
process $E_{D}(\beta,\gamma,\delta,\pi)$, then we have
$\mathbf{P}(Z_{\infty}<\infty)=1$ if $R_{D}\leq 1$ and
$\mathbf{P}(Z_{\infty}<\infty)<1$ if $R_{D}>1.$

Let $Z_{n}$ be the final number infected in the epidemic, then
$\mathbf{P}(Z_{n}>\sqrt{n})\to 0$ as $n\to\infty$ if and only if $R_{D}\leq
1.$

####  3.1.2 The individual reproduction number

The interpretation of $R_{D}$ is complicated in that the two types in the
underlying branching process $E_{D}(\beta,\gamma,\delta,\pi)$ are different
types of objects: type-2 individuals are actual individuals (non-app-users),
but type-1 are ”macro individuals”: the to-be-traced app-using components. In
this section, our aim is to convert $R_{D}$ to an individual reproduction
number.

We first define the average number of individuals (of any type)
$R^{(ind)}_{D-A}$ infected by a newly infected app-user and $R^{(ind)}_{D-N}$
by an non-app-user, respectively. We note that each infection, irrespective of
the type of infector, is with an app-user with probability $\pi$ and a non-
app-user with probability ($1-\pi$). This implies that the $R^{(ind)}_{D}$ can
be expressed as the sum:

$R^{(ind)}_{D}=\pi R^{(ind)}_{D-A}+(1-\pi)R^{(ind)}_{D-N}.$ (11)

It remains to compute the $R^{(ind)}_{D-A}$ and $R^{(ind)}_{D-N}$. From the
previous results in Section 3.1.1, we see that a given app-using component
infects on average $m_{12}$ number of non-app-users. Let $\mu^{(A)}_{c}$ be
the mean number of app-users that ever get infected in an app-using component.
All but one of these $\mu^{(A)}_{c}$ are infected within the component, and
there are, on average, $m_{12}$ number of ”external” infections. So in total,
there are on average $(\mu^{(A)}_{c}-1)+m_{12}$ number of infections by an
app-using component. As a consequence, the average number $R^{(ind)}_{D-A}$ of
infections _per app-user_ equals

$R^{(ind)}_{D-A}=\frac{(\mu^{(A)}_{c}-1)+m_{12}}{\mu^{(A)}_{c}},$ (12)

where

$\mu^{(A)}_{c}=1+\frac{\beta\pi}{\beta\pi+\gamma}\sum_{k=1}^{\infty}\mathbf{P}(N^{(a)}_{c}>k-1)=1+\frac{\beta\pi}{\beta\pi+\gamma}\mathbb{E}[N^{(a)}_{c}].$
(13)

We skip the proof for the computation of $\mu^{(A)}_{c}$; an identical
argument was used in [zhang_analysing_2021].

For non-app-users, it is easier to analyse since a given infectious non-app-
user, on average, infects $m_{21}+m_{22}$ number of individuals (app-users and
non-app-users). Using Equations (5) and (6) it follows that

$R^{(ind)}_{D-N}=m_{21}+m_{22}=\frac{\beta}{\delta+\gamma},$ (14)

which is identical to $R_{0}$ (see Equation (1)). This is not surprising since
non-app-users will never be contact traced.

An individual reproduction number $R^{(ind)}_{D}$ for digital tracing is thus
given by

$\begin{split}R^{(ind)}_{D}&=\pi\frac{(\mu^{(A)}_{c}-1)+m_{12}}{\mu^{(A)}_{c}}+(1-\pi)\frac{\beta}{\delta+\gamma},\end{split}$
(15)

with $m_{12}$ defined in Equation (7) and $\mu^{(A)}_{c}$ in Equation (13).

######  Remark 3.1.

As discussed in [zhang_analysing_2021], this individual reproduction number
$R^{(ind)}_{D}$ does not possess the traditional interpretation: the average
number of infections caused by infected people at the beginning of the
outbreak. This is due to the delicate timing of event issues closely related
to those explained in [ball_trapman_2016]. However, it still has the correct
threshold property as supported numerically in Section 4.

###  3.2 The epidemic model with digital and manual contact tracing

We now analyse the combined model with both manual and digital contact
tracing.

####  3.2.1 Properties of limiting branching process

Also, the combined model can be approximated by a suitable two-type branching
process $E_{DM}(\beta,\gamma,\delta,p,\pi)$ during the early stage of the
epidemic, assuming a large population. Clearly, for the combined model, the
app-users could not only be traced by app-users, but also by non-app-users
with probability $p.$ On the other hand, the non-app-users could be traced by
individuals (either app-users or non-app-users) with probability $p$.
Accordingly, to-be-traced components now make up infections between app-users
and non-app-users who will be traced manually. Moreover, new components are
created if an app-user infects a non-app-user without manual contact tracing
taking place or if a non-app user infects either type and there is no manual
contact tracing. The new components hence differ in how they are created: with
an app-user (infected by a non-app-user without manual contact tracing) or a
non-app-user (without manual contact tracing), and the future evolvement of
such components depend on the current number of infectious app-users and non-
app-users. We hence have two types of to-be-traced components, depending on if
it was initiated by an app-user or non-app-user. A component starting with a
non-app-user increases its size at rate $\beta p$ (only manual tracing works
for non-app-users), whereas a component starting with an app-user increases
its size at rate $\beta(1-\pi)p+\beta\pi$.

Next, we give an example in Figure 3.2 to illustrate how the two types of
components (starting with an app-user or a non-app-user) grow and reproduce.
We start with case 1, which is the root of the component $C^{(NA)}_{1}$, which
generates two new components $C^{(NA)}_{2}$ and $C^{(A)}_{3}$. The component
$C^{(NA)}_{2}$ is with root 2 as a non-app-user and $C^{(A)}_{3}$ starts with
an app-user 3. Once individual 1 is diagnosed, the whole component
$C^{(NA)}_{1}$ is diagnosed.

Figure 3.2: Example of the process of to-be-traced components started by app-
users and non-app-users for the combined model: the circles are app-users and
squares non-app-users. The nodes in white, black, and grey are regarded as
”infectious”, ”diagnosed” and ”naturally recovered”, respectively. The
rectangular region surrounded by the dashed line is for the component, whereas
the area is filled with slashed lines when the whole component is removed.
Full edges stand for contacts that could be traced either by manual or digital
contact tracing (so always full edges between each pair of app-users), whereas
dashed ones are for those non-to-be-traced.

Another difference from the analysis for digital tracing is that the to-be-
traced components may also contain non-app-users that are manually traced. So
it is necessary to keep track of both the number of infectious app-users and
infectious non-app-users in the component.

Let $N^{(i)}_{j}(t)$ be the number of infectious type-$j$ individuals in the
component starting with one type-$i$ individual at time $t$, where we set
type-$1$ to be app-users and type-$2$ to be non-app-users. Suppose that at
time $t$ we have

$N^{(i)}(t)=(k,l),\ \text{i.e.}\ =(N^{(i)}_{1}(t)=k,N^{(i)}_{2}(t)=l),$

i.e. there are $k$ infectious app-users and $l$ infectious non-app-users in a
component starting with a type-$i$ individual. We note that
$N^{(1)}(0)=(1,0)$, whereas $N^{(2)}(0)=(0,1)$.

$N^{(i)}$ increases or decreases by 1 in the first component if there is a
newly infected app-user or one of the $k$ app-users recovers naturally,
respectively. The rate at which $N^{(i)}$ increases by $(1,0)$ is hence
$k\beta\pi+l\beta\pi p$ (the rate that app-users infect plus the rate at which
non-app-users infect), and $N^{(i)}$ decreases by $(-1,0)$ at rate $k\gamma$
(recovery of an app-user). Further, $N^{(i)}$ increases by 1 in the second
component at rate $(k+l)\beta(1-\pi)p$, because this newly infected non-app-
user shall have a manual tracing link, irrespective of the type of infector.
Furthermore, $N^{(i)}$ decreases by 1 in the second component at rate
$l\gamma$ (recovery of a non-app-user). Finally, $N^{(i)}$ drops directly to
$(0,0)$ at rate $(k+l)\delta$ when one of the infectious individuals (of
either type) in the component is diagnosed.

Next, we describe the birth of new components. We note that only infections
with no digital or manual contact tracing links create a new component. This
can only occur to pairs of non-app-users or pairs of app-users and non-app-
users, but it never happens between two app-users. As a consequence, given
that there are $N^{(i)}(t)=(k,l)$ ($k$ infectious app-users and $l$ infectious
non-app-users), the component gives birth to new components with an app-using
root at rate $l\beta\pi(1-p)$, and new components with a non-app-using root at
rate $(k+l)\beta(1-\pi)(1-p).$

In conclusion, at the beginning of an outbreak and for a large population, we
approximate the epidemic with both manual and digital tracing by a two-type
branching process $E_{DM}(\beta,\gamma,\delta,p,\pi)$ with ”individuals” of
type 1 are to-be-traced components starting with an app-user and type 2
”individuals” are components with a non-app-using root. Again, we can apply
the coupling argument (see, e.g., [ball_strong_1995,
andersson_stochastic_2000, britton_stochastic_2010]) to show that the epidemic
process in terms of the two type components converges to the two-type
branching process $E_{DM}(\beta,\gamma,\delta,p,\pi)$ as $n\to\infty$.

Now we derive the corresponding component reproduction number $R_{DM}$ which
is known as the dominant eigenvalue of the corresponding mean offspring matrix
$M^{(c)}$ (see [athreya_branching_1972, becker_multi_type_1990]) given by :

$M^{(c)}=\begin{pmatrix}m^{(c)}_{11}&m^{(c)}_{12}\\\ \\\
m^{(c)}_{21}&m^{(c)}_{22}\end{pmatrix},$ (16)

in which the element $m^{(c)}_{ij}$, is the mean number of new components of
type $j$ produced by a single component of type $i$, for $i,j=1,2.$ The
component reproduction number $R_{DM}$ is hence given by

$R_{DM}=\frac{m^{(c)}_{11}+m^{(c)}_{22}}{2}+\sqrt{\frac{(m^{(c)}_{11}+m^{(c)}_{22})^{2}}{4}-m^{(c)}_{11}m^{(c)}_{22}+m^{(c)}_{12}m^{(c)}_{21}}.$
(17)

Based on the previous discussion, we have derived the birth rates of new
components of type $i$ conditional on $N^{(i)}(t)$. The expected total numbers
$m^{(c)}_{ij}$ of new type-$j$ components produced by a type-$i$ component are
hence the expected values of these expressions integrated over time:

$\displaystyle m^{(c)}_{11}$
$\displaystyle=\int_{0}^{\infty}\mathbb{E}[N^{(1)}_{2}(t)]\beta\pi(1-p)dt,$
(18) $\displaystyle m^{(c)}_{12}$
$\displaystyle=\int_{0}^{\infty}(\mathbb{E}[N^{(1)}_{1}(t)]+\mathbb{E}[N^{(1)}_{2}(t)])\beta(1-\pi)(1-p)dt,$
(19) $\displaystyle m^{(c)}_{21}$
$\displaystyle=\int_{0}^{\infty}\mathbb{E}[N^{(2)}_{2}(t)]\beta\pi(1-p)dt,$
(20) $\displaystyle m^{(c)}_{22}$
$\displaystyle=\int_{0}^{\infty}(\mathbb{E}[N^{(2)}_{1}(t)]+\mathbb{E}[N^{(2)}_{2}(t)])\beta(1-\pi)(1-p)dt.$
(21)

We have not been able to simplify these expressions further analytically. The
main reason for this is that not all the rates of the four events occurring in
the component are linear in the number of infectious individuals. In
particular, the rate at which $N^{(i)}=(k,l)$ increases in the first component
equals $k\beta\pi+l\beta\pi p$ (non-linear in $k+l$).

In Section 4, we perform simulations to derive the $m_{ij}$ and numerically
compute the component reproduction number $R_{DM}$ for given parameters. Based
on the numerical results, we analyze how $R_{DM}$ depends on the manual
tracing probability $p$ and the app-using fraction $\pi$.

####  3.2.2 The reduction effect of manual and digital contact tracing

We recall that $R_{0}={\beta}/({\delta+\gamma})$ is the basic reproduction
number (without any type of contact tracing), $R_{M}$ is the effective
component reproduction number for manual tracing only (derived in
[zhang_analysing_2021]), $R_{D}$ is the component reproduction number for
digital tracing only (Equation (3)), and $R_{DM}$ is the component
reproduction number for the combined model (Equation (17)).

Further, let $r_{M},r_{D}$ and $r_{DM}$ denote the three relative reductions
(of $R_{0}$) by manual, digital contact tracing only and both type of contact
tracing, respectively:

$R_{M}=(1-r_{M})R_{0},R_{D}=(1-r_{D})R_{0},R_{DM}=(1-r_{DM})R_{0}.$

If the two effects from manual and digital contact tracing acted independently
when combined, we would have

$1-r_{DM}=(1-r_{M})(1-r_{D}),$

which implies that the reproduction number for the combined model would be
given by

$(1-r_{M})(1-r_{D})R_{0}.$ (22)

######  Remark 3.2.

It is worth highlighting that the reproduction numbers $R_{0}$, $R_{D}$,
$R_{M}$ and $R_{DM}$ are not in the same scale, however all of them are
epidemic thresholds meaning that the models are super-critical if and only if
the corresponding reproduction number exceeds 1. For this reason, we mainly
focus on the cases when the quantity in Equation (22) equals one: if we have
$R_{DM}<1$ and $R_{0}(1-r_{M})(1-r_{D})>1$, then the combined effect (of
reducing $R_{0}$) would be bigger than the product of each of the preventive
effects.

Later in Section 4, we will plot the critical combinations of $(p,\pi)$ for
which $R_{DM}$ equals 1 and the corresponding values for which
$(1-r_{M})(1-r_{D})R_{0}$ equals 1. And it will show that the reduction
$r_{DM}$ is in fact bigger than $1-(1-r_{M})(1-r_{D})$.

##  4 Numerical Illustration

In this section, we numerically compare the effects of digital and manual
tracing and the combined model. We also investigate the effect of the testing
rate $\delta$, where we vary the fraction getting tested:
$\delta/(\delta+\gamma)$ rather than varying $\delta.$ For all the numerical
results below (except Table 4.1), we set the epidemic parameters to be fixed:
$\beta=6/7,\gamma=1/7$ so that the reproduction number without any testing or
contact tracing is $\beta/\gamma=6$. When also considering diagnosis
($\delta>0$) , we would have the basic reproduction number
$R_{0}={\beta}/({\gamma+\delta})$ smaller than 6.

###  4.1 Comparison of digital and manual tracing only

In Figure 1(a), we plot the necessary testing rate (measured by testing level
($\delta/(\delta+\gamma)$) for which the manual $R_{M}$ equals 1 (see
[zhang_analysing_2021]) and digital component reproduction number $R_{D}$
equals 1 (see Equation (10)), for a given value of $p$ and $\pi$ respectively.
It shows that for a given testing level $\delta/(\delta+\gamma)$, a larger
app-using fraction $\pi$ is required compared to $p$, the fraction of contacts
that are successfully manually traced, for which the corresponding
reproduction number equals 1. The explanation is that the effectiveness of
digital contact tracing is more connected to the square of the fraction of
app-users ($\pi^{2}$): the digital tracing is carried out only if both the
confirmed case and the contact are app-users.

In Figure 1(b), we hence plot the critical combination of
$\delta/(\delta+\gamma)$ against $\pi^{2}$ instead. It can be seen that the
curve for $\pi^{2}$ is closer to the one for $p$, but still not identical: we
still need a larger value on $\pi^{2}$ for the reproduction number to equal
unity.

(a)

(b)

Figure 4.1: Plot of the effective combination of testing fraction
$\delta/(\delta+\gamma)$ and tracing probability (manual tracing probability
$p$ or 1(a): fraction $\pi$ of app-users; 1(b): squared fraction $\pi^{2}$ of
app-users) such that the reproduction number ($R_{M}$ or $R_{D}$) equals one:
the curve (in red) is for model only with digital tracing, whereas the curve
(in blue) is for model only with manual tracing.

###  4.2 The individual and component reproduction numbers for digital tracing
only

In Figure 4.2, we plot how the digital reproduction number $R_{D}$ and
individual reproduction number $R^{(ind)}_{D}$ vary with the testing fraction
$\delta/(\delta+\gamma)$ in $[0,5/6]$ and the fraction of app-users $\pi$ in
$[0,1]$. By looking at the colors at the bottom in Figure 2(a), we observe
that the component reproduction number $R_{D}$ is not monotonically decreasing
in $\pi$, the fraction of app-users. This is a highly surprising result since
we expect a smaller reproduction number for a bigger fraction $\pi$ that use
the app and hence perform digital contact tracing.

This non-monotonicity of $R_{D}$ is explained as follows. First of all, it can
be easily seen from Equations (5) and (6) that the element $m_{21}$ is
increasing and $m_{22}$ is decreasing with $\pi.$ Then, at a very low level of
testing fraction, increasing $\pi$ leads to more app-users, and thus the
_size_ of the app-using component will grow even if fewer non-app-users will
be infected per app-user. The average number of non-app-users infected by one
app-using component $m_{12}$ may hence increase with $\pi$. Then according to
the expression of $R_{D}$ (see Equation (10)), $R_{D}$ could increase with
$\pi$ when the testing fraction $\delta/(\delta+\gamma)$ is low. On the other
hand, we see from Figure 2(b) that the individual reproduction number
$R^{(ind)}_{D}$ is monotonically decreasing in $\pi$ as expected.

(a)

(b)

Figure 4.2: Heatmap of the effective reproduction number, 2(a) for component
reproduction number and 2(b) for individual reproduction number varying with
testing fraction $\delta/(\delta+\gamma)$ and fraction of app-users $\pi$;
with $\beta=6/7,\gamma=1/7$ fixed. The lines indicate where $R_{D}=1$ (in
white) and $R^{(ind)}_{D}=$ 2.5, 2, 1.5 (in yellow dashed, from left to right)
and 1 (in white), respectively.

Further, we check if the individual reproduction number $R^{(ind)}_{D}$ has
the correct threshold property, i.e. see if it equals 1 exactly when $R_{D}$
does. We first fix the testing rate $\delta=1/7,$ so that there are $50\%$ of
the infected individuals are tested and isolated, and $50\%$ recover naturally
while still infectious, implying that $R_{0}=\beta/(\gamma+\delta)=3.$ Then we
compare the app-using fractions $\pi$ such that $R^{(ind)}_{D}$ or $R_{D}$
equals 1. In Figure 4.3, we plot the curves of reproduction numbers
$R^{(ind)}_{D}$ (in red) and $R_{D}$ (in blue) against the fraction of app-
users $\pi$, where it seems to confirm that both reproduction numbers equal 1
at the same $\pi$, thus supporting our conjecture in Remark (3.1).

Figure 4.3: Plot of two curves of $R^{(ind)}_{D}$ (in red) and $R_{D}$ (in
blue) against $\pi$ with $\beta=6/7,\gamma=1/7$ and $\delta=1/7$ fixed. The
horizontal dashed line stands for when the reproduction number equals 1.

###  4.3 Effect of combination of digital and manual tracing

Finally, we perform simulations to derive the reproduction number $R_{DM}$ for
the combined model shown in Figure 4.4. In Figure 4(a), we show the heatmap of
$R_{DM}$ as a function of $p$ and $\pi$ (for $\delta=1/7$) and add two curves:
one curve is the true reproduction number $R_{DM}=1$ in Equation (17) and the
second curve is the naïve guess one $R_{0}(1-r_{D})(1-r_{M})=1$ in Equation
(22).

We see from Figure 4(a) that the reproduction number $R_{DM}$ appears to be
monotonically decreasing both with fraction $p$ of being successfully manually
contact traced and the fraction $\pi$ of app-users. Moreover, we observe from
the two critical curves that for any combination of $(p,\pi)$ such that the
naïve guess $R_{0}(1-r_{D})(1-r_{M})=1$, we have the true reproduction number
$R_{DM}<1$. It implies that the combined effect of having both manual and
digital tracing is bigger than the product of the preventive effect when
having each one of them. An explanation to this is that a contact tracing
event from digital contact tracing also shortens the infectious periods for
some possible infections where manual tracing would take place, and vice
versa.

Nevertheless, we note that for small values of $\delta/(\delta+\gamma)$, the
component reproduction number $R_{D}$ can be increasing in $\pi$. Similarly
for the combined model, if we choose $\delta=1/28$ (not realistic but as an
illustration), we see in Figure 4(b) that the component reproduction number
$R_{DM}$ is not monotonically decreasing with either probability $p$ or $\pi$.
And the curve of $R_{DM}=1$ seems to be monotonically decreasing.

(a) with testing fraction $\frac{\delta}{\delta+\gamma}=0.5$

(b) with testing fraction $\frac{\delta}{\delta+\gamma}=0.2$

Figure 4.4: Plot of reproduction number $R_{DM}$ for the combined model
varying with manual tracing probability $p$ and fraction of app-users $\pi$;
with $\beta=6/7,\gamma=1/7$ fixed, 4(a) for $\delta=1/7$ and 4(b) for smaller
$\delta=1/28$: the curve in purple stands for the case when $R_{DM}=1$ and the
curve in white for the case when $R_{0}(1-r_{D})(1-r_{M})=1$. Moreover, the
dashed lines in black in 4(a) are for the cases when $R_{DM}=2.5,2,1.5,$
respectively from left to right.

Finally in Table 4.1, we compute the reproduction number $R_{DM}$ and its
naïve guess $R_{0}(1-r_{D})(1-r_{M})$ in four different cases: without any
contact tracing ($p=\pi=0$); with digital tracing only ($p=0,\pi=2/3$); with
manual tracing only: ($p=2/3,\pi=0$) and with both manual and digital tracing
($p=\pi=2/3$), keeping all other parameters fixed:
$\beta=0.8,\gamma=1/7,\delta=1/7$. For each case, we perform 10 000
simulations of the epidemic with the chosen parameters and population size
$n=5000$, starting with one initial infective. It can be seen from Table 4.1
that the fraction of major outbreaks (considered as more than $10\%$ have been
infected) and the mean fraction of infected among the major outbreaks both
_decrease_ with the reproduction number, as expected (see Proposition 3.1).

In particular, from the last line of Table 4.1 we note that $R_{DM}$ is below
1 even if the naïve guess $R_{0}(1-r_{D})(1-r_{M})$ is above 1, which confirms
that the combined effect of both manual and digital tracing is bigger than two
separate effects multiplied by each other, i.e. $r_{DM}>1-(1-r_{D})(1-r_{M}).$

Table 4.1: Simulation results in four different cases of $p$ and $\pi$ $p$ | $\pi$ | the reproduction | fraction of | mean fraction of infected
---|---|---|---|---
|  | number | major outbreaks | among major outbreaks
0 | 0 | $R_{0}=2.80$ | 0.64 | 0.93
0 | 2/3 | $R_{D}=2.20$ | 0.49 | 0.81
2/3 | 0 | $R_{M}=1.49$ | 0.46 | 0.75
2/3 | 2/3 | $R_{DM}=0.92$ | 0.01 | 0.14
|  | $R_{0}(1-r_{D})(1-r_{M})=1.17$ |  |

##  5 Discussion

In this paper, we analyzed a Markovian epidemic model with digital as well as
manual contact tracing. For the epidemic model with digital tracing only, the
early stage of the epidemic was approximated by a two-type branching process
with one type being to-be-traced app-using components and the other being non-
app-users, and $R_{D}$ being the largest eigenvalue of the matrix of mean
offspring. The individual reproduction number $R^{(ind)}_{D}$ was also
derived. The dependency of reproduction numbers on the effectiveness of
testing and contact tracing was assessed analytically and numerically.
Surprisingly, the reproduction number $R_{D}$ for digital tracing was not
monotonically decreasing in $\pi$, the fraction of app-users, whereas the
individual reproduction number $R^{(ind)}_{D}$ seems to be. When comparing
digital tracing with manual tracing, a smaller tracing probability $p$ is
needed than the corresponding app-using fraction $\pi$ to reach the same
effectiveness. This result is not very surprising because both sides of the
infection need to be app-users for digital contact tracing to take place.

We then analyzed the combined model and approximated the beginning of the
epidemic by a different two-type branching process, with both types being to-
be-traced components and only differing on the type of the root (app-user or
not). The corresponding reproduction number was derived numerically.
Importantly, the preventive effect of combining the two contact tracing
methods was shown to be bigger than the product of two separate effects.

There are several extensions of the present model that would make it more
realistic. On the one hand, we assume that there is no delay in both manual
and digital contact tracing, i.e., any traced contact (either by manual or
digital tracing) is assumed to be immediately diagnosed. This neglects one
crucial difference between manual and digital tracing: manual tracing is time-
and labor-intensive, so there is often a delay between the case confirmation
and notification of contacts, whereas digital tracing is designed to avoid (or
shorten) the delay. Additionally, one limitation of a contact tracing app is
that there could be close contacts that can lead to transmission but occur too
quickly to be recorded by the app. In this sense, our model could be extended
by considering the fact that not all the contacts between pairs of app-users
could be reached by digital tracing. Further, we make the simplifying
assumption that traced individuals who have by then recovered are also
identified and contact traced (see [muller_contact_2000] which do not make
this assumption). In reality, this may happen for individuals who have
recently recovered, but perhaps not for those who have recovered several weeks
earlier. Therefore, our results in this paper give an upper bound on how
effective the real contact tracing would be.

A different extension would be to consider a structured community (e.g., a
social network) combined with homogeneous random contacts. Then, digital
contact tracing would work in the same way for random contacts and contacts in
the social network (as long as both are app-users), whereas manual contact
tracing would most likely only happen on the social network (very rarely could
you name whom you sat next to in the bus).

On the mathematical side, we only focus on the early stage of the epidemic in
this paper. Analysing later parts of the epidemic remains open. Moreover,
there are problems with obtaining an analytic expression for the reproduction
number for the combined model.

Despite these model limitations, we consider our contribution an important
step in increasing understanding of the effect of different contact tracing
forms on spreading disease.

### Acknowledgments

T.B. is grateful to the Swedish Research Council (grant 2020-04744) for
financial support.

## References


[datatype=bibtex] [overwrite] [fieldsource=shortjournal]
[fieldset=journaltitle,origfieldval]

# 19 Parameters Is All You Need:
Tiny Neural Networks for Particle Physics

Alexander Bogatskiy
Center for Computational Mathematics
Flatiron Institute, New York, NY, U.S.A.
<EMAIL_ADDRESS>
Timothy Hoffman
Department of Physics, University of Chicago
Chicago, IL, U.S.A.
<EMAIL_ADDRESS>
Jan T. Offermann
Department of Physics, University of Chicago
Enrico Fermi Institute
Chicago, IL, U.S.A.
<EMAIL_ADDRESS>

###### Abstract

As particle accelerators increase their collision rates, and deep learning
solutions prove their viability, there is a growing need for lightweight and
fast neural network architectures for low-latency tasks such as triggering. We
examine the potential of one recent Lorentz- and permutation-symmetric
architecture, PELICAN, and present its instances with as few as 19 trainable
parameters that outperform generic architectures with tens of thousands of
parameters when compared on the binary classification task of top quark jet
tagging.

##  1 Introduction

Particle collisions at the Large Hadron Collider at CERN happen every 25
nanoseconds, producing immense amounts of data that have to be processed in
real time. Much of the event filtering is done by the Level-1 trigger [5, 1],
which uses algorithms implemented on FPGAs that need to operate at below-
microsecond latency to avoid loss of valuable data. Low-latency tasks include
charged particle track reconstruction and energy measurements. Implementing
neural networks under such constraints is a significant challenge, however the
most recent attempts to do so have finally surpassed their traditional non-ML
counterparts. The current state-of-the-art implementations in this area are
based on the JEDI-net Graph Neural Network (GNN) architecture, see [14, 18,
19]. The network input data consist of lists of jet constituents, with a
certain number of geometric features describing each constituent.

GNN architectures are inherently permutation-equivariant, providing a
significant boost to efficiency and model size by virtue of weight sharing,
but no other physical symmetries are necessarily respected. Physics-informed
architectures that are inherently equivariant with respect to rotational and
Lorentz-boost symmetries have recently shown themselves to provide state-of-
the-art performance at tasks such as jet tagging (see e.g. [2, 8, 11, 12, 3,
4]), and they do so despite the relatively small model size.

In this work we study the current state-of-the-art architecture for top-quark
jet tagging, PELICAN [4]. It is fully Lorentz-invariant and its permutation-
equivariant layers are based on the general higher-order permutation-
equivariant mappings introduced in [13, 15]. The full reduction of all
relevant symmetries allows small instances of PELICAN with just a few thousand
parameters to perform on par with much larger models with hundreds of
thousands or even millions of parameters. Moreover, the simplicity of the
architecture presents a unique opportunity for explainability and even
interpretability.

Our goal here is to explore the small model size limit of PELICAN and compare
it against the previous state-of-the-art (and also Lorentz-equivariant)
architecture, LorentzNet [8]. The benchmark task for this comparison is that
of top-quark tagging due to the publicly available dataset [9] and the
extensive prior exploration of architectures trained on it [11]. The input
consists of a list of $N$ 4-momenta of jet constituents,
$\\{p_{i}\\}_{i=1}^{N}$ which PELICAN reduces to the $N\times N$ array of
pairwise Lorentz-invariant dot products, $d_{ij}=p_{i}\cdot p_{j}$. Thus the
reduced input is an array with one channel. We find that a stripped down
version of PELICAN consisting of nothing but two linear permutation-
equivariant blocks with just two channels in the hidden layer and exactly one
nonlinear activation function in between outperforms generic architectures
such as the fully-connected TopoDNN which has 59k parameters [11]. This model
nominally has 26 parameters, but through absorption of multiplicative factors
and a simplification of the output layer that number can be effectively
reduced to 19. Despite the costly $N^{2}$ scaling of the memory that PELICAN
requires, its symmetric architecture can provide ultra-lightweight networks
that can be viably used in low-latency and high-throughput applications.

##  2 The original PELICAN architecture

The original PELICAN architecture consists of an input block which encodes the
$N\times N$ array of pairwise dot products $\\{d_{ij}\\}$, followed by a
sequence of so-called $\text{Eq}_{2\to 2}$ permutation-equivariant blocks (the
index $2$ indicates the rank of the input and output arrays) that produce
transformed $N\times N$ arrays. Each of these blocks consists of a fully-
connected “messaging” layer that mixes the channels but is shared among all
components of the $N\times N$ array, and an “aggregation” layer that applies a
general linear permutation-equivariant operation that exchanges information
between the various components of the array. Finally, a similar
$\text{Eq}_{2\to 0}$ block reduces the array to a permutation-invariant (rank
0) scalar, after which an output MLP layer produces the two binary
classification weights $\\{w_{0},w_{1}\\}$. The diagram below summarizes this
architecture, see [4] for details.

${\\{d_{ij}\\}}$Emb${\left[\mathrm{Eq_{2\to 2}}\right]^{L}}$${\mathrm{Eq_{2\to
0}}}$${\mathrm{MLP}}$${\\{w_{c}\\}}$

Notably, the aggregation step inside $\text{Eq}_{2\to 2}$, called
$\text{LinEq}_{2\to 2}$, applies 15 different operations that provide a basis
for the space of all linear permutation-equivariant transformations of rank 2
arrays, which temporarily increases the size of the activation by a factor of
15, marking the peak of PELICAN’s memory utilization. This is followed by a
trainable linear layer that applies $(C_{\text{in}}\times 15)\times
C_{\text{out}}$ weights and adds two biases per channel (one bias added to the
entire $N\times N$ array and one only to the diagonal), where $C_{\text{in}}$
and $C_{\text{out}}$ are the number of input and output channels. Similarly,
$\text{Eq}_{2\to 0}$ involves only 2 aggregators (total sum and trace), a
linear layer of shape $(C_{\text{in}}\times 2)\times C_{\text{out}}$, and one
bias per channel.

##  3 nano PELICAN architecture

In this section we simplify the PELICAN architecture to a single hidden layer
and reduce parameters further based on symmetry arguments, which we refer to
as nano PELICAN (nPELICAN). The only two linear symmetric observables that can
be constructed from the input dot products (assuming sum-based aggregation
that does not explicitly depend on the multiplicity $N$) are the jet mass
$m_{J}^{2}=\sum_{i,j}d_{ij}$ and the total mass $\sum_{i}d_{ii}$. The top-
tagging dataset has only massless constituents, so the latter observable is
irrelevant. A non-parametric top-tagger based on a simple jet mass cut
achieves an AUC of only $90.6\%$, therefore a linear PELICAN cannot exceed
this. To this end, we set out to find the smallest and most interpretable
instance of PELICAN that is still nonlinear and performs competitively on the
top-tagging task. We thus omit the input embedding layer, all messaging layers
(including batch normalizations), and the output MLP, and are left with just
two linear equivariant blocks, $\text{LinEq}_{2\to 2}$ and $\text{LinEq}_{2\to
0}$, separated by a single activation function, which we choose to be ReLU.
The architecture is summarized in the following diagram:

${\\{d_{ij}\\}}$${\text{LinEq}_{2\to
2}^{\text{nano}}}$ReLU${\text{LinEq}_{2\to 0}}$${\\{w_{c}\\}.}$

Here, we also notice that since the array of dot products, $\\{d_{ij}\\}$, is
symmetric, and since the constituents in the top-tagging dataset are massless,
many of the 15 basis aggregators in $\text{LinEq}_{2\to 2}$ are redundant. We
remove 5 aggregators that depend only on the diagonal of the input, and one
from each of 4 pairs of aggregators that attain the same value on symmetric
inputs. We are left with just 6 aggregators which constitute
$\text{LinEq}_{2\to 2}^{\text{nano}}$. To help with training, each equivariant
layer is still preceded by a Dropout layer. Moreover, keeping BatchNorm layers
just before the dropout can also help the model converge, meanwhile the extra
parameters from these layers can be completely absorbed into the linear layers
for inference.

The only remaining hyperparameter is $C_{\text{hidden}}$, the number of
channels in the hidden layer (between $\text{LinEq}_{2\to 2}$ and
$\text{LinEq}_{2\to 0}$). The total number of parameters is then $1\times
6\times C_{\text{hidden}}+2\cdot C_{\text{hidden}}+C_{\text{hidden}}\times
2\times 2+2=12C_{\text{hidden}}+2$ (ignoring BatchNorm). In addition, since
for binary classification only the difference in weights $w_{1}-w_{0}$
matters, it is possible to have only 1 output channel, in which case we have
$10C_{\text{hidden}}+1$ parameters. The models presented below produce only
one output weight called $w$.

##  4 Top tagging performance

The top tagging dataset [9] consists of anti-$k_{T}$ jets [6] corresponding
with top quarks (signal) and light quarks or gluons (background). It includes
up to 200 jet constituents per entry, each represented by a 4-momentum in
Cartesian coordinates. A converted version of the dataset that can be directly
used with PELICAN can be found at [10]. For our models we only use the 80
constituents with the highest transverse momentum
$p_{T}=\sqrt{p_{x}^{2}+p_{y}^{2}}$, which is typically enough to saturate our
network’s performance. We follow a training regime almost identical to that in
[4], using an Nvidia H100 GPU. The only changes are that we disable weight
decay, extend the training to 140 epochs (4 epochs of linear warm-up, 124
epochs of CosineAnnealingLR with $T_{0}=4$ and $T_{\text{mult}}=2$, and 12
epochs of exponential decay with $\gamma=0.5$), and increase the batch size to
256. Training took about $15\,\text{ms}$ per batch, and the evaluation took
about $1.7\,\text{ms}$ per batch (including overhead).

Table 1: Comparison of tiny top-taggers (best out of 10 random seeds). Architecture | Accuracy | AUC | $1/\epsilon_{B}$ | # Params
---|---|---|---|---
$\text{LorentzNet}_{n_{\text{hidden}}=3}$ | 0.9056 | 0.9623 | 136 | 120
$\text{nPELICAN}_{C_{\text{hidden}}=10}$ | 0.9208 | 0.9747 | 332 | 101
$\text{nPELICAN}_{C_{\text{hidden}}=3}$ | 0.9179 | 0.9722 | 272 | 31
$\text{nPELICAN}_{C_{\text{hidden}}=2}$ | 0.9169 | 0.9716 | 255 | 21
$\text{nPELICAN}_{C_{\text{hidden}}=1}$ | 0.8951 | 0.9480 | 70 | 11

We train several models with $C_{\text{hidden}}$ ranging from 1 to 10. For
comparison, we also train instances of LorentzNet with only one message
passing block and the number of channels in the hidden layers set to 3 and the
batch size of 256 (no other changes to hyperparameters and training were
made). The results are reported in Table 1. We report the accuracy, the area
under the ROC, and the background rejection (inverse false positive rate) at
30% signal efficiency (true positive rate). For each architecture, we pick the
model with the lowest cross-entropy loss out of 10 trained instances
initialized with different random seeds.

We observe that nPELICAN achieves competitive AUC and background rejection
with as few as 2 channels in the hidden layer. In fact, its AUC surpasses that
of the fully-connected TopoDNN with 59k parameters [11] (its average accuracy
was $0.929(1)$, AUC $0.964(14)$, and background rejection $424\pm 82$).
Moreover, the AUC of nPELICAN with 10 channels (101 parameters) is only about
1% behind that of ParticleNet (498k parameters) [16] and even ParT (2.1M
parameters) [17]. Meanwhile, LorentzNet with one message-passing block lags
far behind nPELICAN with a similar number of parameters.

##  5 Interpreting nano PELICAN

Considering the extremely low complexity and relatively high performance of
nPELICAN, there is high potential for a full interpretation of the model.
Before attempting to interpret the weights, it is crucial to minimize any
redundancies. In particular, since ReLU is a homogenous function, one
multiplicative factor from the weights in $\text{LinEq}_{2\to 0}$ can be
absorbed into the weights and biases of $\text{LinEq}_{2\to 2}$ in each
channel of the hidden layer. For the model with $C_{\text{hidden}}=2$ this
means that the number of parameters can effectively be reduced to 19.
Explicitly, the model can be written analytically as

$w=b^{2\to 0}+\sum_{h=1}^{C_{\text{hidden}}}c^{2\to
0}_{0h}\frac{1}{\bar{N}^{2}}\sum_{i,j}\text{ReLU}\left(\sum_{b=1}^{6}c^{2\to
2}_{bh}\text{Agg}_{b}(d)_{ij}+b^{2\to 2}_{h}+b^{2\to
2}_{\text{diag},h}\delta_{ij}\right)+\\\
+\sum_{h=1}^{C_{\text{hidden}}}c^{2\to
0}_{1h}\frac{1}{\bar{N}}\sum_{i=j}\text{ReLU}\left(\sum_{b=1}^{6}c^{2\to
2}_{bh}\text{Agg}_{b}(d)_{ij}+b^{2\to 2}_{h}+b^{2\to
2}_{\text{diag},h}\delta_{ij}\right).$ (1)

Here, $w$ is the output score (the jet is tagged as a top quark if $w>0$);
$c^{2\to 2}$, $b^{2\to 2}$, and $b^{2\to 2}_{\text{diag}}$ are the weights and
biases of $\text{LinEq}_{2\to 2}$; $c^{2\to 0}$ and $b^{2\to 0}$ are the
weights and the bias of $\text{LinEq}_{2\to 0}$; index $b$ enumerates the $6$
aggregators of $\text{LinEq}_{2\to 2}$; index $h$ enumerates the channels in
the hidden layer. $\bar{N}$ is a hyperparameter that is used to control the
magnitude of the sums over constituents, here set to be 49 (it is similarly
used inside the aggregators $\text{Agg}_{b}$).

Therefore the ReLU effectively sets a linear constraint on the dot products,
and the output takes a sum over only those pairs $(i,j)$ that satisfy the
constraint. More explicitly, denoting the jet momentum by $J=\sum_{i}p_{i}$,
the argument of the ReLU is a linear combination of relative masses
($m_{ij}^{2}=-(p_{i}-p_{j})^{2}=2d_{ij}$), jet-frame masses $p_{i}\cdot J$,
$p_{j}\cdot J$, the jet mass $m_{J}^{2}=\sum_{ij}d_{ij}$, and a constant term.
In addition, we found that these parameters are stable across multiple random
initializations, indicating that they can be directly interpreted as unique
physical constraints that encode Lorentz-invariant quantities such as the top
quark mass, which we intend elucidate in future work.

##  6 Conclusions

We have presented nPELICAN, a miniaturized a version of the PELICAN
architecture, which is both surprisingly performant relative to much larger
networks and can be rewritten simply as a constraint on Lorentz invariant
quantities with a single ReLU activation function. This represents a novel
development in interpretability for neural networks in particle physics, and
gives hope for the interpretability of much larger networks. Future studies
will exploit the stability of the nPELICAN parameters to determine their
dependencies on features of the training data such as jet energies and
particle masses, as well as relating these parameters to traditional,
discriminating kinematic variables for jet-tagging, such as jet constituent
multiplicity, subjet multiplicity [20] and jet shapes [7]. The code can be
found at https://github.com/abogatskiy/PELICAN-nano.

## References

  * [1] Georges Aad  “Performance of the ATLAS Level-1 topological trigger in Run 2”  In _Eur. Phys. J. C_ 82.1, 2022, pp. 7  DOI: 10.1140/epjc/s10052-021-09807-0
  * [2] Alexander Bogatskiy, Brandon Anderson, Jan T. Offermann, Marwah Roussi, David W. Miller and Risi Kondor  “Lorentz Group Equivariant Neural Network for Particle Physics”  In _ICML 2020_ ICML, 2020  DOI: 10.48550/arXiv.2006.04780
  * [3] Alexander Bogatskiy, Timothy Hoffman, David W. Miller and Jan T. Offermann  “PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics”, 2022  DOI: 10.48550/arXiv.2211.00454
  * [4] Alexander Bogatskiy, Timothy Hoffman, David W. Miller, Jan T. Offermann and Xiaoyang Liu  “Explainable Equivariant Neural Networks for Particle Physics: PELICAN”, 2023  arXiv:2307.16506 [hep-ph]
  * [5] W Buttinger  “The ATLAS Level-1 Trigger System” On behalf of the ATLAS collaboration, 2012  DOI: 10.1088/1742-6596/396/1/012010
  * [6] Matteo Cacciari, Gavin P. Salam and Gregory Soyez  “The anti-$k_{t}$ jet clustering algorithm”  In _JHEP_ 04, 2008, pp. 063  DOI: 10.1088/1126-6708/2008/04/063
  * [7] Stephen D. Ellis, Zoltan Kunszt and Davison E. Soper  “Jets at hadron colliders at order $\alpha-s^{3:}$ A Look inside”  In _Phys. Rev. Lett._ 69, 1992, pp. 3615–3618  DOI: 10.1103/PhysRevLett.69.3615
  * [8] Shiqi Gong et al.  “An efficient Lorentz equivariant graph neural network for jet tagging”  In _JHEP_ 2022.7, 2022, pp. 30  DOI: 10.1007/JHEP07(2022)030
  * [9] Gregor Kasieczka, Tilman Plehn, Jennifer Thompson and Michael Russel  “Top Quark Tagging Reference Dataset”, 2019  URL: https://zenodo.org/record/2603256
  * [10] Gregor Kasieczka, Tilman Plehn, Jennifer Thompson and Michael Russel  “Converted Top Tagging Dataset”, 2020  URL: https://osf.io/7u3fk/?view_only=8c42f1b112ab4a43bcf208012f9db2df
  * [11] Gregor Kasieczka et al.  “The Machine Learning landscape of top taggers”  In _SciPost Phys._ 7 SciPost, 2019, pp. 014  DOI: 10.21468/SciPostPhys.7.1.014
  * [12] Congqiao Li et al.  “Does Lorentz-symmetric design boost network performance in jet physics?”, 2022  arXiv:2208.07814 [hep-ph]
  * [13] Haggai Maron, Heli Ben-Hamu, Nadav Shamir and Yaron Lipman  “Invariant and Equivariant Graph Networks”, 2018  arXiv: http://arxiv.org/abs/1812.09902
  * [14] Eric A. Moreno et al.  “JEDI-net: a jet identification algorithm based on interaction networks”  In _Eur. Phys. J. C_ 80.1, 2020, pp. 58  DOI: 10.1140/epjc/s10052-020-7608-4
  * [15] Horace Pan and Risi Kondor  “Permutation Equivariant Layers for Higher Order Interactions”  In _AISTATS_ PMLR, 2022, pp. 5987–6001  URL: https://proceedings.mlr.press/v151/pan22a.html
  * [16] Huilin Qu and Loukas Gouskos  “Jet tagging via particle clouds”  In _Phys. Rev. D_ 101.5, 2020  DOI: 10.1103/PhysRevD.101.056019
  * [17] Huilin Qu, Congqiao Li and Sitian Qian  “Particle Transformer for Jet Tagging”, 2022  arXiv:2202.03772 [hep-ph]
  * [18] Zhiqiang Que, Marcus Loo, Hongxiang Fan, Maurizio Pierini, Alexander Tapper and Wayne Luk  “Optimizing Graph Neural Networks for Jet Tagging in Particle Physics on FPGAs”  In _2022 32nd International Conference on Field-Programmable Logic and Applications (FPL)_ , 2022, pp. 327–333  DOI: 10.1109/FPL57034.2022.00057
  * [19] Zhiqiang Que et al.  “LL-GNN: Low Latency Graph Neural Networks on FPGAs for High Energy Physics”, 2023  arXiv:2209.14065 [cs.AR]
  * [20] Jesse Thaler and Ken Van Tilburg  “Identifying Boosted Objects with N-subjettiness”  In _JHEP_ 03, 2011, pp. 015  DOI: 10.1007/JHEP03(2011)015