# Tensor categories

Alexei Davydov

###### Abstract

We present here definitions and constructions basic for the theory of monoidal
and tensor categories. We provide references to the original sources, whenever
possible. Group-theoretical categories are used as examples.

Department of Mathematics, Ohio University, Athens, OH 45701, USA
<EMAIL_ADDRESS>

Keywords/Objectives: monoidal categories and functors, braided and symmetric
monoidal categories, rigid and spherical monoidal categories, dimension,
algebras and modules in monoidal categories, linear and abelian monoidal
categories, tensor and fusion categories.

###### Contents

  1. 1 Introduction
  2. 2 Monoidal categories
  3. 3 Braided and symmetric monoidal categories
  4. 4 Rigid monoidal categories
  5. 5 Module categories
  6. 6 Internal algebra
  7. 7 Linear and abelian monoidal categories
  8. 8 Finite tensor and fusion categories
  9. 9 Example: pointed categories
  10. 10 Conclusions

##  1 Introduction

Monoidal categories were defined in [Bénabou (1963)] and further studied in
[1] and [Kelly (1964)]. The notion turned out to be a fundamental one. From
philosophical point of view it is the result of applying the so-called
categorification (in the sense of e.g. [Crane and Frenkel (1994), Baez and
Dolan (1998)]) to the notion of a monoid, i.e. the process of replacing sets
with categories, functions with functors, and equations between functions by
natural isomorphisms between functors, which in turn should satisfy certain
equations of their own, called “coherence laws”. The pseudo-monoid nature of
monoidal categories explains their appearance in low dimensional topology,
through (extended) topological field theories [Turaev (1994)]. Other (somewhat
related) applications of monoidal categories include mathematical logic and
theoretical computer science (see [Baez and Stay (2010)]).

A special kind of monoidal categories is formed by linear (hom-sets are vector
spaces, composition and tensor product of morphisms are bilinear maps) and
abelian categories. The natural source of such categories is the
representation theory of groups and other related structures (Lie algebras,
Hopf algebras, etc.). An abelian monoidal (or tensor) category can thus be
seen as the category of all possible linear realisations of a certain
symmetry. This point of view is beneficial in such areas as the theory of
motives or Langlands correspondence and was driving the development of
(symmetric) tensor categories [Deligne and Milne (1982), Deligne (2002)].

The concept of fusion (finite semi-simple tensor) categories originated in
high energy physics [Moore and Seiberg (1990)]. Braided fusion categories
appear as representation categories of chiral algebras in 2d conformal field
theory or as values on the circle of corresponding 3d topological field
theories. More general fusion categories are formed by boundary data of such
theories. Currently the largest consumer of fusion categories is condensed
matter physics (topological states of matter) [Kitaev (2006)].

For definitions and basic facts of category theory, such as adjoint functors,
limits, ends see [Mac Lane (1971)]. We follow Mac Lane’s notation denoting the
set of morphisms between objects $X$ and $Y$ of a category ${\cal A}$ by
${\cal A}(X,Y)$. For a functor $F:{\cal C}\to{\cal D}$ we denote by ${\cal
I}m_{f}(F)$ the full subcategory of ${\cal D}$ with objects of the form $F(X)$
for $X\in{\cal C}$ (the full image of $F$) and by ${\cal I}m(F)$ is the
subcategory of ${\cal D}$ with objects of the form $F(X)$ for $X\in{\cal C}$
and morphisms of the form $F(f)$ for $f$ a morphism in ${\cal C}$ (the proper
image of $F$). Thus $F$ factors as ${\cal C}\to{\cal I}m(F)\to{\cal
I}m_{f}(F)\to{\cal D}$, where the first functor is full, the second is
faithful, and the last is fully faithful (an embedding).

##  2 Monoidal categories

A category ${\cal C}$ is semi-groupal if it comes equipped with a functor (the
tensor product)

${\cal C}\times{\cal C}\to{\cal C}\qquad(X,Y)\mapsto X{\otimes}Y$

together with a collection of isomorphisms (the associativity constraint)
$a_{X,Y,Z}:X{\otimes}(Y{\otimes}Z)\to(X{\otimes}Y){\otimes}Z$ natural in
$X,Y,Z\in{\cal C}$, such that the diagram

$\textstyle{(X{\otimes}Y){\otimes}(Z{\otimes}W)}$$\textstyle{X{\otimes}(Y{\otimes}(Z{\otimes}W))}$$\textstyle{((X{\otimes}Y){\otimes}Z){\otimes}W}$$\textstyle{X{\otimes}((Y{\otimes}Z){\otimes}W)}$$\textstyle{(X{\otimes}(Y{\otimes}Z)){\otimes}W}$$\scriptstyle{a_{X,Y,Z{\otimes}W}}$$\scriptstyle{a_{X{\otimes}Y,Z,W}}$$\scriptstyle{1{\otimes}a_{Y,Z,W}}$$\scriptstyle{a_{X,Y{\otimes}Z,W}}$$\scriptstyle{a_{X,Y,Z}{\otimes}1}$
(1)

commutes for any $X,Y,Z,W\in{\cal C}$.
A semi-groupal category is strict if $a_{X,Y,Z}=1$ for all $X,Y,Z\in{\cal C}$.

######  Example 2.1.

For a category ${\cal M}$ denote by ${\cal E}nd({\cal M})$ the category of
endofunctors ${\cal M}\to{\cal M}$. This is a strict semi-groupal category
with composition of functors as the tensor product.

A functor $F:{\cal C}\to{\cal D}$ between semi-groupal categories is semi-
groupal if it comes equipped with a collection of isomorphisms (the semi-
groupal constraint) $F_{X,Y}:F(X){\otimes}F(Y)\to F(X{\otimes}Y)$ natural in
$X,Y\in{\cal C}$, such that the diagram

$\textstyle{F(X){\otimes}(F(Y){\otimes}F(Z))}$$\textstyle{F(X){\otimes}F(Y{\otimes}Z)}$$\textstyle{F(X{\otimes}(Y{\otimes}Z))}$$\textstyle{(F(X){\otimes}F(Y)){\otimes}F(Z)}$$\textstyle{F(X{\otimes}Y){\otimes}F(Z)}$$\textstyle{F((X{\otimes}Y){\otimes}Z)}$$\scriptstyle{a_{F(X),F(Y),F(Z)}}$$\scriptstyle{1{\otimes}F_{Y,Z}}$$\scriptstyle{F_{X,Y{\otimes}Z}}$$\scriptstyle{F_{X,Y}{\otimes}1}$$\scriptstyle{F_{X{\otimes}Y,Z}}$$\scriptstyle{F(a_{X,Y,Z})}$
(2)

commutes for any $X,Y,Z\in{\cal C}$.
A natural transformation $c:F\to G$ between semi-groupal functors $F,G:{\cal
C}\to{\cal D}$ is semi-groupal if the diagram

$\textstyle{F(X){\otimes}F(Y)}$$\textstyle{F(X{\otimes}Y)}$$\textstyle{G(X){\otimes}G(Y)}$$\textstyle{G(X{\otimes}Y)}$$\scriptstyle{c_{X}{\otimes}c_{Y}}$$\scriptstyle{F_{X,Y}}$$\scriptstyle{G_{X,Y}}$$\scriptstyle{c_{X{\otimes}Y}}$
(3)

commutes for any $X,Y\in{\cal C}$.
Clearly composites of semi-groupal functors and natural transformations are
semi-groupal.
A semi-groupal category ${\cal C}$ is monoidal if it has an object $I\in{\cal
C}$ (the monoidal unit) together with collections of isomorphisms
$l_{X}:I{\otimes}X\to X$, $r_{X}:X{\otimes}I\to X$ natural in $X\in{\cal C}$
such that the diagram

$\textstyle{X{\otimes}(I{\otimes}Y)}$$\textstyle{(X{\otimes}I){\otimes}Y}$$\textstyle{X{\otimes}Y}$$\scriptstyle{1{\otimes}l_{Y}}$$\scriptstyle{a_{X,I,Y}}$$\scriptstyle{r_{X}{\otimes}1}$
(4)

commutes for any $X,Y\in{\cal C}$.

######  Remark 2.2.

The monoidal unit object is unique up to an isomorphism, i.e. being monoidal
is a property of a semi-groupal category.
In a strict monoidal category one also has $l=r=1$.

A semi-groupal functor $F:{\cal C}\to{\cal D}$ is monoidal if it comes
equipped with an isomorphism $\iota:I\to F(I)$ such that the diagrams

$\textstyle{F(I){\otimes}F(X)}$$\textstyle{F(I{\otimes}X)}$$\textstyle{I{\otimes}F(X)}$$\textstyle{F(X)}$$\scriptstyle{\iota{\otimes}1}$$\scriptstyle{F_{I,X}}$$\scriptstyle{F(l_{X})}$$\scriptstyle{l_{F(X)}}$
$\textstyle{F(X){\otimes}F(I)}$$\textstyle{F(X{\otimes}I)}$$\textstyle{F(X){\otimes}I}$$\textstyle{F(X)}$$\scriptstyle{1{\otimes}\iota}$$\scriptstyle{F_{X,I}}$$\scriptstyle{F(r_{X})}$$\scriptstyle{r_{F(X)}}$
(5)

commute for any $X\in{\cal C}$.

The scalars of a monoidal ${\cal C}$ form the submonoid ${\cal
C}_{s}(I,I)=\\{c\in{\cal C}(I,I)|\
l_{X}(c{\otimes}1_{X})=r_{X}(1_{X}{\otimes}c)\ \forall X\in{\cal C}\\}$ of the
monoid of endomorphisms ${\cal C}(I,I)$. By the Eckman-Hilton argument the
monoid ${\cal C}(I,I)$ is commutative.

For a monoidal ${\cal C}$ the opposite category ${\cal C}^{op}$ (the category
with morphism sets ${\cal C}^{op}(X,Y)={\cal C}(Y,X)$) is monoidal with
$a^{op}_{X,Y,Z}=a_{X,Y,Z}^{-1},\ l^{op}_{X}=l_{X}^{-1},\
r^{op}_{X}=r_{X}^{-1}$. The reverse ${\cal C}^{rev}$ of a monoidal ${\cal C}$
is the category ${\cal C}$ with a new tensor product
$X{\otimes}^{rev}Y=Y{\otimes}X$ and $a^{rev}_{X,Y,Z}=a_{Z,Y,X}^{-1},\
l^{rev}_{X}=r_{X},\ r^{rev}_{X}=l_{X}$. We also write: ${\cal
C}^{oprev}=(C^{rev})^{op}$.

##  3 Braided and symmetric monoidal categories

A monoidal category ${\cal C}$ is braided if it comes equipped with a
collection of isomorphisms $c_{X,Y}:X{\otimes}Y\to Y{\otimes}X$ (braidings)
natural in $X,Y\in{\cal C}$ such that the diagrams

$\textstyle{X{\otimes}(Y{\otimes}Z)}$$\textstyle{(X{\otimes}Y){\otimes}Z}$$\scriptstyle{a_{X,Y,Z}}$$\textstyle{Z{\otimes}(X{\otimes}Y)}$$\scriptstyle{c_{X{\otimes}Y,Z}}$$\textstyle{(Z{\otimes}X){\otimes}Y}$$\scriptstyle{a_{Z,X,Y}}$$\textstyle{X{\otimes}(Z{\otimes}Y)}$$\scriptstyle{1c_{Y,Z}}$$\textstyle{(X{\otimes}Z){\otimes}Y}$$\scriptstyle{a_{X,Z,Y}^{-1}}$$\scriptstyle{c_{X,Z}1}$
$\textstyle{(X{\otimes}Y){\otimes}Z}$$\textstyle{X{\otimes}(Y{\otimes}Z)}$$\scriptstyle{a_{X,Y,Z}^{-1}}$$\textstyle{(Y{\otimes}Z){\otimes}X}$$\scriptstyle{c_{X,Y{\otimes}Z}}$$\textstyle{Y{\otimes}(Z{\otimes}X)}$$\scriptstyle{a_{Y,Z,X}^{-1}}$$\textstyle{(Y{\otimes}X){\otimes}Z}$$\scriptstyle{c_{X,Y}1}$$\textstyle{Y{\otimes}(X{\otimes}Z)}$$\scriptstyle{a_{Y,X,Z}^{-1}}$$\scriptstyle{1c_{X,Z}}$
(6)

commute for all $X,Y,Z\in{\cal C}$. It is also assumed that
$c_{I,Y}=l_{Y}^{-1}r_{Y}$ and $c_{X,I}=r_{X}^{-1}l_{X}$.
A monoidal functor $F:{\cal C}\to{\cal D}$ between braided monoidal categories
is braided if the diagram

$\textstyle{F(X){\otimes}F(Y)\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{F_{X,Y}}$$\scriptstyle{c_{F(X),F(Y)}}$$\textstyle{F(X{\otimes}Y)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{F(c_{X,Y})}$$\textstyle{G(X){\otimes}G(Y)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{G_{X,Y}}$$\textstyle{G(X{\otimes}Y)}$
(7)

commutes for all $X,Y\in{\cal C}$ [Joyal and Street (1993)].
The conjugate $\overline{\cal C}$ of a braided monoidal category ${\cal C}$ is
the monoidal category ${\cal C}$ with the braiding
$\overline{c}_{X,Y}=c_{Y,X}^{-1}$.

######  Example 3.1.

For a monoidal category ${\cal C}$ define ${\cal Z}({\cal C})$ as the category
of pairs $(Z,z)$, where $Z\in{\cal C}$ together with a collection of
isomorphisms $z_{X}:X{\otimes}Z\to Z{\otimes}X$ natural in $X\in{\cal C}$
(half-braiding) and such that the diagram obtained by replacing $c$ with $z$
in the first diagram of (6) commutes. The category ${\cal Z}({\cal C})$ is
monoidal with the tensor product $(Z,z){\otimes}(W,w)=(Z{\otimes}W,z|w)$,
where $(z|w)_{X}:X{\otimes}(Z{\otimes}W)\to(Z{\otimes}W){\otimes}X$ is defined
by a diagram similar to the second diagram of (6). The monoidal category
${\cal Z}({\cal C})$ is braided (with the braiding $c_{(Z,z),(W,w)}=z_{W}$)
and is known as the monoidal (or Drinfeld) centre of ${\cal C}$ [Joyal and
Street (1991b), Majid (1991)].

A braided monoidal category ${\cal C}$ is symmetric if $c_{Y,X}c_{X,Y}=1$ for
all $X,Y\in{\cal C}$. The full subcategory ${\cal Z}_{sym}({\cal
C})=\\{X\in{\cal C}|\ c_{Y,X}c_{X,Y}=1\ \forall Y\in{\cal C}\\}$ is symmetric
(the symmetric or Mueger’s centre of ${\cal C}$).

A braiding of ${\cal C}$ can be thought of as the structure of a monoidal
functor ${\cal C}\to{\cal C}^{rev}$ on the identity functor $Id_{\cal C}$. For
a symmetric ${\cal C}$ the composite ${\cal C}\to{\cal C}^{rev}\to{\cal C}$
coincides with $Id_{\cal C}$ as a monoidal functor.
More generally, a balancing of a braided monoidal ${\cal C}$ is a monoidal
isomorphism between the composite ${\cal C}\to{\cal C}^{rev}\to{\cal C}$ and
$Id_{\cal C}$ [Joyal and Street (1993)]. The group $Aut_{\otimes}(Id_{\cal
C})$ of monoidal automorphisms of the identity functor acts freely on the set
of possible balancings of a balanced category.
Not all structures of a monoidal functor ${\cal C}\to{\cal C}^{rev}$ on the
identity functor $Id_{\cal C}$ come from braidings of ${\cal C}$. A monoidal
${\cal C}$ together with a structure of monoidal functor ${\cal C}\to{\cal
C}^{rev}$ on the identity functor $Id_{\cal C}$ such that the composite ${\cal
C}\to{\cal C}^{rev}\to{\cal C}$ coincides with $Id_{\cal C}$ as a monoidal
functor was called coboundary in [Drinfeld (1989)].

##  4 Rigid monoidal categories

The (left) internal hom $[X,Y]$ of objects $X,Y$ of a semi-groupal category
${\cal C}$ is the terminal object $([X,Y],ev)$ in the category of pairs
$(T,\tau)$ where $T$ is an object and $\tau:T{\otimes}X\to Y$ is a morphism in
${\cal C}$ correspondingly. Equivalently, $[X,-]$ is the right adjoint to the
functor $-{\otimes}X$, i.e. there is a collection of isomorphisms ${\cal
C}(T,[X,Y])\simeq{\cal C}(T{\otimes}X,Y)$ natural in $X,Y,T\in{\cal C}$. The
universal property of the internal hom gives rise to morphisms

$[Y,Z]{\otimes}[X,Y]\longrightarrow[X,Z],\qquad
X{\otimes}[Y,Z]\stackrel{{\scriptstyle}}{{\longrightarrow}}[Y,X{\otimes}Z],\qquad[X,[Y,Z]]\stackrel{{\scriptstyle\sim}}{{\longrightarrow}}[X{\otimes}Y,Z]$

natural in $X,Y,Z\in{\cal C}$. A semi-groupal category ${\cal C}$ is (left)
closed if $[X,Y]$ exists for any $X,Y\in{\cal C}$ [Eilenberg and Kelly
(1966)].

######  Remark 4.1.

The existence of $[X,-]$, as a right adjoint, can e.g. be deduced from the
adjoint functor theorems (see [1]). If $[X,-]$ exists the functor
$-{\otimes}X$ is a left adjoint and hence preserves colimits. Thus the tensor
product of a closed category preserves colimits (in both arguments).

In a monoidal category ${\cal C}$ one has $[I,X]\simeq X$ for any $X\in{\cal
C}$. A monoidal category ${\cal C}$ is (left) rigid of (left) autonomous if it
is (left) closed and $[I,Y]{\otimes}[X,I]\longrightarrow[X,Y]$ is an
isomorphism for any $X,Y\in{\cal C}$. In this case $X^{*}=[X,I]$ is called the
(left) dual of $X$. The (left) dual $X^{*}$ of $X$ is characterised by the
existence of morphisms $ev:X^{*}{\otimes}X\to I,coev:I\to X{\otimes}X^{*}$
such that $(coev{\otimes}1)(1{\otimes}ev)=1_{X}$ and
$(1{\otimes}coev)(ev{\otimes}1)=1_{X^{*}}$ (here we suppress associators).
Such morphisms are unique up to an automorphism of $I$ (see [Saavedra Rivano
(1972)]).

######  Example 4.2.

The (left) dual to an endofunctor $F$ as an object of the monoidal category
${\cal E}nd({\cal M})$ is its left adjoint. Thus a left rigid $F\in{\cal
E}nd({\cal M})$ preserves limits and a right rigid endofunctor preserves
colimits in ${\cal M}$.

######  Remark 4.3.

A monoidal functor $F:{\cal C}\to{\cal D}$ preserves duals
$F(X^{*})=F(X)^{*}$. Thus for a rigid ${\cal C}$ the proper and full images
${\cal I}m(F)$ and ${\cal I}m_{f}(F)$ of a monoidal functor are rigid.

######  Remark 4.4.

The monoidal functor ${\cal C}\to{\cal E}nd({\cal C})$ sending $X$ to the
endofunctor $Y\mapsto X{\otimes}Y$ sends rigid object to functors preserving
limits and colimits. Thus the tensor product of a rigid category preserves
limits and colimits (in both arguments).

For rigid ${\cal C}$ the assignment $X\mapsto X^{*}$ extends to morphisms and
defines a monoidal equivalence ${\cal C}\to{\cal C}^{oprev}$. A pivotal
structure on a rigid monoidal ${\cal C}$ is a monoidal isomorphism $\tau$
between $Id_{\cal C}$ and the composite $(-)^{**}:{\cal C}\to{\cal
C}^{oprev}\to{\cal C}$ [Freyd and Yetter(1989)]. The group
$Aut_{\otimes}(Id_{\cal C})$ acts freely on the set of pivotal structures of a
pivotal category.
For an endomorphism $f:X\to X$ of a pivotal category ${\cal C}$ the composite
$tr(f)=ev_{X^{*}}(\tau_{X}f{\otimes}1)coev_{X}$ is an endomorphism of $I$
called the trace of $f$ (here we suppress associators). A pivotal category
${\cal C}$ is spherical if $tr(f^{*})=tr(f)$ for any endomorphism $f$ in
${\cal C}$ [Barrett and Westbury (1999)].

##  5 Module categories

A category ${\cal M}$ is a (left) module category over a monoidal category
${\cal C}$ if it comes equipped with a monoidal functor ${\cal C}\to{\cal
E}nd({\cal M})$. Equivalently, a ${\cal C}$-module structure of ${\cal M}$ is
encoded in a functor ${\cal C}\times{\cal M}\to{\cal M},\ (X,M)\mapsto X*M$.
The monoidal constraint of ${\cal C}\to{\cal E}nd({\cal M})$ corresponds to a
collection of isomorphisms (the action associativity constraint)
$a_{X,Y,M}:X*(Y*M)\to(X{\otimes}Y)*M$ natural in $X,Y\in{\cal C}$, $M\in{\cal
M}$, such that the diagrams similar to (1),(4) commute. A functor $F:{\cal
M}\to{\cal N}$ between ${\cal C}$-module categories is ${\cal C}$-module
functor if it comes equipped with a collection of isomorphisms (the module
constraint) $F_{X,M}:F(X)*F(M)\to F(X*M)$ natural in $X\in{\cal C}$,
$M\in{\cal M}$, such that the diagrams similar to (2),(5) commute. A natural
transformation $c:F\to G$ between ${\cal C}$-module functors $F,G:{\cal
M}\to{\cal N}$ is ${\cal C}$-module if the diagram similar to (3) commutes.
Clearly composites of ${\cal C}$-module functors and natural transformations
are ${\cal C}$-module. In particular, ${\cal C}$-module endofunctors of a
${\cal C}$-module category ${\cal M}$ together with ${\cal C}$-module natural
transformations form a strict monoidal category ${\cal E}nd_{{\cal
C}{\text{-}}}({\cal M})$.

######  Example 5.1.

The tensor product can be seen as an action of a monoidal category ${\cal C}$
on itself (the regular left ${\cal C}$-module category). The assignment
$F\mapsto F(I)$ is a monoidal equivalence ${\cal E}nd_{{\cal
C}{\text{-}}}({\cal C})\to{\cal C}$. Thus any monoidal category is equivalent
to a strict one (Mac Lane’s coherence theorem).

Similarly one defines right ${\cal C}$-module and ${\cal C}$-bimodule
categories. For a ${\cal C}$-bimodule category ${\cal M}$ denote by ${\cal
E}nd_{{\cal C}{\text{-}}{\cal C}}({\cal M})$ the strict monoidal category of
${\cal C}$-bimodule endofunctors.

######  Example 5.2.

The tensor product turns a monoidal category ${\cal C}$ into a bimodule over
itself. The assignment $F\mapsto F(I)$ extends to a monoidal equivalence
${\cal E}nd_{{\cal C}{\text{-}}{\cal C}}({\cal C})\to{\cal Z}({\cal C})$ with
the ${\cal C}$-bimodule structure of $F$ corresponding to the half braiding on
$F(I)$.

The notion of internal hom extends to module categories. The internal action
hom $[M,N]_{\cal M}$ of objects $M,N$ of a (left) ${\cal C}$-module category
${\cal M}$ is the terminal object $([M,N]_{\cal M},ev)$ in the category of
pairs $(T,\tau)$ where $T\in{\cal C}$ and $\tau:T*M\to N$ is a morphism in
${\cal M}$. Equivalently, $[M,-]_{\cal M}$ is the right adjoint to the functor
$-*M$, i.e. there is a collection of isomorphisms ${\cal C}(T,[M,N]_{\cal
M})\simeq{\cal M}(T*M,N)$ natural in $M,N\in{\cal M},T\in{\cal C}$ [Ostrik
(2003)]. The universal property of the internal hom gives rise to morphisms

$[M,N]_{\cal M}{\otimes}[L,M]_{\cal M}\longrightarrow[L,N]_{\cal M},\qquad
X{\otimes}[M,N]_{\cal
M}\stackrel{{\scriptstyle}}{{\longrightarrow}}[M,X*N]_{\cal
M},\qquad[X,[M,N]_{\cal
M}]\stackrel{{\scriptstyle\sim}}{{\longrightarrow}}[X*M,N]_{\cal M}$

natural in $L,M,N\in{\cal M},X\in{\cal C}$. For rigid ${\cal C}$ the second
morphism is an isomorphism.

##  6 Internal algebra

The language of algebras and their modules internal to a monoidal category is
a useful tool for defining and computing with module categories and for
constructing braided monoidal categories.

An associative unital algebra in a monoidal category ${\cal C}$ is a triple
$(A,\mu,\iota)$ consisting of an object $A\in{\cal C}$ together with a
multiplication $\mu:A\otimes A\to A$ and a unit map $\iota:I\to A$, satisfying
the associativity $\mu\left(\mu\otimes
1_{A}\right)=\mu\left(1_{A}\otimes\mu\right),$ and the unit
$\mu\left(\iota\otimes 1_{A}\right)=1_{A}=\mu\left(1_{A}\otimes\iota\right)$
axioms.
A right module over an algebra $A$ is a pair $(M,\nu)$, where $M$ is an object
of ${\cal C}$ and $\nu:M\otimes A\to M$ is a morphism (action map), such that
$\nu\left(\nu\otimes 1_{A}\right)=\nu\left(1_{M}\otimes\mu\right).$ A
homomorphism of right $A$-modules $M\to N$ is a morphism $f:M\to N$ in ${\cal
C}$ such that $\nu_{N}\left(f\otimes 1_{A}\right)=f\nu_{M}.$
Right modules over an algebra $A\in{\cal C}$ together with module
homomorphisms form a category ${\cal C}_{A}$. The forgetful functor ${\cal
C}_{A}\to{\cal C}$ has a left adjoint, which sends an object $X\in{\cal C}$
into the free $A$-module $X\otimes A$, with $1_{X}{\otimes}\mu$ as the
$A$-module structure. More generally, for $X\in{\cal C}$ and $M\in{\cal
C}_{A}$ the tensor product $X{\otimes}M$ is a right $A$-module under
$1_{X}{\otimes}\nu_{M}$. This categorical pairing is a left $C$-module action
on ${\cal C}_{A}$. The free $A$-module functor $-{\otimes}A:{\cal C}\to{\cal
C}_{A}$ and the forgetful functor ${\cal C}_{A}\to{\cal C}$ are $C$-module
functors.
Conversely, let $M$ be an object of a ${\cal C}$-module category ${\cal M}$
such that internal action homs $[M,-]_{\cal M}$ exist, Then $[M,M]_{\cal M}$
is an algebra and $[M,N]_{\cal M}$ is a right $[M,M]_{\cal M}$-module in
${\cal C}$. For rigid ${\cal C}$ the functor $[M,-]_{\cal M}:{\cal M}\to{\cal
C}_{[M,M]_{\cal M}}$ is ${\cal C}$-module and the free $[M,M]_{\cal M}$-module
functor factorises

---
$\textstyle{{\cal
C}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{-*M}$$\textstyle{{\cal
C}_{[M,M]_{\cal M}}}$$\textstyle{{\cal
M}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{[M,-]_{\cal
M}}$

An algebra $A$ in a braided monoidal category is commutative if $\mu
c_{A,A}=\mu$. For $M,N\in{\cal C}_{A}$ denote by $M{\otimes}_{A}N$ the
coequaliser of $\nu_{M}1,(1\nu_{N})(1c_{A,N}):M{\otimes}A{\otimes}N\to
M{\otimes}N$. When these coequalisers exist for all $M,N\in{\cal C}_{A}$ the
category ${\cal C}_{A}$ becomes monoidal with the tensor product
${\otimes}_{A}$ [Pareigis (1995)]. The free-module functor ${\cal C}\to{\cal
C}_{A}$ is monoidal and central, i.e. lifts to a braided monoidal functor
${\cal C}\to{\cal Z}({\cal C}_{A})$.
Conversely for a central functor $F:{\cal C}\to{\cal D}$ with the right
adjoint $R:{\cal D}\to{\cal C}$ the object $R(I)\in{\cal C}$ has a structure
of commutative algebra. The functor $F$ induces a monoidal functor
$\tilde{F}:{\cal C}_{R(I)}\to{\cal D}$ and $F=\tilde{F}\circ(-{\otimes}R(I))$
[Davydov et al (2013)]. Note that the algebra $R(I)$ coincides with the
internal action hom $[I,I]$ for the ${\cal C}$-action on ${\cal D}$ defined by
the functor $F$.

######  Example 6.1.

For a ${\cal C}$-module category ${\cal M}$ there is defined a central
monoidal functor ${\cal Z}(C)\to{\cal E}nd_{\cal C}({\cal M})$ (called
$\alpha$-induction). The corresponding internal action hom $[Id_{\cal
M},Id_{\cal M}]$ is a commutative algebra $Z({\cal M})$ in ${\cal Z}(C)$
called the full centre of ${\cal M}$. For ${\cal M}={\cal C}_{A}$ the full
centre $Z(A)=Z({\cal C}_{A})$ is the terminal object of the category of the
pairs $(Z,\zeta)$, where $Z\in{\cal Z}({\cal C})$ and $\zeta:Z\to A$ is a
morphism in ${\cal C}$ such that
$\mu(\zeta{\otimes}1_{A})=\mu(1_{A}{\otimes}\zeta)z_{A}$ (where $z_{A}$ is the
half-braiding of $Z$) [Fröhlich et al (2006), Davydov (2010)].

More generally for a commutative algebra $A\in{\cal Z}({\cal C})$ the
coequaliser $M{\otimes}_{A}N$ (with $c_{A,N}$ replaced by the half braiding of
$A$) is a tensor product on ${\cal C}_{A}$ [Schauenburg (2001)].

For closed ${\cal C}$ the category of $A$-modules ${\cal C}_{A}$ is also
closed with the internal hom $[M,N]_{A}$ being by the equaliser of
$[1,\widetilde{\nu}_{N}],[\nu_{M},1]:[M,N]\to[M,[A,N]]$, where
$\widetilde{\nu}_{N}:N\to[A,N]$ is induced by the module structure
$\nu_{N}:N{\otimes}A\to N$ [Pareigis (1995)]. For a rigid ${\cal C}$ the
category ${\cal C}_{A}$ is also rigid if the algebra $A$ is separable, i.e. an
algebra $A$ such that the canonical (trace) pairing
$ev_{A}z_{A^{*}}(\mu{\otimes}1)(1{\otimes}coev_{A})\mu:A{\otimes}A\to A$ is
non-degenerate [Fröhlich et al (2006)].

For a commutative algebra $A$ in a braided monoidal category ${\cal C}$ the
monoidal category ${\cal C}_{A}$ is not braided in general. However it
contains a distinguished full braided subcategory ${\cal C}_{A}^{loc}$ formed
by so-called local modules. An $A$-module $(M,\nu)$ is local if $\nu
c_{A,M}c_{M,A}=\nu$ [Pareigis (1995), Fröhlich et al (2006)]. Note that the
full embedding ${\cal C}_{A}^{loc}\to{\cal C}_{A}$ is a central monoidal
functor.

##  7 Linear and abelian monoidal categories

It is convenient when dealing with additive and linear categories to use the
language of categories enriched in a symmetric monoidal category ${\cal V}$,
or ${\cal V}$-category. In a ${\cal V}$-category ${\cal A}$ homs ${\cal
A}(X,Y)$ are not sets but rather objects of ${\cal V}$ and composition is a
morphism ${\cal A}(Y,Z){\otimes}_{\cal V}{\cal A}(X,Y)\to{\cal A}(X,Z)$ (see
[Kelly (1982)] for all details). For ${\cal V}$ being the category of sets
${\cal S}et$, ${\cal V}$-category is just a category. When ${\cal V}$ is the
category of abelian groups, ${\cal V}$-category is an additive category
(${\cal A}(X,Y)$ is an abelian group and composition is bilinear). When ${\cal
V}$ is the category ${\cal V}{\it ect}$ of vector spaces over a fixed field
$k$, ${\cal V}$-category is a linear category (${\cal A}(X,Y)$ is a vector
space and composition is bilinear).

A ${\cal V}$-category ${\cal C}$ is monoidal if it comes equipped with a
${\cal V}$-functor ${\cal C}\boxtimes_{\cal V}{\cal C}\to{\cal C}$ together
with an associativity constraint and the unit object. Here ${\cal
A}\boxtimes_{\cal V}{\cal B}$ is the ${\cal V}$-category with objects
$Ob({\cal A}\boxtimes_{\cal V}{\cal B})=Ob({\cal A})\times Ob({\cal B})$ and
homs $({\cal A}\boxtimes_{\cal V}{\cal B})((X,Z),(Y,W))={\cal
A}(X,Y){\otimes}_{\cal V}{\cal B}(Z,W)$. For example in a monoidal linear
category the tensor product on morphisms is given by bilinear maps ${\cal
C}(X,Y){\otimes}_{k}{\cal C}(Z,W)\to{\cal C}(X{\otimes}Z,Y{\otimes}W)$. The
addition operation on morphisms can be quite useful. For an additive functor
$F:{\cal C}\to{\cal D}$ the proper image ${\cal I}m(F)$ can be described as
the quotient ${\cal C}/Ker(F)$ by the ideal $Ker(F)$ of all morphisms mapped
by $F$ to zero. When the functor $F$ is monoidal the ideal $Ker(F)$ is also
monoidal, i.e. $Ker(F)$ is closed not just under compositions but also under
tensor products with arbitrary morphisms.

######  Example 7.1.

In a spherical linear category negligible morphisms form a monoidal ideal $N$.
A morphism $f:X\to Y$ is negligible if $tr(fg)=0$ for any $g:Y\to X$ [Turaev
(1994), Barrett and Westbury (1999)].

A pre-abelian category is an additive category with finite limits and colimits
(finitely bicomplete). A pre-abelian category is abelian if any monomorphism
is a kernel and any epimorphism is a cokernel.

For (co)complete ${\cal V}$ (with (co)continuous tensor product) the category
$\widehat{\cal A}={\cal F}unct_{\cal V}({\cal A}^{op},{\cal V})$ of ${\cal
V}$-functors (the ${\cal V}$-presheaf category) is also (co)complete. The
Yoneda embedding $Y:{\cal A}\to\widehat{\cal A}$ sending $X$ to the functor
${\cal A}(X,-)$ is fully faithful.
For a monoidal ${\cal V}$-category ${\cal C}$ the presheaf category
$\widehat{\cal C}$ is monoidal with respect to the Day convolution defined as
the coend $(U{\otimes}V)(X)=\int^{Y,Z\in{\cal C}}{\cal
C}(X,Y{\otimes}Z){\otimes}_{\cal V}U(Y){\otimes}_{\cal V}V(Z)$ [Day (1970)].
If ${\cal V}$ is closed the monoidal presheaf category $\widehat{\cal C}$ is
also closed with the internal hom given by the end $[U,V](X)=\int_{Y\in{\cal
C}}[U(Y),V(X{\otimes}Y)]_{\cal V}$.

######  Example 7.2.

For a linear category ${\cal A}$ the presheaf category $\widehat{\cal A}$ is
abelian.

An abelian linear monoidal category ${\cal C}$ is tensor if it is rigid and
the identity object $I$ is simple and ${\cal C}(I,I)=k$. Here an object is
simple if all morphisms out of it are monomorphisms and morphism in it are
epimorphisms.

##  8 Finite tensor and fusion categories

A tensor category is finite if all hom-spaces are finite dimensional and any
object has a finite length (i.e. has a finite filtration with simple factors).
As an abelian category a finite tensor category is equivalent to the category
of finite dimensional modules over a finite dimensional algebra. As the result
a finite tensor category is finitely complete and cocomplete, and a tensor
functor between finite tensor categories has left and right adjoints. In
particular, internal action homs for a finite module category exist.

The categorical dimension $dim({\cal C})$ of a finite tensor category is the
dimension $d(Z(I))$ of the full centre of the trivial algebra $I\in{\cal C}$.
The categorical dimension has the following properties: $dim(Z({\cal
C}))=dim({\cal C})^{2}$, $dim({\cal C}_{A})d(A)=dim({\cal C})$ and $dim({\cal
C}_{A}^{loc})d(A)^{2}=dim({\cal C})$ for a separable commutative algebra
$A\in{\cal Z}({\cal C})$ such that ${\cal C}(I,A)=k$ [Fröhlich et al (2006),
Etingof et al (2015)].

A finite tensor category ${\cal C}$ is incompressible if any tensor functor
${\cal C}\to{\cal D}$ into a tensor category is fully faithful (see
[Coulembier et al (2023)] and references therein). For an incompressible
finite tensor category ${\cal C}$ there should be no non-trivial separable
commutative algebras $A$ in ${\cal Z}({\cal C})$ with ${\cal C}(I,A)=k$.

######  Example 8.1.

An incompressible symmetric tensor category in zero characteristic is either
the category of vector spaces ${\cal V}{\it ect}$ or its super-analog $s{\cal
V}{\it ect}$ [Deligne (2002)]. A symmetric tensor category ${\cal C}$ is
Tannakian if there is a faithful symmetric tensor functor ${\cal C}\to{\cal
V}ect$ and super-Tannakian if there is a faithful symmetric tensor functor
${\cal C}\to s{\cal V}ect$.
The structure of incompressible symmetric tensor categories in finite
characteristic is much richer (see [Coulembier et al (2023)] and references
therein).

For an incompressible braided finite tensor category ${\cal C}$ there should
be no non-trivial separable commutative algebras $A$ in ${\cal C}$ with ${\cal
C}(I,A)=k$ (complete anisotropy of ${\cal C}$, see [Davydov et al (2013)]).

######  Example 8.2.

A braided tensor category ${\cal C}$ is non-degenerate if ${\cal
Z}_{sym}({\cal C})={\cal V}{\it ect}$ and is slightly-degenerate if ${\cal
Z}_{sym}({\cal C})=s{\cal V}{\it ect}$.
Completely anisotropic braided finite tensor categories are either non-
degenerate or slightly-degenerate.

A finite tensor category is fusion if it is semi-simple, i.e. any object is a
direct sum of simple objects.

######  Example 8.3.

Equivalence classes of completely anisotropic non-degenerate braided fusion
categories form a group, the so-called Witt group [Davydov et al (2013)]. The
same is true for slightly-degenerate braided fusion categories.

##  9 Example: pointed categories

Let $G$ be a semi-group. Denote by ${\cal V}(G)$ the category of vector spaces
(over a field $k$) graded by $G$ with grading preserving linear maps as
morphisms. Note that it is a semi-simple category (any object is a direct
product of simple objects) with simple objects being one dimensional vector
space $I(g)=k$ concentrated in the a single degree $g\in G$. It is also a
monoidal category with the tensor product of vector spaces graded by
$(U{\otimes}_{k}V)_{f}={\oplus}_{gh=f}U_{g}{\otimes}_{k}V_{h}$. The unit
object $I$ is the one dimensional vector space concentrated in the trivial
degree $I=I(e)$. It follows from the naturality that the general form of an
associativity constraint on homogeneous $u\in U_{f},v\in V_{g},w\in W_{h}$ is
$a_{U,V,W}(u{\otimes}(v{\otimes}w))=\alpha(f,g,h)(u{\otimes}v){\otimes}w$,
where $\alpha(f,g,h)\in k^{\times}$ are non-zero scalars. The coherence (1) is
equivalent to the function $\alpha:G^{\times 3}\to k^{\times}$ being a
3-cocycle of $G$ with coefficients in the trivial $G$-module $k^{\times}$.
Denote by ${\cal V}(G,\alpha)$ the category of $G$-vector spaces with the
associator corresponding to a 3-cocycle $\alpha\in Z^{3}(G,k^{\times})$.
The category ${\cal V}(G,\alpha)$ is closed with the left internal hom
$[U,V]=Hom_{k}(U,V)$ being the space of linear maps with the grading
$[U,V]_{f}=\\{l|\ l(U_{g})\subset V_{fg}\\}$ (the right internal hom
$\\{U,V\\}=Hom_{k}(U,V)$ is graded by $\\{U,V\\}_{f}=\\{l|\ l(U_{g})\subset
V_{gf}\\}$). A graded vector space $U$ is dualisable if and only if $U$ is
finite dimensional and supported on invertible elements of $G$. For a (finite)
group $G$ the subcategory ${\cal V}_{fd}(G,\alpha)$ of finite dimensional
graded vector spaces is rigid and fusion (a so-called pointed fusion
category).
A monoidal functor $F:{\cal V}(G,\alpha)\to{\cal V}(Q,\beta)$ corresponds to a
group homomorphism $\varphi:G\to Q$ (the effect of $F$ on simple objects) and
a 2-cochain $\phi\in C^{2}(G,k^{\times})$ (defining the monoidal constraint).
Explicitly, $F(U)_{q}={\oplus}_{\varphi(g)=q}U_{g}$ with the monoidal
constraint $F_{U,V}(u{\otimes}v)=\phi(f,g)u{\otimes}v$ on $u\in U_{f},v\in
V_{g}$. The coherence (2) is equivalent to the coboundary condition
$d(\phi)=\alpha^{-1}\varphi^{*}(\beta)$ in $C^{*}(G,k^{\times})$.

Let $X$ be a $G$-set with the action $G\times X\to X,\ (g,x)\mapsto g.x$. The
category ${\cal M}(X)$ of $X$-graded vector spaces is a module category over
${\cal V}(G)$ with the categorical action $U*M=U{\otimes}_{k}M$ graded by
$(U*M)_{x}={\oplus}_{g.y=x}U_{g}{\otimes}_{k}M_{y}$ and the identity
associativity constraint. To make the category of $X$-graded vector spaces a
module category over ${\cal V}(G,\alpha)$ one needs a coboundary $\xi\in
C^{2}(G,Map(X,k^{\times}))$ for $\alpha$ considered an element of
$Z^{3}(G,k^{\times})\subset Z^{3}(G,Map(X,k^{\times}))$. Here
$Map(X,k^{\times})$ is the multiplicative group of maps $X\to k^{\times}$ with
the $G$-action coming from the $G$-set structure. Such $\xi$ defines an action
associativity constraint $U*(V*M)\to(U{\otimes}V)*M,\ \
u{\otimes}(v{\otimes}m)\mapsto\xi(f,g,x)(u{\otimes}v){\otimes}m$ with $u\in
U_{f},v\in V_{g},m\in M_{x}$. The action coherence is equivalent to the
equation $d(\xi)=\alpha$. The ${\cal V}(G,\alpha)$-module category ${\cal
M}(X,\xi)$ is action closed with the action internal hom
$[M,N]={\oplus}_{g}[M,N]_{g}$ where $[M,N]_{g}=\\{l\in Hom_{k}(M,N)|\
l(U_{x})\subset V_{g.x}\\}$. Note that any semi-simple module category over
${\cal V}(G,\alpha)$ is (equivalent to) such ${\cal M}(X,\xi)$ for some
$(X,\xi)$ (with $X$ being the set of isomorphism classes of simple objects).
Indecomposable semi-simple ${\cal V}(G,\alpha)$-module categories correspond
to transitive $G$-sets and have the form ${\cal M}(G/H,\xi)$ for a subgroup
$H\subset G$. Shapiro’s lemma allows one to replace $\xi\in
C^{2}(G,Map(G/H,k^{\times}))$ by $\tilde{\xi}\in C^{2}(H,k^{\times})$ with
$d(\tilde{\xi})=\alpha|_{H}$. Categories of module endofunctors ${\cal
E}nd_{{\cal V}(G,\alpha)}({\cal M}(G/H,\xi))$ are called group-theoretical
(see [Etingof et al (2015)] and references therein). For example, ${\cal
E}nd_{{\cal V}(G,1)}({\cal M}(G/G,1))$ is the category ${\cal R}ep(G)$ of
representations of $G$.
An associative algebra in ${\cal V}(G,\alpha)$ is a $G$-graded vector space
$A={\oplus}_{g}A_{g}$ with a multiplication, which is graded (or
$\alpha$-)associative, i.e. $a(bc)=\alpha(f,g,h)(ab)c$ for any $a\in
A_{f},b\in A_{g},c\in A_{h}$. The subalgebra $A_{e}={\cal V}(G,\alpha)(I,A)$
is an associative algebra in ${\cal V}ect$. For example, the action internal
end $[I(x),I(x)]$ for a simple object $I(x)\in{\cal M}(X,\xi)$ is supported on
the stabiliser $H=St_{G}(x)$ and coincides with the twisted group algebra
$k[H,\tilde{\xi}]$. The category of modules ${\cal
V}(G,\alpha)_{k[H,\tilde{\xi}]}$ is equivalent to ${\cal M}(G/H,\xi)$ as a
${\cal V}(G,\alpha)$-module category.
The monoidal centre ${\cal Z}({\cal V}(G,1))$ is equivalent to the category
with objects, $G$-graded vector spaces $Z={\oplus}_{g}Z_{g}$ with a linear
$G$-action compatible via $f(Z_{g})=Z_{fgf^{-1}}$. The tensor product is the
standard one. The half-braiding of $Z$ with a $G$-graded vector space $V$ is
given by $z_{U}(u{\otimes}z)=f.z{\otimes}u$ for $u\in U_{f}$. The monoidal
centre ${\cal Z}({\cal V}(G,\alpha))$ has a similar model (see e.g. [Davydov
and Simmons (2013)] for details).

The category ${\cal V}(G,\alpha)$ affords a braiding only if the group $G$ is
abelian. The naturality restricts the general form of a braiding to
$c_{U,V}(u{\otimes}v)=\gamma(f,g)v{\otimes}u$ for $u\in U_{f},v\in V_{g}$,
with some $\gamma(f,g)\in k^{\times}$. The coherence (6) is equivalent to
$(\alpha,\gamma)$ being an abelian 3-cocycle (a 3-cocycle of the Eilenberg-Mac
Lane cohomology complex $C^{*}(G,2,k^{\times})$). The coherence (7) for the
monoidal constraint of a braded monoidal functor $F:{\cal
V}(G,\alpha,\gamma)\to{\cal V}(Q,\beta,\xi)$ is equivalent to the coboundary
condition $d(\phi)=(\alpha,\gamma)^{-1}\varphi^{*}(\beta,\xi)$ in
$C^{*}(G,2,k^{\times})$. In particular, up to a braided equivalence ${\cal
V}(G,\alpha,\gamma)$ depends only on the cohomology class
$[(\alpha,\gamma)]\in H^{3}(G,2,k^{\times})$. Eilenberg and Mac Lane proved
that the assignment $(\alpha,\gamma)\mapsto q(g)=\gamma(g,g)$ gives an
isomorphism between $H^{3}(G,2,k^{\times})$ and the group $Q(G,k^{\times})$ of
quadratic functions, i.e. functions $q:G\to k^{\times}$ such that
$q(g^{-1})=q(g)$ and $q(f)q(g)q(h)q(fgh)=q(fg)q(fh)q(gh)$ (see e.g. [Joyal and
Street (1991b), Davydov et al (2013)] for details and references). It is
convenient to denote (the braided equivalence class of) ${\cal
V}(G,\alpha,\gamma)$ by ${\cal C}(G,q)$. For example, the conjugate
$\overline{{\cal C}(G,q)}$ is ${\cal C}(G,q^{-1})$ and the symmetric centre
${\cal Z}_{sym}({\cal C}(G,q))={\cal C}(Ker(q),1)$, where $Ker(q)=\\{g\in G|\
q(fg)=q(f)\ \forall f\in G\\}$ is the kernel of the quadratic function $q$.
The braided linear category ${\cal C}(G,q)$ is non-degenerate whenever the
quadratic function $q$ is.
The twisted group algebra $k[H,\tilde{\xi}]$ in ${\cal C}(G,q)$ is commutative
if and only if $H$ is an isotropic subgroup, i.e. $q|_{H}=1$. Since
$\tilde{\xi}$ is uniquely determined by $H$ we denote the commutative algebra
$k[H,\tilde{\xi}]$ by $A(H)$. The category ${\cal C}(G,q)_{A(H)}^{loc}$ of
local $A(H)$-modules is equivalent to ${\cal C}(H^{\perp}/H,q)$, where
$H^{\perp}=\\{g\in G|\ q(fg)=q(f)\ \forall f\in H\\}$ is the orthogonal
complement of $H$. The braided linear category ${\cal C}(G,q)$ is completely
anisotropic whenever the pair $G,q)$ is, i.e. whenever $G$ does not have
proper isotropic subgroups. The subgroup of the Witt group generated by
braided pointed fusion categories is the classical Witt group of quadratic
functions studied by C.T.C. Wall) see [Davydov et al (2013)] for details).

##  10 Conclusions

The development of the theory of tensor categories was driven by the
applications in representation theory (resulting in a good understanding of
Tannakian categories [Deligne and Milne (1982)]), high energy and condensed
matter physics (bringing fusion categories to the centre of attension).
Concepts crucial for the emergent theory of tensor categories came from or
play an important role in these applications: non-degenerate braided fusion
categories (representation categories of chiral algebras), module categories
(categories of boundary data), full centres (full state spaces of conformal
field theories, modular invariants), Witt equivalence (boson condensation,
defects in 2d conformal field theories).
Interest to non-rational conformal field theories in two dimensions stimulates
the study of more general (finite) tensor categories.
Higher categorical analogues of tensor categories (fusion 2-categories) play
an important role in 4d topological field theories. Progress in their study is
impossible without better understanding of their partial decategorifications.
Thus it is natural to expect further developments in the area of (finite set-
theoretical) monoidal categories.

Due to the space limitation many important references are not listed here.

## References

  * [Bénabou (1963)] Bénabou J. (1963), Catégories avec multiplication, C.R. Acad. Sci. Paris 256, 1887–1890.
  * [Baez and Dolan (1998)] Baez J., Dolan J. (1998), Categorification, in Higher Category Theory, Contemp. Math., vol. 230, Providence, Rhode Island: American Mathematical Society, pp. 1–36,
  * [Baez and Stay (2010)] Baez J. , Stay M. (2010), Physics, Topology, Logic and Computation: A Rosetta Stone. Lecture Notes in Physics, pp. 95-172.
  * [Barrett and Westbury (1999)] Barrett J., Westbury B. (1999). Spherical Categories. Advances in Mathematics. 143 (2): 357-375.
  * [Coulembier et al (2023)] Coulembier K., Etingof P., Ostrik P. (2023), Incompressible tensor categories, arxiv-2306.09745.
  * [Crane and Frenkel (1994)] Crane L. , Frenkel I. (1994), Four-dimensional Topological Quantum Field Theory, Hopf Categories, and the Canonical Bases, J. Math. Phys. 35 (10), 5136-5145.
  * [Davydov (2010)] Davydov A. (2010), Centre of an algebra, Advances in Mathematics, 225, 319-348, arXiv:0908.1250
  * [Davydov et al (2013)] Davydov A., Müger M., Nikshych D., Ostrik V. (2013), “Witt group of non-degenerate braided fusion categories”. Journal für die Reine und Angewandte Mathematik, 677, pp. 135-177.
  * [Davydov and Simmons (2013)] Davydov A., Simmons D., On Lagrangian algebras in group-theoretical braided fusion categories, J. Algebra, 471, 1 (2017), 149-175.
  * [Day (1970)] Day B. (1970), On closed categories of functors, Reports of the Midwest Category Seminar IV, Lecture Notes in Mathematics Vol. 137. Springer-Verlag, pp 1-38.
  * [Deligne (2002)] Deligne P. (2002), Catégories Tensorielles, Moscow Math. Journal 2, no. 2, 227-248.
  * [Deligne and Milne (1982)] Deligne P. , Milne J. (1982), Tannakian categories, in: Hodge Cycles, Motives, and Shimura Varieties, in: Lecture Notes in Math., vol. 900, Springer-Verlag, Berlin, pp. 101-228.
  * [Drinfeld (1989)] Drinfeld V. (1989), Quasi-Hopf algebras. Algebra i Analiz, 1(6):114–148.
  * [Eilenberg and Kelly (1966)] Eilenberg S., Kelly M. (1966), Closed Categories, in: Proceedings of the Conference on Categorical Algebra - La Jolla 1965, Springer, 421-562
  * [Etingof et al (2015)] Etingof P., Gelaki S., Nikshych D., Ostrik V. (2015), Tensor categories, Mathematical Surveys and Monographs, Volume 205, American Mathematical Society.
  * [Fröhlich et al (2006)] Fröhlich J., Fuchs J., Runkel I., Schweigert C. (2006), Correspondences of ribbon categories. Adv. Math. 199 , no. 1, 192-329.
  * [Freyd and Yetter(1989)] Freyd P., Yetter D. (1989), Braided compact closed categories with applications to low dimensional topology, Adv. in Math. 77 , 156-182.
  * [Joyal and Street (1991a)] Joyal A., Street R. (1991), Tortile Yang-Baxter operators in tensor categories, Journal of Pure and Applied Algebra, 71 (1): 43-51.
  * [Joyal and Street (1991b)] Joyal A., Street R. (1991), The geometry of tensor calculus I, Adv. Math. 88, no. 1, 55-112.
  * [Joyal and Street (1993)] Joyal A., Street R. (1993), Braided tensor categories, Adv. Math. 102, 20-78.
  * [Kelly (1964)] Kelly M. (1964), On MacLane’s Conditions for Coherence of Natural Associativities, Commutativities, etc. , Journal of Algebra 1, pp.397-402.
  * [Kelly (1982)] Kelly M. (1982), Basic concepts of enriched category theory, London Math. Soc. Lec. Note Series 64, Cambridge Univ. Press, Reprints in Theory and Applications of Categories 10 (2005) 1-136
  * [Kitaev (2006)] Kitaev A. (2006), Anyons in an exactly solved model and beyond, Ann. Phys. (N.Y.) 321, 2.
  * [1] S. Mac Lane, Natural associativity and commutativity, Rice University Studies 49 (1963) 28–46.
  * [Mac Lane (1971)] Mac Lane S. (1971), Categories For The Working Mathematician, Springer, Berlin.
  * [Majid (1991)] Majid S. (1991), Representations, duals and quantum doubles of monoidal categories, Rendiconti del Circolo Matematico di Palermo. Serie II. Supplemento. No. 26, 197-206.
  * [Moore and Seiberg (1990)] Moore G. , Seiberg N. (1990), Lectures on RCFT. Superstrings 89 (Trieste, 1989), 1129, World Sci. Publ.
  * [Pareigis (1995)] Pareigis B. (1995), On braiding and dyslexia, J. Algebra 171, 413-425.
  * [Ostrik (2003)] Ostrik V. (2003), Module categories, weak Hopf algebras and modular invariants, Transform Groups 8, 177-206.
  * [Saavedra Rivano (1972)] Saavedra Rivano N. (1972), Catégories Tannakiennes, Bulletin de la Société Mathématique de France 100: 417-430.
  * [Schauenburg (2001)] Schauenburg P. (2001), The monoidal center construction and bimodules. J. Pure Appl. Algebra 158(2-3), 325-346.
  * [Turaev (1994)] Turaev V. (1994), Quantum invariants of knots and 3-manifolds, de Gruyter Studies in Mathematics 18 Walter de Gruyter, Berlin.


††thanks: Also at: CNR-INO, Sesto Fiorentino, Italy††thanks: corresponding
author

#  Atom Interferometry with the Rb Blue Transitions

L. Salvi1 L. Cacciapuoti2 G. M. Tino1 G. Rosi1 1 Dipartimento di Fisica e
Astronomia and LENS, Università di Firenze,
INFN Sezione di Firenze, via Sansone 1, I-50019 Sesto Fiorentino (FI), Italy
2 European Space Agency, Keplerlaan 1, 2201 AZ Noordwijk, Netherlands

###### Abstract

We demonstrate a novel scheme for Raman-pulse and Bragg-pulse atom
interferometry based on the $5\mathrm{S}-6\mathrm{P}$ blue transitions of 87Rb
that provides an increase by a factor $\sim 2$ of the interferometer phase due
to accelerations with respect to the commonly used infrared transition at 780
nm. A narrow-linewidth laser system generating more than 1 W of light in the
420-422 nm range was developed for this purpose. Used as a cold-atom gravity
gradiometer, our Raman interferometer attains a stability to differential
acceleration measurements of $1\times 10^{-8}$ $g$ at 1 s and $2\times
10^{-10}$ $g$ after 2000 s of integration time. When operated on first-order
Bragg transitions, the interferometer shows a stability of $6\times 10^{-8}$ g
at 1 s, averaging to $1\times 10^{-9}$ g after 2000 s of integration time. The
instrument sensitivity, currently limited by the noise due to spontaneous
emission, can be further improved by increasing the laser power and the
detuning from the atomic resonance. The present scheme is attractive for high-
precision experiments as, in particular, for the determination of the
Newtonian gravitational constant.

Atom interferometry Tino and Kasevich (2014), first demonstrated three decades
ago, is now the operating principle of advanced quantum sensors for
fundamental physics experiments Tino (2021) and applications Bongs  _et al._
(2019). It is used to measure the gravitational acceleration Kasevich and Chu
(1992); Peters  _et al._ (1999); Müller  _et al._ (2008a); Le Gouët  _et al._
(2008); D’Amico  _et al._ (2019), the Earth’s gravity gradient McGuirk  _et
al._ (2002); Snadden  _et al._ (1998); Sorrentino  _et al._ (2014); Duan  _et
al._ (2014); Pereira Dos Santos (2015); Wang  _et al._ (2016); D’Amico  _et
al._ (2017) and rotations Gustavson  _et al._ (2000, 1997); Canuel  _et al._
(2006); Gauguet  _et al._ (2009). Atom interferometry experiments have been
performed to test the Einstein’s Equivalence Principle Tarallo  _et al._
(2014); Rosi  _et al._ (2017); Asenbaum  _et al._ (2020), to determine the
value of fundamental constants Rosi  _et al._ (2014); Parker  _et al._ (2018);
Morel  _et al._ (2020) and to search for quantum-gravity effects Amelino-
Camelia  _et al._ (2009). At the same time, space missions designed to study
gravitational waves, dark matter and fundamental aspects of gravity using
ultracold atoms have been proposed Tino  _et al._ (2019); El-Neaj  _et al._
(2020).

The sensitivity of light-pulse atom interferometers improves with the momentum
transferred to the atoms by the beam splitters and the mirrors. Different
large-momentum-transfer schemes have been demonstrated so far. They are based
on repeated stimulated Raman transitions McGuirk  _et al._ (2000), Raman
composite pulses Butts  _et al._ (2013), Raman double diffraction Lévèque  _et
al._ (2009), Raman adiabatic rapid passage Jaffe  _et al._ (2018); Kotru  _et
al._ (2015), Bragg diffraction Müller  _et al._ (2008b); Kovachy  _et al._
(2015) and Floquet atom optics Wilkason  _et al._ (2022). In a rubidium atom
interferometer, the two hyperfine levels of the ground state provide a very
good approximation of a two-level system when coupled with Raman lasers having
a large detuning from the single photon transition. In the case of Bragg
transitions between two momentum states, the larger is the photon recoil, the
better is the two-level atom approximation. Being able to transfer high
momentum in Bragg interferometry is therefore crucial both to control
systematic effects and to increase the efficiency of large-momentum-transfer
methods.

In this paper, we report on a novel approach based on the use of Raman and
Bragg pulses with the blue $5\mathrm{S}-6\mathrm{P}$ transitions of rubidium
to realize an atomic gravity gradiometer. Thanks to the higher photon recoil
($\lambda\simeq 420$ nm) compared to the commonly used D2 line ($\lambda\simeq
780$ nm), the momentum-space splitting is increased by a factor $\sim 1.9$
while preserving the robustness and simplicity of the three-pulse Mach-Zehnder
sequence. The shorter laser wavelength also translates into a smaller
diffraction, thus reducing the systematic effects produced by the Gouy phase
and wavefront distortions Wicht  _et al._ (2005); Bade  _et al._ (2018). This
scheme can find interesting applications in high precision measurements,
especially for the determination of the Newtonian gravitational constant Rosi
_et al._ (2014); Rosi (2017).

---
Figure 1: a) Partial level diagram of the 87Rb atom. The Raman transitions on
the D2 ($5^{2}\mathrm{S}_{1/2}-5^{2}\mathrm{P}_{3/2}$) line at 780.2 nm (light
red), the $5^{2}\mathrm{S}_{1/2}-6^{2}\mathrm{P}_{1/2}$ line at 421.7 nm
(blue) and the $5^{2}\mathrm{S}_{1/2}-6^{2}\mathrm{P}_{3/2}$ line at 420.3 nm
(violet) explored in this work are shown. To streamline the figure, we have
omitted the Bragg transition on the
$5^{2}\mathrm{S}_{1/2}(F=1)-6^{2}\mathrm{P}_{1/2}(F=1)$ line used in this
work. Additional decay channels leading the atoms to the ground state are
indicated in red, dark red and black. b) Experiment cycle consisting of
magneto-optical trapping, atom launch and juggling, final velocity selection
and atom interferometry sequence. The velocity selection consists of three
Raman $\pi$ pulses at 780 nm alternated with blow away pulses. The two Mach-
Zehnder atom interferometers are vertically separated by 30 cm.

We demonstrate the three-pulse Raman gravity gradiometer both on the
$5\mathrm{S}_{1/2}-6\mathrm{P}_{1/2}$ and on the
$5\mathrm{S}_{1/2}-6\mathrm{P}_{3/2}$ transitions of 87Rb at 421.7 nm and
420.3 nm, respectively Glaser  _et al._ (2020); the Bragg interferometer is
operated on the $5\mathrm{S}_{1/2}-6\mathrm{P}_{3/2}$ transition (see Fig. 1).
A critical aspect of this scheme is the laser power required to achieve high
pulse efficiency and low single-photon scattering rates. For a given ratio $R$
between the two-photon Rabi frequency and the single-photon scattering rate,
the Rabi frequency is proportional to $\lambda^{3}$ and to the branching ratio
of the transition sup . Therefore, the smaller wavelength of the blue
transition leads to a reduction of the Rabi frequency by a factor 0.16
compared to the usual Raman interferometers operated at 780 nm. The relevant
branching ratios (see Fig. 1a) are 0.19 and 0.24 for the 421.7 nm and 420.3 nm
transitions respectively Safronova  _et al._ (2004). As a result, a higher
power by a factor $\sim 30$ is required for the Raman lasers to achieve the
same ratio $R$ and the same Rabi frequency as for the D2 line at 780 nm. A
similar reasoning applies to the Bragg lasers. This technical difficulty has
probably hindered so far the use of the $5\mathrm{S}-6\mathrm{P}$ transitions
for atom interferometry. For this experiment, we have developed a compact,
narrow-linewidth ($\simeq$ 250 kHz) blue laser system with an output power of
more than 1 W resulting in $\sim 0.5$ W for the Raman and Bragg pulses on the
atoms. It should be noticed that two-photon ionization processes are
negligible for the laser intensity and detuning from resonance employed in
this work Anderlini  _et al._ (2004); sup .

The atom interferometer apparatus was described in detail in Rosi  _et al._
(2014); Prevedelli  _et al._ (2014). Briefly, a three-dimensional magneto-
optical trap (3D-MOT) loaded by a two-dimensional MOT confines about $10^{9}$
rubidium atoms in the vacuum chamber at the bottom of the experimental setup.
The sample is cooled down to a temperature of $\sim 4$ $\mu$K while it is
launched vertically inside the interferometer tube with the moving-molasses
technique. Two atomic clouds are loaded in rapid sequence and juggled to have
them simultaneously reaching the apogees of their ballistic trajectories at
about 70 cm and 100 cm above the 3D-MOT (Fig. 1b). The interferometer tube
encloses a magnetically shielded region where a uniform B-field oriented along
the vertical direction defines the quantization axis. The blue interferometer
lasers are superimposed to the infrared Raman beams on a dichroic mirror. An
achromatic doublet pair and an achromatic wave plate collimate and polarize
them before entering the vacuum system. After exiting from the top of the
interferometer tube, they pass through a quarter-wave plate and a retro-
reflection mirror. The lasers are prepared to have circular polarization in
the Raman interferometry experiment and linear polarization in the Bragg
interferometer. The retro-reflection mirror is mounted on actuators that
provide tip-tilt control and compensate for the Coriolis acceleration. The two
atomic samples are velocity selected and prepared in the $|F=1,m_{F}=0\rangle$
state with a combination of Raman $\pi$ pulses and resonant blow-away laser
pulses (Fig. 1b). Infrared Raman lasers are used in the preparation phase to
reduce atom losses due to off-resonance scattering. The two clouds, each
composed of about $10^{5}$ atoms, are simultaneously interrogated with a
$\pi/2-\pi-\pi/2$ pulse sequence by the Raman or Bragg lasers (Fig. 1b).
Finally, the normalized population at the two output ports of the
interferometers is measured via fluorescence detection.

The blue laser system developed for this experiment comprises three infrared
lasers. Each of them is composed of an extended cavity diode laser (ECDL) with
an intracavity interference filter optically injecting a tapered amplifier
(TA). The first laser is used as a reference for the two interferometry
lasers, which are operated in a master-slave configuration. The TA of the
reference laser delivers about 250 mW in the infrared. This light is fiber
coupled and sent towards a periodically poled lithium niobate (PPLN) waveguide
to produce 8-10 mW of blue light. The main beam is arranged in a pump-probe
configuration to detect the saturated absorption signal of 87Rb from a vapour
cell. The ECDL frequency is stabilized on the crossover signal of the
$5\mathrm{S}_{1/2}(F=2)-6\mathrm{P}_{1/2}(F=1,2)$ or of the
$5\mathrm{S}_{1/2}(F=2)-6\mathrm{P}_{3/2}(F=1,2)$ transitions when operating
the interferometer on the 421.7 nm and 420.3 nm lines, respectively. A small
portion of the infrared radiation from the reference laser is used to offset-
lock the interferometry master laser. This frequency offset determines the
detuning of the two interferometry beams from the $6\mathrm{P}$ manifold. The
master and slave laser systems are identical. The infrared ECDL is in this
case amplified to more than 1 W in a TA. This beam is matched to the
fundamental transverse mode of a bow-tie enhancement cavity built around a
lithium triborate (LBO) doubling crystal. The conversion efficiency of the
second harmonic generation process reaches a maximum of 60%, with more than
700 mW of blue light generated from 1.2 W of injected power. The slave
interferometry laser is stabilized on the master by a two-stage optical phase
locked loop (OPLL). The primary OPLL detects the beatnote between the ECDL
infrared beams before they inject the TAs. In this way, we minimize the signal
propagation delay and maximize the loop bandwidth. In the Raman configuration,
the beatnote is mixed with the 3.4 GHz reference frequency, which corresponds
to half the hyperfine splitting of the Rb ground state . The downconverted
signal is then compared with the signal generated by direct digital synthesis
(DDS) in a fast digital phase-frequency detector (PFD). The DDS frequency is
swept around 20 MHz with a phase-continuous linear frequency ramp to
compensate the varying Doppler shift during the interferometer sequence. The
second OPLL reduces the phase noise introduced by the frequency doubling
process. The beatnote of the blue radiation from the Raman lasers is detected
before they enter the optical fiber. This signal is mixed with the 6.8 GHz
output of our reference oscillator and then compared in a PFD with the doubled
(2$\times$20 MHz) frequency ramp acting on the primary OPLL. The error signal
is then used to control a piezo actuator, which translates one of the mirrors
along the optical path of the master Raman laser. Our setup is very versatile
and it can be easily rearranged to generate both Raman and Bragg lasers for
atoms interrogation.

Figure 2: Phase noise power spectra of the infrared beatnote with the primary
OPLL closed (blue line) and of the blue beatnote with the secondary OPLL open
(black line) and closed (red line). Inset: Beatnote signal between the two
Raman lasers on the fundamental (black line) and frequency doubled (red line)
light with the primary OPLL closed (resolution bandwidth: 10 kHz).

To evaluate the contribution of the frequency doubling process to the
interferometry lasers phase noise, we characterize the master-slave beatnote
both at the fundamental frequency and at the second harmonic. The beatnote of
the freely running lasers shows a FWHM linewidth of $147(5)$ kHz in the
infrared and $255(11)$ kHz in the blue. Figure 2 (inset) shows the beatnote
signals in the infrared (black line) and in the blue (red line) after closing
the primary OPLL. The frequency doubling process increases the phase noise at
frequencies below 250 kHz. On the contrary, a reduction of the blue beatnote
servo-bumps is observed, due to the filtering effect introduced by the finite
width ($\simeq 500$ kHz HWHM) of the doubling cavity resonance. Figure 2 shows
the phase noise power spectrum of the infrared beatnote when the primary OPLL
is closed (blue line) and of the blue beatnote with the secondary OPLL open
(black line) and closed (red line). The excess of noise introduced by the
frequency doubling process increases at an average rate of 20 dB/decade as the
frequency decreases. The secondary OPLL reduces the noise below 1 kHz and
restores the phase coherence. This is sufficient for our experiment due to the
excellent suppression factor that a gravity gradiometer can ensure on the
common-mode phase noise of the interrogation lasers. For the operation of the
interferometer as an accelerometer, this performance, currently limited by the
bandwidth of our piezo, can be improved and extended to higher frequencies by
using a faster actuator (e.g. an electro-optical modulator).

We characterize the Raman gravity gradiometer sensitivity on both the
$5\mathrm{S}_{1/2}-6\mathrm{P}_{1/2}$ and
$5\mathrm{S}_{1/2}-6\mathrm{P}_{3/2}$ transitions and compare it to the
traditional interferometer on the D2 line. The Raman beams are collimated to a
waist of $9$ mm for the blue and $13$ mm for the infrared. With respect to the
typical parameters of our infrared Raman interferometer, we maintain a pulse
separation of $T=$ 160 ms and a repetition rate of 0.5 Hz, but we increase the
$\pi$ pulse duration from 24 to 48 $\mu$s. The intensity ratio $R_{21}$
between the lasers interacting with the $F=2$ and $F=1$ levels of the ground
state is tuned to cancel the differential AC Stark shift Peters  _et al._
(2001); sup . For the interferometer on the
$5\mathrm{S}_{1/2}-6\mathrm{P}_{1/2}$ transition, we use a total maximum Raman
power of 0.44 W (measured after fiber), an intensity ratio $R_{21}\approx 1$,
and a positive detuning of 50 MHz from the $F=2-F^{\prime}=2$ transition. On
the $5\mathrm{S}_{1/2}-6\mathrm{P}_{3/2}$ transition, we have a total maximum
power of 0.55 W and $R_{21}\approx 2$, with a negative detuning of 80 MHz from
the $F=2-F^{\prime}=1$ transition. The spontaneous emission probability for
the overall interferometer is $\sim 13$% for both blue wavelengths.

---
Figure 3: Experimental ellipses (360 points each) obtained with Raman
interferometry at 420.3 nm (red open squares), at 421.7 (blue open circles)
and at 780 nm (black triangles). Elliptical fits are indicated with red, blue
and black solid lines, respectively. To avoid overlaps and improve clarity,
the centers of the ellipses in the figure have been translated.

Figure 3 shows typical Lissajous curves obtained by plotting the interference
fringes of the upper interferometer as a function of the fringes of the lower
interferometer when using Raman pulses on the blue transitions (420.3 and
421.7 nm) and on the infrared transition (780 nm) during the interferometer
sequence. By comparing the results obtained at the three different
wavelengths, we experimentally confirm the expected increase by a factor 1.9
of the gradiometer phase shift: from the 0.6 rad observed at 780 nm to the 1.1
rad obtained with the blue transitions. Figure 3 also shows that the fringe
contrast is reduced by 10-20% with respect to the interferometer on the
infrared transition, which is partially explained by spontaneous emission.
Moreover, the ellipses obtained on the blue transitions reveal a higher noise
level. After analysing the atomic fluorescence signals at detection, we could
attribute this extra noise to the higher single-photon scattering rate on the
blue transitions, which affects the atom counting at the output ports of the
interferometers sup . To characterize the instrument sensitivity and the
medium-term stability, we evaluate the Allan deviation of the gravity gradient
measurements normalized to the local gravitational acceleration over a total
measurement duration of about 6 h. As shown in Fig. 4, the Allan deviation
decreases as $1/\sqrt{t}$, where $t$ is the integration time, showing that
gravity gradient measurements are affected by a white noise process. In this
respect the interferometers operated on the blue and infrared transitions have
similar performance within a maximum difference of 30%. Indeed, for our
experimental conditions, the higher signal ($\times 1.9$) measured on the blue
transitions is compensated by the higher noise levels that we observe due to
single-photon scattering processes. This important noise contribution can be
reduced considerably by using higher power laser sources that allow to operate
the Raman lasers at a larger detuning from the excited states. The resulting
interferometer sensitivity to differential acceleration measurements is
$1\times 10^{-8}$ g at 1 s, reaching $2\times 10^{-10}$g after 2000 s of
integration time.

---
Figure 4: Allan deviation plots of the gravity gradiometer measurements
normalized to the local gravitational acceleration, obtained with Raman
interferometry at 420.3 nm (red diamonds), at 421.7 nm (blue triangles) and at
780 nm (black squares).

In a second experiment, we tune the blue lasers on the
$5\mathrm{S}_{1/2}-6\mathrm{P}_{3/2}$ transition at 420.3 nm and configure
them to operate the gravity gradiometer with first-order Bragg pulses. To
reduce the single-photon scattering rate and improve the signal-to-noise level
of the atom number measurements at the output ports of the interferometer,
Bragg lasers are set to a negative detuning of 252 MHz with respect to the
$F=1-F^{\prime}=1$ transition, about three times larger than for the Raman
interferometer. As a consequence, for a total laser power of 0.5 W, the beam
waist had to be reduced to 7 mm. Rubidium atoms are interrogated in the Bragg
interferometer on a $\pi/2-\pi-\pi/2$ sequence of Gaussian pulses. The $\pi$
pulse has a Gaussian standard deviation of 21 $\mu$s and a separation of
$T=80$ ms with respect to the beam splitter pulses. The three-pulse sequence
takes place during the ascending phase of the atomic clouds to maximize the
spatial separation between the two output ports of the interferometers. Due to
the higher photon recoil carried by the blue lasers, first-order Bragg
transitions are already sufficient to use the standard position-resolved
detection systems for atom counting. Momentum-resolved detection methods Cheng
_et al._ (2018) will also benefit from the larger velocity difference.
Finally, during the interferometric sequence we change the frequency detuning
of the Bragg $\pi$ pulse by 40 MHz to introduce a differential phase shift and
mimic the presence of a gravity gradient Roura (2017); D’Amico  _et al._
(2017) to further open the ellipse and improve the best fit estimate. Figure 5
(inset) shows the Lissajous plot obtained from the gravity gradiometer
measurements using Bragg-pulse interferometry. The fringes contrast is similar
to the one observed for the Raman interferometers on the blue transitions, but
it is obtained at a significantly larger frequency detuning and for a shorter
interferometer duration. As for the Raman interferometer, spontaneous emission
is the main contributor to the noise that we observe in our measurements. The
stability of the first-order Bragg gravity gradiometer is characterized by
evaluating the Allan deviation of the differential acceleration measurements
(see Fig. 5). The slope of the Allan deviation curve indicates the presence of
a white noise process. The instrument stability is $5.9\times 10^{-8}$ g at 1
s, averaging down to $1\times 10^{-9}$ g after 2000 s of integration time.
Compared to the results obtained with the third-order Bragg gravity
gradiometer on the infrared transition characterized in D’Amico  _et al._
(2016), the sensitivity to differential acceleration measurements is degraded
by a factor 1.6, which exactly corresponds to the ratio of the photon momenta
transferred to the atoms in the two experiments. This result would confirm
that the factor 1.9 improvement on the instrument sensitivity can indeed be
obtained when the Bragg lasers have adequate detuning from the single-photon
transition. The laser power available in the blue is not sufficient to operate
our gravity gradiometer on higher-order Bragg transitions therefore a direct
comparison, blue vs infrared, is not possible.

---
Figure 5: Allan deviation plot of the gravity gradiometer measurements
normalized to the local gravitational acceleration obtained with Bragg
interferometry at 420.3 nm. Inset: Lissajous plot of the 12500 data points
collected during the measurement run, together with the elliptical best fit
(red line).

In conclusion, we have demonstrated Raman-pulse and Bragg-pulse atom
interferometry on the 420.3 nm and 421.7 nm transitions of 87Rb. The larger
momentum transferred to the atoms on the blue transitions provides an increase
by a factor 1.9 of the gravity gradiometer differential phase with respect to
the infrared transition at 780 nm, while preserving the simplicity and
robustness of the three-pulse Mach-Zehnder interferometer. With a stability of
$1\times 10^{-8}$ g at 1 s, the Raman gravity gradiometer noise averages down
to $2\times 10^{-10}$g after 2000 s of integration time at both blue
wavelengths. When the atoms are interrogated in the first-order Bragg
interferometer, we obtain a stability of $5.9\times 10^{-8}$ g at 1 s,
reaching $1\times 10^{-9}$ g after 2000 s of integration time. Due to the
higher intensity required for the blue transitions compared to the infrared,
the blue interrogation lasers had to be shaped to a smaller beam diameter and
operated to deliver longer $\pi$ pulses at a relatively smaller detuning from
the resonance with the single-photon transition. In this configuration, the
lower number of atoms at detection and the higher spontaneous emission rate
reduce the fringe contrast and introduce substantial noise in the differential
phase measurement. As also shown in the Bragg-pulse interferometry experiment,
higher laser power and larger detuning from resonance are required to reach
optimal working conditions. They can be obtained by adding an amplification
stage on the infrared radiation before the second-harmonic generation or by
injection-locking schemes of Ti:sapphire lasers that have recently been
demonstrated to produce several watts of infrared light with an excellent
spatial profile Takano  _et al._ (2021). We expect that such an upgrade will
allow to reduce the noise due to spontaneous emission and unravel the full
potential of the presented interferometer scheme. We also point out that the
near-unity intensity ratio of the Raman beams for which the differential AC
Stark shift is canceled at 421.7 nm allows to optimally use the available
laser power as the two-photon Rabi frequency is maximized by this ratio.
Moreover, the deviation from the unity ratio remains on the order of 10% for a
positive detuning up to 300 MHz. For the 420.3 nm transition, the optimal
intensity ratio is found at about 1.36 and it is detuning-independent to first
order for a significantly higher negative detuning of 458 MHz sup . This is
important, for example, in schemes where it is necessary to change the laser
frequency during the interferometer sequence to mimic the presence of a
gravity gradient Roura (2017); D’Amico  _et al._ (2017), as also done in this
work. Large-momentum-transfer schemes already demonstrated with the infrared
transition can be implemented in the blue to further increase the gradiometer
signal. With this respect, the larger photon recoil transferred by the blue
lasers in a Bragg transition is also providing a larger separation between the
two coupled momentum states thus promising a better control of systematic
effects and a higher efficiency of the large-momentum-transfer pulses.
Furthermore, the shorter laser wavelength will lead to a reduction of
systematic effects like the Gouy phase and wavefront distortion currently
limiting several atom interferometry experiments. Therefore, we anticipate
that the scheme demonstrated in this work will be instrumental for high-
precision experiments as, in particular, for the determination of the
Newtonian gravitational constant Jain  _et al._ (2021).

###### Acknowledgements.

This work was supported by INFN under the OLAGS project and the QuantERA grant
SQUEIS. GR acknowledges financial support from the European Research Council,
Grant No. 804815 (MEGANTE) and from MIUR (Italian Ministry of Education,
Universities and Research) under the FARE-TENMA project. LS acknowledges
support from Università di Firenze under the QuEGI-Quantum Enhanced Gravity
Interferometry project. The authors thank Giovanni Gangale for help in an
early stage of the experiment.

## References

  * Tino and Kasevich (2014) G. M. Tino and M. A. Kasevich,  _Atom interferometry_ (SIF and IOS Press, Bologna, Amsterdam, 2014).
  * Tino (2021) G. M. Tino, Quantum Science and Technology 6, 024014 (2021).
  * Bongs  _et al._ (2019) K. Bongs, M. Holynski, J. Vovrosh, P. Bouyer, G. Condon, E. Rasel, C. Schubert, W. P. Schleich, and A. Roura, Nature Reviews Physics 1, 731 (2019).
  * Kasevich and Chu (1992) M. Kasevich and S. Chu, Appl. Phys. B 54, 321 (1992).
  * Peters  _et al._ (1999) A. Peters, K. Y. Chung, and S. Chu, Nature 400, 849 (1999).
  * Müller  _et al._ (2008a) H. Müller, S.-w. Chiow, S. Herrmann, S. Chu, and K.-Y. Chung, Phys. Rev. Lett. 100, 031101 (2008a).
  * Le Gouët  _et al._ (2008) J. Le Gouët, T. Mehlstäubler, J. Kim, S. Merlet, A. Clairon, A. Landragin, and F. P. Dos Santos, Appl. Phys. B 92, 133 (2008).
  * D’Amico  _et al._ (2019) G. D’Amico, L. Cacciapuoti, M. Jain, S. Zhan, and G. Rosi, Eur. Phys. J. D 73, 1 (2019).
  * McGuirk  _et al._ (2002) J. M. McGuirk, G. Foster, J. Fixler, M. Snadden, and M. Kasevich, Phys. Rev. A 65, 033608 (2002).
  * Snadden  _et al._ (1998) M. Snadden, J. McGuirk, P. Bouyer, K. Haritos, and M. Kasevich, Phys. Rev. Lett. 81, 971 (1998).
  * Sorrentino  _et al._ (2014) F. Sorrentino, Q. Bodart, L. Cacciapuoti, Y.-H. Lien, M. Prevedelli, G. Rosi, L. Salvi, and G. M. Tino, Phys. Rev. A 89, 023607 (2014).
  * Duan  _et al._ (2014) X.-C. Duan, M.-K. Zhou, D.-K. Mao, H.-B. Yao, X.-B. Deng, J. Luo, and Z.-K. Hu, Phys. Rev. A 90, 023617 (2014).
  * Pereira Dos Santos (2015) F. Pereira Dos Santos, Phys. Rev. A 91, 063615 (2015).
  * Wang  _et al._ (2016) Y.-P. Wang, J.-Q. Zhong, X. Chen, R.-B. Li, D.-W. Li, L. Zhu, H.-W. Song, J. Wang, and M.-S. Zhan, Opt. Commun. 375, 34 (2016).
  * D’Amico  _et al._ (2017) G. D’Amico, G. Rosi, S. Zhan, L. Cacciapuoti, M. Fattori, and G. M. Tino, Phys. Rev. Lett. 119, 253201 (2017).
  * Gustavson  _et al._ (2000) T. Gustavson, A. Landragin, and M. Kasevich, Class. Quantum Grav. 17, 2385 (2000).
  * Gustavson  _et al._ (1997) T. Gustavson, P. Bouyer, and M. Kasevich, Phys. Rev. Lett. 78, 2046 (1997).
  * Canuel  _et al._ (2006) B. Canuel, F. Leduc, D. Holleville, A. Gauguet, J. Fils, A. Virdis, A. Clairon, N. Dimarcq, C. J. Bordé, A. Landragin, _et al._ , Phys. Rev. Lett. 97, 010402 (2006).
  * Gauguet  _et al._ (2009) A. Gauguet, B. Canuel, T. Lévèque, W. Chaibi, and A. Landragin, Phys. Rev. A 80, 063604 (2009).
  * Tarallo  _et al._ (2014) M. G. Tarallo, T. Mazzoni, N. Poli, D. Sutyrin, X. Zhang, and G. M. Tino, Phys. Rev. Lett. 113, 023005 (2014).
  * Rosi  _et al._ (2017) G. Rosi, G. D’Amico, L. Cacciapuoti, F. Sorrentino, M. Prevedelli, M. Zych, Č. Brukner, and G. M. Tino, Nat. Commun. 8, 1 (2017).
  * Asenbaum  _et al._ (2020) P. Asenbaum, C. Overstreet, M. Kim, J. Curti, and M. A. Kasevich, Phys. Rev. Lett. 125, 191101 (2020).
  * Rosi  _et al._ (2014) G. Rosi, F. Sorrentino, L. Cacciapuoti, M. Prevedelli, and G. M. Tino, Nature 510, 518 (2014).
  * Parker  _et al._ (2018) R. H. Parker, C. Yu, W. Zhong, B. Estey, and H. Müller, Science 360, 191 (2018).
  * Morel  _et al._ (2020) L. Morel, Z. Yao, P. Cladé, and S. Guellati-Khélifa, Nature 588, 61 (2020).
  * Amelino-Camelia  _et al._ (2009) G. Amelino-Camelia, C. Lämmerzahl, F. Mercati, and G. M. Tino, Phys. Rev. Lett. 103, 171302 (2009).
  * Tino  _et al._ (2019) G. M. Tino, A. Bassi, G. Bianco, K. Bongs, P. Bouyer, L. Cacciapuoti, S. Capozziello, X. Chen, M. L. Chiofalo, A. Derevianko, _et al._ , Eur. Phys. J. D 73, 228 (2019).
  * El-Neaj  _et al._ (2020) Y. A. El-Neaj, C. Alpigiani, S. Amairi-Pyka, H. Araújo, A. Balaž, A. Bassi, L. Bathe-Peters, B. Battelier, A. Belić, E. Bentine, _et al._ , EPJ Quant. Tech. 7, 228 (2020).
  * McGuirk  _et al._ (2000) J. M. McGuirk, M. J. Snadden, and M. A. Kasevich, Phys. Rev. Lett. 85, 4498 (2000).
  * Butts  _et al._ (2013) D. L. Butts, K. Kotru, J. M. Kinast, A. M. Radojevic, B. P. Timmons, and R. E. Stoner, J. Opt. Soc. Am. B 30, 922 (2013).
  * Lévèque  _et al._ (2009) T. Lévèque, A. Gauguet, F. Michaud, F. Pereira Dos Santos, and A. Landragin, Phys. Rev. Lett. 103, 080405 (2009).
  * Jaffe  _et al._ (2018) M. Jaffe, V. Xu, P. Haslinger, H. Müller, and P. Hamilton, Phys. Rev. Lett. 121, 040402 (2018).
  * Kotru  _et al._ (2015) K. Kotru, D. L. Butts, J. M. Kinast, and R. E. Stoner, Phys. Rev. Lett. 115, 103001 (2015).
  * Müller  _et al._ (2008b) H. Müller, S.-w. Chiow, Q. Long, S. Herrmann, and S. Chu, Phys. Rev. Lett. 100, 180405 (2008b).
  * Kovachy  _et al._ (2015) T. Kovachy, P. Asenbaum, C. Overstreet, C. A. Donnelly, S. M. Dickerson, A. Sugarbaker, J. M. Hogan, and M. A. Kasevich, Nature 528, 530–533 (2015).
  * Wilkason  _et al._ (2022) T. Wilkason, M. Nantel, J. Rudolph, Y. Jiang, B. E. Garber, H. Swan, S. P. Carman, M. Abe, and J. M. Hogan, Phys. Rev. Lett. 129, 183202 (2022).
  * Wicht  _et al._ (2005) A. Wicht, E. Sarajlic, J. M. Hensley, and S. Chu, Phys. Rev. A 72, 023602 (2005).
  * Bade  _et al._ (2018) S. Bade, L. Djadaojee, M. Andia, P. Cladé, and S. Guellati-Khelifa, Phys. Rev. Lett. 121, 073603 (2018).
  * Rosi (2017) G. Rosi, Metrologia 55, 50 (2017).
  * Glaser  _et al._ (2020) C. Glaser, F. Karlewski, J. Kluge, J. Grimmel, M. Kaiser, A. Günther, H. Hattermann, M. Krutzik, and J. Fortágh, Phys. Rev. A 102, 012804 (2020).
  * (41) See Supplemental Material at URL_will_be_inserted_by_publisher for more information.
  * Safronova  _et al._ (2004) M. S. Safronova, C. J. Williams, and C. W. Clark, Phys. Rev. A 69, 022509 (2004).
  * Anderlini  _et al._ (2004) M. Anderlini, E. Courtade, D. Ciampini, J. H. Müller, O. Morsch, and E. Arimondo, J. Opt. Soc. Am. B 21, 480 (2004).
  * Prevedelli  _et al._ (2014) M. Prevedelli, L. Cacciapuoti, G. Rosi, F. Sorrentino, and G. M. Tino, Phil. Trans. R. Soc. A. 372, 20140030 (2014).
  * Peters  _et al._ (2001) A. Peters, K. Y. Chung, and S. Chu, Metrologia 38, 25 (2001).
  * Cheng  _et al._ (2018) Y. Cheng, K. Zhang, L.-L. Chen, T. Zhang, W.-J. Xu, X.-C. Duan, M.-K. Zhou, and Z.-K. Hu, Phys. Rev. A 98, 043611 (2018).
  * Roura (2017) A. Roura, Phys. Rev. Lett. 118, 160401 (2017).
  * D’Amico  _et al._ (2017) G. D’Amico, G. Rosi, S. Zhan, L. Cacciapuoti, M. Fattori, and G. M. Tino, Phys. Rev. Lett. 119, 253201 (2017).
  * D’Amico  _et al._ (2016) G. D’Amico, F. Borselli, L. Cacciapuoti, M. Prevedelli, G. Rosi, F. Sorrentino, and G. M. Tino, Phys. Rev. A 93, 063628 (2016).
  * Takano  _et al._ (2021) T. Takano, H. Ogawa, C. Ohae, and M. Katsuragawa, Opt. Express 29, 6927 (2021).
  * Jain  _et al._ (2021) M. Jain, G. M. Tino, L. Cacciapuoti, and G. Rosi, Eur. Phys. J. D 75 (2021).


# Automatic tumour segmentation in H&E-stained whole-slide images of the
pancreas.

P. Vendittelli  Department of Pathology, Radboud University Medical Center,
Nijmegen, The Netherlands  E.M.M. Smeets  Department of Pathology, Radboud
University Medical Center, Nijmegen, The Netherlands  G.J.S. Litjens
Department of Pathology, Radboud University Medical Center, Nijmegen, The
Netherlands

###### Abstract

Pancreatic cancer will soon be the second leading cause of cancer-related
death in Western society. Imaging techniques such as CT, MRI and ultrasound
typically help providing the initial diagnosis, but histopathological
assessment is still the gold standard for final confirmation of disease
presence and prognosis. In recent years machine learning approaches and
pathomics pipelines have shown potential in improving diagnostics and
prognostics in other cancerous entities, such as breast and prostate cancer. A
crucial first step in these pipelines is typically identification and
segmentation of the tumour area. Ideally this step is done automatically to
prevent time consuming manual annotation. In this paper, we propose a multi-
task convolutional neural network to balance disease detection and
segmentation accuracy. We validated our approach on a dataset of 29 patients
(for a total of 58 slides) at different resolutions. The best single task
segmentation network achieved a median Dice of 0.885 (0.122) IQR at a
resolution of 15.56 $\mu$m. Our multi-task network improved on that with a
median Dice score of 0.934 (0.077) IQR.

###### keywords:

Digital Pathology, Deep Learning, Pancreatic Ductal Adenocarcinoma,
Segmentation

##  1 INTRODUCTION

Pancreatic ductal adenocarcinoma (PDAC) is now the seventh leading cause of
cancer-related deaths worldwide [1] and will soon become the second leading
cause of cancer-related death in Western society [2]. Europe has the highest
burden of PDAC in the world, with 150 000 new cases in 2018 and 95 000
deaths/year. Worldwide there were over 500 000 deaths in 2020[3]. Moreover,
PDAC has the lowest survival of all types of cancers [4] (median survival time
4.6 months), with patients losing 98% of their healthy life expectancy.

Although imaging techniques as computed tomography (CT), magnetic resonance
imaging (MRI) and endoscopic ultrasound (EUS) [2] are widely used for an
initial diagnosis of pancreatic cancer, biopsies are the gold standard for
final confirmation of disease presence and prognosis. In recent years machine
learning approaches and pathomics pipelines have shown potential in improving
diagnostics and prognostics in other cancerous entities, such as breast and
prostate cancer [5][6], but very little research has been done on the analysis
of pancreatic histopathological images through automatic systems. As far as
the authors are aware, Fu et al.[7] proposed an automatic method based on
convolutional neural networks (CNNs) for pancreatic cancer detection and
segmentation. They perform patch- and Whole-slide Image- level classification
to determine which areas of an image are most critical for PDAC identification
and use the segmentation network as control study for their classification
network. A key challenge when combining detection and segmentation in a single
convolutional neural network is to balance detection and segmentation
accuracy. This paper proposes a multi-task convolutional neural network for
segmenting PDAC on whole-slide images (WSIs) and compares it to a single-task
segmentation network. Furthermore, we explore which image resolution is most
suitable for PDAC detection and segmentation.

##  2 METHODOLOGY

### Data

For this study, 29 patients were selected from a PET-cohort of 158 patients
with histologically proven pancreatic ductal adenocarcinoma (PDAC) who
underwent [18F]FDG PET/CT in the period between 2004 and 2015 at Radboud
University Medical Center, Nijmegen (The Netherlands) during diagnostic work-
up for a total of 58 (2 slides per patient) H&E stained slides. Fifty-one
slides contained cancerous tissue, while seven only had normal tissue. All the
positive cases were collected with tumour annotations (Figure 1) made under
the supervision of a pathologist. Additionally, the annotations were masked
with the tissue masks generated by a tissue-background segmentation
algorithm[8] in order to remove possible presence of the background. Slides
were organised in 5 folds for cross-validation, maintaining a ratio for
training - validation of 0.8 - 0.2. There were no slides coming from the same
patient in both the training and validation folds. Due to the limited data
available, we did not use a separate test fold.

Figure 1: Example of two cases with coarse tumour annotations, masked with
tissue masks. Note that the annotations include both stromal cells and tumour
cells.

### Network architecture and experiment setup

WSIs have a typical dimension of $\sim$ 100,000 x $\sim$ 100,000 pixels at the
highest resolution, around 0.25 $\mu$m. Forwarding even a single slide through
any CNN would be nearly impossible for modern GPU as it would require a
computational power and memory higher than any of the current hardware, unless
using specialized end-to-end approaches[9] [10]. For segmentation, a patch-
based approach is typically preferred due to its simplicity and because
generally only local patch-level information is required to solve the task.

The best resolution at which to sample patches is often unknown a-priori, thus
in our study, we compared the performances of the same model architecture over
three different patch sets sampled at three different resolutions,
respectively: 3.89$\mu$m, 7.78$\mu$m and 15.56$\mu$m. Patches of shape 256x256
pixels were sampled with equal probability from normal tissue and cancerous
tissue in both the training and validation sets. We augmented the sampled
training patches for every experiment by applying flipping (both horizontal
and vertical), blurring, modification of HSV channels, contrast and
brightness.

We used a U-Net[11] model provided by the open-source Segmentation library[12]
as our segmentation model. We selected a five-layers efficientnet-b0
structure, pre-trained on ImageNet, as the backbone for the encoder and a
five-layers decoder with skip-connection for reconstructing the output. For
each experiment, we trained the U-Net for 30 epochs and used Adam optimizer
(learning rate =$1e-5$) with a mini-batch of 16 patches. A Dice loss was used
during training. During inference, we divide the test WSIs into sequential
patches sampled at the exact resolution the model was trained and pass it
through the network to obtain the cancer probability map then binarized with a
threshold of 0.5 for generating the segmentation of the tumour.

For the multi-task network, we added a single neuron acting as a binary
classifier to the last layer (latent space) of the encode. For each patch
passed through the network, there would then be two outputs: a segmentation
mask and a label. To assign a ground truth label for the patch, we tested two
different criteria: in the first (Seg + Cl1), the patch is given label 1
(cancer) if at least 51% of its pixels belong to the tumour class, while in
the latter (Seg +Cl2) a patch is considered cancer if there is at least 1
pixel belonging to the tumour class in it.
To optimize this small binary classifier we used the Binary Cross-Entropy
(BCE) loss. Thus, the total loss function used for training the multi-task
network was the arithmetic sum of the segmentation loss and the classification
loss, therefore the Dice loss function and the Binary Cross-Entropy loss. As
reported in Table 1, the single-task network achieved higher median dice when
trained at lower resolution, therefore we trained the multi-task network with
samples extracted at 15.56$\mu$m and for 100 epochs. The reason why the
previous experiments were done with 30 epochs of training is that we noticed a
plateau after that amount of epochs, while adding this classification task
allowed us to train the network longer without overfitting. During inference,
we rely on the classification layer to produce the output mask. If a patch is
classified as positive, so containing cancer, then the product of the
segmentation branch is kept, while if a patch is classified as negative, the
result of the segmentation branch is masked with an array full of zeros. This
ideally should lead to a reduction of false positive patches.

### Results

Figure 2: Boxplots of the network over the three different resolutions and multi-task network on the positive cases. Table 1: Median and Median positive Dice for the three resolutions and over the Segmentation + Classification Task. Experiment | Med. Dice (IQR) | Med. Pos. Dice (IQR)
---|---|---
3.89$\mu$m | 0.831 (0.151) | 0.846 (0.110)
7.78$\mu$m | 0.848 (0.149) | 0.867 (0.124)
15.56$\mu$m | 0.874 (0.160) | 0.885 (0.122)
Seg + Cl 1 | 0.926 (0.143) | 0.934 (0.077)
Seg + Cl 2 | 0.871 (0.129) | 0.893 (0.117)

We evaluated the performances of our models calculating the Dice coefficient
on the validation images per each fold. Figure 2 shows the boxplots of the
experiments over the five-fold cross-validation. When lowering the resolution
at which to extract patches, the model has a higher median Dice coefficient
for segmenting the tumour. This might be due to the fact that for pancreatic
cancer segmentation architectural features of the whole tissue are more
important than cellular features. Therefore, to achieve good performance is
not needed to increase the resolution. Table 1 reports the median Dice
coefficient with inter-quartile range calculated for the whole dataset (58
cases) and the positive cases only (51). From the table is possible to see
that adding the classification branch to the network does improve the results,
but this is dependant on the labeling criteria assigned. Indeed, with the
second labeling criteria (Seg + Cl 2) nearly no additional information is
given to the single-task network, and this might be due to the fact that this
might be a really permissive labeling criteria, as a patch is given label 1 if
there’s at least one pixel belonging to the tumour class in its ground truth
patch. Thus, results are comparable with the single-task network, while on the
other hand, when using the first labeling criteria (Seg + Cl 1), the network
actually feels the effect of the more restrictive criteria. the Figure 2 shows
some outliers with Dice of 0. Most of them are the negative cases, for which
the model generated false positive patches, but for the multi-task network and
the lower resolution approach, despite having the overall best results there
is also a missed positive case, for which the segmentation mask was negative,
therefore for that case as well the Dice was set to 0.

### Discussion

Figure 3: Example case containing PDAC. Tumor segmentation is overlayed in
yellow. 3.89$\mu$m (Dice: 0.926), 7.78$\mu$m (Dice: 0.960), 15.56$\mu$m (Dice:
0.959), Seg+Cl1 (Dice: 0.978), Seg+Cl2 (Dice: 0.967) Figure 4: Example case
containing PDAC. Tumor segmentation is overlayed in yellow. 3.89$\mu$m (Dice:
0.823), 7.78$\mu$m (Dice: 0.849), 15.56$\mu$m (Dice: 0.939), Seg+Cl1 (Dice:
0.947), Seg+Cl2 (Dice: 0.901) Figure 5: Example case without PDAC. The lower
resolution networks show fewer false positives both for the single- and multi-
task networks. Where small false positive patches are generated, the Dice is
set to 0. Figure 6: Example case without PDAC. The lower resolution networks
show fewer false positives both for the single- and multi-task networks. Where
small false positive patches are generated, the Dice is set to 0.

Figure 3 and Figure 4 compare the segmentation masks produced on two positive
cases by the model trained on the three resolutions and the model trained on
the lower resolution in the multi-task settings. For the first case, the
models obtained a high Dice score for all the experiments, whereas in the
second case the models obtained a Dice score within the inter-quartile range.
Figure 3 and Figure 4 show, the lower the resolution, the better the network
can segment the edges of the tumour. This is because at higher resolutions,
the patch contains less spatial information; thus, the model would nearly-
always deal with patches completely positive or completely negative, while at
lower resolution this happens less frequently, resulting in the edges of the
segmented tumour being better defined. The network also is forced to focus
more on architectural aspects of the tissue instead of cellular features,
which might be more important in PDAC. The addition of the multi-task loss
shows a further improvement in median Dice and a reduction in inter-quartile
range. The multi-task network and the network trained at lower resolution miss
classify a single positive case. This might due to the composition of the
tissue itself which looks very different from the others in the dataset, thus
at higher resolution it was miss classified, while detected (with false
positive patches) at higher resolution.

In PDAC-negative cases the networks trained at higher resolutions usually
produce more false-positive patches, while training the network at lower
resolution decreases the number of false-positive patches (Figure 5 and Figure
6). In the multi-task setting this is even further improved. In general, the
percentage of miss-classified tissue for the seven negative cases in our
dataset varies from 3.2$\pm$ 1.66% for the model trained at 3.89$\mu$m to 0.60
$\pm$ 0.57% for the best multi-task configuration.
This work can be considered as the first step for more detailed tissue
analysis. Once the main tumour area is accurately segmented, discrimination on
the tumour microenviroenment (TME) components, such as epithelial cells and
stromal cells, can be done, yielding to possible prognostic markers such as
Tumour to stroma ratio (TSR).
Our work does have several limitations. Due to the small dataset size we
needed to perform five-fold cross-validation to adequately train the networks.
In future work we want to expand the dataset, include data from external
sources and analyse whether the improvement provided by the multi-task network
is statistically significant. Additionally, we did not explore more
sophisticated classification branches for the multi-task network or different
weights for the classification loss, which could help further improve
performance. Last, due to the fact that we only have coarse tumor annotation
we cannot separate between different tumour components, such as stroma, immune
cells and tumor cells, which will be important for extracting relevant
pathomics features. Summarizing, we have shown that standard segmentation
architectures can segment PDAC accurately, even at low-resolution.
Furthermore, in a multi-task setting performance is further improved.

###### Acknowledgements.

This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement No 101016851, project
PANCAIM.

## References

  * [1] Rawla, P., Sunkara, T., and Gaduputi, V., “Epidemiology of pancreatic cancer: global trends, etiology and risk factors,” World journal of oncology 10(1), 10 (2019).
  * [2] McGuigan, A., Kelly, P., Turkington, R. C., Jones, C., Coleman, H. G., and McCain, R. S., “Pancreatic cancer: A review of clinical diagnosis, epidemiology, treatment and outcomes,” World journal of gastroenterology 24(43), 4846 (2018).
  * [3] Sung, H., Ferlay, J., Siegel, R. L., Laversanne, M., Soerjomataram, I., Jemal, A., and Bray, F., “Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries,” CA: a cancer journal for clinicians 71(3), 209–249 (2021).
  * [4] Siegel, R. L., Miller, K. D., and Jemal, A., “Cancer statistics, 2020,” Ca-a Cancer Journal for Clinicians 70(1), 7–30 (2020).
  * [5] van Rijthoven, M., Balkenhol, M., Siliņa, K., van der Laak, J., and Ciompi, F., “Hooknet: Multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images,” Medical Image Analysis 68, 101890 (2021).
  * [6] Bulten, W., Pinckaers, H., van Boven, H., Vink, R., de Bel, T., van Ginneken, B., van der Laak, J., Hulsbergen-van de Kaa, C., and Litjens, G., “Automated deep-learning system for gleason grading of prostate cancer using biopsies: a diagnostic study,” The Lancet Oncology 21, 233–241 (Feb 2020).
  * [7] Fu, H., Mi, W., Pan, B., Guo, Y., Li, J., Xu, R., Zheng, J., Zou, C., Zhang, T., Liang, Z., et al., “Automatic pancreatic ductal adenocarcinoma detection in whole slide images using deep convolutional neural networks,” Frontiers in oncology 11, 2464 (2021).
  * [8] Bándi, P., Balkenhol, M., van Ginneken, B., van der Laak, J., and Litjens, G., “Resolution-agnostic tissue segmentation in whole-slide histopathology images with convolutional neural networks,” PeerJ 7, e8242 (2019).
  * [9] Pinckaers, H., van Ginneken, B., and Litjens, G., “Streaming convolutional neural networks for end-to-end learning with multi-megapixel images,” arXiv preprint arXiv:1911.04432 (2019).
  * [10] Tellez, D., Litjens, G., van der Laak, J., and Ciompi, F., “Neural image compression for gigapixel histopathology image analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence 43(2), 567–578 (2021).
  * [11] Ronneberger, O., Fischer, P., and Brox, T., “U-net: Convolutional networks for biomedical image segmentation,” in [International Conference on Medical image computing and computer-assisted intervention ], 234–241, Springer (2015).
  * [12] Yakubovskiy, P., “Segmentation models pytorch.” https://github.com/qubvel/segmentation_models.pytorch (2020).