# On convergence of intrinsic volumes of Riemannian manifolds.

Semyon Alesker 111Partially supported by ISF grant 865/16 and the US - Israel
BSF grant 2018115.
Department of Mathematics, Tel Aviv University, Ramat Aviv
69978 Tel Aviv, Israel
e-mail<EMAIL_ADDRESS>
###### Abstract

Let $\pi\colon M\longrightarrow B$ be a Riemannian submersion of two compact
smooth Riemannian manifolds, $B$ is connected. Let $M(\varepsilon)$ denote the
manifold $M$ equipped with the new Riemannian metric which is obtained from
the original one by multiplying by $\varepsilon$ along the vertical subspaces
(i.e. along the fibers) and keeping unchanged along the (orthogonal to them)
horizontal subspaces. Let $V_{i}(M(\varepsilon))$ denote the $i$th intrinsic
volume. The main result of this note says that
$\lim_{\varepsilon\longrightarrow+0}V_{i}(M(\varepsilon))=\chi(Z)V_{i}(B)$
where $\chi(Z)$ denotes the Euler characteristic of a fiber of $\pi$.

##  1 Introduction.

  1. 1

Intrinsic volumes of convex subsets of a Euclidean space have been studied in
convexity for a long time, see e.g. §4.1 of Schneider’s book [21]. In
dimension 2 and 3 this notion goes back to Steiner (1819). They can be defined
using the Steiner formula as follows. Let $\mathbb{R}^{n}$ denote the
Euclidean space. Let $B\subset\mathbb{R}^{n}$ denote the unit Euclidean ball.
Let $K\subset\mathbb{R}^{n}$ be an arbitrary convex compact subset. Set
$K_{\varepsilon}$ denotes $\varepsilon$-neighborhood of $K$. The Steiner
formula says that volume of $K_{\varepsilon}$ is a polynomial of degree $n$ in
$\varepsilon\geq 0$:

$\displaystyle
vol(K_{\varepsilon})=\sum_{i=0}^{n}\varepsilon^{n-i}\kappa_{n-i}V_{i}(K),$
(1.1)

where $\kappa_{n-i}$ are normalizing constants (see [21] for an explicit
expression). For $i=0,1,\dots,n$ the coefficient $V_{i}(K)$ is called the
$i$th intrinsic volume of $K$. Note that $V_{n}(K)$ equals the volume of $K$
while $V_{0}(K)=1$. Among important properties of intrinsic volumes of convex
bodies let us mention their invariance under isometries of $\mathbb{R}^{n}$,
continuity in the Hausdorff metric, and the valuation property (i.e. they are
finitely additive measures on convex compact sets); see [21] for details.
Intrinsic volumes are distinguished by the famous Hadwiger’s characterization
theorem [16] saying that a functional on all convex compact subsets of
$\mathbb{R}^{n}$ is a linear combination of intrinsic volumes if and only if
it is invariant under Euclidean isometries, continuous in the Hausdorff
metric, and is a valuation. Besides that, intrinsic volumes satisfy the
Alexandrov-Fenchel inequality (see Ch. 7 in [21]).

  2. 2

In 1939 H. Weyl [22] has extended the definition of intrinsic volumes to
smooth closed Riemannian manifolds. In this context intrinsic volumes are
often called Lipschitz-Killing curvatures. This notion is central for the
present paper. The definition is based on a non-trivial result of him stated
below. However we will state it in greater generality of manifolds with
boundary or, still more generally, simplicial corners. Such an extension seems
to be a folklore, and this generality seems to be more natural to us.

Let $(M^{n},g)$ be a compact smooth Riemannian manifold, possibly with
simplicial corners (e.g. with boundary). By the Nash imbedding theorem there
exists a smooth isometric imbedding $\iota\colon
M\hookrightarrow\mathbb{R}^{L}$ into a Euclidean space of sufficiently high
dimension. The Euclidean volume of the $\varepsilon$-neighborhood of the image
of $\iota$ turns out to be polynomial for small $\varepsilon\geq 0$:

$vol_{L}(\iota
M)_{\varepsilon}=\varepsilon^{L-n}\sum_{i=0}^{n}c_{L,n,i}\varepsilon^{n-i}V_{i}(M),$

where $c_{L,n,i}$ are appropriately normalized coefficients independent of $M$
and $\iota$. The Weyl theorem says that the coefficients $V_{i}(M)$ are
independent of the imbedding $\iota$. Weyl himself proved it for closed
manifolds; he gave an explicit formula (see below) in terms of the Riemann
curvature tensor. The case of boundary and corners seems to be known for a
long time, but I do not have a classical reference. But this is a special case
of Theorem 3.11 in a recent paper [15]. This definition is obviously
consistent with the definition for convex sets from the previous paragraph
when convex sets have smooth boundary.

Note that $V_{n}(M^{n})$ equals to the Riemannian volume of $M$, $V_{0}(M)$
equals to the Euler characteristic of $M$ by the generalized Gauss-Bonnet
formula discussed below. For a closed $M$ one has $V_{i}(M^{n})=0$ provided
$n-i$ is odd. Moreover for closed $M$ the intrinsic volume $V_{n-2}(M)$ is
proportional to the integral of the scalar curvature
$\int_{M}Sc(x)dvol_{M}(x)$.

  3. 3

As we have mentioned, an important property of intrinsic volumes on convex
sets is continuity in the Hausdorff metric. The convergence properties of
intrinsic volumes of manifolds seem to be rare in literature. The earliest
known to me result in this direction is due to Cheeger, Müller, and Schrader
[13] who proved a convergence of intrinsic volumes for polyhedral
approximations of smooth Riemannian manifolds.

A more recent example is the work in progress by Lebedeva and Petrunin [19]
where it is claimed that for a non-collapsing sequence $\\{M_{i}^{n}\\}$ of
closed Riemannian manifolds with uniformly bounded below sectional curvature
converging in the Gromov-Hausdorff sense to an Alexandrov space always there
exists $\lim_{i\longrightarrow\infty}V_{n-2}(M^{n}_{i})$.

The author (based on discussions with A. Petrunin [20]) has formulated a few
conjectures [6] on the behavior of intrinsic volumes under the Gromov-
Hausdorff convergence of Riemannian manifolds, and more generally Alexandrov
spaces, with sectional curvature uniformly bounded below. The main result of
the present paper verifies one of these conjectures in a very special case. In
fact in this case the assumption of uniform lower boundedness of sectional
curvature is unnecessary.

  4. 4

Let us formulate the main result.

######  1.1 Definition.

A smooth map $\pi\colon M\longrightarrow B$ of two smooth Riemannian manifolds
is called a Riemannian submersion if for any $o\in M$ the differential

$d\pi_{o}\colon T_{o}M\longrightarrow T_{\pi(o)}B$

being restricted to the orthogonal complement of its kernel
$(Ker(d\pi_{o}))^{\perp}$ is an isometry.

Note that $Ker(d\pi_{o})$ is the tangent space $T_{o}Z_{o}$ of the fiber
$Z_{o}:=\pi^{-1}\pi(o)$.

Let $\pi\colon M\longrightarrow B$ be a Riemannian submersion. Let us change
the original Riemannian metric $g$ on $M$ as follows. Fix any $o\in M$. We
have the orthogonal with respect to $g$ decomposition of the tangent space

$T_{o}M=T_{o}Z_{o}\oplus(T_{o}Z_{o})^{\perp}.$

For $\varepsilon>0$ let us denote by $g(\varepsilon)$ the metric which is
equal to $\varepsilon g$ on $T_{o}Z_{o}$, $g$ on $(T_{o}Z_{o})^{\perp}$, and
the two subspaces are still orthogonal to each other with respect to
$g(\varepsilon)$. We denote by $M(\varepsilon)$ the manifold $M$ equipped with
the metric $g(\varepsilon)$. The main result is

######  1.2 Theorem.

Let $\pi\colon M\longrightarrow B$ be a Riemannian submersion of smooth
compact manifolds. Let us assume that $B$ is connected. Let $Z$ denote a fiber
of $\pi$. Then for any $i\geq 0$

$\lim_{\varepsilon\longrightarrow+0}V_{i}(M(\varepsilon))=\chi(Z)V_{i}(B),$

where $\chi(Z)$ denotes the Euler characteristic of $Z$.

Just to compare with the conjectures [6], we show in Proposition 5.1 that
under the assumptions of Theorem 1.2 the manifolds $M(\varepsilon)$ have
uniform lower bound on sectional curvature when $\varepsilon\longrightarrow+0$
if and only if all the fibers of $\pi$ have non-negative sectional curvature.

  5. 5

Intrinsic volumes on convex sets and on manifolds play an important role in
valuations theory on convex sets [5], [21] and its generalizations to
manifolds [1]-[4], [15]. They play a key role in integral geometry [8], [9],
[17].

Recently there were constructed analogues of intrinsic volumes in pseudo-
Riemannian geometry and obtained non-trivial results for them [10], [11],
[12].

  6. 6

Acknowledgement. Part of this work was done during my sabbatical stay at the
Kent State University in the academic year 2018/19. I am grateful to this
institution for hospitality.

##  2 Reminder on Weyl’s and generalized Gauss-Bonnet formulas.

First let us remind the H. Weyl [22] formula for intrinsic volumes of a closed
smooth Riemannian manifold $M^{n}$. Let $R_{abcd}$ denote the Riemann
curvature tensor for a Riemannian metric $g_{ab}$. Let us denote

$R^{pq}_{rs}:=g^{pa}g^{qb}R_{abrs},$

where $(g^{ab})$ denotes the inverse matrix of $(g_{ab})$, and we use the
summation convention over repeated indices.

H. Weyl [22] has shown that for a closed manifold $M^{n}$ for odd $e$ one has
$V_{n-e}(M^{n})=0$, and for even $e$ one has

$\displaystyle
V_{n-e}(M^{n})=(2\pi)^{-e/2}\int_{M}\sum_{[p,q]}sgn(p,q)R^{q_{1}q_{2}}_{p_{1}p_{2}}\dots
R^{q_{e-1}q_{e}}_{p_{e-1}p_{e}}dvol_{M},$ (2.1)

where $dvol_{M}$ is the Riemannian volume form and the sum runs over all
couplings $[p,q]$. Here the numbers $p_{1},\dots,p_{e}$ are distinct numbers
between 1 and $n$, and $q_{1},\dots,q_{e}$ are any permutation of the
$p_{i}$’s. The sign $sgn(p,q)$ is the sign of the permutation taking
$p_{1},\dots,p_{e}$ to $q_{1},\dots,q_{e}$. The coupling $[p,q]$ is a
partition of pairs

$\displaystyle\begin{array}[]{c|c|c}(p_{1}p_{2})&(p_{3}p_{4})&\dots\\\
(q_{1}q_{2})&(q_{3}q_{4})&\dots\end{array}.$

The pairs $(p_{1}p_{2})$ etc. of two distinct numbers is unordered; due to
symmetries of the curvature tensor and the term $sgn(p,q)$ the switch of the
order of $p_{1}$ and $p_{2}$ does not change the summand. The coupling is also
unchanged under any permutation of any of $e/2$ columns of it.

In 1943 Allendoerfer and Weil [7] have shown that $V_{0}(M)$ defined by the
formula (2.1) (with $e=n$, $n$ is even) coincides with the Euler
characteristic $\chi(M)$ thus obtaining a generalization of the Gauss-Bonnet
formula. Actually they got a more general result for compact manifolds with
simplicial corners. In 1944 Chern [14] published a simpler proof of this fact
for the case of closed manifolds.

##  3 A few technical lemmas.

Let $\pi\colon M\longrightarrow B$ be a Riemannian submersion of compact
manifolds. We fix a point $o\in M$. Define $o^{*}:=\pi(o)$. There exists a
neighborhood $U$ of $o^{*}$ such that $\pi^{-1}(U)$ is diffeomorphic to
$Z\times U$ and under this identification $\pi$ is the obvious projection
$Z\times U\longrightarrow U$ where $Z$ is a fiber of $\pi$. In all local
computations throughout this paper we will fix an arbitrary such a
diffeomorphism.

We fix coordinates $\\{x^{\alpha}\\}$ near $o^{*}$. Let us denote
$Z_{o^{*}}:=\pi^{-1}(o^{*})$. Let us fix coordinates $x^{i}$ on $Z_{o^{*}}$
near $o$. Throughout this paper we denote by Greek letters
$\alpha,\beta,\gamma,\delta\dots$ the coordinates on an open subset $U\subset
B$, by Latin letters $i,j,k,l,m,n$ coordinates on the fiber $Z_{o^{*}}$, and
by Latin letters $p,q,r,s,t$ coordinates of either kind on $Z\times U$. Let
$(g(\varepsilon))$ the metric on $M$ as in Theorem 1.2.

######  3.1 Lemma.

At any point $o\in M$ we have

$\displaystyle(g(\varepsilon))=\left[\begin{array}[]{cc}\varepsilon\cdot(g_{ij})&\varepsilon\cdot(g_{i\alpha})\\\
\varepsilon\cdot(g_{\alpha i})&g_{\alpha\beta}\end{array}\right].$

Proof. At any point $o\in M$ we denote by $(\cdot,\cdot)$ the scalar product
on $T_{0}M$ for the original metric $g$, and by $(\cdot,\cdot)_{\varepsilon}$
for the metric $g(\varepsilon)$.

First it is obvious that $g(\varepsilon)_{ij}=\varepsilon\cdot g_{ij}$. Next
in the orthogonal complement to the tangent space at $o$ to the fiber there
exists a unique basis $\\{\xi_{\alpha}\\}$ such that
$d\pi(\xi_{\alpha})=\partial_{\alpha}$ where $\partial_{\alpha}$ are the
obvious vector fields corresponding to chosen coordinate $x^{\alpha}$. Clearly

$\displaystyle
g(\varepsilon)_{\alpha\beta}=(\partial_{\alpha},\partial_{\beta})_{\varepsilon}=(\xi_{\alpha},\xi_{\beta})_{\varepsilon}=(\xi_{\alpha},\xi_{\beta})=g_{\alpha\beta},$
(3.2)

where the second equality follows from the Riemannian submersion property of
$\pi$.

It remains to prove $g(\varepsilon)_{i\alpha}=\varepsilon g_{i\alpha}$. Let us
denote by $(\tilde{g}^{ij})$ the matrix inverse to $(g_{ij})$ (note that both
are matrices of size $N\times N$ where $N$ is the dimension of the fiber of
$\pi$). First let us express $\xi_{\alpha}$ via $\partial_{p}$’s. Since
$d\pi(\xi_{\alpha})=\partial_{\alpha}$ we have

$\xi_{\alpha}=\partial_{\alpha}+h^{i}_{\alpha}\partial_{i},$

where $h^{i}_{\alpha}$ are some coefficients. Then

$\displaystyle
0=(\xi_{\alpha},\partial_{j})=(\partial_{\alpha},\partial_{j})+h^{i}_{\alpha}(\partial_{i},\partial_{j})=g_{\alpha
j}+h^{i}_{\alpha}g_{ij}.$

Consequently $h^{i}_{\alpha}=-g_{\alpha j}\tilde{g}^{ji}.$ Hence

$\displaystyle\xi_{\alpha}=\partial_{\alpha}-g_{\alpha
j}\tilde{g}^{ji}\partial_{i}.$ (3.3)

Using this we have

$\displaystyle
g(\varepsilon)_{i\alpha}=(\partial_{i},\partial_{\alpha})_{\varepsilon}=(\partial_{i},\xi_{\alpha}+g_{\alpha
j}\tilde{g}^{jk}\partial_{k})_{\varepsilon}=g_{\alpha
j}\tilde{g}^{jk}(\partial_{i},\partial_{k})_{\varepsilon}=$
$\displaystyle\varepsilon g_{\alpha j}\tilde{g}^{jk}g_{ik}=\varepsilon
g_{\alpha i}=\varepsilon g_{i\alpha}.$

Q.E.D.

######  3.2 Lemma.

In the above notation the Riemannian metric on $B$ at $o^{*}=\pi(o)$ is equal
to $(g_{\alpha\beta})$.

Proof immediately follows from 3.2. Q.E.D.

######  3.3 Lemma.

Let us choose the normal coordinate system near $o^{*}=\pi(o)$, (in particular
$\partial_{\gamma}g_{\alpha\beta}|_{o}=0$). Then as
$\varepsilon\longrightarrow+0$ one has

$\displaystyle\partial_{p}g_{qr}(\varepsilon)|_{o}=O(\varepsilon),$
$\displaystyle\Gamma_{pqr}(\varepsilon)|_{o}=O(\varepsilon).$

Proof. Obviously the second part follows from the first one since

$\Gamma_{pqr}(\varepsilon)=\frac{1}{2}(\partial_{r}g_{pq}(\varepsilon)+\partial_{q}g_{pr}(\varepsilon)-\partial_{p}g_{qr}(\varepsilon)).$

To prove the first part observe that if at least one of $q,r$ is $i,j,...$
then $g_{qr}(\varepsilon)=\varepsilon g_{qr}$, and the result follows in this
case. In the remaining case $q=\alpha,r=\beta$ we have

$\partial_{p}g_{\alpha\beta}(\varepsilon)|_{o}=\partial_{p}g_{\alpha\beta}|_{o}.$

By Lemma 3.2 $g_{\alpha\beta}$ is constant along the fibers of $\pi$. Hence
$\partial_{i}g_{\alpha\beta}|_{o}=0$. By the choice of normal coordinate
system near $o^{*}$ we have $\partial_{\gamma}g_{\alpha\beta}|_{o}=0$. Q.E.D.

######  3.4 Lemma.

Let

$\displaystyle G=\left[\begin{array}[]{cc}A&B\\\ C&D\end{array}\right]$

be a block matrix with $A,D$ invertible square matrices of not necessarily the
same size. Then for $\varepsilon\longrightarrow 0$ one has

$\displaystyle\left[\begin{array}[]{cc}\varepsilon A&\varepsilon B\\\
\varepsilon
C&D\end{array}\right]^{-1}=\left[\begin{array}[]{cc}\varepsilon^{-1}A^{-1}+O(1)&O(1)\\\
O(1)&D^{-1}+O(\varepsilon)\end{array}\right]$

while the left hand side is automatically invertible for small
$\varepsilon>0$.

Proof. We have

$\displaystyle\left[\begin{array}[]{cc}\varepsilon A&\varepsilon B\\\
\varepsilon C&D\end{array}\right]=\left[\begin{array}[]{cc}A&0\\\
0&D\end{array}\right]\left[\begin{array}[]{cc}\varepsilon I&\varepsilon X\\\
\varepsilon Y&I\end{array}\right],$

where $X=A^{-1}B$, $Y=D^{-1}C$. Hence it suffices to prove the lemma for
$A=I,D=I$. It is easy to see that

$\displaystyle\left[\begin{array}[]{cc}\varepsilon I&\varepsilon X\\\
\varepsilon Y&I\end{array}\right]=\left[\begin{array}[]{cc}I&\varepsilon X\\\
0&I\end{array}\right]\left[\begin{array}[]{cc}\varepsilon(I-\varepsilon
XY)&0\\\ 0&I\end{array}\right]\left[\begin{array}[]{cc}I&0\\\ \varepsilon
Y&I\end{array}\right].$

Taking the inverse matrix and computing the inverse of each term we finally
obtain

$\displaystyle\left[\begin{array}[]{cc}\varepsilon I&\varepsilon X\\\
\varepsilon
Y&I\end{array}\right]^{-1}=\left[\begin{array}[]{cc}\varepsilon^{-1}(I-\varepsilon
XY)^{-1}&-(I-\varepsilon XY)^{-1}X\\\ -Y(I-\varepsilon XY)^{-1}&I+\varepsilon
Y(I-\varepsilon XY)^{-1}X\end{array}\right].$

The lemma follows. Q.E.D.

######  3.5 Lemma.

At any point $o\in M$ one has

$\displaystyle R_{\alpha\beta\gamma\delta}(\varepsilon)=O(1),$ (3.9)
$\displaystyle R_{ipqr}(\varepsilon)=O(\varepsilon).$ (3.10)

where the estimates in $O(\varepsilon),O(1)$ depend on $||g_{pq}||_{C^{2}}$
only.

######  3.6 Remark.

The lemma says that if at least one of the indices is not Greek then
$R(\varepsilon)=(\varepsilon)$, otherwise it is just bounded. Indeed this
follows from the symmetries of the Riemann curvature tensor.

Proof. Let us choose near $o^{*}=\pi(o)$ normal coordinates. There is a known
formula for the Riemann curvature tensor (see e.g. formula (92.1) in [18]):

$\displaystyle 2R_{pqrs}(\varepsilon)=$ (3.11)
$\displaystyle\partial_{q}\partial_{r}g_{ps}(\varepsilon)+\partial_{p}\partial_{s}g_{qr}(\varepsilon)-\partial_{q}\partial_{s}g_{pr}(\varepsilon)-\partial_{p}\partial_{r}g_{qs}(\varepsilon)+2(\Gamma_{tqr}(\varepsilon)\Gamma^{t}_{ps}(\varepsilon)-\Gamma_{tqs}(\varepsilon)\Gamma^{t}_{pr}(\varepsilon)).$
(3.12)

Let us show that each of the two $\Gamma\Gamma$ summands is $O(\varepsilon)$.
Let us estimate the first one, the second one is estimated similarly. We have

$\displaystyle\Gamma_{tqr}(\varepsilon)\Gamma^{t}_{ps}(\varepsilon)=\Gamma_{tqr}(\varepsilon)g^{tt^{\prime}}(\varepsilon)\Gamma_{t^{\prime}ps}(\varepsilon)=O(\varepsilon),$

where we have used that by Lemma 3.3 $\Gamma(\varepsilon)=O(\varepsilon)$ and
by Lemmas 3.1 and 3.4 $g^{tt^{\prime}}(\varepsilon)=O(\varepsilon^{-1})$.

If at least one of $p,q$ is $i,j,k,l$ then
$g_{pq}(\varepsilon)=O(\varepsilon)$ and all of its derivatives. Hence in
(3.12) the only contributions which may not be $O(\varepsilon)$ have the form
$\partial_{p}\partial_{q}g_{\alpha\beta}(\varepsilon)$. By Lemma 3.2 (with
$(g)$ replaced with $(g(\varepsilon))$) we have

$\partial_{i}(g_{\alpha\beta}(\varepsilon))=\partial_{i}g_{\alpha\beta}=0.$

Differentiating by $\partial_{p}$ (recall that $p$ corresponds to any
coordinate on $M$) we get

$\partial_{p}\partial_{i}(g_{\alpha\beta}(\varepsilon))=0.$

This implies the lemma. Q.E.D.

##  4 Proof of Theorem 1.2.

Let us apply (2.1) to $(M,g(\varepsilon))=:M(\varepsilon)$. We denote by $N$
the dimension of the fibers of $\pi$. We have

$\displaystyle
V_{n-e}(M(\varepsilon))=(2\pi)^{-e/2}\varepsilon^{N/2}\int_{M}\sum_{[p,q]}sgn(p,q)R^{q_{1}q_{2}}_{p_{1}p_{2}}(\varepsilon)\dots
R^{q_{e-1}q_{e}}_{p_{e-1}p_{e}}(\varepsilon)dvol_{M},$ (4.1)

where $dvol_{M}$ is the Riemannian volume form corresponding to the original
metric $g$ (rather than $g(\varepsilon)$). Indeed it is easy to see that for
the metric $g(\varepsilon)$ the volume form is equal to
$dvol_{M(\varepsilon)}=\varepsilon^{N/2}dvol_{M}$.

######  4.1 Theorem.

For a Riemannian submersion $\pi\colon M^{n}\longrightarrow B$ of closed
Riemannian manifolds

$\lim_{\varepsilon\longrightarrow+0}V_{i}(M(\varepsilon))=\chi(Z)\cdot
V_{i}(B),$

where $Z$ is a fiber of $\pi$.

Proof. If $e$ is odd then $V_{n-e}(M(\varepsilon))=0$ for all $\varepsilon>0$.
If $\dim B-n+e$ is odd then $V_{n-e}(B)=0$, and the theorem follows when both
$e$ and $\dim B-n+e$ are odd. If $e$ is odd and $\dim B-n+e$ is even then the
dimension $N=n-\dim B$ of a fiber $Z$ is odd and hence $\chi(Z)=0$. Thus the
result follows in this case too.

Thus let us assume that $e$ is even and consider $V_{n-e}(M(\varepsilon))$
defined by (4.1). By Lemmas 3.5 and 3.4 we have uniformly on $M$

$\displaystyle R^{\alpha\beta}_{\gamma\delta}(\varepsilon)=O(1),$
$\displaystyle R^{\alpha\beta}_{ip}(\varepsilon)=O(\varepsilon),$
$\displaystyle R^{i\alpha}_{pq}(\varepsilon)=O(1),$ $\displaystyle
R^{ij}_{pq}(\varepsilon)=O(\varepsilon^{-1}).$

Let us consider a summand $R^{q_{1}q_{2}}_{p_{1}p_{2}}(\varepsilon)\dots
R^{q_{e-1}q_{e}}_{p_{e-1}p_{e}}(\varepsilon)$ in (4.1). Let us denote by
$\tau$ the number of terms of the form $R^{ij}_{pq}(\varepsilon)$, and by
$\omega$ the number of terms of the form $R^{\alpha\beta}_{ip}(\varepsilon)$.
Then the above formulas imply that uniformly on $M$

$R^{q_{1}q_{2}}_{p_{1}p_{2}}(\varepsilon)\dots
R^{q_{e-1}q_{e}}_{p_{e-1}p_{e}}(\varepsilon)=O(\varepsilon^{-\tau+\omega}).$

Since all $p_{1},\dots,p_{e}$ are pairwise distinct we have $\tau\leq N/2$.
Hence uniformly on $M$

$\varepsilon^{N/2}R^{q_{1}q_{2}}_{p_{1}p_{2}}(\varepsilon)\dots
R^{q_{e-1}q_{e}}_{p_{e-1}p_{e}}(\varepsilon)=O(\varepsilon^{N/2-\tau+\omega}).$

Thus the following claim holds.

######  4.2 Claim.

The expression $\varepsilon^{N/2}R^{q_{1}q_{2}}_{p_{1}p_{2}}(\varepsilon)\dots
R^{q_{e-1}q_{e}}_{p_{e-1}p_{e}}(\varepsilon)=O(\varepsilon^{N/2-\tau+\omega})$
(1) is always uniformly bounded on $M$ as $\varepsilon\longrightarrow+0$;
(2) vanishes in the limit when either $\tau<N/2$ or $\omega>0$.

Hence it suffices to consider the summands with $\tau=N/2,\omega=0$
simultaneously. In particular $N$ must be even for such summands to exist.
Thus if $N$ is odd then
$\lim_{\varepsilon\longrightarrow+0}V_{n-e}(M(\varepsilon))=0$. At the same
time $\chi(Z)=0$ and the theorem follows for odd $N$.

Let us assume that $N$ is even. Consider all the summands with $\tau=N/2$ and
$\omega=0$. For any of them all the indices corresponding to coordinates of
the fiber $Z$ appear among upper indices of the terms of the form
$R^{ij}_{pq}(\varepsilon)$, and there are no more such indices to appear in
terms of the form $R^{i\alpha}_{pq}(\varepsilon)$. In other words if
$\tau=N/2$ there are no terms of the form $R^{i\alpha}_{pq}(\varepsilon)$.
Hence the only terms which appear are
$R^{\alpha\beta}_{\gamma\delta}(\varepsilon)$ and $R^{ij}_{pq}(\varepsilon)$.
Furthermore since all lower indices are a permutation of upper indices, the
total numbers of Greek upper and lower indices should be the same. Hence the
term $R^{ij}_{pq}(\varepsilon)$ should be $R^{ij}_{kl}(\varepsilon)$. Hence

$\displaystyle(\star):=\lim_{\varepsilon\longrightarrow+0}\varepsilon^{N/2}\sum_{[p,q]}sgn(p,q)R^{q_{1}q_{2}}_{p_{1}p_{2}}(\varepsilon)\dots
R^{q_{e-1}q_{e}}_{p_{e-1}p_{e}}(\varepsilon)=$
$\displaystyle\lim_{\varepsilon\longrightarrow+0}\varepsilon^{N/2}\left(\sum_{[\alpha,\beta]}sgn(\alpha,\beta)R^{\alpha_{1}\alpha_{2}}_{\beta_{1}\beta_{2}}(\varepsilon)\dots
R^{\alpha_{e-N-1}\alpha_{e-N}}_{\beta_{e-N-1}\beta_{e-N}}(\varepsilon)\right)\cdot$
$\displaystyle\left(\sum_{[i,j]}sgn(i,j)R^{i_{1}i_{2}}_{j_{1}j_{2}}(\varepsilon)\dots
R^{i_{N-1}i_{N}}_{j_{N-1}j_{N}}(\varepsilon)\right).$

We claim that this limit exists and

$\displaystyle(\star)=\left(\sum_{[\alpha,\beta]}sgn(\alpha,\beta)\hat{R}^{\alpha_{1}\alpha_{2}}_{\beta_{1}\beta_{2}}\dots\hat{R}^{\alpha_{e-N-1}\alpha_{e-N}}_{\beta_{e-N-1}\beta_{e-N}}\right)\cdot\left(\sum_{[i,j]}sgn(i,j)\tilde{R}^{i_{1}i_{2}}_{j_{1}j_{2}}\dots\tilde{R}^{i_{N-1}i_{N}}_{j_{N-1}j_{N}}\right)$
(4.2)

where $\hat{R}^{\alpha\beta}_{\gamma\delta}$ denotes the curvature tensor of
$B$ at the point $o^{*}$, and $\tilde{R}^{mn}_{kl}$ denotes the curvature
tensor of the fiber $Z=\pi^{-1}\pi(o)$ at $o$. To prove that it suffices to
prove that following two statements:

$\displaystyle\lim_{\varepsilon\longrightarrow+0}R^{\alpha\beta}_{\gamma\delta}(\varepsilon)=\hat{R}^{\alpha\beta}_{\gamma\delta},$
(4.3) $\displaystyle\lim_{\varepsilon\longrightarrow+0}\varepsilon
R^{mn}_{kl}(\varepsilon)=\tilde{R}^{mn}_{kl}.$ (4.4)

For let us chose normal coordinates near $o^{*}$. First let us prove (4.3). By
(3.11) one has at the point $o$

$\displaystyle 2R^{\alpha\beta}_{\gamma\delta}(\varepsilon)=g^{\alpha
p}(\varepsilon)g^{\beta q}(\varepsilon)(\partial_{\gamma}\partial_{q}g_{\delta
p}(\varepsilon)+\partial_{\delta}\partial_{p}g_{\gamma
q}(\varepsilon)-\partial_{\gamma}\partial_{p}g_{\delta
q}(\varepsilon)-\partial_{\delta}\partial_{q}g_{\gamma p}(\varepsilon)+$ (4.5)
$\displaystyle 2(\Gamma_{t\delta
p}(\varepsilon)g^{ts}(\varepsilon)\Gamma_{s\gamma
q}(\varepsilon)-\Gamma_{t\delta
q}(\varepsilon)g^{ts}(\varepsilon)\Gamma_{s\gamma p}(\varepsilon))).$ (4.6)

By Lemma 3.3 $\Gamma(\varepsilon)=O(\varepsilon)$. By Lemmas 3.1 and 3.4
$g^{st}(\varepsilon)=O(\varepsilon^{-1})$. Hence the expression (4.6) tends to
0 when $\varepsilon\longrightarrow+0$. By Lemma 3.1 $g_{\alpha
i}(\varepsilon)$ and all of its derivatives are proportional to $\varepsilon$.
By Lemma 3.4 $g^{\alpha p}(\varepsilon)=O(1)$. Consequently in (4.5) the
summands with indices $p,q$ corresponding to coordinates of the fiber
$Z=\pi^{-1}\pi o$ tend to 0. Hence

$\displaystyle
2R^{\alpha\beta}_{\gamma\delta}(\varepsilon)=g^{\alpha\mu}(\varepsilon)g^{\beta\nu}(\varepsilon)(\partial_{\gamma}\partial_{\nu}g_{\delta\mu}(\varepsilon)+\partial_{\delta}\partial_{\mu}g_{\gamma\nu}(\varepsilon)-\partial_{\gamma}\partial_{\mu}g_{\delta\nu}(\varepsilon)-\partial_{\delta}\partial_{\nu}g_{\gamma\mu}(\varepsilon))+O(\varepsilon)$

Let us denote by $\hat{g}_{\alpha\beta}$ the Riemannian metric of the base $B$
at the point $o^{*}$. By Lemma 3.1
$g_{\alpha\beta}(\varepsilon)|_{x}=\hat{g}_{\alpha\beta}|_{\pi(x)}$ for any
$x$ in a neighborhood of $o$ and for any small $\varepsilon$. Hence
$\partial_{\gamma}\partial_{\delta}g_{\alpha\beta}(\varepsilon)|_{o}=\partial_{\gamma}\partial_{\delta}\hat{g}_{\alpha\beta}|_{o^{*}}$.
By Lemma 3.4
$g^{\alpha\beta}(\varepsilon)=\hat{g}^{\alpha\beta}+O(\varepsilon)$. Hence at
the point $o$ one has

$\displaystyle
2R^{\alpha\beta}_{\gamma\delta}(\varepsilon)=\hat{g}^{\alpha\mu}\hat{g}^{\beta\nu}(\partial_{\gamma}\partial_{\nu}\hat{g}_{\delta\mu}+\partial_{\delta}\partial_{\mu}\hat{g}_{\gamma\nu}-\partial_{\gamma}\partial_{\mu}\hat{g}_{\delta\nu}-\partial_{\delta}\partial_{\nu}\hat{g}_{\gamma\mu})+O(\varepsilon)$

Since at $o^{*}$ we have chosen normal coordinates the Christoffel symbols of
$\hat{g}$ vanish at $o^{*}$. By (3.11) the right hand side of the last
equality can be rewritten

$2R^{\alpha\beta}_{\gamma\delta}(\varepsilon)=2\hat{R}^{\alpha\beta}_{\gamma\delta}+O(\varepsilon).$

Thus (4.3) follows. Let us now prove (4.4). By (3.11) we have (below as usual
$m,n,k,l$ denote indices of coordinates along the fiber $Z$, and $p,q,r,s,t$
denote arbitrary indices)

$\displaystyle\varepsilon 2R^{mn}_{kl}(\varepsilon)=\varepsilon
g^{mp}(\varepsilon)g^{nq}(\varepsilon)(\partial_{l}\partial_{p}g_{kq}(\varepsilon)+\partial_{k}\partial_{q}g_{lp}(\varepsilon)-\partial_{k}\partial_{p}g_{lq}(\varepsilon)-\partial_{l}\partial_{q}g_{kp}(\varepsilon)+$
(4.7) $\displaystyle
2\left[\Gamma_{tlp}(\varepsilon)g^{ts}(\varepsilon)\Gamma_{skq}(\varepsilon)-\Gamma_{tkp}(\varepsilon)g^{ts}(\varepsilon)\Gamma_{slq}(\varepsilon)\right]).$
(4.8)

By Lemma 3.1 each of the first summands in the round brackets is proportional
to $\varepsilon$, e.g.

$\displaystyle g_{kq}(\varepsilon)=\varepsilon g_{kq}.$ (4.9)

By Lemmas 3.1, 3.4, and 3.3

$\displaystyle
g^{mp}(\varepsilon)=O(\varepsilon^{-1}),\,\,\,\Gamma(\varepsilon)=O(\varepsilon).$

It follows that one opens brackets in (4.7)-(4.8) then each summand is $O(1)$.
Moreover if one of the terms in a summand has better estimate (e.g.
$g^{m\alpha}(\varepsilon)=O(1)$ by Lemma 3.4) then the corresponding term is
$O(\varepsilon)$. Hence we may keep in (4.7) only terms of the form
$g^{mi}(\varepsilon)$ (i.e. both indices $m,i$ correspond to coordinates on
$Z$). For the same reason $g^{st}(\varepsilon)$ in (4.8) can be replaced with
$g^{i^{\prime}j^{\prime}}(\varepsilon)$ (i.e. summation is only over the
indices corresponding to the fiber $Z$). Again by Lemma 3.4

$g^{mi}(\varepsilon)=\varepsilon^{-1}(g|_{Z})^{mi}+O(1).$

Substituting all that into (4.7)-(4.8) we get

$\displaystyle\varepsilon
2R^{mn}_{kl}(\varepsilon)=\varepsilon^{-1}(g|_{Z})^{mi}(g|_{Z})^{nj}(\varepsilon(\partial_{l}\partial_{i}g_{kj}+\partial_{k}\partial_{j}g_{li}-\partial_{k}\partial_{i}g_{lj}-\partial_{l}\partial_{j}g_{ki})+$
$\displaystyle
2\varepsilon^{-1}\left[\Gamma_{i^{\prime}li}(\varepsilon)(g|_{Z})^{i^{\prime}j^{\prime}}\Gamma_{j^{\prime}kj}(\varepsilon)-\Gamma_{i^{\prime}ki}(\varepsilon)(g|_{Z})^{i^{\prime}j^{\prime}}\Gamma_{j^{\prime}lj}(\varepsilon)\right])+O(\varepsilon).$

In the last formula all the indices correspond to coordinates of the fiber
$Z$. For them obviously
$\Gamma_{i^{\prime}ki}(\varepsilon)=\varepsilon\Gamma_{i^{\prime}ki}$ where
$\Gamma_{i^{\prime}ki}$ is the Christoffel symbol of the original metric on
$M$. Hence

$\displaystyle\varepsilon
2R^{mn}_{kl}(\varepsilon)=(g|_{Z})^{mi}(g|_{Z})^{nj}(\partial_{l}\partial_{i}g_{kj}+\partial_{k}\partial_{j}g_{li}-\partial_{k}\partial_{i}g_{lj}-\partial_{l}\partial_{j}g_{ki})+$
$\displaystyle
2\left[\Gamma_{i^{\prime}li}(g|_{Z})^{i^{\prime}j^{\prime}}\Gamma_{j^{\prime}kj}-\Gamma_{i^{\prime}ki}(g|_{Z})^{i^{\prime}j^{\prime}}\Gamma_{j^{\prime}lj}\right])+O(\varepsilon)=$
$\displaystyle 2\tilde{R}^{mn}_{ij}+O(\varepsilon).$

Clearly statement (4.4) follows. Consequently (4.2) is proven as well.

Let us prove the theorem now. By (4.1)

$V_{n-e}(M(\varepsilon))=(2\pi)^{-e/2}\int_{M}\varepsilon^{N/2}\sum_{[p,q]}sgn(p,q)R^{q_{1}q_{2}}_{p_{1}p_{2}}(\varepsilon)\dots
R^{q_{e-1}q_{e}}_{p_{e-1}p_{e}}(\varepsilon)dvol_{M}$

By Claim 4.2(1) the expression under the integral is uniformly bounded on $M$.
By (4.2) and the Lebesgue convergence theorem

$\displaystyle\lim_{\varepsilon\longrightarrow+0}V_{n-e}(M(\varepsilon))=$
$\displaystyle(2\pi)^{-e/2}\int_{M}\left(\sum_{[\alpha,\beta]}sgn(\alpha,\beta)\hat{R}^{\alpha_{1}\alpha_{2}}_{\beta_{1}\beta_{2}}\dots\hat{R}^{\alpha_{e-N-1}\alpha_{e-N}}_{\beta_{e-N-1}\beta_{e-N}}\right)|_{o}\cdot$
$\displaystyle\left(\sum_{[i,j]}sgn(i,j)\tilde{R}^{i_{1}i_{2}}_{j_{1}j_{2}}\dots\tilde{R}^{i_{N-1}i_{N}}_{j_{N-1}j_{N}}\right)|_{\pi(o)}dvol_{M}(o),$

where, as previously, $\hat{R}^{\alpha\beta}_{\gamma\delta}|_{o}$ denotes the
curvature of the fiber $\pi^{-1}\pi(o)$ at the point $o$, and
$\tilde{R}^{ij}_{kl}|_{\pi(o)}$ denotes the curvature of the base $B$ at the
point $\pi(o)$.

For $p^{*}\in B$ we denote by $Z_{p^{*}}:=\pi^{-1}(p^{*})$, and by
$dvol_{Z_{p^{*}}}$ the volume form on $Z_{p^{*}}$ with respect to the
restriction of the original Riemannian metric $g$. Integrating along the
fibers we have

$\displaystyle\lim_{\varepsilon\longrightarrow+0}V_{n-e}(M(\varepsilon))=$
$\displaystyle(2\pi)^{-\frac{e-N}{2}}\int_{B}dvol_{B}(p^{*})\left(\sum_{[\alpha,\beta]}sgn(\alpha,\beta)\hat{R}^{\alpha_{1}\alpha_{2}}_{\beta_{1}\beta_{2}}\dots\hat{R}^{\alpha_{e-N-1}\alpha_{e-N}}_{\beta_{e-N-1}\beta_{e-N}}\right)|_{p^{*}}\cdot$
$\displaystyle(2\pi)^{-\frac{N}{2}}\int_{p\in
Z_{p^{*}}}dvol_{Z_{p^{*}}}(p)\left(\sum_{[i,j]}sgn(i,j)\tilde{R}^{i_{1}i_{2}}_{j_{1}j_{2}}\dots\tilde{R}^{i_{N-1}i_{N}}_{j_{N-1}j_{N}}\right)\big{|}_{p}$

The inner integral is equal to $\chi(Z_{p^{*}})$ by the generalized Gauss-
Bonnet formula. Hence

$\displaystyle\lim_{\varepsilon\longrightarrow+0}V_{n-e}(M(\varepsilon))=$
$\displaystyle\chi(Z)\cdot(2\pi)^{-\frac{e-N}{2}}\int_{B}dvol_{B}(p^{*})\left(\sum_{[\alpha,\beta]}sgn(\alpha,\beta)\hat{R}^{\alpha_{1}\alpha_{2}}_{\beta_{1}\beta_{2}}\dots\hat{R}^{\alpha_{e-N-1}\alpha_{e-N}}_{\beta_{e-N-1}\beta_{e-N}}\right)\big{|}_{p^{*}}=$
$\displaystyle\chi(Z)V_{n-e}(B),$

where the last equality is by the definition of $V_{n-e}$. Q.E.D.

##  5 When $M(\varepsilon)$ has uniform lower bound on sectional curvature?

######  5.1 Proposition.

The sectional curvature of $M(\varepsilon)$ stays uniformly bounded below as
$\varepsilon\longrightarrow+0$ if and only if the sectional curvature of all
fibers of $\pi$ is non-negative.

Proof. Below we use the convention of lifting and lowering indices using the
matrix $(g_{pq})$ of size $\dim M$. For $p\neq q$ let us denote by
$K_{pq}(\varepsilon)$ the Gauss curvature of $g(\varepsilon)$ of the
$(x^{p}x^{q})$-plane. By the known formula

$K_{pq}(\varepsilon)=\frac{2R_{pqpq}(\varepsilon)}{g_{pp}(\varepsilon)g_{qq}(\varepsilon)-g_{pq}^{2}(\varepsilon)}.$

Case 1. Assume $p=\alpha,q=\beta$. By Lemma 3.1

$\displaystyle
K_{\alpha\beta}(\varepsilon)=\frac{2R_{\alpha\beta\alpha\beta}(\varepsilon)}{g_{\alpha\alpha}g_{\beta\beta}-g_{\alpha\beta}^{2}}\overset{(\ref{E:curv2})}{=}\frac{O(1)}{g_{\alpha\alpha}g_{\beta\beta}-g_{\alpha\beta}^{2}}=O(1).$

Case 2. Assume $p=\alpha,q=i$. Similarly by Lemma 3.1

$\displaystyle K_{\alpha i}(\varepsilon)=\frac{2R_{\alpha i\alpha
i}(\varepsilon)}{g_{\alpha\alpha}g_{ii}-g_{\alpha
i}^{2}}\overset{(\ref{E:curv1})}{=}\frac{O(\varepsilon)}{\varepsilon
g_{\alpha\alpha}g_{ii}-\varepsilon^{2}g_{\alpha i}^{2}}=O(1).$

Case 3. Assume $p=i,q=j$. Again

$\displaystyle
K_{ij}(\varepsilon)=\frac{2R_{ijij}(\varepsilon)}{\varepsilon^{2}(g_{ii}g_{jj}-g_{ij}^{2})}.$
(5.1)

We have

$\displaystyle 2R_{ijij}(\varepsilon)\overset{(\ref{E:RC-formula})}{=}$ (5.2)
$\displaystyle
2\partial_{i}\partial_{j}g_{ij}(\varepsilon)-\partial_{i}\partial_{i}g_{jj}(\varepsilon)-\partial_{j}\partial_{j}g_{ii}(\varepsilon)+2(\Gamma_{tij}(\varepsilon)\Gamma^{t}_{ij}(\varepsilon)-\Gamma_{tii}(\varepsilon)\Gamma^{t}_{jj}(\varepsilon)).$
(5.3)

The $\Gamma\Gamma$ summand can be rewritten more explicitly

$\displaystyle\Gamma_{tij}(\varepsilon)\Gamma_{sij}(\varepsilon)g^{st}(\varepsilon)-\Gamma_{tii}(\varepsilon)\Gamma_{sjj}(\varepsilon)g^{st}(\varepsilon).$

Using Lemma 3.4 for the matrix $G=g(\varepsilon)$ the last expression is equal

$\displaystyle\Gamma_{kij}(\varepsilon)\Gamma_{lij}(\varepsilon)(\tilde{g}^{kl}(\varepsilon)+O(1))-\Gamma_{kii}(\varepsilon)\Gamma_{ljj}(\varepsilon)(\tilde{g}^{kl}(\varepsilon)+O(1)).$
(5.4)

We may choose furthermore the coordinates near $o^{*}$ to be normal. Then by
Lemma 3.3 the expression (5.4) is equal to

$\displaystyle\varepsilon(\Gamma_{kij}\Gamma_{lij}\tilde{g}^{kl}-\Gamma_{kii}\Gamma_{ljj}\tilde{g}^{kl})+O(\varepsilon^{2}).$

Substituting this back to (5.3) instead of the $\Gamma\Gamma$ term we get

$\displaystyle 2R_{ijij}(\varepsilon)=$ (5.5)
$\displaystyle\varepsilon(2\partial_{i}\partial_{j}g_{ij}-\partial_{i}\partial_{i}g_{jj}-\partial_{j}\partial_{j}g_{ii})+\varepsilon(\Gamma_{kij}\Gamma_{lij}\tilde{g}^{kl}-\Gamma_{kii}\Gamma_{ljj}\tilde{g}^{kl})+O(\varepsilon^{2}).$
(5.6)

Substituting this into (5.1) we get

$\displaystyle
K_{ij}(\varepsilon)=\frac{\varepsilon(2\partial_{i}\partial_{j}g_{ij}-\partial_{i}\partial_{i}g_{jj}-\partial_{j}\partial_{j}g_{ii}+\Gamma_{kij}\Gamma_{lij}\tilde{g}^{kl}-\Gamma_{kii}\Gamma_{ljj}\tilde{g}^{kl})+O(\varepsilon^{2})}{\varepsilon^{2}(g_{ii}g_{jj}-g_{ij}^{2})}=$
$\displaystyle\varepsilon^{-1}\frac{2\tilde{R}_{ijij}}{g_{ii}g_{jj}-g_{ij}^{2}}+O(1)=\varepsilon^{-1}\tilde{K}_{ij}+O(1),$

where the last summand $O(1)$ is uniform on $M$, and $\tilde{K}_{ij}$ denotes
the sectional curvature of the fiber at $o$. We see that when
$\varepsilon\longrightarrow+0$ the sectional curvature $K_{ij}(\varepsilon)$
is uniformly on $M$ bounded below if and only if $\tilde{K}_{ij}\geq 0$.
Q.E.D.

## References

  * [1] Alesker, Semyon; Theory of valuations on manifolds. I. Linear spaces. Israel J. Math. 156 (2006), 311–339.
  * [2] Alesker, Semyon; Theory of valuations on manifolds. II. Adv. Math. 207 (2006), no. 1, 420–454.
  * [3] Alesker, Semyon; Fu, Joseph H. G.; Theory of valuations on manifolds. III. Multiplicative structure in the general case. Trans. Amer. Math. Soc. 360 (2008), no. 4, 1951–1981.
  * [4] Alesker, Semyon; Theory of valuations on manifolds. IV. New properties of the multiplicative structure. Geometric aspects of functional analysis, 1–44, Lecture Notes in Math., 1910, Springer, Berlin, 2007.
  * [5] Alesker, Semyon; Introduction to the theory of valuations. CBMS Regional Conference Series in Mathematics, 126. Published for the Conference Board of the Mathematical Sciences, Washington, DC; by the American Mathematical Society, Providence, RI, 2018.
  * [6] Alesker, Semyon; Some conjectures on intrinsic volumes of Riemannian manifolds and Alexandrov spaces. Arnold Math. J. 4 (2018), no. 1, 1–17.
  * [7] Allendoerfer, Carl B.; Weil, André; The Gauss-Bonnet theorem for Riemannian polyhedra. Trans. Amer. Math. Soc. 53 (1943), 101–129.
  * [8] Bernig, Andreas; Fu, Joseph H. G.; Hermitian integral geometry. Ann. of Math. (2) 173 (2011), no. 2, 907–945.
  * [9] Bernig, Andreas; Fu, Joseph H. G.; Solanes, Gil; Integral geometry of complex space forms. Geom. Funct. Anal. 24 (2014), no. 2, 403–492.
  * [10] Bernig, Andreas; Faifman, Dmitry; Valuation theory of indefinite orthogonal groups. J. Funct. Anal. 273 (2017), no. 6, 2167–2247.
  * [11] Bernig, Andreas; Faifman, Dmitry; Solanes, Gil; Curvature Measures of Pseudo-Riemannian Manifolds. arXiv:1910.09635
  * [12] Bernig, Andreas; Faifman, Dmitry; Solanes, Gil; Uniqueness of curvature measures in pseudo-Riemannian geometry. arXiv:2009.02230
  * [13] Cheeger, Jeff; Müller, Werner; Schrader, Robert; On the curvature of piecewise flat spaces. Comm. Math. Phys. 92 (1984), no. 3, 405–454.
  * [14] Chern, Shiing-shen; A simple intrinsic proof of the Gauss-Bonnet formula for closed Riemannian manifolds. Ann. of Math. (2) 45 (1944), 747–752.
  * [15] Fu, Joseph H. G.; Wannerer, Thomas; Riemannian curvature measures. Geom. Funct. Anal. 29 (2019), no. 2, 343–381.
  * [16] Hadwiger, Hugo; Vorlesungen über Inhalt, Oberfläche und Isoperimetrie. (German) Springer-Verlag, Berlin-Göttingen-Heidelberg 1957.
  * [17] Klain, Daniel A.; Rota, Gian-Carlo; Introduction to geometric probability. Lezioni Lincee. [Lincei Lectures] Cambridge University Press, Cambridge, 1997.
  * [18] Landau, L. D.; Lifshitz, E. M.; Course of theoretical physics, Vol. 2. The classical theory of fields. Fourth edition. Translated from the Russian by Morton Hamermesh. Pergamon Press, Oxford-New York-Toronto, Ont., 1975.
  * [19] Lebedeva, Nina; Petrunin, Anton; Curvature tensors on Alexandrov spaces. In preparation (2017).
  * [20] Petrinin, Anton; Personal discussions.
  * [21] Schneider, Rolf; Convex bodies: the Brunn-Minkowski theory. Second expanded edition. Encyclopedia of Mathematics and its Applications, 151. Cambridge University Press, Cambridge, 2014.
  * [22] Weyl, Hermann; On the Volume of Tubes. Amer. J. Math. 61 (1939), no. 2, 461-472.


# Interpretation of Intracardiac Electrograms
Through Textual Representations

William Jongwon Han<EMAIL_ADDRESS>
Carnegie Mellon University  USA  Diana Gomez<EMAIL_ADDRESS>
Carnegie Mellon University  USA  Avi Alok<EMAIL_ADDRESS>
Carnegie Mellon University  USA  Chaojing Duan<EMAIL_ADDRESS>
Allegheny Health Network  USA  Michael A. Rosenberg
<EMAIL_ADDRESS>
University of Colorado  USA  Douglas Weber<EMAIL_ADDRESS>
Carnegie Mellon University  USA  Emerson Liu<EMAIL_ADDRESS>
Allegheny Health Network  USA  Ding Zhao<EMAIL_ADDRESS>
Carnegie Mellon University  USA

###### Abstract

Understanding the irregular electrical activity of atrial fibrillation (AFib)
has been a key challenge in electrocardiography. For serious cases of AFib,
catheter ablations are performed to collect intracardiac electrograms (EGMs).
EGMs offer intricately detailed and localized electrical activity of the heart
and are an ideal modality for interpretable cardiac studies. Recent
advancements in artificial intelligence (AI) has allowed some works to utilize
deep learning frameworks to interpret EGMs during AFib. Additionally, language
models (LMs) have shown exceptional performance in being able to generalize to
unseen domains, especially in healthcare. In this study, we are the first to
leverage pretrained LMs for finetuning of EGM interpolation and AFib
classification via masked language modeling. We formulate the EGM as a textual
sequence and present competitive performances on AFib classification compared
against other representations. Lastly, we provide a comprehensive
interpretability study to provide a multi-perspective intuition of the model’s
behavior, which could greatly benefit the clinical use.

#### Data and Code Availability

We recorded intracardiac electrograms in the left atrium of two patients, one
healthy and the other afflicted with atrial fibrillation (AFib), via an
Octoray catheter from Biosense Webster Inc. The recording was done at an
anonymous hospital site. More details about the data is provided in Section 4.

#### Institutional Review Board (IRB)

Since we are using an anonymous dataset that has been collected previous to
this work and de-identified, we will not be needing an IRB.

##  1 Introduction

Atrial fibrillation (AFib) is one of the most common types of arrhythmia
globally, affecting more than 60 million people over the last thirty years
(Elliott et al., 2023). AFib is characterized by the chaotic, irregular, and
often rapid, beating pattern of the heart’s upper chambers, known as the
atria. This causes the atria to become out of sync with the heart’s lower
chambers, called the ventricles. Due to these debilitating properties, people
with AFib are at risk of stroke, heart failure, and a number of other cardiac
diseases.

Figure 1: The overall pipeline of our model. We formulate EGM signal
interpolation and AFib classification as a masked language modeling task.
denotes the mask.

Catheter ablations are performed for patients with more serious cases of AFib,
during which intracardiac electrograms (EGMs) representative of electrical
activity along the endocardial surface of the atria are collected by means of
navigational catheters embedded with multiple electrodes. Because EGMs have
distinct and spatially specific signatures for different types of cardiac
arrhythmia, they are the ideal modality for highly interpretable cardiac
studies, in particular for AFib.

Due to their locality and richness in information, there are works utilizing
deep learning methods with EGMs for AFib classification (Alhusseini et al.,
2020), patient-specific therapy (John et al., 2022), and surface
electrocardiogram (ECG) reconstruction (Zhang et al., 2021). In addition, the
representation of EGMs are carefully considered (Alhusseini et al., 2020; Tang
et al., 2022) and are most commonly represented as visual or time series
modalities (Kong et al., 2023; Zhang et al., 2021; Alhusseini et al., 2020).
Despite the impressive performances of these EGM algorithms, the model’s
interpretability is a critical component that requires further exploration
before deployment in the clinical setting. Although some works attempt to
interpret the model’s decisions, to our best knowledge, only a uniperspective
interpretability metric is provided, such as visualizing the Grad-CAM heatmap
(Alhusseini et al., 2020), which can often be misleading (Ribeiro et al.,
2016; Donoso-Guzmán et al., 2023).

In this study, we introduce a tokenization schema, inspired by Chen et al.
(2022), that represents an EGM as a textual sequence. The tokenization process
discretizes continuous EGM signal amplitudes and maps each to a unique token
ID, similarly to textual data. Representing EGM signals as a textual sequences
allows us to leverage powerful Language Models (LMs) pretrained on billions of
texts. Specifically, we use a Masked Language Model (MLM) that is optimized
for predicting randomly masked positions in a given sequence. As seen in
Figure 1, the AFib label is also tokenized input. By masking 75% of the signal
tokens and the AFib label token, the model is able to interpolate the masked
EGM values and simultaneously classify the EGM as AFib or not. Additionally,
we provide a multi-perspective interpretability procedure to expose the
intuition behind the model’s decisions at the token level.

In summary, our main contributions are the following:

  * •

To our best knowledge, this is the first work to represent EGMs as a textual
sequence. We introduce an effective tokenization schema that maintains the low
level information of the original signal.

  * •

We utilize a MLM pretrained on textual data to finetune for interpreting EGM
signals by interpolation and classification for AFib.

  * •

We also perform a comprehensive interpretability procedure via attention maps,
integrated gradients (Sundararajan et al., 2017), and counterfactual analysis
to provide clarity of the model’s decisions for clinicians.

##  2 Related Work

###  2.1 Language Models for Healthcare

Language Models (LMs) have been an extremely popular medium for many different
downstream tasks in healthcare. Qiu et al. (2023a) utilizes LMs to translate
surface ECG signals into descriptive clinical notes. Choi et al. (2023) first
cluster surface ECG signals into 70 groups and uses them to create a wave
vocabulary for pretraining a MLM. They then employ the pretrained MLM model
for various downstream tasks, such as AFib classification, heartbeat
classification, and user identification (Choi et al., 2023). There has also
been large efforts in pretraining LMs on solely clinical text, such as
Alsentzer et al. (2019) and Li et al. (2022). Mehandru et al. (2023) observed
that as the role of LMs become larger in the medical world, they can act as
agents by providing clinical decision support and stakeholder interactions.
Therefore, they propose new evaluation frameworks, termed Artificial-
intelligence Structured Clinical Examinations (AI-SCI), to assess LMs in real-
world clinical tasks (Mehandru et al., 2023).

###  2.2 Machine Learning for EGM

Machine learning in electrogram (EGM) analysis, though limited, has seen
notable advancements. Alhusseini et al. (2020) used a 64-electrode catheter to
create 8x8 spatial activation heatmaps from atrial EGM signals, feeding them
into a CNN for atrial fibrillation (AFib) classification. Zhang et al. (2021)
developed RT-RCG, a neural network tailored for ECG data reconstruction from
EGMs, incorporating a Differentiable Acceleration Search for efficient
hardware accelerator optimization. Duque et al. (2017) presented a genetic
algorithm and K-NN based classifier to categorize EGM signals, enhancing AF
ablation therapy guidance. Tang et al. (2022) employed a multimodal approach,
integrating surface ECG, EGM signals, and clinical data in a CNN, improving
AFib classification accuracy. Our study extends these works by applying LMs,
specifically MLMs, pretrained on general text to interpret complex EGM signals
in AFib cases.

###  2.3 Interpretability for Machine Learning in Healthcare

Interpretability in healthcare machine learning is crucial but challenging for
clinical use (Amann et al., 2020). Jin et al. (2021) outline four attribution
methods for machine learning interpretability: backpropagation, feature
perturbation, attention, and model distillation. Notably, Alhusseini et al.
(2020) and Vaid et al. (2022) use Grad-CAM heatmaps and attention saliency
maps, respectively, to interpret CNN and Vision Transformer models. Shi and
Norgeot (2022) advocates for interpretability in healthcare models, with
Sanchez et al. (2022) and Prosperi et al. (2020) emphasizing counterfactual
analysis and addressing biases in machine learning. Our study merges several
interpretability methods (e.g., attention, integrated gradients,
counterfactual analysis) for a more holistic understanding of model decisions.

##  3 Methods

Our method consists of three main steps: (1) translating the EGM signals from
time series to text through tokenization, (2) utilizing MLMs to predict the
masked portions of the input sequence, and (3) interpreting the decisions of
the models through attention weights, integrated gradients, and counterfactual
analysis.

###  3.1 Tokenization

Our tokenization approach, inspired by Chen et al. (2022), discretizes the
continuous amplitudes of EGM signals to create a textual representation of the
time series information. In this section, we intricately explain the process
of tokenization for a single sequence. Let
$S=\\{s_{1},s_{2},\ldots,s_{M}\\}\in\mathbb{R}^{M}$ be a discretized EGM
signal, where $M=1000$ for a one second signal.

The normalization of $S$ scales the amplitude values to [0, 1]. For the
sequence $S$ and each $s_{i}$, this is given by:

$S_{\text{norm}}=\left\\{\frac{s_{i}-s_{\text{min}}}{s_{\text{max}}-s_{\text{min}}}:s_{i}\in
S\right\\}$

where $s_{\text{min}}=\min(S)$, $s_{\text{max}}=\max(S)$, and
$S_{\text{norm}}$ is the normalized sequence.

Quantization converts $S_{\text{norm}}$ into discrete levels $V$, where
$V=250$. The quantized sequence $S_{\text{quant}}$ is:

$S_{\text{quant}}=\left\\{\left\lfloor S_{\text{norm}}\times
V\right\rfloor:s_{i}\in S_{\text{norm}}\right\\}$

where $\left\lfloor\cdot\right\rfloor$ denotes the floor function. The
decision of the value of $V$ were purposely chosen through experimentation. We
observed that $250$ was the minimum amount of discrete levels that represents
the fine-grained details of an EGM signal as well as obtain the best
performance for the signal interpolation task.

We then map $S_{\text{quant}}$ to a unique token ID and expand the tokenizer’s
embedding table by adding the new tokens. For signals, we define:

$T_{\text{S}}=\left\\{\text{``signal\\_"}+\text{str}(q):q\in
S_{\text{quant}}\right\\}$

where $\text{str}(\cdot)$ converts numbers to strings. We also create the
token IDs for the AFib label by introducing the following tokens:

$T_{\text{A}}=\left\\{\text{``afib\\_"}+\text{str}(a):a\in{0,1}\right\\}$

where 0 and 1 denotes a normal and AFib signal respectively.

###  3.2 Masked Language Model

The Masked Language Models (MLMs) we use are BigBird (Zaheer et al., 2021),
LongFormer (Beltagy et al., 2020), Clinical BigBird (Li et al., 2022), and
Clinical LongFormer (Li et al., 2022), which are optimized to handle sequences
of lengths up to 4,096, for finetuning for EGM signal interpolation and AFib
classification. Clinical BigBird and Clinical Longformer uses the same
architecture as their predecessors, however, they are pretrained on 2 million
clinical notes from the MIMIC-III dataset (Johnson et al., 2016) instead of a
general web-crawled text dataset. More details on the models are available in
the appendix.

###  3.3 Masking Strategy

To prepare our input for the MLM, we adopt a high masking strategy. We
randomly mask out $75\%$ of $T_{S}$. We choose $75\%$, as suggested in He et
al. (2021), to ensure the model learns a significant portion of the EGM
waveform and to diminish the possibility of interpolating from the surrounding
unmasked areas. We always mask out $T_{A}$ to ensure the model is not being
fed the classification label. After applying the masks, our input is
formulated as follows:

$T_{I}=[\text{CLS}]\oplus T_{S}\oplus[\text{SEP}]\oplus
T_{A}\oplus[\text{SEP}]$

where $\oplus$ is the concatenation operation, [CLS] is the classification
token indicating the start of the sequence, and [SEP] is the separator token
denoting both the end of a sequence as well as distinguishing boundaries for
different types of input (Devlin et al., 2019).

###  3.4 Learning Objectives

#### Masked Language Model Loss

Let $M$ be the set of indices of the masked tokens in $T_{I}$. For each masked
token index $i\in M$, the original token at this position is $t_{i}$. The MLM
objective is to minimize the following loss function:

$L_{MLM}(\theta)=-\sum_{i\in M}\log P(t_{i}|T_{I};\theta),$

where $\theta$ represents the parameters of the MLM model,
$P(t_{i}|T_{I};\theta)$ is the probability predicted by the model for the
token $t_{i}$ given the masked input sequence $T_{I}$, and the sum is over all
masked tokens in the sequence.

#### Atrial Fibrillation Classification Loss

We add another loss $L_{AFib}$ based on the prediction of $T_{A}$ within the
sequence $T_{I}$. The cross-entropy loss for this classification task is
computed as follows:

$L_{AFib}(\theta)=-\sum_{c=1}^{C}y_{c}\log(p_{c}),$

where $C$ is the number of classes for AFib classification, $y_{c}$ is a
binary indicator (0 or 1) if class label $c$ is the correct classification for
$T_{A}$, and $p_{c}$ is the predicted probability for the token $T_{A}$ being
of class $c$.

#### Final Learning Objective

The total loss for the model is given by:

$L(\theta)=\alpha_{1}L_{MLM}(\theta)+\alpha_{2}L_{AFib}(\theta),$

where $\alpha_{i}\in[0,1],i\in\\{1,2\\}$.

###  3.5 Model Interpretability

All of our model interpretability procedures have been conducted using the
BigBird (Zaheer et al., 2021) model with the final learning objective.

#### Attention

Visualizing the attention map is an extremely common technique for
interpretability in Transformers (Vaswani et al., 2023). There has been
arguments in the interpretability community that debate whether attention maps
are reliable or not (Jain and Wallace, 2019; Wiegreffe and Pinter, 2019). The
general consensus seems to be that although attention can be informative, it
can also be misleading if it is the only metric of interpretability (Wen et
al., 2023). Therefore, we visualize the attention maps of our models, while
providing other perspectives as well. In our study, we visualize the averaged
attention map over all heads and layers.

#### Integrated Gradients

Integrated Gradients ($IG$), introduced by (Sundararajan et al., 2017), is a
method for attributing a model’s prediction to its input features, providing
insight into the model’s behavior. Given an input $x$ and a baseline
$x^{\prime}$, $IG$ computes the gradient of the model’s output with respect to
the input along the path $\alpha$ from the baseline to the input. The
attribution of each feature $i$ is given by:

$IG_{i}(x)=(x_{i}-x^{\prime}_{i})\times\int_{\alpha=0}^{1}\frac{\partial
F(x^{\prime}+\alpha\times(x-x^{\prime}))}{\partial x_{i}}d\alpha$

where $F$ is the model function. $IG$ satisfies two key axioms: Sensitivity,
ensuring non-zero attribution for features that change the output, and
Implementation Invariance, guaranteeing consistent attributions for
functionally equivalent models. In our setting, the input $x$ is our masked,
tokenized sequence, $T_{I}$. The baseline $x^{\prime}$ is defined to be a
vector of padding tokens [PAD] of the same dimension as $T_{I}$.

#### Counterfactual Analysis

Counterfactual analysis observes the change of the predicted outcome by
directly manipulating the input (Feder et al., 2021; Pearl, 2009). We
individually conduct three different counterfactual analysis methods: Token
Substitution, Token Addition, and Label Flipping. We view the effects of the
introduced counterfactuals in two settings. The first setting is where we
finetune the MLM without the counterfactuals and inference with modified and
unmodified inputs. The second setting is where we finetune the MLM with the
counterfactuals alongside unmodified inputs, and inference on both modified
and unmodified inputs. In both settings, we randomly choose 25% of the batch
to be modified with the counterfactuals.

#### Token Substitution

Given our input signal $T_{S}$, we apply token substitution by replacing the
original sequence with a smoothed out version via a moving average filter. The
intuition behind this is aligned from the clinicians’ perspective such that
the sharp oscillations in a given signal is where most of the information is
stored for the model or clinician to determine whether it is a normal
heartbeat or AFib. By smoothing out the oscillations via a moving average
filter, we want to observe the model’s robustness against this counterfactual
as well as its attention and gradients.

#### Token Addition

Given our input signal $T_{S}$, we apply token addition by introducing the
following $V$ new augmentation signal tokens and adding them to tokenizer’s
embedding table:

$T_{\text{AUG}}=\left\\{\text{``augsig\\_"}+\text{str}(q):q\in
S_{\text{quant}}\right\\}$

Please note that $T_{\text{AUG}}$ is still derived from the same
$S_{\text{quant}}$ as $T_{S}$. We then randomly choose 25% (a 250 length
segment) of $T_{\text{AUG}}$ and append it to $T_{S}$ therefore getting the
input sequence

$T_{I}^{\prime}=[\text{CLS}]\oplus T_{S}\oplus
T_{\text{AUG}}\oplus[\text{SEP}]\oplus T_{A}\oplus[\text{SEP}]$

These augmented tokens will not be masked. The ground truth label to these
augmented tokens will be the padding token [PAD]. Essentially, the augmented
tokens will simply serve as noise to $T_{S}$ to observe whether the model
leverages the noise or still attends to the meaningful, non-augmented portion
of the signal.

#### Label Flipping

Given our masked input AFib label $T_{A}$, we apply label flipping by
replacing the ground truth label with its opposite. This is used to understand
the model’s behavior under adversarial examples.

tab:main_results

Table 1: AFib classification results with different baselines and
representations.

Method Representation Sensitivity % Specificity % PPV % NPV % Accuracy %
CatBoost (Tang et al., 2022) Time Series 88.5 62.7 - - 70.1 CNN (Alhusseini et
al., 2020) Image 97.0 93.0 93.1 97.0 95.0 K-Means (Alhusseini et al., 2020)
Image 77.0 82.3 83.5 75.5 79.4 KNN (Alhusseini et al., 2020) Image 75.3 84.0
87.1 70.3 78.9 LDA (Alhusseini et al., 2020) Image 85.0 74.6 76.4 83.7 79.7
SVM (Alhusseini et al., 2020) Image 82.9 76.7 77.4 82.3 79.7 ViT (Dosovitskiy
et al., 2021) Image 1.0 99.1 99.1 1.0 99.7 BEiT (Bao et al., 2022) Image 99.9
99.3 98.8 99.9 99.5 BigBird (Zaheer et al., 2021) Time Series 0.0 1.0 0.0 63.6
63.6 LongFormer (Beltagy et al., 2020) Time Series 0.0 1.0 0.0 63.6 63.6
BigBird (Zaheer et al., 2021) Text 97.8 96.1 93.5 98.7 96.7 LongFormer
(Beltagy et al., 2020) Text 69.0 98.4 96.1 84.7 87.7 Clinical BigBird (Li et
al., 2022) Text 12.8 97.3 72.8 66.1 66.6 Clinical LongFormer (Li et al., 2022)
Text 85.2 97.9 95.8 92.1 93.3 Ours BigBird (Zaheer et al., 2021) Text 96.7
99.8 99.6 98.2 99.2 Ours LongFormer (Beltagy et al., 2020) Text 1.0 99.4 99.0
99.8 99.5 Ours Clinical BigBird (Li et al., 2022) Text 99.0 99.0 98.1 99.4
99.0 Ours Clinical LongFormer (Li et al., 2022) Text 99.9 99.6 99.9 99.3 99.7

Figure 2: Visualization of all 65 (left) Octoray catheters and a single
Octoray Catheter (right) placements inside the left atrium.

##  4 Experiments

###  4.1 Dataset and Preprocessing

Our dataset consists of 20 different placements of catheter ablation for a
patient with normal heartbeat rhythm and 45 different placements of catheter
ablation for a patient with AFib. The catheter ablation was performed with the
Octoray catheter from Biosense Webster Inc. The catheter consists of 8 splines
with 6 electrodes on each spline, summing up to 48 electrodes. The catheter
ablation performed on the patient with AFib was for 30 seconds and for the
patient with a normal heartbeat, the ablation was for 29 seconds. Both studies
were sampled at a rate of 1000 Hz. It is important to note that our dataset is
from two patients. However, this limitation is overcome by the diversity of
catheter placement locations as seen in Fig 2. We treat each electrode on all
65 catheters placements as a separate instance. In summary, we have a total
number of $59*48*65=184,080$ unique samples we finetune on.

Let $\mathbf{X}\in\mathbb{R}^{I\times J\times K}$ represent the normal and
AFib data tensor, with dimensions $I,J$ and $K$ denoting the time length,
number of electrodes, and number of catheter placements, respectively. Z-score
normalization is applied across the recording time and electrodes for each
placement $k$. The normalized tensor $\mathbf{Z}$ is computed as:

$Z_{i,j,k}=\frac{X_{i,j,k}-\mu_{k}}{\sigma_{k}}$

where $\mu_{k}$ and $\sigma_{k}$ are the mean and standard deviation of the
elements in $\mathbf{X}$ across the first two dimensions for each $k$. We then
segment $Z_{i,j,k}$ into a sequence of non-overlapping segments of length $M$,
arriving at $S=\\{s_{1},s_{2},\ldots,s_{L}\\}$. Unless specified otherwise, we
report the results where $M=1000$.

###  4.2 Experimental Setting

We finetuned the masked language model utilizing the AdamW optimizer (Kingma
and Ba, 2017) with a learning weight of $1e-4$, weight decay rate of $1e-2$
(Loshchilov and Hutter, 2017). We conducted all experiments with a batch size
of 8 and 1 for finetuning and inference, respectively. For our final learning
objective $L$, $\alpha_{1}=\alpha_{2}=1$. Our experiments were conducted on 2
NVIDIA A6000 and 2 NVIDIA A5000 GPUs.

During inference, we evaluate our models on two tasks, namely EGM signal
interpolation and AFib classification. For EGM signal interpolation, we use
the Mean Squared Error and Mean Absolute Error. For AFib classification, we
use the sensitivity, specificity, positive predicted values (PPV), negative
predicted values (NPV), and accuracy metrics for evaluation.

##  5 Results

###  5.1 AFib Classification

We compare our results with different representations and baselines in
tab:main_results. We divide our table into three sections. The top section
contains prior works that do AFib classification with EGMs. It is important to
note that the datasets used in this top section are different from ours (i.e.,
different patients, catheter, sampling rate), therefore they are not directly
comparable. However, we decide to include them in this table to report the
current state of AFib classification using EGMs.

The middle section contains the results from different base models and
representations with our dataset. For ViT (Dosovitskiy et al., 2021) and BEiT
(Bao et al., 2022), we transform the EGMs into image representations via
Markov Transition Field (MTF), Gramian Angular Field (GAF) (Wang and Oates,
2014), and Recurrence Plot (RP) (Eckmann et al., 1987), which have proven to
be excellent image representations of time series signals (Qiu et al., 2023b).
We load in the pretrained weights of ViT and BEiT, ‘google/vit-base-
patch16-224-in21k’ and ‘microsoft/beit-base-patch16-224-pt22k’ respectively,
and apply the 75% masking rate on the images during finetuning. For the time
series representation results using LongFormer (Beltagy et al., 2020) and
BigBird (Zaheer et al., 2021), we simply utilize the normalized signal
amplitudes as input embeddings by projecting them through a linear layer
before passing them into their respective models. For the text representation
results, we report the results for only utilizing the pretrained base model
with masked language model loss.

Finally, in the bottom section of the table, we report our results for
utilizing the textual representation and full learning objective. We can
observe that our method of representing the complex EGM signal as a simple
textual sequence achieves competitive results with the image modality.

tab:inter  Method Interpolation MSE $\downarrow$ MAE $\downarrow$ BigBird
(Zaheer et al., 2021) 0.44 0.16 LongFormer (Beltagy et al., 2020) 0.37 0.13
Clinical BigBird (Li et al., 2022) 0.50 0.17 Clinical LongFormer (Li et al.,
2022) 0.36 0.13 Ours BigBird (Zaheer et al., 2021) 0.80 0.37 Ours LongFormer
(Beltagy et al., 2020) 0.40 0.14 Ours Clinical BigBird (Li et al., 2022) 0.44
0.15 Ours Clinical LongFormer (Li et al., 2022) 0.40 0.14

Table 2: Results on interpolation.

###  5.2 EGM Interpolation

We report the results of the EGM interpolation task in tab:inter. We visualize
the comparison of the reconstructed (blue) and ground truth (yellow) signal in
the appendix. From tab:inter, we can see the models with only the MLM learning
objective opposed to our final learning objective achieves slightly lower MSE
and MAE scores. We can see a clear trade-off between the interpolation results
and the AFib classification results in adding the Cross Entropy (CE) loss on
top of the MLM loss. However, this trade-off is acceptable due to the increase
in performance for AFib classification.

###  5.3 Ablation Study

#### Pretrained vs Non-Pretrained

We observe the effects of utilizing pretrained vs non-pretrained versions of
the MLM in tab:pt. For clarification, pretrained means we load in the
checkpoint that was pretrained on the large text corpus and conduct
finetuning. Conversely, non-pretrained means we do not load the checkpoint
that was pretrained on the text corpus and conduct finetuning. Although the
BigBird model (Zaheer et al., 2021) and LongFormer model (Beltagy et al.,
2020) were pretrained on a text corpora that was not directly related to
clinical subjects, we see that the pretrained setting greatly outperforms the
non-pretrained setting. From this, we can observe the characteristic of
pretrained models being able to leverage the language knowledge to well adapt
to unseen domains.

tab:pt  Model Pretrained Interpolation AFib Classification MSE $\downarrow$
MAE $\downarrow$ Accuracy % BigBird Zaheer et al. (2021) ✗ 5.42 1.42 63.6
BigBird Zaheer et al. (2021) ✓ 0.44 0.16 96.7 LongFormer Beltagy et al. (2020)
✗ 6.32 2.21 60.9 LongFormer Beltagy et al. (2020) ✓ 0.37 0.13 87.7

Table 3: Ablation study on pretrained (✓) vs non-pretrained (✗) MLM. Figure 3:
The averaged attention weights (yellow) and attribution scores (red) of an
AFib EGM signal during inference. ‘$T_{A}$ Masked’ refers to running inference
when only the $T_{A}$ token is masked. ‘$T_{A}$ \+ $T_{S}$ Masked’ denotes our
regular paradigm of masking out 75% of the $T_{S}$ tokens and always masking
out the $T_{A}$ token. Figure 4: The averaged attribution scores for
checkpoints that have been finetuned on their respective counterfactuals. We
also include the visualization of the average attribution score for the
checkpoint that has been finetuned without counterfactuals for comparison.

###  5.4 Interpretability

In our study, we analyzed AFib EGM signal attention maps and attribution
scores during inference (Figure 3), examining both before and after
finetuning, and under different masking conditions ($T_{A}$ Masked,
$T_{A}+T_{S}$ Masked).

Attention maps pre-finetuning showed slightly more oscillation compared to
post-finetuning, though the overall distribution remained largely unchanged in
both settings and masking conditions. Notably, the model focused more on the
start and end of sequences, a behavior attributed to its pretraining on
natural language data, where salient information is often found at these
positions. This tendency seems to have generalized to EGM data.

Regarding attribution scores, a marked difference pre- and post-finetuning was
observed in score stability. Pre-finetuning, scores were random across the
signal when masking only the $T_{A}$ token and somewhat concentrated at the
start but still randomly distributed with our original masking strategy. Post-
finetuning, the distribution changed significantly. With only the $T_{A}$
token masked, the scores peaked around signal oscillations, aligning with
clinical significance. For full masking, scores were evenly distributed across
the sequence, with notable attribution at the start, reflecting the MLM’s use
of the entire sequence for contextual understanding, including predicting the
$T_{A}$ token. We visualize more random samples of attribution scores
overlayed on both normal and AFib EGMs in the appendix.

tab:aug  Representations Counterfactuals Finetuned Interpolation AFib
Classification MSE $\downarrow$ MAE $\downarrow$ Accuracy % Image Token
Substitution ✓ 0.57 0.46 99.4 ✗ 0.72 0.55 99.0 2-6 Token Addition ✓ 1.12 0.64
99.1 ✗ 1.92 1.24 73.8 2-6 Label Flipping ✓ 2.31 1.12 74.0 ✗ 2.51 2.08 74.5
Time Series Token Substitution ✓ 11.63 6.40 37.1 ✗ 11.96 6.34 36.4 2-6 Token
Addition ✓ 8.81 5.10 54.2 ✗ 9.92 5.50 50.7 2-6 Label Flipping ✓ 7.22 4.51 56.9
✗ 10.41 9.56 56.1 Text Token Substitution ✓ 0.29 0.26 96.3 ✗ 0.25 0.25 74.7
2-6 Token Addition ✓ 0.70 0.34 99.0 ✗ 0.61 0.19 77.4 2-6 Label Flipping ✓ 0.47
0.17 93.5 ✗ 0.81 0.38 91.3

Table 4: Results for finetuning with (✓) and without (✗) Token Substitution,
Token Addition, and Label Flipping, where we highlight the best results for
each counterfactual in orange, blue, and red, respectively.

###  5.5 Counterfactual Analysis

We first analyzes model performance with three counterfactuals (Token
Substitution, Token Addition, Label Flipping) across two finetuning settings
(i.e., with/without counterfactuals) and three representations (i.e., text,
image, time series), using ViT and BigBird architectures for image and time
series respectively. Textual representations exhibit higher robustness to
counterfactuals in interpolation tasks compared to others.

Attribution scores from checkpoints finetuned on respective counterfactuals
were compared against non-counterfactual methods (Figure 4). For Token
Substitution, full masking results in typical attribution score distributions,
while partial masking ($T_{A}$ token) shows random distributions, indicating a
forced learning of a generalized signal distribution. Token Addition
demonstrates a focused attribution on oscillating signal parts when $T_{A}$ is
masked, reflecting model adaptation to identify key sequence elements amidst
noisy tokens, enhancing interpolation and AFib classification (see tab:aug).

For Label Flipping, the attribution scores under full masking align with our
standard method, indicating robustness to adversarial examples. However,
partial masking shows arbitrary peaks across the sequence, diverging from the
focused attribution of our non-counterfactual method. This suggests inherent
model resilience against incorrect labels without specific adversarial
training objectives.

##  6 Discussion and Conclusion

In this study, we demonstrated the effectiveness of using pretrained MLMs with
textual representations for EGM signal interpretation. Our approach yielded
impressive results in AFib classification and interpolation, achieving 99.7%
accuracy, with MSE of 0.40 and MAE of 0.14. This outperformed both time series
and image-based EGM signal representations. We compared the use of a general
text corpus pretrained MLM versus a non-pretrained MLM for EGM interpretation.
The pretrained model significantly outperformed the non-pretrained model in
both AFib classification and interpolation, showing a 38.8% increase in
accuracy and decreases of 5.92 in MSE and 2.07 in MAE. Our analysis included
visualizing the attention and attribution scores. While attention weights were
not insightful regarding the model’s decision-making process, the attribution
scores under full masking indicated substantial reliance on pretrained
knowledge for interpreting EGM signals. Interestingly, partial masking
revealed the model’s focus on oscillating signal parts for AFib
classification, aligning with clinical interpretations. Further, we
demonstrated our method’s robustness by finetuning it with predefined
counterfactuals, which showed superior performance over other representations.
Analyzing the attribution scores of these finetuned models revealed distinct
attribution distributions for each counterfactual. The distinct behaviors
observed in the attribution scores of these finetuned models not only confirm
the adaptability of our approach but also open avenues for future research.
Specifically, these findings encourage further exploration into optimizing the
model’s focus on critical signal segments, maintaining accuracy even when
confronted with varying counterfactual scenarios. Additionally, this work not
only contributes to the field of EGM signal interpretation via machine
learning but also sets a foundation for future studies aimed at expanding the
capabilities of LM-based approaches in EGM analysis.

#### Limitations

While our study highlights the potential of utilizing general pretrained MLMs
for interpreting complicated EGM signals during AFib, there are notable
limitations worth mentioning. The most obvious limitation of this study is the
data. Due to the invasive and complex nature of collecting intracardiac EGM
data, it is difficult to find and conduct comparable baselines. However, we
try to mitigate this limitation in our work by conducting several baseline
models with different representations.

This research is supported by the Allegheny Health Network and the Mario
Lemieux Center for Innovation and Research in EP.

## References

  * Alhusseini et al. (2020) Mahmood I. Alhusseini, Firas Abuzaid, Albert J. Rogers, Junaid A.B. Zaman, Tina Baykaner, Paul Clopton, Peter Bailis, Matei Zaharia, Paul J. Wang, Wouter-Jan Rappel, and Sanjiv M. Narayan.  Machine learning to classify intracardiac electrical patterns during atrial fibrillation.  _Circulation: Arrhythmia and Electrophysiology_ , 13, 08 2020.  10.1161/circep.119.008160.
  * Alsentzer et al. (2019) Emily Alsentzer, John R. Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew B. A. McDermott.  Publicly available clinical bert embeddings, 2019.
  * Amann et al. (2020) Julia Amann, Alessandro Blasimme, Effy Vayena, Dietmar Frey, and Vince Madai.  Explainability for artificial intelligence in healthcare: a multidisciplinary perspective.  _BMC Medical Informatics and Decision Making_ , 20, 11 2020.  10.1186/s12911-020-01332-6.
  * Bao et al. (2022) Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei.  Beit: Bert pre-training of image transformers, 2022.
  * Beltagy et al. (2020) Iz Beltagy, Matthew E. Peters, and Arman Cohan.  Longformer: The long-document transformer, 2020.
  * Chen et al. (2022) Ting Chen, Saurabh Saxena, Lala Li, David J. Fleet, and Geoffrey Hinton.  Pix2seq: A language modeling framework for object detection, 2022.
  * Choi et al. (2023) Seokmin Choi, Sajad Mousavi, Phillip Si, Haben G. Yhdego, Fatemeh Khadem, and Fatemeh Afghah.  Ecgbert: Understanding hidden language of ecgs with self-supervised representation learning, 2023.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding, 2019.
  * Donoso-Guzmán et al. (2023) Ivania Donoso-Guzmán, Jeroen Ooge, Denis Parra, and Katrien Verbert.  Towards a comprehensive human-centred evaluation framework for explainable ai, 2023.
  * Dosovitskiy et al. (2021) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale, 2021.
  * Duque et al. (2017) S.I. Duque, A. Orozco-Duque, V. Kremen, D. Novak, C. Tobón, and J. Bustamante.  Feature subset selection and classification of intracardiac electrograms during atrial fibrillation.  _Biomedical Signal Processing and Control_ , 38:182–190, 2017.  ISSN 1746-8094.  https://doi.org/10.1016/j.bspc.2017.06.005.  URL https://www.sciencedirect.com/science/article/pii/S1746809417301088.
  * Eckmann et al. (1987) Jean-Pierre Eckmann, Sylvie Kamphorst, and D. Ruelle.  Recurrence plots of dynamical systems.  _Europhysics Letters (epl)_ , 4:973–977, 11 1987.  10.1209/0295-5075/4/9/004.
  * Elliott et al. (2023) Adrian D. Elliott, Melissa E. Middeldorp, Isabelle C. Van Gelder, Christine M. Albert, and Prashanthan Sanders.  Epidemiology and modifiable risk factors for atrial fibrillation.  _Nature Reviews Cardiology_ , page 1–14, 01 2023.  10.1038/s41569-022-00820-8.  URL https://www.nature.com/articles/s41569-022-00820-8#ref-CR1.
  * Feder et al. (2021) Amir Feder, Nadav Oved, Uri Shalit, and Roi Reichart.  Causalm: Causal model explanation through counterfactual language models.  _Computational Linguistics_ , page 1–54, May 2021.  ISSN 1530-9312.  10.1162/coli_a_00404.  URL http://dx.doi.org/10.1162/coli_a_00404.
  * He et al. (2021) Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick.  Masked autoencoders are scalable vision learners, 2021.
  * Jain and Wallace (2019) Sarthak Jain and Byron C. Wallace.  Attention is not explanation, 2019.
  * Jin et al. (2021) Di Jin, Elena Sergeeva, Wei-Hung Weng, Geeticka Chauhan, and Peter Szolovits.  Explainable deep learning in healthcare: A methodological survey from an attribution view, 2021.
  * John et al. (2022) Mathews M. John, Anton Banta, Allison Post, Skylar Buchan, Behnaam Aazhang, and Mehdi Razavi.  Artificial intelligence and machine learning in cardiac electrophysiology.  _Texas Heart Institute Journal_ , 49, 03 2022.  10.14503/thij-21-7576.
  * Johnson et al. (2016) Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, Li wei H. Lehman, Mengling Feng, Mohammad Mahdi Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark.  Mimic-iii, a freely accessible critical care database.  _Scientific Data_ , 3, 2016.  URL https://api.semanticscholar.org/CorpusID:33285731.
  * Kingma and Ba (2017) Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization, 2017.
  * Kong et al. (2023) Xiangzhen Kong, Vasanth Ravikumar, Siva K. Mulpuru, Henri Roukoz, and Elena G. Tolkacheva.  A data-driven preprocessing framework for atrial fibrillation intracardiac electrocardiogram analysis.  _Entropy_ , 25(2), 2023.  ISSN 1099-4300.  10.3390/e25020332.  URL https://www.mdpi.com/1099-4300/25/2/332.
  * Li et al. (2022) Yikuan Li, Ramsey M. Wehbe, Faraz S. Ahmad, Hanyin Wang, and Yuan Luo.  Clinical-longformer and clinical-bigbird: Transformers for long clinical sequences, 2022.
  * Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  _arXiv preprint arXiv:1711.05101_ , 2017.
  * Mehandru et al. (2023) Nikita Mehandru, Brenda Y. Miao, Eduardo Rodriguez Almaraz, Madhumita Sushil, Atul J. Butte, and Ahmed Alaa.  Large language models as agents in the clinic, 2023.
  * Pearl (2009) Judea Pearl.  _Causality_.  Cambridge University Press, Cambridge, UK, 2 edition, 2009.  ISBN 978-0-521-89560-6.  10.1017/CBO9780511803161.
  * Prosperi et al. (2020) Mattia Prosperi, Yi Guo, Matt Sperrin, James S. Koopman, Jae S. Min, Xing He, Shannan Rich, Mo Wang, Iain E. Buchan, and Jiang Bian.  Causal inference and counterfactual prediction in machine learning for actionable healthcare.  _Nature Machine Intelligence_ , 2:369–375, 07 2020.  10.1038/s42256-020-0197-y.
  * Qiu et al. (2023a) Jielin Qiu, William Han, Jiacheng Zhu, Mengdi Xu, Michael Rosenberg, Emerson Liu, Douglas Weber, and Ding Zhao.  Transfer knowledge from natural language to electrocardiography: Can we detect cardiovascular disease through language models?, 2023a.
  * Qiu et al. (2023b) Jielin Qiu, Jiacheng Zhu, Shiqi Liu, William Han, Jingqi Zhang, Chaojing Duan, Michael Rosenberg, Emerson Liu, Douglas Weber, and Ding Zhao.  Automated cardiovascular record retrieval by multimodal learning between electrocardiogram and clinical report, 2023b.
  * Ribeiro et al. (2016) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  ”why should i trust you?”: Explaining the predictions of any classifier, 2016.
  * Sanchez et al. (2022) Pedro Sanchez, Jeremy P. Voisey, Tian Xia, Hannah I. Watson, Alison Q. O’Neil, and Sotirios A. Tsaftaris.  Causal machine learning for healthcare and precision medicine.  _Royal Society Open Science_ , 9, 08 2022.  10.1098/rsos.220638.
  * Shi and Norgeot (2022) Jingpu Shi and Beau Norgeot.  Learning causal effects from observational data in healthcare: A review and summary.  _Frontiers in Medicine_ , 9, 07 2022.  10.3389/fmed.2022.864882.
  * Sundararajan et al. (2017) Mukund Sundararajan, Ankur Taly, and Qiqi Yan.  Axiomatic attribution for deep networks, 2017.
  * Tang et al. (2022) Siyi Tang, Orod Razeghi, Ridhima Kapoor, Mahmood Alhusseini, Muhammad Fazal, Albert J Rogers, Miguel Rodrigo Bort, Paul Clopton, Paul J Wang, Daniel L Rubin, Sanjiv M Narayan, and Tina Baykaner.  Machine learning–enabled multimodal fusion of intra-atrial and body surface signals in prediction of atrial fibrillation ablation outcomes.  _Circulation-arrhythmia and Electrophysiology_ , 15, 08 2022.  10.1161/circep.122.010850.
  * Vaid et al. (2022) Akhil Vaid, Joy Jiang, Ashwin Sawant, Stamatios Lerakis, Edgar Argulian, Yuri Ahuja, Joshua Lampert, Alexander Charney, Hayit Greenspan, Benjamin Glicksberg, Jagat Narula, and Girish Nadkarni.  Heartbeit: Vision transformer for electrocardiogram data improves diagnostic performance at low sample sizes, 2022.
  * Vaswani et al. (2023) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need, 2023.
  * Wang and Oates (2014) Zhiguang Wang and Tim Oates.  Encoding time series as images for visual inspection and classification using tiled convolutional neural networks.  2014\.  URL https://api.semanticscholar.org/CorpusID:16409971.
  * Wen et al. (2023) Kaiyue Wen, Yuchen Li, Bingbin Liu, and Andrej Risteski.  Transformers are uninterpretable with myopic methods: a case study with bounded dyck grammars, 2023.
  * Wiegreffe and Pinter (2019) Sarah Wiegreffe and Yuval Pinter.  Attention is not not explanation, 2019.
  * Zaheer et al. (2021) Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed.  Big bird: Transformers for longer sequences, 2021.
  * Zhang et al. (2021) Yongan Zhang, Anton Banta, Yonggan Fu, Mathews M. John, Allison Post, Mehdi Razavi, Joseph Cavallaro, Behnaam Aazhang, and Yingyan Lin.  Rt-rcg: Neural network and accelerator search towards effective and real-time ecg reconstruction from intracardiac electrograms, 2021.

##  Appendix A Additional Visualizations

#### Visualization of Tokenized Sequence

We compare a three second tokenized signal when $V=\\{50,100,250\\}$ to its
original representation (far left) in Figure 6.

#### Visualization of Reconstructed EGM Signal

We visualize the reconstructed EGM signal and compare between using only the
MLM learning objective versus our full learning objective in Figure 5. The
reconstructed EGM signal using only the MLM learning objective follows the
wave morphology more closely but results in lower AFib classification results.
The reconstructed EGM signal using our full learning objective still closely
resembles the signal’s overall morphology, thus we accept this trade-off for
the improvement in AFib classification.

#### Visualization of Attribution Scores

We randomly sampled 4 normal and 4 AFib EGMs and visualize the attribution
score distributions in Figure 7. The visualizations are based off of the
finetuned checkpoint of our full learning objective with the BigBird model
(Zaheer et al., 2021). We also visualize the attribution score when the
$T_{A}$ token is only masked out, due to the clinical significance. From
Figure 7, we can observe that for normal EGMs, the attribution scores are
precisely localized around the fluctuating sections of the signal, which is
aligned with the clinician’s perspective. For AFib EGMs, we can still observe
a pattern formulating where the rise and fall of the attribution scores are
aligned with the irregularly oscillating portions as well.

##  Appendix B Details on Masked Language Models

#### BigBird

The BigBird model employs a sparse attention mechanism to efficiently process
long sequences. This approach is characterized by three types of attention
patterns: random, windowed, and global. The random pattern enables long-range
interactions modeled after the Erdős-Rényi graph. The windowed attention
captures local context within a fixed-size window, while the global tokens are
included to maintain overall sequence context. These elements are integrated
into a modified attention calculation, reducing the complexity from quadratic
to linear with respect to sequence length, thereby facilitating scalability
for datasets with long textual information. We use the pretrained weights
‘google/bigbird-roberta-base’ provided by the authors (Zaheer et al., 2021).

Figure 5: Comparison of $L_{MLM}$ and $L_{MLM}+L_{CE}$ reconstructed AFib EGM
signals with the BigBird model (Zaheer et al., 2021) using our final learning
objective.

#### LongFormer

The Longformer model is also optimized for processing long sequences of data.
Beltagy et al. (2020) introduce a sparse attention mechanism that combines
sliding window and global attention, reducing complexity from quadratic to
linear and enabling handling of up to 4096 tokens. The sliding window
attention applies standard self-attention within a fixed-size window, ensuring
local context is maintained. Global attention allows designated tokens to
interact with the entire sequence. We use the pretrained weights
‘allenai/longformer-base-4096’ provided by the authors (Beltagy et al., 2020).

#### Clinical BigBird and LongFormer

Clinical BigBird and Clinical LongFormer utilize the same architectures as
BigBird and LongFormer, respectively. The difference among these two models is
that they are pretrained on 2 million clinical notes from the MIMIC-III
dataset (Johnson et al., 2016). We use the pretrained weights
‘yikuan8/Clinical-BigBird’ and ‘yikuan8/Clinical-Longformer’ for Clinical
BigBird and Clinical LongFormer respectively, provided by the authors (Li et
al., 2022).

tab:sig_len  $M$ Interpolation AFib Classification MSE $\downarrow$ MAE
$\downarrow$ Accuracy % 4 0.88 0.40 98.3 3 0.92 0.42 99.3 2 1.18 1.04 97.4 1
0.80 0.37 99.2

Table 5: Ablation Study on Signal Length $M$

##  Appendix C Additional Experiments

#### Only MLM vs MLM + Classification head

We compare our method with a traditional formulation of doing classification
with a MLM. The traditional formulation is that given some input, in this case
$[\text{CLS}]\oplus T_{S}\oplus[\text{SEP}]$, to the MLM, the output of it
will be inputted into an additional linear layer for classification. In our
traditional formulation, we first finetune the MLM for the interpolation task
then further finetune for AFib classification with an added linear layer. In
tab:mlm_lin, we can see that there is no performance difference between the
two methods. Therefore, we highlight the efficiency of our method since we
only require the MLM to be finetuned once.

tab:mlm_lin  Method AFib Classification Accuracy % BigBird (Zaheer et al.,
2021) \+ Linear Layer 98.7 Ours BigBird (Zaheer et al., 2021) 98.7

Table 6: Ablation study on MLM vs MLM + classification head Figure 6: The
tokenized representation of an EGM signal for a patient with a normal
heartbeat (Top) and AFib (Bottom) when $V=\\{50,100,250\\}$ levels from left
to right and starting with the the original time series EGM representation.
Figure 7: We visualize the attribution scores of the finetuned checkpoint of
our full learning objective using the BigBird model (Zaheer et al., 2021). We
visualize randomly sampled normal and AFib EGMs with only masking the $T_{A}$
token for clinical significance.

#### Performance on Varying Signal Length $M$

We compare the performance of our model when we vary the signal length $M$ in
tab:sig_len, using the BigBird model (Zaheer et al., 2021) with our final
learning objective. We observe that for interpolation, $M=1$ gives the best
results, whereas for AFib classification, $M=3$ does slightly better. In
general, our model seems to be robust to varying signal length inputs.