# On the existence of global solutions for the $3$d chemorepulsion system

Tomasz Cieślak  Institute of Mathematics, Polish Academy of Sciences,
Śniadeckich 8, 00-656 Warsaw, Poland<EMAIL_ADDRESS>,  Mario Fuest  Leibniz
Universität Hannover, Institut für Angewandte Mathematik, Welfengarten 1,
30167 Hannover, Germany<EMAIL_ADDRESS>,  Karol Hajduk  Institute
of Mathematics, Polish Academy of Sciences, Śniadeckich 8, 00-656 Warsaw,
Poland<EMAIL_ADDRESS>and  MikoŁaj Sierżega  Faculty of Mathematics,
Informatics and Mechanics, University of Warsaw, Banacha 2, 02-097 Warsaw,
Poland<EMAIL_ADDRESS>
###### Abstract.

In this paper, we give sufficient conditions for global-in-time existence of
classical solutions for the fully parabolic chemorepulsion system posed on a
convex, bounded three-dimensional domain. Our main result establishes global-
in-time existence of regular nonnegative solutions provided that
$\nabla\sqrt{u}\in L^{4}(0,T;L^{2}(\Omega))$. Our method is related to the
Bakry–Émery calculation and appears to be new in this context.

###### Key words and phrases:

boundedness of solutions, chemotaxis, chemorepulsion, Lyapunov-like
functional, Bernis-type inequality

###### 2010 Mathematics Subject Classification:

Primary 35B45, 35K45; Secondary 92C17

##  1\. Introduction

In this paper, we study the problem of global existence of solutions of the
fully parabolic chemorepulsion system. The two-dimensional case was solved in
[5]. Unlike in the more-widely known chemoattraction case, $2$D chemorepulsion
leads to the global-in-time existence of classical solutions regardless of the
size of the initial data. The question of global existence in three and higher
dimensions remains open. In the present paper, we look into the $3$D case and
establish a conditional global regularity result for this model. First, we
introduce the model.

Let $\Omega\subset{\mathbb{R}}^{n}$ be an open, bounded domain with a
sufficiently smooth boundary. We consider the following fully parabolic
chemorepulsion system

(1.1) $\left\\{\begin{split}\partial_{t}u&=\nabla\cdot(\nabla u+u\nabla v)\\\
\partial_{t}v&=\Delta
v-v+u\end{split}\right.\qquad\mbox{in}\qquad(0,\infty)\times\Omega,$

with homogeneous Neumann boundary conditions (no-flux through the boundary)

(1.2) $\displaystyle\left.\frac{\partial
u}{\partial\nu}\right\rvert_{\partial\Omega}=0\qquad\&\qquad\left.\frac{\partial
v}{\partial\nu}\right\rvert_{\partial\Omega}=0,$

where $\nu$ is the unit outward normal to the boundary, and with nonnegative
initial conditions

(1.3) $\displaystyle u(0,x)=u_{0}(x)\geq 0\qquad\&\qquad v(0,x)=v_{0}(x)\geq
0.$

The functions $u$ and $v$ describe densities of some living organisms and of a
chemical substance which repels them, respectively. The ’$+$‘ sign on the
right-hand-side of the first equation in $(\ref{CRu})$ corresponds to the
repulsion mechanism. The opposite phenomenon appears in the widely studied
chemoattraction case described by the Keller–Segel system. For an overview of
results for such systems, we refer to the surveys [2, 12]. The latter survey
contains a chapter concerning the construction of solutions, including the
irregular ones, to the fully parabolic chemorepulsion system.

Three and higher-dimensional cases of the chemorepulsion system are still far
from being understood. While we know that global weak solutions exist (see
[5]), it is unclear whether regular bounded solutions exist for all $t>0$.
Some results concerning the perturbation of the parabolic-elliptic case are
known, see [4]. Moreover, for the problem with nonlinear, sufficiently strong
diffusion, see the global existence result in [7]. Similarly, it is known that
nonlinear, sufficiently weak chemorepulsion leads to global-in-time solutions,
see [17]. However, the main basic problem lacks a definitive answer. In the
present note we establish a conditional result.

We emphasise that our result is rather of methodological meaning. On the one
hand, our method applied to the 3D chemorepulsion yields only a conditional
result. Moreover, as communicated to us by M. Winkler and one of the referees,
this result can be improved, see Appendix B. Indeed, the result in Appendix B
covers our Thereom 1.1 and is applicable also in non-convex domains. Notice
however, that our method of estimating the Fisher information along the
solution of a system of partial differential equations seems promising in
other type of problems. The inequality from Appendix A turns out to be very
helpful in such an approach. Indeed, in [3] Fisher‘s information together with
inequality (A.1) were succesfully applied to obtain global-in-time unique
regular solutions to the 1D thermoelasticity problem. Very recently, another
application yielding progress in 1D combustion theory was performed in [13].
Last, but not least, notice that our approach gives also an qualitative
conditional result, namely it implies that concavity of $v$ is sufficient to
obtain global solution to (1.1), see Section 6.

The problem (1.1)–(1.3) captured attention of groups of researchers, in
particular due to its role in the attraction-repulsion competition, which
appears to play a role in the modeling of, among other things, Alzheimer‘s
disease, see for instance [15]. The biological meaning of the attraction-
repulsion competition was widely investigated, see for instance the
contributions in [14, 16, 18].

Let us formulate our main result.

######  Theorem 1.1.

Let $\Omega\subset{\mathbb{R}}^{3}$ be a convex, smooth bounded domain and let
$u_{0},v_{0}\in W^{1,p}(\Omega)$ for some $p>3$ with $0\not\equiv u_{0}\geq 0$
and $v_{0}\geq 0$ in $\Omega$. Suppose that $T_{\max}\in(0,\infty]$ is the
maximal existence time of the classical solution to the system (1.1)–(1.3),
constructed in [5, Theorem 2.1] (cf. Lemma 3.1 below). If $\nabla\sqrt{u}\in
L^{4}\left({0,T_{\max};L^{2}(\Omega)}\right)$, then $T_{\max}=\infty$.

The paper is organised as follows: In Section 2, we introduce some technical
tools such as the Winkler version of the Bernis-type inequality, Bochner‘s
formula, the behaviour of the normal derivative of the gradients of regular
functions at the boundaries of convex domains and further preparatory
inequalities. In Section 3, we then recall well-known properties of solutions
to (1.1)–(1.3).

Section 4 is devoted to our entropy estimate. It is a core and the main
novelty of our approach. We estimate the time derivative of the entropy
production term occurring in the Lyapunov functional. The latter entropy
production term resembles the Fisher information along the heat flow and we
utilize this similarity. Having this estimate established, in Section 5 we
proceed with the proof of Theorem 1.1.

In Appendix A, we prove a new functional inequality, which we arrived at as a
byproduct of our investigations. It seems interesting in its own right.
Appendix B is devoted to the proof of an observation due to M.Ẇinkler and one
of the anonymous referees, which improves the conditional result.

##  2\. Preliminaries

In this section, we collect some computations and known results which will be
useful in the sequel.

We begin with the flat case of the well-known Bochner formula.

######  Proposition 2.1.

Let $\Omega\subset{\mathbb{R}}^{n}$, $n\in{\mathbb{N}}$, be a smooth domain
and let $u\in C^{3}(\bar{\Omega})$. Then

(2.1) $\frac{1}{2}\Delta\left\lvert{\nabla
u}\right\rvert^{2}=\nabla\left({\Delta u}\right)\cdot\nabla
u+\left\lvert{D^{2}u}\right\rvert^{2}\quad\text{in}\quad\bar{\Omega}.$

###### Proof.

This can be checked by a direct calculation. ∎

The following lemma informs us about the normal derivative of the square of
the gradient of a function on the boundary of a convex set, provided the
function‘s normal derivative vanishes.

######  Lemma 2.2.

Let $\Omega\subset{\mathbb{R}}^{n}$, $n\in{\mathbb{N}}$, be a convex bounded
domain with smooth boundary. Suppose that a function $u\in
C^{2}(\bar{\Omega})$ satisfies $\frac{\partial u}{\partial\nu}=0$ on
$\partial\Omega$. Then

$\left.\frac{\partial\left\lvert{\nabla
u}\right\rvert^{2}}{\partial\nu}\right\rvert_{\partial\Omega}\leq 0.$

###### Proof.

See [6, p. 95]. ∎

We will use a higher-dimensional version of the Bernis-type inequality given
by Winkler [20, Lemma 3.3] (with $h(\varphi)=\varphi$).

######  Lemma 2.3.

Let $\Omega\subset{\mathbb{R}}^{n}$, $n\in{\mathbb{N}}$, be a smooth, bounded
domain. For all positive $\varphi\in C^{2}\left({\bar{\Omega}}\right)$ with
$\frac{\partial\varphi}{\partial\nu}=0$ on $\partial\Omega$, we have the
following inequality

(2.2)
$\displaystyle\int_{\Omega}{\frac{|\nabla\varphi|^{4}}{\varphi^{3}}}\leq\left({2+\sqrt{n}}\right)^{2}\int_{\Omega}{\varphi\left\lvert{D^{2}\log\varphi}\right\rvert^{2}}.$

Next, we prove two estimates holding in three-dimensional domains. The first
one relates the Hessian of a function $\varphi$ with $\nabla\Delta\varphi$ (in
contrast to the full third-order derivative).

######  Lemma 2.4.

Let $\Omega\subset{\mathbb{R}}^{3}$ be a smooth, bounded domain. Then there is
a positive constant $C$ such that for every $\varphi\in C^{3}(\bar{\Omega})$
with $\frac{\partial\varphi}{\partial\nu}=0$ on $\partial\Omega$ we have

$\displaystyle\left\lVert{D^{2}\varphi}\right\rVert_{L^{6}(\Omega)}\leq
C\left\lVert{\nabla\Delta\varphi}\right\rVert_{L^{2}(\Omega)}.$

###### Proof.

According to [8, Theorem $19.1$], there is $c_{1}>0$ such that

$\displaystyle\left\lVert{D^{2}\varphi}\right\rVert_{L^{6}}\leq
c_{1}\left\lVert{\Delta\varphi}\right\rVert_{L^{6}}+c_{1}\left\lVert{\varphi-\bar{\varphi}}\right\rVert_{L^{6}}$

for every $\varphi\in C^{2}(\bar{\Omega})$ with
$\frac{\partial\varphi}{\partial\nu}=0$ on $\partial\Omega$, where
$\bar{\varphi}=\frac{1}{|\Omega|}\int_{\Omega}\varphi$. As
$W^{1,2}(\Omega)\hookrightarrow L^{6}(\Omega)$, we can further estimate

$\displaystyle\left\lVert{D^{2}\varphi}\right\rVert_{L^{6}}\leq
c_{2}\left({\left\lVert{\nabla\Delta\varphi}\right\rVert_{L^{2}}^{2}+\left\lVert{\Delta\varphi}\right\rVert_{L^{2}}^{2}}\right)^{1/2}+c_{2}\left({\left\lVert{\nabla\varphi}\right\rVert_{L^{2}}^{2}+\left\lVert{\varphi-\bar{\varphi}}\right\rVert_{L^{2}}^{2}}\right)^{1/2}$

for every $\varphi\in C^{3}(\bar{\Omega})$ with
$\frac{\partial\varphi}{\partial\nu}=0$ on $\partial\Omega$, for some
$c_{2}>0$. The statement then follows by the Poincaré inequality. See, for
example, [9, Lemma A.1] which states that there exists a constant $c>0$ such
that

$\displaystyle\left\lVert{\varphi-\bar{\varphi}}\right\rVert_{L^{2}}^{2}\leq
c\left\lVert{\nabla\varphi}\right\rVert_{L^{2}}^{2},\quad\left\lVert{\nabla\varphi}\right\rVert_{L^{2}}^{2}\leq
c\left\lVert{\Delta\varphi}\right\rVert_{L^{2}}^{2}\quad\&\quad\left\lVert{\Delta\varphi}\right\rVert_{L^{2}}^{2}\leq
c\left\lVert{\nabla\Delta\varphi}\right\rVert_{L^{2}}^{2}$

for all $\varphi\in C^{3}(\bar{\Omega})$. ∎

Finally, we combine several of the lemmata above to obtain an estimate
required in the proof of our main result.

######  Lemma 2.5.

Let $\Omega\subset{\mathbb{R}}^{3}$ be a convex, smooth bounded domain and let
${\varepsilon}>0$ and $M>0$. Then there exists $C>0$ such that for every
$0<\varphi\in C^{2}(\bar{\Omega})$ with $\int_{\Omega}\varphi=M>0$ and
$\psi\in C^{3}(\bar{\Omega})$ that satisfy
$\partial_{\nu}\varphi=\partial_{\nu}\psi=0$ on $\partial\Omega$ we have

$\displaystyle\mathrel{\hphantom{=}}\int_{\Omega}|(\nabla\sqrt{\varphi})^{T}D^{2}\psi(\nabla\sqrt{\varphi})|$
(2.3) $\displaystyle\leq
C\left({\int_{\Omega}{\left\lvert{\nabla\sqrt{\varphi}}\right\rvert^{2}}}\right)^{3}+C+{\varepsilon}\int_{\Omega}{\varphi\left\lvert{D^{2}\log{\varphi}}\right\rvert^{2}}+{\varepsilon}\int_{\Omega}{\left\lvert{\nabla\Delta\psi}\right\rvert^{2}}.$

###### Proof.

By Hölder’s inequality, we have

$\displaystyle\mathrel{\hphantom{=}}\int_{\Omega}|(\nabla\sqrt{\varphi})^{T}D^{2}\psi(\nabla\sqrt{\varphi})|$
$\displaystyle\leq\frac{1}{2}\int_{\Omega}{\left\lvert{\nabla\sqrt{\varphi}}\right\rvert\left\lvert{D^{2}\psi}\right\rvert\frac{\left\lvert{\nabla\varphi}\right\rvert}{\varphi^{3/4}}\varphi^{1/4}}$
(2.4)
$\displaystyle\leq\frac{1}{2}\left\lVert{\nabla\sqrt{\varphi}}\right\rVert_{L^{2}(\Omega)}\left\lVert{D^{2}\psi}\right\rVert_{L^{6}(\Omega)}\left\lVert{\frac{\nabla\varphi}{\varphi^{\frac{3}{4}}}}\right\rVert_{L^{4}(\Omega)}\left\lVert{\varphi^{1/4}}\right\rVert_{L^{12}(\Omega)}$

for all $0<\varphi\in C^{2}(\bar{\Omega})$ and $\psi\in C^{3}(\bar{\Omega})$.

Since $W^{1,2}(\Omega)\hookrightarrow L^{6}(\Omega)$ and
$\int_{\Omega}{\varphi}=M$ there is $c_{1}>0$ such that

$\displaystyle\left\lVert{\varphi^{1/4}}\right\rVert_{L^{12}(\Omega)}$
$\displaystyle=\left\lVert{\varphi^{1/2}}\right\rVert^{1/2}_{L^{6}(\Omega)}\leq
c_{1}\left\lVert{\varphi^{1/2}}\right\rVert^{1/2}_{W^{1,2}(\Omega)}$
$\displaystyle\leq
c_{1}\left\lVert{\nabla\varphi^{1/2}}\right\rVert^{1/2}_{L^{2}(\Omega)}+c_{1}\left\lVert{\varphi^{1/2}}\right\rVert^{1/2}_{L^{2}(\Omega)}=c_{1}\left({\left\lVert{\nabla\sqrt{\varphi}}\right\rVert^{1/2}_{L^{2}(\Omega)}+M^{1/4}}\right)$

for all $0<\varphi\in C^{2}(\bar{\Omega})$ with $\int_{\Omega}\varphi=M$. In
combination with (2), Lemma 2.4, Winkler‘s inequality (2.2), the elementary
estimate

$a(\sqrt{a}+\sqrt{b})\leq 2(a+b)\sqrt{a+b}\quad\text{for}\quad a,b\geq 0$

and Young’s inequality, we see that by taking
$a=\left\lVert{\nabla\sqrt{\varphi}}\right\rVert_{L^{2}(\Omega)}$, $b=M^{1/2}$
and with some $c_{2}>0$ and $C>0$, we have

$\displaystyle\mathrel{\hphantom{=}}\int_{\Omega}|(\nabla\sqrt{\varphi})^{T}D^{2}\psi\nabla(\sqrt{\varphi})|$
$\displaystyle\leq
c_{2}\left\lVert{\nabla\sqrt{\varphi}}\right\rVert_{L^{2}(\Omega)}\left\lVert{\nabla\Delta\psi}\right\rVert_{L^{2}(\Omega)}\left({\int_{\Omega}{\varphi\left\lvert{D^{2}\log{\varphi}}\right\rvert^{2}}}\right)^{1/4}\left({\left\lVert{\nabla\sqrt{\varphi}}\right\rVert^{1/2}_{L^{2}(\Omega)}+M^{1/4}}\right)$
$\displaystyle\leq
2c_{2}\left({\left\lVert{\nabla\sqrt{\varphi}}\right\rVert_{L^{2}}+M^{1/2}}\right)^{\frac{3}{2}}\left({\int_{\Omega}{\varphi\left\lvert{D^{2}\log{\varphi}}\right\rvert^{2}}}\right)^{1/4}\left\lVert{\nabla\Delta\psi}\right\rVert_{L^{2}}$
$\displaystyle\leq
C\left\lVert{\nabla\sqrt{\varphi}}\right\rVert^{6}_{L^{2}}+C+{\varepsilon}\int_{\Omega}{\varphi\left\lvert{D^{2}\log{\varphi}}\right\rvert^{2}}+{\varepsilon}\left\lVert{\nabla\Delta\psi}\right\rVert^{2}_{L^{2}}$

for all $0<\varphi\in C^{2}(\bar{\Omega})$ and $\psi\in C^{3}(\bar{\Omega})$
with $\int_{\Omega}\varphi=M$ and $\partial_{\nu}\varphi=\partial_{\nu}\psi=0$
on $\partial\Omega$. ∎

##  3\. Known properties of the solution

Next, we list some known properties of solutions to (1.1)–(1.3) constructed in
[5].

######  Lemma 3.1.

Let

(3.1) $\displaystyle\Omega\subset{\mathbb{R}}^{n},n\in{\mathbb{N}},\text{ be a
smooth, bounded domain}$

and let

(3.2) $\displaystyle u_{0},v_{0}\in W^{1,p}(\Omega)\text{ for some
$p>n$}\quad\text{with}\quad 0\not\equiv u_{0}\geq 0\text{ and }v_{0}\geq
0\text{ in $\Omega$}.$

Then the system (1.1)–(1.3) has a maximal unique classical solution

(3.3) $\displaystyle(u,v)\in C^{0}([0,T_{\max});W^{1,p}(\Omega))\cap
C^{\infty}(\bar{\Omega}\times(0,T_{\max}))$

and if $T_{\max}<\infty$, then $\limsup_{t\nearrow
T_{\max}}(\|u(\cdot,t)\|_{L^{\infty}(\Omega)}+\|v(\cdot,t)\|_{L^{\infty}(\Omega)})=\infty$.
Moreover, $u$ and $v$ are positive in $\bar{\Omega}\times(0,T_{\max})$.

###### Proof.

The existence of a $C^{\infty}(\bar{\Omega}\times(0,T_{\max}))$ solution that
is nonnegative has been proved in [5, Theorem 2.1]. Applying the Hopf lemma to
each of the equations in (1.1) separately (in view of the regularity (3.3),
$\Delta v$ is bounded on $\bar{\Omega}\times[\tau,T]$ for every
$0<\tau<T<T_{\max}$), due to zero Neumann boundary data (1.2), we arrive at
the positivity of $u$ and $v$ for each $t\in(0,T_{\max})$. ∎

As noted in [5, (3)], integrating both equations in (1.1) immediately ensures,
that both solution components are uniformly in time bounded in
$L^{1}(\Omega)$.

######  Lemma 3.2.

Suppose the assumptions of Lemma 3.1 hold. Then the solution $(u,v)$ of
(1.1)–(1.3) given by Lemma 3.1 fulfills

$\displaystyle\|u(\cdot,t)\|_{L^{1}(\Omega)}$
$\displaystyle=\|u_{0}\|_{L^{1}(\Omega)}\quad\text{and}$
$\displaystyle\left\lVert{v(\cdot,t)}\right\rVert_{L^{1}(\Omega)}$
$\displaystyle={\rm
e}^{-t}\left({\left\lVert{v_{0}}\right\rVert_{L^{1}(\Omega)}-\left\lVert{u_{0}}\right\rVert_{L^{1}(\Omega)}}\right)+\left\lVert{u_{0}}\right\rVert_{L^{1}(\Omega)}$

for all $t\in(0,T_{\max})$.

Moreover, [5] has identified a Lyapunov functional, which served as the main
ingredient for solving the question of global existence the two-dimensional
case.

######  Lemma 3.3.

Under the assumptions of Lemma 3.1 the solution $(u,v)$ satisfies

(3.4) $\displaystyle\frac{{\rm d}}{{\rm
d}t}\left({\int_{\Omega}{u\log{u}}+\frac{1}{2}\int_{\Omega}{\left\lvert{\nabla
v}\right\rvert^{2}}}\right)=-\left({\int_{\Omega}{\left\lvert{\Delta
v}\right\rvert^{2}+\int_{\Omega}\left\lvert{\nabla
v}\right\rvert^{2}+\int_{\Omega}\frac{\left\lvert{\nabla
u}\right\rvert^{2}}{u}}}\right)$

for all $t\in(0,T_{\max})$. In particular,

(3.5) $\displaystyle\int_{0}^{T_{\max}}\left({\int_{\Omega}{\left\lvert{\Delta
v}\right\rvert^{2}+\int_{\Omega}\left\lvert{\nabla
v}\right\rvert^{2}+\int_{\Omega}\frac{\left\lvert{\nabla
u}\right\rvert^{2}}{u}}}\right)<\infty.$

###### Proof.

The differential inequality (3.4) is entailed in [5, Lemma 2.2], upon which
(3.5) results by an integration in time as the Lyapunov functional is bounded
from below. ∎

##  4\. The main estimate

This section contains our main contribution, a calculation of the evolution of
the Fisher information along the trajectories of (1.1)–(1.3). It is related to
the Bakry-Émery calculation, see [1], applied however to a system of
equations.

Throughout this section, we fix a domain and initial data fulfilling (3.1) and
(3.2) as well as the solution $(u,v)$ of (1.1)–(1.3), with maximal existence
time $T_{\max}$ given by Lemma 3.1. Moreover, we denote

(4.1) $\displaystyle\frac{{\rm d}}{{\rm
d}t}\left({\int_{\Omega}{{\left\lvert{\Delta
v}\right\rvert^{2}+\int_{\Omega}\left\lvert{\nabla
v}\right\rvert^{2}+\int_{\Omega}\frac{\left\lvert{\nabla
u}\right\rvert^{2}}{u}}}}\right)$ $\displaystyle\eqqcolon\frac{{\rm d}}{{\rm
d}t}I(t).$

Our aim is to obtain an estimate of $I$. Notice that $I$ is an extended
version of the Fisher information. Indeed, in the case of a single heat
equation, the quantity $\int_{\Omega}\frac{|\nabla u|^{2}}{u}$ is called
Fisher‘s information. The following remark explains our strategy.

######  Remark 4.1.

We note that the inequality $\dot{I}\leq cI^{2}$ would imply boundedness of
$I$ due to (3.5). Indeed, using Ladyzhenskaya‘s trick (see [11]) we would have

$\frac{{\rm d}}{{\rm d}t}\left({I(t){\rm e}^{-\int_{0}^{t}{cI(s)\,{\rm
d}s}}}\right)\leq 0.$

Below we formulate and prove our main contribution. It extends the calculation
controlling the evolution of the Fisher information to the case of a system of
equations.

######  Lemma 4.2.

For all $t\in(0,T_{\max})$, the estimate

$\displaystyle\dot{I}(t)$
$\displaystyle\leq-2\int_{\Omega}{u\left\lvert{D^{2}\log{u}}\right\rvert^{2}}+8\int_{\Omega}{\left({\nabla\sqrt{u}}\right)^{T}D^{2}v\left({\nabla\sqrt{u}}\right)}$
$\displaystyle\mathrel{\hphantom{\leq}}-2\int_{\Omega}|\nabla\Delta
v|^{2}-4\int_{\Omega}|\Delta v|^{2}-2\int_{\Omega}|\nabla
v|^{2}+2\int_{\Omega}{\nabla u\cdot\nabla v}$

holds.

###### Proof.

We notice

$\displaystyle\frac{{\rm d}}{{\rm d}t}\int_{\Omega}{\left\lvert{\Delta
v}\right\rvert^{2}}=2\int_{\Omega}{\Delta{v}_{t}\Delta
v}\quad\text{in}\quad(0,T_{\max}).$

From the second equation in $(\ref{CRu})$ we can substitute $\Delta
v=v_{t}+v-u$ (equivalently, we can take the inner product of the second
equation in $(\ref{CRu})$ with $\Delta v_{t}$), to get

$\displaystyle\frac{{\rm d}}{{\rm d}t}\int_{\Omega}{\left\lvert{\Delta
v}\right\rvert^{2}}$ $\displaystyle=2{\int_{\Omega}{\Delta
v_{t}\left({v_{t}+v-u}\right)}}=-2{\int_{\Omega}{\nabla
v_{t}\cdot\left({\nabla v_{t}+\nabla v-\nabla u}\right)}}$ (4.2)
$\displaystyle=-2{\int_{\Omega}{\left\lvert{\nabla
v_{t}}\right\rvert^{2}}}-\frac{{\rm d}}{{\rm
d}t}{\int_{\Omega}{\left\lvert{\nabla
v}\right\rvert^{2}}}+2{\int_{\Omega}{\nabla v_{t}\cdot\nabla
u}}\quad\text{in}\quad(0,T_{\max}).$

From $(\ref{lapderiv})$ we get

(4.3) $\displaystyle\frac{{\rm d}}{{\rm
d}t}\left({\int_{\Omega}{{\left\lvert{\Delta
v}\right\rvert^{2}+\left\lvert{\nabla
v}\right\rvert^{2}}}}\right)=-2{\int_{\Omega}{\left\lvert{\nabla
v_{t}}\right\rvert^{2}}}+2{\int_{\Omega}{\nabla v_{t}\cdot\nabla
u}}\quad\text{in}\quad(0,T_{\max})$

and from $(\ref{LF2})$ and $(\ref{partderiv})$ we obtain

(4.4) $\displaystyle\dot{I}(t)=-2{\int_{\Omega}{\left\lvert{\nabla
v_{t}}\right\rvert^{2}}}+2{\int_{\Omega}{\nabla v_{t}\cdot\nabla
u}}+\frac{{\rm d}}{{\rm d}t}\int_{\Omega}{\frac{\left\lvert{\nabla
u}\right\rvert^{2}}{u}}\quad\text{for all}\quad t\in(0,T_{\max}).$

Next, we compute the last term on the right-hand-side,

(4.5) $\displaystyle\frac{{\rm d}}{{\rm
d}t}\int_{\Omega}{\frac{\left\lvert{\nabla u}\right\rvert^{2}}{u}}=4\frac{{\rm
d}}{{\rm
d}t}\int_{\Omega}{\left\lvert{\nabla{\sqrt{u}}}\right\rvert^{2}}=8\int_{\Omega}{\nabla{{\varrho}_{t}}\cdot\nabla{{\varrho}}}\quad\text{in}\quad(0,T_{\max}),$

where we applied the substitution ${\varrho}\coloneqq\sqrt{u}$. From the first
equation in $(\ref{CRu})$ we have

(4.6) $\displaystyle\varrho_{t}$
$\displaystyle=\frac{u_{t}}{2\sqrt{u}}=\frac{\Delta u+\nabla u\cdot\nabla
v+u\Delta v}{2\sqrt{u}}=\frac{\Delta u}{2\sqrt{u}}+\nabla{\varrho}\cdot\nabla
v+\frac{1}{2}{\varrho}\Delta v,$

where

(4.7) $\displaystyle\Delta{\varrho}$ $\displaystyle=\rm
div\left({\nabla{\varrho}}\right)=\rm div\left({\frac{\nabla
u}{2\sqrt{u}}}\right)=\frac{\Delta u}{2\sqrt{u}}-\frac{\left\lvert{\nabla
u}\right\rvert^{2}}{4u^{3/2}}=\frac{\Delta
u}{2\sqrt{u}}-\frac{\left\lvert{\nabla{\varrho}}\right\rvert^{2}}{{\varrho}}$

in $\Omega\times(0,T_{\max})$. So, by plugging $(\ref{Lapvro})$ into
$(\ref{rhot2})$, we find

$\displaystyle\varrho_{t}$
$\displaystyle=\Delta{\varrho}+\frac{\left\lvert{\nabla{\varrho}}\right\rvert^{2}}{{\varrho}}+\nabla{\varrho}\cdot\nabla
v+\frac{1}{2}{\varrho}\Delta v\quad\text{in}\quad\Omega\times(0,T_{\max}).$

Hence, $(\ref{FI})$ becomes

$\displaystyle\frac{{\rm d}}{{\rm d}t}\int_{\Omega}{\frac{\left\lvert{\nabla
u}\right\rvert^{2}}{u}}$
$\displaystyle=8\int_{\Omega}{\nabla{{\varrho}}\cdot\nabla\left({\Delta{\varrho}+\frac{\left\lvert{\nabla{\varrho}}\right\rvert^{2}}{{\varrho}}+\nabla{\varrho}\cdot\nabla
v+\frac{1}{2}{\varrho}\Delta v}\right)}$
$\displaystyle=8\left[{\int_{\Omega}{\nabla{\varrho}\cdot\nabla\left({\Delta{\varrho}}\right)}+\int_{\Omega}{2\frac{\left({\nabla{\varrho}}\right)^{T}D^{2}{\varrho}\left({\nabla{\varrho}}\right)}{{\varrho}}}-\int_{\Omega}{\frac{\left\lvert{\nabla{\varrho}}\right\rvert^{4}}{{\varrho}^{2}}}}\right]$
$\displaystyle\mathrel{\hphantom{=}}+8\left[{\int_{\Omega}{\left({\nabla{\varrho}}\right)^{T}D^{2}{\varrho}\left({\nabla
v}\right)}+\int_{\Omega}{\left({\nabla{\varrho}}\right)^{T}D^{2}v\left({\nabla{\varrho}}\right)}}\right]$
(4.8)
$\displaystyle\mathrel{\hphantom{=}}+4\int_{\Omega}{\nabla\left({{\varrho}\Delta
v}\right)\cdot\nabla{\varrho}}\quad\text{in}\quad(0,T_{\max}).$

Due to the Bochner formula $(\ref{bochnerformula})$,

$\nabla{\varrho}\cdot\nabla\left({\Delta{\varrho}}\right)=-\left\lvert{D^{2}{\varrho}}\right\rvert^{2}+\frac{1}{2}\Delta\left({\left\lvert{\nabla{\varrho}}\right\rvert^{2}}\right)\quad\text{in}\quad\Omega\times(0,T_{\max}),$

we get

$\displaystyle\mathrel{\hphantom{=}}\int_{\Omega}{\nabla{\varrho}\cdot\nabla\left({\Delta{\varrho}}\right)}+\int_{\Omega}{2\frac{\left({\nabla{\varrho}}\right)^{T}D^{2}{\varrho}\left({\nabla{\varrho}}\right)}{{\varrho}}}-\int_{\Omega}{\frac{\left\lvert{\nabla{\varrho}}\right\rvert^{4}}{{\varrho}^{2}}}$
$\displaystyle=-\int_{\Omega}{\left\lvert{D^{2}{\varrho}-\frac{\nabla{\varrho}\otimes\nabla{\varrho}}{{\varrho}}}\right\rvert^{2}}+\frac{1}{2}\int_{\Omega}{\Delta\left({\left\lvert{\nabla{\varrho}}\right\rvert^{2}}\right)}\quad\text{in}\quad(0,T_{\max}).$

We note that the boundary condition $\left.\frac{\partial
u}{\partial\nu}\right\rvert_{\partial\Omega}=0$ implies that

$\left.\frac{\partial{\varrho}}{\partial\nu}\right\rvert_{\partial\Omega}=\left.\frac{\frac{\partial
u}{\partial\nu}}{2\sqrt{u}}\right\rvert_{\partial\Omega}=0,$

so that an integration by parts and an application of Lemma 2.2, which is
possible thanks to the convexity of the domain $\Omega$, yield

$\int_{\Omega}{\Delta\left({\left\lvert{\nabla{\varrho}}\right\rvert^{2}}\right)}=\int_{\partial\Omega}{\frac{\partial\left({\left\lvert{\nabla{\varrho}}\right\rvert^{2}}\right)}{\partial\nu}}\leq
0\quad\text{in}\quad(0,T_{\max}).$

Plugging the above into $(\ref{FI2})$, we obtain

$\displaystyle\frac{{\rm d}}{{\rm d}t}\int_{\Omega}{\frac{\left\lvert{\nabla
u}\right\rvert^{2}}{u}}$
$\displaystyle\leq-2\int_{\Omega}{u\left\lvert{D^{2}\log{u}}\right\rvert^{2}}$
$\displaystyle+8\left[{\int_{\Omega}{\left({\nabla{\varrho}}\right)^{T}D^{2}{\varrho}\left({\nabla
v}\right)}+\int_{\Omega}{\left({\nabla{\varrho}}\right)^{T}D^{2}v\left({\nabla{\varrho}}\right)}}\right]$
(4.9) $\displaystyle+4\int_{\Omega}{\nabla\left({{\varrho}\Delta
v}\right)\cdot\nabla{\varrho}}\quad\text{in}\quad(0,T_{\max}),$

where we also used the relations

$\int_{\Omega}{\left\lvert{D^{2}{\varrho}-\frac{\nabla{\varrho}\otimes\nabla{\varrho}}{{\varrho}}}\right\rvert^{2}}=\int_{\Omega}{{\varrho}^{2}\left\lvert{D^{2}\log{{\varrho}}}\right\rvert^{2}}=\frac{1}{4}\int_{\Omega}u\left\lvert{D^{2}\log
u}\right\rvert^{2}$

in the first term on the right-hand side.

We now focus on the last term in $(\ref{FI3})$,

(4.10) $\displaystyle 4\int_{\Omega}{\nabla\left({{\varrho}\Delta
v}\right)\cdot\nabla{\varrho}}=4\int_{\Omega}{\left\lvert{\nabla{\varrho}}\right\rvert^{2}\Delta
v}+4\int_{\Omega}{{\varrho}\nabla{\varrho}\cdot\nabla\left({\Delta
v}\right)}.$

Integration by parts yields

(4.11) $4\int_{\Omega}{\left\lvert{\nabla{\varrho}}\right\rvert^{2}\Delta
v}=-4\int_{\Omega}{\nabla\left({\left\lvert{\nabla{\varrho}}\right\rvert^{2}}\right)\cdot\nabla
v}=-8\int_{\Omega}{\left({\nabla{\varrho}}\right)^{T}D^{2}{\varrho}\left({\nabla
v}\right)}$

in $(0,T_{\max})$. For the second term in $(\ref{gradLapv})$ we substitute
$\Delta v=v_{t}+v-u$ from the second equation in $(\ref{CRu})$ to obtain

$\displaystyle\mathrel{\hphantom{=}}4\int_{\Omega}{{\varrho}\nabla{\varrho}\cdot\nabla\left({\Delta
v}\right)}=2\int_{\Omega}{\nabla{\varrho}^{2}\cdot\nabla\left({v_{t}+v-u}\right)}$
(4.12) $\displaystyle=2\int_{\Omega}{\nabla u\cdot\nabla
v_{t}}+2\int_{\Omega}{\nabla u\cdot\nabla v}-2\int_{\Omega}{\left\lvert{\nabla
u}\right\rvert^{2}}\quad\text{in}\quad(0,T_{\max}).$

Inserting $(\ref{D2vro})$ and $(\ref{gradLapv2})$ in $(\ref{FI3})$ gives

$\displaystyle\frac{{\rm d}}{{\rm d}t}\int_{\Omega}{\frac{\left\lvert{\nabla
u}\right\rvert^{2}}{u}}$
$\displaystyle\leq-2\int_{\Omega}{u\left\lvert{D^{2}\log{u}}\right\rvert^{2}}+8\int_{\Omega}{\left({\nabla{\varrho}}\right)^{T}D^{2}v\left({\nabla{\varrho}}\right)}$
$\displaystyle\mathrel{\hphantom{=}}+2\int_{\Omega}{\nabla u\cdot\nabla
v_{t}}+2\int_{\Omega}{\nabla u\cdot\nabla v}-2\int_{\Omega}{\left\lvert{\nabla
u}\right\rvert^{2}}\quad\text{in}\quad(0,T_{\max}).$

Therefore, going back to $(\ref{mainineq})$, we have

$\displaystyle\dot{I}(t)$
$\displaystyle\leq-2\int_{\Omega}{u\left\lvert{D^{2}\log{u}}\right\rvert^{2}}+8\int_{\Omega}{\left({\nabla\sqrt{u}}\right)^{T}D^{2}v\left({\nabla\sqrt{u}}\right)}$
$\displaystyle\mathrel{\hphantom{=}}-2{\int_{\Omega}{\left\lvert{\nabla
v_{t}}\right\rvert^{2}}}+4{\int_{\Omega}{\nabla v_{t}\cdot\nabla
u}}+2\int_{\Omega}{\nabla u\cdot\nabla v}-2\int_{\Omega}{\left\lvert{\nabla
u}\right\rvert^{2}}$

for all $t\in(0,T_{\max})$. Since

$\displaystyle\mathrel{\hphantom{=}}-2{\int_{\Omega}{\left\lvert{\nabla
v_{t}}\right\rvert^{2}}}+4{\int_{\Omega}{\nabla v_{t}\cdot\nabla
u}}-2\int_{\Omega}{\left\lvert{\nabla u}\right\rvert^{2}}$
$\displaystyle=-2\int_{\Omega}|\nabla(v_{t}-u)|^{2}=-2\int_{\Omega}|\nabla(\Delta
v-v)|^{2}$ $\displaystyle=-2\int_{\Omega}|\nabla\Delta
v|^{2}-4\int_{\Omega}|\Delta v|^{2}-2\int_{\Omega}|\nabla
v|^{2}\quad\text{in}\quad(0,T_{\max}),$

we obtain the desired estimate. ∎

Next, we simplify the previous differential inequality, which will allow us to
argue in a more straightforward manner in the sequel.

######  Lemma 4.3.

Throughout $(0,T_{\max})$, it holds that

$\displaystyle\mathrel{\hphantom{=}}\frac{{\rm d}}{{\rm
d}t}\left(4\int_{\Omega}|\nabla\sqrt{u}|^{2}+\int_{\Omega}|\Delta
v|^{2}\right)$ $\displaystyle\leq-2\int_{\Omega}u|D^{2}\log
u|^{2}-2\int_{\Omega}|\nabla\Delta v|^{2}-2\int_{\Omega}|\Delta v|^{2}$
$\displaystyle\mathrel{\hphantom{=}}+8\int_{\Omega}(\nabla\sqrt{u})^{T}D^{2}v\nabla\sqrt{u}.$

###### Proof.

This follows immediately from Lemma 4.2 and the fact that

$\displaystyle\frac{{\rm d}}{{\rm d}t}\int_{\Omega}|\nabla v|^{2}$
$\displaystyle=-2\int_{\Omega}\Delta vv_{t}=-2\int_{\Omega}|\Delta
v|^{2}-2\int_{\Omega}|\nabla v|^{2}+2\int_{\Omega}\nabla v\cdot\nabla v$

in $(0,T_{\max})$. ∎

##  5\. Proof of the main theorem

We are now in a position to utilize our calculation from Lemma 4.2 and
complete the proof of the announced result. As in the previous section, we fix
a domain $\Omega$ and initial data $u_{0},v_{0}$ satisfying (3.1) and (3.2) as
well as the solution $(u,v)$ of (1.1)–(1.3) given by Lemma 3.1. Moreover, as
the solution is unique by Lemma 3.1, $T_{\max}$ is infinite if and only if the
solution with initial data $(u(\cdot,t_{0}),v(\cdot,t_{0}))$ for some
$t_{0}\in(0,T_{\max})$ exists globally. Thus, by switching to the solution
with these initial data and recalling (3.3), we may assume $u,v\in
C^{\infty}(\bar{\Omega}\times[0,T_{\max}))$.

The following lemma is the first step in a bootstrapping procedure yielding
the required regularity of the solution.

######  Lemma 5.1.

Suppose $n=3$ and that $\Omega$ is convex. Let
$T\in(0,T_{\max}]\cap(0,\infty)$ and suppose that $\nabla\sqrt{u}\in
L^{4}(0,T;L^{2}(\Omega))$. Then there is $C>0$ such that

$\displaystyle\int_{\Omega}u^{3}(\cdot,t)\leq C\qquad\text{for all}\quad
t\in(0,T).$

###### Proof.

We define

$J(t)\coloneqq 4\int_{\Omega}|\nabla\sqrt{u}|^{2}+\int_{\Omega}|\Delta
v|^{2}.$

Taking $\varphi=u$, $\psi=v$ and $\varepsilon=\frac{1}{4}$ in Lemma 2.5 and
making use of Lemma 4.3, we arrive at

$\displaystyle\dot{J}(t)$ $\displaystyle\leq
c_{1}\left({\int_{\Omega}{\left\lvert{\nabla\sqrt{u}}\right\rvert^{2}}}\right)^{3}+c_{1}\leq
c_{1}\left({\int_{\Omega}{\left\lvert{\nabla\sqrt{u}}\right\rvert^{2}}}\right)^{2}J(t)+c_{1}$

in $(0,T)$ for some $c_{1}>0$. Thus, with $K(t)\coloneqq
c_{1}\int_{0}^{t}\left({\int_{\Omega}{\left\lvert{\nabla\sqrt{u}}\right\rvert^{2}}}\right)^{2}$,
$t\in(0,T)$, we have

$\displaystyle J(t)\leq{\rm e}^{K(t)}J(0)+c_{1}\int_{0}^{t}{\rm
e}^{K(t-s)}\,{\rm d}s\leq{\rm e}^{K(T)}J(0)+c_{1}T{\rm e}^{K(T)}$

for all $t\in(0,T)$. Since $K(T)<\infty$ by assumption, we obtain boundedness
of $\sup_{t\in(0,T)}\|\nabla\sqrt{u(\cdot,t)}\|_{L^{2}(\Omega)}$, which, in
conjunction with Lemma 3.2, implies the desired estimate as $W^{1,2}(\Omega)$
embeds continuously into $L^{6}(\Omega)$. ∎

Next, we show the higher regularity of the obtained solution.

######  Lemma 5.2.

Under the assumptions of Lemma 5.1 there is $C>0$ such that

$\displaystyle\|u(\cdot,t)\|_{L^{\infty}(\Omega)}+\|v(\cdot,t)\|_{L^{\infty}(\Omega)}\leq
C\qquad\text{for all}\quad t\in(0,T).$

###### Proof.

We fix $3<r<q<\infty$. Making use of well-known semigroup estimates (cf. [19,
Lemma 1.3 (ii) and (iii)]), we obtain

$\displaystyle\mathrel{\hphantom{\leq}}\|\nabla v(\cdot,t)\|_{L^{q}(\Omega)}$
$\displaystyle\leq\|\nabla\mathrm{e}^{t(\Delta-1)}v_{0}\|_{L^{q}(\Omega)}+\int_{0}^{t}\|\mathrm{e}^{(t-s)(\Delta-1)}u(\cdot,s)\|_{L^{q}(\Omega)}\,\mathrm{d}s$
$\displaystyle\leq c_{1}\mathrm{e}^{-t}\|\nabla
v_{0}\|_{L^{q}(\Omega)}+c_{2}\int_{0}^{t}\left(1+(t-s)^{-\frac{1}{2}-\frac{3}{2}(\frac{1}{3}-\frac{1}{q})}\right)\mathrm{e}^{-(t-s)}\|u(\cdot,s)\|_{L^{3}(\Omega)}\,\mathrm{d}s$
$\displaystyle\leq c_{1}\|\nabla
v_{0}\|_{L^{q}(\Omega)}+c_{2}\sup_{s\in(0,T)}\|u(\cdot,s)\|_{L^{3}(\Omega)}\int_{0}^{T}\left(1+s^{-\frac{1}{2}-\frac{3}{2}(\frac{1}{3}-\frac{1}{q})}\right)\,\mathrm{d}s$

for all $t\in(0,T)$ and some $c_{1},c_{2}>0$. Since
$-\frac{1}{2}-\frac{3}{2}(\frac{1}{3}-\frac{1}{q})>-1$ and recalling Lemma
5.1, we conclude that there is $c_{3}>0$ such that $\|\nabla
v(\cdot,t)\|_{L^{q}(\Omega)}\leq c_{3}$ for all $t\in(0,T)$. Since $q>3$,
$W^{1,q}(\Omega)$ embeds continuously into $L^{\infty}(\Omega)$ and so the
above estimate in conjunction with Lemma 3.2 imply that
$\sup_{t\in(0,T)}\|v(\cdot,t)\|_{L^{\infty}(\Omega)}$ is finite as well.

Relying on the maximum principle and again on well-known semigroup estimates
(cf. [19, Lemma 1.3 (iv)]), we further estimate

$\displaystyle\mathrel{\hphantom{\leq}}\left\|u(\cdot,t)\right\|_{L^{\infty}(\Omega)}$
$\displaystyle\leq\|\mathrm{e}^{t\Delta}u_{0}\|_{L^{\infty}(\Omega)}+\int_{0}^{t}\left\|\mathrm{e}^{(t-s)\Delta}\nabla\cdot(u\nabla
v)(\cdot,s)\,\mathrm{d}s\right\|_{L^{\infty}(\Omega)}$
$\displaystyle\leq\left\|u_{0}\right\|_{L^{\infty}(\Omega)}+c_{4}\int_{0}^{t}\left(1+(t-s)^{-\frac{1}{2}-\frac{3}{2}(\frac{1}{r}-\frac{1}{\infty})}\right)\|(u\nabla
v)(\cdot,s)\|_{L^{r}(\Omega)}\,\mathrm{d}s$
$\displaystyle\leq\left\|u_{0}\right\|_{L^{\infty}(\Omega)}+c_{4}\sup_{s\in(0,t)}\|(u\nabla
v)(\cdot,s)\|_{L^{r}(\Omega)}\int_{0}^{T}\left(1+s^{-\frac{1}{2}-\frac{3}{2r}}\right)\,\mathrm{d}s$

for all $t\in(0,T)$ and some $c_{4}>0$. Since with
$\lambda\coloneqq\frac{rq}{q-r}$ and
$\theta\coloneqq\frac{\lambda-1}{\lambda}\in(0,1)$ we have

$\displaystyle\|(u\nabla v)(\cdot,s)\|_{L^{r}(\Omega)}$
$\displaystyle\leq\|u(\cdot,s)\|_{L^{\lambda}(\Omega)}\|\nabla
v(\cdot,s)\|_{L^{q}(\Omega)}$
$\displaystyle\leq\|u(\cdot,s)\|_{L^{\infty}(\Omega)}^{\theta}\|u(\cdot,s)\|_{L^{1}(\Omega)}^{1-\theta}\|\nabla
v(\cdot,s)\|_{L^{q}(\Omega)}$

for all $s\in(0,T)$, we see that there is $c_{5}>0$ such that

$\displaystyle\|u(\cdot,t)\|_{L^{\infty}(\Omega)}$ $\displaystyle\leq
c_{5}+c_{5}\sup_{s\in(0,t)}\|u(\cdot,s)\|_{L^{\infty}(\Omega)}^{\theta}$

for all $t\in(0,T)$. For $t\in(0,T)$, we set
$A(t)\coloneqq\sup_{s\in(0,t)}\|u(\cdot,t)\|_{L^{\infty}(\Omega)}$. Since
$\theta\in(0,1)$, by means of Young‘s inequality we obtain $A(t)\leq
c_{5}+c_{5}A^{\theta}(t)\leq c_{6}+\frac{1}{2}A(t)$ for some $c_{6}>0$ (not
depending on $t$) and hence also $A(t)\leq 2c_{6}$ for all $t\in(0,T)$. Taking
$t\nearrow T$ shows that also $u$ remains bounded in $\Omega\times(0,T)$. ∎

We are now in position to prove Theorem 1.1.

###### Proof of Theorem 1.1.

Suppose that $T_{\max}<\infty$, then Lemma 5.2 asserts boundedness of $u$ and
$v$ in $\Omega\times(0,T_{\max})$. However, this contradicts the extensibility
criterion in Lemma 3.1. ∎

##  6\. Conclusion

On the one hand, we obtained a condition which guarantees global existence of
solutions for the chemorepulsion system in three-dimensional space. This
result can be improved, as suggested to us by M. Winkler and an anonymous
referee, see Appendix B. On the other hand, we notice from our computations
that the concavity of the function $v$ would greatly simplify our argument. It
would lead to boundedness of the function $I(t)$, and hence to the global
existence of solutions, as shown in this paper. Indeed, we see from
$(\ref{notconvex})$ that if the function $v$ is concave, i.e., its Hessian is
negative-semidefinite,

$x^{T}\,D^{2}v\,x\leq 0\qquad\mbox{for every}\qquad x\in{\mathbb{R}}^{n},$

we have the following differential inequality for $I(t)$

$\displaystyle\dot{I}(t)$ $\displaystyle\leq 0.$

From the equation (1.1) we see that $v$ is not far from being concave. Taking
a simplified version of the equation for $v$ [assuming $v_{t}=0$ and
neglecting $v$ on the right-hand side of the second equation in (1.1)], we
have

$\Delta v=-u\leq 0,$

which would also hold if the Hessian of $v$ was negative-semidefinite.

Verifying concavity of a solution of a parabolic boundary value problem posed
on a convex domain has been studied before. In the context of one parabolic
equation of certain type, some positive results can be found in [10], for
example. However, we are not aware of any result in this direction for systems
of equations.

##  Appendix A A new inequality

As a byproduct of our arguments, we discovered a differential inequality
relating the second norm of the Hessian of the square-root of a positive
function with the dissipation of the Fisher information along the heat flow.
Due to the fact that both of these quantities appear in the calculation of the
evolution of the Fisher information along the heat flow, the following
inequality is interesting in its own right and may have further applications.
In particular, it was used by the first author in [3], where global-in-time
regular unique solution to a $1$D thermoelasticity system is obtained. Next, a
very recent application of the inequality (A.1) led to the interesting result
in the theory of $1$D combustion, see [13].

######  Lemma A.1.

Let $\Omega\subset{\mathbb{R}}^{n}$ be a smooth bounded domain. For every
positive function $u\in C^{2}(\bar{\Omega})$ with the boundary condition
$\left.\frac{\partial u}{\partial\nu}\right\rvert_{\partial\Omega}=0$, we have

(A.1) $\int_{\Omega}{\left\lvert{D^{2}\sqrt{u}}\right\rvert^{2}}\leq
C\int_{\Omega}{u\left\lvert{D^{2}\log{u}}\right\rvert^{2}},$

where $C=1+\frac{\sqrt{n}}{2}+\frac{n}{8}$.

######  Remark A.2.

We note that the inequality $(\ref{cieslakineq})$ does not hold pointwise,
i.e., there is no constant $C>0$ such that, for every positive $u\in
C^{2}(\bar{\Omega})$,

$\left\lvert{D^{2}\sqrt{u}}\right\rvert^{2}\leq
Cu\left\lvert{D^{2}\log{u}}\right\rvert^{2}\quad\text{in}\quad\bar{\Omega}.$

###### Proof of Lemma A.1.

We first note that

$\left[{D^{2}\sqrt{u}}\right]^{2}_{ij}=\left({\frac{\partial_{x_{i}x_{j}}u}{2u^{1/2}}-\frac{\partial_{x_{i}}u\partial_{x_{j}}u}{4u^{3/2}}}\right)^{2}=\frac{1}{4}\left({\frac{\partial_{x_{i}x_{j}}u}{u^{1/2}}-\frac{1}{2}\frac{\partial_{x_{i}}u\partial_{x_{j}}u}{u^{3/2}}}\right)^{2}$

and

$u\left[{D^{2}\log{u}}\right]^{2}_{ij}=u\left({\frac{\partial_{x_{i}x_{j}}u}{u}-\frac{\partial_{x_{i}}u\partial_{x_{j}}u}{u^{2}}}\right)^{2}=\left({\frac{\partial_{x_{i}x_{j}}u}{u^{1/2}}-\frac{\partial_{x_{i}}u\partial_{x_{j}}u}{u^{3/2}}}\right)^{2}$

in $\bar{\Omega}$. Using the simple fact that $\left({a+b}\right)^{2}\leq
2\left({a^{2}+b^{2}}\right)$, we get

$\displaystyle\left[{D^{2}\sqrt{u}}\right]^{2}_{ij}$
$\displaystyle=\frac{1}{4}\left({\frac{\partial_{x_{i}x_{j}}u}{u^{1/2}}-\frac{\partial_{x_{i}}u\partial_{x_{j}}u}{u^{3/2}}+\frac{1}{2}\frac{\partial_{x_{i}}u\partial_{x_{j}}u}{u^{3/2}}}\right)^{2}$
$\displaystyle\leq\frac{1}{4}\left[{2\left({\frac{\partial_{x_{i}x_{j}}u}{u^{1/2}}-\frac{\partial_{x_{i}}u\partial_{x_{j}}u}{u^{3/2}}}\right)^{2}+\frac{1}{2}\left({\frac{\partial_{x_{i}}u\partial_{x_{j}}u}{u^{3/2}}}\right)^{2}}\right]$
$\displaystyle=\frac{1}{2}{u\left[{D^{2}\log{u}}\right]^{2}_{ij}+\frac{1}{8}\left({\frac{\partial_{x_{i}}u\partial_{x_{j}}u}{u^{3/2}}}\right)^{2}}.$

Therefore, we obtain

$\displaystyle{\left\lvert{D^{2}\sqrt{u}}\right\rvert^{2}}$
$\displaystyle={\sum_{i,j=1}^{n}{\left[{D^{2}\sqrt{u}}\right]^{2}_{ij}}}\leq{\sum_{i,j=1}^{n}{\left[{\frac{1}{2}{u\left[{D^{2}\log{u}}\right]^{2}_{ij}+\frac{1}{8}\left({\frac{\partial_{x_{i}}u\partial_{x_{j}}u}{u^{3/2}}}\right)^{2}}}\right]}}$
(A.2)
$\displaystyle=\frac{1}{2}{u\left\lvert{D^{2}\log{u}}\right\rvert^{2}}+\frac{1}{8}{\frac{\left\lvert{\nabla
u}\right\rvert^{4}}{u^{3}}}\quad\text{in}\quad\bar{\Omega}.$

Applying Lemma 2.3 to $(\ref{cieslakfinal})$, we get

$\displaystyle\int_{\Omega}{\left\lvert{D^{2}\sqrt{u}}\right\rvert^{2}}\leq\left({\frac{1}{2}+\frac{1}{8}\left({2+\sqrt{n}}\right)^{2}}\right)\int_{\Omega}{u\left\lvert{D^{2}\log{u}}\right\rvert^{2}},$

as required. ∎

##  Appendix B Alternative proof

This section is devoted to the presentation of the result communicated to us
by M. Winkler as well as one of the anonymous referees. It gives an
alternative conditional result, extending our Theorem 1.1. On the one hand, no
convexity of the domain is required, on the other hand, only a zero order
estimate of $u$ is required.

It is based on the well-known fact that in dimension $3$, bounding the
$L^{\infty}((0,T_{\max});L^{p}(\Omega))$ norm of $u$ for any $p>\frac{3}{2}$
allows prolongation of a solution to (1.1), see for instance [2].

######  Lemma B.1.

Let $\Omega\subset{\mathbb{R}}^{3}$ be a smooth, bounded domain and let
$u_{0},v_{0}$ be as in (3.2). Suppose that the maximal existence time
$T_{\max}$ of the solution $(u,v)$ to (1.1)–(1.3) given by Lemma 3.1 is
finite. Then

(B.1)
$\displaystyle\int_{0}^{T_{\max}}{\left\lVert{u(t)}\right\rVert_{L^{3}(\Omega)}^{2}\,\mathrm{d}t}=\infty.$

In particular,

$\int_{0}^{T_{\max}}{\left\lVert{\nabla\sqrt{u(t)}}\right\rVert_{L^{2}(\Omega)}^{4}\,\mathrm{d}t}=\infty.$

###### Proof.

In view of the Gagliardo–Nirenberg inequality and conservation of mass, we
only need to verify (B.1). To this end, we suppose that (B.1) does not hold,
which allows us to first apply maximal Sobolev regularity theory to the second
equation in (1.1) to see that

(B.2) $\displaystyle\int_{\frac{T_{\max}}{2}}^{T_{\max}}{\left\lVert{\Delta
v(t)}\right\rVert_{L^{3}(\Omega)}^{2}\,\mathrm{d}t}<\infty.$

Then, we use the first equation in (1.1) along with an integration by parts to
get

$\displaystyle\frac{1}{2}\frac{\mathrm{d}}{\mathrm{d}t}\int_{\Omega}{u^{2}}+\int_{\Omega}{\left\lvert{\nabla
u}\right\rvert^{2}}+\int_{\Omega}{u^{2}}$
$\displaystyle\leq\frac{1}{2}\int_{\Omega}{u^{2}\Delta
v}+\int_{\Omega}{{u}^{2}}$
$\displaystyle\leq\frac{1}{2}\left\lVert{u}\right\rVert_{L^{3}(\Omega)}^{2}\left\lVert{\Delta
v}\right\rVert_{L^{3}(\Omega)}+\int_{\Omega}{{u}^{2}}$

in $(0,\infty)$. Next, applications of the Gagliardo–Nirenberg interpolation
and Young’s inequality give for all $t\in(0,T_{\max})$,

$\displaystyle\frac{1}{2}\frac{\mathrm{d}}{\mathrm{d}t}\int_{\Omega}{u^{2}}+\int_{\Omega}{\left\lvert{\nabla
u}\right\rvert^{2}}+\int_{\Omega}{u^{2}}\leq\int_{\Omega}{\left({\left\lvert{\nabla
u}\right\rvert^{2}+u^{2}}\right)}+c_{1}\left({\left\lVert{\Delta
v}\right\rVert_{L^{3}(\Omega)}^{2}+1}\right)\left\lVert{u}\right\rVert_{L^{2}(\Omega)}^{2}$

for some constant $c_{1}>0$.

Writing $y(t)\coloneqq\int_{\Omega}{u^{2}(\cdot,t)}$ and $h(t)\coloneqq
2c_{1}\left({\left\lVert{\Delta
v(\cdot,t)}\right\rVert_{L^{3}(\Omega)}^{2}+1}\right)$ for $t\in(0,T_{\max})$,
we arrive at a differential inequality

(B.3) $\dot{y}(t)\leq h(t)y(t)\qquad\mbox{for all }t\in(0,T_{\max}).$

Since $h\in L^{1}(\frac{T_{\max}}{2},T_{\max})$ by (B.2), integration in time
of (B.3) shows boundedness of $u$ in $L^{\infty}((0,T_{\max});L^{2}(\Omega))$,
contradicting finiteness of $T_{\max}$. ∎

## Acknowledgments

T.C. was supported by the National Science Center of Poland grant SONATA BIS 7
number UMO-2017/26/E/ST1/00989. K.H. was partially supported by the National
Science Center of Poland grant SONATA BIS 10 number UMO-2020/38/E/ST1/00469.
M.S. was supported by the National Science Center of Poland grant SONATA BIS
10 number UMO-2020/38/E/ST1/00596. The authors are grateful to both anonymous
referees as well as to Michael Winkler for careful reading of the manuscript
and suggestions leading to an essential improvement of our results. In
particular, Appendix B was observed by M. Winkler and the anonymous referee.

## References

  * [1] Bakry, D. and Émery, M. Diffusions hypercontractives. Séminaire de probabilités, XIX, 1983/84, pp. 177–206, Lecture Notes in Math., 1123, Springer, Berlin, 1985.
  * [2] Bellomo, N., Bellouquid, A., Tao, Y. and Winkler, M. Toward a mathematical theory of Keller–Segel models of pattern formation in biological tissues. Math. Models Methods Appl. Sci. 25, 09 (2015), 1663–1763.
  * [3] Bies, P.M., Cieślak, T. Global-in-time regular unique solutions with positive temperature to the $1$d thermoelasticity, to appear in SIAM J. Math. Anal., arXiv:2303.12522.
  * [4] Cieślak, T., and Fujie, K. Some remarks on well-posedness of the higher-dimensional chemorepulsion system. Bull. Pol. Acad. Sci. Math. 67, 2 (2019), 165–178.
  * [5] Cieślak, T., Laurençot, Ph., and Morales-Rodrigo, C. Global existence and convergence to steady states in a chemorepulsion system. In Parabolic and Navier–Stokes equations. Part 1, vol. 81 of Banach Center Publ. Polish Acad. Sci. Inst. Math., Warsaw, 2008, pp. 105–117.
  * [6] Evans, L. C. Entropy and Partial Differential Equations. CreateSpace Independent Publishing Platform, 2014.
  * [7] Freitag, M. Global existence and boundedness in a chemorepulsion system with superlinear diffusion. Discrete Contin. Dyn. Syst. 38, 11 (2018), 5943–5961.
  * [8] Friedman, A. Partial differential equations, original ed. Robert E. Krieger Publishing Co., Huntington, N.Y., 1976.
  * [9] Fuest, M. Global solutions near homogeneous steady states in a multidimensional population model with both predator- and prey-taxis. SIAM J. Math. Anal. 52, 6 (2020), 5865–5891.
  * [10] Korevaar, N.J. Convex solutions to nonlinear elliptic and parabolic boundary value problems. Indiana Univ. Math. J. 32 (1983), no. 4, 603–614.
  * [11] Ladyzenskaja, O. A. Solution ’’in the large‘‘ of the nonstationary boundary value problem for the Navier–Stokes system with two space variables. Comm. Pure Appl. Math. 12 (1959), 427–433.
  * [12] Lankeit, J., Winkler, M. Facing low regularity in chemotaxis systems. Jahresber. Dtsch. Math.-Ver. 122 (2020), 35–64.
  * [13] Li, S., Yang, J. Characterisations for the depletion of reactant in a one-dimensional dynamic combustion model, arXiv:2308.16506.
  * [14] Lin, K. Mu, C. Global existence and convergence to steady states for an attraction-repulsion chemotaxis system. Nonlinear Anal. Real World Appl. 31 (2016), 630–642.
  * [15] Luca, M., Chavez-Ross, A. Edelstein-Keshet, L., Mogilner, A. Chemotactic signaling, microglia, and Alzheimer’s disease senile plaques: Is there a connection? Bull. Math. Biol. 65, (2003), 693–730.
  * [16] Nagai, T., Seki, Y., Yamada, T. Global existence of solutions to a parabolic attraction-repulsion chemotaxis system in ${\mathbb{R}}^{2}$: the attractive dominant case. Nonlinear Anal. Real World Appl. 62 (2021), Paper No. 103357, 16 pp.
  * [17] Tao, Y. Global dynamics in a higher-dimensional repulsion chemotaxis model with nonlinear sensitivity. Discrete Contin. Dyn. Syst. Ser. B 18, 10 (2013), 2705–2722.
  * [18] Tao, Y., and Wang, Z.-A. Competing effects of attraction vs. repulsion in chemotaxis. Math. Models Methods Appl. Sci. 23, 1 (2013), 1–36.
  * [19] Winkler, M. Aggregation vs. global diffusive behavior in the higher-dimensional Keller–Segel model. J. Differ. Equ. 248, 12 (2010), 2889–2905,
  * [20] Winkler, M. Global large-data solutions in a chemotaxis-(Navier–)Stokes system modeling cellular swimming in fluid drops. Comm. Partial Differential Equations 37, 2 (2012), 319–351.


# Continual Barlow Twins: continual self-supervised learning for remote
sensing semantic segmentation

Valerio Marsocci, Simone Scardapane Valerio Marsocci is with the Department of
Computer, Control and Management Engineering Antonio Ruberti (DIAG), Sapienza
University of Rome, 00185, Rome, Italy, mail:
<EMAIL_ADDRESS>Scardapane is with the Department of
Information Engineering, Electronics and Telecommunication (DIET), Sapienza
University of Rome, 00184, Rome, Italy, mail<EMAIL_ADDRESS>
###### Abstract

In the field of Earth Observation (EO), Continual Learning (CL) algorithms
have been proposed to deal with large datasets by decomposing them into
several subsets and processing them incrementally. The majority of these
algorithms assume that data is (a) coming from a single source, and (b) fully
labeled. Real-world EO datasets are instead characterized by a large
heterogeneity (e.g., coming from aerial, satellite, or drone scenarios), and
for the most part they are unlabeled, meaning they can be fully exploited only
through the emerging Self-Supervised Learning (SSL) paradigm. For these
reasons, in this paper we propose a new algorithm for merging SSL and CL for
remote sensing applications, that we call Continual Barlow Twins (CBT). It
combines the advantages of one of the simplest self-supervision techniques,
i.e., Barlow Twins, with the Elastic Weight Consolidation method to avoid
catastrophic forgetting. In addition, for the first time we evaluate SSL
methods on a highly heterogeneous EO dataset, showing the effectiveness of
these strategies on a novel combination of three almost non-overlapping
domains datasets (airborne Potsdam dataset, satellite US3D dataset, and drone
UAVid dataset), on a crucial downstream task in EO, i.e., semantic
segmentation. Encouraging results show the superiority of SSL in this setting,
and the effectiveness of creating an incremental effective pretrained feature
extractor, based on ResNet50, without the need of relying on the complete
availability of all the data, with a valuable saving of time and resources.

###### Index Terms:

Self-supervised learning, continual learning, semantic segmentation, remote
sensing

##  I Introduction

Figure 1: t-Stochastic Neighbor Embedding (t-SNE) visualization results of the
features of three selected RS datasets (see Section IV for a description of
the datasets). It can easily be seen that the images from the different
settings can be considered independent but non-identically distributed.

In recent years, improvements in speed and acquisition technologies have
drastically increased the amount of available Earth Observation (EO) images.
These improvements bring challenging issues to the widespread use of Remote
Sensing (RS) classification and semantic segmentation techniques, due to (a)
the continuous arrival of new data, possibly belonging to partially
overlapping domains, (b) the sheer size of the datasets, requiring vast
amounts of processing power, and (c) the increasing quantity of data which has
not been labeled by a domain expert. The main aim of this paper is to propose
an algorithm to deal with these three characteristics simultaneously, that we
call Continual Barlow Twins (CBT). This algorithm combines the strengths of
two separates lines of research: Continual Learning (CL) for processing a
large heterogeneous dataset in an incremental way, satisfying constraints (a)
and (b) above, and Self-Supervised Learning (SSL) to deal with the lack of
labeling information, satisfying constraint (c) above. In the following, we
describe briefly the two issues separately, before introducing our proposed
solution.

### Problem #1: EO datasets are heterogeneous

Consider the situation where we trained a semantic segmentation network on a
dataset of Italian satellite images. If we receive a new labeled dataset of
similar images from a different nation, ideally, we would like our network to
be able to segment equally well images coming from the two countries. However,
most of the algorithms developed for classification or segmentation in EO
suffer from catastrophic forgetting problems in this context, requiring to
discard the acquired knowledge to retrain the model from scratch on the
combination of the two datasets [1]. In a wide range of EO applications, the
strategy of retraining the whole model is computationally expensive and
costly. Therefore, there is a need to ensure that the newly-developed models
have the ability to learn new tasks while retaining satisfying performance on
previous ones. The cause of the catastrophic forgetting problem is that
different tasks or datasets are independent but not identically distributed in
the feature domain, as it is known that the distribution of various RS
datasets vary greatly [2], due to different resolutions, acquisitions,
textures, and captured scenes. This is even more evident in urban scenes and
in datasets made of images acquired from different types of sensors, e.g.,
drone, airborne, satellite (see Fig. 1 for a visualization of this
phenomenon).

In this paper, we leverage a CL algorithm [1] to mitigate catastrophic
forgetting problem and allow our algorithm to generalize to different feature
distributions without the requirement of accessing already seen data.

### Problem #2: EO datasets are largely unlabeled

Most methods, especially in EO applications, are framed as supervised systems,
relying on annotated data. More than in other fields, for drone, aerial and
satellite images, it is difficult to rely on a labeled dataset, in light of
the high cost and the amount of effort and time that are required, along with
domain expertise. In computer vision (CV), SSL has been proposed to handle
this problem, reducing the amount of annotated data needed [3, 4]. The goal of
SSL is to learn an effective visual representation of the input using a
massive quantity of data provided without any label [5]. We can see the task
as the need to build a well-structured and relevant set of features, able to
represent an image which is useful for several downstream tasks. There is a
growing research line demonstrating how SSL techniques increase performance in
EO applications [6], although evaluations have been limited to a single
dataset or domain.

Figure 2: Schematic representation of the Continual Barlow Twins algorithm.
When computing $\mathcal{L}_{\mathcal{C}\mathcal{B}\mathcal{T}}$, $C$, and $I$
contribute to the Barlow Twins loss term, $f_{\theta,T1}$, $F$, and
$f_{\theta,T2}$ to the $EWC$ regularization term.

### Contributions of the paper

In this paper, we propose and experimentally evaluate a novel algorithm that
is able to train a deep network for RS by exploiting vast amounts of
heterogeneous, unlabeled, continually-arriving data. Specifically, we show
that it is possible to exploit the potential of SSL incrementally, to obtain
an efficient and effective pretrained model trained in several successive
steps, without the need to re-train it from scratch every time new data is
added [7]. The proposed Continual Barlow Twins (CBT) algorithm trains a
feature extractor (ResNet50) with Barlow Twins (BT) [3], whose loss is
integrated with a regularization term, borrowed from Elastic Weight
Consolidation (EWC) [8], to avoid catastrophic forgetting. With the obtained
feature extractor, we train a UNet++ [9] to perform semantic segmentation.
When acquiring new RS data, CBT can be trained quickly, as it will be
necessary to update it on the new data only, discarding all old data. Our
method also provides computational efficiency, potentially allowing small
realities to train a large model on huge amounts of data that would be
unfeasible otherwise.

Since the generalisation capabilities and benefits of SSL on RS data from non-
overlapping domains (as shown in Fig. 1) is still unexplored, we also propose
a new benchmark by combining three datasets with images captured with
different sensors (drone, airborne and satellite data), with different
resolutions and acquired under different conditions, representing different
objects and scenes. We show that SSL targeted to RS images can outperform
standard pre-trained strategies (e.g., ImageNet), and we expect this to become
a useful benchmark scenario for further research in SSL and CL in remote
sensing. We also show that the proposed CBT algorithm offers significantly
more versatility and less computing time compared to a standard approach.

##  II Related Works

In this section, we briefly review the relevant literature on SSL (Section
II-A) and CL (Section II-B) in EO. We note that in CV, the combination of
these two technologies is starting to be explored, and some works have already
started to demonstrate how SSL methods are well prepared, after small
modifications, to learn incrementally [7, 10]. On the other hand, in EO, this
combination has not yet been explored, to our knowledge, apart from a few
embryonic contributions combining weakly-supervision with CL [11].

###  II-A Self-supervised Learning in EO

In [12], the authors train CMC [4] on three large datasets both with RGB and
multispectral bands. Then, they evaluate the effectiveness of the learned
features on four datasets to solve downstream tasks of both single-label and
multi-label classification. The same authors, in [6], applies a split-brain
autoencoder on aerial images. In [13], the authors perform a semantic
segmentation downstream task on the Vaihingen dataset [14] to learn the
features of the encoder of the net that solves the segmentation. A different
approach is proposed in [15], where the authors adopt as contrastive strategy
the use of RS images of the same areas in different time frames, introducing a
loss term based on the geolocation of the tiles. Other contrastive strategies
are proposed by [16] and [17]. [18] learns visual representations inferring
information on the visible spectrum from the other bands on BigEarthNet [19].
Finally in [20], the authors propose a net that, imitating the discriminator
of a generative adversarial network (GAN), identifies patches taken from two
temporal images. A similar approach, with multi-view images, is proposed in
[21].

###  II-B Continual Learning in EO

In [22], the authors proposed a two-block network for RS land cover
classification tasks, where one module minimizes the error among classes
during the new task training, and another module learns how to effectively
distinguish among tasks, based on representing past data stored in a linear
memory. Similarly, [23] proposes a framework based on two purposes: adapting
and remembering. Concerning the former, the authors save a copy of the
trained-on-the-previous task net, to store the info of the already seen
classes. The latter stores some old data, which feed the net during the
sequential training steps. Shaped for semantic segmentation, [2] propose two
regularization components: representation consistency structure loss and pixel
affinity structure loss. The first retains the information in the isolated
pixels. The second saves the high-frequency information, shared throughout the
tasks.

##  III Methodology

###  III-A Overview of the components

We consider a RS scenario where (a) data is coming incrementally from multiple
domains (e.g., drone, airborne and/or satellite images); (b) we cannot re-
train from scratch the model when new data is received; (c) the majority of
the data is unlabeled. We refer to each domain (or subset of the dataset) as a
task, in accordance with the CL literature. To achieve our compound objective,
the intuition is to embed a CL strategy in a self-supervised framework, by
combining two algorithms that are considered state-of-the-art in their
respective fields: (i) Barlow Twins (BT) [3], which trains a network based on
measuring the cross-correlation matrix between the outputs of two identical
networks fed with distorted versions of a sample, and making it as close to
the identity matrix as possible; (ii) Elastic Weight Consolidation (EWC) [8]
consisting of constraining important weights of the network to stay close to
the value obtained in previous tasks. In the next section, we highlight in
detail how Continual Barlow Twins (CBT) works. A schematic overview of the
method is provided in Fig. 2.

###  III-B Continual Barlow Twins

Consider for now a single task, and denote by $X$ a batch of unlabeled images.
Our main training step, taken from BT, produces two disturbed views of $X$,
$Y_{A}$ and $Y_{B}$, based on a set of data augmentations strategies $S$
(e.g., random rotations and scalings). In this paper we consider standard sets
of data augmentations (see Section V), although augmentations specific to RS
could also be considered. The two views are fed to a convolutional neural
network with weights $\theta$, that produces, respectively, two embeddings
$Z_{A}$ and $Z_{B}$ (assumed to be mean-centered along the batch dimension).
To learn effective representations of the input images in a self-supervised
fashion we leverage the BT loss, which is composed of two terms called
invariance and redundancy reduction terms:

$\mathcal{L}_{\mathcal{B}\mathcal{T}}(X)=\underbrace{\sum_{i}\left(1-\mathcal{C}_{ii}\right)^{2}}_{\text{invariance
term }}+\mu\underbrace{\sum_{i}\sum_{j\neq
i}\mathcal{C}_{ij}^{2}}_{\text{redundancy reduction term }}$ (1)

where $\mu$ is a positive constant balancing the invariant and the redundancy
reduction terms of the loss, and $\mathcal{C}$ is the cross-correlation
matrix, with values comprised between -1 (i.e. total anti-correlation) and 1
(i.e. total correlation), computed between the outputs of the two identical
networks along the batch dimension. Practically, the first term of the loss
has the goal to make the diagonal elements of $\mathcal{C}$ equal to 1. In
this way, the embeddings become invariant to the applied augmentations. On the
other hand, the second term of the loss has the aim to bring to 0 the off-
diagonal elements of $\mathcal{C}$. This ensures that the various components
of the embeddings are decorrelated with each other, making the information
non-redundant, enhancing the representations of the images.

Suppose now that the network has been trained on images coming from a task
$T_{1}$ using the loss (1) (e.g., drone images), and we receive a new dataset
of images coming from a second task $T_{2}$ (e.g., satellite images). We
denote the weights obtained at the end of the first training as
$\theta^{T_{1}}$, and the data of the two tasks respectevely as $D_{T_{1}}$
and $D_{T_{2}}$. To retain old knowledge from $T_{1}$ and avoid catastrophic
forgetting, we complement the BT loss (1) with a EWC regularization term [8]
which forces the weights to stay close to $\theta^{T_{1}}$ depending on their
importance, given by the diagonal of the Fisher information matrix $F$, which
is a positive semidefinite matrix corresponding to the second derivative of
the loss near the minimum. In our scenario, the loss cannot be decomposed for
each individual data point, as it depends on the cross-correlations between
data in a mini-batch and its corresponding augmentations. To this end,
denoting by $B_{T_{1}}$ the number of mini-batches $X$ that can be extracted
from $D_{T_{1}}$, we approximate the $i$-th element of the diagonal Fisher
information matrix as:

$F_{i}=\frac{1}{B_{T_{1}}}\sum_{X\in
D_{T_{1}}}\left[\frac{\partial\mathcal{L}_{\mathcal{B}\mathcal{T}}(X)}{\partial\theta_{i}^{T_{1}}}\right]^{2}$
(2)

where $\mathcal{L}_{\mathcal{B}\mathcal{T}}(X)$ denotes the BT loss computed
on mini-batch $X$, as in (1). Intuitively, each weight of the network is given
an importance that depends on the square of the corresponding loss gradient.
Given this approximation, the new loss for a batch of images $X$ taken from
the second task is given by:

$\mathcal{L}(X)=\mathcal{L}_{\mathcal{B}\mathcal{T}}(X)+\sum_{i}\frac{\lambda}{2}F_{i}\left(\theta_{i}-\theta^{T_{1}}_{i}\right)^{2}$
(3)

where $\mathcal{L}_{\mathcal{B}\mathcal{T}}(X)$ is the BT loss (1) computed on
the data from task $T2$, and $\lambda$ weights the constraint on the previous
task. If moving to a third task, we repeat the computation of the Fisher
information matrix at the end of training for the second task and replace it.
The CBT approach is summarized in Figure 2, and the associated code is
available online.111https://github.com/VMarsocci/CBT After the self-supervised
pre-training, the network can be exploited for any downstream task of interest
in EO. In particular, we explore in Section V a fine-tuning for a semantic
segmentation task.

##  IV Datasets

To perform the experiments we build a novel dataset which is a combination of
three previously introduced datasets. Each contains images from a different
source: airborne, satellite, and drone. As previously stated, the construction
of a novel mixed dataset is crucial since the data is vastly heterogeneous,
presenting almost non-overlapping domains, as shown in Fig. 1. In fact, the
choice was dictated by the desire to demonstrate the effectiveness of the SSL
on a challenging task (that is semantic segmentation), extending its validity
even in the case of highly variable data, while most previous works focused on
a single domain (see Section II). We briefly summarize next each dataset.
Salient information are summed up in Table I.

TABLE I: Summary of the datasets used for the experiments. Dataset | Type of images | Number of images
---|---|---
Potsdam [14] | Aerial |  $\sim$5000
UAVid [24] | UAV |  $\sim$7500
US3D [25] | Satellite |  $\sim$11000

####  IV-1 Potsdam

The ISPRS Potsdam dataset [14] consists of 38 high-resolution aerial true
orthophoto (TOP), with four available bands (Near-Infrared, Red, Green and
Blue). Each image is $6000\times 6000$ pixels, with a Ground Sample Distance
(GSD) of 5 cm, ending up in covering 3.42 $km^{2}$. For our experiments, we
took in consideration only the 38 RGB TOPs. These are annotated with pixel-
level labels of six classes: background, impervious surfaces, cars, buildings,
low vegetation, trees. We used the eroded mask, and we selected 24 images for
training, 13 for testing and 1 for validation, without considering the
background class, similarly to [26]. We cropped each image in $512\times 512$
non-overlapping patches, ending up in 2640 images for training, 120 for
validation and 1680 for test.

####  IV-2 UAVid

Unmanned Aerial Vehicle semantic segmentation dataset (UAVid) [24] consists in
42 video sequences, captured with 4K high-resolution by an oblique point of
view. UAVid is a challenging dataset due to the very high resolution of
images, large-scale variation, and complexities in the scenes. The authors
extracted ten labelled images per each sequence, ending up in 420 images with
$3840\times 2160$ pixels. The annotated classes are 8: building, road, static
car, tree, low vegetation, human, moving car, background clutter. The images
are already divided into train, validation and test, by the authors, however
the test segmentation maps have not yet been released. For this reason, we
used a part (80%) of the validation set as test set in our experiments.
Moreover, we cropped the images in $512\times 512$ non-overlapping patches,
ending up in $\sim 7500$ images.

####  IV-3 US3D

The US3D dataset [25] includes approximately 100 $km^{2}$ coverage for the
United States cities of Jacksonville, Florida and Omaha, Nebraska. Sources
include incidental satellite images, airborne LiDAR, and feature annotations
derived from LiDAR. The dataset is composed of 2783 images, $1024\times 1024$,
obtained from the WorldView-3 satellite: they are non-orthorectified and
multi-view. The images have 8 bands, six of which are part of the visible
spectrum, and two of the near infrared. Semantic labels for the US3D dataset,
derived automatically from Homeland Security Infrastructure Program (HSIP),
are five: ground, trees, water, building and clutter. For our experiments we
considered only RGB bands and all the classes. Also for this dataset, we
cropped the images in $512\times 512$ non-overlapping patches, ending up in
more than 11000 images, randomly divided in train ($\sim 70\%$), validation
($\sim 10\%$) and test ($\sim 20\%$).

##  V Experimental Setup

For the training phase, a single Tesla V100-SXM2 32 GB GPU has been used. For
the semantic segmentation task, we use UNet++ [9], with the Squeeze and
Excitation strategy [27] and the softmax function as activation on the last
layer. For the experiments on all the three datasets, we fix the batch size to
8, the number of epochs to 200 and the learning rate to 0.0001. Moreover, we
used Adam as optimizer, Jaccard loss as the cost function and the following
set of augmentations: random horizontal flip, random geometric transformation
(i.e. shifting, scaling, rotating), random gaussian noise, random radiometric
transformation (i.e. brightness, contrast, saturation variations). The overall
accuracy (OA), mean intersection over union (mIoU), and F1-score (F1) are the
selected evaluation metrics. Under these conditions, we test different pre-
training strategies for UNet++. As baselines, we use ResNet50 pretrained on
ImageNet in a supervised manner, and ResNet50 pretrained on ImageNet data with
BT222Downloaded at https://github.com/facebookresearch/barlowtwins. For the
proposed CBT approach, we pretrain ResNet50 incrementally on the three
datasets in this order: US3D, UAVid, Potsdam (with $\lambda=10e-2)$. Then, we
left the rest of parameters as in [3]. As additional baseline, we consider
ResNet50 pretrained on the three datasets with standard BT. We used the same
set of parameters provided in [3]. With these encoders, we trained the
semantic segmentation models in a supervised way, with different percentages
(10%, 50%, 100%) of labeled data for the three datasets. The results are shown
and commented in the next Section (Sec. VI).

##  VI Experimental Results

Figure 3: mIoU metrics on experiments with an increasing amount of labeled
data of respectevely a) UAVid, b) US3D and c) Potsdam.

The results of the experiments on the downstream task are shown both in Figure
3 and Tables III, IV, V. As can be seen, the feature extractors that perform
best are the ones obtained from the CBT and BT training on the three selected
EO datasets. Precisely, these conformations outperform, with reference to
mIoU, their counterpart trained with ImageNet supervised pretraining
respectively by 3.45% and by 3.75% in average. Moreover, it is interesting to
note that the performance of the models with the CBT feature extractor is only
slightly lower (average drop of $\approx$0.3%, referring to mIoU) than that
obtained with an encoder trained by means of BT, demonstrating how the
proposed approach gives up only a slight part of optimal performance, against
clear advantages in terms of computational efficiency and general versatility.
In absolute terms, it is necessary to notice once again how self-supervision
strategies lead to better results than exclusively supervised ones [12, 15],
and, above all, how this is even more true when combining EO data from domains
that are not homogeneous in terms of type of sensor, acquisition, resolution
and objects represented. In particular, in the next paragraphs, we will go in
the depth of some specific evidences regarding: computational times (Sec.
VI-A), UAVid experiments (Sec. VI-B), Potsdam experiments (Sec. VI-C) and US3D
experiments (Sec. VI-D).

###  VI-A Computational times

As stated earlier, one of the best advantages of the proposed new method is
the shorter computational time with a very limited performance drop in the
case of incremental data availability. As already stressed, this situation is
especially likely in the field of EO, where new data often arrives
continuously [23], due to satellite revisit times, scheduled acquisition
campaigns and other variable parameters. In Table II we can observe the
results of the experiments. Concerning the traditional BT strategy, for the
training of the three considered datasets, we simulate an incremental arrival
of data as follows:

  1. (1)

training of the only US3D;

  2. (2)

joint training of US3D and UAVid;

  3. (3)

training of the three datasets together.

This strategy is employed to create a salient feature extractor for all
datasets, with the mean of avoiding catastrophic forgetting, in situations of
incremental data availability. On the other hand, for CBT, step 1) is referred
to training on US3D, step 2) on UAVid and step 3) on Potsdam, as previously
stated. We can easily affirm, observing Table II, that our method could save
nearly 50% of times, when the data are available incrementally. It is also
interesting to state that, also in case of complete and immediate availability
of all data, the computational times are comparable (28.75 h for CBT vs 24.61
h for BT, where the computational times of the latter consist of just the
third step).

TABLE II: Elapsed training times. CBT offers important advantages when data are provided in an incremental fashion. Strategy | Step 1 (s) | Step 2 (s) | Step 3 (s) | Tot (h)
---|---|---|---|---
BT | 41400 | 61500 | 88600 | 53.19
CBT | 42000 | 36400 | 25100 | 28.75

###  VI-B UAVid

According to the results shown in Table III and represented in Figure 3 a),
when using a limited amount of data, the performance of the supervised
pretrained encoder, still being inferior overall, do not differ a lot from
those obtained with the self-supervised encoders. Particularly, the training
with 50% of data are the most similar along the different pretrained encoders,
with just $\sim$0.5% mIoU gap between the the worst result (74.93% mIoU,
obtained with ImageNet encoder) and the best (75.42% mIoU, achieved with BT
encoder). This trend can be mainly explained by what has been stated above
with respect to the image domain. In fact, the images captured by drone are
definitely more similar to close-range camera taken images, like ImageNet
ones, than those from the other two RS datasets. This is mainly due to the
point of view from which the images were captured. For Potsdam and US3D, the
viewpoint is almost nadiral, while for UAVid it is oblique, more precisely the
camera angle is set to around 45 degrees to the vertical direction, at a
flight height of about 50m. As also stated by the authors [24], a non-nadiral
view allows easier reconstruction of object geometry (i.e. volume, shape,
etc…), making the use of more sophisticated feature extractors less effective.
These reasons favour the high performance of ImageNet pretrained models,
especially with a limited number of labels. However, by increasing the number
of labels, the models with encoders trained on the proposed datasets are able
to match their features to the best conformation to solve the task with the
best performance (80.62% mIoU with respect to 77.84% mIoU of the ImageNet
encoder experiment). In addition, the fact that the pretraining on UAVid was
the second of the three steps slightly affected the performance of the CBT
pretraining strategy (80.13% mIoU), with a very limited drop in performance
($\sim$0.5%).

TABLE III: UAVid results for different % of training data. The highest score is marked in red bold. The second highest in blue bold. Encoder | % | OA | mIoU | F1
---|---|---|---|---
| 10% | 95.04 | 69.78 | 80.34
ImageNet | 50% | 96.28 | 74.93 | 84.63
| 100% | 96.61 | 77.84 | 86.67
| 10% | 95.3 | 71.01 | 81.33
BT ImageNet | 50% | 96.31 | 75.19 | 84.83
| 100% | 96.68 | 78.49 | 86.72
| 10% | 95.38 | 71.47 | 81.52
CBT | 50% | 96.00 | 75.24 | 84.71
| 100% | 96.95 | 80.13 | 88.44
| 10% | 95.41 | 71.61 | 81.66
BT | 50% | 96.34 | 75.42 | 84.81
| 100% | 97.17 | 80.62 | 88.43

###  VI-C Potsdam

As far as the results on Potsdam are concerned, in Table IV and Figure 3 c),
we see that the gap between results with self-supervised (71.4% mIoU) and
supervised encoder (64.09% mIoU) is the largest among all experiments. This
trend is true also for the experiments with a limited amount of training data.
For example, with 10% of data, the gap between the ImageNet encoder (58.31%
mIoU) and BT encoder (62.7% mIoU) is $\sim$4.4%. This can be explained by the
fact that this dataset is the one with the least amount of data among the
three available (see Table I). This insight is supported by the fact that the
gap between performance with ImageNet encoders and performance with CBT and BT
encoders is wider also for the other datasets when only 10% of the data is
used. Therefore, it is definitely the one that benefits the most from a more
efficient encoder feature selection. This could be confirmed by the fact that
the Potsdam domain is comparable with that of US3D, a very wide dataset,
capable of improving the representations that can be used during the training
of the Potsdam semantic segmentation, confirming similar intuitions reached,
for example, in [28].

TABLE IV: Potsdam results for different % of training data. The highest score is marked in red bold. The second highest in blue bold. Encoder | % | OA | mIoU | F1
---|---|---|---|---
| 10% | 89.05 | 58.31 | 70.69
ImageNet | 50% | 90.12 | 61.66 | 73.51
| 100% | 91.05 | 64.09 | 75.91
| 10% | 90.04 | 61.47 | 73.28
BT ImageNet | 50% | 91.61 | 66.86 | 77.43
| 100% | 92.36 | 70.12 | 79.46
| 10% | 89.91 | 60.87 | 72.43
CBT | 50% | 91.81 | 67.21 | 77.87
| 100% | 92.43 | 70.87 | 79.98
| 10% | 90.49 | 62.7 | 74.44
BT | 50% | 91.84 | 67.28 | 77.73
| 100% | 92.84 | 71.4 | 80.64

###  VI-D US3D

As far as the US3D dataset is concerned, again self-supervision techniques
lead to better results on downstream tasks, highlighting how, in this case,
CBT performs best of all (CBT 84.56% vs ImageNet 83.32% mIoU). In fact, this
is the first evidence that can be easily argued from Table V and Figure 3 b).
Therefore, we observe that there are no profound differences in performance
between the other encoders (BT ImageNet 84.41% vs CBT 84.56% vs BT 84.48%
mIoU), since the US3D is a large dataset, composed of several images of the
same area but captured from different points of view (i.e. multiview). This
redundancy, working as data augmentation itself, facilitates the resolution of
the task on this dataset, as once an efficient feature extractor is engaged,
convergence is achieved quite effectively. It is not surprising that similar
results are presented in [12], where self-supervision is applied on other
large datasets.

TABLE V: US3D results for different % of training data. The highest score is marked in red bold. The second highest in blue bold. Encoder | % | OA | mIoU | F1
---|---|---|---|---
| 10% | 93.88 | 74.01 | 84.56
ImageNet | 50% | 95.02 | 78.37 | 87.58
| 100% | 96.25 | 83.32 | 90.56
| 10% | 94.21 | 75.46 | 85.41
BT ImageNet | 50% | 95.2 | 79.24 | 87.93
| 100% | 96.52 | 84.41 | 91.24
| 10% | 94.13 | 75.23 | 85.22
CBT | 50% | 95.37 | 79.67 | 88.17
| 100% | 96.49 | 84.56 | 91.29
| 10% | 94.31 | 75.87 | 85.68
BT | 50% | 95.59 | 80.71 | 88.91
| 100% | 96.53 | 84.48 | 91.27

##  VII Conclusions

In this paper, we have shown that the combination of CL and SSL offers an
optimal compromise between performance and training efficiency and versatility
for RS applications. In particular, we demonstrated a combined approach
(Continual Barlow Twins) leading to consistent performance in a novel
combination of datasets with RS images that are heterogeneous in terms of
sensors, resolution, acquisition and scenes represented. Since the
availability of unlabeled data is increasing at a great speed, and it is not
possible for everyone to train repeatedly large models, a framework like CBT
offers a potential solution. However, more work remains to be done. First, the
validity of these results could be extended to new datasets and new tasks.
Among the use of new datasets, we mention datasets containing multispectral
images (i.e., not only with RGB bands). Second, other SSL and CL strategies
can be combined into an effective and efficient framework.

## References

  * [1] M. Delange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars, “A continual learning survey: Defying forgetting in classification tasks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.
  * [2] Y. Feng, X. Sun, W. Diao, J. Li, X. Gao, and K. Fu, “Continual learning with structured inheritance for semantic segmentation in aerial imagery,” IEEE Transactions on Geoscience and Remote Sensing, 2021.
  * [3] J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny, “Barlow twins: Self-supervised learning via redundancy reduction,” arXiv preprint arXiv:2103.03230, 2021.
  * [4] Y. Tian, D. Krishnan, and P. Isola, “Contrastive multiview coding,” in 16th European Conference on Compute Vision (ECCV), pp. 776–794, Springer, 2020\.
  * [5] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in Int. Conf. on Mach. Lear., pp. 1597–1607, PMLR, 2020.
  * [6] V. Stojnić and V. Risojević, “Evaluation of split-brain autoencoders for high-resolution remote sensing scene classification,” in 2018 International Symposium ELMAR, pp. 67–70, IEEE, 2018.
  * [7] E. Fini, V. G. T. da Costa, X. Alameda-Pineda, E. Ricci, K. Alahari, and J. Mairal, “Self-supervised models are continual learners,” arXiv preprint arXiv:2112.04215, 2021.
  * [8] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, et al., “Overcoming catastrophic forgetting in neural networks,” Proceedings of the national academy of sciences, vol. 114, no. 13, pp. 3521–3526, 2017.
  * [9] Z. Zhou, M. M. Rahman Siddiquee, N. Tajbakhsh, and J. Liang, “Unet++: A nested u-net architecture for medical image segmentation,” in Deep learning in medical image analysis and multimodal learning for clinical decision support, pp. 3–11, Springer, 2018.
  * [10] L. Caccia and J. Pineau, “Special: Self-supervised pretraining for continual learning,” arXiv preprint arXiv:2106.09065, 2021.
  * [11] G. Lenczner, A. Chan-Hon-Tong, N. Luminari, and B. L. Saux, “Weakly-supervised continual learning for class-incremental segmentation,” arXiv preprint arXiv:2201.01029, 2022.
  * [12] V. Stojnic and V. Risojevic, “Self-supervised learning of remote sensing scene representations using contrastive multiview coding,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1182–1191, 2021.
  * [13] V. Marsocci, S. Scardapane, and N. Komodakis, “Mare: Self-supervised multi-attention resu-net for semantic segmentation in remote sensing,” Remote Sensing, vol. 13, no. 16, p. 3275, 2021.
  * [14] F. Rottensteiner, G. Sohn, J. Jung, M. Gerke, C. Baillard, S. Benitez, and U. Breitkopf, “The isprs benchmark on urban object classification and 3d building reconstruction,” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences I-3 (2012), Nr. 1, vol. 1, no. 1, pp. 293–298, 2012.
  * [15] K. Ayush, B. Uzkent, C. Meng, K. Tanmay, M. Burke, D. Lobell, and S. Ermon, “Geography-aware self-supervised learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10181–10190, 2021\.
  * [16] C. Tao, J. Qi, W. Lu, H. Wang, and H. Li, “Remote sensing image scene classification with self-supervised paradigm under limited labeled samples,” IEEE Geoscience and Remote Sensing Letters, 2020.
  * [17] J. Kang, R. Fernandez-Beltran, P. Duan, S. Liu, and A. J. Plaza, “Deep unsupervised embedding for remotely sensed images based on spatially augmented momentum contrast,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 3, pp. 2598–2610, 2020.
  * [18] S. Vincenzi, A. Porrello, P. Buzzega, M. Cipriano, P. Fronte, R. Cuccu, C. Ippoliti, A. Conte, and S. Calderara, “The color out of space: learning self-supervised representations for earth observation imagery,” in 2020 25th International Conference on Pattern Recognition (ICPR), pp. 3034–3041, IEEE, 2021.
  * [19] G. Sumbul, M. Charfuelan, B. Demir, and V. Markl, “Bigearthnet: A large-scale benchmark archive for remote sensing image understanding,” in IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, pp. 5901–5904, IEEE, 2019.
  * [20] H. Dong, W. Ma, Y. Wu, J. Zhang, and L. Jiao, “Self-supervised representation learning for remote sensing image change detection based on temporal prediction,” Remote Sens., vol. 12, no. 11, p. 1868, 2020.
  * [21] Y. Chen and L. Bruzzone, “Self-supervised change detection in multi-view remote sensing images,” arXiv preprint arXiv:2103.05969, 2021.
  * [22] N. Ammour, “Continual learning using data regeneration for remote sensing scene classification,” IEEE Geoscience and Remote Sensing Letters, 2021\.
  * [23] O. Tasar, Y. Tarabalka, and P. Alliez, “Incremental learning for semantic segmentation of large-scale remote sensing data,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 9, pp. 3524–3537, 2019.
  * [24] Y. Lyu, G. Vosselman, G.-S. Xia, A. Yilmaz, and M. Y. Yang, “Uavid: A semantic segmentation dataset for uav imagery,” ISPRS journal of photogrammetry and remote sensing, vol. 165, pp. 108–119, 2020.
  * [25] M. Bosch, K. Foster, G. Christie, S. Wang, G. D. Hager, and M. Brown, “Semantic stereo for incidental satellite images,” in 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1524–1532, IEEE, 2019\.
  * [26] R. Li, S. Zheng, C. Zhang, C. Duan, J. Su, L. Wang, and P. M. Atkinson, “Multiattention network for semantic segmentation of fine-resolution remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing, 2021\.
  * [27] A. G. Roy, N. Navab, and C. Wachinger, “Recalibrating fully convolutional networks with spatial and channel “squeeze and excitation” blocks,” IEEE transactions on medical imaging, vol. 38, no. 2, pp. 540–549, 2018.
  * [28] C. J. Reed, X. Yue, A. Nrusimha, S. Ebrahimi, V. Vijaykumar, R. Mao, B. Li, S. Zhang, D. Guillory, S. Metzger, K. Keutzer, and T. Darrell, “Self-supervised pretraining improves self-supervised pretraining,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 2584–2594, January 2022.