# Manipulating Vortices with Domain Walls in Superconductor-Ferromagnet
Heterostructures

Sebastián A. Díaz  Faculty of Physics and Center for Nanointegration Duisburg-
Essen (CENIDE), University of Duisburg-Essen, 47057 Duisburg, Germany
Department of Physics, University of Konstanz, 78457 Konstanz, Germany  Jonas
Nothhelfer  Faculty of Physics and Center for Nanointegration Duisburg-Essen
(CENIDE), University of Duisburg-Essen, 47057 Duisburg, Germany  Kjetil M. D.
Hals  Department of Engineering Sciences, University of Agder, NO-4879
Grimstad, Norway  Karin Everschor-Sitte  Faculty of Physics and Center for
Nanointegration Duisburg-Essen (CENIDE), University of Duisburg-Essen, 47057
Duisburg, Germany

(August 28, 2024)

###### Abstract

Vortices are point-like topological defects in superconductors whose motion
dictates superconducting properties and controls device performance. In
superconductor-ferromagnet heterostructures, vortices interact with
topological defects in the ferromagnet such as line-like domain walls. While
in previous heterostructure generations, vortex-domain wall interactions were
mediated by stray fields; in new heterostructure families, more important
become exchange fields and spin-orbit coupling. However, spin-orbit coupling’s
role in vortex-domain wall interactions remains unexplored. Here we uncover,
via numerical simulations and Ginzburg-Landau theory, that Rashba spin-orbit
coupling induces magnetoelectric interactions between vortices and domain
walls that crucially depend on the wall’s winding direction—its helicity. The
wall’s helicity controls whether vortices are pushed or dragged by Néel walls,
and their gliding direction along Bloch walls. Our work capitalizes on
interactions between topological defects from different order parameters and
of different dimensionality to engineer enhanced functionality.

##  I Introduction

Nature provides us with numerous examples where nanostructures of dissimilar
kind and dimensionality work together to achieve specific functions:
supramolecular capsules trap small molecules and act as catalytic centers
Zhang and Tiefenbacher (2015); Syntrivanis  _et al._ (2020), lipid bilayer
membranes enclose organelles that determine cells’ functionality Alberts  _et
al._ (2022), RNA polymerase glides along a DNA strand to transcribe genetic
information Lee and Young (2000); Suzuki  _et al._ (2012). In the spirit of
these examples from Nature, topological defects of ordered media in condensed
matter Mermin (1979); Chaikin and Lubensky (2000), such as point-like
skyrmions Mühlbauer  _et al._ (2009); Yu  _et al._ (2010); Bogdanov  _et al._
(2003); Everschor-Sitte  _et al._ (2018); Shen and Dierking (2021) and
vortices Hubert and Schäfer (1998); Shinjo  _et al._ (2000); Blatter  _et al._
(1994); Lounasmaa and Thuneberg (1999); Salomaa and Volovik (1987), as well as
line-like dislocations Hull and Bacon (2011); Kleman (1989); Schoenherr  _et
al._ (2018); Azhar  _et al._ (2022) and domain walls Malozemoff and
Slonczewski (1979); Meier  _et al._ (2020); Nataf  _et al._ (2020); Meier and
Selbach (2021), have been actively investigated for their potential use in
technological applications. For instance, recent experiments have reported on
the confinement of skyrmions using domain walls in liquid crystals and magnets
Foster  _et al._ (2019); Tang  _et al._ (2021). Similarly, recent theoretical
studies Knapman  _et al._ (2021); Kong  _et al._ (2021) and their experimental
realization Song  _et al._ (2022) have demonstrated the guidance of magnetic
skyrmions along the tracks provided by the helical phase in helimagnets Ezawa
(2011); Müller  _et al._ (2017); Stepanova  _et al._ (2021). The main
challenge faced by these recent reports was ensuring the coexistence of
topological defects with different dimensionality and of the same order
parameter.

An alternative to overcome this challenge and a path towards engineering
enhanced functionality, as in the above examples from Nature, is to employ
different order parameters. Progress in this direction has already been made
in superconductor-ferromagnet heterostructures Lyuksyutov and Pokrovsky
(2005); Buzdin (2005); Aladyshkin  _et al._ (2009), where magnetic domain
walls have been shown to interact with superconducting vortices via stray
fields Helseth  _et al._ (2002); Vestgarden  _et al._ (2007). Moreover, both
the displacement Goa  _et al._ (2002); Vlasko-Vlasov  _et al._ (2008) and the
confinement Iavarone  _et al._ (2011) of superconducting vortices by the stray
field of magnetic domain walls have already been experimentally demonstrated.

Current interest in proximity-coupled superconductor-magnet heterostructures
stems from research on topological superconductivity Fu and Kane (2008);
Lutchyn  _et al._ (2010); Oreg  _et al._ (2010) and the promise of realizing
Majorana bound states Kitaev (2001); Alicea (2012) whose manipulation, for
example, could be achieved via controlling magnetic skyrmions Nothhelfer  _et
al._ (2019); Nothhelfer (2019); Díaz  _et al._ (2021); Nothhelfer  _et al._
(2022). Spin-orbit coupling (SOC), necessary for topological
superconductivity, is also present in heterostructures hosting superconducting
vortices and magnetic domain walls, but its role has so far been neglected and
remains unexplored.

Figure 1: Superconducting vortices manipulated by magnetic domain walls in
superconductor-ferromagnet heterostructures. A magnetic domain wall in the
ferromagnetic thin film (FM) proximity-coupled to a superconducting layer (SC)
interacts with superconducting vortices (blue tubes) via the magnetoelectric
effect induced by spin-orbit coupling. The winding of the domain wall
(helicity) and the vortices (vorticity) control whether vortices glide along
the domain wall, get dragged, or (as depicted) pushed.

Here we exploit the rich interplay between the distinct order parameters in
proximity-coupled superconductor-ferromagnet heterostructures, arising from
the magnetoelectric effect induced by Rashba SOC Edelstein (1995, 1996); Yip
(2002), to manipulate superconducting vortices with magnetic domain walls.

##  II Textured ferromagnet proximity-coupled to a superconductor

We assume the magnetic domain wall dynamics to be externally controlled, for
instance by in-plane magnetic fields Fang and Thiele (1990); Kabanov  _et al._
(2010) or temperature gradients Berger (1985); Torrejon  _et al._ (2012);
Jiang  _et al._ (2013), and aim to model its effect on the superconductor.
Moreover, the phenomena we are interested in possess a natural separation of
time scales. While magnetic domain walls attain speeds of $\sim 10$
$\mu\rm{m/s}$ when driven by in-plane magnetic fields Kabanov  _et al._ (2010)
and of $\sim 10^{2}$ $\mu\rm{m/s}$ when driven by temperature gradients Jiang
_et al._ (2013), itinerant electrons in clean superconductors are orders of
magnitude faster due to typical Fermi velocities of $\sim 10^{6}$ m/s Ashcroft
and Mermin (1976). Therefore, we adopt an adiabatic approach where we simulate
the time evolution of the superconducting order parameter as a sequence of
static configuration steps. At each step, the magnetic texture in the
proximity-coupled ferromagnet is a parameter set that modifies the electronic
energy spectrum and eigenstates in the superconductor through the tight-
binding Hamiltonian

$\displaystyle H=$ $\displaystyle-t\sum_{\langle
ij\rangle}\bm{c}_{i}^{\dagger}\bm{c}_{j}-\mu\sum_{i}\bm{c}_{i}^{\dagger}\bm{c}_{i}-\sum_{i}\bm{c}_{i}^{\dagger}(\bm{h}_{i}\cdot\bm{\sigma})\bm{c}_{i}$
(1) $\displaystyle+\mathrm{i}\,\alpha_{\text{R}}\sum_{\langle
ij\rangle}\bm{c}_{i}^{\dagger}\hat{\bm{z}}\cdot(\bm{\hat{d}}_{ij}\times\bm{\sigma})\bm{c}_{j}+\sum_{i}(\Delta_{i}c_{i\uparrow}^{\dagger}c_{i\downarrow}^{\dagger}+h.c.)\,.$

Itinerant electrons of spin $\alpha$ are created (annihilated) at site $i$ of
the square lattice, on the $xy$ plane and with lattice constant $a$, via the
operator $c^{\dagger}_{i\alpha}$($c_{i\alpha}$), and
$\bm{c}_{i}=(c_{i\uparrow},c_{i\downarrow})^{T}$. Here $t$ is the hopping
parameter, $\mu$ is the chemical potential, and $\bm{h}_{i}$ is the exchange
field due to the magnetic texture in the proximity-coupled ferromagnet. Rashba
SOC enters through the fourth term where $\alpha_{\text{R}}$ is the coupling
strength, the unit vector $\bm{\hat{d}}_{ij}$ is oriented along the nearest-
neighbor link from $j$ to $i$, $\bm{\sigma}$ is the vector of Pauli matrices,
and the coefficient $\mathrm{i}$ is the imaginary unit. Finally, $\Delta_{i}$
is the complex-valued and spatially-dependent superconducting order parameter.

Since we aim to describe a ferromagnet proximity-coupled to a superconductor
whose order parameter is spatially-dependent, a self-consistent treatment is
mandatory. A successfully convergent self-consistent calculation yields the
electronic energy spectrum and eigenstates as well as the spatially-dependent
superconducting order parameter. Details of this self-consistent calculation
approach can be found in Nothhelfer  _et al._ (2022).

With the adiabatic, self-consistent model outlined above at our disposal, we
will now characterize the possible interactions between magnetic domain walls
and superconducting vortices.

##  III Vortex response to a moving domain wall

Figure 2:  The vortex response to a moving domain wall depends on the wall’s
helicity. Snapshots show the time evolution of a vortex in response to a
domain wall (vertical red line) moving rightward with helicity (red arrow) (a)
$\gamma=\pi$, (b) $\gamma=0$, (c) $\gamma=3\pi/2$, and (d) $\gamma=\pi/2$.
Lengths are in units of the lattice constant $a$. The spatial configuration of
the superconducting order parameter is color-coded (central color wheel,
normalization constant $R_{\rm{norm}}=0.75$) by its curl and divergence. The
simulation starts with the vortex on the right domain. The supercurrent (light
blue arrows) is largest in the vicinity of the vortex and of the domain wall.
A Néel domain wall with (a) $\gamma=\pi$ constantly pushes the vortex. In
contrast, a Néel domain wall with (b) $\gamma=0$ initially attracts the
vortex, lets it move to the left domain, and then drags the vortex. After
getting trapped by a Bloch domain wall, the vortex glides along the domain
wall in the $\bm{\hat{y}}$ direction for (c) $\gamma=3\pi/2$ and in the
$-\bm{\hat{y}}$ direction for (d) $\gamma=\pi/2$.

To reveal the interaction between a superconducting vortex and a magnetic
domain wall we simulate their adiabatic time evolution. The exchange field
induced by the magnetic domain wall, aligned parallel to the $y$-axis and
centered at $x=0$, is given by

$\displaystyle\bm{h}^{(dw)}=h_{0}\big{[}\sin\Theta(x)\,\bm{\hat{n}}^{(dw)}(\gamma)+\cos\Theta(x)\bm{\hat{z}}\big{]}\,,$
(2)

where $h_{0}$ is the exchange field strength,
$\bm{\hat{n}}^{(dw)}(\gamma)=\cos\gamma\,\bm{\hat{x}}+\sin\gamma\,\bm{\hat{y}}$,
and $\gamma$ is the domain wall _helicity_. We choose a wall profile with the
magnetic moments pointing asymptotically upward (downward) to the left (right)
of the domain wall: $\cos\Theta(x)=-\tanh(x/d)$, where $d$ denotes the domain
wall width.

In the vicinity of the vortex, the superconducting order parameter phase
adopts the form $\theta^{(v)}=q_{v}\phi+\varphi$. The winding number of
$\theta^{(v)}$ about the vortex core, its _vorticity_ $q_{v}$, as well as its
_chirality_ $\varphi$ are determined by the background exchange field at the
vortex location and the Rashba SOC strength $\alpha_{\text{R}}$. For
concreteness, we will focus on superconducting vortices with $q_{v}=1$ 111We
do so by identifying model parameters that make a $q_{v}=1$ vortex, in a field
polarized magnetic background, the lowest-energy topologically nontrivial
configuration of the superconducting order parameter..

We simulate four distinct helicity cases: Néel domain walls with
$\gamma=0,\pi$ and Bloch domain walls with $\gamma=\pi/2,3\pi/2$ (see App. A).
Implementing a rigid, rightward domain wall motion yields strikingly different
superconducting vortex responses for the four different cases.

A Néel domain wall with $\gamma=\pi$, when sufficiently close, constantly
pushes the superconducting vortex to the right as shown in Fig. 2(a) (see
video Neel-Push.mp4). However, if the Néel domain wall has helicity
$\gamma=0$, as Fig. 2(b) depicts, a drastically different behavior occurs. The
superconducting vortex is attracted towards the domain wall and eventually
crosses to the left domain. Once in the “wake” of the domain wall, the vortex
settles at a distance and the domain wall begins to drag the vortex (see video
Neel-Drag.mp4).

In contrast, both Bloch domain walls initially attract the superconducting
vortex. Once sufficiently close, the vortex chirality changes and the vortex
motion acquires a vertical component. Eventually, the vortex gets trapped and
then it glides along the domain wall: for $\gamma=3\pi/2$ in the
$\bm{\hat{y}}$ direction, as shown in Fig. 2(c) (see video Bloch-Glide-
Up.mp4); and for $\gamma=\pi/2$ in the $-\bm{\hat{y}}$ direction, as shown in
Fig. 2(d) (see video Bloch-Glide-Down.mp4).

The diverse features of the above four cases admit a unified description
underpinned by the magnetoelectric effect induced by Rashba SOC Edelstein
(1995, 1996); Yip (2002). The exchange field due to the magnetic domain wall
generates a magnetoelectric supercurrent that interacts with the supercurrent
circulating the superconducting vortex, as we describe in the following.

##  IV Vortex-domain wall interactions

The numerical simulations presented above can be comprehended qualitatively
through the Ginzburg-Landau theory of superconductors Tinkham (1996). The
spatially asymmetric Rashba SOC introduces coupling mechanisms between the
superconducting condensate and the ferromagnetic layer, referred to as
magnetoelectric interactions Edelstein (1995, 1996); Yip (2002); Pershoguba
_et al._ (2015); Hals  _et al._ (2016). These interactions are incorporated
into the Ginzburg-Landau phenomenology through the following magnetoelectric
free energy functional $F_{me}\left[\psi^{\ast},\psi,\bm{A},\bm{h}\right]=\int
d\bm{r}\,\mathcal{F}_{me}^{(1)}$, where the free energy density
$\mathcal{F}_{me}^{(1)}$ is a SOC-induced Lifshitz invariant given by

$\displaystyle\mathcal{F}_{me}^{(1)}=-\kappa^{(1)}(\bm{\hat{z}}\times\bm{h})\cdot\bm{\mathcal{P}}\,,$
(3)

and the positive constant $\kappa^{(1)}$ parameterizes the magnetoelectric
coupling to first order in the SOC 222The contribution to the magnetoelectric
coupling to second order in the SOC, as our calculations in App. D show, is
negligible.. Here, $\bm{h}$ is the exchange field induced in the
superconductor by the adjacent ferromagnetic layer via the exchange proximity
effect,
$\bm{\mathcal{P}}=\real\left[\psi^{\ast}(-\mathrm{i}\hbar\bm{\nabla}-2e\bm{A}/c)\psi\right]$
is the momentum density of the superconducting condensate,
$\psi(\bm{r})=|\psi(\bm{r})|\exp(\mathrm{i}\theta(\bm{r}))$ is the complex
order parameter field of the superconductor, $2e$ (where $e=-|e|$) is the
charge of a Cooper pair, $c$ is the speed of light, and $\bm{A}$ is the
magnetic vector potential. We concentrate on the case of a thin film type-II
superconductor where the penetration depth of the magnetic field is
significantly larger than the typical length scales that characterize the
spatial variations of magnetic domain walls and superconducting vortices. In
this case, we can disregard the magnetic vector potential Kopnin (2001). The
magnetoelectric free energy functional $F_{me}$ establishes a connection
between the orientation of the ferromagnet’s magnetization and the momentum
density of the condensate. An essential consequence of this interaction is the
emergence of an anomalous supercurrent density given by (see App. B)

$\bm{j}_{as}=-2e|\psi|^{2}\kappa^{(1)}(\bm{\hat{z}}\times\bm{h})\,.$ (4)

This is an additional contribution to the conventional supercurrent
$\bm{j}_{cs}=e\bm{\mathcal{P}}/m$ generated by phase gradients in $\psi$,
resulting in a net supercurrent density of
$\bm{j}_{s}=\bm{j}_{cs}+\bm{j}_{as}$. Therefore, in proximity-coupled
superconductor-ferromagnet heterostructures, textures with an in-plane
component such as ferromagnetic domain walls, according to Eq. (4), produce
anomalous supercurrents.

Incorporating the exchange field defined in Eq. (2), we find the following
anomalous supercurrent density associated with a domain wall of general
helicity

$\bm{j}_{as}^{(dw)}(x,\gamma)=-g^{(1)}(x)\;\bm{\hat{z}}\times\bm{\hat{n}}^{(dw)}(\gamma)\,,$
(5)

where $g^{(1)}(x)=\kappa^{(1)}2eh_{0}|\psi|^{2}\sech\left(x/d\right)$
represents the current originating from $\mathcal{F}_{me}^{(1)}$. While Néel
domain walls generate supercurrents along the wall, Bloch domain walls
generate supercurrents across the wall.

On the other hand, a superconducting vortex centered at
$\bm{R}=R_{x}\,\bm{\hat{x}}+R_{y}\,\bm{\hat{y}}$ can be characterized by the
order parameter field Tinkham (1996)

$\psi^{(v)}(\bm{r})=\psi_{0}\tanh(|\bm{r}-\bm{R}|/\xi)\exp(\mathrm{i}\theta^{(v)}(\bm{r}))\,,$
(6)

where $\theta^{(v)}$ varies by $2\pi$ in making a complete circuit
counterclockwise about the center of the vortex, $\psi_{0}$ denotes the
absolute value of the order parameter far from the vortex core, and $\xi$
represents the coherence length. This vortex profile generates the
conventional supercurrent density

$\bm{j}_{cs}^{(v)}=\frac{\hbar
e\psi_{0}^{2}}{m}\frac{\tanh^{2}(|\bm{r}-\bm{R}|/\xi)}{|\bm{r}-\bm{R}|^{2}}\bm{\hat{z}}\times(\bm{r}-\bm{R})\,,$
(7)

which is strongest when $|\bm{r}-\bm{R}|\sim\xi$ and approaches zero in the
limits $|\bm{r}-\bm{R}|\rightarrow 0$ and $|\bm{r}-\bm{R}|\rightarrow\infty$.

The vortex-domain wall magnetoelectric interactions are governed by the
effective free energy

$F_{\rm
eff}(\bm{R},\gamma)=F_{me}[\psi^{(v)\ast},\psi^{(v)},\bm{h}^{(dw)}]\,,$ (8)

which is obtained by substituting $\bm{h}^{(dw)}$, the exchange field of a
domain wall with helicity $\gamma$ centered at $x=0$ Eq. (2), and
$\psi^{(v)}$, the profile of a vortex centered at $\bm{R}$ Eq. (6), into the
magnetoelectric energy functional $F_{me}$, followed by integrating over the
spatial coordinate $\bm{r}$. Through its dependence on $\bm{R}$ and $\gamma$,
$F_{\rm eff}$ determines how the superconductor’s free energy varies with the
vortex’s position relative to the domain wall. Additionally, from Eq. (8) we
can deduce the magnetoelectric force $\bm{f}_{me}=-\partial F_{\rm
eff}/\partial\bm{R}$, which drives the vortex towards the effective free
energy minimum.

We numerically compute the effective free energy, assuming the domain wall
width and the coherence length are similar $d\sim\xi$, in the vicinity of Néel
[Fig. 3(a)] and Bloch [Fig. 3(b)] domain walls. For $\gamma=\pi$ ($\gamma=0$)
Néel walls, $F_{\rm eff}$ exhibits a global minimum at $R_{x}\sim\xi$
($R_{x}\sim-\xi$) from the wall’s center. This explains why in the simulation
for $\gamma=\pi$ [Fig. 2(a)] the vortex is pushed in front of the domain wall
as it moves to the right, while for $\gamma=0$ [Fig. 2(b)] the vortex crosses
into the left domain and is dragged behind the moving wall. In contrast, for
$\gamma=\pi/2$ ($\gamma=3\pi/2$) Bloch walls, $F_{\rm eff}$ exhibits a global
minimum at the end of the wall towards $-\bm{\hat{y}}$ ($+\bm{\hat{y}}$) where
it intersects the sample boundary. Therefore, the effective free energy gives
rise to a $y$-component in the vortex motion [Figs. 2(c),(d)], which explains
its gliding along the wall.

Our analytical calculations (see App. C) corroborate these numerical results
and their helicity-dependence. For Bloch walls, they also clearly show the
removal of the minima and maxima of $F_{\rm eff}$ as the sample size
approaches infinity—these minima and maxima are finite-size effects.

Figure 3:  Effective free energy landscapes of a vortex interacting with a
domain wall. The minima of $F_{\rm eff}$ Eq. (8), with the domain wall fixed
at $R_{x}=0$, indicates the vortex equilibrium position. The numerical
calculation considered a sample of size
$[-\infty,+\infty]\times[-10\,\xi,10\,\xi]$, the domain wall width is
$d=0.4\,\xi$, and $F_{0}^{(1)}=\kappa^{(1)}\pi\hbar\,d\,h_{0}\psi_{0}^{2}$.
(a) For a Néel wall with helicity $\gamma=\pi$, $F_{\rm eff}$ attains its
global minimum to the right of the wall’s center at $(R_{x}\sim\xi,R_{y}=0)$;
this explains the simulation [Fig. 2(a)] where the wall pushes the vortex. (b)
For a Bloch wall with helicity $\gamma=\pi/2$, $F_{\rm eff}$ attains its
global minimum at one of the wall ends on the sample boundary at
$(R_{x}=0,R_{y}=-10\,\xi)$; this explains the simulation [Fig. 2(d)] where the
vortex glides along the wall.

A simple physical picture of vortex-domain wall interactions, which
complements the above free energy calculations, follows from the interplay
between anomalous and conventional supercurrents densities. When the
superconductor-ferromagnet heterostructure contains both a domain wall and a
vortex, the superconductor aims to minimize its free energy by relocating the
vortex to reduce the total supercurrent density
$\bm{j}_{s}=\bm{j}_{cs}^{(v)}+\bm{j}_{as}^{(dw)}$ to a minimum (see App. B).
In the case of a Néel wall, the vortex reaches an equilibrium position at a
distance $|R_{x}|\sim\xi$ from the wall’s center because this ensures that the
area with the strongest conventional supercurrent density around the vortex
partly offsets the anomalous supercurrent density induced along the wall.
While for Bloch walls, the vortex attains its equilibrium position at one of
the wall ends to counteract the anomalous supercurrent density induced in the
sample interior.

The detailed understanding of the helicity-dependent vortex-domain wall
interactions obtained thus far will be exploited in the following section
where we demonstrate practical routes to manipulate superconducting vortices
with magnetic domain walls.

##  V Manipulating vortices with domain walls

Vortex manipulation is relevant for technological applications. For instance,
directional guidance of vortices underpins vortex-based logic platforms
Nakajima  _et al._ (1976); Reichhardt  _et al._ (2004); Nothhelfer  _et al._
(2022); Keren  _et al._ (2023). Additionally, confining vortices to narrow
constrictions enhances vortex pinning, which extends the magnetic field range
of the zero-resistance state Córdoba  _et al._ (2013) and improves the
performance of superconducting electronic devices Dobrovolskiy (2017). We
employ the helicity-dependent vortex-domain wall interactions detailed in the
previous sections to manipulate vortices with domain walls.

A Néel wall with helicity $\gamma=\pi$ can be utilized to push away and thus
remove vortices from a specific sample area, as illustrated in Fig. 4.
Alternatively, by the same mechanism, an empty sample area can be populated
with vortices. Being mobile barriers, these domain walls also have the
potential to suppress or trigger superconducting vortex avalanches Field  _et
al._ (1995); Altshuler and Johansen (2004); Laviano (2017).

Bloch walls themselves, in finite-sized setups, can serve as unidirectional
vortex channels where the wall’s helicity controls the vortices propagation
direction [see Figs. 2(c),(d)]. Another route to construct a vortex channel is
depicted in Fig. 5 (for a simulation see video Vortex-Chain-Assembly.mp4)
where two Néel walls are employed, also taking advantage of repulsive vortex-
vortex interactions, to assemble a chain of vortices. Although vortex chains
have been reported before in anisotropic Matsuda  _et al._ (2001); Grigorenko
_et al._ (2001); Bending and Dodgson (2005) and nanopatterned superconductors
Dobrovolskiy (2017); Córdoba  _et al._ (2019), and between twin boundaries
Song  _et al._ (2023); mobile magnetic domain walls represent an attractive
non-invasive avenue to construct reconfigurable channels for vortices and
vortex chains.

Our simulations demonstrate that exploiting the dependence on the wall’s
helicity of the vortex-domain wall interactions leads to enhanced
functionality.

Figure 4: Wiping the slate clean of superconducting vortices. Configurational
sequence of a rightward-moving Néel domain wall with helicity $\gamma=\pi$
that pushes vortices away from the sample area, effectively acting as a vortex
“eraser” or “rake” (see video Vortex-Eraser.mp4). Figure 5: Assembling a
superconducting vortex chain using two magnetic domain walls. Configurational
sequence of superconducting vortices confined between two Néel domain walls
with helicity $\gamma=\pi$ (see video Vortex-Chain-Assembly.mp4). As the
separation between the walls decreases, the vortices rearrange (due to domain
wall pushing and repulsive vortex-vortex interactions) and form a chain
parallel to the walls.

##  VI Discussion and Conclusion

The experimental realization of our predictions requires superconductor-
ferromagnet heterostructures in which vortices and domain walls interact via
Rashba SOC. The high-quality interfaces needed for Rashba SOC can be assembled
using van der Waals (vdW) materials. For example, evidence of Rashba SOC has
been reported at the interface of: a quasi-2D layer of the vdW ferromagnet
Fe0.29TaS2 with NbN Cai  _et al._ (2021), and a monolayer of the vdW
ferromagnet CrBr3 with NbSe2 Kezilebieke  _et al._ (2020). Both NbN and NbSe2
are conventional, type-II superconductors which admit vortices. Furthermore,
it has been established that the CrBr3/NbSe2 heterostructure supports vortices
Kezilebieke  _et al._ (2022) and CrBr3 hosts Bloch domain walls Matricardi
_et al._ (1967); Sun  _et al._ (2021). Gliding vortices along Bloch walls
[Figs. 2(c),(d)] we propose to observe in the existing CrBr3/NbSe2
heterostructure. Another vdW ferromagnet that exhibits Bloch domain walls is
Fe3GeTe2 Yang  _et al._ (2022); but by decreasing its thickness or engineering
its capping layer, Fe3GeTe2 can favor Néel domain walls Birch  _et al._
(2022); Peng  _et al._ (2021). Vortex pushing or dragging by Néel walls [Figs.
2(a),(b)] we suggest to observe in Fe3GeTe2/NbSe2 heterostructures. Therefore,
superconductor-ferromagnet heterostructures suitable for the experimental
realization of our predictions are available.

Conclusive experimental confirmation of our predictions calls for tracking
domain wall and vortex positions. The observation of domain walls and the
determination of their helicity requires the measurement of the three
components of the magnetization, which can be achieved using spin-polarized
scanning tunneling microscopy (SP-STM) Bode  _et al._ (2007), spin-polarized
low-energy electron microscopy (SPLEEM) Chen and Schmid (2015), or Lorentz
transmission electron microscopy (TEM) even without tilting the sample Chess
_et al._ (2017). Additionally, note that our findings of helicity-dependent
vortex-domain wall interactions provide an indirect method to measure the
domain wall helicity. Vortex detection can be done via Lorentz TEM Harada  _et
al._ (1992), magneto-optical imaging Goa  _et al._ (2002); Vestgarden  _et
al._ (2007), magnetic force microscopy (MFM) Moser  _et al._ (1995);
Auslaender  _et al._ (2008), STM Hess  _et al._ (1989), and superconducting
quantum interference device (SQUID) magnetometry Embon  _et al._ (2017).
Employing a single technique to image vortices and domain walls
simultaneously, facilitates data collection and minimizes system disruption.
The simultaneous measurement of domain walls and vortices has been
demonstrated using MFM Iavarone  _et al._ (2011). There exist plenty of
imaging techniques that allow to confirm our predictions.

We have explained how Rashba spin-orbit coupling in superconductor-ferromagnet
heterostructures induces interactions between superconducting vortices and
magnetic domain walls. Leveraging the dependence of these interactions on the
domain walls’ helicity, we have showcased the enhanced functionalities that
result from manipulating vortices with domain walls. Our work epitomizes
Nature’s paradigm of functionality via the interaction of dissimilar
nanostructures, and embraces its extension to the condensed matter realm of
topological defects of different order parameters and dimensionality.

###### Acknowledgements.

Jonas Nothhelfer and Sebastián A. Díaz contributed equally to this work. We
thank Lucas Görzen for assistance in this work’s preliminary stages. We thank
Stephan Kessler, Christos Panagopoulos, Danilo Nikolic, Michael Hein, and
Wolfgang Belzig for helpful discussions. S. A. Díaz acknowledges funding by a
Blue Sky project led by W. Belzig in the framework of the Excellence Strategy
of the University of Konstanz. We acknowledge funding from the German Research
Foundation (DFG) Project No. 320163632 (Emmy Noether), Project No. 403233384
(SPP2137 Skyrmionics), the Research Council of Norway via Project No. 334202,
and support from the Max Planck Graduate Center. This research was supported
in part by the National Science Foundation under Grant No. NSF PHY-1748958.

##  Appendix A Vortex and domain wall simulations

In the simulations presented in this work, we numerically computed the
superconducting order parameter $\Delta$ following the self-consistent
calculation detailed in our previous work Nothhelfer  _et al._ (2022), and we
used the system parameters shown in Table 1. The domain walls, on the other
hand, were inputs for the numerical calculation of $\Delta$; as described in
Sec. III in the main text, we assumed the domain wall profile
$\cos\Theta(x)=-\tanh(x/d)$. Here, $d$ denotes the domain wall width, and in
our simulations we used $d=3a$, where $a$ is the lattice constant of the
tight-binding Hamiltonian in Eq. (1) of the main text.

Figure 2 in the main text shows selected snapshots of four simulations whose
videos (Neel-Push.mp4, Neel-Drag.mp4, Bloch-Glide-Up.mp4, Bloch-Glide-
Down.mp4) are included in the Supplemental Material. In these four
simulations, a single vortex is initially placed sufficiently to the right of
the domain wall (at a distance of $19$ lattice sites) where the magnetic
texture is effectively field polarized.

Also included in the Supplemental Material are simulation videos of the vortex
manipulations introduced in Figs. 4-5: Vortex-Eraser.mp4 and Vortex-Chain-
Assembly.mp4.

To obtain the initial vortex configuration of the superconducting order
parameter we use a seed at the beginning of the self-consistent calculation.
Subsequent simulation steps use the vortex configuration from the previous
step as their seed (for details see our previous work Nothhelfer  _et al._
(2022)).

Parameters | Symbol | Value ($t$)
---|---|---
Chemical potential | $\mu$ | $-4$
Spin-Orbit-Coupling | $\alpha_{\text{R}}$ | 0.75
Thermal energy | $k_{B}T$ | 0.001
Debye frequency | $\omega_{D}$ | 100
Effective attractive interaction | $V$ | 5
Table 1: System parameters used in this work in units of the hopping $t$.

##  Appendix B Ginzburg-Landau theory of noncentrosymmetric superconductors

For noncentrosymmetric superconductors, the Ginzburg-Landau free energy takes
the form

$F\left[\psi^{\ast},\psi,\bm{A},\bm{h}\right]=\int
d\bm{r}\left[\mathcal{F}_{0}+\mathcal{F}_{me}^{(1)}+\mathcal{F}_{me}^{(2)}\right],$
(9)

where $\mathcal{F}_{0}$ is given by

$\mathcal{F}_{0}=\frac{1}{4m}(\bm{\Pi}\psi)^{\ast}\cdot\bm{\Pi}\psi+a|\psi|^{2}+\frac{b}{2}|\psi|^{4}+\frac{|\bm{B}|^{2}}{8\pi}.$
(10)

Here, $\bm{\Pi}=-\mathrm{i}\hbar\bm{\nabla}-2e\bm{A}/c$, and $a$ and $b$
determine the absolute value of the order parameter field $\psi$. Note that
$\psi$ and $\Delta$ are proportional to each other Edelstein (1996). The
magnetoelectric free energy densities $\mathcal{F}_{me}^{(1)}$ and
$\mathcal{F}_{me}^{(2)}$ are

$\displaystyle\mathcal{F}_{me}^{(1)}$
$\displaystyle=-\kappa^{(1)}(\bm{\hat{z}}\times\bm{h})\cdot\bm{\mathcal{P}}\,,$
(11) $\displaystyle\mathcal{F}_{me}^{(2)}$
$\displaystyle=\kappa^{(2)}(\bm{\hat{z}}\times\bm{\nabla}h_{z})\cdot\bm{\mathcal{P}}\,,$
(12)

with
$\bm{\mathcal{P}}=\real\left[\psi^{\ast}(-\mathrm{i}\hbar\bm{\nabla}-2e\bm{A}/c)\psi\right]$,
while the constants $\kappa^{(1)}$ and $\kappa^{(2)}$ parameterize the
magnetoelectric coupling respectively to first and second order in the SOC.
Varying the free energy functional in Eq. (9) with respect to $\bm{A}$ and
using that $\bm{j}_{s}=(c/4\pi)(\bm{\nabla}\times\bm{B})$ lead to the
supercurrent density $\bm{j}_{s}=\bm{j}_{cs}+\bm{j}_{as}$, where
$\bm{j}_{cs}=e\bm{\mathcal{P}}/m$ and, disregarding $\bm{A}$ as justified in
the main text,

$\displaystyle\bm{j}_{as}=2e|\psi|^{2}\left[-\kappa^{(1)}(\bm{\hat{z}}\times\bm{h})+\kappa^{(2)}(\bm{\hat{z}}\times\bm{\nabla}h_{z})\right]\,.$
(13)

Plugging in the domain wall exchange field (Eq. (2) in the main text) yields

$\displaystyle\bm{j}_{as}^{(dw)}(x,\gamma)=-g^{(1)}(x)\;\bm{\hat{z}}\times\bm{\hat{n}}^{(dw)}(\gamma)-g^{(2)}(x)\;\bm{\hat{y}}\,,$
(14)

where $g^{(1)}(x)=\kappa^{(1)}2eh_{0}|\psi|^{2}\sech\left(x/d\right)$ and
$g^{(2)}(x)=\kappa^{(2)}2eh_{0}|\psi|^{2}\sech^{2}(x/d)/d$. The contribution
of the term proportional to $\kappa^{(2)}$, as we show in App. D, is
negligible. For this reason, in the main text, this term is absent.

Neglecting $\kappa^{(2)}$ in $\bm{j}_{as}$ [Eq. (13)] and $\bm{A}$ in
$\bm{j}_{cs}$, as in the main text, allows the rewrite
$\mathcal{F}_{me}^{(1)}=k\,\bm{j}_{as}\cdot\bm{j}_{cs}$ with
$k=m/(2e^{2}|\psi|^{2})>0$, shedding light to our results. The superconducting
vortex, to minimize $\mathcal{F}_{me}^{(1)}$, places itself in the region near
the domain wall where $\bm{j}_{as}$ and $\bm{j}_{cs}$ are antiparallel and the
strongest.

##  Appendix C Effective free energy

To derive approximate analytical expressions for the effective free energy
$F_{\rm eff}$, Eq. (8) in the main text, we consider a sample that extends
infinitely along the $x$-axis and has a width of $2L_{y}$ along the $y$-axis.
A domain wall is positioned at $x=0$. We assume that the vortex is located
near the domain wall and the origin of the sample, satisfying
$\tilde{\bm{R}}=\bm{R}/\xi\ll 1$. Additionally, we examine narrow domain walls
where $d\rightarrow 0$. By performing a series expansion of $F_{\rm eff}$ in
terms of $\tilde{\bm{R}}$ to third order and considering the scenarios where
the heterostructure hosts Néel and Bloch walls, we obtain the following
expressions for the effective free energies

$\displaystyle F_{\rm eff}^{(nw)}=$ $\displaystyle(\epsilon
F_{0}^{(1)}+F_{0}^{(2)})\left[c_{1}\tilde{R}_{x}-c_{2}\tilde{R}_{x}^{3}-c_{3}\tilde{R}_{x}\tilde{R}_{y}^{2}\right]\,,$
(15) $\displaystyle F_{\rm eff}^{(bw)}=$ $\displaystyle\epsilon
F_{0}^{(1)}\left[c_{4}\tilde{R}_{y}+c_{5}\tilde{R}_{y}^{3}-c_{6}\tilde{R}_{x}^{2}\tilde{R}_{y}\right]$
$\displaystyle-
F_{0}^{(2)}\left[c_{2}\tilde{R}_{x}^{3}-c_{1}\tilde{R}_{x}+c_{3}\tilde{R}_{x}\tilde{R}_{y}^{2}\right]\,.$

Here, $F_{\rm eff}^{(nw)}=F_{\rm eff}|_{\gamma=(1-\epsilon)\pi/2}$ and $F_{\rm
eff}^{(bw)}=F_{\rm eff}|_{\gamma=(2-\epsilon)\pi/2}$ respectively represent
the effective free energies for systems containing Néel and Bloch walls, where
$\epsilon\in\\{1,-1\\}$ depending on the domain wall’s helicity. We have also
introduced $F_{0}^{(1)}=\pi\hbar d\kappa^{(1)}h_{0}\psi_{0}^{2}$ and
$F_{0}^{(2)}=2\hbar\kappa^{(2)}h_{0}\psi_{0}^{2}$. In deriving the above
expressions, we have utilized the approximations $(1/\pi
d)\sech(x/d)\approx\delta(x)$ and $(1/2d)\sech^{2}(x/d)\approx\delta(x)$ in
the limit $d\rightarrow 0$ (where $\delta(x)$ is the Dirac delta function).
Since a vortex at $\bm{R}$ and a domain wall in $\bm{h}$ are related by a
global rotation to a configuration with $-\bm{R}$ and $-\bm{h}$, their
effective free energy is the same: $F_{\rm eff}(\bm{R},\bm{h})=F_{\rm
eff}(-\bm{R},-\bm{h})$. Additionally, the effective free energy changes sign
upon reversing the exchange field, i.e., $F_{\rm eff}(\bm{R},-\bm{h})=-F_{\rm
eff}(\bm{R},\bm{h})$; therefore $F_{\rm eff}(-\bm{R},\bm{h})=-F_{\rm
eff}(\bm{R},\bm{h})$. This explains why $F_{\rm eff}^{(nw)}$ and $F_{\rm
eff}^{(bw)}$ are odd functions in $\tilde{\bm{R}}$, and in particular the
second-order contributions vanish in both cases. The expansion coefficients in
Eqs. (15) are determined by the integrals
$c_{i}=\int_{-\tilde{L}_{y}}^{\tilde{L}_{y}}\eta_{i}d\tilde{y}$, where
$\tilde{y}=y/\xi$, $\tilde{L}_{y}=L_{y}/\xi$, and the dimensionless functions
$\eta_{i}$ are given by the expressions

$\displaystyle\eta_{1}=$
$\displaystyle\frac{1}{\tilde{y}^{2}}\tanh^{2}(\tilde{y})\,,$ (16)
$\displaystyle\eta_{2}=$
$\displaystyle\frac{1}{\tilde{y}^{4}}\tanh^{2}(\tilde{y})[1-\tilde{y}\csch(\tilde{y})\sech(\tilde{y})]\,,$
(17) $\displaystyle\eta_{3}=$
$\displaystyle\frac{1}{\tilde{y}^{4}}[(3+2\tilde{y}^{2}+4\tilde{y}\tanh(\tilde{y}))\sech^{2}(\tilde{y})$
$\displaystyle\quad\quad-3-3\tilde{y}^{2}\sech^{4}(\tilde{y})]\,,$ (18)
$\displaystyle\eta_{4}=$
$\displaystyle\frac{1}{\tilde{y}^{2}}\tanh(\tilde{y})[2\tilde{y}\sech^{2}(\tilde{y})-\tanh(\tilde{y})]\,,$
(19) $\displaystyle\eta_{5}=$
$\displaystyle\frac{1}{3\tilde{y}^{4}}[(3+6\tilde{y}^{2}+2\tilde{y}(3+2\tilde{y}^{2})\tanh(\tilde{y}))\sech^{2}(\tilde{y})$
$\displaystyle\quad\quad-3-3\tilde{y}^{2}(3+4\tilde{y}\tanh(\tilde{y}))\sech^{4}(\tilde{y})]\,,$
(20) $\displaystyle\eta_{6}=$ $\displaystyle\eta_{3}\,.$ (21)

While exact analytic expressions for $c_{1}$ and $c_{2}$ are not obtainable,
the other coefficients read

$\displaystyle c_{3}=$
$\displaystyle\frac{2}{\tilde{L}_{y}^{3}}\left[1-\sech^{2}(\tilde{L}_{y})(1+\tilde{L}_{y}\tanh(\tilde{L}_{y}))\right]\,,$
(22) $\displaystyle c_{4}=$
$\displaystyle\frac{2\tanh^{2}(\tilde{L}_{y})}{\tilde{L}_{y}}\,,$ (23)
$\displaystyle c_{5}=$
$\displaystyle\frac{\sech^{4}(\tilde{L}_{y})}{12\tilde{L}_{y}^{3}}[\cosh(4\tilde{L}_{y})+16\tilde{L}_{y}^{2}$
$\displaystyle-1-8\tilde{L}_{y}^{2}\cosh(2\tilde{L}_{y})-8\tilde{L}_{y}\sinh(2\tilde{L}_{y})]\,,$
(24) $\displaystyle c_{6}=$ $\displaystyle c_{3}\,.$ (25)

Importantly, all the expansion coefficients are positive for $L_{y}/\xi>2$,
depend on the system size $L_{y}$, and approach the values $c_{1}\approx
3.41$, $c_{2}\approx 1.32$, $c_{3}\approx 2(\xi/L_{y})^{3}$, $c_{4}\approx
2\xi/L_{y}$, $c_{5}\approx(2/3)(\xi/L_{y})^{3}$, and $c_{6}\approx
2(\xi/L_{y})^{3}$ when $L_{y}/\xi\gg 1$. Thus, the terms proportional to
$c_{3-6}$ account for finite-size effects, as these coefficients vanish in the
limit $L_{y}/\xi\rightarrow\infty$.

The expressions in Eqs. (15) capture the functional form of the effective free
energy for vortices located in proximity to Néel and Bloch walls. In the case
of Néel walls, the series expansions of the $F_{0}^{(1)}$ and $F_{0}^{(2)}$
terms are identical, resulting in an energy minimum at $\bm{R}_{0}=-{\rm
sign}(\epsilon F_{0}^{(1)}+F_{0}^{(2)})\xi\sqrt{c_{1}/3c_{2}}\;\bm{\hat{x}}$.
In contrast, for heterostructures containing Bloch walls, the $F_{0}^{(1)}$
term exhibits an energy minimum at $R_{y}=-{\rm sign}(\epsilon
F_{0}^{(1)})L_{y}$. Note that this phenomenon is solely a finite-size effect,
as it stems from the $c_{4-6}$ terms in $F_{\rm eff}^{(bw)}$ that vanish as
$L_{y}/\xi$ tends to infinity. The $F_{0}^{(2)}$ term, on the other hand,
still has a minimum at $\bm{R}_{0}=-{\rm
sign}(F_{0}^{(2)})\xi\sqrt{c_{1}/3c_{2}}\;\bm{\hat{x}}$.

The main features of the numerically computed effective free energies in Fig.
3 of the main text follow from Eqs. (15) after setting $F_{0}^{(2)}=0$. This
is equivalent to a vanishing $\kappa^{(2)}$; whose contribution is negligible
when compared to $\kappa^{(1)}$, as we justify in App. D. Furthermore, our
numerical simulations are consistent with the $F_{0}^{(1)}$-term in Eqs. (15).
For a Néel wall with $\gamma=0$, the $F_{0}^{(1)}$-term exhibits a minimum at
$\bm{R}=-\xi\sqrt{c_{1}/3c_{2}}\;\bm{\hat{x}}$. This explains why the vortex
crosses into the left domain and is dragged behind the moving wall.
Conversely, for a Néel wall with $\gamma=\pi$, the vortex reaches equilibrium
at $\bm{R}=\xi\sqrt{c_{1}/3c_{2}}\;\bm{\hat{x}}$ and is pushed in front of the
domain wall as it moves to the right. In the case of Bloch walls, the
$F_{0}^{(1)}$-term has a minimum at the end of the wall, given by
$R_{y}=L_{y}$ ($R_{y}=-L_{y}$) for $\gamma=3\pi/2$ ($\gamma=\pi/2$). This
effective free energy landscape gives rise to a $y$-component in the vortex’s
motion, which explains its gliding movement along the wall.

##  Appendix D Supercurrent density and second-order SOC term significance

The vector fields of the supercurrent density $\bm{j}_{s}$ plotted in Fig. 2
in the main text were numerically computed using the eigenenergies and
eigenstates of the tight-binding Hamiltonian in Eq. (1) of the main text
written in Bogoliubov-de Gennes form. For this numerical calculation of
$\bm{j}_{s}$ we used the lattice expression from the appendix of Ref. Hals
(2016). Comparing the vector fields of $\bm{j}_{s}$ in Fig. 2 with the
prediction from Ginzburg-Landau theory in App. B, reveals that the term in
$\bm{j}_{as}$, Eq. (13), proportional to $\kappa^{(2)}$ does not contribute
significantly.

To assess the significance of the term proportional to $\kappa^{(2)}$ (also
$F_{0}^{(2)}$ in Eqs. (15)), we performed a numerical analysis of the relative
magnitude of $j_{as,x}^{(dw)}$ and $j_{as,y}^{(dw)}$ in Eq. (14) for a system
containing a Bloch wall without a vortex. In this case, the
$\kappa^{(1)}$-term generates a current along the $x$-axis, while the
$\kappa^{(2)}$-term produces a current parallel to the $y$-axis. Consequently,
the ratio $j_{as,y}^{(dw)}/j_{as,x}^{(dw)}$ provides a direct measure of
$\kappa^{(2)}/\kappa^{(1)}$. Our numerical investigation shows that the value
of $j_{as,y}^{(dw)}/j_{as,x}^{(dw)}$ is significantly below the machine
precision, implying that the contribution from $\kappa^{(2)}$ is negligible.

## References

  * Zhang and Tiefenbacher (2015) Q. Zhang and K. Tiefenbacher, Nature Chemistry 7, 197 (2015).
  * Syntrivanis  _et al._ (2020) L.-D. Syntrivanis, I. Némethová, D. Schmid, S. Levi, A. Prescimone, F. Bissegger, D. T. Major, and K. Tiefenbacher, Journal of the American Chemical Society 142, 5894 (2020).
  * Alberts  _et al._ (2022) B. Alberts, R. Heald, A. Johnson, D. Morgan, M. Raff, K. Roberts, and P. Walter,  _Molecular Biology of the Cell_ (W. W. Norton & Co., 2022).
  * Lee and Young (2000) T. I. Lee and R. A. Young, Annual Review of Genetics 34, 77 (2000).
  * Suzuki  _et al._ (2012) Y. Suzuki, M. Shin, A. Yoshida, S. H. Yoshimura, and K. Takeyasu, FEBS Letters 586, 3187 (2012).
  * Mermin (1979) N. D. Mermin, Reviews of Modern Physics 51, 591 (1979).
  * Chaikin and Lubensky (2000) P. Chaikin and T. Lubensky,  _Principles of Condensed Matter Physics_ (Cambridge University Press, 2000).
  * Mühlbauer  _et al._ (2009) S. Mühlbauer, B. Binz, F. Jonietz, C. Pfleiderer, A. Rosch, A. Neubauer, R. Georgii, and P. Böni, Science 323, 915 (2009).
  * Yu  _et al._ (2010) X. Z. Yu, Y. Onose, N. Kanazawa, J. H. Park, J. H. Han, Y. Matsui, N. Nagaosa, and Y. Tokura, Nature 465, 901 (2010).
  * Bogdanov  _et al._ (2003) A. N. Bogdanov, U. K. Rößler, and A. A. Shestakov, Physical Review E 67, 016602 (2003).
  * Everschor-Sitte  _et al._ (2018) K. Everschor-Sitte, J. Masell, R. M. Reeve, and M. Kläui, Journal of Applied Physics 124, 240901 (2018).
  * Shen and Dierking (2021) Y. Shen and I. Dierking, Physical Review Applied 15, 054023 (2021).
  * Hubert and Schäfer (1998) A. Hubert and R. Schäfer,  _Magnetic Domains: The Analysis of Magnetic Microstructures_ (Springer Berlin Heidelberg, 1998).
  * Shinjo  _et al._ (2000) T. Shinjo, T. Okuno, R. Hassdorf, †. K. Shigeto, and T. Ono, Science 289, 930 (2000).
  * Blatter  _et al._ (1994) G. Blatter, M. V. Feigel'man, V. B. Geshkenbein, A. I. Larkin, and V. M. Vinokur, Reviews of Modern Physics 66, 1125 (1994).
  * Lounasmaa and Thuneberg (1999) O. V. Lounasmaa and E. Thuneberg, Proceedings of the National Academy of Sciences 96, 7760 (1999).
  * Salomaa and Volovik (1987) M. M. Salomaa and G. E. Volovik, Reviews of Modern Physics 59, 533 (1987).
  * Hull and Bacon (2011) D. Hull and D. Bacon,  _Introduction to Dislocations_ (Elsevier, 2011).
  * Kleman (1989) M. Kleman, Reports on Progress in Physics 52, 555 (1989).
  * Schoenherr  _et al._ (2018) P. Schoenherr, J. Müller, L. Köhler, A. Rosch, N. Kanazawa, Y. Tokura, M. Garst, and D. Meier, Nature Physics 14, 465 (2018).
  * Azhar  _et al._ (2022) M. Azhar, V. P. Kravchuk, and M. Garst, Physical Review Letters 128, 157204 (2022).
  * Malozemoff and Slonczewski (1979) A. P. Malozemoff and J. C. Slonczewski,  _Magnetic Domain Walls in Bubble Materials_ (Elsevier, 1979).
  * Meier  _et al._ (2020) D. Meier, J. Seidel, M. Gregg, and R. Ramesh,  _Domain Walls: From Fundamental Properties to Nanotechnology Concepts_ (Oxford University Press, 2020).
  * Nataf  _et al._ (2020) G. F. Nataf, M. Guennou, J. M. Gregg, D. Meier, J. Hlinka, E. K. H. Salje, and J. Kreisel, Nature Reviews Physics 2, 634 (2020).
  * Meier and Selbach (2021) D. Meier and S. M. Selbach, Nature Reviews Materials 7, 157 (2021).
  * Foster  _et al._ (2019) D. Foster, C. Kind, P. J. Ackerman, J.-S. B. Tai, M. R. Dennis, and I. I. Smalyukh, Nature Physics 15, 655 (2019).
  * Tang  _et al._ (2021) J. Tang, Y. Wu, W. Wang, L. Kong, B. Lv, W. Wei, J. Zang, M. Tian, and H. Du, Nature Nanotechnology 16, 1086 (2021).
  * Knapman  _et al._ (2021) R. Knapman, D. R. Rodrigues, J. Masell, and K. Everschor-Sitte, Journal of Physics D: Applied Physics 54, 404003 (2021).
  * Kong  _et al._ (2021) L. Kong, X. Chen, W. Wang, D. Song, and H. Du, Physical Review B 104, 214407 (2021).
  * Song  _et al._ (2022) D. Song, W. Wang, J.-X. Yu, P. Zhang, S. S. Pershoguba, G. Yin, W. Wei, J. Jiang, B. Ge, X. Fan, M. Tian, A. Rosch, J. Zang, and H. Du, (2022), arXiv:2212.08991 [cond-mat.mes-hall] .
  * Ezawa (2011) M. Ezawa, Physical Review B 83, 100408 (2011).
  * Müller  _et al._ (2017) J. Müller, J. Rajeswari, P. Huang, Y. Murooka, H. M. Rønnow, F. Carbone, and A. Rosch, Physical Review Letters 119, 137201 (2017).
  * Stepanova  _et al._ (2021) M. Stepanova, J. Masell, E. Lysne, P. Schoenherr, L. Köhler, M. Paulsen, A. Qaiumzadeh, N. Kanazawa, A. Rosch, Y. Tokura, A. Brataas, M. Garst, and D. Meier, Nano Letters 22, 14 (2021).
  * Lyuksyutov and Pokrovsky (2005) I. F. Lyuksyutov and V. L. Pokrovsky, Advances in Physics 54, 67 (2005).
  * Buzdin (2005) A. I. Buzdin, Reviews of Modern Physics 77, 935 (2005).
  * Aladyshkin  _et al._ (2009) A. Y. Aladyshkin, A. V. Silhanek, W. Gillijns, and V. V. Moshchalkov, Superconductor Science and Technology 22, 053001 (2009).
  * Helseth  _et al._ (2002) L. E. Helseth, P. E. Goa, H. Hauglin, M. Baziljevich, and T. H. Johansen, Physical Review B 65, 132514 (2002).
  * Vestgarden  _et al._ (2007) J. I. Vestgarden, D. V. Shantsev, A. A. F. Olsen, Y. M. Galperin, V. V. Yurchenko, P. E. Goa, and T. H. Johansen, Physical Review Letters 98, 117002 (2007).
  * Goa  _et al._ (2002) P. E. Goa, H. Hauglin, A. A. F. Olsen, D. Shantsev, and T. H. Johansen, Applied Physics Letters 82, 79 (2002).
  * Vlasko-Vlasov  _et al._ (2008) V. Vlasko-Vlasov, U. Welp, G. Karapetrov, V. Novosad, D. Rosenmann, M. Iavarone, A. Belkin, and W.-K. Kwok, Physical Review B 77, 134518 (2008).
  * Iavarone  _et al._ (2011) M. Iavarone, A. Scarfato, F. Bobba, M. Longobardi, G. Karapetrov, V. Novosad, V. Yefremenko, F. Giubileo, and A. M. Cucolo, Physical Review B 84, 024506 (2011).
  * Fu and Kane (2008) L. Fu and C. L. Kane, Physical Review Letters 100, 096407 (2008).
  * Lutchyn  _et al._ (2010) R. M. Lutchyn, J. D. Sau, and S. D. Sarma, Physical Review Letters 105, 077001 (2010).
  * Oreg  _et al._ (2010) Y. Oreg, G. Refael, and F. von Oppen, Physical Review Letters 105, 177002 (2010).
  * Kitaev (2001) A. Y. Kitaev, Physics-Uspekhi 44, 131 (2001).
  * Alicea (2012) J. Alicea, Reports on Progress in Physics 75, 076501 (2012).
  * Nothhelfer  _et al._ (2019) J. Nothhelfer, K. M. D. Hals, K. Everschor-Sitte, and M. Rizzi, European Patent Application EP3751472A1/International Patent Application WO2020249630A1 and Pending Patent CN114008639A (to be published) (2019).
  * Nothhelfer (2019) J. Nothhelfer, Master’s thesis, Johannes Gutenberg University Mainz (2019).
  * Díaz  _et al._ (2021) S. A. Díaz, J. Klinovaja, D. Loss, and S. Hoffman, Physical Review B 104, 214501 (2021).
  * Nothhelfer  _et al._ (2022) J. Nothhelfer, S. A. Díaz, S. Kessler, T. Meng, M. Rizzi, K. M. D. Hals, and K. Everschor-Sitte, Physical Review B 105, 224509 (2022).
  * Edelstein (1995) V. M. Edelstein, Physical Review Letters 75, 2004 (1995).
  * Edelstein (1996) V. M. Edelstein, Journal of Physics: Condensed Matter 8, 339 (1996).
  * Yip (2002) S. K. Yip, Physical Review B 65, 144508 (2002).
  * Fang and Thiele (1990) T. Fang and A. Thiele, IEEE Transactions on Magnetics 26, 1524 (1990).
  * Kabanov  _et al._ (2010) Y. P. Kabanov, Y. L. Iunin, V. I. Nikitenko, A. J. Shapiro, R. D. Shull, L. Y. Zhu, and C. L. Chien, IEEE Transactions on Magnetics 46, 2220 (2010).
  * Berger (1985) L. Berger, Journal of Applied Physics 58, 450 (1985).
  * Torrejon  _et al._ (2012) J. Torrejon, G. Malinowski, M. Pelloux, R. Weil, A. Thiaville, J. Curiale, D. Lacour, F. Montaigne, and M. Hehn, Physical Review Letters 109, 106601 (2012).
  * Jiang  _et al._ (2013) W. Jiang, P. Upadhyaya, Y. Fan, J. Zhao, M. Wang, L.-T. Chang, M. Lang, K. L. Wong, M. Lewis, Y.-T. Lin, J. Tang, S. Cherepov, X. Zhou, Y. Tserkovnyak, R. N. Schwartz, and K. L. Wang, Physical Review Letters 110, 177202 (2013).
  * Ashcroft and Mermin (1976) N. W. Ashcroft and N. D. Mermin,  _Solid State Physics_ (Holt, Rinehart and Winston, New York, U.S.A, 1976).
  * Note (1) We do so by identifying model parameters that make a $q_{v}=1$ vortex, in a field polarized magnetic background, the lowest-energy topologically nontrivial configuration of the superconducting order parameter.
  * Tinkham (1996) M. Tinkham,  _Introduction to Superconductivity_ (Dover Publications, New York, 1996).
  * Pershoguba  _et al._ (2015) S. S. Pershoguba, K. Björnson, A. M. Black-Schaffer, and A. V. Balatsky, Physical Review Letters 115, 116602 (2015).
  * Hals  _et al._ (2016) K. M. Hals, M. Schecter, and M. S. Rudner, Physical Review Letters 117, 017001 (2016).
  * Note (2) The contribution to the magnetoelectric coupling to second order in the SOC, as our calculations in App. D show, is negligible.
  * Kopnin (2001) N. Kopnin,  _Theory of Nonequilibrium Superconductivity_ (Oxford University PressOxford, 2001) p. 238.
  * Nakajima  _et al._ (1976) K. Nakajima, Y. Onodera, and Y. Ogawa, Journal of Applied Physics 47, 1620 (1976).
  * Reichhardt  _et al._ (2004) C. O. Reichhardt, C. Reichhardt, M. Hastings, and B. Jankó, Physica C: Superconductivity 404, 266 (2004).
  * Keren  _et al._ (2023) I. Keren, A. Gutfreund, A. Noah, N. Fridman, A. D. Bernardo, H. Steinberg, and Y. Anahory, Nano Letters 23, 4669 (2023).
  * Córdoba  _et al._ (2013) R. Córdoba, T. I. Baturina, J. Sesé, A. Y. Mironov, J. M. D. Teresa, M. R. Ibarra, D. A. Nasimov, A. K. Gutakovskii, A. V. Latyshev, I. Guillamón, H. Suderow, S. Vieira, M. R. Baklanov, J. J. Palacios, and V. M. Vinokur, Nature Communications 4, 2437 (2013).
  * Dobrovolskiy (2017) O. V. Dobrovolskiy, Physica C: Superconductivity and its Applications 533, 80 (2017).
  * Field  _et al._ (1995) S. Field, J. Witt, F. Nori, and X. Ling, Physical Review Letters 74, 1206 (1995).
  * Altshuler and Johansen (2004) E. Altshuler and T. H. Johansen, Reviews of Modern Physics 76, 471 (2004).
  * Laviano (2017) F. Laviano, in  _Vortices and Nanostructured Superconductors_ (Springer International Publishing, 2017) pp. 133–157.
  * Matsuda  _et al._ (2001) T. Matsuda, O. Kamimura, H. Kasai, K. Harada, T. Yoshida, T. Akashi, A. Tonomura, Y. Nakayama, J. Shimoyama, K. Kishio, T. Hanaguri, and K. Kitazawa, Science 294, 2136 (2001).
  * Grigorenko  _et al._ (2001) A. Grigorenko, S. Bending, T. Tamegai, S. Ooi, and M. Henini, Nature 414, 728 (2001).
  * Bending and Dodgson (2005) S. J. Bending and M. J. W. Dodgson, Journal of Physics: Condensed Matter 17, R955 (2005).
  * Córdoba  _et al._ (2019) R. Córdoba, P. Orús, Ž. L. Jelić, J. Sesé, M. R. Ibarra, I. Guillamón, S. Vieira, J. J. Palacios, H. Suderow, M. V. Milosević, and J. M. D. Teresa, Scientific Reports 9, 41598 (2019).
  * Song  _et al._ (2023) S. Y. Song, C. Hua, L. Bell, W. Ko, H. Fangohr, J. Yan, G. B. Halász, E. F. Dumitrescu, B. J. Lawrie, and P. Maksymovych, Nano Letters 23, 2822 (2023).
  * Cai  _et al._ (2021) R. Cai, Y. Yao, P. Lv, Y. Ma, W. Xing, B. Li, Y. Ji, H. Zhou, C. Shen, S. Jia, X. C. Xie, I. Žutić, Q.-F. Sun, and W. Han, Nature Communications 12, 41467 (2021).
  * Kezilebieke  _et al._ (2020) S. Kezilebieke, M. N. Huda, V. Vaňo, M. Aapro, S. C. Ganguli, O. J. Silveira, S. Głodzik, A. S. Foster, T. Ojanen, and P. Liljeroth, Nature 588, 424 (2020).
  * Kezilebieke  _et al._ (2022) S. Kezilebieke, V. Vaňo, M. N. Huda, M. Aapro, S. C. Ganguli, P. Liljeroth, and J. L. Lado, Nano Letters 22, 328 (2022).
  * Matricardi  _et al._ (1967) V. R. Matricardi, W. G. Lehmann, N. Kitamura, and J. Silcox, Journal of Applied Physics 38, 1297 (1967).
  * Sun  _et al._ (2021) Q.-C. Sun, T. Song, E. Anderson, A. Brunner, J. Förster, T. Shalomayeva, T. Taniguchi, K. Watanabe, J. Gräfe, R. Stöhr, X. Xu, and J. Wrachtrup, Nature Communications 12, 41467 (2021).
  * Yang  _et al._ (2022) H.-H. Yang, N. Bansal, P. Rüßmann, M. Hoffmann, L. Zhang, D. Go, Q. Li, A.-A. Haghighirad, K. Sen, S. Blügel, M. L. Tacon, Y. Mokrousov, and W. Wulfhekel, 2D Materials 9, 025022 (2022).
  * Birch  _et al._ (2022) M. T. Birch, L. Powalla, S. Wintz, O. Hovorka, K. Litzius, J. C. Loudon, L. A. Turnbull, V. Nehruji, K. Son, C. Bubeck, T. G. Rauch, M. Weigand, E. Goering, M. Burghard, and G. Schütz, Nature Communications 13, 41467 (2022).
  * Peng  _et al._ (2021) L. Peng, F. S. Yasin, T.-E. Park, S. J. Kim, X. Zhang, T. Nagai, K. Kimoto, S. Woo, and X. Yu, Advanced Functional Materials 31, 2103583 (2021).
  * Bode  _et al._ (2007) M. Bode, M. Heide, K. von Bergmann, P. Ferriani, S. Heinze, G. Bihlmayer, A. Kubetzka, O. Pietzsch, S. Blügel, and R. Wiesendanger, Nature 447, 190 (2007).
  * Chen and Schmid (2015) G. Chen and A. K. Schmid, Advanced Materials 27, 5738 (2015).
  * Chess  _et al._ (2017) J. J. Chess, S. A. Montoya, E. E. Fullerton, and B. J. McMorran, AIP Advances 7, 4977500 (2017).
  * Harada  _et al._ (1992) K. Harada, T. Matsuda, J. Bonevich, M. Igarashi, S. Kondo, G. Pozzi, U. Kawabe, and A. Tonomura, Nature 360, 51 (1992).
  * Moser  _et al._ (1995) A. Moser, H. J. Hug, I. Parashikov, B. Stiefel, O. Fritz, H. Thomas, A. Baratoff, H.-J. Güntherodt, and P. Chaudhari, Physical Review Letters 74, 1847 (1995).
  * Auslaender  _et al._ (2008) O. M. Auslaender, L. Luan, E. W. J. Straver, J. E. Hoffman, N. C. Koshnick, E. Zeldov, D. A. Bonn, R. Liang, W. N. Hardy, and K. A. Moler, Nature Physics 5, 35 (2008).
  * Hess  _et al._ (1989) H. F. Hess, R. B. Robinson, R. C. Dynes, J. M. Valles, and J. V. Waszczak, Physical Review Letters 62, 214 (1989).
  * Embon  _et al._ (2017) L. Embon, Y. Anahory, Ž. Jelić, E. O. Lachman, Y. Myasoedov, M. E. Huber, G. P. Mikitik, A. V. Silhanek, M. V. Milošević, A. Gurevich, and E. Zeldov, Nature Communications 8, 41467 (2017).
  * Hals (2016) K. M. D. Hals, Physical Review B 93, 115431 (2016).


11institutetext: Technical University of Munich, Munich, Germany
22institutetext: Georg-August-University Göttingen, Göttingen, Germany
33institutetext: GEOMAR Helmholtz Centre for Ocean Research Kiel, Kiel,
Germany 44institutetext: Helmholtz AI, Helmholtz Munich - German Research
Center for Environmental Health, Neuherberg, Germany
44email<EMAIL_ADDRESS>
# LUCYD: A Feature-Driven Richardson-Lucy Deconvolution Network

Tomáš Chobola 1144 0009-0000-3272-9996 Gesine Müller 22 Veit Dausmann 33
0000-0003-3281-9208 Anton Theileis 33 Jan Taucher 33 0000-0001-9944-0775 Jan
Huisken 22 0000-0001-7250-3756 Tingying Peng✉ 44 0000-0002-7881-1749

###### Abstract

The process of acquiring microscopic images in life sciences often results in
image degradation and corruption, characterised by the presence of noise and
blur, which poses significant challenges in accurately analysing and
interpreting the obtained data. This paper proposes LUCYD, a novel method for
the restoration of volumetric microscopy images that combines the Richardson-
Lucy deconvolution formula and the fusion of deep features obtained by a fully
convolutional network. By integrating the image formation process into a
feature-driven restoration model, the proposed approach aims to enhance the
quality of the restored images whilst reducing computational costs and
maintaining a high degree of interpretability. Our results demonstrate that
LUCYD outperforms the state-of-the-art methods in both synthetic and real
microscopy images, achieving superior performance in terms of image quality
and generalisability. We show that the model can handle various microscopy
modalities and different imaging conditions by evaluating it on two different
microscopy datasets, including volumetric widefield and light-sheet
microscopy. Our experiments indicate that LUCYD can significantly improve
resolution, contrast, and overall quality of microscopy images. Therefore, it
can be a valuable tool for microscopy image restoration and can facilitate
further research in various microscopy applications. We made the source code
for the model accessible under https://github.com/ctom2/lucyd-deconvolution/.

###### Keywords:

Deconvolution Deblurring Denoising Microscopy

##  1 Introduction

Microscopy is one of the most widely used imaging techniques that allows life
scientists to analyse cells, tissues and subcellular structures with a high
level of detail. However, microscopy images often suffer from degradation such
as blur, noise and other artefacts, which may result in an inaccurate
quantification and hinder downstream analysis. Therefore, deconvolution
techniques are necessary to restore the images to improve their quality, thus
increasing the accuracy of downstream tasks [16, 12, 7]. Image deconvolution
is a well-studied task in computer vision and imaging sciences that aims to
recover a sharp and clear object out of a degraded input. The mathematical
representation of image corruption can be expressed as:

$y=x*K+n,$ (1)

where $*$ represents convolution, $y$ denotes the resulting image of an object
$x$, which has been blurred with a point spread function (PSF) $K$, and
degraded by noise $n$.

Two classic image deconvolution methods widely used in microscopy and medical
imaging are Wiener filter [18] and Richardson-Lucy algorithm (RL) [11, 9]. The
Wiener filter is a linear filter that is applied to the frequency domain
representation of the blurred image. It assumes the Gaussian noise
distribution and thus minimises the mean squared error between the restored
image and the original one. The RL method, on the other hand, is an iterative
algorithm that works in the spatial domain, usually leading to better
reconstruction than Wiener filter. It assumes Poisson noise distribution and
seeks to estimate the corresponding sharp image $x$ in a fixed number of
iterations or until a convergence criterion is met [5]. While being simple and
effective, the main limitations of both methods are their susceptibility to
noise amplification [13, 3] and the assumption that the accurate PSF is known.
In practice, however, PSF is challenging to obtain and is often unknown or
varies across the image, which leads to inaccurate reconstructions of the
sharp image. Moreover, as an iterative method, RL is computationally costly
for three-dimensional (3D) data [4].

In the computer vision field, numerous deep learning models have been trained
on large datasets with the objective to learn a direct mapping between input
and output domains [6, 1, 10, 14, 15]. Some of these models have also been
adapted for use in microscopy, such as the U-Net-based content-aware image
restoration networks (CARE) [17]. These methods have exhibited exceptional
performance in tasks such as super-resolution and denoising. However, the
interpretability of these methods is limited, and given their data-driven
nature, the quantity and quality of training data can be a restricting factor,
particularly in biomedical applications where data pairs are often scarce or
not available.

Inspired by the RL algorithm, the Richardson-Lucy Network (RLN) [8] was
recently designed to overcome the problem of data-driven models by embedding
the RL formula for iterative image restoration into a neural network and
substituting convolutions of the measured PSF kernel with learnable
convolutional layers. Although being more compact than U-Net, the low capacity
of RLN makes it insufficiently robust to different blur intensities and noise
levels, requiring the network to be re-trained whenever there is a shift in
the input image domain. This reduces the efficacy of the method.

To address the limitations of existing methods, we propose a novel lightweight
model called LUCYD, which integrates the RL deconvolution formula and a
U-shaped network. The main contributions of this paper can be summarised as:

  1. 1.

LUCYD is a lightweight deconvolution method that embeds the RL deconvolution
formula into a deep convolutional network that leverages the features
extracted by a U-shaped module while maintaining low computational costs for
processing 3D microscopy images and a high level of interpretability.

  2. 2.

The proposed method outperforms existing deconvolution methods on both real
and synthetic datasets, based on qualitative assessment and quantitative
evaluation metrics, respectively.

  3. 3.

We show that LUCYD has strong resistance to noise and can generalise well to
new and unseen data. This ability makes it a valuable tool for practical
applications in microscopy imaging fields where image quality is critical for
downstream tasks yet training data are often scarce or unavailable.

##  2 Method

The overall architecture of the proposed model is illustrated in Figure 1 and
it comprises three main components: a correction module, an update module, and
a bottleneck that is shared between the two modules. The data flow in the
model is based on the following iterative RL formula:

$z_{(k)}=\underbrace{\vphantom{\left(\frac{0}{0_{(x)}}\right)}z_{(k-1)}}_{x\text{
estimate}}\cdot\underbrace{\left(\frac{y}{z_{(k-1)}*K}*K^{\top}\right)}_{\text{update
term}},$ (2)

which aims to recover $x$ in $k$ steps. We bypass the requirement of $k{-}1$
preceding iterations with the correction module that generates a mask $M$ to
form an intermediate sharp image estimation through a single forward pass,
allowing to rapidly process 3D data, as follows:

$\tilde{z}=y+M.$ (3)

Next, inspired by Li et al. [8], we adopt the three-step approach to decompose
the RL update term from Equation 2 in the update module:

$\text{(a)}\;\text{FP}=y*f\text{, (b)}\;\text{DV}={y}/\text{FP}\text{,
(c)}\;u=\text{DV}*b.$ (4)

Specifically, we replace convolutions with a known PSF in steps (a) and (c)
with forward projector $f$ and backward projector $b$, which consist of sets
of learnable convolutional layers. The produced update term $u$ allows us to
recondition the estimate $\tilde{z}$ from the correction module into a sharp
image through multiplication, i.e. the last step of image formation in the RL
formula: $x^{\prime}=\tilde{z}\cdot u$. The whole network can then be
expressed as follows,

$x^{\prime}=\tilde{z}\cdot\left(\frac{y}{y*f}*b\right).$ (5)

By adhering to the image formation steps as prescribed by the RL formula, we
maintain a high degree of interpretability, critical for real-world scenarios,
where the accuracy and reliability of the generated results are of utmost
importance.

Figure 1: The architecture of LUCYD consists of a correction module, an update
module and a bottleneck that is shared between the two modules.

(a)

(b)

Figure 2: The architecture of submodules: (a) Feature Fusion Block (FFBlock),
(b) Richardson-Lucy Division Block (RLDiv).

###  2.1 Correction Module & Bottleneck

The proposed correction module and bottleneck architectures consist of encoder
blocks (EBs), decoder blocks (DBs), and multi-scale feature fusion blocks to
facilitate efficient information exchange across different scales within the
model.

####  2.1.1 Feature Encoding

The features of the volumetric input image $y\in\mathbb{R}^{C\times D\times
H\times W}$ are obtained through the first encoder block $\text{EB}_{1}$ in
the correction module, and then encoded by a convolutional layer with a stride
2. Subsequently, the downsampled features are concatenated with the encoded
features of the forward projection $f$ from the update module and then fed to
the bottleneck encoder $\text{EB}_{2}$ to integrate the information from both
modules.

####  2.1.2 Feature Fusion Block

Similarly to Cho et al. [2], we enhance the connections between encoders and
decoders and allow information flow from different scales within the network
through Feature Fusion Blocks (FFBlocks). The features from $\text{EB}_{1}$
and $\text{EB}_{2}$ are refined as follows,

$\displaystyle\text{FFBlock}_{1}^{\text{out}}$
$\displaystyle=\text{FFBlock}_{1}\left(\text{EB}_{1}^{\text{out}},(\text{EB}_{2}^{\text{out}})^{\uparrow}\right),$
(6) $\displaystyle\text{FFBlock}_{2}^{\text{out}}$
$\displaystyle=\text{FFBlock}_{2}\left((\text{EB}_{1}^{\text{out}})^{\downarrow},\text{EB}_{2}^{\text{out}}\right),$
(7)

where up-sampling ($\uparrow$) and down-sampling ($\downarrow$) is applied to
allow for feature concatenation. The multi-scale features are then combined
and processed by $1\times 1$ and $3\times 3$ convolutional layers,
respectively, to allow the decoder blocks $\text{DB}_{1}$ and $\text{DB}_{2}$
to utilise information obtained on different scales. The structure of the
blocks is shown in Figure 2a.

####  2.1.3 Feature Decoding

Initially, the refined features are decoded in the bottleneck using a
convolutional layer and residual block within the $\text{DB}_{2}$. Next, these
features are expanded with a convolutional layer to match the dimensions in
both the correction and update modules. The resulting features are then
concatenated with the output of $\text{FFBlock}_{1}$ and subsequently fed into
decoder $\text{DB}_{2}$ within the correction module. The features are then
mapped to the image dimensions resulting in mask $M$, which is summed with $y$
to form $\tilde{z}$.

###  2.2 Update Module

Inspired by the forward and backward projector functions [8], we substitute
the PSF convolution operations from Richardson-Lucy algorithm with learnable
convolutional layers and residual blocks.

During forward projection (FP), shallow features are initially extracted by a
single convolutional layer and then refined by a residual block. The output of
$f$ is then passed to Richardson-Lucy Division Block (RLDiv) which embeds the
division of the raw image $y$ by the channel-wise mean of the refined FP
features. Next, we project the division result to a feature map to extract
more information about the image. The visualisation of the process is in
Figure 2b. These features are then concatenated with the features extracted by
the bottleneck and combined by a convolutional layer which initiates the
backward projection with $b$. The output is then summed with the output of
RLDiv, forming a skip-connection, and passed through a residual block. The
features are then refined by a convolutional layer and their channel-wise mean
is taken to be the update term $u$, which is used to obtain the final model
output $x^{\prime}$ through multiplication with $\tilde{z}$ (denoted as
RLMul).

###  2.3 Loss Function

The entire model is trained end-to-end with a single loss function that
combines the Mean Squared Error (MSE) and the Structural Similarity Index
Measure (SSIM) as follows:

$\mathcal{L}(x^{\prime},x)=\text{MSE}(x^{\prime},x)-\ln\left(\frac{1+\text{SSIM}(x^{\prime},x)}{2}\right),$
(8)

where $x$ is the ground truth sharp image and $x^{\prime}$ is the model
estimation of $x$.

##  3 Experiments

(a) $y$

(b) $\tilde{z}$

(c) $u$

(d) $x^{\prime}$

(e) $x$ (GT)

Figure 3: Overview of the deconvolution process given an input (a). The
outputs of the correction module and the update module are shown in (b) and
(c), respectively, and the final output obtained through their multiplication
is in (d).

###  3.1 Setup

Table 1: Number of learnable parameters, comparing CARE, RLN and LUCYD. CARE [17] | RLN [8] | LUCYD (ours)
---|---|---
1 M | 15,900 | 24,964

####  3.1.1 Datasets

We assess the performance of LUCYD on both simulated phantom objects and real
microscopy images. To achieve this, we use five sets of 3D grayscale volumes
generated by Li et al. [8], consisting of dots, solid spheres, and ellipsoidal
surfaces, which are provided along with their sharp ground truth volumes of
dimensions $128\times 128\times 128$ (one exemplary image shown in Figure 3e).
To test the generalization capabilities of our method, we also include two
blurry and noisy versions of the dataset, $\mathcal{D}_{\text{nuc}}$ and
$\mathcal{D}_{\text{act}}$, which utilize different image degradation
processes for embryonic nuclei and membrane data. Additionally, we generate a
mixed dataset by applying permutations of three Gaussian blur intensities
($\sigma_{b}=[1.0,1.2,1.5]$) and three levels of additive Gaussian noise
($\sigma_{n}=[0,15,30]$) to the ground truth volumes, and then test the
ability of the model to generalize to volumes blurred with Gaussian kernels
($\sigma_{b}=[0.5,2.0]$) and additive Gaussian noise
($\sigma_{n}=[20,50,70,100]$) levels outside of the training dataset. The
model is trained on patches of dimensions $32\times 64\times 64$ that are
randomly sampled from the training datasets. Moreover, we evaluate the model
trained using synthetic phantom shapes on a real 3D light-sheet image of a
starfish (private data) and widefield microscopy image of U2OS cell (from the
dataset of [8]), to explore the generalisation capabilities.

####  3.1.2 Baseline & Metrics

We employ one classic U-Net-based fluorescence image restoration model CARE
[17] and one RL-based convolutional model RLN [8] as baselines. We
quantitatively evaluate the deconvolution performance on simulated data using
two metrics: Structural Similarity Index Measure (SSIM) and Peak Signal-to-
Noise Ratio (PSNR).

Table 2: Performance on synthetic datasets (SSIM/PSNR (dB)) degraded with blur and noise levels not present in the training dataset. The models are trained on phantom objects blurred with $\sigma_{b}=[1.0,1.2,1.5]$ and corrupted with Gaussian noise intensities $\sigma_{n}=[0,15,30]$. Blur intensity $\sigma_{b}$ | Noise level $\sigma_{n}$ | CARE [17] | RLN [8] | LUCYD (ours)
---|---|---|---|---
0.5 | 20 | $0.9166/21.62$ | $0.9571/25.60$ | $\mathbf{0.9725/26.85}$
0.5 | 50 | $0.7589/15.96$ | $0.8519/21.67$ | $\mathbf{0.9463/24.35}$
0.5 | 70 | $0.6828/14.32$ | $0.7235/18.52$ | $\mathbf{0.9040/21.78}$
0.5 | 100 | $0.5856/12.56$ | $0.5644/15.91$ | $\mathbf{0.7233/17.47}$
2.0 | 20 | $0.8582/20.36$ | $0.9040/22.34$ | $\mathbf{0.9271/23.49}$
2.0 | 50 | $0.7057/16.35$ | $0.7443/18.85$ | $\mathbf{0.8575/21.00}$
2.0 | 70 | $0.6259/15.06$ | $0.6051/16.69$ | $\mathbf{0.7995/19.38}$
2.0 | 100 | $0.5154/13.08$ | $0.4495/14.86$ | $\mathbf{0.6311/16.42}$
Table 3: Performance on synthetic datasets (SSIM/PSNR (dB)) given varying training data. | Train dataset | Test dataset | CARE [17] | RLN [8] | LUCYD (ours)
---|---|---|---|---|---
In-domain | $\mathcal{D}_{\text{nuc}}$ | $\mathcal{D}_{\text{nuc}}$ | $0.7895/18.00$ | $0.9247/26.43$ | $\mathbf{0.9525/28.57}$
$\mathcal{D}_{\text{act}}$ | $\mathcal{D}_{\text{act}}$ | $0.7666/17.44$ | $0.8966/26.10$ | $\mathbf{0.9450/27.83}$
Cross-domain | $\mathcal{D}_{\text{nuc}}$ | $\mathcal{D}_{\text{act}}$ | $0.7623/17.68$ | $0.8841/24.33$ | $\mathbf{0.9024/24.82}$
$\mathcal{D}_{\text{act}}$ | $\mathcal{D}_{\text{nuc}}$ | $0.7584/17.00$ | $0.9081/27.23$ | $\mathbf{0.9336/27.63}$
Figure 4: Quantitative comparison of RLN and LUCYD on lateral and axial
maximum-intensity projections of a starfish acquired by 3D light-sheet
microscopy is shown in (a). Additional analysis of the deconvolution results
of CARE, RLN and LUCYD trained on synthetic phantom objects in (b) shows
patches of four-colour lateral maximum-intensity projections of a fixed U2OS
cell acquired by widefield microscopy (from the dataset of [8]). LUCYD
exhibits superior performance in recovering fine details and structures as
compared to CARE and RLN, while simultaneously maintaining low levels of noise
and haze surrounding the objects.

###  3.2 Results

In Table 2, we present the quantitative results of all three methods on
simulated phantom objects degraded with blur and noise levels that were not
present in the training dataset. LUCYD achieves the best performance even in
cases where the amount of additive noise exceeds the maximum level included in
the training dataset. This is in contrast to CARE and RLN, which did not
demonstrate such exceptional generalisation capabilities and noise resistance.
We further examine LUCYD’s performance on datasets simulating widefield
microscopy imaging of embryo nuclei and membrane data. As shown in Table 3,
LUCYD outperforms CARE and RLN in both in-domain and cross-domain assessments,
further supporting the model’s capabilities in cross-domain applications.

We finally apply LUCYD on two real microscopy test samples, as illustrated in
Figure 4. On the 3D light-sheet image of starfish, LUCYD recovers more details
and structures than RLN while maintaining low levels of noise and haze
surrounding the object in both lateral and axial projections. On the other
test sample of a fixed U2OS cell acquired by widefield microscopy, LUCYD also
suppresses noise and haze to a higher degree compared to RLN and CARE and
retrieves finer and sharper details.

##  4 Conclusion

In this paper, we introduce LUCYD, an innovative technique for deconvolving
volumetric microscopy images that combines a classic image deconvolution
formula with a U-shaped network. LUCYD takes advantages of both approaches,
resulting in a highly efficient method capable of processing 3D data with high
efficacy. We have demonstrated through experiments on both synthetic and real
microscopy datasets that LUCYD exhibits strong generalization capabilities, as
well as robustness to noise. These qualities make it an excellent tool for
cross-domain applications in various domains, such as biology and medical
imaging. Additionally, the lightweight nature of LUCYD makes it
computationally feasible for real-time applications, which can be crucial in
various settings.

####  4.0.1 Acknowledgements

Tomáš Chobola is supported by the Helmholtz Association under the joint
research school "Munich School for Data Science - MUDS".

## References

  * [1] Chen, J., Sasaki, H., Lai, H., Su, Y., Liu, J., Wu, Y., Zhovmer, A., Combs, C.A., Rey-Suarez, I., Chang, H.Y., Huang, C.C., Li, X., Guo, M., Nizambad, S., Upadhyaya, A., Lee, S.J.J., Lucas, L.A.G., Shroff, H.: Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes. Nature Methods 18(6), 678–687 (May 2021). https://doi.org/10.1038/s41592-021-01155-x
  * [2] Cho, S.J., Ji, S.W., Hong, J.P., Jung, S.W., Ko, S.J.: Rethinking coarse-to-fine approach in single image deblurring. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4641–4650 (2021)
  * [3] Dell’Acqua, F., Scifo, P., Rizzo, G., Catani, M., Simmons, A., Scotti, G., Fazio, F.: A modified damped richardson–lucy algorithm to reduce isotropic background effects in spherical deconvolution. NeuroImage 49(2), 1446–1458 (2010). https://doi.org/https://doi.org/10.1016/j.neuroimage.2009.09.033
  * [4] Dey, N., Blanc-Feraud, L., Zimmer, C., Roux, P., Kam, Z., Olivo-Marin, J.C., Zerubia, J.: Richardson–lucy algorithm with total variation regularization for 3d confocal microscope deconvolution. Microscopy Research and Technique 69(4), 260–266 (2006). https://doi.org/10.1002/jemt.20294
  * [5] Eichstädt, S., Schmähling, F., Wübbeler, G., Anhalt, K., Bünger, L., Krüger, U., Elster, C.: Comparison of the richardson–lucy method and a classical approach for spectrometer bandpass correction. Metrologia 50(2), 107 (2013)
  * [6] Guo, M., Li, Y., Su, Y., Lambert, T., Nogare, D.D., Moyle, M.W., Duncan, L.H., Ikegami, R., Santella, A., Rey-Suarez, I., Green, D., Beiriger, A., Chen, J., Vishwasrao, H., Ganesan, S., Prince, V., Waters, J.C., Annunziata, C.M., Hafner, M., Mohler, W.A., Chitnis, A.B., Upadhyaya, A., Usdin, T.B., Bao, Z., Colón-Ramos, D., Riviere, P.L., Liu, H., Wu, Y., Shroff, H.: Rapid image deconvolution and multiview fusion for optical microscopy. Nature Biotechnology 38(11), 1337–1346 (Jun 2020). https://doi.org/10.1038/s41587-020-0560-x
  * [7] Kaderuppan, S.S., Wong, E.W.L., Sharma, A., Woo, W.L.: Smart nanoscopy: A review of computational approaches to achieve super-resolved optical microscopy. IEEE Access 8, 214801–214831 (2020). https://doi.org/10.1109/ACCESS.2020.3040319
  * [8] Li, Y., Su, Y., Guo, M., Han, X., Liu, J., Vishwasrao, H.D., Li, X., Christensen, R., Sengupta, T., Moyle, M.W., Rey-Suarez, I., Chen, J., Upadhyaya, A., Usdin, T.B., Colón-Ramos, D.A., Liu, H., Wu, Y., Shroff, H.: Incorporating the image formation process into deep learning improves network performance. Nature Methods 19(11), 1427–1437 (Oct 2022). https://doi.org/10.1038/s41592-022-01652-7, https://doi.org/10.1038/s41592-022-01652-7
  * [9] Lucy, L.B.: An iterative technique for the rectification of observed distributions. The Astronomical Journal 79, 745 (Jun 1974). https://doi.org/10.1086/111605
  * [10] Qiao, C., Li, D., Guo, Y., Liu, C., Jiang, T., Dai, Q., Li, D.: Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nature Methods 18(2), 194–202 (Jan 2021). https://doi.org/10.1038/s41592-020-01048-5
  * [11] Richardson, W.H.: Bayesian-based iterative method of image restoration$\ast$. J. Opt. Soc. Am. 62(1), 55–59 (Jan 1972). https://doi.org/10.1364/JOSA.62.000055, http://www.osapublishing.org/abstract.cfm?URI=josa-62-1-55
  * [12] Sage, D., Donati, L., Soulez, F., Fortun, D., Schmit, G., Seitz, A., Guiet, R., Vonesch, C., Unser, M.: DeconvolutionLab2: An open-source software for deconvolution microscopy. Methods 115, 28–41 (Feb 2017). https://doi.org/10.1016/j.ymeth.2016.12.015
  * [13] Tan, K., Li, W., Zhang, Q., Huang, Y., Wu, J., Yang, J.: Penalized maximum likelihood angular super-resolution method for scanning radar forward-looking imaging. Sensors 18(3), 912 (Mar 2018). https://doi.org/10.3390/s18030912
  * [14] Vizcaíno, J.P., Saltarin, F., Belyaev, Y., Lyck, R., Lasser, T., Favaro, P.: Learning to reconstruct confocal microscopy stacks from single light field images. IEEE Transactions on Computational Imaging 7, 775–788 (2021). https://doi.org/10.1109/TCI.2021.3097611
  * [15] Wagner, N., Beuttenmueller, F., Norlin, N., Gierten, J., Boffi, J.C., Wittbrodt, J., Weigert, M., Hufnagel, L., Prevedel, R., Kreshuk, A.: Deep learning-enhanced light-field imaging with continuous validation. Nature Methods 18(5), 557–563 (May 2021). https://doi.org/10.1038/s41592-021-01136-0
  * [16] Wallace, W., Schaefer, L.H., Swedlow, J.R.: A workingperson’s guide to deconvolution in light microscopy. BioTechniques 31(5), 1076–1097 (Nov 2001). https://doi.org/10.2144/01315bi01
  * [17] Weigert, M., Schmidt, U., Boothe, T., Müller, A., Dibrov, A., Jain, A., Wilhelm, B., Schmidt, D., Broaddus, C., Culley, S., Rocha-Martins, M., Segovia-Miranda, F., Norden, C., Henriques, R., Zerial, M., Solimena, M., Rink, J., Tomancak, P., Royer, L., Jug, F., Myers, E.W.: Content-aware image restoration: pushing the limits of fluorescence microscopy. Nature Methods 15(12), 1090–1097 (Nov 2018). https://doi.org/10.1038/s41592-018-0216-7
  * [18] Wiener, N.: Extrapolation, Interpolation, and Smoothing of Stationary Time Series. The MIT Press (1949). https://doi.org/10.7551/mitpress/2946.001.0001