# Transient nuclear inversion by X-Ray Free Electron Laser in a tapered x-ray
waveguide

Yu-Hsueh Chen  Department of Physics, National Central University, Taoyuan
City 32001, Taiwan  Max-Planck-Institut für Kernphysik, Saupfercheckweg 1,
D-69117 Heidelberg, Germany  Po-Han Lin  Department of Physics, National
Central University, Taoyuan City 32001, Taiwan  Guan-Ying Wang  Department of
Physics, National Central University, Taoyuan City 32001, Taiwan  Adriana
Pálffy  adriana.palffy-buss.fau.de Department of Physics, Friedrich-Alexander-
Universität Erlangen-Nürnberg, D-91058, Erlangen, Germany  Max-Planck-Institut
für Kernphysik, Saupfercheckweg 1, D-69117 Heidelberg, Germany  Wen-Te Liao
<EMAIL_ADDRESS>Department of Physics, National Central University,
Taoyuan City 32001, Taiwan  Max-Planck-Institut für Kernphysik,
Saupfercheckweg 1, D-69117 Heidelberg, Germany  Physics Division, National
Center for Theoretical Sciences, Hsinchu 30013, Taiwan

###### Abstract

By restricting the spatial energy transmission of an electromagnetic wave,
dielectric waveguides transmit light over long distances at sustained
intensity. Waveguides have been used in the microwave and optical range to
maintain strong signal intensities in connection with lasers, but guiding of
intense short-wavelength radiation such as x-rays has proven more cumbersome.
Here we investigate theoretically how tapered x-ray waveguides can focus and
guide radiation from x-ray free electron lasers. Elliptical waveguides using a
cladding material with high atomic number such as platinum can maintain an
x-ray intensity up to three orders of magnitude larger than in free space.
This feature can be used to significantly enhance resonant interactions of
x-rays, for instance driving nuclear transitions up to transient nuclear
population inversion. This could be the first breakthrough in nuclear state
population control. Our results anticipate the important role of tapered x-ray
waveguides in the emerging field of x-ray quantum optics with nuclear
transitions.

x-ray quantum optics, x-ray free electron laser, waveguides, nuclear quantum
optics

More than sixty years ago, it was the invention of the laser that
revolutionized atomic physics, leading to a better understanding of atomic and
molecular dynamics and to a vast number of applications in both fundamental
and applied sciences. With only optical frequencies available, the interaction
of coherent light with matter was for a long time mainly restricted to driving
and control of atomic transitions. Fortunately, since 2009 we have witnessed
the commissioning and operation of the first x-ray free electron laser (XFEL)
facilities Emma et al. (2010); Pellegrini et al. (2016), opening for study the
interaction of high-frequency lasers with matter. X-ray photons carry large
momentum, are easy to detect, can be focused to much smaller spot sizes, can
penetrate more deeply into materials and, in principle, can support faster
information processing because of their higher frequencies Kuznetsova and
Kocharovskaya (2017). As such, they would be very desirable as information
carriers for quantum technologies, for exploring the boundary between the
classical and quantum world Raimond et al. (2001), or for matter-probing
techniques based on quantum effects.

Quantum optics with visible and infrared photons relies on resonant
interactions of photons with atoms Raimond et al. (2001). In contrast, with
their high frequencies, x-rays are no longer resonant to valence electron
transitions in atoms, but rather with inner-shell transitions in highly
charged ions, or even with keV electromagnetic transitions in the atomic
nucleus Adams et al. (2013). An advantage of interactions based on nuclear
over electronic transitions is that nuclear transitions have narrow spectral
linewidths even in bulk solids at room temperature. This renders them very
clean quantum systems, with long coherence times. These desirable properties
are due to the large mass and small size of nuclei, as well as the possibility
for recoilless absorption of gamma rays and x-rays (the Mössbauer effect)
Kuznetsova and Kocharovskaya (2017); Vagizov et al. (2014). The downside is
that due to the tiny nuclear size, the light-nuclear coupling is typically
five orders of magnitude weaker than the electromagnetic coupling of any
atomic system. Thus, x-ray quantum optics experiments have been successfully
performed at the single-photon level with a very narrow Mössbauer nuclear
transition at 14.4 keV in the stable 57Fe nucleus Röhlsberger et al. (2010,
2012); Heeg et al. (2013); Adams et al. (2013); Vagizov et al. (2014); Heeg et
al. (2015a, b, 2017); Haber et al. (2017); Chumakov et al. (2018); Heeg et al.
(2021). However, even intense XFEL radiation is still far from achieving
efficient nuclear pumping or nuclear population inversion Chumakov et al.
(2018). Apart from x-ray quantum optics incentives, efficient driving of
nuclear transitions and nuclear population control would be desirable for a
safe and clean energy storage solution based on nuclear isomers, i.e., long-
lived nuclear excited states Walker and Dracoulis (1999).

An intuitive solution to this probem is given by x-ray nano-focusing, either
in free space on in specially designed structures. At present, tight x-ray
nano-focusing optics is yet to be developed at XFEL facilities LCL (a); SAC ;
Eur ; Swi ; Pal . Thus, x-ray and nuclear quantum optics applications would
very much benefit from the development of structures that enhance the light-
matter interactions such as waveguides or cavities. So far, grazing incidence
x-ray cavities with embedded Mössbauer nuclei Röhlsberger et al. (2010, 2012);
Heeg et al. (2013, 2015a, 2015b); Haber et al. (2016, 2017) or normal-
incidence cavities based on diamond mirrors Shvyd’ko et al. (2010, 2011) have
been shown to be very lossy. Theoretical prospects of strong nuclear
excitation in x-ray thin-film cavities were discussed in Ref. Heeg et al.
(2016). X-ray waveguides have been designed and fabricated to work with x-ray
radiation from synchrotron sources Pfeiffer et al. (2002); Jarre et al.
(2005); Krüger et al. (2012); Chen et al. (2015); Hoffmann-Urlaub et al.
(2016). Such waveguides have been successfully used as point-like hard x-ray
sources for imaging and in particular for holographical imaging Salditt et al.
(2015).

Here, we investigate for the first time the theoretical prospects of tapered
x-ray waveguides (WG) for efficient nuclear pumping of a nuclear transition by
an XFEL. A tapered WG presents a gradual narrowing structure design of the
cladding material. Such a WG not only confines but also focuses x-rays Stern
et al. (1988); Denisov et al. (1991); Bergemann et al. (2003); Krüger et al.
(2012); Chen et al. (2015). This boosts the x-ray capability of pumping
nuclear excitations. An additional important advantage of the WG is that
nuclear resonance fluorescence therein will proceed by photons emitted in
designed modes, leading to directional emission even without the generation of
a superradiant state Röhlsberger et al. (2010). This is relevant for
downstream detection and applications. The waveguide setup is illustrated in
Fig. 1(a). A nuclear sample is embedded around the focal point of an X-ray
cladding waveguide (WG) depicted in Fig. 1(a) by the gray bottle-shaped
volume. The typical parameters describing a tapered WG are the focusing length
$L_{f}$, the input (output) port radius $r_{i}$ ($r_{o}$), the cladding
material, and the cladding-vacuum interface geometry illustrated in Fig. 1(b).
As a main example we consider the Mössbauer nuclear transition from the stable
57Fe ground state to the first excited nuclear state at 14.4 keV. This
corresponds to an x-ray wavelength of approx. 1 Å and has magnetic dipole
multipolarity. Similar transitions in 133Ba, 187Os, and 169Tm that will be
also addressed are presented in Table 1.

Figure 1:  Nuclear sample embedded in an X-ray waveguide. (a) XFEL radiation
is focused on the tapered WG. A nuclear sample (yellow cylinder) is placed at
the focal point of the WG (sketch not to scale). The x-ray intensity in the
waveguide is illustrated below. (b) A tapered WG has a bottle-like shape, with
input radius $r_{i}$, output radius $r_{o}$ and focusing length $L_{f}$. (c)
57Fe nuclear level scheme. A linearly polarized XFEL pulse (blue arrows)
drives the two $\Delta m=0$ transitions. We label the two ground states with
$|1\rangle$, $|2\rangle$ and the relevant excited states with $|3\rangle$ and
$|4\rangle$, respectively.

In the theoretical modelling we consider in the following the XFEL pulse to be
linearly polarized and fully coherent. The effects of limited coherence will
be modeled and discussed separately in the Results. Fully coherent x-ray
pulses can be delivered by seeded XFELs Feldhaus et al. (1997); Amann et al.
(2012); Serkez (2016); Emma et al. (2017); Inoue et al. (2019); Nam et al.
(2021), an XFEL oscillator (XFELO) Kim et al. (2008); Zemella et al. (2012);
Adams et al. (2019); Shvyd’ko (2019); Rauer et al. (2019); Marcus et al.
(2020), or their combination Li et al. (2018); Paraskaki et al. (2019). We
would like to highlight here the very recent report of successful seeding at
the Pohang Accelerator Laboratory XFEL in South Korea, which demonstrated 40
times higher peak brightness than in self-amplified spontaneous emission, with
pulses close to the Fourier limit and energies up to 14.6 keV Nam et al.
(2021). This energy range would cover all examples of nuclear transitions
discussed here. The nuclear transitions usually experience hyperfine splitting
according to their respective spins. For all four considered transitions,
$I_{u}=3/2$ and $I_{d}=1/2$ are the nuclear spins of the ground (down, $d$)
and excited (up, $u$) states, respectively. We consider a setup geometry which
allows the XFEL pulse to drive the two transitions $\Delta m=m_{u}-m_{d}=0$,
where $m_{u}$ ($m_{d}$) are the spin projections of the excited (ground)
nuclear state on the quantization axis. The dynamics of the system can be
modelled theoretically by the so-called Maxwell-Bloch equations Scully and
Zubairy (2006); Agarwal (2012) which combine the dynamics of the density
matrix of the nuclear system with the Maxwell equation describing the coherent
field propagation. The nuclear dynamics can be expressed in the Liouville
equation Bürvenich et al. (2006); Liao et al. (2011, 2012a, 2012b, 2013); Kong
et al. (2014); Kong and Pálffy (2016); Wang and Liao (2018)

$\dot{\hat{\rho}}=-\frac{i}{\hbar}\left[\hat{H},\hat{\rho}\right]+\mathcal{L}[\hat{\rho}]\,,$
(1)

where $\hat{\rho}$ is the density matrix describing the nuclear states,
$\hbar$ stands for the reduced Planck constant, and $\hat{H}$ is the
Hamiltionian of the system. Furthermore, the losses in the system are
described by the Lindblad superoperator $\mathcal{L}[\hat{\rho}]$. The Maxwell
wave equation describing the coherent propagation of the x-ray electric field
modulus $E=|\vec{E}|$ through the medium is given by Shvyd’ko et al. (1998);
Fuhse and Salditt (2006); Liao et al. (2012a); Kong et al. (2014)

$\displaystyle\left(\partial_{x}^{2}+\partial_{y}^{2}\right)E+2ik\left(\partial_{z}+\frac{1}{c}\partial_{t}\right)E+k^{2}\left[n\left(\vec{r}\right)^{2}-1\right]E$
(2) $\displaystyle=$
$\displaystyle-\left[\frac{8\pi\hbar\Gamma_{s}N\left(\vec{r}\right)}{\Pi
k}\frac{2I_{u}+1}{2I_{d}+1}\frac{f_{lm}}{\alpha+1}\right]\left(\rho_{31}+\rho_{42}\right).$

Here, $k$ is the x-ray wave vector and $c$ denotes the speed of light in
vacuum. The complex x-ray index of refraction determined by the electrons in
the sample is denoted by
$n\left(\vec{r}\right)=1-\delta\left(\vec{r}\right)+i\beta\left(\vec{r}\right)$
Fuhse and Salditt (2006); Bukreeva et al. (2006); Chen et al. (2015), with the
values for different materials presented in Table 1. Furthermore, $\Gamma_{s}$
is the spontaneous decay rate of the nuclear excited state,
$N\left(\vec{r}\right)$ the nuclear particle number density distribution,
$f_{lm}$ is the Lamb-Mössbauer factor, and $\alpha$ is the internal conversion
coefficient, respectively. The notation $\Pi$ defined in the Methods stands
for the nuclear matrix element and $\rho_{31}$ and $\rho_{42}$ are the off-
diagonal density matrix elements for the nuclear-four-level scheme depicted in
Fig. 1(c). The two coupled equations (1) and (2) are solved numerically in
cylindrical coordinates $(r,z)$ (see Methods).

##  I Results

Figure 2:  Focusing X-rays in a tapered waveguide. (a) Intrawaveguide x-ray
concentration ratio $R_{c}(r=0,z)$ (left vertical axis) and transmission $T$
(right vertical axis) of the CWG (blue empty squares and blue-dashed line) and
EWG (red dots and red-dashed-dotted line). For comparison we consider both WG
with ($r_{i}$, $r_{o}$) = (150 nm, 15 nm). (b, c) Contour plots of x-ray
concentration distribution $R_{c}\left(r,z\right)$ for a Pt-cladding (b) CWG
and (c) EWG. The common WG parameters are ($r_{i}$, $L_{f}$, $r_{o}$) = (150
nm, 160 $\mu$m, 15 nm). The input x-ray pulse has a transverse Gaussian form
with $\sigma=$120 nm. The red-upward arrow pinpoints the EWG focal point.
White-dashed lines indicate the platinum cladding and vacuum interface. Insets
in (b, c) illustrate radial and axial $R_{c}$ values about the WG focal point.
The black horizontal arrows indicate the focal spot size (left) and
intrawaveguide Rayleigh length (right).

We first characterize the quality of the x-ray waveguide. To this end we
define and calculate the intrawaveguide x-ray concentration ratio $R_{c}$ and
the transmission $T$

$\displaystyle R_{c}\left(r,z\right)$ $\displaystyle=$
$\displaystyle\frac{|E\left(r,z\right)|^{2}}{|E\left(0,0\right)|^{2}},$ (3)
$\displaystyle T$ $\displaystyle=$
$\displaystyle\frac{\int_{0}^{r_{o}}|E\left(r,z_{f}\right)|^{2}rdr}{\int_{0}^{r_{i}}|E\left(r,0\right)|^{2}rdr},$
(4)

for different waveguide geometries. Here $z=z_{f}$ is the axial position where
the maximum intrawaveguide x-ray intensity occurs. From their definitions it
is clear that $R_{c}$ depicts the degree of x-ray focusing and the spatial
intensity distribution, while $T$ reveals the percentage of transmitted (and
therefore not lost) x rays in a WG. In Fig. 2 we present our steady-state
numerical solutions of equation (2) with only the electronic contribution of
the cladding index of refraction (see Table 1). The photon loss usually
accompanies the reflection on the cladding-vacuum interface. Intuitively, an
elliptical waveguide (EWG), whose interface is elliptical, will guide x rays
from one focal point to another via only a single reflection on the interface.
One can therefore take this advantage to reduce photon loss when channelling
XFEL through a EWG. In order to show this advantage, we compare the EWG with a
conical waveguide (CWG), whose cladding-vacuum interface is a cone.

We consider an EWG and a CWG, both sharing the geometrical parameters
($r_{i},r_{o}$) = (150 nm, 15 nm), illuminated by the same input XFEL of beam
waist $\sigma=$120 nm at WG input port. X-rays experience focusing for $z\leq
160\ \mu$m in both WGs, but behave very differently for $z>160\ \mu$m. By
varying the $L_{f}$, Fig. 2(a) demonstrates the performance of the two types
of tapered WG and shows the $L_{f}$-dependent axial x-ray concentration
$R_{c}\left(0,z_{f}\right)$ and the transmission $T$. We notice that the
performance of EWG is generally better than that of CWG. The EWG transmission
becomes higher than that of CWG when $L_{f}\geq 80\ \mu$m, and the gap between
the two $R_{c}$ curves is increasing for $L_{f}>40\ \mu$m. In contrast to the
maximum CWG $R_{c}\approx 250$ at $L_{f}=90\ \mu$m, EWG $R_{C}$ remarkably
reaches the greatest concentration ratio of approx. 800 at $L_{f}=150\ \mu$m.
The product $TR_{c}\left(0,z_{f}\right)$ can be used as the WG figure of
merit. The EWG’s value reaches the optimized case around $L_{f}=160\ \mu$m,
where it offers both tight X-ray focusing and high transmission.

In order to demonstrate a detailed comparison between the two geometries,
Figs. 2(b-c) illustrate the spatial distribution of $R_{c}\left(r,z\right)$
for Pt-cladding CWG and EWG at $L_{f}$ = 160 $\mu$m where the EWG is
optimized. The axial dependence defines the intrawaveguide Rayleigh length as
the full width half maximum (FWHM) of $R(0,z)$. The EWG $R_{c}$ value is four
times that of CWG at the maximal point $z_{\rm max}\approx 161~{}\mu$m where
$R_{c}\left(0,z_{\mathrm{max}}\right)\approx 800$ (see red arrow in Fig. 2c).
The EWG is capable of better focusing and maintaining higher transmission than
the CWG. Although the CWG provides looser focusing compared to the EWG, its
more uniform field distribution may render it a good platform for observing
collective emission Röhlsberger et al. (2010). For the present purpose of
producing as strong a nuclear excitation as possible, we will focus on EWG in
the following.

XFEL-pumped transient nuclear inversion in an elliptical waveguide. Let us now
investigate the efficiency of nuclear excitation in an EWG. To this end we
numerically solve Eqs. (1) and (2) (see Methods). The nuclear inversion is
defined as $I_{v}=\rho_{33}+\rho_{44}-\rho_{11}-\rho_{22}$ Bürvenich et al.
(2006) (see Methods). We can express it with the help of the x-ray
concentration $R_{c}(r,z)$ and further pulse parameters such as the beam waist
$\sigma$ and the number of photons in the pulse $n_{p}$ (see Methods). The
required maximum XFEL spot radius $\sigma_{\mathrm{max}}$ for achieving full
nuclear inversion $I_{v}=1$ can be estimated via Eq. (10). Considering
$R_{c}=1$, i.e., no waveguide focusing effects, pulse duration $\tau=42.5$ fs
corresponding to the FWHM $2\sqrt{2\ln 2}\tau=100$ fs of the European XFEL
Geloni et al. (2010), and the values of isotopes listed in Table 1, we arrive
at $\sigma_{\mathrm{max}}\approx\sqrt{n_{p}}\times 10^{-5}$ nm. Given
$n_{p}=10^{12}$ for European XFEL Geloni et al. (2010),
$\sigma_{\mathrm{max}}\lesssim 10$ nm is required to achieve full nuclear
inversion for all the presently considered isotopes.

However, these requirements are easied by the large values that can be
achieved for $R_{c}$ in the EWG. Based on Eq. (10) in the Methods, three
requirements determine the efficiency of a tapering structure: (i) a few-
nanometer x-ray focal spot, which is defined by the radial FWHM of
$R_{c}(r,z_{\mathrm{max}})$ at the focus and indicated by the leftward arrow
in the left insets of Figs. 2(b, c), (ii) a long intrawaveguide Rayleigh
length, which is given by the axial FWHM of $R_{c}(0,z)$ at the focus and
indicated by the horizontal arrows in the right insets of Figs. 2(b, c), and
(iii) low photon loss. The first two requirements maximize the nuclear
excitation volume for a given input x-ray photon number, and the last
criterion maximizes the use of photons in focusing. As demonstrated in Fig. 2,
the WG enhances the free-space Rabi frequency with the maximum factor of
$\sqrt{R_{c}\left(0,z_{f}\right)}$ and therefore reduces the required input
XFEL photon number $n_{p}$ for achieving nuclear inversion with a factor of 28
for the EWG and of 14 for the CWG. The focusing spot radius and the Rayleigh
length are respectively about 5 nm and 2 $\mu$m. This results in an active
volume containing $10^{6}$ nuclei with solid-state particle number density.
Moreover, the EWG can achieve $T=0.8$ at the focus and so fulfil the condition
(iii).

Figure 3:  XFEL-pumped transient nuclear inversion in an elliptical waveguide.
(a) Axial x-ray concentration $R_{c}\left(0,z,t\right)$ as a function of time
and axial coordinate $z$ for 14.4 keV x-ray propagation in a Pt-cladding EWG.
(b) Time- and axial space-dynamics of nuclear population inversion $I_{v}$
around the EWG focus, and (c) spatial distribution of $I_{v}\left(r,z\right)$
for 57Fe driven by a propagating XFEL with ($n_{p}$, $\sigma$ , $\tau$) =
(2.88$\times 10^{11}$, 120 nm, 42.5 fs). The nuclear population inversion
maximum max$(I_{v})$ for nuclear transitions in 133Ba (unfilled red square),
57Fe (filled blue triangle), 187Os (filled yellow circle), and 169Tm (unfilled
greed diamond) is depicted as a function of the number of photons in the pulse
in a (d) platinum- (e) silicon-cladding EWG. We use the parameter set
($r_{i}$, $L_{f}$, $r_{o}$) = (150 nm, 160 $\mu$m, 15 nm) for all graphs.

The full XFEL-nuclear dynamics is presented in Fig. 3. Figure 3(a)
demonstrates the propagation of an XFEL pulse through the EWG illustrated in
Fig. 2(c) where a 57Fe nuclear sample is embedded in $160\mu$m$\leq z\leq
165\mu$m. The axial $R_{c}\left(0,z\right)$ shows that the XFEL intensity
reaches the maximum when arriving at $z\approx 161\ \mu$m as also predicted by
the steady-state solution in Fig. 2(c). With XFEL parameters ($n_{p}$,
$\sigma$ , $\tau$) = (2.88$\times 10^{11}$, 120 nm, 42.5 fs), the
corresponding 57Fe nuclear dynamics is investigated in Fig. 3(b) which
illustrates the axial nuclear inversion $I_{v}\left(0,z,t\right)$. Figure 3(b)
shows that 57Fe is coherently excited by the coming x ray when the XFEL
propagates through the WG neck. On the time scale of 100 fs XFEL FWHM, nuclei
located near the focal point are fully inverted. As showed in Fig. 3(c) which
considers now the nuclear inversion at the end of the pulse, the XFEL leaves
behind an active volume of $\pi\times$3 nm$\times$3 nm$\times 1.7\ \mu$m,
containing $2.5\times 10^{6}$ inverted 57Fe nuclei out of $4.1\times 10^{6}$
totally embedded isotopes In Figs. 3(d-e) we show the $n_{p}$-dependent
maximum of the nuclear population inversion max$(I_{v})$ for nuclear
transitions in the isotopes 133Ba, 57Fe, 187Os and 169Tm using Pt- and Si-
cladding EWG, respectively. Each set of numerical results is fitted by Eq.
(10) with a correspondingly colored solid line.

Three conclusions can be drawn based on the results presented in Figs. 3(d-e).
First, all four considered isotopes can be fully inverted around the focus in
a Pt-cladding EWG in the input-photon-number range of $10^{11}<n_{p}<10^{12}$.
Significant inversion occurs also for photon numbers in the range
$10^{10}<n_{p}<10^{11}$ in a Pt-cladding EWG for 133Ba, 187Os, and 169Tm.
Population inversion in tapered waveguides appears to occur more efficiently
and to require smaller photon numbers $n_{p}$ than in x-ray thin-film cavities
according to theoretical predictions in Ref. Heeg et al. (2016). The required
x-ray photon number lies within the reach of modern XFEL facilities LCL (a);
SAC ; Eur ; Swi ; Pal  and also of seeded XFEL facilities LCL (b); Nam et al.
(2021). Second, the coherent x-ray pulse can drive a Rabi oscillation
Bürvenich et al. (2006); Pálffy et al. (2008). Finally, a comparison between
Figs. 3(d) and 3(e) demonstrates that the Pt-cladding EWG provides a better
X-ray focusing than the silicon-cladding EWG. This behaviour is due to the
difference in index of refraction, which affects the critical reflection angle
and further leads to the enhanced x-ray focusing ability.

The effect of waveguide roughness. For a more realistic modelling of the WG,
we investigate now the effect of surface imperfections at the vacuum-cladding
interface. To this end we consider the cladding surface as ellipical with an
additional white noise of amplitude $A_{p}$. Depending on the fabrication
procedure, this amplitude can be on the order of several nanometers Osterhoff
and Salditt (2009). Figure 4(a) illustrates an example of steady-state
$R_{c}\left(r,z\right)$ using $A_{p}=$ 5 nm, where the maximum $R_{c}$
degrades from 800 [which was the case in Fig. 2(c)] to 320. The direct
consequence of the roughness is the reduction of nuclear excitation. Compared
with Fig. 3(c), our full calculation in Fig. 4(b) shows that the maximum
$I_{v}$ becomes 0.4, and the inversion volume where $I_{v}>0$ shrinks by a
factor of 0.6 resulting in $1.35\times 10^{6}$ inverted nuclei out of
$2.5\times 10^{6}$ totally embedded 57Fe isotopes. Note here that the active
volume and therefore number of addressed nuclei changes when considering the
WG roughness. In order to investigate the noise-amplitude-dependent total
number of inverted nuclei, we numerically simulate 100 realizations, with
random sets of noise for each amplitude value $A_{p}$ in Fig. 4(c) considering
the case of $L_{f}=160\ \mu$m. The results are illustrated for the four
isotopes 187Os, 169Tm, 57Fe, and 133Ba. The shaded regions illustrate the
error bars averaged over 100 realizations at each $A_{p}$. The quite small
error bars reveal that the amount of inverted nuclei is very stable from noise
to noise for a given $A_{p}$. The number of inverted nuclei sustains the
increasing $A_{p}$ from 0 to 10 nm, and remains on the order of $10^{6}$.

Figure 4:  The effect of waveguide roughness. (a) 14.4 keV EWG x-ray
concentration distribution $R_{c}$ with a rough cladding interface of
($r_{i}$, $L_{f}$, $r_{o}$, $A_{p}$) = (150 nm, 160 $\mu$m, 15 nm, 5 nm). The
white line indicates the Pt-cladding-vacuum interface. (b) the corresponding
XFEL-pumped 57Fe $I_{v}\left(r,z\right)$ in the EWG taking into account the
surface interface roughness using the XFEL pulse parameters ($n_{p}$, $\sigma$
, $\tau$) = (2.88$\times 10^{11}$, 120 nm, 42.5 fs). (c) $A_{p}$-dependent
number of inverted 187Os (red solid line), 169Tm (green dashed line), 57Fe
(blue dotted line), and 133Ba (orange dashed-dotted line) nuclei. We consider
a WG with $L_{f}=160\ \mu$m. The shaded regions depict the error bars averaged
over 100 realizations.

The effect of XFEL coherence time. So far we have considered a completely
coherent XFEL pulse. However, at present most XFEL facilities generate
partially coherent x-rays by self-amplified spontaneous emission (SASE) with
good transverse coherence but rather short longitudinal coherence time
$\tau_{c}$ of few tens of femtosecond Emma et al. (2010). Thus, in reality the
time structure of a SASE XFEL pulse can be rather spiky. Using self-seeding
Feldhaus et al. (1997), the coherence time can be substantially improved, as
already demonstrated at LCLS and SACLA Amann et al. (2012); Inoue et al.
(2019). We illustrate the effect of limited coherence on the XFEL pulse
structure in Fig. 5(a) which shows the pulse intensity for three different
coherence times. The phase jumps randomly between neighbouring spikes and
degrades the ability of producing nuclear excitation. Moreover, the pulse
energy fluctuates from shot to shot following the Gamma distribution Pfeifer
et al. (2010). In order to model these effects in calculating the nuclear
population inversion, we use the partial-coherence method to model SASE XFEL
Pfeifer et al. (2010) combined with Eq. (10) (see Methods). The results are
illustrated in Fig. 5(b) for the four isotopes considered so far. The shaded
regions illustrate the error bars resulting from an averaging over 5000 SASE
realizations at each $\tau_{c}$ with the EWG considered in Fig.2(c). The error
bars are scaled by a factor of 0.2 for better visualization. The quite large
error bars reflect the SASE Gamma distribution. For all nuclear species, the
amount of inverted nuclei reaches the $10^{4}$ region when $\tau_{c}>10$ fs
and approaches the result of fully coherent XFEL as increasing $\tau_{c}$. The
statistics at $\tau_{c}=50$ fs shows that $10^{6}$ inverted 187Os, 169Tm, and
57Fe nuclei are already produced by an average of $\left\langle
n_{p}\right\rangle\approx 2\times 10^{11}$ photons per pulse. Thus, strong
nuclear inversion is potentially in reach also for just partially coherent
XFEL pulses in an EWG.

Figure 5:  The effect of XFEL coherence time. (a) XFEL intensity for $\tau_{c}=50$ fs (green dashed dotted line), $\tau_{c}=10$ fs (red solid line), and considering full coherence (blue dashed line). The XFEL FWHM is 100fs for all cases. (b) $\tau_{c}$-dependent number of inverted 187Os (red solid line), 169Tm (green dashed line), 57Fe (blue dotted line), and 133Ba (orange dash-dotted line) nuclei for an EWG with ($r_{i}$, $L_{f}$, $r_{o}$, $A_{p}$) = (150 nm, 160 $\mu$m, 15 nm, 0). The shaded regions depict the error bars resulting from an average over 5000 SASE realizations, for illustration purposes scaled by a factor of 0.2.  Table 1:  Nuclear and waveguide cladding parameters nnd ; lbl ; ipm . For each nucleus with atomic mass number $A$ we present the nuclear transition energy $E_{t}$, the number density in solid-state $N$, the calculated Lamb-Mössbauer factor $f_{lm}$, the reduced nuclear transition probability $B(M1)$ in Weisskopf units (W.u.), the internal conversion coefficient $\alpha$ and the spontaneous radiative rate $\Gamma_{s}$ in MHz. The corresponding X-ray index of refraction is $n=1-\delta+i\beta$ for the respective nuclear target and the cladding materials are given in the last six columns.  ${}^{A}X$ |  $E_{t}$ (keV) |  $N$ ($10^{28}$/m3) | $f_{lm}$ |  $B(M1)$ (W.u.) | $\alpha$ |  $\Gamma_{s}$ (MHz) | $\delta(10^{-6})$ | $\beta(10^{-7})$
---|---|---|---|---|---|---|---|---
|  |  |  |  |  |  | ${}^{A}X$ | Pt | Si | ${}^{A}X$ | Pt | Si
169Tm | 8.410 | 3.32 | 0.84 | 0.0342 | 263 | 169 | 19.0 | 47.5 | 6.94 | 13.4 | 43.7 | 1.46
187Os | 9.756 | 7.15 | 0.96 | 0.0260 | 280 | 291 | 36.0 | 34.7 | 5.14 | 24.6 | 25.7 | 0.81
133Ba | 12.327 | 1.54 | 0.38 | 0.0230 | 69.5 | 99.0 | 3.93 | 21.2 | 3.21 | 3.10 | 28 | 0.32
57Fe | 14.413 | 8.49 | 0.80 | 0.0078 | 8.65 | 7.05 | 7.44 | 16.1 | 2.34 | 3.39 | 24.9 | 0.17

##  II Discussion

Our results demonstrate that x-ray waveguides can increase the x-ray intensity
by up to three orders of magnitude and therefore substantially boost its
efficiency to induce strong nuclear excitation. Combined with temporally
coherent pulses, prospects of nuclear population inversion finally come in
reach at XFEL facilities. We emphasize that in this work we focus on the new
aspect of waveguide-induced enhancement rather than on collective nuclear
dynamics previously investigated in the context of nuclear forward scattering
Chumakov et al. (2018). Thus, we expect that our estimates of required photon
number per XFEL pulse for achieving nuclear inversion are rather conservative.
A more sophisticated calculation including on equal footing both wavevuide
field enhancement and nuclear collective effects deserves further
investigation.

Experimentally, a stronger intensity carries with it more destructive power in
form of heat load. This could be solved by decreasing the off-resonant
components of the incoming x rays. Possible implementations are for instance
narrow-band monochromatization using a 57FeBO3 or 57Fe2O3 crystal Van Burck et
al. (1980); Gerdau et al. (1985); Shvyd’ko et al. (1996). This kind of nuclear
monochromator has $<$10 neV bandwidth and stretches the pulse duration to 100
ns Van Burck et al. (1980); Gerdau et al. (1985); Shvyd’ko et al. (1996).
Furthermore, the nuclear excitation detection requires event-based electronics
which record the scattered photons ideally in both time and energy. Such a so-
called 2D-spectrum technique has been recently developed Goerttler et al.
(2019); Heeg et al. (2021); Kuan and Liao (2020). It is anticipated that with
higher count rates also the quantum properties of the scattered x-ray photons
- for instance bunching or anti-bunching and photon correlations will become
accessible to experiments.

The enhanced x-ray intensity in tapered waveguides will have a strong impact
for the future development of x-ray quantum optics using the resonant
interaction of x-rays with nuclear systems. Once demonstrated, the successful
nuclear population control could be used for depleting nuclear isomers and
thus releasing the energy stored therein. A classical example is the 2.4 MeV
93mMo isomer with 6.8 hours halflife, which could be depleted by driving a 4.8
keV transition, well within the reach of XFEL facilities, upwards to a state
which then directly decays within ns to the ground state Gunst et al. (2014);
Wu et al. (2018). Our calculations for this case show that, although nuclear
population inversion requires higher photon per pulse than presently available
at XFEL facilities, an EWG enhances the achievable excitation by a factor of
approx. 200.

##  III Methods

Optical-Bloch equations. The Liouville equation (1) describing the coupling
between x rays and nuclei can be written in terms of the density matrix
elements to yield the so-called optical Bloch equations (Scully and Zubairy,
2006; Agarwal, 2012)

$\displaystyle\partial_{t}{\rho_{dd}}$ $\displaystyle=$
$\displaystyle\left(\frac{\alpha}{1+\alpha}\right)\Gamma_{s}\rho_{uu}+\frac{i}{2}\left(\Omega^{\ast}\rho_{ud}-\Omega\rho_{ud}^{\ast}\right),$
(5) $\displaystyle\partial_{t}{\rho_{ud}}$ $\displaystyle=$
$\displaystyle-\left(\frac{\alpha}{1+\alpha}\right)\frac{\Gamma_{s}}{2}\rho_{ud}-\frac{i}{2}\Omega\left(\rho_{uu}-\rho_{dd}\right),$
(6) $\displaystyle\partial_{t}{\rho_{uu}}$ $\displaystyle=$
$\displaystyle-\left(\frac{\alpha}{1+\alpha}\right)\Gamma_{s}\rho_{uu}-\frac{i}{2}\left(\Omega^{\ast}\rho_{ud}-\Omega\rho_{ud}^{\ast}\right),$
(7)

where the subscripts $\left(u,d\right)=\left(3,1\right)$ or $\left(4,2\right)$
label the nuclear states as illustrated in Fig. 1(c). In Eq. (5-7), the
incoherent decay of the nuclear excitation is attributed to the internal
conversion rate $\frac{\alpha}{1+\alpha}\Gamma_{s}$, and the WG mode-nuclei
coupling is described by the coherent process. We solve the optical Bloch
equations together with the wave equation for the propagation of the x-ray
electric field (2) using the initial conditions
$\rho_{33}\left(r,z,0\right)=\rho_{44}\left(r,z,0\right)=\rho_{31}\left(r,z,0\right)=\rho_{42}\left(r,z,0\right)=E\left(r,z,0\right)=0$,
$\rho_{11}\left(r,z,0\right)=\rho_{22}\left(r,z,0\right)=1/2$. The boundary
condition Pálffy et al. (2008); Liao et al. (2011, 2013) at $z=0$ is

$E\left(r,0,t\right)=\sqrt{\frac{n_{p}E_{t}\sqrt{2}}{c\epsilon_{0}\sigma^{2}\tau\sqrt{\pi^{3}}}}\exp\left(-\frac{r^{2}}{\sigma^{2}}-\frac{\left(t-t_{c}\right)^{2}}{4\tau^{2}}\right).$
(8)

Here , $\epsilon_{0}$ is the vacuum permittivity, $t_{c}=250$fs is the XFEL
peak time, and $E_{t}$ the nuclear transition energy, respectively.
Furthermore, $n_{p}$, $\sigma$, and $\tau$ are the photon number per pulse,
beam waist, and pulse duration of the input XFEL pulse, respectively. We note
that, together with the dispersion relation in the wave Eq. (2), the Gaussian
pulse modeling in Eq. (8) is more realistic than the originally used effective
intensity approach Bürvenich et al. (2006) and renders larger excited state
populations $\rho_{uu}$. Numerically, in Eq. (2) the contribution of the
nuclear coherences term on the right-hand side is dominated by the waveguide
terms.

Following Ref. Scully and Zubairy (2006); Pálffy et al. (2008); Liao et al.
(2011, 2012a), we define the Rabi frequency $\Omega=\Pi E/\hbar$, where

$\Pi=\frac{4a\mu}{3}\sqrt{1.79\left(2I_{u}+1\right)\pi B\left(M1\right)}\ ,$
(9)

$\mu=0.105e\cdot$fm is the nuclear magneton, $a=\sqrt{2/3}$ is the
corresponding Clebsch-Gordan coefficient, and $B\left(M1\right)$ is the
reduced transition probability for the magnetic dipole nuclear transition. All
nuclear and WG cladding parameters used in our calculations are listed in
Table 1. For simplicity, we define the nuclear inversion
$I_{v}=\rho_{33}+\rho_{44}-\rho_{11}-\rho_{22}$ Bürvenich et al. (2006). When
obtaining the x-ray concentration distribution $R_{c}$ via a steady-state
numerical calculation, i.e., solving Eq. (2) without nuclear coherence and
temporal derivative terms Pfeiffer et al. (2002); Fuhse and Salditt (2006),
one can calculate the XFEL pulse area in a typical two-level model and obtain
the nuclear inversion

$I_{v}\left(r,z\right)=-\cos\left[2\frac{\Pi}{\sigma\hbar}\sqrt{\frac{\tau
n_{p}TR_{c}\left(r,z\right)E_{t}\sqrt{2}}{c\epsilon_{0}\sqrt{\pi}}}\right].$
(10)

Coherence time for SASE XFEL. We use the partial-coherence method to
numerically simulate the SASE XFEL pulse structure Pfeifer et al. (2010). The
pulse energy $P=2\pi
c\epsilon_{0}\int_{-\infty}^{\infty}\int_{0}^{\infty}|E\left(r,0,t\right)|^{2}rdrdt$
follows a Gamma distribution:

$\Gamma\left(P\right)=\frac{M^{M}}{\Gamma\left(M\right)}\left(\frac{P}{\left\langle
P\right\rangle}\right)^{M-1}\frac{1}{\left\langle
P\right\rangle}\exp\left(-M\frac{P}{\left\langle P\right\rangle}\right),$ (11)

with the mode number

$M=\frac{\left\langle P\right\rangle^{2}}{\left\langle\left(P-\left\langle
P\right\rangle\right)^{2}\right\rangle}.$ (12)

In order to investigate the effect of SASE XFEL as demonstrated in Fig. 5, we
obtain the lowest average pulse energy $\left\langle P\right\rangle$ for
achieving the full nuclear inversion $I_{v}=1$ in Eq. (10) with the maximum
EWG x-ray concentration $R_{c}$:

$\left\langle P\right\rangle=\left\langle n_{p}\right\rangle
E_{t}=\frac{c\epsilon_{0}\pi^{3/2}\sigma^{2}\hbar^{2}}{4\sqrt{2}\tau\Pi^{2}\mathrm{max}\left(R_{c}\right)}.$
(13)

For each isotope listed in Table 1, $\left\langle
P\right\rangle/E_{t}=\left\langle n_{p}\right\rangle\approx 2\times 10^{11}$
photons per pulse. The number of inverted nuclei can be calculated as

$\int_{I_{v}>0}N\left(\vec{r}\right)\left[\rho_{33}\left(\vec{r}\right)+\rho_{44}\left(\vec{r}\right)\right]d^{3}\vec{r}$
(14)

for Figs. 4(c) and 5(b).

##  IV Acknowledgements

Y.-H. C., P.-H. L, G.-Y. W., and W.-T. L. are supported by the Ministry of
Science and Technology, Taiwan (Grant No. MOST 107-2112-M-008-007-MY3 & MOST
109-2639-M-007-002-ASP). AP gratefully acknowledges support from the
Heisenberg Program of the Deutsche Forschungsgemeinschaft (DFG).

##  V Author contributions

Y.-H. C. derived the physics model and developed the computational code. Y.-H.
C., P.-H. L, and G.-Y. W. performed the numerical calculation. A. P. derived
the nuclear condensed matter input of the calculation. W.-T. L. conceived the
idea and conducted the project. All the authors discussed the results and
wrote the manuscript.

## References

  * Emma et al. (2010) P. Emma, R. Akre, J. Arthur, R. Bionta, C. Bostedt, J. Bozek, A. Brachmann, P. Bucksbaum, R. Coffee, F.-J. Decker, et al., Nature Photon. 4, 641 (2010).
  * Pellegrini et al. (2016) C. Pellegrini, A. Marinelli, and S. Reiche, Rev. Mod. Phys. 88, 015006 (2016), URL https://link.aps.org/doi/10.1103/RevModPhys.88.015006.
  * Kuznetsova and Kocharovskaya (2017) E. Kuznetsova and O. Kocharovskaya, Nature Photon. 11, 685 (2017).
  * Raimond et al. (2001) J. M. Raimond, M. Brune, and S. Haroche, Rev. Mod. Phys. 73, 565 (2001).
  * Adams et al. (2013) B. W. Adams, C. Buth, S. M. Cavaletto, J. Evers, Z. Harman, C. H. Keitel, A. Pálffy, A. Picón, R. Röhlsberger, Y. Rostovtsev, et al., Journal of modern optics 60, 2 (2013).
  * Vagizov et al. (2014) F. Vagizov, V. Antonov, Y. Radeonychev, R. N. Shakhmuratov, and O. Kocharovskaya, Nature 508, 80 (2014).
  * Röhlsberger et al. (2010) R. Röhlsberger, K. Schlage, B. Sahoo, S. Couet, and R. Rüffer, Science 328, 1248 (2010).
  * Röhlsberger et al. (2012) R. Röhlsberger, H. C. Wille, K. Schlage, and B. Sahoo, Nature 482, 199 (2012).
  * Heeg et al. (2013) K. P. Heeg, H.-C. Wille, K. Schlage, T. Guryeva, D. Schumacher, I. Uschmann, K. S. Schulze, B. Marx, T. Kämpfer, G. G. Paulus, et al., Phys. Rev. Lett. 111, 073601 (2013).
  * Heeg et al. (2015a) K. P. Heeg, J. Haber, D. Schumacher, L. Bocklage, H.-C. Wille, K. S. Schulze, R. Loetzsch, I. Uschmann, G. G. Paulus, R. Rüffer, et al., Phys. Rev. Lett. 114, 203601 (2015a).
  * Heeg et al. (2015b) K. P. Heeg, C. Ott, D. Schumacher, H.-C. Wille, R. Röhlsberger, T. Pfeifer, and J. Evers, Phys. Rev. Lett. 114, 207401 (2015b).
  * Heeg et al. (2017) K. P. Heeg, A. Kaldun, C. Strohm, P. Reiser, C. Ott, R. Subramanian, D. Lentrodt, J. Haber, H.-C. Wille, S. Goerttler, et al., Science 357, 375 (2017).
  * Haber et al. (2017) J. Haber, X. Kong, C. Strohm, S. Willing, J. Gollwitzer, L. Bocklage, R. Rüffer, A. Pálffy, and R. Röhlsberger, Nature Photonics 11, 720 (2017).
  * Chumakov et al. (2018) A. I. Chumakov, A. Q. Baron, I. Sergueev, C. Strohm, O. Leupold, Y. Shvyd’ko, G. V. Smirnov, R. Rüffer, Y. Inubushi, M. Yabashi, et al., Nature Physics 14, 261 (2018).
  * Heeg et al. (2021) K. Heeg, A. Kaldun, C. Strohm, C. Ott, R. Subramanian, D. Lentrodt, J. Haber, H.-C. Wille, S. Goerttler, R. Rüffer, et al., Nature 590, 401 (2021).
  * Walker and Dracoulis (1999) P. Walker and G. Dracoulis, Nature 399, 35 (1999).
  * LCL (a) _Lcls xfel beamline specifications_ , URL https://lcls.slac.stanford.edu/instruments.
  * (18) _Sacla xfel beamline specifications_ , URL http://xfel.riken.jp/eng/users/bml04-11.html.
  * (19) _European xfel beamline specifications_ , URL https://www.xfel.eu/facility/instruments/index_eng.html.
  * (20) _Swissfel beamline specifications_ , URL https://www.psi.ch/en/swissfel/beamlines-and-instruments.
  * (21) _Pal-xfel beamline specifications_ , URL http://pal.postech.ac.kr/paleng/Menu.pal?method=menuView&pageMode=paleng&top=7&sub=2&sub2=1&sub3=0.
  * Haber et al. (2016) J. Haber, K. S. Schulze, K. Schlage, R. Loetzsch, L. Bocklage, T. Gurieva, H. Bernhardt, H.-C. Wille, R. Rüffer, I. Uschmann, et al., Nature Photonics 10, 445 (2016).
  * Shvyd’ko et al. (2010) Y. Shvyd’ko, S. Stoupin, A. Cunsolo, A. H. Said, and X. Huang, Nature Phys. 6, 196 (2010).
  * Shvyd’ko et al. (2011) Y. Shvyd’ko, S. Stoupin, V. Blank, and S. Terentyev, Nature Photon. 5, 539 (2011).
  * Heeg et al. (2016) K. P. Heeg, C. H. Keitel, and J. Evers, arXiv:1607.04116 [quant-ph] (2016).
  * Pfeiffer et al. (2002) F. Pfeiffer, C. David, M. Burghammer, C. Riekel, and T. Salditt, Science 297, 230 (2002).
  * Jarre et al. (2005) A. Jarre, C. Fuhse, C. Ollinger, J. Seeger, R. Tucoulou, and T. Salditt, Phys. Rev. Lett. 94, 074801 (2005), URL https://link.aps.org/doi/10.1103/PhysRevLett.94.074801.
  * Krüger et al. (2012) S. Krüger, H. Neubauer, M. Bartels, S. Kalbfleisch, K. Giewekemeyer, P. Wilbrandt, M. Sprung, and T. Salditt, Journal of synchrotron radiation 19, 227 (2012).
  * Chen et al. (2015) H.-Y. Chen, S. Hoffmann, and T. Salditt, Applied Physics Letters 106, 194105 (2015).
  * Hoffmann-Urlaub et al. (2016) S. Hoffmann-Urlaub, P. Höhne, M. Kanbach, and T. Salditt, Microelectronic Engineering 164, 135 (2016).
  * Salditt et al. (2015) T. Salditt, S. Hoffmann, M. Vassholz, J. Haber, M. Osterhoff, and J. Hilhorst, Phys. Rev. Lett. 115, 203902 (2015), URL https://link.aps.org/doi/10.1103/PhysRevLett.115.203902.
  * Stern et al. (1988) E. A. Stern, Z. Kalman, A. Lewis, and K. Lieberman, Applied optics 27, 5135 (1988).
  * Denisov et al. (1991) E. Denisov, V. Glebov, and N. Zhevago, Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment 308, 400 (1991).
  * Bergemann et al. (2003) C. Bergemann, H. Keymeulen, and J. F. van der Veen, Phys. Rev. Lett. 91, 204801 (2003), URL https://link.aps.org/doi/10.1103/PhysRevLett.91.204801.
  * Feldhaus et al. (1997) J. Feldhaus, E. Saldin, J. Schneider, E. Schneidmiller, and M. Yurkov, Optics Communications 140, 341 (1997).
  * Amann et al. (2012) J. Amann, W. Berg, V. Blank, F.-J. Decker, Y. Ding, P. Emma, Y. Feng, J. Frisch, D. Fritz, J. Hastings, et al., Nature photonics 6, 693 (2012).
  * Serkez (2016) S. Serkez, Synchrotron Radiation News 29, 10 (2016).
  * Emma et al. (2017) C. Emma, A. Lutman, M. W. Guetg, J. Krzywinski, A. Marinelli, J. Wu, and C. Pellegrini, Applied Physics Letters 110, 154101 (2017), eprint https://doi.org/10.1063/1.4980092, URL https://doi.org/10.1063/1.4980092.
  * Inoue et al. (2019) I. Inoue, T. Osaka, T. Hara, T. Tanaka, T. Inagaki, T. Fukui, S. Goto, Y. Inubushi, H. Kimura, R. Kinjo, et al., Nature Photonics 13, 319 (2019).
  * Nam et al. (2021) I. Nam et al., Nature Photonics https://doi.org/10.1038/s41566-021-00777-z (2021).
  * Kim et al. (2008) K.-J. Kim, Y. Shvyd’ko, and S. Reiche, Phys. Rev. Lett. 100, 244802 (2008), URL https://link.aps.org/doi/10.1103/PhysRevLett.100.244802.
  * Zemella et al. (2012) J. Zemella et al., in _Proceedings of the 32th International Free-Electron Laser Conference_ (2012), pp. 429–432.
  * Adams et al. (2019) B. Adams, G. Aeppli, T. Allison, A. Q. Baron, P. Bucksbaum, A. I. Chumakov, C. Corder, S. P. Cramer, S. DeBeer, Y. Ding, et al., arXiv:1903.09317 (2019).
  * Shvyd’ko (2019) Y. Shvyd’ko, Phys. Rev. Accel. Beams 22, 100703 (2019), URL https://link.aps.org/doi/10.1103/PhysRevAccelBeams.22.100703.
  * Rauer et al. (2019) P. Rauer, I. Bahns, W. Hillert, J. Roßbach, W. Decking, and H. Sinn, in _Proceedings of the 39th International Free-Electron Laser Conference_ (2019), pp. 62–65.
  * Marcus et al. (2020) G. Marcus, A. Halavanau, Z. Huang, J. Krzywinski, J. MacArthur, R. Margraf, T. Raubenheimer, and D. Zhu, Phys. Rev. Lett. 125, 254801 (2020), URL https://link.aps.org/doi/10.1103/PhysRevLett.125.254801.
  * Li et al. (2018) K. Li, J. Yan, C. Feng, M. Zhang, and H. Deng, Phys. Rev. Accel. Beams 21, 040702 (2018), URL https://link.aps.org/doi/10.1103/PhysRevAccelBeams.21.040702.
  * Paraskaki et al. (2019) G. Paraskaki, S. Ackermann, B. Faatz, G. Geloni, V. Grattoni, W. Hillert, C. Lechner, M. Mehrjoo, S. Serkez, and T. Tanikawa, in _39th Free Electron Laser Conf.(FEL’19), Hamburg, Germany, 26-30 August 2019_ (JACOW Publishing, Geneva, Switzerland, 2019), pp. 234–237.
  * Scully and Zubairy (2006) M. O. Scully and M. S. Zubairy, _Quantum Optics_ (Cambridge University Press, 2006).
  * Agarwal (2012) G. S. Agarwal, _Quantum Optics_ (Cambridge University Press, 2012).
  * Bürvenich et al. (2006) T. J. Bürvenich, J. Evers, and C. H. Keitel, Phys. Rev. Lett. 96, 142501 (2006).
  * Liao et al. (2011) W.-T. Liao, A. Pálffy, and C. H. Keitel, Physics Letters B 705, 134 (2011).
  * Liao et al. (2012a) W.-T. Liao, A. Pálffy, and C. H. Keitel, Phys. Rev. Lett. 109, 197403 (2012a).
  * Liao et al. (2012b) W.-T. Liao, S. Das, C. H. Keitel, and A. Pálffy, Phys. Rev. Lett. 109, 262502 (2012b).
  * Liao et al. (2013) W.-T. Liao, A. Pálffy, and C. H. Keitel, Phys. Rev. C 87, 054609 (2013).
  * Kong et al. (2014) X. Kong, W.-T. Liao, and A. Pálffy, New Journal of Physics 16, 013049 (2014).
  * Kong and Pálffy (2016) X. Kong and A. Pálffy, Phys. Rev. Lett. 116, 197402 (2016).
  * Wang and Liao (2018) G.-Y. Wang and W.-T. Liao, Phys. Rev. Applied 10, 014003 (2018), URL https://link.aps.org/doi/10.1103/PhysRevApplied.10.014003.
  * Shvyd’ko et al. (1998) Y. V. Shvyd’ko, U. van Bürck, W. Potzel, P. Schindelmann, E. Gerdau, O. Leupold, J. Metge, H. D. Rüter, and G. V. Smirnov, Phys. Rev. B 57, 3552 (1998), URL https://link.aps.org/doi/10.1103/PhysRevB.57.3552.
  * Fuhse and Salditt (2006) C. Fuhse and T. Salditt, Applied optics 45, 4603 (2006).
  * Bukreeva et al. (2006) I. Bukreeva, A. Popov, D. Pelliccia, A. Cedola, S. B. Dabagov, and S. Lagomarsino, Phys. Rev. Lett. 97, 184801 (2006), URL https://link.aps.org/doi/10.1103/PhysRevLett.97.184801.
  * Geloni et al. (2010) G. Geloni, E. Saldin, L. Samoylova, E. Schneidmiller, H. Sinn, T. Tschentscher, and M. Yurkov, New Journal of Physics 12, 035021 (2010).
  * LCL (b) _Lcls-ii xfel beamline specifications_ , URL https://lcls.slac.stanford.edu/lcls-ii/design-and-performance.
  * Pálffy et al. (2008) A. Pálffy, J. Evers, and C. H. Keitel, Phys. Rev. C 77, 044602 (2008), URL https://link.aps.org/doi/10.1103/PhysRevC.77.044602.
  * Osterhoff and Salditt (2009) M. Osterhoff and T. Salditt, Optics communications 282, 3250 (2009).
  * Pfeifer et al. (2010) T. Pfeifer, Y. Jiang, S. Düsterer, R. Moshammer, and J. Ullrich, Optics letters 35, 3441 (2010).
  * (67) URL https://www.nndc.bnl.gov/ensdf/.
  * (68) URL https://henke.lbl.gov/optical_constants/asf.html.
  * (69) URL http://purple.ipmt-hpm.ac.ru/xcalc/xcalc_mysql/ref_index.php.
  * Van Burck et al. (1980) U. Van Burck, R. Mossbauer, G. Smirnov, and H. Maurus, Journal of Physics C: Solid State Physics 13, 4511 (1980).
  * Gerdau et al. (1985) E. Gerdau, R. Rüffer, H. Winkler, W. Tolksdorf, C. P. Klages, and J. P. Hannon, Phys. Rev. Lett. 54, 835 (1985), URL https://link.aps.org/doi/10.1103/PhysRevLett.54.835.
  * Shvyd’ko et al. (1996) Y. V. Shvyd’ko, T. Hertrich, U. van Bürck, E. Gerdau, O. Leupold, J. Metge, H. D. Rüter, S. Schwendy, G. V. Smirnov, W. Potzel, et al., Phys. Rev. Lett. 77, 3232 (1996).
  * Goerttler et al. (2019) S. Goerttler, K. Heeg, A. Kaldun, P. Reiser, C. Strohm, J. Haber, C. Ott, R. Subramanian, R. Röhlsberger, J. Evers, et al., Phys. Rev. Lett. 123, 153902 (2019), URL https://link.aps.org/doi/10.1103/PhysRevLett.123.153902.
  * Kuan and Liao (2020) Y.-H. Kuan and W.-T. Liao, Phys. Rev. A 101, 023836 (2020), URL https://link.aps.org/doi/10.1103/PhysRevA.101.023836.
  * Gunst et al. (2014) J. Gunst, Y. A. Litvinov, C. H. Keitel, and A. Pálffy, Phys. Rev. Lett. 112, 082501 (2014), URL https://link.aps.org/doi/10.1103/PhysRevLett.112.082501.
  * Wu et al. (2018) Y. Wu, J. Gunst, C. H. Keitel, and A. Pálffy, Phys. Rev. Lett. 120, 052504 (2018), URL https://link.aps.org/doi/10.1103/PhysRevLett.120.052504.


# SamDSK: Combining Segment Anything Model with Domain-Specific Knowledge for
Semi-Supervised Learning in Medical Image Segmentation

Yizhe Zhang, Tao Zhou, Shuo Wang, Ye Wu, Pengfei Gu, Danny Z. Chen Y. Zhang,
T. Zhou, and Y. Wu are with the School of Computer Science and Engineering,
Nanjing University of Science and Technology, Nanjing, China. S. Wang is with
the Digital Medical Research Center, School of Basic Medical Sciences, Fudan
University, and Shanghai Key Laboratory of MICCAI, Shanghai, China. P. Gu and
D. Chen are with the Department of Computer Science and Engineering,
University of Notre Dame, Notre Dame, IN 46556, USA.
E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The Segment Anything Model (SAM) exhibits a capability to segment a wide array
of objects in natural images, serving as a versatile perceptual tool for
various downstream image segmentation tasks. In contrast, medical image
segmentation tasks often rely on domain-specific knowledge (DSK). In this
paper, we propose a novel method that combines the segmentation foundation
model (i.e., SAM) with domain-specific knowledge for reliable utilization of
unlabeled images in building a medical image segmentation model. Our new
method is iterative and consists of two main stages: (1) segmentation model
training; (2) expanding the labeled set by using the trained segmentation
model, an unlabeled set, SAM, and domain-specific knowledge. These two stages
are repeated until no more samples are added to the labeled set. A novel
optimal-matching-based method is developed for combining the SAM-generated
segmentation proposals and pixel-level and image-level DSK for constructing
annotations of unlabeled images in the iterative stage (2). In experiments, we
demonstrate the effectiveness of our proposed method for breast cancer
segmentation in ultrasound images, polyp segmentation in endoscopic images,
and skin lesion segmentation in dermoscopic images. Our work initiates a new
direction of semi-supervised learning for medical image segmentation: the
segmentation foundation model can be harnessed as a valuable tool for label-
efficient segmentation learning in medical image segmentation.

##  1 Introduction

Segmentation foundational models, such as the Segment Anything Model (SAM)
[13], have opened up new opportunities in medical image segmentation studies.
Many recent endeavors have focused on fine-tuning and adapting SAM for medical
image segmentation tasks. In cases where SAM generates high-quality
segmentation proposals for medical image segmentation datasets, domain-
specific knowledge can be integrated with SAM to annotate unlabeled images.
This approach can be beneficial and enhance medical image segmentation,
especially in scenarios where plentiful unlabeled images are accessible while
manually annotated images are scarce.

Figure 1: A high-level comparison between a conventional approach (left) and
our proposed SamDSK (right) for semi-supervised learning (SSL) in medical
image segmentation.

Leveraging unlabeled samples in training medical image segmentation models is
a semi-supervised learning (SSL) problem, and has been extensively studied in
the last decade. Zhang et al. [31] proposed to use adversarial training when
using unlabeled samples. Adversarial training encourages a segmentation model
to produce outputs for unlabeled samples that are indistinguishable from
outputs for labeled samples in segmentation quality. Peiris et al. [17] added
multi-view segmentation components to the adversarial training framework.
Recently, Wang et al. [25] proposed to incorporate anatomical constraints such
as connectivity, convexity, and symmetry in adversarial training for learning
from unlabeled images. Luo et al. [15] combined a consistency criterion with
adversarial training for semi-supervised learning of medical image
segmentation models. Designing losses and network structures that enhance
segmentation consistencies is another widely used approach for semi-supervised
learning in medical image segmentation [27]. Another track of research for
semi-supervised learning is based on pseudo-labels. Introduced in [14],
pseudo-label techniques have been widely utilized for semi-supervised learning
in medical image segmentation (e.g., [19]).

Adversarial training for semi-supervised learning is typically built on the
assumption that the unlabeled samples are drawn from the same distribution of
the labeled samples. The effectiveness of the min-max optimization in
improving the segmentation quality depends on factors such as the architecture
of the discriminator, the initial model parameters, the distribution changes
(if any) between the unlabeled and labeled samples. Contrastive learning has
shown great performance in representation learning, but, its effectiveness
depends on the design of pre-tasks and the training process. Similarly,
consistency loss requires identifying correct notation of consistency between
unlabeled images and multiple views of the images. Incorrect or inappropriate
design of the constrastive loss and consistency loss could make the learning
ineffective and sometimes even harmful to the segmentation performance.
Pseudo-label (PL) based techniques are often a go-to approach for utilizing
unlabeled images in training a prediction model (e.g., classification model,
segmentation model). Despite its effectiveness, if a large portion of the
pseudo-labels is incorrect, then using PLs to train the model would suffer the
risk of lower performance.

Previous work of semi-supervised learning commonly concentrated on network
training and architecture design, but fewer attempts were made on constructing
and expanding the (labeled) training set. In this paper, we cast the task of
utilizing unlabeled images in training a medical image segmentation model as
an iterative and gradual process of annotation acquisition of unlabeled images
together with model training (and re-training). Both the training set and
segmentation model evolve during the learning process. We develop a novel
method that combines the emerging SAM with Domain-Specific Knowledge (DSK) for
annotation acquisition and labeled set expansion. More specifically, suppose a
labeled image set is initially given to our method. We train an initial
medical image segmentation model using the labeled images. We then apply the
trained model to the unlabeled images and obtain segmentation probability maps
for all the unlabeled images. We refer to these maps as pixel-level domain-
specific knowledge (pixel-level DSK). We further introduce a high-level DSK
called image-level DSK, which includes prior knowledge of the segmentation
task, such as the potential number of RoIs (Regions of Interests) for each
segmentation class in an image. For an unlabeled image, an optimization
process is performed to construct its segmentation annotation by matching the
segmentation proposals (generated by SAM) with the pixel-level DSK,
constrained by the image-level DSK. Unlabeled images with segmentation
annotations attained by the above matching process are then examined according
to the optimal matching scores, and those images (with the constructed
annotations) with high matching scores will be added to the labeled set as
machine-labeled samples for the next round of segmentation model training. The
segmentation model receives a further improvement in each round of training
since more labeled data become available, and through multiple rounds of such
annotation acquisition and model training/retraining, we expect more unlabeled
samples with high-quality segmentation to be added to the labeled set, and
thus a better medical image segmentation model will be obtained. A high-level
overview of our proposed method is illustrated in Fig. 1.

In summary, the main contributions of this work are four-fold:

  * $\bullet$

We propose a schematic shift of utilizing unlabeled images in training a
medical image segmentation model. Our proposition is that once unlabeled data
can be “transformed” into labeled data, the training of the segmentation model
becomes similar to the model that is trained under the fully supervised
learning setup.

  * $\bullet$

We develop a novel optimization-based method that combines the segmentation
foundation model (SAM) with domain-specific knowledge (DSK) for semi-
supervised learning in medical image segmentation. Our approach integrates the
strengths of SAM and pseudo-label based learning, enhancing the
controllability and reliability of the process of utilizing unlabeled images
to train a segmentation model.

  * $\bullet$

Given that many medical image segmentation tasks involve and benefit from
image-level domain-specific knowledge, we extend our method by developing an
image-level DSK within the optimization process. More specifically, we
directly apply and utilize the count of regions of interest in our proposed
method, aiming to regularize the process of constructing annotations for
unlabeled images.

  * $\bullet$

Analyses and experiments confirm the effectiveness of our proposed method. We
also illustrate and discuss current limitations and future research directions
for utilizing SAM to leverage unlabeled images in medical image segmentation.

Figure 2: The main steps of our proposed SamDSK method. Step-0 to step-4 are
repeated iteratively until a certain criterion is met (e.g., no more samples
are added to the labeled set). Step-0 may need to run only once if SAM remains
unchanged during the process.

##  2 Related Work

###  2.1 Pseudo-labels

Our proposed SamDSK uses probability maps generated by a medical image
segmentation network for unlabeled images. Although probability maps are not
exactly taken as labels or pseudo-labels, these maps are often referred to as
“soft” pseudo-labels. Hence, below we give a review of the methods that use
pseudo-labels for semi-supervised learning in medical image segmentation.

Wu et al. [27] proposed Mutual Consistency Learning (MCL) for utilizing
unlabeled samples in segmentation model training. MCL creates multiple
decoders which all share one encoder for generating segmentations. Because of
the random initialization, these decoders generate possibly different
segmentation predictions, and the differences between these predictions are
used to compute prediction uncertainties. In addition, mutual learning [30] is
performed during model training to enforce the output from each decoder to be
close (consistent) to the pseudo-labels generated by the other decoders. At
the end of the training, only one decoder is used for model deployment. Since
pseudo-labels inherently contain errors, Thompson et al. [23] proposed to use
super-pixels to refine pseudo-labels in semi-supervised learning. The proposed
process largely follows the classical approach of using pseudo-labels for SSL
[14], with additional steps which refine pseudo-labels according to super-
pixels. The main motivation of this method is to impose a spatial structure
induced from the super-pixels for regularizing and refining pseudo-labels so
that pseudo-labels would yield better quality and benefit the subsequent
segmentation model training. Seibold et al. [20] proposed to use reference
images (with annotations) for generating pseudo-labels. This approach works
for medical image segmentation tasks with well-defined and stable object
structures (e.g., chest radiographic anatomy segmentation). For tasks under
more dynamic scenes where objects can be at arbitrary locations and with large
variations of shapes (e.g., polyp segmentation in endoscopic images),
reference-based pseudo-label generation could become less reliable and
effective. Recently, Basak et al. [4] proposed to utilize pseudo-labels in
conjunction with contrastive learning.

Our proposed method utilizes probability maps generated by a task-specific
medical image segmentation model. Similar to [23], we employ an external
component (i.e., SAM in our case) to work with the medical image segmentation
model. Instead of refining pseudo-labels using super-pixels, we aim to use DSK
to match and select SAM-generated segmentation proposals and construct
segmentations for unlabeled images for model re-training. Comparing to the
super-pixel based methods, SAM is a much more advanced model and using SAM
allows us to design a more controllable optimization process (e.g, multiple
rounds of optimal matching) in the usage of domain-specific knowledge both at
the pixel level and image level for generating segmentations of unlabeled
images.

###  2.2 SAM for Medical Image Segmentation

Since the introduction of the Segment Anything Model (SAM), there has been a
wave of attempts to develop a more accurate and robust medical image
segmentation system with the utilization of SAM [11, 32, 16, 26, 28, 18].
Recent work has shown that SAM alone, without further fine-tuning and/or
adaptation, often delivers unsatisfactory results for medical image
segmentation tasks [11, 32]. To utilize SAM more effectively, Ma et al. [16]
proposed to fine-tune SAM using labeled images. Wu et al. [26] proposed to add
additional layers to adapt SAM for a medical image segmentation task. Our work
aims at using SAM for semi-supervised learning of a medical image segmentation
model.

###  2.3 SAM and Pseudo-labels

Recently, Chen et al. [7] proposed to use SAM to refine pseudo-labels in
weakly-supervised learning. SAM is employed in three places: (1) refining CAM-
generated attention maps, (2) refining segmentation generated by post-
processing, and (3) refining segmentations generated by a segmentation model
(DeepLab). In this paper, we utilize probability maps as pixel-level DSK for
selecting segmentation proposals produced by SAM. To make the selection more
reliable, we further employ an image-level DSK (e.g., the potential number of
regions of interests) during the selection process to constrain the
optimization process as a way of realizing the idea of combining SAM with
domain-specific knowledge (at both the pixel level and image level).

Figure 3: Matching segmentation proposals generated by SAM with pixel-level
DSK generated by the current trained segmentation model. The matching is
constrained by additional image-level DSK, i.e., potential counts of RoIs.
Green arrows mark the obtained solution.

##  3 Methodology

The input to our SamDSK model consists of four components: a labeled image set
$D=\\{(x_{1},y_{1}),(x_{2},y_{2}),\dots,(x_{n},y_{n})\\}$, an unlabeled image
set $U=\\{u_{1},u_{2},\dots,u_{m}\\}$, Segment Anything Model SAM, and a
medical image segmentation model ${M}$ with an encoder initialized using
ImageNet [9] pre-trained weights and a decoder initialized randomly. SamDSK
performs five main steps (step-0 to step-4). An overview of the five steps and
their relations are illustrated in Fig. 2. In Sections 3.2, 3.3, and 3.4, we
describe the key step (step-3) for generating annotations of unlabeled images.
In Section 3.5, we present the process of segmentation model training using
both human-labeled and machine-labeled images. In Section 3.6, we analyze the
dynamics of our proposed multi-round SSL method. Finally, in Section 3.7, we
give discussions of several medical image segmentation tasks that SamDSK is
applicable to.

###  3.1 Applying SAM to Unlabeled Medical Images

Given a set $U$ of unlabeled medical images, we apply the state-of-the-art SAM
to these images to obtain segmentation proposals in each image. A key
consideration in this step is to use relatively lower threshold settings in
SAM to ensure the inclusion of all potential Regions of Interest (RoIs) in the
generated segmentation proposals. By default, we use the following parameter
values when employing SAM: “crop_nms_threshold = 0.5, box_nms_thresh = 0.5,
pred_iou_thresh = 0.5, stability_score_thresh = 0.5”. With the ongoing
development of the segmentation foundation model, future SAM variants are
expected to provide improved coverage of RoIs for various medical image
segmentation tasks.

###  3.2 Matching Segmentation Proposals with Pixel-level DSK

Suppose we already train a segmentation model ${M}$ using the currently
available labeled data. Also, suppose for an unlabeled image $u$ with width
$w$ and height $h$, SAM provides a set of $K$ segmentation proposals,
$S=\\{s_{1},s_{2},\dots,s_{K}\\}$, where $s_{i}\in\\{0,1\\}^{h\times w}$ for
$i=1,2,\dots,K$. The segmentation model ${M}$ gives $C$ distinct probability
maps $\\{p_{1},p_{2},\dots,p_{C}\\}$, where $p_{c}\in\mathbb{R}^{h\times w}$
for $c=1,2,\dots,C$. $C$ is the number of classes in the segmentation task.
The background class is taken always as the class $C$. For each class
$c\in\\{1,2,\dots,C\\}$, we create a set of binary scalar indicators,
$Z_{c}=\\{z_{c,1},z_{c,2},\dots,z_{c,K}\\}$, where $z_{c,k}\in\\{0,1\\}$ for
$k=1,2,\dots,K$. Each indicator $z_{c,k}$ is multiplied with the corresponding
segmentation proposal $s_{k}$ in constructing the segmentation map for class
$c$. We aim to find an optimal configuration of
$z_{1,1},z_{1,2},\dots,z_{C,K}$ which maximizes the total sum of IoU scores
between the segmentation proposals and the segmentation probability maps, as:

$z_{1,1}^{*},\dots,z_{C,K}^{*}=\operatorname*{argmax}_{z_{1,1},z_{1,2},\dots,z_{C,K}}\sum_{c=1}^{C-1}\textrm{IoU}(p_{c},\tau(\sum_{k=1}^{K}z_{c,k}s_{k})),$
(1) $\displaystyle s.t.,$ $\displaystyle\sum_{c=1}^{C-1}z_{c,k}\leq 1,$ (2)
$\displaystyle z_{c,k}\in\\{0,1\\},\ c=1,2,\dots,C,\ k=1,2,\dots,K,$

where $\tau(\cdot)$ converts all values which are larger than 1 to 1, and
$\textrm{IoU}(\cdot,\cdot)$ computes the Intersection over Union score between
a probability map and a segmentation proposal. The constraints in Eq. (2)
ensure that each segmentation proposal can be assigned to only one class
label. Note that there can be segmentation proposals not being selected in
forming the optimal matching.

In Fig. 3, we provide a high-level illustration of the mechanism that we use
for this optimal matching formulation. The optimization problem in Eq. (1)
(with the constraints in Eq. (2)) is a maximum bipartite matching problem
which can be efficiently solved in polynomial time [3].

###  3.3 Incorporating Image-level DSK

In this section, we aim to improve the effectiveness and reliability of the
above proposed segmentation annotation construction by adding image-level DSK
as new constraints to the optimal matching formulation. Suppose the number of
RoIs for an image of a given medical image segmentation task ranges from
$v_{c}^{lower}$ to $v_{c}^{upper}$ for class $c$. We add these constraints to
Eq. (2). The optimization objective remains unchanged, but the constraints are
updated with additional constraints imposed by $v_{c}^{lower}$ and
$v_{c}^{upper}$. Formally, we aim to optimize the objective in Eq. (1), with
the following constraints:

$\displaystyle s.t.,$ $\displaystyle\sum_{c=1}^{C-1}z_{c,k}\leq 1,$ (3)
$\displaystyle v_{c}^{lower}\leq\sum_{k=1}^{K}z_{c,k}\leq
v_{c}^{upper},c\in\\{1,2,\dots,C-1\\},$ $\displaystyle z_{c,k}\in\\{0,1\\},\
c=1,2,\dots,C,\ k=1,2,\dots,K.$

In each round of the model training and labeled set expansion, we control the
$v_{c}^{lower}$ and $v_{c}^{upper}$ values to ensure a stable way of gradually
including more samples with more RoIs into the labeled set. For the first
round, we can set $v^{lower}$ and $v^{upper}$ as both equal to 1. This
indicates that the optimal matching is only a one-to-one matching between the
segmentation proposals and the probability maps. In the second round, we can
set $v_{c}^{lower}$ as 1 and $v_{c}^{upper}$ as 2 to allow a two-to-one
matching. Depending on the specific segmentation task, we can use a larger
increment of $v_{c}^{upper}$ to allow a more effective inclusion of unlabeled
samples into the labeled set.

Figure 4: Image examples of polyp segmentation ((a) and (b)), skin lesion
segmentation ((c) and (d)), and breast cancer segmentation ((e) and (f)). Row
1: raw images. Row 2: ground truth annotations. Row 3: Boundary maps of the
SAM-generated segmentation proposals.

###  3.4 Segmentation Annotation Construction

With the identified values $z_{1,1}^{*},z_{1,2}^{*},\dots,z_{C,K}^{*}$ through
optimizing Eq. (1) and Eq. (3), we compute the overall score $\beta$ for the
image, as follows:

$\displaystyle\beta=\frac{1}{C-1}\sum_{c=1}^{C-1}\textrm{IoU}(p_{c},\tau(\sum_{k=1}^{K}z_{c,k}^{*}s_{k})).$
(4)

A sample with a $\beta$ score higher than a pre-defined threshold (denoted as
$\beta^{*}$; its default value is 0.9) is selected to advance to the next
round of segmentation model training. For a selected sample, its annotation
map is constructed as $q_{c}=\tau(\sum_{k=1}^{K}z_{c,k}^{*}s_{k})$, for
$c=1,2,\dots,C-1$. Pixel areas not covered by any segmentation proposals
selected by the optimal matching are considered as belonging to the background
class (class $C$).

###  3.5 Segmentation Model Training

Suppose the unlabeled samples with their annotation maps generated by the
above optimization procedure are denoted as
${D^{machine}}=\\{(u_{j},q_{j,c})\\}$, for $j=1,2,\dots,m^{\prime}$,
$m^{\prime}\leq m$, and $c=1,2,\dots,C$. The original human-labeled set is
denoted as $D=\\{(x_{i},y_{i,c})\\}$, for $i=1,2,\dots,n$, $c=1,2,\dots,C$. We
aim to optimize the following objective function with respect to the
parameters $\theta$ of a medical image segmentation model ${M}$, as:

$\operatorname*{argmin}_{\theta}(\sum_{c=1}^{C}\sum_{i=1}^{n}{L}(M(x_{i},y_{i,c}))+\lambda\sum_{c=1}^{C}\sum_{j=1}^{m^{\prime}}{L}(M(u_{j},q_{j,c}))),$
(5)

where $\lambda$ is set as 1 by default. Assuming that the medical image
segmentation model ${M}$ generates probability maps for each class, the loss
function $L$ can take the form of a region-based loss (e.g., Dice loss), a
pixel-level entropy-based loss, or a combination of these two types of losses.
A mini-batch based stochastic gradient descent method (e.g., Adam) can be
applied to optimize the loss via updating the parameters $\theta$ of the model
${M}$.

###  3.6 Analyses

The SamDSK method combines SAM-generated segmentation proposals with domain-
specific knowledge both at the pixel-level and image-level in order to
generate annotations for unlabeled images. Pixel-level DSK is provided by the
segmentation model ${M}$ trained on the current labeled image set, and optimal
matching is performed with constraints imposed by image-level DSK as well as
the base constraints that only one segmentation proposal can be assigned to
one class label. There exist clearly two cases after the optimal matching
process for constructing annotations of the unlabeled images.

  * •

Case-1: Prediction maps produced by the current segmentation model ${M}$ match
well with a subset of the segmentation proposals (generated by SAM). In this
scenario, the optimal matching gives a high $\beta$ matching score (i.e.,
higher than $\beta^{*}$), and this image with the constructed annotations is
added to the labeled image set for the subsequent rounds of segmentation model
retraining.

  * •

Case-2: Prediction maps produced by the current segmentation model ${M}$ do
not match well with any subsets of the segmentation proposals generated by
SAM. In this situation, optimal matching gives a low $\beta$ matching score,
and this image is not added to the labeled set for the next round of model
retraining.

For simplicity, we use a binary segmentation task to illustrate, and focus on
its segmentation class #1 (the foreground class) to provide the analyses
below. The same logic applies to multi-class segmentation tasks with multiple
classes of foreground objects.

Assumption 1: For an unlabeled image $u$, there exists a subset of
segmentation proposals, $\Gamma_{1}$, generated by SAM, such that the union of
its elements closely approximates the ground truth of segmentation class 1 in
$u$.111SamDSK is not yet ready to be applied to those medical image
segmentation tasks for which SAM fails to generate sufficiently good
segmentation proposals. More formally, this assumption can be described as:

$\textrm{IoU}(\bigcup_{s\in\Gamma_{1}}s,gt_{1})>1-\epsilon,$ (6)

where $gt_{1}$ represents the ground truth annotation map of class 1 for the
image $u$, and $\epsilon$ is a small positive value (e.g., $\epsilon$ = 0.02).
It is important to note that we do not have access to the ground truth
annotations of the unlabeled images.

Proposition 1: If an unlabeled image falls into the Case-2 category described
above, then its probability maps generated by the segmentation model ${M}$ are
not closely aligned with their corresponding ground truth annotation maps.

Proof: For an unlabeled image in Case-2, the optimal matching identifies no
subset of the segmentation proposals that yields a sufficiently high
Intersection over Union (IoU) score (e.g., a score higher than 0.9) for the
probability map of class 1 (i.e., $p_{1}$) generated by the segmentation model
${M}$. Given that $\Gamma_{1}$ is a feasible solution attained when seeking an
optimal matching, it follows that the union of elements in $\Gamma_{1}$ is not
well-matched with the probability map $p_{1}$. This can be formally expressed
as:

$\textrm{IoU}(\bigcup_{s\in\Gamma_{1}}s,p_{1})<\beta^{*}.$ (7)

Since the union of elements in $\Gamma_{1}$ closely approximates the ground
truth and $p_{1}$ is not well aligned with the union of elements in
$\Gamma_{1}$, it follows that $p_{1}$ is NOT closely aligned with the ground
truth annotation map $gt_{1}$. Formally, an upper bound of the Intersection
over Union (IoU) between the ground truth annotation map and the prediction
map of class #1 is given by:

$\textrm{IoU}(gt_{1},p_{1})\lesssim\beta^{*}+\epsilon.$ (8)

With the above argument, we have demonstrated that the unlabeled samples which
are not added to the labeled set (Case-2) do not yet contain sufficiently
accurate annotations (e.g., $p_{1}$ is not sufficiently accurate with respect
to the ground truth). On the other hand, for the unlabeled images with
annotations that are added to the labeled set (Case-1), it is still possible
that although their annotations match well with a subset of the segmentation
proposals, denoted as $\Gamma_{1}$, the ground truth annotation map is
actually closer to another subset $\Gamma_{1}^{\prime}$ of the segmentation
proposals, and $\Gamma_{1}$ may or may not be equal to $\Gamma_{1}^{\prime}$.
Consequently, we cannot provide an explicit guarantee on the correctness of
$p_{1}$ and its corresponding $\bigcup_{s\in\Gamma_{1}}s$ for the samples in
Case-1. Nevertheless, in our experiments, we present empirical evidence to
demonstrate that the annotations of samples in Case-1 (those added to the
labeled set) are closer to their corresponding ground truth annotations than
those in Case-2.

Table 1: Breast cancer segmentation in ultrasound images (in Dice coefficient). The best results are marked in bold, and the second best results are underlined. The red color highlights performance degradation compared to the baseline. The same markings are used in the other tables. Method | Network | Labeled | Unlabeled | Dice
---|---|---|---|---
Baseline | TransUNet | 30% | 0% | 52.7
PL [14] | 70% | 50.8
SP-Refine [23] | 53.2
BCP [2] | 53.7
UPS [19] | 54.3
SamDSK (ours) | 59.4
Baseline | HSNet | 30% | 0% | 64.3
PL [14] | 70% | 69.5
SP-Refine [23] | 69.7
BCP [2] | 64.1
UPS [19] | 70.8
SamDSK (ours) | 73.6
Table 2: Segmentation performances of different methods on five polyp segmentation datasets (in Dice coefficient). Method | Network | Labeled | Unlabeled | CVC-300 | CVC-ClinicDB | Kvasir | CVC-ColonDB | ETIS
---|---|---|---|---|---|---|---|---
Baseline | PraNet | 10% | 0% | 82.3 | 76.2 | 83.1 | 63.7 | 60.2
PL [14] | 90% | 83.8 | 77.7 | 85.1 | 63.4 | 63.3
BCP [2] | 83.4 | 78.3 | 85.3 | 65.6 | 64.3
UPS [19] | 84.3 | 81.7 | 86.6 | 62.3 | 58.2
SamDSK (ours) | 89.4 | 84.4 | 88.0 | 67.6 | 63.5
Baseline | HSNet | 10% | 0% | 85.1 | 81.7 | 86.9 | 66.4 | 68.9
PL [14] | 90% | 87.5 | 84.3 | 89.6 | 71.5 | 72.9
BCP [2] | 85.8 | 84.9 | 88.1 | 68.5 | 68.3
UPS [19] | 88.3 | 86.3 | 89.8 | 72.9 | 76.7
SamDSK (ours) | 88.3 | 85.2 | 90.2 | 76.6 | 72.7
Baseline | PraNet | 30% | 0% | 89.7 | 83.2 | 86.5 | 63.2 | 64.7
PL [14] | 70% | 90.8 | 86.7 | 88.3 | 68.5 | 61.1
BCP [2] | 88.3 | 84.3 | 89.1 | 69.0 | 69.7
UPS [19] | 87.8 | 84.2 | 87.1 | 64.8 | 58.3
SamDSK (ours) | 91.7 | 85.2 | 87.6 | 71.4 | 67.4
Baseline | HSNet | 30% | 0% | 87.1 | 86.5 | 90.9 | 74.9 | 76.4
PL [14] | 70% | 87.8 | 88.9 | 91.2 | 77.3 | 76.9
BCP [2] | 88.3 | 88.4 | 90.9 | 75.5 | 74.0
UPS [19] | 87.6 | 89.4 | 91.8 | 77.8 | 78.5
SamDSK (ours) | 88.9 | 89.6 | 92.0 | 79.8 | 77.5
Table 3: Skin lesion segmentation on the ISIC 2018 dataset (in Dice coefficient). Method | Network | Labeled | Unlabeled | Dice
---|---|---|---|---
Baseline | PraNet | 10% | 0% | 86.1
PL [14] | 90% | 87.3
SP-Refine [23] | 87.2
BCP [2] | 87.5
SamDSK (ours) | 88.1
Baseline | HSNet | 10% | 0% | 86.5
PL [14] | 90% | 87.7
SP-Refine [23] | 88.2
BCP [2] | 88.4
SamDSK (ours) | 89.9
Figure 5: Mean IoU (mIoU) of segmentation predictions and ground truth
annotations for selected samples. The $\beta$ score (in Eq. (4)) enables
SamDSK to select samples with higher segmentation quality (higher mIoU)
compared to samples selected by using the model’s own uncertainty and
confidence measures (as conducted in UPS [19]). Figure 6: Ground truth
annotations and segmentation predictions after dimensionality reduction (PCA).
The segmentation predictions for samples in Case-1 are closer to the ground
truth annotations than those in Case-2. Top row: samples of the polyp
segmentation dataset. Bottom row: samples of the skin lesion segmentation
dataset.  Figure 7: Segmentation performances (in Dice coefficient) across
different rounds of processing in SamDSK.

###  3.7 Applicable Segmentation Tasks

Medical image segmentation encompasses a multitude of imaging modalities
(e.g., MRI, CT, microscopy), covering a wide range of diverse objects of
interest and clinical tasks. SamDSK relies on SAM to provide segmentation
proposals. Therefore, SamDSK is especially applicable to those medical image
segmentation tasks in which the regions of interest can be adequately
addressed by segmentation proposals generated by SAM. It is important to note
that our optimal matching formulation allows for the utilization of multiple
segmentation proposals to cover one or more regions of interest. Consequently,
SamDSK accommodates segmentation proposals that consist of over-segmented
regions with respect to the regions of interest.

For illustration, we highlight several specific medical image segmentation
tasks for which SamDSK is well-suited: (1) polyp segmentation in endoscopic
images, (2) skin lesion segmentation in dermoscopic images, and (3) tumor
region segmentation in ultrasound images. Fig. 4 presents visual examples of
these tasks and the corresponding segmentation proposals generated by SAM. We
conduct comprehensive experiments to demonstrate the effectiveness of SamDSK
in tackling these segmentation tasks, as shown in the next section.

##  4 Experiments

We utilize three public datasets to demonstrate the effectiveness of our
proposed SamDSK method. We compare SamDSK with several classical methods as
well as state-of-the-art methods: (1) Pseudo-labels (PL) [14]; (2) SP-Refine
[23]: utilizing superpixels for refining pseudo-annotations; (3) Bidirectional
Copy-Paste (BCP) [2]; (4) Uncertainty-aware Pseudo-label Selection (UPS) [19]:
a multi-round SSL method based on uncertainty-aware pseudo-label selection.

In addition to comparing with the known methods, we conduct ablation and
additional studies to validate the effectiveness of the key components in
SamDSK. In Section 4.4.1, we validate the effectiveness of utilizing SAM for
sample selection. In Section 4.4.2, we validate the effectiveness of multi-
round SSL with SAM. In Section 4.4.3, we demonstrate empirical coverage of the
SAM-generated segmentation proposals with respect to ground truth annotations.

###  4.1 Breast Cancer Segmentation in Ultrasound Images

We first utilize the breast cancer segmentation task in ultrasound images [1]
to validate the effectiveness of our proposed SamDSK. This task is a binary
segmentation problem of segmenting breast cancer regions in ultrasound images.
We randomly split the original dataset into two equally-sized sets, one set
for model training and the other set for testing. In the training set, we
randomly select 30% of samples and treat them as labeled images, and treat the
remaining 70% of samples as unlabeled images.222All the data split information
will be published together with the code release. Two state-of-the-art medical
image segmentation models, TransUNet [6] and HSNet [29], are taken as the
segmentation model $M$ for the experiments. We perform SamDSK for three rounds
of processing (step-0 to step-4 as illustrated in Fig. 2 are one round of
processing). For Eq. (3), both $v_{1}^{lower}$ and $v_{1}^{upper}$ are set as
1. In Table 1, the results show that SamDSK outperforms the state-of-the-art
methods for SSL. SamDSK yields considerably better segmentation performance
than the closely related BPC method, indicating that SAM plays a critical role
in sample selection and annotation construction of unlabeled images. In
addition, the results show that the PL method may degrade the segmentation
performance (see the TransUNet case) when the initial segmentation model
incurs too many errors in pseudo-labels.

###  4.2 Polyp Segmentation in Endoscopic Images

Automatic polyp segmentation in endoscopic images can help improve the
efficiency and accuracy of clinical screenings and tests for gastrointestinal
diseases. Many deep learning (DL) based methods have been proposed for robust
and automatic segmentation of polyps. Here, we utilize the SOTA polyp
segmentation model HSNet [29] and the widely-used polyp segmentation model
PraNet [10] for evaluating our proposed SamDSK method. HSNet uses the PVT
backbone (Transformer-based) and PraNet uses the Res2Net backbone (CNN-based).
We perform SamDSK for three rounds of processing. For Eq. (3), $v_{1}^{lower}$
is set as 1, 1, and 1 for the first, second, and third rounds, respectively,
and $v_{1}^{upper}$ is set as 1, 2, and 3 for the first, second, and third
rounds, respectively (the polyp class is set as class 1). That is, the
constraints on the number of RoIs for the polyp class are gradually changed
when we perform more rounds of model training and labeled set expansion, and
we allow an increase in the number of segmentation proposals for the
foreground-class segmentation. Following the data split settings in [10, 32],
900 images from Kvasir [12] and 550 images from CVC-ClinicDB [5] are randomly
selected to form the training set. The remaining samples from these two
datasets (i.e., Kvasir and CVC-ClinicDB) and the samples from CVC-ColonDB
[22], ETIS [21], and CVC-300 [24] form the test set.

The results are reported in Table 2, which show that SamDSK improves the
segmentation performances of the HSNet and PraNet models for the settings
where 10% and 30% of the total labeled samples from the training set are
available. Compared to the classical pseudo-label method [14] and the recently
proposed methods, SamDSK achieves similar or better segmentation performances.
Notably, we observe that SamDSK is stable and reliable in providing
segmentation improvement, while some of the known methods can encounter
situations where worse segmentation performances might be yielded in some
cases.

###  4.3 Skin Lesion Segmentation in Dermoscopic Images

The ISIC 2018 skin lesion segmentation dataset [8] contains 2594 training
dermoscopic images and 1000 test images for melanoma segmentation. Once again,
we employ HSNet and PraNet for the experiments. We are aware that the image-
level DSK for skin lesion segmentation in most of these images is that they
contain only one region of interest. Consequently, we set $v_{1}^{lower}$ to 1
and $v_{1}^{upper}$ to 1 in Eq. (3) (the melanoma class is designated as class
1) throughout all the rounds of processing. The results in Table 3 demonstrate
that SamDSK enhances the segmentation performances of the state-of-the-art
models when utilizing 10% of the total labeled samples. SamDSK achieves a
higher performance gain in comparison to the competing methods.

###  4.4 Ablation and Additional Studies

####  4.4.1 Effectiveness of Using SAM

A key question that we seek to answer in this section is whether using the
$\beta$ score obtained from optimal matching actually helps in selecting
samples of better segmentation quality. Furthermore, we examine whether using
the model’s own segmentation map uncertainty and confidence serves as a better
criterion for sample selection. In Fig. 5, we show that by using the $\beta$
score obtained from optimal matching, we can select samples with significantly
better segmentation quality (higher mIoU) than using the model’s own
segmentation map uncertainty and confidence measures. For polyp segmentation,
samples with the top 10% of $\beta$ scores achieve a 95% mIoU. For skin lesion
segmentation, samples with the top 10% of $\beta$ scores achieve a 92% mIoU.
Additionally, in Fig. 6, we provide visualization plots for ground truth
annotations and segmentation predictions for samples in Case-1 and Case-2,
which are identified after the optimal matching process. We observe that the
samples in Case-1 (selected and added to the labeled set) exhibit a tighter
alignment with their corresponding ground truth annotations than the samples
in Case-2 (not yet added to the labeled set).

####  4.4.2 Effectiveness of Multi-round SSL

In Fig. 7, we show the segmentation performances with respect to the rounds of
processing in SamDSK, along with that of the initial model (the baseline
model). It is evident that the model’s performance is improved through
multiple rounds of processing, as more samples are added to the labeled set
for model training/retraining.

####  4.4.3 Performances of SAM in Generating Segmentation Proposals

For the polyp segmentation task, we apply optimal matching between the
segmentation proposals (generated by SAM) and the ground truth annotations. We
use those segmentation proposals identified by optimal matching to construct
segmentation annotation maps for each selected unlabeled sample. By comparing
the generated annotation maps with the ground truths, we find that the
segmentation annotations achieve an 89.1% Dice coefficient. We apply the same
procedure to the breast cancer segmentation dataset, and find that the Dice
coefficient is 75.3% for the benign cases and 64.4% for the malignant cases.
These results suggest that SAM performs better on endoscopic images than on
ultrasound images (note that ultrasound images in the breast cancer
segmentation dataset are in grayscale). This is probably due to the fact that
SAM was trained using natural scene color images, which in appearance are
closer to endoscopic images than ultrasound images. Additionally, in the case
of ultrasound images, the benign cases consist of objects with more regular
shapes and better contrast around the object boundaries, which may lead SAM to
yield better coverage with its generated segmentation proposals.

##  5 Conclusions

In this paper, we proposed a new semi-supervised learning method, SamDSK,
which utilizes the Segment Anything Model (SAM) for training a medical image
segmentation model. SAM is employed in our proposed optimal matching system
for selecting and refining segmentation predictions of unlabeled images
(obtained from the current trained segmentation model). A multi-round
iterative procedure of model training and labeled set expansion is performed
on top of the optimal matching system to gradually improve the target
segmentation model while enlarging the labeled training set. Experiments on
three datasets, compared with several recently developed SSL methods,
demonstrated the effectiveness and advantages of our proposed SamDSK method.
Future work may study further expanding the applicable tasks of SamDSK to
other medical imaging modalities and segmentation scenarios.

## References

  * [1] Walid Al-Dhabyani, Mohammed Gomaa, Hussien Khaled, and Aly Fahmy.  Dataset of breast ultrasound images.  Data in Brief, 28:104863, 2020.
  * [2] Yunhao Bai, Duowen Chen, Qingli Li, Wei Shen, and Yan Wang.  Bidirectional copy-paste for semi-supervised medical image segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11514–11524, 2023.
  * [3] Jørgen Bang-Jensen and Gregory Gutin.  Section 2.3.4: The Bellman-Ford-Moore Algorithm. Digraphs: Theory, Algorithms and Applications.  Springer-Verlag, London, UK, 2000.
  * [4] Hritam Basak and Zhaozheng Yin.  Pseudo-label guided contrastive learning for semi-supervised medical image segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19786–19797, 2023.
  * [5] Jorge Bernal, F Javier Sánchez, Gloria Fernández-Esparrach, Debora Gil, Cristina Rodríguez, and Fernando Vilariño.  WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians.  Computerized Medical Imaging and Graphics, 43:99–111, 2015.
  * [6] Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou.  TransUNet: Transformers make strong encoders for medical image segmentation.  arXiv preprint arXiv:2102.04306, 2021.
  * [7] Tianle Chen, Zheda Mai, Ruiwen Li, and Wei-lun Chao.  Segment Anything Model (SAM) enhanced pseudo labels for weakly supervised semantic segmentation.  arXiv preprint arXiv:2305.05803, 2023.
  * [8] Noel Codella, Veronica Rotemberg, Philipp Tschandl, M Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, et al.  Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (ISIC).  arXiv preprint arXiv:1902.03368, 2019.
  * [9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  ImageNet: A large-scale hierarchical image database.  In CVPR, pages 248–255, 2009.
  * [10] Deng-Ping Fan, Ge-Peng Ji, Tao Zhou, Geng Chen, Huazhu Fu, Jianbing Shen, and Ling Shao.  PraNet: Parallel reverse attention network for polyp segmentation.  In International Conference on Medical Image Computing and Computer-assisted Intervention, pages 263–273. Springer, 2020.
  * [11] Yuhao Huang, Xin Yang, Lian Liu, Han Zhou, Ao Chang, Xinrui Zhou, Rusi Chen, Junxuan Yu, Jiongquan Chen, Chaoyu Chen, et al.  Segment Anything Model for medical images?  arXiv preprint arXiv:2304.14660, 2023.
  * [12] Debesh Jha, Pia H Smedsrud, Michael A Riegler, Pål Halvorsen, Thomas de Lange, Dag Johansen, and Håvard D Johansen.  Kvasir-SEG: A segmented polyp dataset.  In MultiMedia Modeling: 26th International Conference, Part II, pages 451–462. Springer, 2020.
  * [13] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al.  Segment anything.  arXiv preprint arXiv:2304.02643, 2023.
  * [14] Dong-Hyun Lee et al.  Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks.  In Workshop on Challenges in Representation Learning, ICML, volume 3, page 896. Atlanta, 2013.
  * [15] Xiangde Luo, Jieneng Chen, Tao Song, and Guotai Wang.  Semi-supervised medical image segmentation through dual-task consistency.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8801–8809, 2021.
  * [16] Jun Ma and Bo Wang.  Segment anything in medical images.  arXiv preprint arXiv:2304.12306, 2023.
  * [17] Himashi Peiris, Zhaolin Chen, Gary Egan, and Mehrtash Harandi.  Duo-SegNet: Adversarial dual-views for semi-supervised medical image segmentation.  In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Proceedings, Part II 24, pages 428–438. Springer, 2021.
  * [18] Yu Qiao, Chaoning Zhang, Taegoo Kang, Donghun Kim, Shehbaz Tariq, Chenshuang Zhang, and Choong Seon Hong.  Robustness of SAM: Segment anything under corruptions and beyond.  arXiv preprint arXiv:2306.07713, 2023.
  * [19] Mamshad Nayeem Rizve, Kevin Duarte, Yogesh S Rawat, and Mubarak Shah.  In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning.  In International Conference on Learning Representations, 2021.
  * [20] Constantin Marc Seibold, Simon Reiß, Jens Kleesiek, and Rainer Stiefelhagen.  Reference-guided pseudo-label generation for medical semantic segmentation.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 2171–2179, 2022.
  * [21] Juan Silva, Aymeric Histace, Olivier Romain, Xavier Dray, and Bertrand Granado.  Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer.  International Journal of Computer Assisted Radiology and Surgery, 9:283–293, 2014.
  * [22] Nima Tajbakhsh, Suryakanth R Gurudu, and Jianming Liang.  Automated polyp detection in colonoscopy videos using shape and context information.  IEEE Transactions on Medical Imaging, 35(2):630–644, 2015.
  * [23] Bethany H Thompson, Gaetano Di Caterina, and Jeremy P Voisey.  Pseudo-label refinement using superpixels for semi-supervised brain tumour segmentation.  In 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pages 1–5. IEEE, 2022.
  * [24] David Vázquez, Jorge Bernal, F Javier Sánchez, Gloria Fernández-Esparrach, Antonio M López, Adriana Romero, Michal Drozdzal, and Aaron Courville.  A benchmark for endoluminal scene segmentation of colonoscopy images.  Journal of Healthcare Engineering, 2017, 2017.
  * [25] Ping Wang, Jizong Peng, Marco Pedersoli, Yuanfeng Zhou, Caiming Zhang, and Christian Desrosiers.  CAT: Constrained adversarial training for anatomically-plausible semi-supervised segmentation.  IEEE Transactions on Medical Imaging, 2023.
  * [26] Junde Wu, Rao Fu, Huihui Fang, Yuanpei Liu, Zhaowei Wang, Yanwu Xu, Yueming Jin, and Tal Arbel.  Medical SAM adapter: Adapting Segment Anything Model for medical image segmentation.  arXiv preprint arXiv:2304.12620, 2023.
  * [27] Yicheng Wu, Zongyuan Ge, Donghao Zhang, Minfeng Xu, Lei Zhang, Yong Xia, and Jianfei Cai.  Mutual consistency learning for semi-supervised medical image segmentation.  Medical Image Analysis, 81:102530, 2022.
  * [28] Chaoning Zhang, Sheng Zheng, Chenghao Li, Yu Qiao, Taegoo Kang, Xinru Shan, Chenshuang Zhang, Caiyan Qin, Francois Rameau, Sung-Ho Bae, et al.  A survey on Segment Anything Model (SAM): Vision foundation model meets prompt engineering.  arXiv preprint arXiv:2306.06211, 2023.
  * [29] Wenchao Zhang, Chong Fu, Yu Zheng, Fangyuan Zhang, Yanli Zhao, and Chiu-Wing Sham.  HSNet: A hybrid semantic network for polyp segmentation.  Computers in Biology and Medicine, 150:106173, 2022.
  * [30] Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu.  Deep mutual learning.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4320–4328, 2018.
  * [31] Yizhe Zhang, Lin Yang, Jianxu Chen, Maridel Fredericksen, David P Hughes, and Danny Z Chen.  Deep adversarial networks for biomedical image segmentation utilizing unannotated images.  In Medical Image Computing and Computer Assisted Intervention- MICCAI 2017: 20th International Conference, Proceedings, Part III 20, pages 408–416. Springer, 2017.
  * [32] Tao Zhou, Yizhe Zhang, Yi Zhou, Ye Wu, and Chen Gong.  Can SAM segment polyps?  arXiv preprint arXiv:2304.07583, 2023.