# NeRT: Implicit Neural Representations for General Unsupervised Turbulence
Mitigation

Michael Shell, and Jane Doe, M. Shell is with the Department of Electrical and
Computer Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.
E-mail: see http://www.michaelshell.org/contact.html J. Doe is with Anonymous
University. Weiyun Jiang  Vivek Boominathan Ashok Veeraraghavan
Rice University
{wyjiang, vivekb<EMAIL_ADDRESS>
###### Abstract

The atmospheric and water turbulence mitigation problems have emerged as
challenging inverse problems in computer vision and optics communities over
the years. However, current methods either rely heavily on the quality of the
training dataset or fail to generalize over various scenarios, such as static
scenes, dynamic scenes, and text reconstructions. We propose a general
implicit neural representation for unsupervised atmospheric and water
turbulence mitigation (NeRT). NeRT leverages the implicit neural
representations and the physically correct tilt-then-blur turbulence model to
reconstruct the clean, undistorted image, given only dozens of distorted input
images. Moreover, we show that NeRT outperforms the state-of-the-art through
various qualitative and quantitative evaluations of atmospheric and water
turbulence datasets. Furthermore, we demonstrate the ability of NeRT to
eliminate uncontrolled turbulence from real-world environments. Lastly, we
incorporate NeRT into continuously captured video sequences and demonstrate
$48\times$ speedup.

###### Index Terms:

seeing through turbulence, seeing through water, implicit neural
representations

TABLE I: How is our method, NeRT, different from other SOTA? NeRT is the first generalizable unsupervised physically grounded model. ∗Section 7 discusses the anytime convergence property of NeRT. Method | Physically grounded? | Correct tilt-then-blur model? | Unsupervised? | Generalizable? | Time-efficient?
---|---|---|---|---|---
CLEAR [1] | ✔ | ✗ | ✔ | ✔ |  $\sim$5 secs
Mao et al. [2] | ✔ | ✗ | ✔ | ✔ | 10 secs $\sim$1 hr
TurbuGAN [3] | ✔ | ✔ | ✔ | ✗ |  $\sim$3 hrs
TurbNet [4] | ✔ | ✔ | ✗ | ✗ |  $\sim$1 sec
TSR-WGAN [5] | ✗ | ✗ | ✗ | ✗ |  $\sim$60 secs
Li et al. [6] | ✗ | ✗ | ✗ | ✗ |  $\sim$1 sec
Tian et al. [7] | ✔ | ✗ | ✔ | ✔ |  $\sim$10 secs
Oreifej et al. [8] | ✔ | ✗ | ✔ | ✔ |  $\sim$3 hrs
NDIR [9] | ✗ | ✗ | ✔ | ✔ | Anytime∗
NeRT (Ours) | ✔ | ✔ | ✔ | ✔ | Anytime∗

##  1 Introduction

General turbulence distortion is a common phenomenon that we encounter in our
daily lives. Whenever light waves pass through nonuniform mediums before
reaching the imaging system, they are refracted and distorted. In a uniform
medium, such as free space and outer space, the imaging system always captures
clean and sharp images. However, when light passes from a dense medium to a
sparse medium, it is refracted according to Snell’s law, resulting in a
sequence of distorted images. Seeing through water is a simple example of such
distortion, where water and air act as two mediums with different refractive
indices. When the water surface moves up and down randomly, it becomes
challenging to mitigate water turbulence and reconstruct the original
undistorted scene underwater. In addition, seeing through atmospheric
turbulence presents an even more challenging task, especially for ground-based
long-range imaging systems. The intermediate medium between the objects and
the imaging systems may extend for several kilometers, and various factors,
such as wind, humidity, temperature, and distance, can affect the medium’s
refractive index along the light path. Unlike water turbulence, where we only
need to consider the water-air interface, atmospheric turbulence involves an
unlimited number of nonuniform interfaces that refract light arbitrarily,
resulting in spatiotemporally varying tilting and blurring in distorted images
[10].

Existing deep learning approaches [3, 11, 2, 5, 12, 13, 6] for seeing through
atmospheric turbulence require a huge amount of training dataset. In the first
place, it is a challenging task to build a physically accurate and fast
simulator [14, 15, 16, 17]. In addition, by leveraging domain-specific priors,
these existing deep learning approaches inevitably have dataset biases [18,
19, 20] and poor performance for out-of-domain distributions.

Classical non-deep learning approaches [2, 21, 1, 22] for atmospheric
turbulence mitigation do not require a huge amount of training dataset.
However, they need to rely on some non-rigid registration techniques, such as
B-spline functions [21], optical flow [2, 22], and diffeomorphism [23], to
model the grid deformation under atmospheric turbulence. All of these non-
rigid registration methods, such as B-spline functions, optical flow, and
diffeomorphism, require a reference frame to start with. The reference frames
selected by these methods do not accurately represent the original sharp
image. Mao et al. [2] select the sharpest distorted input frame, which still
contains blurring and tilting, as the reference frame for the optical flow
algorithm. Additionally, Shimizu et al. [21] select the temporal average of
all the distorted input frames, which also contain noise, blurring, and
tilting, as the reference frame. Furthermore, Gilles et al. [23] choose the
reference frame as the temporal median of all the distorted input frames,
which is inherently distorted by noise, blurring, and tilting.

To address the above challenges, we primarily focus on atmospheric turbulence
and design an implicit neural representation for unsupervised atmospheric
mitigation (NeRT), removing temporally and spatially tilting and blurring.
Building upon this success, we also extend NeRT to address other types of
turbulence distortions, including mitigating water turbulence and handling
water ripple reflection turbulence in uncontrolled environments. The key idea
is to constrain the network to learn the physically correct air turbulence
forward model, the tilt-then-blur model [24]. Inspired by NDIR [9], NeRT can
model temporally and spatially tilting using deformed grids and implicit
neural representations. For instance, if the implicit neural representations
take uniform undistorted coordinates, they render clean and sharp images. If
the implicit neural representations take distorted coordinates, they will
output the corresponding distorted images under atmospheric turbulence.

The overall architecture of our learning framework, NeRT, is depicted in
Figure 1. The network contains three major components, grid deformers
$\mathcal{G}$ that estimate the spatially and temporally varying tilting at
each pixel location, an image generator $\mathcal{I}$ that outputs pixel
values at corresponding coordinates, and shift-varying blurring $\mathcal{P}$
that approximates the spatially blurring at each pixel location.

We perform extensive experiments on both atmospheric and water turbulence
datasets. Our study demonstrates that NeRT outperforms the state-of-the-art
supervised methods in mitigating atmospheric turbulence. In addition, NeRT
performs comparably to the state-of-the-art methods in removing water
turbulence. We further demonstrate NeRT’s robustness in uncontrolled
environments, where it effectively mitigates water ripple reflection
turbulence. Our specific contributions include as follows:

  * •

We are the first to propose an unsupervised and physically grounded deep
learning method for general turbulence mitigation. The pipeline follows the
physically correct forward turbulence model — the tilt-then-blur model[24].

  * •

Our unsupervised algorithm is highly generalizable as it can recover clean and
distortion-free images without domain-specific priors such as distorted-clean
image pairs. We validate this generalizability by testing on image datasets
seeing through atmospheric and water turbulence.

  * •

We test our method on newly collected videos of water reflection turbulence in
uncontrolled environments and further validate our method’s generalizability.

  * •

Finally, we show a path toward faster and real-time reconstruction by
processing the current video frame based on network parameters learned from
previous frames. We call this “anytime” reconstruction.

##  2 Related work

Implicit neural representations. Implicit neural representations, which use
multi-layer perceptions (MLPs) as the backbone networks, store 2D images [25,
26] and 3D shapes [27, 28] as continuous functions. The inputs of implicit
neural representations are $2$D or $3$D coordinates, while the outputs are the
corresponding signal. This kind of continuous representation shows not only
extraordinary results in overfitting a single image or multiple images but
also exceeds other state-of-the-art architectures in solving inverse problems,
such as single-image superresolution [29], medical image reconstruction [30]
and medical image registration [31]. Our work, NeRT, uses implicit neural
representations as $2$D image functions to render distorted images under
atmospheric turbulence and clean, undistorted images.

Figure 1: The overall architecture of NeRT. The network predicts a clean and
sharp image $\hat{\mathbf{J}}$, given a series of observed atmospheric
turbulence distorted images $\mathbf{I}$. We compute $L_{1}$ loss between
predicted tilted-then-blurred images
$\mathbf{I}_{\mathcal{B}\circ\mathcal{T}}$, resampled tilted-then-blurred
images $\mathbf{I}^{G}_{\mathcal{B}\circ\mathcal{T}}$, and observed images
$\mathbf{I}$ during optimization to update the parameters in image generators
$\mathcal{I}_{\phi}$, grid deformers $\mathcal{G}_{\theta}$, and shift-varying
blurring $\mathcal{P}_{\alpha}$.

Atmospheric turbulence mitigation. Many works have been proposed to undistort
the effects of atmospheric turbulence. Some recent works have demonstrated the
uses of transformer architectures as supervised turbulence removal networks
for single frame [4, 32, 33, 34, 35, 36, 37] and muti-frame [11, 38, 5, 39,
40] atmospheric turbulence mitigation tasks. These proposed supervised
transformer architectures must rely on fast and physically accurate simulators
to generate a huge collection of paired distorted-clean image pairs for
training datasets. Although they have fast inference speed, they are hard to
generalize over out-of-the-domain datasets. Our proposed unsupervised learning
architecture, NeRT, does not require any datasets for pretraining and, thus,
can generalize over all kinds of datasets. TurbuGAN [3] proposes a self-
supervised approach for imaging through turbulence by leveraging an
adversarial learning framework and a fast turbulence simulator [15]. This
approach requires no paired training datasets; however, it is hard to
generalize domain-specific priors over out-of-the-domain distributions. NDIR
[9] is the closest to our work. This method exploits convolutional neural
networks to model non-rigid distortion. However, it relies on an off-the-shelf
physically incorrect spatially invariant deblurring algorithm [41]. Our
method, NeRT, incorporates a physically grounded, spatially and temporally
varying deblurring approach to restoring the sharp image.

Seeing through water. Over the years, numerous methods have been proposed to
restore the clear underwater scene using multiple images or a single image,
distorted by water turbulence. Tian et al. [7] propose to build a compact
spatial distortion model of the water surface using the wave equation.
Together with a novel tracking technique, they propose to use the compact
model to remove water turbulence. Oreifej et al. [8] presents a two-stage
approach for seeing through the water. During the first stage, they
simultaneously estimate the spatially-varying blur kernel and non-rigid
distortion to de-warp the distorted frames. During the second stage, they
apply an off-the-shelf low-rank optimization algorithm to denoise the
registered frames. Li et al. [6] propose to train a deep convolution neural
network to undistort the dynamic refractive effects of water turbulence using
only a single image. Given the supervised method requires only a single image
and thousands of carefully collected training image pairs, Li et al. [6] have
a very poor ability for generalization.

##  3 Physically grounded restoration network

Given dozens of turbulence-distorted input images,
$\mathbf{I}\in\mathbb{R}^{m\times n\times 3}$, NeRT predicts a clean and sharp
image without any distortions, $\hat{\mathbf{J}}\in\mathbb{R}^{m\times n\times
3}$ without any domain-specific image priors.

The key idea is to optimize an unsupervised model to precisely represent both
the spatially and temporally varying tiltings and the spatially and temporally
varying blurrings from general turbulence, such as water and atmospheric
turbulence. Once we have the ability to estimate the forward model, reversing
it becomes a simple task that allows us to reconstruct the undistorted image,
denoted as $\hat{\mathbf{J}}\in\mathbb{R}^{m\times n\times 3}$.

Our model has three major components, grid deformers, image generators, and
shift-varying blurring. Each grid deformer, $\mathcal{G}_{\theta}$ aims to
estimate the spatially varying and temporally varying tiltings (non-rigid
deformations) of every distorted input image. Then, given the estimated non-
rigid deformed $2$D pixel coordinates, the image generator,
$\mathcal{I}_{\phi}$ generates the tilted image
$\mathbf{I}_{\mathcal{T}}\in\mathbb{R}^{m\times n\times 3}$. At the same time,
given the uniform and undistorted $2$D pixel coordinates,
$\mathbf{G}_{U}\in\mathbb{R}^{m\times n\times 2}$, the image generator,
$\mathcal{I}_{\phi}$ generates the clean and sharp image
$\hat{\mathbf{J}}\in\mathbb{R}^{m\times n\times 3}$. We also resample the
clean and sharp image $\hat{\mathbf{J}}\in\mathbb{R}^{m\times n\times 3}$
based on the estimated deformed $2$D pixel coordinates,
$\mathbf{G}_{\mathcal{T}}\in\mathbb{R}^{m\times n\times 2}$ to predict the
resampled tilted image $\mathbf{I}^{G}_{\mathcal{T}}\in\mathbb{R}^{m\times
n\times 3}$. Finally, given generated tilted image
$\mathbf{I}_{\mathcal{T}}\in\mathbb{R}^{m\times n\times 3}$ and resampled
tilted image $\mathbf{I}_{\mathcal{T}}\in\mathbb{R}^{m\times n\times 3}$, the
shift-varying blurring predicts the generated tilted-then-blurred image
$\mathbf{I}_{\mathcal{B}\circ\mathcal{T}}\in\mathbb{R}^{m\times n\times 3}$
and resampled tilted-then-blurred image
$\mathbf{I}^{G}_{\mathcal{B}\circ\mathcal{T}}\in\mathbb{R}^{m\times n\times
3}$ respectively.

One crucial aspect of NeRT is its unsupervised optimization of grid reformers,
$\mathcal{G_{\theta}}$, the image generator, $\mathcal{I}_{\phi}$, and shift-
varying blurring, $\mathcal{P}_{\alpha}$, which enables the identification of
perfect spatiotemporal blurring kernels and deformed grids for each distorted
input frame. Section 4 showcases NeRT’s superiority over other state-of-the-
art methods in effectively mitigating various types of turbulence, including
atmospheric and water turbulence. Furthermore, Section 6 presents qualitative
results that demonstrate the robustness of NeRT in eliminating uncontrolled
turbulence from real-world environments.

###  3.1 Forward atmospheric turbulence model

Imagine the light reflected from a scene, represented as a clean and sharp
image $\mathbf{J}$, travels through space with a spatially and temporally
varying index of refraction. The light finally arrives at a passive imaging
device, such as a digital single-lens reflex (DSLR) camera, forming many
distorted images $\mathbf{I}$ over time. Each $\mathbf{I}$ has stochastic
distortion at different pixel locations and time stamps. The generalized
forward atmospheric turbulence model can be written as [24]:

$\mathbf{I}(x,y,t)=\mathcal{H}_{t}(\mathbf{J}(x,y,t)),$ (1)

where $\mathcal{H}$ is a general linear distortion operator. However, we
desire to decompose $\mathcal{H}$ into simpler operations for the
computational tractability of inverting the distortion. Fortunately,
representation using Zernike bases allows us to do so.

The distortion from atmospheric turbulence can be parameterized by
coefficients of Zernike polynomials in the phase space [14]. The
representation in the Zernike space allows us to decouple the distortion into
interpretable operations of tilting operation ($\mathcal{T}$) and blurring
operation ($\mathcal{B}$). The tilt $\mathcal{T}$ is encoded by the first two
Zernike bases (barring the constant term) and is the most significant
contributor to shifting the center of mass of the distortion spread. The blur
$\mathcal{B}$ is encoded by the remaining Zernike bases and describes the
distortion spread. The question is how to compose the decoupled operations to
describe the actual atmospheric distortion accurately. There are two possible
options:

  * Blur-then-tilt: $\mathbf{I}=[\mathcal{T}\circ\mathcal{B}]\mathbf{J}$,

  * Tilt-then-blur: $\mathbf{I}=[\mathcal{B}\circ\mathcal{T}]\mathbf{J}$,

where $\circ$ is a functional composition operator, and the composition is
read from right to left.

Figure 2: Simulation of turbulence using $\mathcal{B}\circ\mathcal{T}$,
$\mathcal{T}\circ\mathcal{B}$, and the full model, along with their respective
error maps with respect to the full model. The error of the blur-then-tilt
model, $\mathcal{T}\circ\mathcal{B}$ occurs at edges and high-resolution
regions.

Many of the previous works [1, 2, 42, 23] opt to use the blur-then-tilt model,
whose inversion is to untilt first and then to deblur. They choose this route
because it is relatively easier to estimate the tilt first using well-known
non-rigid registration computations such as optical flow [2, 22],
diffeomorphism [23], and B-spline function [21]. Then, after untilting, an
off-the-shelf deblurring algorithm is used to deblur [9].

However, Chan [24] showed, by careful analysis, that the blur-then-tilt model
is inaccurate and that the tilt-then-blur model is physically more accurate.
In our work, we use the correct model of tilt-then-blur to represent our
forward model, which is given as follows [24]:

$\mathbf{I}(x,y,t)=[\mathcal{B}\circ\mathcal{T}]\mathbf{J}(x,y,t),$ (2)

where $\mathcal{B}$ denotes a temporally and spatially varying blurring
operator, $\mathcal{T}$ represents a temporally and spatially varying tilting
operator and $\circ$ is a functional composition operator. We apply spatially
and temporally varying tilting operators to the clean image $\mathbf{J}(x,y)$
first to obtain multiple tilted images $\mathbf{I}_{\mathcal{T}}$ at different
time stamps. Then, we apply spatially varying blurring operators to those
tilted images $\mathbf{I}_{\mathcal{T}}$ to render corresponding final tilted-
then-blurred images $\mathbf{I}=\mathbf{I}_{\mathcal{B}\circ\mathcal{T}}$
under atmospheric turbulence.

Figure 3: Qualitative results from the static scene atmospheric turbulence
datasets, including Siemens star dataset [43], text dataset [4], and door
dataset [43]. We compare NeRT with other supervised [4] and unsupervised [9,
1] SOTA. NeRT is able to achieve high spatial resolution, recover high-
contrast text, and reconstruct fine details, such as wire fences. CLEAR [1],
TSR-WGAN [5], and TurbNet [4] fails to mitigate the atmospheric turbulence,
while NDIR [9] fails to preserve the wire fences.

###  3.2 Why choose tilt-then-blur?

Although the two compositions $\mathcal{B}\circ\mathcal{T}$ (tilt-then-blur)
and $\mathcal{T}\circ\mathcal{B}$ (blur-then-tilt) are analytically different,
their impacts on images of natural scenes tend to be similar, and the
differences might be imperceptible [24]. However, the errors in the incorrect
$\mathcal{T}\circ\mathcal{B}$ model can quickly accumulate at the edges and
high-resolution regions, as shown in the analysis below.

For simplicity, let us assume that the blurring is spatially invariant. We can
write the equations of the two models as [24]:

$\displaystyle\mathbf{I}_{\mathcal{T}\circ\mathcal{B}}$
$\displaystyle=\sum_{j=1}^{N}g(\bm{x}_{i}-\bm{u}_{j})\bm{J}(\bm{u}_{j}-\bm{t}_{i}),$
(3) $\displaystyle\mathbf{I}_{\mathcal{B}\circ\mathcal{T}}$
$\displaystyle=\sum_{j=1}^{N}g(\bm{x}_{i}-\bm{u}_{j})\bm{J}(\bm{u}_{j}-\bm{t}_{j}),$
(4)

where $g$ is the spatially invariant blur, and $t$ is the tilt. Note that the
subtlety is captured in the indexing of $t$. More details regarding the
derivation can be found in [24].

We may now evaluate the difference between the correct tilt-then-blur model
and the incorrect blur-then-tilt model as

$\begin{split}\mathbf{I}_{\mathcal{T}\circ\mathcal{B}}&-\mathbf{I}_{\mathcal{B}\circ\mathcal{T}}\\\
&=\sum_{j=1}^{N}g(\bm{x}_{i}-\bm{u}_{j})[\bm{J}(\bm{u}_{j}-\bm{t}_{i})-\bm{J}(\bm{u}_{j}-\bm{t}_{j})]\\\
&\approx\sum_{j=1}^{N}g(\bm{x}_{i}-\bm{u}_{j})\nabla\bm{J}(\bm{u}_{j}^{T})(\bm{t}_{i}-\bm{t}_{j}),\end{split}$
(5)

where $\nabla\bm{J}(\bm{u}_{j}^{T})$ stands for the image gradient, and
$\bm{t}_{i}-\bm{t}_{j}$ represents the random tilt. For natural scene images,
the image gradients $\nabla\bm{J}(\bm{u}_{j}^{T})$ are typically sparse, and
the error between the two models is close to zero for most of the image
regions. However, the image gradients are strong at edges and high-resolution
regions, and there will be a significant error between the two models (Figure
2). Thus, it is sub-optimal to solve atmospheric mitigation problems following
the incorrect blur-then-tilt model. Using the correct tilt-then-blur model
gives us the opportunity to recover edges and high-resolution details from a
time window of dynamically distorted frames.

###  3.3 Network Structure

Figure 1 demonstrates the architecture of NeRT. Our model has three major
components, grid deformers, image generators, and shift-varying blurring.

Grid deformers $\mathcal{G_{\theta}}$ take uniform grid
$\mathbf{G}_{U}\in\mathbb{R}^{2\times m\times n}$ as inputs, where $m$ and $n$
are the image sizes, and output deformed grid,
$\mathbf{G}_{\mathcal{T}}\in\mathbb{R}^{2\times m\times n}$. Like NDIR [9],
each grid deformer $\mathcal{G}_{\theta}$ consists of four convolutional
layers with 256 channels and ReLU activation layers. Similarly, we have a
dedicated grid deformer $\mathcal{G}_{\theta}$ for each distorted image
$\mathbf{I}\in\mathbb{R}^{3\times m\times n}$.

Image generator $\mathcal{I_{\phi}}$ take deformed $2$D pixel coordinates
$\mathbf{G}_{\mathcal{T}}\in\mathbb{R}^{m\times n\times 2}$ as inputs, and
output 3D RGB pixel value that corresponds to the tilted images
$\mathbf{I}_{\mathcal{T}}\in\mathbb{R}^{m\times n\times 3}$. It can also take
uniform $2$D pixel coordinates $\mathbf{G}_{U}\in\mathbb{R}^{m\times n\times
2}$ as inputs and output colored pixel value of the clean image
$\mathbf{J}\in\mathbb{R}^{m\times n\times 3}$. We build our image generator as
an implicit representation consisting of five fully connected layers of hidden
size 256 with ReLU activation and positional encoding. We implement our
coordinate-based MLPs following previous work, SIREN [25] and Fourier feature
network [26]. Specifically, we reshape the 2D pixel coordinates
$\mathbf{G}\in\mathbb{R}^{{m\times n\times 2}}$ as
$\mathbf{G}\in\mathbb{R}^{m\cdot n\times 2}$ to parse into the coordinate-
based MLPs. Additionally, we reshape the output of the coordinate-based MLPs,
corresponding 3D RGB pixel value, $\mathbf{J}\in\mathbb{R}^{m\cdot n\times 3}$
as $\mathbf{J}\in\mathbb{R}^{m\times n\times 3}$.

TABLE II: Quantitative performance on synthetic atmospheric turbulence dataset created using simulator [15]. $\uparrow$ means the higher the better. We highlight the best results using bold text. Strengths | Metrics | CLEAR [1] | TurbNet [4] | TSR-WGAN [5] | NDIR [9] | NeRT (Ours)
---|---|---|---|---|---|---
Weak ($D/r_{0}$ = $1.5$) | PSNR $\uparrow$ (dB) | $20.164$ | $20.532$ | $20.428$ | $21.366$ | $\mathbf{22.109}$
SSIM $\uparrow$ | $0.704$ | $0.601$ | $0.600$ | $0.716$ | $\mathbf{0.766}$
Medium ($D/r_{0}$ = $3$) | PSNR $\uparrow$ (dB) | $19.341$ | $18.220$ | $18.811$ | $19.606$ | $\mathbf{20.576}$
SSIM $\uparrow$ | $0.611$ | $0.440$ | $0.457$ | $0.603$ | $\mathbf{0.659}$
Strong ($D/r_{0}$ = $4.5$) | PSNR $\uparrow$ (dB) | $17.715$ | $17.786$ | $17.198$ | $18.812$ | $\mathbf{19.311}$
SSIM $\uparrow$ | $0.488$ | $0.512$ | $0.347$ | $0.544$ | $\mathbf{0.567}$
Figure 4: Qualitative results from the dynamic scene atmospheric turbulence
dataset, moving car dataset [1]. We compare NeRT with other supervised [4, 5]
and unsupervised [9, 1] SOTA. We are able to recover high-contrast and fine
details of the license plate while other methods show blurry and low-contrast
license plate numbers. NDIR [9], TSR-WGAN [5], and NeRT choose to recover the
clean image based on the newest frame while CLEAR [1] and TurbNet [4] decide
to recover the clean image based on the oldest frame.

Shift-varying blurring $\mathcal{P_{\alpha}}$ take generated tilted images
$\mathbf{I}_{\mathcal{T}}\in\mathbb{R}^{m\times n\times 3}$ as inputs and
output generated tilted-then-blurred images
$\mathbf{I}_{\mathcal{B}\circ\mathcal{T}}\in\mathbb{R}^{m\times n\times 3}$.
Shift-varying blurring leverages the Phase-to-Space (P2S) transform [15] to
apply pixel-wise spatially and temporally varying blurring. We initialize per-
pixel correlated Zernike coefficients
$\bm{\alpha}=[\alpha_{1},\dots,\alpha_{K}]$ by multiplying independent and
identically distributed Gaussian vectors with pre-computed correlation
matrices. In addition, we apply P2S transform network during the optimization
to convert Zernike coefficients $\bm{\alpha}=[\alpha_{1},\dots,\alpha_{K}]$ to
PSF basis coefficients $\bm{\beta}=[\beta_{1},\dots,\beta_{K}]$. Together with
the pre-computed PSF basis, we are able to use the converted basis coefficient
$\bm{\beta}$ to compute spatially and temporally varying PSF for the shift-
varying blurring operation. Note that we don’t consider the first two Zernike
bases since they are already accounted for as tilt by the grid deformers.

###  3.4 Parameter initialization

Figure 5: Qualitative results from the real water turbulence tank datasets
[6]. We compare NeRT with other optimization-based water turbulence mitigation
methods [7, 8] and the state-of-the-art deep learning-based approaches [9, 1].
The qualitative results of NeRT are on par with SOTA performances. We
highlight the best results using bold text. Figure 6: Qualitative results
obtained from uncontrolled environments featuring water ripple reflection
turbulence. We compare NeRT with other water and atmospheric turbulence
mitigation methods. NeRT demonstrates the ability to effectively eliminate
water ripple turbulence from reflections and produce clean images. The
recovered images are shown mirrored for ease of viewing.

Since our method is unsupervised, parameter initialization is rather
important. A good initial point could help our model avoid saddle points and
local minimums during the optimization. First, $D/r_{0}$, which characterizes
the strength of the atmospheric turbulence and typically ranges from $1.0$ to
$5.0$, determines the variance of the i.i.d. Gaussian vector during $\alpha$
initialization. Higher $D/r_{0}$ means stronger atmospheric turbulence. When
the observed images have relatively high turbulence strength, our model
converges better with a higher $D/r_{0}$. Second, $corr$ [15] refers to how
correlated these nearby PSFs are and typically ranges from $-5$ to $-0.01$. A
higher value means a stronger correlation. When the observed images have
relatively high turbulence strength, our model converges better with a higher
$corr$. Third, the kernel size of the PSF basis should vary as the image size
varies. Large image dimensions usually require a large kernel size of the PSF
basis.

###  3.5 Two-step optimization

We follow the network initialization in NDIR [9]. During the first
initialization step, the grid deformers $\mathcal{G}_{\theta}$ are constrained
to learn an identity mapping from uniform grid $\mathbf{G_{U}}$ to be close to
the uniform grid $\mathbf{G_{U}}$. In this way, we can limit the grid
deformation from extreme pixel mixing. The image generator must learn an
average of all the distorted input images. The loss function of the first
initialization step is formulated as

$\min_{\theta,\phi}\sum_{k}\|\mathcal{G}_{\theta}^{k}(\mathbf{G}_{U})\
-\mathbf{G}_{U}\|_{1}+\|\mathcal{I}_{\phi}(\mathbf{G}_{U})-\mathbf{I}_{k}\|_{1}.$
(6)

During the second iterative optimization step, we initialize the $\bm{\alpha}$
in shift-varying blurring, choosing appropriate $D/r_{0}$, $corr$, and PSF
kernel size. The loss function is then formulated as

$\begin{split}\min_{\theta,\phi,\alpha}\sum_{k}\|\mathcal{P}_{\alpha}(\mathcal{I}_{\phi}(\mathcal{G}_{\theta}^{k}(\mathbf{G}_{U})))-\bm{I}_{k}\|_{1}&\\\
+\|\mathcal{P}_{\alpha}(\mathbf{I}_{\mathcal{T}}^{G})-\bm{I}_{k}\|_{1}&\\\
+\|\mathcal{P}_{\alpha}(\mathcal{I}_{\phi}(\mathcal{G}_{\theta}^{k}(\mathbf{G}_{U})))-\mathcal{P}_{\alpha}(\mathbf{I}_{\mathcal{T}}^{G})\|_{1},\end{split}$
(7)

where $\mathbf{I}_{\mathcal{T}}^{G}$ is a resampled tilted image given
deformed grids $\mathbf{G}_{\mathcal{T}}$. We would like to enforce
consistency between predicted tilted-then-blurred images, observed images, and
resampled tilted-then-blurred images.

##  4 Experiments and results

In this section, we compare NeRT with other state-of-the-art supervised,
unsupervised, and optimization-based methods, such as CLEAR [1], TurbNet [4],
TSR-WGAN [5] and NDIR [9] on both real and synthetic atmospheric turbulence
mitigation datasets. CLEAR [1] is an optimization-based multi-frame
restoration method. TurbNet [4] is a deep learning-based supervised single-
frame restoration method. TSR-WGAN [5] is a deep learning-based supervised
multi-frame restoration method. NDIR [9] is a deep learning-based unsupervised
multi-frame restoration method. We show that NeRT exhibits superior
performance in both qualitative and quantitative assessments when compared to
the state-of-the-art unsupervised and supervised approaches.

Furthermore, we compare NeRT with other state-of-the-art optimization-based
and deep learning-based water turbulence removal approaches, such as Tian et
al. [7], Oreifej et al. [8] and NDIR [9]. Tian et al. [7] and Oreifej et al.
[8] are optimization-based restoration methods. NDIR is the state-of-the-art
deep learning-based unsupervised restoration method. We demonstrate the
robustness of NeRT in addressing water turbulence mitigation problems.
Specifically, both the qualitative and quantitative outcomes of NeRT are on
par with the state-of-the-art methods.

###  4.1 Implementation details

We implement our model in Pytorch with one NVIDIA A100 80GB GPU. We use Adam
[44] optimizer with a learning rate of $1\times 10^{-4}$ to update parameters
in grid deformers $\mathcal{G}_{\theta}$, image generator $\mathcal{I}_{\phi}$
and shift-varying blurring $\mathcal{P}_{\alpha}$. We use 1000 epochs for the
first initialization and the second iterative optimization steps,
respectively. We empirically choose $D/r_{0}=5.0$, $corr=-5.0$, and a PSF
kernel size of $11$ for all atmospheric turbulence mitigation experiments. For
all the water turbulence mitigation experiments, we choose $D/r_{0}=0.1$,
$corr=-5.0$, and a PSF kernel size of $5$. We resize all the distorted input
images to $256\times 256$. We randomly choose $20$ distorted images as input
for all the experiments.

###  4.2 Atmospheric turbulence mitigation

####  4.2.1 Evaluation on synthetic atmospheric turbulence datasets

We use the physics-based atmospheric turbulence simulator [15] to create our
synthetic distorted video sequences for evaluation. We choose three different
levels of turbulence strength. We use $D/r_{0}=1.5$ as weak turbulence,
$D/r_{0}=3$ as medium turbulence, and $D/r_{0}=4.5$ as strong turbulence.
Table II demonstrates the quantitative comparison between our method, NeRT,
and other SOTA. NeRT outperforms other methods in terms of peak signal-to-
noise ratio (PSNR) and structural similarity index measure (SSIM) [45] and is
robust to different turbulence strengths.

TABLE III: Quantitative results from the real water turbulence tank datasets [6]. $\uparrow$ means the higher the better. Results obtained using our method, NeRT, are on par with the current state-of-the-art NDIR [9]. We highlight the best and second-best results using bold and underlined text, respectively. Method |  | Tian et al.
---
[7]
| Oreifej et al.
---
[8]
| NDIR
---
[9]
NeRT (Ours)
PSNR $\uparrow$ | $15.653$ | $17.031$ | $\mathbf{17.457}$ | $\underline{17.213}$
SSIM $\uparrow$ | $0.349$ | $0.524$ | $\mathbf{0.649}$ | $\underline{0.645}$

####  4.2.2 Evaluation on real atmospheric turbulence datasets

Static scenes. We include three real datasets for static scenes, the Siemens
star dataset [43], the text dataset [4], and the door dataset [43]. Figure 3
shows the qualitative results of these static scenes. NeRT achieves the best
overall performance in terms of spatial resolution, high-contrast text
reconstruction, and fine details recovery. Further, NeRT is able to preserve
the fine details, such as the wire fences, while suppressing the blurring and
tilting caused by atmospheric turbulence.

Dynamic scenes. We present the moving car dataset [1] for dynamic scenes. The
distorted input image sequences depict a car moving from back to front and
from left to right. Figure 4 presents the qualitative results from the dynamic
scene. Again we compare with other SOTA. NeRT is able to recover a high-
contrast license plate with higher fidelity. To handle the dynamic scene, the
image generator $\mathcal{I}_{\alpha}$ in our unsupervised model converges to
a reference frame as a starting point during the first initialization step.
During the second iterative optimization step, the clean image generated by
the image generator $\mathcal{I}_{\alpha}$ is further optimized, given dozens
of distorted images.

###  4.3 Seeing through water

We evaluate NeRT as well as other state-of-the-art methods on the real water
turbulence dataset [6]. Li et al. [6] utilize a camera, agitating pump, and
tank full of water to capture a real water turbulence tank dataset in their
laboratory. This tank test dataset contains six different kinds of underwater
scenes, including surf shops, dolphins, planes, ships, skulls, and turtles. We
demonstrate the quantitative and qualitative results of NeRT in Table III and
Figure 5, respectively. We observe that the results obtained by NeRT are on
par with the state-of-the-art performance.

##  5 Ablation studies

We perform a series of ablation studies to validate our architecture design.
Throughout the ablation studies, we use the physics-based atmospheric
turbulence simulator [15] to generate synthetic atmospheric turbulence
datasets under different turbulence strengths ($D/r_{0}$).

Number of distorted input images. Table IV demonstrates the quantitative
performance of NeRT with the different number of distorted input images. On
one hand, we can observe that as the number of distorted input images
increases, NeRT’s performance improves gradually. On the other hand, we also
observe the manifestation of the law of diminishing marginal utility.
Specifically, as the quantity of distorted input images rises, NeRT’s
performance improvement rate decreases.

TABLE IV: Average PSNR, SSIM comparison with the different number of distorted input images. The number in the parentheses indicates the performance improvement rates. $\uparrow$ means the higher, the better. | # of
---
inputs
$2$ | $5$ | $10$ | $15$ | $20$
PSNR | $19.410$ |  | $20.076$
---
$(+3.43\%)$
| $20.492$
---
$(+2.07\%)$
| $20.627$
---
$(+0.66\%)$
| $20.712$
---
$(+0.41\%)$
SSIM | $0.601$ |  | $0.639$
---
$(+6.32\%)$
| $0.655$
---
$(+2.50\%)$
| $0.660$
---
$(+0.76\%)$
| $0.664$
---
$(+0.61\%)$
Figure 7: NeRT converges $\sim 48\times$ faster after two-step optimization
(initialization) over multiple datasets: (a) Siemens star dataset [43], (b)
door dataset [43], (c) moving car dataset [1], (d) text dataset [4], and (e)
water tank dataset [6], and (f) water ripple reflection dataset. It takes a
total of $2000$ epochs ($\sim 8$ minutes) to converge during the two-step
optimization stage, while it only takes $60$ epochs ($\sim 10$ seconds) to
converge during anytime reconstruction (red mark).

Consistency between $\mathbf{I}_{\mathcal{B}\circ\mathcal{T}}$,
$\mathbf{I}^{G}_{\mathcal{B}\circ\mathcal{T}}$, and $\mathbf{I}$. we also
conducted thorough ablation studies to analyze the significance of consistency
between predicted tilted-then-blurred images
$\mathbf{I}_{\mathcal{B}\circ\mathcal{T}}$, resampled tilted-then-blurred
images $\mathbf{I}^{G}_{\mathcal{B}\circ\mathcal{T}}$, and observed images
$\mathbf{I}$ in (7)). Table V exhibits the effect of different consistency
losses on the performance of NeRT.

TABLE V: Average PSNR, SSIM comparison with different consistency losses in (7). $Loss_{1}$ stands for the $L_{1}$ loss between predicted tilted-then-blurred images $\mathbf{I}_{\mathcal{B}\circ\mathcal{T}}$ and observed image $\mathbf{I}$. $Loss_{2}$ stands for the $L_{1}$ loss between resampled tilted-then-blurred images $\mathbf{I}^{G}_{\mathcal{B}\circ\mathcal{T}}$ and observed image $\mathbf{I}$. $Loss_{3}$ stands for the $L_{1}$ loss between resampled tilted-then-blurred images $\mathbf{I}^{G}_{\mathcal{B}\circ\mathcal{T}}$ and predicted tilted-then-blurred images $\mathbf{I}_{\mathcal{B}\circ\mathcal{T}}$. Method |  | NeRT
---
w/o $Loss_{1}$
| NeRT
---
w/o $Loss_{2}$
| NeRT
---
w/o $Loss_{3}$
| NeRT
---
(Proposed)
PSNR | 20.075 | 20.035 | 20.350 | 20.712
SSIM | 0.629 | 0.628 | 0.649 | 0.664

##  6 Water ripple turbulence mitigation in the wild

To showcase the versatility of NeRT in addressing a wide range of turbulence
mitigation challenges, we venture into uncontrolled environments and capture a
couple of real-life video recordings featuring water ripple turbulence. These
recordings comprise both clean and undistorted scenes, as well as their
corresponding mirror images that are distorted by water ripple turbulence. We
apply our method to the parts of the video that are distorted by the water
ripples and reconstruct clear images. Figure 6 exhibits some qualitative
results of mitigating the water ripple turbulence in the wild.

##  7 Anytime reconstruction of continuous video frames

One drawback of unsupervised implicit neural representation methods is that
they are iterative and have a slower reconstruction time. The parameters of
the network need to be optimized for every new scene. Here, we show a path of
extending our method towards real-time reconstruction by processing the
current video frame based on network parameters learned from previous frames.
This speeds up the convergence time, leading to what we call the “anytime”
reconstruction.

Our method NeRT, like NDIR [9], has a separate grid deformer
$\mathcal{G_{\theta}}$ and a separate shift-varying blurring
$\mathcal{P_{\alpha}}$ for each distorted input image, and shares a single
image generator $\mathcal{I_{\phi}}$ across all the distorted images. During
anytime reconstruction, we initialize a new separate grid deformer
$\mathcal{G_{\theta}}$ and shift-varying blurring $\mathcal{P_{\alpha}}$ for
the most recent frame that is captured. All the other grid deformers
$\mathcal{G_{\theta}}$, shift-varying blurring $\mathcal{P_{\alpha}}$ and the
image generator $\mathcal{I_{\phi}}$ can contain all the information of the
previously observed distorted frame, speeding up the anytime reconstruction.

Figure 7 demonstrates $48\times$ speedup of our method during anytime
convergence. It takes about $8$ minutes to complete the two-step
initialization step for the first set of frames. However, converging for every
newly captured video frame only takes about $10$ seconds.

##  8 Conclusions and discussions

We have proposed the first unsupervised and physically grounded model for
general turbulence mitigation. Given multiple observed distorted images, our
model leveraged the physically correct tilt-then-blur model to reconstruct a
clean and undistorted image. Our model could generalize and outperform other
SOTA in various scenarios, such as seeing through water, seeing through
turbulence, and mitigating water reflection turbulence. After the two-step
initialization, our method converged $48\times$ faster on the latest captured
frame.

Limitations and future directions. Our shift-varying deblurring did not have
any regularization. Thus, the reconstructed clean image inevitably consisted
of some noise due to blind deconvolution. A more sophisticated shift-varying
deblurring process remains a future research direction. Additionally, one
might also leverage the implicit neural network for image superresolution. As
we know, the implicit neural network is a continuous representation of the
image. More pixel coordinates queried into the implicit image function lead to
higher resolution images generated.

_Note: A much smaller version of this paper was submitted for CVPR Workshop.
We have uploaded that version as supplementary for full disclosure._

## References

  * [1] N. Anantrasirichai, A. Achim, and D. Bull, “Atmospheric turbulence mitigation for sequences with moving objects using recursive image fusion,” in _2018 25th IEEE international conference on image processing (ICIP)_. IEEE, 2018, pp. 2895–2899.
  * [2] Z. Mao, N. Chimitt, and S. H. Chan, “Image reconstruction of static and dynamic scenes through anisoplanatic turbulence,” _IEEE Transactions on Computational Imaging_ , vol. 6, pp. 1415–1428, 2020.
  * [3] B. Y. Feng, M. Xie, and C. A. Metzler, “Turbugan: An adversarial learning approach to spatially-varying multiframe blind deconvolution with applications to imaging through turbulence,” _IEEE Journal on Selected Areas in Information Theory_ , 2023.
  * [4] Z. Mao, A. Jaiswal, Z. Wang, and S. H. Chan, “Single frame atmospheric turbulence mitigation: A benchmark study and a new physics-inspired transformer model,” in _Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIX_. Springer, 2022, pp. 430–446.
  * [5] D. Jin, Y. Chen, Y. Lu, J. Chen, P. Wang, Z. Liu, S. Guo, and X. Bai, “Neutralizing the impact of atmospheric turbulence on complex scene imaging via deep learning,” _Nature Machine Intelligence_ , vol. 3, no. 10, pp. 876–884, 2021.
  * [6] Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi, and M. Chandraker, “Learning to see through turbulent water,” in _2018 IEEE Winter Conference on Applications of Computer Vision (WACV)_. IEEE, 2018, pp. 512–520.
  * [7] Y. Tian and S. G. Narasimhan, “Seeing through water: Image restoration using model-based tracking,” in _2009 IEEE 12th International Conference on Computer Vision_. Ieee, 2009, pp. 2303–2310.
  * [8] O. Oreifej, G. Shu, T. Pace, and M. Shah, “A two-stage reconstruction approach for seeing through water,” in _CVPR 2011_. IEEE, 2011, pp. 1153–1160.
  * [9] N. Li, S. Thapa, C. Whyte, A. W. Reed, S. Jayasuriya, and J. Ye, “Unsupervised non-rigid image distortion removal via grid deformation,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 2522–2532.
  * [10] M. C. Roggemann, B. M. Welsh, and B. R. Hunt, _Imaging through turbulence_. CRC press, 1996.
  * [11] X. Zhang, Z. Mao, N. Chimitt, and S. H. Chan, “Imaging through the atmosphere using turbulence mitigation transformer,” _arXiv preprint arXiv:2207.06465_ , 2022.
  * [12] J. Gao, N. Anantrasirichai, and D. Bull, “Atmospheric turbulence removal using convolutional neural network,” _arXiv preprint arXiv:1912.11350_ , 2019.
  * [13] R. Nieuwenhuizen and K. Schutte, “Deep learning for software-based turbulence mitigation in long-range imaging,” in _Artificial Intelligence and Machine Learning in Defense Applications_ , vol. 11169. SPIE, 2019, pp. 153–162.
  * [14] N. Chimitt and S. H. Chan, “Simulating anisoplanatic turbulence by sampling intermodal and spatially correlated zernike coefficients,” _Optical Engineering_ , vol. 59, no. 8, pp. 083 101–083 101, 2020.
  * [15] Z. Mao, N. Chimitt, and S. H. Chan, “Accelerating atmospheric turbulence simulation via learned phase-to-space transform,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 14 759–14 768.
  * [16] N. Chimitt, X. Zhang, Z. Mao, and S. H. Chan, “Real-time dense field phase-to-space simulation of imaging through atmospheric turbulence,” _IEEE Transactions on Computational Imaging_ , 2022.
  * [17] R. C. Hardie, J. D. Power, D. A. LeMaster, D. R. Droege, S. Gladysz, and S. Bose-Pillai, “Simulation of anisoplanatic imaging through optical turbulence using numerical wave propagation with new validation analysis,” _Optical Engineering_ , vol. 56, no. 7, pp. 071 502–071 502, 2017.
  * [18] A. Torralba and A. A. Efros, “Unbiased look at dataset bias,” in _CVPR 2011_. IEEE, 2011, pp. 1521–1528.
  * [19] E. Cole, Q. Meng, J. Pauly, and S. Vasanawala, “Learned compression of high dimensional image datasets,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 1748–1752.
  * [20] T. Zhang, T. Feng, S. Alam, S. Lee, M. Zhang, S. S. Narayanan, and S. Avestimehr, “Fedaudio: A federated learning benchmark for audio tasks,” _arXiv preprint arXiv:2210.15707_ , 2022.
  * [21] M. Shimizu, S. Yoshimura, M. Tanaka, and M. Okutomi, “Super-resolution from image sequence under influence of hot-air optical turbulence,” in _2008 IEEE Conference on Computer Vision and Pattern Recognition_. IEEE, 2008, pp. 1–8.
  * [22] T. Caliskan and N. Arica, “Atmospheric turbulence mitigation using optical flow,” in _2014 22nd International Conference on Pattern Recognition_. Ieee, 2014, pp. 883–888.
  * [23] J. Gilles, T. Dagobert, and C. De Franchis, “Atmospheric turbulence restoration by diffeomorphic image registration and blind deconvolution,” in _Advanced Concepts for Intelligent Vision Systems: 10th International Conference, ACIVS 2008, Juan-les-Pins, France, October 20-24, 2008. Proceedings 10_. Springer, 2008, pp. 400–409.
  * [24] S. H. Chan, “Tilt-then-blur or blur-then-tilt? clarifying the atmospheric turbulence model,” _IEEE Signal Processing Letters_ , vol. 29, pp. 1833–1837, 2022.
  * [25] V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein, “Implicit neural representations with periodic activation functions,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 7462–7473, 2020.
  * [26] M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng, “Fourier features let networks learn high frequency functions in low dimensional domains,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 7537–7547, 2020.
  * [27] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” _Communications of the ACM_ , vol. 65, no. 1, pp. 99–106, 2021\.
  * [28] R. Martin-Brualla, N. Radwan, M. S. Sajjadi, J. T. Barron, A. Dosovitskiy, and D. Duckworth, “Nerf in the wild: Neural radiance fields for unconstrained photo collections,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 7210–7219.
  * [29] Y. Chen, S. Liu, and X. Wang, “Learning continuous image representation with local implicit image function,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 8628–8638.
  * [30] L. Shen, J. Pauly, and L. Xing, “Nerp: implicit neural representation learning with prior embedding for sparsely sampled image reconstruction,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2022.
  * [31] J. M. Wolterink, J. C. Zwienenberg, and C. Brune, “Implicit neural representations for deformable image registration,” in _International Conference on Medical Imaging with Deep Learning_. PMLR, 2022, pp. 1349–1359.
  * [32] S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, M.-H. Yang, and L. Shao, “Multi-stage progressive image restoration,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 14 821–14 831.
  * [33] S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, and M.-H. Yang, “Restormer: Efficient transformer for high-resolution image restoration,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 5728–5739.
  * [34] D. Park, D. U. Kang, J. Kim, and S. Y. Chun, “Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training,” in _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16_. Springer, 2020, pp. 327–343.
  * [35] R. Yasarla and V. M. Patel, “Learning to restore images degraded by atmospheric turbulence using uncertainty,” in _2021 IEEE International Conference on Image Processing (ICIP)_. IEEE, 2021, pp. 1694–1698.
  * [36] K. Mei and V. M. Patel, “Ltt-gan: Looking through turbulence by inverting gans,” _IEEE Journal of Selected Topics in Signal Processing_ , 2023.
  * [37] N. G. Nair, K. Mei, and V. M. Patel, “At-ddpm: Restoring faces degraded by atmospheric turbulence using denoising diffusion probabilistic models,” in _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , 2023, pp. 3434–3443.
  * [38] K. C. Chan, S. Zhou, X. Xu, and C. C. Loy, “Basicvsr++: Improving video super-resolution with enhanced propagation and alignment,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2022, pp. 5972–5981.
  * [39] J. Liang, J. Cao, Y. Fan, K. Zhang, R. Ranjan, Y. Li, R. Timofte, and L. Van Gool, “Vrt: A video restoration transformer,” _arXiv preprint arXiv:2201.12288_ , 2022.
  * [40] O. Oreifej, X. Li, and M. Shah, “Simultaneous video stabilization and moving object detection in turbulence,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 35, no. 2, pp. 450–462, 2012.
  * [41] L. Xu, S. Zheng, and J. Jia, “Unnatural l0 sparse representation for natural image deblurring,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2013, pp. 1107–1114.
  * [42] X. Zhu and P. Milanfar, “Removing atmospheric turbulence via space-invariant deconvolution,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 35, no. 1, pp. 157–170, 2012.
  * [43] J. Gilles and N. B. Ferrante, “Open turbulent image set (otis),” _Pattern Recognition Letters_ , vol. 86, pp. 38–41, 2017.
  * [44] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [45] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” _IEEE transactions on image processing_ , vol. 13, no. 4, pp. 600–612, 2004.


# Electroweak Monopole-antimonopole Pair in the Standard Model

Dan Zhu  School of Physics, Universiti Sains Malaysia, 11800 USM, Penang,
Malaysia  Khai-Ming Wong and Guo-Quan Wong  School of Physics, Universiti
Sains Malaysia, 11800 USM, Penang, Malaysia

###### Abstract

We present numerical solutions corresponding to a pair of Cho-Maison monopole
and antimonopole (MAP) in the SU(2)$\times$U(1) Weinberg-Salam (WS) theory.
Each pole carries magnetic charge $\pm 4\pi/e$. This configuration is
important because if Cho-Maison monopoles were detected in the lab, it would
be through pair production. The Cho-Maison MAP was investigated for a range of
Weinberg angle, $0.4675\leq\tan\theta_{\scalebox{0.5}{\mbox{W}}}\leq 10$ and
Higgs self-coupling, $0\leq\beta\leq 1.7704$. Magnetic dipole moment
($\mu_{m}$) and pole separation ($d_{z}$) of the numerical solutions are
calculated and analyzed. Total energy of the system, however, is infinite due
to point singularities at the location of monopoles.

##  1 Introduction

Magnetic monopoles have remained a topic of extensive studies ever since
P.A.M. Dirac [1] introduced the idea into Maxwell’s theory. It was generalized
to non-Abelian gauge theories in 1968 by Wu and Yang [2]. However, both Dirac
and Wu-Yang monopoles possess infinite energy due to the presence of a point
singularity at the origin. The first finite energy magnetic monopole solution
is the ’t Hooft-Polyakov monopole [3] in the SU(2) Yang-Mills-Higgs (YMH)
theory, found independently by G. ’t Hooft and A.M. Polyakov in 1974. The mass
of their monopole was estimated to be around 137
$m_{\scalebox{0.5}{\mbox{W}}}$, where $m_{\scalebox{0.5}{\mbox{W}}}$ is the
mass of intermediate vector boson.

Since then, numerous solutions were found in the SU(2) YMH theory. In the BPS
limit of vanishing Higgs potential, there exist exact monopole [4] and
multimonopole [5] solutions. Other well-known cases are the axially symmetric
MAP of Kleihaus and Kunz [6], and the monopole-antimonopole chain (MAC) of
Kleihaus et al. [7]. These solutions possess finite energy and represent a
chain of magnetic monopoles and antimonopoles lying in alternating order along
the symmetrical axis.

Magnetic monopole research started a new chapter when the Cho-Maison monopole
[8] was reported in the Standard Model, which is an electroweak generalization
of the Dirac monopole. It acquires the W-boson dressing and becomes a hybrid
between the Dirac monopole and ’t Hooft-Polyakov monopole. Moreover, its
magnetic charge is twice as large because the period of the electromagnetic
U(1) is $4\pi$ in the SU(2)$\times$U(1) Weinberg-Salam (WS) theory.

The importance of Cho-Maison monopole comes from the following fact. As the
electroweak generalization of the Dirac monopole, it must exist if the
Standard Model is correct [8, 9, 10]. One might argue that the standard WS
theory cannot accommodate magnetic monopoles because the second homotopy of
quotient space, SU(2)$\times$U(1)Y/U(1)${}_{\scalebox{0.5}{\mbox{em}}}$, is
trivial. However, it is not the only monopole topology. In fact, the WS
theory, with hypercharge U(1), can be viewed as a gauged CP1 model in which
the normalized Higgs doublet plays the role of the CP1 field [8]. This way,
the SU(2) part of WS theory has exactly the same monopole topology as the
Georgi-Glashow model. That is, $\pi_{2}\left(S^{2}\right)=\mathbb{Z}$.
Originally, the electroweak monopole and dyon solutions were obtained by
numerical integration, but a mathematically rigorous proof of existence was
established later on [9].

The mass of Cho-Maison monopole is infinite due to the point singularity at
the origin. However, it is premature to deny its existence simply because it
has infinite energy at classical level. A priori, there is nothing wrong with
this. Classically, the electron has an infinite electric energy but a finite
mass [10]. Additionally, it has been shown that the solution can be
regularized. Ref. [10] estimated the mass of a Cho-Maison monopole to be
around 4 to 10 TeV. More recently, different methods were used to regularize
the solution, it is reported that the new BPS bound for Cho-Maison monopole
may not be smaller than 2.98 TeV, more probably 3.75 TeV [11]. Another
estimate puts the lower bound of the mass of Cho-Maison monopole at 2.37 TeV
[12].

These predictions of mass strongly indicate that it could actually be produced
at LHC in the near future. So, when discovered, it will become the first
magnetically charged topological elementary particle in the history of
physics. Second, it could have deep implications in cosmology. This is because
it could induce the density perturbation in the early universe and generate
primordial magnetic black holes which could explain the dark matter. As
importantly, it could become the seed of the large scale structures of the
universe and become the source of the intergalactic magnetic field [13]. For
these reasons MoEDAL and ATLAS at LHC are actively searching for the monopole
[14, 15].

If magnetic monopoles were detected in the lab, it would be through pair
production, and therefore, the importance of studying a pair of monopole and
antimonopole (MAP) is self-evident. In 1977, Y. Nambu [16] theoretically
predicted the existence of a pair of magnetic monopole and antimonopole
bounded by a Z0 flux string. Monopoles in an Nambu MAP carry magnetic charge
$\pm 4\pi\sin^{2}\theta_{\scalebox{0.5}{\mbox{W}}}/e$. The existence of Nambu
MAPs was confirmed numerically by Teh et al. [17] using an axially symmetic
magnetic ansatz. They also confirmed that Nambu MAPs are actually electroweak
sphalerons reported by Kleihaus et al. [18].

It is interesting to know whether an axially symmetric magnetic ansatz can
produce a finitely separated Cho-Maison MAP that does not have a Z0 flux
string connecting the poles. In this work, we demonstrate that it is possible.
Magnetic charge carried by each pole of the MAP solutions found in this study
is $\pm 4\pi/e$, confirming that they are indeed Cho-Maison monopoles.
Gervalle and Volkov [19] studied Cho-Maison multimonopole solutions, which are
also axially symmetric, but it is important to note that multimonopoles and
MAPs are fundamentally different.

The Cho-Maison MAP solutions were investigated at physical Weinberg angle,
$\tan\theta_{\scalebox{0.5}{\mbox{W}}}=0.53557042$, while the Higgs self-
coupling constant, $\beta$, runs from 0 to 1.7704 and at physical
$\beta=0.77818833$, while $\tan\theta_{\scalebox{0.5}{\mbox{W}}}$ is allowed
to vary ($0.4675\leq\tan\theta_{\scalebox{0.5}{\mbox{W}}}\leq 10$). The
investigated quantities include magnetic dipole moment ($\mu_{m}$) and pole
separation ($d_{z}$). Total energy of the configuration is infinite due to
point singularities at the location of monopoles.

##  2 MAP Ansatz in Standard Model

The Lagrangian of the bosonic sector of SU(2)$\times$U(1) WS theory is given
by

$\displaystyle\mathcal{L}=$
$\displaystyle-\frac{1}{4}F^{a}_{\mu\nu}F^{a\mu\nu}-\frac{1}{4}G_{\mu\nu}G^{\mu\nu}$
$\displaystyle-\left(\mathcal{D}_{\mu}\boldsymbol{\phi}\right)^{\dagger}\left(\mathcal{D}^{\mu}\boldsymbol{\phi}\right)-\frac{\lambda}{2}\left(\boldsymbol{\phi}^{\dagger}\boldsymbol{\phi}-\frac{\mu_{\scalebox{0.5}{\mbox{H}}}^{2}}{\lambda}\right)^{2}.$
(1)

Here, $\mathcal{D}_{\mu}$ is the covariant derivative of SU(2)$\times$U(1)
group and is defined as

$\mathcal{D}_{\mu}=D_{\mu}-\frac{i}{2}g^{\prime}B_{\mu}=\partial_{\mu}-\frac{i}{2}gA^{a}_{\mu}\sigma_{a}-\frac{i}{2}g^{\prime}B_{\mu},$
(2)

where $D_{\mu}$ is the covariant derivative of SU(2) group only.

The SU(2) gauge coupling constant, potential and electromagnetic tensor are
$g$, $A^{a}_{\mu}$ and $F^{a}_{\mu\nu}$. Their counterparts in U(1) gauge
field are denoted as $g^{\prime}$, $B_{\mu}$ and $G_{\mu\nu}$. The term,
$\sigma_{a}$, is the Pauli matrices, $\boldsymbol{\phi}$ and $\lambda$ are the
complex scalar Higgs doublet and Higgs field self-coupling constant. Higgs
boson mass and $\mu_{\scalebox{0.5}{\mbox{H}}}$ are related through
$m_{\scalebox{0.5}{\mbox{H}}}=\sqrt{2}\mu_{\scalebox{0.5}{\mbox{H}}}$. In
addition, the Higgs field can be expressed as
$\boldsymbol{\phi}=H\boldsymbol{\xi}/\sqrt{2}$, where $\boldsymbol{\xi}$ is a
column 2-vector that satisfies $\boldsymbol{\xi}^{\dagger}\boldsymbol{\xi}=1$.
Metric used in this paper is ($-$+++).

Through Lagrangian (1), three equations of motion can be obtained as the
following,

$\displaystyle\mathcal{D}^{\mu}\mathcal{D}_{\mu}\boldsymbol{\phi}$
$\displaystyle=\lambda\left(\boldsymbol{\phi}^{\dagger}\boldsymbol{\phi}-\frac{\mu_{\scalebox{0.5}{\mbox{H}}}^{2}}{\lambda}\right)\boldsymbol{\phi},$
(3) $\displaystyle D^{\mu}F^{a}_{\mu\nu}$
$\displaystyle=\frac{ig}{2}\left[\boldsymbol{\phi}^{\dagger}\sigma^{a}\left(\mathcal{D}_{\nu}\boldsymbol{\phi}\right)-\left(\mathcal{D}_{\nu}\boldsymbol{\phi}\right)^{\dagger}\sigma^{a}\boldsymbol{\phi}\right],$
(4) $\displaystyle\partial^{\mu}G_{\mu\nu}$
$\displaystyle=\frac{ig^{\prime}}{2}\left[\boldsymbol{\phi}^{\dagger}\left(\mathcal{D}_{\nu}\boldsymbol{\phi}\right)-\left(\mathcal{D}_{\nu}\boldsymbol{\phi}\right)^{\dagger}\boldsymbol{\phi}\right].$
(5)

The magnetic ansatz used to obtain the Cho-Maison MAP is:

$\displaystyle gA^{a}_{i}=$
$\displaystyle-\frac{1}{r}\psi_{1}(r,\theta)\hat{n}^{a}_{\phi}\hat{\theta}_{i}+\frac{n}{r}\psi_{2}(r,\theta)\hat{n}^{a}_{\theta}\hat{\phi}_{i}$
$\displaystyle+\frac{1}{r}R_{1}(r,\theta)\hat{n}^{a}_{\phi}\hat{r}_{i}-\frac{n}{r}R_{2}(r,\theta)\hat{n}^{a}_{r}\hat{\phi}_{i},$
$\displaystyle g^{\prime}B_{i}=$ $\displaystyle\
\frac{n}{r\sin\theta}B_{s}(r,\theta)\hat{\phi}_{i},\;gA^{a}_{0}=g^{\prime}B_{0}=0,$
$\displaystyle\Phi^{a}=$ $\displaystyle\
\Phi_{1}(r,\theta)\hat{n}^{a}_{r}+\Phi_{2}(r,\theta)\hat{n}^{a}_{\theta}=H(r,\theta)\hat{\Phi}^{a}.$
(6)

In addition, the Higgs unit vector, $\hat{\Phi}^{a}$, can be written as,

$\displaystyle\hat{\Phi}^{a}=$
$\displaystyle-\boldsymbol{\xi}^{\dagger}\sigma^{a}\boldsymbol{\xi}$
$\displaystyle=$
$\displaystyle\cos(\alpha-\theta)\hat{n}^{a}_{r}+\sin(\alpha-\theta)\hat{n}^{a}_{\theta}=h_{1}\hat{n}^{a}_{r}+h_{2}\hat{n}^{a}_{\theta},$
$\displaystyle\boldsymbol{\xi}=$ $\displaystyle\
i\begin{pmatrix}\sin\frac{\alpha(r,\theta)}{2}e^{-in\phi}\\\
-\cos\frac{\alpha(r,\theta)}{2}\end{pmatrix}.$ (7)

The functions, $\cos\alpha$ and $\sin\alpha$ are defined as

$\displaystyle\cos\alpha=$
$\displaystyle\frac{\Phi_{1}\cos\theta-\Phi_{2}\sin\theta}{\sqrt{\Phi_{1}^{2}+\Phi_{2}^{2}}}=h_{1}\cos\theta-
h_{2}\sin\theta,$ $\displaystyle\sin\alpha=$
$\displaystyle\frac{\Phi_{1}\sin\theta+\Phi_{2}\cos\theta}{\sqrt{\Phi_{1}^{2}+\Phi_{2}^{2}}}=h_{1}\sin\theta+h_{2}\cos\theta.$
(8)

Moreover, the angle, $\alpha(r,\theta)\rightarrow p\theta$ asymptotically
[17], where $p$ is the parameter controlling the number of poles in the
solution and is set to two. Finally,
$H(r,\theta)=\mathinner{\\!\left\lvert\Phi\right\rvert}=\sqrt{\Phi_{1}^{2}+\Phi_{2}^{2}}$
is the Higgs modulus.

In the magnetic ansatz, the spatial spherical coordinate unit vectors are
defined as

$\displaystyle\hat{r}_{i}$
$\displaystyle=\sin\theta\cos\phi\,\delta_{i1}+\sin\theta\sin\phi\,\delta_{i2}+\cos\theta\,\delta_{i3},$
$\displaystyle\hat{\theta}_{i}$
$\displaystyle=\cos\theta\cos\phi\,\delta_{i1}+\cos\theta\sin\phi\,\delta_{i2}-\sin\theta\,\delta_{i3},$
$\displaystyle\hat{\phi}_{i}$
$\displaystyle=-\sin\phi\,\delta_{i1}+\cos\phi\,\delta_{i2}.$ (9)

Similarly, the unit vectors for isospin coordinate system are given by

$\displaystyle\hat{n}^{a}_{r}$ $\displaystyle=\sin\theta\cos
n\phi\,\delta^{a}_{1}+\sin\theta\sin
n\phi\,\delta^{a}_{2}+\cos\theta\,\delta^{a}_{3},$
$\displaystyle\hat{n}^{a}_{\theta}$ $\displaystyle=\cos\theta\cos
n\phi\,\delta^{a}_{1}+\cos\theta\sin
n\phi\,\delta^{a}_{2}-\sin\theta\,\delta^{a}_{3},$
$\displaystyle\hat{n}^{a}_{\phi}$ $\displaystyle=-\sin
n\phi\,\delta^{a}_{1}+\cos n\phi\,\delta^{a}_{2},$ (10)

where $n$ is the $\phi$-winding number and is set to one in this research.

Upon substituting the magnetic ansatz (6) into the equations of motion (3) -
(5), the system of equations was reduced to seven coupled second-order partial
differential equations, which were further simplified with the following
substitutions,

$x=m_{\scalebox{0.5}{\mbox{W}}}r,\;\widetilde{H}=\frac{H}{H_{0}},\;\tan\theta_{\scalebox{0.5}{\mbox{W}}}=\frac{g^{\prime}}{g},\;\beta^{2}=\frac{\lambda}{g^{2}}.$
(11)

Here $H_{0}=\sqrt{2}\mu_{\scalebox{0.5}{\mbox{H}}}/\sqrt{\lambda}$ and
$m_{\scalebox{0.5}{\mbox{W}}}=gH_{0}/2$. The new radial coordinate, $x$, is
dimensionless, which is then compactified through $\widetilde{x}=x/(x+1)$. The
rescaled Higgs field, $\widetilde{H}$, approaches one asymptotically. Only two
free parameters were left after the transformation,
$\tan\theta_{\scalebox{0.5}{\mbox{W}}}$ and $\beta$. Both of these parameters
can be expressed in terms of the mass of elementary particles and by adopting
$m_{\scalebox{0.5}{\mbox{H}}}=125.10$ GeV,
$m_{\scalebox{0.5}{\mbox{W}}}=80.379$ GeV and
$m_{\scalebox{0.5}{\mbox{Z}}}=91.1876$ GeV[20], these parameters are
calculated to be $\beta=0.77818833$ and
$\tan\theta_{\scalebox{0.5}{\mbox{W}}}=0.53557042$.

The seven coupled equations are then subject to the following boundary
conditions. Along the positive and negative $z$-axis, when $\theta=0$ and
$\pi$,

$\displaystyle\partial_{\theta}\psi_{A}=R_{A}=\partial_{\theta}\Phi_{1}=\Phi_{2}=\partial_{\theta}B_{s}=0,$
(12)

where $A=1$, 2. Asymptotically, when $r$ approaches infinity,

$\displaystyle\psi_{A}(\infty,\theta)$
$\displaystyle=2,R_{A}(\infty,\theta)=B_{s}(\infty,\theta)=0,$
$\displaystyle\Phi_{1}(\infty,\theta)$
$\displaystyle=\cos\theta,\Phi_{2}(\infty,\theta)=\sin\theta,$ (13)

and finally, at the origin,

$\displaystyle\psi_{A}(0,\theta)=R_{A}(0,\theta)=0,B_{s}(0,\theta)=-2,$
$\displaystyle\Phi_{1}(0,\theta)\sin\theta+\Phi_{2}(0,\theta)\cos\theta=0,$
$\displaystyle\partial_{r}(\Phi_{1}(r,\theta)\cos\theta-\Phi_{2}(r,\theta)\sin\theta)|_{r=0}=0.$
(14)

Using finite difference method (central difference approximation), the set of
seven coupled partial differential equations was converted into a system of
non-linear equations, which was then discretized onto a non-equidistant grid
of $M\times N$, where $M=70$, $N=60$. The associated error with the numerical
method employed is $\mathcal{O}\left(1/M^{2}\right)$ in $r$ direction and
$\mathcal{O}\left(\pi^{2}/N^{2}\right)$ in $\theta$ direction. The region of
integration covers all space which translates to $0\leq\widetilde{x}\leq 1$
and $0\leq\theta\leq\pi$. Good initial guesses are needed in order for the
numerical computation to converge.

##  3 Properties of Cho-Maison MAP

In the SU(2)$\times$U(1) WS model, energy density of Cho-Maison MAP solutions
is obtained from the energy-momentum tensor, $T_{\mu\nu}$:

$\displaystyle\varepsilon=T_{00}$
$\displaystyle=\frac{1}{4}G_{ij}G_{ij}+\frac{1}{4}F^{a}_{ij}F^{a}_{ij}+\left(\mathcal{D}_{i}\boldsymbol{\phi}\right)^{\dagger}\left(\mathcal{D}_{i}\boldsymbol{\phi}\right)$
$\displaystyle\quad\,+\frac{\lambda}{2}\left(\boldsymbol{\phi}^{\dagger}\boldsymbol{\phi}-\frac{\mu_{\scalebox{0.5}{\mbox{H}}}^{2}}{\lambda}\right)^{2}.$
(15)

The curve of weighted energy density
($\varepsilon_{\scalebox{0.5}{\mbox{W}}}=r^{2}\sin\theta\cdot\varepsilon$)
near the $z$-axis was plotted to show that its value blows up and the
singularities appear at the location of the monopoles.

To investigate the magnetic properties, the following gauge was applied:

$\displaystyle G$
$\displaystyle=-i\begin{pmatrix}\cos\frac{\alpha}{2}&\sin\frac{\alpha}{2}e^{-in\phi}\\\
\sin\frac{\alpha}{2}e^{in\phi}&-\cos\frac{\alpha}{2}\end{pmatrix}$
$\displaystyle=\cos\frac{-\pi}{2}+i\hat{u}^{a}_{r}\sigma^{a}\sin\frac{-\pi}{2},$
$\displaystyle\hat{u}^{a}_{r}$
$\displaystyle=\sin\frac{\alpha}{2}\cos{n\phi}\,\delta^{a}_{1}+\sin\frac{\alpha}{2}\sin{n\phi}\,\delta^{a}_{2}+\cos\frac{\alpha}{2}\,\delta^{a}_{3}.$
(16)

The transformed gauge potential has the following form:

$\displaystyle gA^{\prime a}_{i}=$
$\displaystyle-\frac{2n}{r}\left[\psi_{2}\sin\left(\theta-\frac{\alpha}{2}\right)+R_{2}\cos\left(\theta-\frac{\alpha}{2}\right)\right]\hat{u}^{a}_{r}\hat{\phi}_{i}$
$\displaystyle-
gA^{a}_{i}+\frac{2n\sin\frac{\alpha}{2}}{r\sin\theta}\hat{u}_{\theta}^{a}\hat{\phi}_{i}-\partial_{i}\alpha\;\hat{u}_{\phi}^{a}.$
(17)

Note here, when $a=3$, equation (17) becomes the ’t Hooft gauge potential,

$\displaystyle gA^{{}^{\prime}3}_{i}$
$\displaystyle=\frac{n}{r}\left(\psi_{2}h_{2}-R_{2}h_{1}-\frac{1-\cos\alpha}{\sin\theta}\right)\hat{\phi}_{i}$
$\displaystyle=\frac{A_{s}}{r\sin\theta}\hat{\phi}_{i}.$ (18)

Now, as

$\displaystyle\begin{pmatrix}A_{i}^{\scalebox{0.5}{\mbox{em}}}\\\
Z_{i}\end{pmatrix}$
$\displaystyle=\begin{pmatrix}\cos\theta_{\scalebox{0.5}{\mbox{W}}}&\sin\theta_{\scalebox{0.5}{\mbox{W}}}\\\
-\sin\theta_{\scalebox{0.5}{\mbox{W}}}&\cos\theta_{\scalebox{0.5}{\mbox{W}}}\end{pmatrix}\begin{pmatrix}B_{i}\\\
A^{\prime 3}_{i}\end{pmatrix}$ $\displaystyle=\frac{1}{\sqrt{g^{\prime
2}+g^{2}}}\begin{pmatrix}g&g^{\prime}\\\
-g^{\prime}&g\end{pmatrix}\begin{pmatrix}B_{i}\\\ A^{\prime
3}_{i}\end{pmatrix},$ (19)

the real electromagnetic potential could be expressed as

$\displaystyle eA_{i}^{\scalebox{0.5}{\mbox{em}}}$
$\displaystyle=\cos^{2}\theta_{\scalebox{0.5}{\mbox{W}}}\left(g^{\prime}B_{i}\right)+\sin^{2}\theta_{\scalebox{0.5}{\mbox{W}}}\left(gA^{\prime
3}_{i}\right),$ (20)

where $e$ is the unit electric charge. The ‘em’ magnetic field could then be
calculated according to the mixing shown in equation (20),

$\displaystyle B^{\scalebox{0.5}{\mbox{em}}}_{i}$
$\displaystyle=-\frac{1}{2}\varepsilon_{ijk}F^{\scalebox{0.5}{\mbox{em}}}_{jk}$
$\displaystyle=-\frac{1}{e}\varepsilon_{ijk}\partial_{j}\\{\cos^{2}\theta_{\scalebox{0.5}{\mbox{W}}}B_{s}+\sin^{2}\theta_{\scalebox{0.5}{\mbox{W}}}A_{s}\\}\partial_{k}\phi.$
(21)

Figure 1: 3D Higgs modulus plots of (a) an MAP found in the SU(2) YMH theory,
(b) a Nambu MAP (electroweak sphaleron), and (c) a Cho-Maison MAP. All with
$\beta=0.77818833$. For (b) and (c),
$\tan\theta_{\scalebox{0.5}{\mbox{W}}}=0.53557042$. Figure 2: The curve of
$\Phi_{1}\left(x,\pi\right)$ versus $x$ for the Cho-Maison MAP shown in Fig.
1(c). Figure 3: Plot of $\varepsilon_{\scalebox{0.5}{\mbox{W}}}$ versus $z$ at
$\rho=0.1077$ for a Cho-Maison MAP with $\beta=0.77818833$ and
$\tan\theta_{\scalebox{0.5}{\mbox{W}}}=0.53557042$. The values of
$\varepsilon_{\scalebox{0.5}{\mbox{W}}}$ clearly indicate that there are two
point singularities at the location of monopoles.

Through Gauss’s law, the magnetic charge enclosed in a Gaussian surface, $S$,
with surface element, $dS^{i}$, can be obtained with the following integral:

$Q_{\scalebox{0.5}{\mbox{M}}(S)}=\oiint_{\scalebox{0.5}{\mbox{S}}}B_{i}^{\scalebox{0.5}{\mbox{em}}}dS^{i}=\iiint_{\scalebox{0.5}{\mbox{V}}}\partial^{i}B_{i}^{\scalebox{0.5}{\mbox{em}}}dV,$
(22)

and for the magnetic charge enclosed in the upper hemisphere, the following
Gaussian surface was defined:

$\displaystyle S_{+}=H^{2}_{+}\cup D^{2}_{\scalebox{0.5}{\mbox{xy}}},$ (23)

where $H^{2}_{+}$ is a half sphere above the $xy$-plane and
$D^{2}_{\scalebox{0.5}{\mbox{xy}}}$ denotes a disk in the $xy$-plane centered
at the origin, both $H^{2}_{+}$ and $D^{2}_{\scalebox{0.5}{\mbox{xy}}}$ have
the same radius. Taking into account the correct orientation of the surface
elements, the magnetic charge enclosed can be calculated as:

$\displaystyle Q_{\scalebox{0.5}{\mbox{M}}(S_{+})}=$
$\displaystyle\iint_{H^{2}_{+}}B_{i}^{\scalebox{0.5}{\mbox{em}}}r^{2}\sin\theta\,d\theta\,d\phi\;\hat{r}^{i}$
$\displaystyle-\iint_{D^{2}_{\scalebox{0.5}{\mbox{xy}}}}B_{i}^{\scalebox{0.5}{\mbox{em}}}r\,dr\,d\phi\;\hat{z}^{i}.$
(24)

Upon applying the boundary conditions (12) - (14), the first integral
vanishes, while the second one becomes:

$\displaystyle\iint_{D^{2}_{\scalebox{0.5}{\mbox{xy}}}}B_{i}^{\scalebox{0.5}{\mbox{em}}}r\,dr\,d\phi\;\hat{z}^{i}$
$\displaystyle=\frac{2\pi}{e}\left[\cos^{2}\theta_{\scalebox{0.5}{\mbox{W}}}\left(-2\right)+\sin^{2}\theta_{\scalebox{0.5}{\mbox{W}}}\left(-2\right)\right]$
$\displaystyle=-\frac{4\pi}{e}.$ (25)

Therefore, the magnetic charge carried by the monopole in the upper space is
$Q_{\scalebox{0.5}{\mbox{M}}(S_{+})}=4\pi/e$, which is characteristic of a
Cho-Maison monopole. Similar calculation shows that magnetic charge in the
lower space is $Q_{\scalebox{0.5}{\mbox{M}}(S_{-})}=-4\pi/e$.

Moreover, the magnetic dipole moment, $\mu_{m}$, of a Cho-Maison MAP can be
calculated according to the mixing shown in equation (20) and considering at
large $r$, $g^{\prime}B_{i}=gA^{\prime 3}_{i}$,

$\displaystyle
A_{i}^{\scalebox{0.5}{\mbox{em}}}\longrightarrow\frac{1}{e}\left(g^{\prime}B_{i}\right)=\frac{1}{e}\frac{nB_{s}}{r\sin\theta}\hat{\phi}_{i}.$
(26)

Here, we perform an asymptotic expansion,

$\displaystyle nB_{s}\longrightarrow-\frac{\mu_{m}\sin^{2}\theta}{r},$ (27)

and therefore, $\mu_{m}=-nrB_{s}/\sin^{2}\theta$. The value of $\mu_{m}$ (in
unit of $1/(e\cdot m_{\scalebox{0.5}{\mbox{W}}})$) is evaluated at
$\theta=\pi/2$.

##  4 Results

Figure 1 shows a comparison of 3D Higgs modulus for three MAP configurations,
(a) an SU(2) MAP [6], (b) a Nambu MAP (electroweak sphaleron) [17], and (c) a
Cho-Maison MAP. In Fig. 1, the $\rho$-axis is defined as
$\rho=\sqrt{x^{2}+y^{2}}$. Physical Higgs self-coupling, $\beta=0.77818833$,
was chosen for all three solutions and physical Weinberg angle,
$\tan\theta_{\scalebox{0.5}{\mbox{W}}}=0.53557042$, was used for the ones
shown in Fig. 1(b) and (c). Evidently, in a Nambu MAP, the poles are connected
through a Z0 flux string, but in a Cho-Maison MAP, they are two separate
entities, just like an SU(2) MAP.

Visually, the pole separation, $d_{z}$, of the Cho-Maison MAP is significantly
larger than that of the MAP found in the SU(2) YMH theory. The value of
$d_{z}$ for Cho-Maison MAP can be obtained numerically from the curve of
$\Phi_{1}\left(x,0\right)$ and $\Phi_{1}\left(x,\pi\right)$. This is because
$\Phi_{2}\left(x,\theta\right)$ is zero along the $z$-axis in boundary
condition (12) and hence, the behavior of $|\Phi|$ when $\theta=0$ or $\pi$ is
solely determined by $\Phi_{1}$.

Figure 2 shows the curve of $\Phi_{1}\left(x,\pi\right)$ for the MAP solution
shown in Fig. 1(c). The magnetic antimonopole is located at
$\Phi_{1}(x,\pi)=0$, which is labelled $C$ in Fig. 2. On the other hand, the
magnetic monopole is located at $\Phi_{1}(x,0)=0$ which shares the same value
as that of point $C$ in Fig. 2. The pole separation is then defined as
$d_{z}=2\times AC$.

Figure 4: Plots of (a) $d_{z}$ versus $\beta$ for a Cho-Maison MAP with a
zoomed-in version of the highlighted region shown in (b).

The plot of $\varepsilon_{\scalebox{0.5}{\mbox{W}}}$ versus $z$ at
$\rho=0.1077$ was shown in Fig. 3 for the particular solution displayed in
Fig. 1(c). The weighted energy density increases rapidly near the location of
monopoles, which clearly indicates there are two point singularities. As a
result, total energy of the system is infinite.

By fixing Weinberg angle at
$\tan\theta_{\scalebox{0.5}{\mbox{W}}}=0.53557042$, the Cho-Maison MAP
configuration is investigated for a range of $\beta$ from 0 to 1.7704. It is
found that $d_{z}$ varies with $\beta$ and the plot is shown in Fig. 4(a). The
value of $d_{z}$ starts off as 9.7140 when $\beta=0$, then monotonically
decreases until $\beta_{\scalebox{0.5}{\mbox{min}}}=1.25$, where the local
minimum $d_{z}=8.0920$ (green dot) is reached. For physical
$\beta=0.77818833$, $d_{z}$ is measured to be 8.2012 (red dot). Instead of
reaching a constant value, $d_{z}$ increases after
$\beta_{\scalebox{0.5}{\mbox{min}}}=1.25$ until
$\beta_{\scalebox{0.5}{\mbox{c}}}=1.7704$, where no solution can be found for
$\beta>\beta_{\scalebox{0.5}{\mbox{c}}}$. The existence of an upperbound in
$\beta$ is unexpected as SU(2) MAPs do not possess such a feature. The
corresponding value for $d_{z}$ when $\beta=\beta_{\scalebox{0.5}{\mbox{c}}}$
is 8.2760 and the behavior of $d_{z}$ near $\beta_{\scalebox{0.5}{\mbox{c}}}$
is shown in Fig. 4(b). It can be seen that gradient of the curve drastically
increases after $\beta=1.765$.

Figure 5: Plots of (a) $\mu_{m}$ versus $\beta$ when $\tan\theta_{\scalebox{0.5}{\mbox{W}}}=0.53557042$, (b) $\mu_{m}$ versus $\tan\theta_{\scalebox{0.5}{\mbox{W}}}$ when $\beta=0.77818833$ for a Cho-Maison MAP. Table 1: Table of $\mu_{m}$ and $d_{z}$ for selected $\beta$ of Cho-Maison MAP solutions at physical Weinberg angle $\tan\theta_{\scalebox{0.5}{\mbox{W}}}=0.53557042$ (Numbers in the table are rounded). $\beta$ | 0 | 0.2 | 0.4 | 0.6 | 0.7782 | 0.9 | 1 | 1.25 | 1.4 | 1.6 | 1.7704
---|---|---|---|---|---|---|---|---|---|---|---
$\mu_{m}$ | 10.2122 | 9.1227 | 8.8754 | 8.6719 | 8.5514 | 8.4983 | 8.4684 | 8.4350 | 8.4385 | 8.4715 | 8.5958
$d_{z}$ | 9.7140 | 8.7356 | 8.5170 | 8.3198 | 8.2012 | 8.1496 | 8.1208 | 8.0920 | 8.0994 | 8.1398 | 8.2760
Table 2: Table of $\mu_{m}$ and $d_{z}$ for selected $\tan\theta_{\scalebox{0.5}{\mbox{W}}}$ of Cho-Maison MAP solutions at physical Higgs self-coupling constant $\beta=0.77818833$ (Numbers in the table are rounded). $\tan\theta_{\scalebox{0.5}{\mbox{W}}}$ | 0.4675 | 0.5356 | 0.6 | 0.8 | 1 | 2 | 2.89 | 4 | 6 | 8 | 10
---|---|---|---|---|---|---|---|---|---|---|---
$\mu_{m}$ | 10.3178 | 8.5514 | 7.5528 | 5.6489 | 4.6219 | 3.0113 | 2.8098 | 2.9240 | 3.0831 | 3.1329 | 3.1513
$d_{z}$ | 9.9384 | 8.2012 | 7.2484 | 5.4582 | 4.4912 | 2.8624 | 2.6120 | 2.7568 | 2.9584 | 3.0228 | 3.0470

The plot of $\mu_{m}$ versus $\beta$ for a Cho-Maison MAP is shown in Fig.
5(a). The shape of the curve is basically identical to Fig. 4(a), which is
expected, considering the close relations between $\mu_{m}$ and $d_{z}$. The
physical value is measured to be $\mu_{m}=8.5514$ (red dot), while the minimum
$\mu_{m}$ is 8.4350 around $\beta_{\scalebox{0.5}{\mbox{min}}}=1.25$ (green
dot). We also investigate the behavior of $\mu_{m}$ by fixing
$\beta=0.77818833$, while allowing $\tan\theta_{\scalebox{0.5}{\mbox{W}}}$ to
vary between 0.4675 and 10, Fig. 5(b). It is found that $\mu_{m}\rightarrow
10.3178$ as $\tan\theta_{\scalebox{0.5}{\mbox{W}}}\rightarrow 0.4675$, where a
lower bound is reached, below which no solutions can be found. In a similar
manner, $\mu_{m}$ decreases until
$\tan\theta_{\scalebox{0.5}{\mbox{W}}}=2.89$, then increases slightly with
increasing $\tan\theta_{\scalebox{0.5}{\mbox{W}}}$ before converging to a
limiting value. The minimum value found this way is much lower,
$\mu_{m}=2.8098$ (green dot). Selected data of Cho-Maison MAP is tabulated in
Table 1 and 2.

##  5 Conclusion

In conclusion, we have found numerical solutions in the SU(2)$\times$U(1) WS
theory corresponding to a pair of Cho-Maison monopole and antimonopole. Poles
of this MAP configuration are not connected through a Z0 flux string and each
pole carries magnetic charge $\pm 4\pi/e$. This configuration is important
because if Cho-Maison monopoles were detected in the lab, it would be through
pair production, which is exactly the MAP configuration discussed in this
study.

When $\tan\theta_{\scalebox{0.5}{\mbox{W}}}$ is fixed at 0.53557042, there
exists an upperbound for the Cho-Maison MAP at
$\beta_{\scalebox{0.5}{\mbox{c}}}=1.7704$, beyond which no solutions can be
found. This is a feature that MAP solutions found in the SU(2) YMH theory do
not possess. When $\beta$ is fixed at 0.77818833, a lower bound exists at
$\tan\theta_{\scalebox{0.5}{\mbox{W}}}=0.4675$, where both $\mu_{m}$ and
$d_{z}$ reach their maximum values of 10.3178 and 9.9384. However, as
solutions in the SU(2)$\times$U(1) WS theory are controlled by two parameters,
$\beta$ and $\tan\theta_{\scalebox{0.5}{\mbox{W}}}$, finding global extrema
must be done on a 2-dimensional curve, like
$d_{z}\left(\beta,\tan\theta_{\scalebox{0.5}{\mbox{W}}}\right)$. Lastly, we
were able to accurately measure the physical values of $\mu_{m}$ and $d_{z}$
when $\beta=0.77818833$ and
$\tan\theta_{\scalebox{0.5}{\mbox{W}}}=0.53557042$. The magnetic dipole moment
of a Cho-Maison MAP is $\mu_{m}=8.5514$ in unit of $1/(e\cdot
m_{\scalebox{0.5}{\mbox{W}}})$ and the pole separation is $d_{z}=8.2012$ in
unit of $1/m_{\scalebox{0.5}{\mbox{W}}}$.

Although the numerical method employed in this research is a direct
generalization of a well-known procedure used in Refs. [6, 18], the solutions
obtained in this study are unique and new. Indeed, the originality of this
work lies in the actual construction of a novel numerical solution, which is a
finitely separated Cho-Maison MAP.

Finally, due to the presence of point singularities at the location of
monopoles, total energy of the system is infinite. However, it is possible to
regularize the solutions by introducing a non-trivial U(1) hypercharge
permeability in the form of a dimensionless function,
$\epsilon\left(\boldsymbol{\phi}\right)=\left(H/H_{0}\right)^{8}$ [10, 11].
This will be reported in a future work. Additionally, electric charges can be
introduced into the system forming a pair of dyon and antidyon.

## References

  * [1] P.A.M. Dirac, Proc. R. Soc. Lond. A 133, 60 (1931); Phys. Rev. 74, 817 (1948).
  * [2] T.T. Wu and C.N. Yang, in Properties of Matter under Unusual Conditions, edited by H. Mark and S. Fernbach (Intersience, New York, 1969), p. 349; Nucl. Phys. B 107, 365 (1976); Phys. Rev. D 16, 1018 (1977).
  * [3] G. ’t Hooft, Nucl. Phys. B 79, 276 (1974); A.M. Polyakov, JETP Lett. 20, 194 (1974).
  * [4] M.K. Prasad and C.M. Sommerfield, Phys. Rev. Lett. 35, 760 (1975).
  * [5] P. Forgács, Z. Horváth, and L. Palla, Phys. Lett. B 99, 232 (1981); M. K. Prasad and P. Rossi, Phys. Rev. D 24, 2182 (1981).
  * [6] B. Kleihaus and J. Kunz, Phys. Rev. D 61, 025003 (1999).
  * [7] B. Kleihaus, J. Kunz and Y. Shnir, Phys. Lett. B 570, 237 (2003); Phys. Rev. D 70, 065010 (2004).
  * [8] Y.M. Cho, D. Maison, Phys. Lett. B 391, 360 (1997).
  * [9] Yisong Yang, Proc. Roy. Soc. A454, 155 (1998); Solitons in Field Theory and Nonlinear Analysis (Springer Monographs in Mathematics), p. 322 (Springer-Verlag) 2001.
  * [10] Kyoungtae Kimm, J.H. Yoon, and Y.M. Cho, Eur. Phys. J. C75, 67 (2015); Kyoungtae Kimm, J. H. Yoon, S. H. Oh and Y. M. Cho, Mod. Phys. Lett. A 31, 1650053 (2016).
  * [11] P. Zhang, L. Zou, Y. M. Cho, Eur. Phys, J. C 80, 280 (2020).
  * [12] F. Blaschke and P. Benes, Prog. Theor. Exp. Phys. 073B03 (2018).
  * [13] Y. M. Cho, Phil. Trans. R. Soc. A377, 0038 (2019).
  * [14] B. Acharya et al. (MoEDAL Collaboration), Phys. Rev. Lett. 118, 061801 (2017); Phys. Rev. Lett. 123, 021802 (2019); Phys. Rev. Lett. 126, 071801 (2021).
  * [15] ATLAS Collaboration, Phys. Rev. Lett. 124, 031802 (2020).
  * [16] Y. Nambu, Nucl. Phys. B 130, 505 (1977).
  * [17] R. Teh, B.L. Ng and K.M. Wong, Ann. Phys. (N. Y.), 362, 170 (2016).
  * [18] B. Kleihaus, J. Kunz and M. Leißner, Phys. Lett. B 678, 313 (2009).
  * [19] R. Gervalle and M.S. Volkov, Nucl. Phys. B 984, 115937. (2022).
  * [20] P.A. Zyla et al. (Particle Data Group), Prog. Theor. Exp. Phys. 2020, 083C01 (2020).
  * [21] M. Hindmarsh, M. James, Phys. Rev. D 49, 6109. (1994).
  * [22] E. Radu, M.S. Volkov, Phys. Rev. D 79, 065021. (2009).