# Implicit Neural Representation in Medical Imaging: A Comparative Survey

Amirali Molaei1 Amirhossein Aminimehr1 Armin Tavakoli1 Amirhossein Kazerouni2
Bobby Azad3 Reza Azad4 Dorit Merhof${}^{~{}5,6,}$ Corresponding author: Dorit
Merhof, Tel.: +49 (941) 943-68509, E-mail<EMAIL_ADDRESS>1 _School of
Computer Engineering, Iran University of Science and Technology, Tehran, Iran_
2 _School of Electrical Engineering, Iran University of Science and
Technology, Tehran, Iran_
3 _South Dakota State University, Brookings, South Dakota, USA_
4 _Faculty of Electrical Engineering and Information Technology ,RWTH Aachen
University, Aachen, Germany_
5 _Institute of Image Analysis and Computer Vision, Faculty of Informatics and
Data Science, University of Regensburg, Regensburg, Germany_
6 _Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany_

###### Abstract

Implicit neural representations (INRs) have gained prominence as a powerful
paradigm in scene reconstruction and computer graphics, demonstrating
remarkable results. By utilizing neural networks to parameterize data through
implicit continuous functions, INRs offer several benefits. Recognizing the
potential of INRs beyond these domains, this survey aims to provide a
comprehensive overview of INR models in the field of medical imaging. In
medical settings, numerous challenging and ill-posed problems exist, making
INRs an attractive solution. The survey explores the application of INRs in
various medical imaging tasks, such as image reconstruction, segmentation,
registration, novel view synthesis, and compression. It discusses the
advantages and limitations of INRs, highlighting their resolution-agnostic
nature, memory efficiency, ability to avoid locality biases, and
differentiability, enabling adaptation to different tasks. Furthermore, the
survey addresses the challenges and considerations specific to medical imaging
data, such as data availability, computational complexity, and dynamic
clinical scene analysis. It also identifies future research directions and
opportunities, including integration with multi-modal imaging, real-time and
interactive systems, and domain adaptation for clinical decision support. To
facilitate further exploration and implementation of INRs in medical image
analysis, we have provided a compilation of cited studies along with their
available open-source implementations on GitHub. Finally, we aim to
consistently incorporate the most recent and relevant papers regularly.

##  1 Introduction

Figure 1: Chart (a) visually displays the relative proportions of published
papers according to their application, and chart (b) illustrates the count of
published papers based on INR design in medical tasks during different time
periods (with ”H” indicating the first or second half of the year). The
assessment of the statistics is based on a sample of 64 research papers
published during the years 2021 to 2023.

Knowledge representation is one of the fundamental pillars of artificial
intelligence (AI). Its importance stems from its significant impact on the
success of machines in learning many AI-related tasks, such as classification
and decision-making. Humans construct task-specific representations to
facilitate their interaction with the world around them [36]. Consequently, a
substantial amount of AI research involves proposing algorithms and models
that mimic this human cognition process to solve machine learning tasks with a
degree of performance that equals, or even surpasses, human capabilities [49,
7, 8, 4].

Our visual world can be represented continuously, which is a fundamental
principle in fields such as computer vision. Data obtained through observation
and sensing manifests in various forms, including images and audio.
Conventional approaches to encoding input signals as representations typically
follow an explicit paradigm, where the input space is discretized or
partitioned into separate elements (_e.g_., point clouds, voxel grids, and
meshes). However, in recent years, an alternative approach to representation,
known as implicit representations, has gained popularity due to its efficient
memory usage [23]. Unlike explicit (or discrete) representations that directly
encode the features or signal values, implicit representations are defined as
a generator function that maps input coordinates to their corresponding value
within the input space.

In computer vision, the quality of representing an image signal holds central
importance. Deep neural networks have emerged as the de facto tools for
complex tasks across various AI domains, particularly in computer vision,
owing to their remarkable representation learning ability [27, 5]. As a
result, there has been an exploration of leveraging their capacity to function
as implicit functions, yielding promising results [50, 63]. Within this
context, a Multi-Layer Perceptron (MLP) is trained to parameterize the signal
of interest, such as an image or shape, utilizing coordinates as input. The
objective is to predict the corresponding data values at those coordinates.
Thus, the MLP serves as an Implicit Neural Representation function that
encodes the signal’s representation within its weights. For instance, in the
case of image signals, feeding the MLP with pixel coordinates leads to the
generation of its RGB value as the output.

These implicit neural functions find extensive application in tasks such as
image generation, super-resolution, 3D object shape reconstruction, and
modeling complex signals [33, 12, 34, 62]. The utilization of MLPs for image
and shape parameterization offers several advantages. Firstly, they are
resolution-agnostic as they operate in the continuous domain. This
characteristic enables them to generate values for coordinates in-between
pixel or voxel-wise grids, thereby facilitating vision tasks. Secondly, the
memory requirements for representing the signal are not limited by its
resolution. Consequently, MLP models exhibit enhanced memory efficiency while
remaining effective, especially when compared to the grid or voxel
representations. The memory demand scales according to the complexity of the
signal itself. Thirdly, MLPs assist INRs in mitigating potential locality
biases that could hinder performance, unlike convolutional neural networks
(CNNs). Finally, MLP models are differentiable, endowing them with significant
adaptability across a wide range of applications. Their weights can be
adjusted using gradient-based techniques, allowing for versatility in handling
different tasks [15, 37, 50, 30, 63, 55, 42, 72].

Figure 2: The figure illustrates various modifications to alleviate the
spectral bias problem in INRs, provides an overview of their underlying
principles, and introduces NeRF as an additional background method, as
discussed in section 2.

To address the challenges posed by the high cost of labeled data and limited
memory resources, INRs have garnered increasing attention from the medical
community, as evident from the exponential growth in research papers dedicated
to this domain (Figure 1). This surge of interest in INR within the medical
imaging community has resulted in a multitude of applications across diverse
medical imaging scenarios. In recent years, their predominant usage has been
in enhancing resolution and synthesizing missing information. They offer a
solution to alleviate the burden of collecting labeled data in medical domains
by eliminating the need for training data and explicit labels. Instead, they
leverage available measurements or signals without the requirement of labeled
data for each instance, enabling the reconstruction of 3D anatomical
structures or generation of 2D scans [54, 40, 48]. For instance, in MRI,
super-resolution techniques can enhance the spatial resolution of images,
providing clearer visualization of structural features and diagnostic
information [31, 61, 62]. Moreover, they can be employed in synthesis and
inverse problems, such as reconstructing CT and MRI data from projection and
frequency domains while reducing radiation exposure [54, 69, 40, 48, 64].
Additionally, INRs find utility in neural rendering to model complex
relationships in scenes, enabling detailed visualizations of anatomical
structures or aiding in robotic surgery by reconstructing deformable surgical
scenes [16, 58].

To present a comprehensive review of these emerging architectures, this paper
provides an overview of their core principles and diverse applications, as
well as their advantages and limitations. To the best of our knowledge, this
is the first survey paper that covers the application of INR in medical
imaging and sheds light on new directions and research opportunities, serving
as a roadmap and systematic guide for researchers. We also aspire to generate
increased interest within the vision community to delve into the exploration
of implicit neural representations in the medical domain. Our key
contributions are as follows:

$\bullet$ We conduct a systematic and comprehensive review of the applications
of INR in the field of medical imaging. We analyze and compare state-of-the-
art (SOTA) approaches for each specific task.

$\bullet$ We provide a detailed and organized taxonomy, as illustrated in
Figure 4, which allows for a structured understanding of the research progress
and limitations in different areas.

$\bullet$ Additionally, we discuss the challenges and open issues associated
with INR in medical imaging. We identify new trends, raise important
questions, and propose future directions for further exploration.

_Search Strategy_. To conduct a comprehensive search, we utilized DBLP, Google
Scholar, and Arxiv Sanity Preserver, employing customized search queries
tailored to retrieve relevant scholarly publications. Our search queries
consisted of keywords such as (INR $|$ implicit neural representation $|$
medical $|$ Task), (INR $|$ medical $|$ Neural rendering), and (INR $|$
medical $|$ NeRF). Here, Task refers to one of the applications covered
(Figure 4). To ensure the selection of relevant papers, we conducted a
meticulous evaluation based on factors such as novelty, contribution, and
significance. Priority was given to papers that were pioneering in the field
of medical imaging. Subsequently, we selected papers with the highest rankings
for further examination.

##  2 Background

Implicitly representing signals with neural networks has gathered pace in
recent years. Instead of parametrizing signals with discrete representations
such as grids, voxels, point clouds, and meshes, a simple MLP can be learned
to continuously represent the signal of interest as an implicit function
$\Psi:\mathrm{x}\rightarrow\Psi{(\mathrm{x})}$, mapping their spatial
coordinates $\mathrm{x}\in\mathbb{R}^{M}$ from $M$ dimensional space to their
corresponding $N$ dimensional value $\psi{(\mathrm{x})}\in\mathbb{R}^{N}$
(_e.g_., occupancy, color, etc.). While INRs have shown promising, they can
fail to encode high-frequency details compared to discrete representations,
leading to a suppressed representation quality. Rahaman _et al_. [39] have
made significant strides in uncovering limitations within conventional ReLU-
based MLPs and their ability to represent fine details in underlying signals
accurately. These MLPs have shown a propensity to learn low-frequency details,
leading to a phenomenon known as spectral bias in piece-wise linear networks.
In order to address this issue, several approaches have been explored to
redirect the network’s focus toward capturing high-frequency details and
effectively representing the signal with finer-grained details. To enhance the
representation of the input signal, three avenues can be pursued within an MLP
framework based on its structure. Firstly, one can consider changing the input
type by mapping it to a higher-dimensional space to enable the network to
capture more intricate details within the signal. Secondly, another approach
involves replacing the ReLU activation function with a new activation function
that better facilitates the learning of high-frequency components. Lastly, one
can explore altering the output of the MLP to a higher-dimensional space,
where each node is responsible for reconstructing a specific part of the
signal. In this section, we will provide a background based on the
modifications that can be made to mitigate the spectral bias issue.
Additionally, we will cover a neural volume rendering model called NeRF [34]
as a pioneering approach to bridge implicit representations and novel view
synthesis. Figure 2 illustrates the overview of our proposed background.

###  2.1 Input

The conventional approach in INR treats the spatial coordinate of each element
in the signal, such as pixels in an image, as the input to an MLP. However,
this approach tends to learn low-frequency functions, limiting its ability to
effectively represent complex signals. To address this limitation, recent
progress suggests using a sinusoidal mapping of the Cartesian coordinates to a
higher dimensional space, which enables the learning of high-frequency details
more effectively [55]:

  1. 1.

Basic:
$\gamma(\mathbf{v})=[\text{cos}(2\pi\mathbf{v},\text{sin}(2\pi\mathbf{v})]^{T}.$

  2. 2.

PE:
$\gamma(\mathbf{v})=[...,\text{cos}(2\pi\sigma^{j/m}\mathbf{v},\text{sin}(2\pi\sigma^{j/m}\mathbf{v}),...]^{T}$
for $j=0,...,m-1$. PE denotes Positional Encoding, and the scale $\sigma$ is
determined for individual tasks and datasets through a process of
hyperparameter sweep.

  3. 3.

Gaussian:
$\gamma(\mathbf{v})=[\text{cos}(2\pi\mathbf{B}\mathbf{v},\text{sin}(2\pi\mathbf{B}\mathbf{v}]^{T},$
where the variable $\mathbf{v}$ represents the signal coordinates, while
$\mathbf{B}$ is a random Gaussian matrix, where each entry is independently
sampled from a normal distribution $\mathcal{N}(0,\sigma^{2})$. Similarly, the
scale $\sigma$ is selected through a hyperparameter sweep for each task and
dataset.

These encoding processes are known as Fourier features mapping.

###  2.2 Activation Function

In general, the intuition behind activation functions is to apply non-
linearity to the neural network. As for implicit representations,
nonlinearities can be either periodic or non-periodic. However, non-periodic
functions, such as $ReLU$ or $tanh$, are not conducive to the effective
learning of high-frequency signals. To handle this issue, Sinusoidal
Representation Networks (SIRENs) [50] utilize $sine$ as the activation
function of the MLP to parametrize complex data:

$\displaystyle\mathbf{\Psi}(\mathbf{x})=\mathbf{W}_{n}(\psi_{n-1}\circ\psi_{n-2}\circ\dots\psi_{0})(\mathbf{x})+\mathbf{b_{n}},$
(1)
$\displaystyle\mathbf{x_{i}}\mapsto\psi_{i}(\mathbf{x_{i}})=\mathrm{sin}(\mathbf{W_{i}}\mathbf{x_{i}}+\mathbf{b_{i}}),$

where $\psi_{i}$ indicates the $i^{th}$ layer of the neural network,
$\mathbf{x}$ is the signal of interest, $\mathbf{W}$ and $\mathbf{b_{i}}$
represents the weight matrix and bias, respectively. The use of sine as the
activation boils down to its derivative being a shifted sine (cosine), which
enables the network to efficiently parametrize higher-order derivatives, such
as the image Laplacian or Helmholtz equation. Furthermore, the sine function
helps to effectively represent signals containing high-frequency details.
SIREN authors suggest a unique initialization technique to prevent vanishing
gradients in traditional activation functions. They initialize weights of each
layer as $W\sim U(\frac{-c}{\sqrt{n}},\frac{c}{\sqrt{n}})$, where $W$ is the
weight of the layers, $U(.)$ is a uniform distribution, $c$ denotes a constant
that controls the range of the weight values and $n$ is the number of inputs
neurons.

###  2.3 Output

Target signals such as images and audio typically exhibit local structure and
dependencies among neighboring elements, which can be effectively utilized to
enhance ReLU networks during training. Aftab _et al_. [1] introduce a multi-
head network architecture where the main body learns global features of the
signal, while the output layer consists of multiple heads. These heads
reconstruct separate parts of the signal and learn its local features. For
instance, in the case of images, they divide the image into equal grid cells.
Each cell is then processed by an MLP in the main body to capture global
features, and the output sparse nodes reconstruct the details of each cell
individually. This approach aims to reduce the network’s bias towards low-
frequency components by exploiting the inherent properties of the target
signal. Therefore, changing the output to a higher dimensional space can
effectively alleviate the issue of spectral bias.

###  2.4 NeRF

Neural Radiance Fields (NeRFs) [34] unites INRs with volume rendering by using
fully connected MLPs to implicitly represent scenes and objects with the goal
of novel view synthesis. The objective of novel view synthesis is to develop a
system that can generate novel viewpoints of an object from any direction by
observing a few images of that particular object. The process is defined as:

$\displaystyle
F_{\theta}(\mathrm{x},\mathrm{d})\longrightarrow(\mathrm{c},\sigma),$ (2)

where $\mathrm{x}$ indicates the 3D location $(x,y,z)$, $\mathrm{d}$
represents the 2D vector of viewing direction $(\theta,\phi)$, $\mathrm{c}$
denotes the color values $(r,g,b)$, and $\mathrm{\sigma}$ is the volume
intensity. The primary idea is to overfit an implicit function on the training
data such that, given $(x,y,z)$ as the spatial coordinates and ($\theta,\phi$)
as the viewing direction, the network outputs the color and volume density of
the particular location. Unlike SIREN [50], the NeRF architecture is equipped
with ReLU as its activation function, but similar to Fourier features [55],
adopts a positional encoding approach to map coordinates to higher dimensions
as follows:

$\displaystyle\gamma(p)=(sin(2^{0}\pi p),cos(2^{0}\pi p),\dots,$ (3)
$\displaystyle sin(2^{L-1}\pi p),cos(2^{L-1}\pi p)),$

where $p$ can be each of the coordinate or viewing direction components.
NeRF’s architecture is designed in a two-stage matter to obtain density and
color values as follows:

$\displaystyle\mathrm{First\
Stage}:\mathrm{\sigma},\mathrm{h}=MLP(\mathrm{x})$ (4)
$\displaystyle\mathrm{Second\
Stage}:\mathrm{c}=MLP(concat[\mathrm{h},\mathrm{d}])$ (5)

where in the first stage the 3D coordinate $\mathrm{x}$ is passed through the
MLP to obtain the density $\mathrm{\sigma}$ and a feature representation
$\mathrm{h}\in\mathbb{R}^{N}$ ($N$ equals 256 in original implementation), and
in the second stage, $\mathrm{h}$ is input to the MLP to output the color
values $\mathrm{c}$ (the original implementation [34] adopts identical MLPs to
accomplish this). Finally, volume rendering [26] is utilized to generate novel
views by tracing camera rays through every pixel of the target synthesized
image.

Figure 3: Comparison of the aligned Pedunculopontine Nucleus (PPN) region
(blue) and manually segmented PPN region (lavender) by radiologists in a
Parkinson’s disease patient [29]. The brain planes (axial, sagittal, and
coronal) show the results of affine and Diffeomorphic (DARTEL) registration,
aligning the myelin staining histological atlas and the enhanced atlas
generated through INR-based super-resolution. Expert radiologists manually
segmented the last two columns to validate the INR-based method.

##  3 Clinical Importance

Thanks to the memory and data efficiency of INRs, they are widely utilized in
numerous medical imaging tasks. One of the most significant challenges in
automated medical imaging is the collection of ground truth annotated data
from reliable sources, such as clinicians and medical professionals [24, 3,
27]. This process is painstaking, expensive, time-consuming, and requires
significant effort. Unlike simple scenes that can be easily recognized and
labeled (_e.g_., categorizing an indoor scene), annotating medical images
should be performed by medical professionals and clinicians. This reliance on
experts, coupled with privacy concerns and the need for patient authorization,
creates a major bottleneck in the annotation process for medical imaging. INRs
offer significant advantages in various applications without the need for
external training annotations [54, 38, 40, 48, 60]. For example, in the field
of medical imaging, INRs are particularly beneficial for tasks like super-
resolution. During the medical imaging procedures like CT scans, PET scans,
MRI, and ultrasound, patient movement can also cause motion artifacts and
result in blurred images or poorly defined structures, especially in upper
abdominal regions such as the chest which are negatively affected by patient
motion. Additionally, in Cone Beam Computed Tomography (CBCT), which is
commonly used in dental and maxillofacial imaging, slow imaging speed combined
with patient movement can result in motion artifacts and lead to poorly
defined structure boundaries [51, 18, 72]. Furthermore, obtaining high-quality
MRI scans poses a challenge due to longer scan times [61, 62]. Conventional
approaches are not suitable for effectively handling these issues through
super-resolution or image reconstruction as they are not resolution-agnostic
and require notable amounts of data. However, INRs can address super-
resolution more effectively by considering inputs from the continuous
coordinate domain and being resolution-agnostic.

Implicit neural models are also widely used in biomedical applications,
particularly in solving inverse imaging problems [69, 40, 48, 48]. These
problems involve learning the structure of an object (organ of interest in
medical cases) from observations or measurements. Using INRs, it becomes
possible to reconstruct CT or MRI scans directly from the sensor domain.
Moreover, they can even facilitate the tracking of tissue progression by
incorporating prior scans from earlier time steps, subsequently reconstructing
the updated scan for the current time.

In practical applications, the reconstruction of images from sparsely sampled
data plays a crucial role. This need arises in various domains, including
medical imaging, where it has proven particularly valuable in specific
applications such as reducing radiation dose in CT imaging and accelerating
MRI scans [54, 40, 48]. Notably, Single Image Super-Resolution (SISR)
techniques have attracted considerable attention due to their potential to
restore a high-resolution (HR) image solely based on a low-resolution (LR)
input [62, 31]. The ability of SISR methods to enhance image details and
fidelity has significant implications for improving diagnostic accuracy and
aiding medical professionals in their decision-making processes. This cannot
be accomplished using convolution-based techniques as they are trained
exclusively for particular up-scaling tasks. Additionally, the time-consuming
process of retraining them for a new up-scaling task hinders their usefulness
in clinical applications [61].

INRs have also found application in assisting robotic surgery [46, 45, 58, 71,
2]. The integration of INRs within robotic surgical systems allows for
enhanced perception and understanding of the surgical environment. By
leveraging INRs, robotic surgical systems can better interpret intraoperative
images, providing real-time feedback and guidance to the surgeon. This can
assist in accurate tissue segmentation, localization of critical anatomical
structures, and precise surgical tool manipulation.

To underscore the practical applicability of INRs in addressing real-world
challenges, it is crucial to highlight the validation of these models through
human expertise. By corroborating the findings of INRs with the insights of
medical professionals, we can establish the reliability and effectiveness of
these novel diagnostic tools. This notion is exemplified by a compelling study
that utilized the expertise of radiologists to validate the outcomes. In the
depicted case study (Figure 3), the aligned Pedunculopontine Nucleus (PPN) and
the manually segmented PPN region by two radiologists are compared for a
patient with Parkinson’s disease [29]. The study focuses on enhancing the
visibility and localization of the PPN, a deep brain structure crucial for
Parkinson’s disease treatment, through a combination of a Quantitative
Susceptibility Mapping Atlas (QSM) and an INR network. The INR-based process
significantly improves the spatial resolution of the atlas, effectively
overcoming limitations and minimizing artifacts and blurring effects. As a
result, it allows for better delineation of the specific region (PPN) on the
atlas, indicating the usability and effectiveness of this approach in clinical
settings.

In conclusion, INRs models offer significant advantages in medical imaging
tasks, addressing challenges such as the lack of annotated data and artifacts
in scans. We believe that research can exploit experts to validate their
method’s practicality, therefore better demonstrating the usefulness of INRs.
In conclusion, INRs have emerged as a valuable and adaptable tool in clinical
settings, successfully tackling a diverse range of imaging challenges. Their
widespread use is anticipated to continue to grow in the future, offering new
possibilities for medical imaging research.

##  4 Taxonomy

In this section, we provide a taxonomy with a focus on the application of INRs
in several medical imaging tasks to acquaint researchers with the notable
operation and functionality of these models.

Figure 4: The taxonomy subsections delineate six distinct sub-fields in
medical imaging: (1) Reconstruction, (2) Segmentation, (3) Registration, (4)
Compression, and (5) Neural Rendering. We use the numbering of the methods in
ascending order and provide the reference for their paper as follows: 1. [61],
2. [48], 3. [40], 4. [62], 5. [54], 6\. [59], 7\. [60], 8\. [53], 9\. [9],
10\. [70], 11\. [22], 12\. [29], 13\. [67], 14\. [66], 15\. [16], 16\. [58],
17\. [18].

###  4.1 Reconstruction

Image reconstruction is a critical task in medical analysis, enabling
professionals to obtain high-quality images for clinical applications. Many
studies have explored the use of convolutional neural networks (CNNs) to learn
a mapping function that transforms raw data into reconstructed images.
However, this approach faces challenges, including the need for large-scale
training datasets, instability in the presence of structural modifications,
and difficulties in generalizing to diverse image modalities or anatomical
locations [6]. Overcoming these obstacles is crucial to improve the
reliability and applicability of image reconstruction in medical settings. To
use INR here, the task is conventially defined as an inverse problem in
medical image reconstruction takes noisy or undersampled measurements of a
medical image as input and aims to generate a reconstructed, complete image as
output. The input can come from various imaging modalities such as CT, MRI, or
ultrasound, and the incompleteness may be due to time constraints, reduced
radiation exposure, or patient movement. The INR model learns to map the input
measurements to the corresponding complete images, recovering the missing
information and producing high-quality images that resemble the ground truth
obtained from fully sampled acquisitions.

To address the aforementioned challenges, numerous INR-based reconstruction
methods have been developed in recent years. For instance, NeRP [48] framework
proposes to integrate implicit neural networks for reconstructing sparsely
sampled medical images through three stages without demanding any training
data. As illustrated in Figure 5, in the first stage, a neural network’s
weights are encoded with a CT image as prior knowledge. Next, the implicit
network is optimized on sparsely sampled sinogram measurements to learn
reconstruction. Finally, the network is applied to all associated spatial
coordinates to generate the final reconstructed CT image. The effectiveness of
NeRP in reconstructing tumor structural progression and its versatile
applicability across various imaging modalities have been demonstrated through
experiments conducted on 2D and 3D data, including CT clinical scans and brain
tumor progression MRI data images.

Additionally, Reed _et al_. [40] proposed a method (DCTR) for reconstructing
dynamic, time-varying scenes using computed tomography (4D-CT). The INR is
utilized to estimate a template reconstruction of the 3D volume’s linear
attenuation coefficients (LACs) in the scene, acting as a prior model that
captures the spatial distribution of LACs. Here, the template refers to a
representation or approximation of the scene’s properties, specifically the
LACs in the 3D volume. By using the INR, DCTR generates a template
reconstruction of the LACs based on available CT measurements or sinograms
through learning a mapping between coordinates $(x,y,z)$ and the template
reconstruction of the LACs, which serves as a starting point for the overall
reconstruction process. DCTR then employs a parametric motion field, a set of
parameters describing how the template should be warped over time to account
for scene motion. Finally, the warped template reconstruction is used to
synthesize sinograms through a differentiable Radon transform, which is then
compared to the actual sinogram to evaluate the accuracy of the
reconstruction. The proposed method demonstrates robust reconstruction of
images with deformable and periodic motion and is validated on their synthetic
D4DCT [40] dataset and the thoracic CT data [11].

Figure 5: Overview of the NeRP framework (taken from [48]). NeRP solves the
reconstruction problem in three stages: (a) Embedding a prior medical image in
an MLP by implicitly encoding it in its weights. (b) Adapting the MLP on the
target image by minimizing the loss between the measurements of the prior and
the target image. (c) Inferencing the actual target CT or MRI image from each
of the coordinates.

###  4.2 Segmentation

Medical image segmentation is a critical task in healthcare systems, aiding in
disease diagnosis and treatment planning. Deep learning methods have shown
promising results in achieving accurate segmentation results. However, these
methods often suffer from computational inefficiency and difficulty in
handling complex topologies [56]. Complex topologies refer to intricate
structural relationships and variations within medical images, such as
lesions, tumors, and intricate vessel structures.

To address these limitations, Barrowclough  _et al_. [9] introduced a novel
approach called BS-ISR that combines convolutional neural networks (CNNs) with
INRs. INRs were specifically selected for their ability to handle complex,
high-dimensional medical imaging data and capture intricate topologies.
Instead of directly generating images, the model utilizes spline
representations to capture geometric boundaries and structures. The authors
also introduced new loss functions tailored for modeling implicit splines,
utilizing binary inside-outside masks. Evaluation on the Congenital Heart
Disease dataset [65] demonstrated the superior performance of the model
compared to other SOTA methods, as measured by the average volumetric test
Dice score metric. In another research, Gu _et al_. [22] proposed a self-
distillation-based INR method for segmentation of retinal vessels for ocular
disease diagnosis (Retinal INR). They utilized the Vision Transformer (ViT)
[17] to capture global dependencies in retinal images by treating the image as
a sequence of patches rather than focusing solely on local features. The self-
distillation method extracted key features for blood vessel segmentation. The
primary benefit of the suggested approach lies in its ability to enhance the
resolution of retinal images and magnify the finer details of capillaries
through the use of INR. To ensure accurate results, they utilized an improved
centerline dice (clDice) loss function to constrain blood vessel topology. The
proposed model was evaluated on the Drive [52] and Chase [19] datasets,
showcasing its superiority over non-INR methods in terms of segmentation
accuracy, detection of detailed structures, and robustness to variations in
image quality and content.

###  4.3 Registration

Figure 6: An illustration of the IDIR registration framework (taken from
[60]), which uses Implicit Neural Representations (INRs) in a multi-layer
perceptron (MLP) to directly optimize the deformation function
$\Phi(x)=u(x)+x$, which maps image coordinates to the deformation vector
$u(x)$.

Medical image registration is a process that aligns multiple images, volumes,
or surfaces within a common coordinate system to identify common areas. It
requires learning a transformation function that geometrically aligns
coordinates between a source and target image. Traditional methods often
require complex, multi-step processes and assumptions about the nature of the
transformations. However, INRs are capable to model the transformation
function without external assumptions. This allows them to handle the
variations and warping in the image in a smooth, coherent manner and at any
image resolution, making them ideally suited for tasks such as image
registration.

In this regard, Wolterink _et al_. [60] proposed IDIR, that employs INR to
model the transformation function with a SIREN-based design for deformable
registration, which attempts spatial alignment of images to account for
changes in the shape, position, or size of anatomical structures, such as
organs, tumors, or other features of interest. As depicted in Figure 6, the
transformation function $\phi(x)=u(x)+x$, which maps each coordinate $x$ in a
fixed image to a coordinate in a moving image, is represented using the MLP.
The MLP takes a continuous coordinate $x$ from the image domain as input and
predicts a deformation vector $u(x)$. The addition of $u(x)$ and $x$ gives the
output $\phi(x)=u(x)+x$. Moreover, the periodic activation function in the MLP
allows for higher-order derivatives, enabling advanced regularization
techniques for accurate and flexible image registration without relying on
CNNs. This model was tested on 4D chest CT registration using the DIR-LAB
dataset [11] and surpassed all deep learning-based methods without folding or
training data needed.

In another study, Sun _et al_. [53] developed mirnf, which can model both
displacement vector fields and velocity vector fields, providing two different
approaches for performing image registration. While displacement vector fields
are used for deformable registration, velocity vector fields are employed for
diffeomorphic registration, both utilizing INRs to model the transformations
between the target and moving images. The network in registration based on
velocity predicts velocity vectors $[v_{p_{x}},v_{p_{y}},v_{p_{z}}]$ from 3D
coordinates in the target image. These vectors are integrated over time using
a Neural ODE Solver to generate a deformation field by mapping each point in
the target image to its corresponding deformed position in the moving image.
By applying the deformation field to the target image, it can be aligned with
the moving image. Alternatively, another approach trains an MLP to directly
predict displacement vectors $[\phi_{p_{x}},\phi_{p_{y}},\phi_{p_{z}}]$ for
each coordinate in the source image. These vectors describe how each point in
the source image should be shifted or deformed to align with the target image.
The target volume is deformed to match the source volume by applying these
displacement vectors, which involves adding the displacement vector to each
point’s position in the target volume. The authors conducted experiments on
two 3D MR brain scan datasets, Mindboggle101 [28] and OASIS [32], and found
that INR achieves SOTA performance in terms of registration accuracy,
optimization speed, and regularity compared to traditional methods.

###  4.4 Compression

Figure 7: The Surgical Neural Rendering framework proposed by Wang _et al_.
(figure taken from [58]). The surgical scenes are represented using a
canonical radiance field $F_{\theta}(x,d)$ and a time-dependent displacement
field $G_{\Phi}(x,t)$. Both models are designed using MLPs, where the
canonical radiance field takes as input the spatial coordinates
$\mathrm{x}\in\mathbb{R}^{3}$ and the unit view-in directions
$d\in\mathbb{R}^{3}$, and the displacement field takes as input the space-time
coordinates $(x,t)$. The output of these MLPs is the RGB colors
$c(x,d)\in\mathbb{R}^{3}$ and space occupancy $\sigma(x)\in\mathbb{R}$ for the
canonical radiance field and the displacement vector at point $x$ and time $t$
for the displacement field.

With increasing volumes of biomedical data, efficient compression methods are
needed for storage, transmission, and secure sharing. While compression
techniques for natural image/video data exist, they are not effective for
biomedical data due to their unique characteristics. Biomedical data contains
diverse tissue types, complex structures, and high-resolution details, which
pose challenges for conventional compression techniques. In recent years,
target-data-specific approaches like INR have shown promise in effectively
compressing diverse visual data.

For instance, Yang _et al_. [67] presented a mathematical interpretation and
adaptive partitioning for the design of an INR-based compressor called SCI.
SCI partitions the data into blocks, and each block is compressed separately
using an MLP network. The first layer is allocated a wide set of neurons to
capture a broader range of frequencies, with a proportional reduction of layer
size with increasing depth. This choice is based on the observation that
increasing the depth of the network rather than the width (number of neurons)
is more efficient to represent a larger range of frequencies or higher-order
harmonics. To maintain high reconstruction fidelity, the allocation of
parameters to blocks is done in accordance with the range of frequencies they
cover. After the compression is completed for each block, the network
parameters, including the weights and biases of the neural network that
contains the learned representations and encoding information for that
specific block, are serialized. Using the HiP-CT dataset [57] as a testbed,
Yang _et al_. found that their method outperformed conventional techniques
(JPEG, H.264, HEVC), data-driven techniques (DVC, SGA+BB, SSF), and existing
INR-based techniques (SIREN [50], NeRF [34], and NeRV [44]) on a wide variety
of biological and medical data.

In another attempt to improve the compression fidelity of INR, a Tree-
structured Implicit Neural Compression (TINC) was proposed [66]. TINC uses
MLPs to fit segmented local regions, and these MLPs are arranged in a tree
structure to enable parameter sharing based on spatial distance. The
parameter-sharing mechanism ensures a smooth transition between neighboring
regions and eliminates redundancy, whether it exists locally or non-locally.
The experiments on the HiP-CT dataset [57] demonstrate the superiority of TINC
over conventional techniques. However, its limitations, similar to those of
other INR-based methods, are its slower compression speed, although its
decompression speed is high.

###  4.5 Neural Rendering

Neural rendering refers to a class of approaches that involve training a
neural network to model the complex relationships between scene geometry,
lighting, and details, which allows for the generation of novel views based on
existing scenes. Implicit representations can be applied in the context of
neural rendering of medical images, allowing for the creation of more detailed
and accurate visualizations of complex anatomical structures and other medical
data.

In 3D CT imaging, the long exposure time of patients to harmful ionizing
radiation imposes a noticeable challenge. Consequently, to alleviate this
problem, MedNeRF [16] proposes to incorporate GRAF [47] (which integrates NeRF
[34] with a CNN) to render CT projections from a single or multi-view X-rays.
The intention behind GRAF boils down to NeRF struggling to handle complex
scenes with large amounts of geometric complexity. To handle this limitation,
the NeRF is trained to minimize the difference between the rendered and ground
truth images, while the GAN [21] is trained to distinguish between the
generated image and a ground truth image, and utilized to refine the NeRF
outputs and improve image quality. The conducted evaluations of MedNeRF on
X-ray chest and knee datasets demonstrate reconstruction improvements in terms
of volumetric depth estimation compared to the neural radiance field methods.

The application of neural rendering for the purpose of reconstructing surgical
scenes in 3D was first introduced by Wang _et al_. [58]. As shown in the
Figure 7, the proposed method (Surgical Neural Rendering) employs Implicit
Neural Representations (INRs) to capture the dynamic and deformable nature of
surgical scenes through a canonical radiance field and a time-dependent
displacement field, represented using an MLP that maps coordinates and view-in
directions to RGB colors and space occupancy. By making the volume rendering
process differentiable, it becomes possible to backpropagate gradients through
the rendering operations, allowing for end-to-end learning of the implicit
neural fields, and enabling the optimization of these parameters to
reconstruct the surgical scenes. To generate renderings for supervision, the
approach utilizes differentiable volume rendering, where camera rays are shot
into the scene, and the color and optical depth of each ray are evaluated
using the volume rendering integral. Sampled points along the rays provide the
necessary inputs to obtain color and space occupancy from the neural fields.
The network parameters of the implicit neural fields are optimized to
reconstruct the shapes, colors, and deformations of the surgical scene. This
optimization is achieved by jointly supervising the rendered color and optical
depth with ground-truth data.

Table 1: A comparison of different proposed methods in different medical
imaging task. The table is divided into different sections based on the
imaging task, including Reconstruction, Segmentation, Neural Rendering,
Compression, and Registration.

4 silver Application | Method | Input | Output |  Activation Function | Modality | Highlight | Ref
---|---|---|---|---|---|---|---
4 llightgreen lightgreen | NeRP | $(x,y,z)$ | Intensity | ReLU |  MRI CT |  Reconstructs high-quality CT and MRI images from sparsely sampled measurements by embedding a prior image from an earlier scan, training the network to learn the neural representation of the target image, and inferring the trained network to generate the final reconstructed image. |  white[48]
llightgreen lightgreen | DCTR | $(x,y,z)$ |  Linear Attenuation Coefficient | ReLU | CT |  Calculating the loss between the new CT measurement and the measurement taken in the previous time step, then propagates the loss back to the MLP in order to remove noise from the new scan and reconstruct its CT image. |  white[40]
llightgreen lightgreen | IREM | $(x,y,z)$ | Intensity | ReLU | MRI |  Generates 3D MRI volumes by fitting implicit neural representations on sagittal, coronal, and axial planes of the brain to reconstruct the vacant voxels between them. |  white[62]
llightgreen lightgreen | ArSSR |  $(x,y,z)$ Image Embedding | Intensity | ReLU | MRI |  represents the 3D brain MRI surface using high-resolution (HR) and low-resolution (LR) image pairs. The LR image is processed through task-specific super-resolution CNN to extract features, which are concatenated with the HR image coordinates. This concatenated input is then used by a decoder MLP to generate the 3D voxel intensity. |  white[61]
llightgreen lightgreen | INR-LCS |  $(x,y,z)$ $\tau$ (Temporal variable) $z$ (latent code) | SDF | Sine |  Fluorescent Microscopy |  $\bullet$ Uses an MLP to represent the signed distance function (SDF) of evolving cell shapes at any point given its spatial coordinates (x, y, z), a temporal parameter (t), and a conditioning latent code that provides additional information and influences the generated cell shapes.
$\bullet$ Can aid researchers in studying subjects such as cell division, migration, or modifications in cellular structures. |  white[59]
llightgreen lightgreenReconstruction | CoiL | ($\theta,l$) | Response ($r$) | ReLU | CT |  By utilizing the viewing angle $\theta$ and the spatial location $l$ of the relevant detector on the sensor plane as inputs, an MLP learns a mapping to sensor responses (CT measurement) in order to represent the measurement field of the organ of interest. |  white[54]
llightpurple lightpurple | BS-ISR | $(x,y)$ | Spline Coefficient | ReLU | CT |  Used a combination of INR and CNNs to model the segmentation boundary by mapping CT slice coordinates to spline coefficients. |  white[9]
llightpurple lightpurple | NeRD | $(d_{t},d_{r},d_{b},d_{l})$ | $(\mu,\Sigma)$ | ReLU | MRI |  Addressed the spatial invariance challenge caused by pooling and padding operations by using a pixel-wise 4D position vector (distance from top, right, bottom, and left) as input to train an MLP representing pixel-wise distributation in order to generate the mean and covariance matrix of that pixel. |  white[70]
llightpurple lightpurpleSegmentation | Retinal INR | $(x,y)$ | RGB Value | ✗ |  Fundus Photography |  An INR model enhances the retinal image resolution, while a Vision Transformer (ViT) conducts self-distillation on the original image to extract the key features. These features are utilized for the segmentation of retinal vessels, therefore aiding in the detection of ocular diseases. |  white[22]
llightpink lightpink | MedNeRF | $(x,y,z,\theta,\phi)$ | (RGB, $\sigma$) | ReLU | CT |  Combines NeRF and a CNN (inspired by GRAF [47]) to generate CT projections from X-rays by training NeRF as the generator to output image patches and a CNN as the discriminator to refine NeRF outputs, enhancing image quality and addressing NeRF’s struggles with complex scenes. |  white[16]
llightpink lightpink |  Surgical Neural Rendering | $(x,y,z,\theta,\phi)$ | (RGB, $\sigma$) | ReLU |  Endoscopic imaging |  NeRF-based rendering in robotic surgery that captures non-rigid deformations and reconstruct the 3D structures of the surgical scenes. from single-viewpoint stereo endoscopes. It handles occlusion caused by surgical instruments by utilizing a canonical radiance field and a time-dependent displacement field. The canonical field maps coordinates and viewing directions to colors and space occupancy, while the displacement field maps input space-time coordinates to displacement vectors. |  white[58]
llightpink lightpinkNeural Rendering | SNAF | $(x,y,z)$ |  Attenuation Coefficient | ReLU | CBCT |  $\bullet$ Neural rendering is utilized as part of the CBCT reconstruction process to enhance the quality of the rendered CBCT images by learning neural attenuation fields using a multi-resolution hash table and employing volume rendering, high-quality CBCT images are generated from sparse 2D projections.
$\bullet$ The deblurring network takes as input a rendered novel view from the learned neural attenuation field, along with its neighboring views to mitigate the blurring effect resulting from limited input projections. |  white[18]
llyellow lyellow | SCI |  $(x,y,z)$ |  Compressed Representation | Sine | CT |  $\bullet$ Mitigates INR’s limitations on broad-spectrum data by introducing adaptive partitioning, which divides the data into blocks within INR’s spectrum envelop and compresses each block using a funnel-shaped neural network architecture (wider beginning and narrower end) to capture its spectrum and characteristics, resulting in compressed data obtained through parameter optimization and serialization.
$\bullet$ Their findings demonstrate that INR struggles to accurately represent data with a wide range of frequencies, impacting its fidelity in capturing diverse spectral components. |  white[68]
llyellow lyellowCompression | TINC | $(x,y,z)$ |  Compressed Representation | Sine | Phase-Contrast CT |  Uses octree partitioning to enable visually similar blocks to share parameters within a tree-shaped neural network structure, enhancing representation compactness and resulting in a more efficient and concise architecture, achieved through an MLP-based implicit neural function representation of each equal-sized block of target data. |  white[66]
lskyblue skyblue | IDIR | $(x,y,z)$ | Deformation Vector | Sine | CT |  Proposes an alternative approach to image registration by optimizing a MLP instead of a CNN. The authors leverage insights from differentiable rendering to demonstrate how an implicit deformable image registration model can be combined with regularization terms using automatic differentiation techniques. |  white[60]
lskyblue skyblueRegistration | mirnf | $(x,y,z)$ |  Displacement Vector Velocity Vector | Sine | MRI |  A novel framework that combines optimization with deep neural networks for image registration. It utilizes neural fields to represents the transformation between pair of images, offering two methods for generating deformation fields. The optimal registration is achieved through parameters of neural field update via stochastic gradient descent. The paper also discusses ways to enhance model optimization. |  white[53]
4 |  |

##  5 Comparative Overview

To provide a comparative overview, we have organized comparative information
and findings in the Table 1. According to the table, it is evident that image
reconstruction has attracted more interest than tasks like segmentation,
compression, registration, and others. This preference is mainly driven by
their great ability to enhance resolution and reduce noise, especially in
medical scenarios where the imaging device is prone to uncertainty. We discuss
and compare noteworthy elements in the following:

Defining Parameters: The parameters used as the input to INR are not always
cartesian coordinates and depend on the task and the signal distribution that
the neural network is defining. For instance, CoiL [54] tried to define the
measurement field by using the parameters that characterize a sensor response
including the viewing angle and spatial location of the detector. Likewise,
NeRD [70] used the positional distance in cardinal directions to define the
pixel-wise distribution function.

Local Information: It is worth noting that the methods employing CNNs, such as
ArSSR [61], BS-ISR [9], and MedNeRF [16], use specifically leverage the power
of CNNs to incorporate local semantic information during the representation
process. By utilizing the convolutional layers, these methods can capture and
encode local features and spatial relationships, enabling more accurate and
context-aware representations for tasks such as noise removal, boundary
modeling, and super-resolution.

Sparse View CT Reconstruction: As aforementioned in section 3, reducing
exposure of patients to radiation dose plays a significant role in improving
health care systems. As a result, a notable number of works have developed
various strategies to reconstruct CT images with sparse and limited
measurements and projection data. Both NeRP [48] and CoiL [54] address the
challenge of sparse CT reconstruction by leveraging prior information or
geometric relationships. DCTR [40] addresses this challenge in the context of
dynamic 4D-CT reconstruction, which is suited for moving structures, such as
organs affected by respiration or cardiac motion. In cone-beam computed
tomography (CBCT), only the area of interest is exposed to radiation, which
reduces radiation exposure to surrounding tissues and organs. SNAF [18]
studied reconstructing scans for this special medical imaging technique by
utilizing a neural rendering method to implicitly learn an attenuation field.
However, due to the use of limited input projections, the resulting outputs
are blurry and require additional effort. It’s important to note that sparse
view reconstruction is a technique that trades off radiation dose reduction
with the potential loss of image quality and accuracy, which is why INR
gathered a lot of research attention in this field.

Network type: SIREN-based vs NeRF-based: Most of the works reviewed are using
ReLU MLPs with Fourier mapping applied to their input to mitigate spectral
bias. Since neural volume rendering is based on NeRF [34] design for view
synthesis and continuous representation, the activation function is ReLU with
Fourier features as input to model 3D structure of the scene accurately [16,
58, 18]. Nonetheless, volume rendering in medical scenarios differs in terms
of the surface boundary, as the entire organ holds valuable diagnostic
information compared to other domains using NeRF. The type of network is
influenced by the objective of the task it aims to solve. Higher-order
differentiability of periodic activations enables incorporating more advanced
regularization terms into the optimization process of the registration, such
as Jacobian regularizer, hyperelastic regularizer [10], and bending energy
penalty [41], which is used in IDIR [60] method.

##  6 Future Work and Open Challenges

Despite the benefits of INRs, particularly in the field of medicine, they are
still limited in various aspects and require further research efforts to
become viable for practical applications, given the high-stakes nature of the
medical domain. We discuss these limitations briefly in the following.

Computational complexity and training time: Learning a neural representation
for each signal separately involves a considerable amount of memory and
computational resources. Furthermore, fitting an INR for applications
involving high-dimensional data like 3D volumes can be time-consuming [42].
This can pose challenges for real-time applications that require immediate
responses. The complexity arises from factors such as the size of the input
data and the model architecture. Meta-learning and multi-scale representations
help accelerate training time and optimize memory utilization in several
domains [20, 43], which provide pathways for representing anatomical and
biological structures with reduced training time and greater practicality.

Scaling to more complex signals: To better represent higher-resolution signals
or complex 3D shapes with fine detail can be challenging. The mapping involved
in such representations is often highly nonlinear, making it difficult to
scale up without incurring significant computational costs. Both widening and
deepening the MLP can enhance its representation capability, but the
backpropagation algorithm used for training deep neural networks becomes more
computationally intensive as the depth increases, and the vanishing/exploding
gradient problem may arise. Researchers often need to strike a balance between
model complexity and available computational resources. Various techniques
have been developed [14, 35, 25].

Video-based INR: When it comes to decoding time, video compression methods
employing INR is better compared to other models [13]. This functionality
allows parallel processing in feed-forwarding, enabling the independent
computation of each frame during decoding. As a result, they got the most
attention in robotic-assisted surgery where both speed and accuracy are
critical [46, 45, 58]. However, modeling semantic relationships between frames
in high-frequency videos (i.e., high frame rate) presents considerable
challenges [73], and ongoing research and development are crucial to fully
utilize the potential of INRs in this field.

##  7 Conclusion

In conclusion, this survey has offered a comprehensive overview of INRs within
the realm of medical imaging. Through the utilization of neural networks and
implicit continuous functions, INRs have demonstrated substantial potential in
tackling complex issues within medical settings. The survey has emphasized the
benefits of employing INRs and has delved into their application across
various medical imaging tasks. Additionally, it has identified open challenges
and areas of future research, providing valuable insights for researchers in
the field.

## References

  * [1] Arya Aftab, Alireza Morsali, and Shahrokh Ghaemmaghami.  Multi-head relu implicit neural representation networks.  In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2510–2514. IEEE, 2022.
  * [2] Dieuwertje Alblas, Marieke Hofman, Christoph Brune, Kak Khee Yeung, and Jelmer M Wolterink.  Implicit neural representations for modeling of abdominal aortic aneurysm progression.  In International Conference on Functional Imaging and Modeling of the Heart, pages 356–365. Springer, 2023.
  * [3] Fouzia Altaf, Syed MS Islam, Naveed Akhtar, and Naeem Khalid Janjua.  Going deep in medical image analysis: concepts, methods, challenges, and future directions.  IEEE Access, 7:99540–99572, 2019.
  * [4] Amirhossein Aminimehr, Pouya Khani, Amirali Molaei, Amirmohammad Kazemeini, and Erik Cambria.  Tbexplain: A text-based explanation method for scene classification models with the statistical prediction correction.  arXiv preprint arXiv:2307.10003, 2023.
  * [5] Amirhossein Aminimehr, Amirali Molaei, and Erik Cambria.  Entri: Ensemble learning with tri-level representations for explainable scene recognition.  arXiv preprint arXiv:2307.12442, 2023.
  * [6] Vegard Antun, Francesco Renna, Clarice Poon, Ben Adcock, and Anders C Hansen.  On instabilities of deep learning in image reconstruction and the potential costs of ai.  Proceedings of the National Academy of Sciences, 117(48):30088–30095, 2020.
  * [7] Reza Azad, Ehsan Khodapanah Aghdam, Amelie Rauland, Yiwei Jia, Atlas Haddadi Avval, Afshin Bozorgpour, Sanaz Karimijafarbigloo, Joseph Paul Cohen, Ehsan Adeli, and Dorit Merhof.  Medical image segmentation review: The success of u-net.  arXiv preprint arXiv:2211.14830, 2022.
  * [8] Reza Azad, Amirhossein Kazerouni, Moein Heidari, Ehsan Khodapanah Aghdam, Amirali Molaei, Yiwei Jia, Abin Jose, Rijo Roy, and Dorit Merhof.  Advances in medical image analysis with vision transformers: A comprehensive review.  arXiv preprint arXiv:2301.03505, 2023.
  * [9] Oliver JD Barrowclough, Georg Muntingh, Varatharajan Nainamalai, and Ivar Stangeby.  Binary segmentation of medical images using implicit spline representations and deep learning.  Computer Aided Geometric Design, 85:101972, 2021.
  * [10] Martin Burger, Jan Modersitzki, and Lars Ruthotto.  A hyperelastic regularization energy for image registration.  SIAM Journal on Scientific Computing, 35(1):B132–B148, 2013.
  * [11] Richard Castillo, Edward Castillo, Rudy Guerra, Valen E Johnson, Travis McPhail, Amit K Garg, and Thomas Guerrero.  A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets.  Physics in Medicine & Biology, 54(7):1849, 2009.
  * [12] Rohan Chabra, Jan E Lenssen, Eddy Ilg, Tanner Schmidt, Julian Straub, Steven Lovegrove, and Richard Newcombe.  Deep local shapes: Learning local sdf priors for detailed 3d reconstruction.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16, pages 608–625. Springer, 2020.
  * [13] Hao Chen, Bo He, Hanyu Wang, Yixuan Ren, Ser Nam Lim, and Abhinav Shrivastava.  Nerv: Neural representations for videos.  Advances in Neural Information Processing Systems, 34:21557–21568, 2021.
  * [14] Zhiqin Chen, Thomas Funkhouser, Peter Hedman, and Andrea Tagliasacchi.  Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16569–16578, 2023.
  * [15] Zhiqin Chen and Hao Zhang.  Learning implicit fields for generative shape modeling.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5939–5948, 2019.
  * [16] Abril Corona-Figueroa, Jonathan Frawley, Sam Bond-Taylor, Sarath Bethapudi, Hubert PH Shum, and Chris G Willcocks.  Mednerf: Medical neural radiance fields for reconstructing 3d-aware ct-projections from a single x-ray.  In 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pages 3843–3848. IEEE, 2022\.
  * [17] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [18] Yu Fang, Lanzhuju Mei, Changjian Li, Yuan Liu, Wenping Wang, Zhiming Cui, and Dinggang Shen.  Snaf: Sparse-view cbct reconstruction with neural attenuation fields.  arXiv preprint arXiv:2211.17048, 2022.
  * [19] Muhammad Moazam Fraz, Paolo Remagnino, Andreas Hoppe, Bunyarit Uyyanonvara, Alicja R Rudnicka, Christopher G Owen, and Sarah A Barman.  An ensemble classification-based approach applied to retinal blood vessel segmentation.  IEEE Transactions on Biomedical Engineering, 59(9):2538–2548, 2012\.
  * [20] Harry Gao, Weijie Gan, Zhixin Sun, and Ulugbek S Kamilov.  Sinco: A novel structural regularizer for image compression using implicit neural representations.  In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023.
  * [21] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  Advances in neural information processing systems, 27, 2014.
  * [22] Jia Gu, Fangzheng Tian, and Il-Seok Oh.  Retinal vessel segmentation based on self-distillation and implicit neural representation.  Applied Intelligence, pages 1–18, 2022.
  * [23] Zekun Hao, Arun Mallya, Serge Belongie, and Ming-Yu Liu.  Implicit neural representations with levels-of-experts.  Advances in Neural Information Processing Systems, 35:2564–2576, 2022.
  * [24] Mohammad Hesam Hesamian, Wenjing Jia, Xiangjian He, and Paul Kennedy.  Deep learning techniques for medical image segmentation: achievements and challenges.  Journal of digital imaging, 32:582–596, 2019.
  * [25] Milad Soltany Kadarvish, Hesam Mojtahedi, Hossein Entezari Zarch, Amirhossein Kazerouni, Alireza Morsali, Azra Abtahi, and Farokh Marvasti.  Ensemble neural representation networks.  arXiv preprint arXiv:2110.04124, 2021.
  * [26] James T Kajiya and Brian P Von Herzen.  Ray tracing volume densities.  ACM SIGGRAPH computer graphics, 18(3):165–174, 1984.
  * [27] Amirhossein Kazerouni, Ehsan Khodapanah Aghdam, Moein Heidari, Reza Azad, Mohsen Fayyaz, Ilker Hacihaliloglu, and Dorit Merhof.  Diffusion models in medical imaging: A comprehensive survey.  Medical Image Analysis, page 102846, 2023.
  * [28] Arno Klein and Jason Tourville.  101 labeled brain images and a consistent human cortical labeling protocol.  Frontiers in neuroscience, 6:171, 2012.
  * [29] Jun Li, Xiaojun Guan, Qing Wu, Chenyu He, Weimin Zhang, Xiyue Lin, Chunlei Liu, Hongjiang Wei, Xiaojun Xu, and Yuyao Zhang.  Direct localization and delineation of human pedunculopontine nucleus based on a self-supervised magnetic resonance image super-resolution method.  Human Brain Mapping, 2023.
  * [30] Shaohui Liu, Yinda Zhang, Songyou Peng, Boxin Shi, Marc Pollefeys, and Zhaopeng Cui.  Dist: Rendering deep implicit signed distance function with differentiable sphere tracing.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2019–2028, 2020.
  * [31] Shuangming Mao and Seiichiro Kamata.  Mri super-resolution using implicit neural representation with frequency domain enhancement.  In Proceedings of the 7th International Conference on Biomedical Signal and Image Processing, pages 30–36, 2022.
  * [32] Daniel S Marcus, Tracy H Wang, Jamie Parker, John G Csernansky, John C Morris, and Randy L Buckner.  Open access series of imaging studies (oasis): cross-sectional mri data in young, middle aged, nondemented, and demented older adults.  Journal of cognitive neuroscience, 19(9):1498–1507, 2007.
  * [33] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger.  Occupancy networks: Learning 3d reconstruction in function space.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460–4470, 2019.
  * [34] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng.  Nerf: Representing scenes as neural radiance fields for view synthesis.  Communications of the ACM, 65(1):99–106, 2021.
  * [35] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller.  Instant neural graphics primitives with a multiresolution hash encoding.  ACM Trans. Graph., 41(4):102:1–102:15, July 2022.
  * [36] Yael Niv.  Learning task-state representations.  Nature neuroscience, 22(10):1544–1553, 2019.
  * [37] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove.  Deepsdf: Learning continuous signed distance functions for shape representation.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 165–174, 2019.
  * [38] Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, and Steven M Seitz.  Hypernerf: A higher-dimensional representation for topologically varying neural radiance fields.  arXiv preprint arXiv:2106.13228, 2021.
  * [39] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville.  On the spectral bias of neural networks.  In International Conference on Machine Learning, pages 5301–5310. PMLR, 2019.
  * [40] Albert W Reed, Hyojin Kim, Rushil Anirudh, K Aditya Mohan, Kyle Champley, Jingu Kang, and Suren Jayasuriya.  Dynamic ct reconstruction from limited views with implicit neural representations and parametric motion fields.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2258–2268, 2021.
  * [41] Daniel Rueckert, Luke I Sonoda, Carmel Hayes, Derek LG Hill, Martin O Leach, and David J Hawkes.  Nonrigid registration using free-form deformations: application to breast mr images.  IEEE transactions on medical imaging, 18(8):712–721, 1999.
  * [42] Vishwanath Saragadam, Daniel LeJeune, Jasper Tan, Guha Balakrishnan, Ashok Veeraraghavan, and Richard G Baraniuk.  Wire: Wavelet implicit neural representations.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18507–18516, 2023.
  * [43] Vishwanath Saragadam, Jasper Tan, Guha Balakrishnan, Richard G Baraniuk, and Ashok Veeraraghavan.  Miner: Multiscale implicit neural representation.  In European Conference on Computer Vision, pages 318–333. Springer, 2022.
  * [44] Kripasindhu Sarkar, Dushyant Mehta, Weipeng Xu, Vladislav Golyanik, and Christian Theobalt.  Neural re-rendering of humans from a single image.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, pages 596–613. Springer, 2020.
  * [45] Adam Schmidt, Omid Mohareri, Simon DiMaio, and Septimiu E Salcudean.  Fast graph refinement and implicit neural representation for tissue tracking.  In 2022 International Conference on Robotics and Automation (ICRA), pages 1281–1288. IEEE, 2022.
  * [46] Adam Schmidt, Omid Mohareri, Simon DiMaio, and Septimiu E Salcudean.  Recurrent implicit neural graph for deformable tracking in endoscopic videos.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 478–488. Springer, 2022.
  * [47] Katja Schwarz, Yiyi Liao, Michael Niemeyer, and Andreas Geiger.  Graf: Generative radiance fields for 3d-aware image synthesis.  Advances in Neural Information Processing Systems, 33:20154–20166, 2020.
  * [48] Liyue Shen, John Pauly, and Lei Xing.  Nerp: implicit neural representation learning with prior embedding for sparsely sampled image reconstruction.  IEEE Transactions on Neural Networks and Learning Systems, 2022\.
  * [49] David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al.  A general reinforcement learning algorithm that masters chess, shogi, and go through self-play.  Science, 362(6419):1140–1144, 2018.
  * [50] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein.  Implicit neural representations with periodic activation functions.  Advances in Neural Information Processing Systems, 33:7462–7473, 2020.
  * [51] Jan-Jakob Sonke, Lambert Zijp, Peter Remeijer, and Marcel Van Herk.  Respiratory correlated cone beam ct.  Medical physics, 32(4):1176–1186, 2005.
  * [52] Joes Staal, Michael D Abràmoff, Meindert Niemeijer, Max A Viergever, and Bram Van Ginneken.  Ridge-based vessel segmentation in color images of the retina.  IEEE transactions on medical imaging, 23(4):501–509, 2004.
  * [53] Shanlin Sun, Kun Han, Deying Kong, Chenyu You, and Xiaohui Xie.  Mirnf: Medical image registration via neural fields.  arXiv preprint arXiv:2206.03111, 2022.
  * [54] Yu Sun, Jiaming Liu, Mingyang Xie, Brendt Wohlberg, and Ulugbek S Kamilov.  Coil: Coordinate-based internal learning for imaging inverse problems.  arXiv preprint arXiv:2102.05181, 2021.
  * [55] Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng.  Fourier features let networks learn high frequency functions in low dimensional domains.  Advances in Neural Information Processing Systems, 33:7537–7547, 2020.
  * [56] Jiapeng Tang, Xiaoguang Han, Junyi Pan, Kui Jia, and Xin Tong.  A skeleton-bridged deep learning approach for generating meshes of complex topologies from single rgb images.  In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 4541–4550, 2019.
  * [57] CL Walsh, P Tafforeau, WL Wagner, DJ Jafree, A Bellier, C Werlein, MP Kühnel, E Boller, S Walker-Samuel, JL Robertus, et al.  Imaging intact human organs with local resolution of cellular structures using hierarchical phase-contrast tomography.  Nature methods, 18(12):1532–1541, 2021.
  * [58] Yuehao Wang, Yonghao Long, Siu Hin Fan, and Qi Dou.  Neural rendering for stereo 3d reconstruction of deformable tissues in robotic surgery.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 431–441. Springer, 2022.
  * [59] David Wiesner, Julian Suk, Sven Dummer, David Svoboda, and Jelmer M Wolterink.  Implicit neural representations for generative modeling of living cell shapes.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 58–67. Springer, 2022.
  * [60] Jelmer M Wolterink, Jesse C Zwienenberg, and Christoph Brune.  Implicit neural representations for deformable image registration.  In International Conference on Medical Imaging with Deep Learning, pages 1349–1359. PMLR, 2022.
  * [61] Qing Wu, Yuwei Li, Yawen Sun, Yan Zhou, Hongjiang Wei, Jingyi Yu, and Yuyao Zhang.  An arbitrary scale super-resolution approach for 3d mr images via implicit neural representation.  IEEE Journal of Biomedical and Health Informatics, 27(2):1004–1015, 2022.
  * [62] Qing Wu, Yuwei Li, Lan Xu, Ruiming Feng, Hongjiang Wei, Qing Yang, Boliang Yu, Xiaozhao Liu, Jingyi Yu, and Yuyao Zhang.  Irem: high-resolution magnetic resonance image reconstruction via implicit neural representation.  In Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part VI 24, pages 65–74. Springer, 2021.
  * [63] Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar.  Neural fields in visual computing and beyond.  In Computer Graphics Forum, volume 41, pages 641–676. Wiley Online Library, 2022.
  * [64] Junshen Xu, Daniel Moyer, Borjan Gagoski, Juan Eugenio Iglesias, P Ellen Grant, Polina Golland, and Elfar Adalsteinsson.  Nesvor: Implicit neural representation for slice-to-volume reconstruction in mri.  IEEE Transactions on Medical Imaging, 2023.
  * [65] Xiaowei Xu, Tianchen Wang, Jian Zhuang, Haiyun Yuan, Meiping Huang, Jianzheng Cen, Qianjun Jia, Yuhao Dong, and Yiyu Shi.  Imagechd: A 3d computed tomography image dataset for classification of congenital heart disease.  In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 77–87. Springer, 2020.
  * [66] Runzhao Yang.  Tinc: Tree-structured implicit neural compression.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18517–18526, 2023.
  * [67] Runzhao Yang, Tingxiong Xiao, Yuxiao Cheng, Qianni Cao, Jinyuan Qu, Jinli Suo, and Qionghai Dai.  Sci: A spectrum concentrated implicit neural compression for biomedical data.  arXiv preprint arXiv:2209.15180, 2022.
  * [68] Runzhao Yang, Tingxiong Xiao, Yuxiao Cheng, Qianni Cao, Jinyuan Qu, Jinli Suo, and Qionghai Dai.  Sci: A spectrum concentrated implicit neural compression for biomedical data.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 4774–4782, 2023.
  * [69] Guangming Zang, Ramzi Idoughi, Rui Li, Peter Wonka, and Wolfgang Heidrich.  Intratomo: self-supervised learning-based tomography via sinogram synthesis and prediction.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1960–1970, 2021.
  * [70] Hang Zhang, Rongguang Wang, Jinwei Zhang, Chao Li, Gufeng Yang, Pascal Spincemaille, Thanh Nguyen, and Yi Wang.  Nerd: Neural representation of distribution for medical image segmentation.  arXiv preprint arXiv:2103.04020, 2021.
  * [71] Heng Zhang, Lifeng Zhu, Jiangwei Shen, and Aiguo Song.  Implicit neural field guidance for teleoperated robot-assisted surgery.  In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 6866–6872. IEEE, 2023.
  * [72] You Zhang, Hua-Chieh Shao, Tinsu Pan, and Tielige Mengke.  Dynamic cone-beam ct reconstruction using spatial and temporal implicit neural representation learning (stinr).  Physics in Medicine and Biology, 2023.
  * [73] Qi Zhao, M Salman Asif, and Zhan Ma.  Dnerv: Modeling inherent dynamics via difference neural representation for videos.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2031–2040, 2023.