[orcid=0000-0002-8428-288X] [cor1]Corresponding author: Yi Wang
<EMAIL_ADDRESS>
# A Review of Image Processing Methods in Prostate Ultrasound

Haiqiao Wang  Hong Wu  Zhuoyuan Wang  Peiyan Yue  Dong Ni  Pheng-Ann Heng  Yi
Wang  Medical UltraSound Image Computing (MUSIC) Lab, Smart Medical Imaging,
Learning and Engineering (SMILE) Lab, School of Biomedical Engineering,
Shenzhen University Medical School, Shenzhen University, Shenzhen, China
Department of Computer Science and Engineering, The Chinese University of Hong
Kong, Hong Kong, China

###### Abstract

Prostate cancer (PCa) poses a significant threat to men’s health, with early
diagnosis being crucial for improving prognosis and reducing mortality rates.
Transrectal ultrasound (TRUS) plays a vital role in the diagnosis and image-
guided intervention of PCa. To facilitate physicians with more accurate and
efficient computer-assisted diagnosis and interventions, many image processing
algorithms in TRUS have been proposed and achieved state-of-the-art
performance in several tasks, including prostate gland segmentation, prostate
image registration, PCa classification and detection, and interventional
needle detection. The rapid development of these algorithms over the past two
decades necessitates a comprehensive summary. In consequence, this survey
provides a systematic analysis of this field, outlining the evolution of image
processing methods in the context of TRUS image analysis and meanwhile
highlighting their relevant contributions. Furthermore, this survey discusses
current challenges, and suggests future research directions to possibly
advance this field further.

###### keywords:

Transrectal ultrasound Prostate cancer Medical image processingDeep learning
Machine learning Medical image segmentation Medical image
registrationClassification Computer assisted detectionComputer assisted
diagnosis

##  1 Introduction

Prostate cancer (PCa) is a malignant disease of the prostate gland. According
to (Siegel et al.,, 2023; Xia et al.,, 2022), in 2022, the number of PCa
patients ranked second among male cancer patients worldwide, accounting for
14.2%, while the number of deaths from PCa ranked fifth among male cancer
patients, accounting for 7.3%. Early diagnosis and treatment is the crucial
key to the cure of progressive PCa. If this can not be carried out
effectively, PCa can invade nearby tissues and spread to other parts of the
body, such as bones, seminal vesicles, and the rectum (Rawla,, 2019). Such
spread not only severely affects the patient’s quality of life but also
increases the risk of death. Therefore, early diagnosis for PCa is crucial for
improving the success rate of treatment and reducing mortality.

The diagnostic procedure for PCa requires the comprehensive use of various
detection tools to ensure accuracy. Firstly, digital rectal examination (DRE)
is a traditional method for preliminary assessment of the prostate condition,
where the size, shape, and texture of the prostate are checked through rectal
palpation (Naji et al.,, 2018). Secondly, prostate specific antigen (PSA)
blood testing is an important screening tool that assesses the risk of
prostate disease by measuring the level of PSA in the blood, despite the
possibility of its elevation due to non-cancerous reasons (Lilja et al.,,
2008). Additionally, imaging examinations, including ultrasound, computed
tomography (CT) and magnetic resonance imaging (MRI), provide physicians with
more detailed views of the prostate structure (Hricak et al.,, 2007), with
multi-parametric MRI (mpMRI) offering higher resolution images that help to
more accurately locate tumors. Ultimately, prostate biopsy is a key step in
confirming prostate cancer, determining the nature of the tumor through
pathological analysis. However, early cancerous lesions are often small and
difficult to accurately locate and obtain sufficient samples in one biopsy,
usually requiring multiple attempts. Multiple biopsies not only increase
patient discomfort but can also lead to complications such as prostatitis or
urinary tract infections (Borghesi et al.,, 2017). To reduce this over-
diagnosis phenomenon, additional imaging guidance for targeted prostate biopsy
is essential (Brown et al.,, 2015), such as MRI and ultrasound.

Transrectal ultrasound (TRUS) guided prostate biopsy is the standard method
for the diagnosis of PCa, which is widely used in clinical practice (Ahmed et
al.,, 2017). TRUS scans the prostate through the rectal wall, generating
images of the prostate, including B-mode, color Doppler, 3D imaging, and
elastography, to meet various clinical needs. The real-time B-mode provides
two-dimensional images commonly used for the diagnosis of benign prostate
hyperplasia (BPH) and prostatitis, and intraoperative guidance. While 3D TRUS
generates comprehensive 3D volumetric information that is particularly
important for preoperative planning. Color Doppler TRUS provides blood flow
information, which helps identifying blood vessels around tumors. TRUS
elastography assesses tissue stiffness, aiding in the early detection of PCa.

Here are several key clinical tasks where TRUS is needed as imaging tool for
the diagnosis and treatment of PCa: 1) PCa classification and detection: TRUS
can help physicians identify different types of PCa, including localized and
invasive PCa. Through TRUS images, physicians can roughly assess the size,
location, and invasion of tumors into surrounding tissues. 2) Prostate gland
segmentation: accurate prostate boundary delineation from TRUS images can
provide the target area for interventions, which facilitates physicians with
effective treatment planning, biopsy needle placement, and brachytherapy. 3)
Image guidance for prostate intervention: TRUS is the routine imaging tool for
image-guided biopsy and therapy of PCa, due to its real-time advantage.
Moreover, other preoperative imaging modalities such as MRI can be
registered/fused with TRUS to provide more comprehensive visual information,
thereby improving the efficacy of the intervention. 4) Intraoperative needle
localization: during the procedure of biopsy, brachytherapy, cryotherapy and
so on, TRUS is used to monitor the position of the biopsy or treatment needle,
ensuring the accuracy and safety of the intervention.

Although TRUS has been widely used for the imaging of prostate, there are
still limitations in practical applications. The image quality may be degraded
by speckle noise and shadow, leading to blurred details and artifacts.
Especially in the apex and base regions of the prostate, the gland’s contour
in TRUS is often unclear, increasing the complexity of diagnosis. In addition,
the interpretation of TRUS depends on the experience and skill of the
physicians, leading to subjectivity. Moreover, the modalities and dimensions
of TRUS are various, making it difficult for physicians to quickly obtain
potentially useful information during diagnosis and treatment. Therefore,
applying image processing technology to analyze TRUS is with clinical
significance.

Figure 1: Illustration of conventional image processing tasks on transrectal
ultrasound (TRUS), including (a) prostate gland segmentation, (b) prostate
image registration, (c) prostate cancer (PCa) classification and detection,
and (d) interventional needle detection.

The development of image processing technology has greatly promoted the
progress of the medical imaging field (Shen et al.,, 2017; Litjens et al.,,
2017; Duncan and Ayache,, 2000), especially playing an important role in
computer assisted diagnosis (CAD) and computer assisted intervention (CAI).
With the help of advanced image processing methods, it is beneficial for the
physicians to make more accurate and efficient clinical decisions. In the
field of TRUS image processing, lots of relevant methods have been developed
and applied in different tasks, mainly including prostate gland segmentation,
prostate image registration, PCa classification and detection, and
interventional needle detection, as illustrated in Fig. 1.

This survey summarizes the related image processing methods in TRUS over the
past 20 years, including gland segmentation, image registration, PCa
classification and detection, and interventional needle detection. We classify
and summarize these methods according to modality, dimension, and technical
route, presenting the similarities and differences of these methods. We also
list the primary evaluation metrics for different tasks, as well as the
results of representative methods under their datasets and metrics, making a
rough comparison of different methods. Finally, we also summarize these
methods to propose current challenges faced by each task and possible future
research directions.

The organization of this survey is as follows. In Section 2, we define each
task of TRUS image processing, and list the definitions and formulas of their
primary evaluation metrics. Section 3 is the main part of this survey, where
we review and summarize the methods of prostate gland segmentation, prostate
image registration, PCa classification and detection, and needle detection in
Section 3.1, 3.2, 3.3 and 3.4, respectively. In Section 4, we discuss and
analyze existing challenges and suggest future research directions for these
four tasks. A summary of the entire survey is provided in Section 5.

##  2 Task Definition

###  2.1 Prostate Gland Segmentation

Image segmentation is one of the most common task in the field of medical
image processing. The purpose of prostate segmentation is to accurately obtain
the contour/region of the prostate gland, see Fig. 1(a). Prostate segmentation
plays a crucial role in different stages of the clinical decision-making
process. For example, prostate volume, which can be directly determined
through prostate segmentation, aids in the diagnosis of BPH. Moreover, the
boundary information of the prostate is important and useful in various
treatment scenarios, such as prostate brachytherapy, high-intensity focused
ultrasound (HIFU) therapy, cryotherapy, and transurethral microwave therapy.
In addition, prostate segmentation also facilitates other image processing
tasks, such as surface-based multi-modality image registration. To achieve
prostate segmentation, lots of shape-/region-based methods, as well as
conventional machine learning methods and advanced deep learning methods have
been developed. We classify and summarize these segmentation methods in
Section 3.1, according to TRUS modality, dimension, and technical route.

####  2.1.1 Evaluation Metrics

The evaluation of segmentation accuracy can be divided into qualitative and
quantitative evaluations. For qualitative evaluation, the resulting contours
are visually compared to the ground truth. For quantitative evaluation, the
numerical calculation involves measuring the similarity between the obtained
contours and the ground truth. The commonly used quantitative evaluation
metrics consist of region-based metrics, boundary-based metrics, and
classification-based metrics. Since traditional segmentation methods often
rely on contour constraints, they always employ boundary-based metrics for the
evaluation. In contrast, recent deep learning methods usually use region-based
metrics.

Dice similarity coefficient (DSC) is currently the most popular region-based
evaluation metric for the segmentation task, which measures the overlap
between the segmented result and the ground truth:

$\text{DSC}=\frac{2\times TP}{FP+FN+2\times TP},$ (1)

where TP, FP, FN denotes true positive, false positive, and false negative
regions, respectively.

Intersection over union (IoU) is another commonly used region-based metric,
also known as the Jaccard coefficient. It is defined as follows:

$\text{IoU}=\text{Jaccard}=\frac{TP}{TP+FP+FN}.$ (2)

Larger values of DSC and IoU indicate better segmentation accuracy.

As for the boundary-based metrics, mean surface distance (MSD), mean absolute
distance (MAD), and average symmetric surface distance (ASD) are often
employed to measure the similarity between the segmented boundary (in
2D)/surface (in 3D) and the ground truth:

$\text{MSD}=\frac{1}{N}\sum_{i=1}^{N}d_{i},$ (3)

$\text{MAD}=\frac{1}{N}\sum^{N}_{i=1}|d_{i}|,$ (4)

$\text{ASD}=\frac{1}{N}\sum^{N}_{i=1}(|d_{i}|+|d_{i}^{\prime}|),$ (5)

where $N$ is the total number of points from the segmented boundary/surface.
$d_{i}$ represents the shortest distance from the i-th point on the
segmentation result to the ground truth, while $d_{i}^{\prime}$ represents the
shortest distance from the i-th point on the ground truth to the segmentation
result. It can be seen that MSD, MAD, and ASD measures the average distance,
mean absolute distance, and average symmetric distance, respectively.

In contrast to MSD, MAD, and ASD, maximum distance (MAXD) and Hausdorff
distance (HD) are used to represent the dissimilarity between the segmented
boundary/surface and the ground truth:

$\text{MAXD}=\max|d_{i}|,$ (6)

$\text{HD}=\max\left(\max|d_{i}|,\max|d_{i}^{\prime}|\right).$ (7)

MAXD measures the maximum distance between each point from the segmentation
result and its corresponding point from the ground truth. HD measures the
maximum distance between the segmentation result and ground truth, and with
95HD denoting the 95$th$ percentile of HD. For all aforementioned boundary-
based metrics, smaller values denotes better segmentation accuracy.

In addition to region-/boundary-based metrics, point-wise classification-based
metrics can also be applied to measure segmentation accuracy. Accuracy (ACC)
represents the proportion of correctly predicted pixels to the total number of
pixels in the segmentation result:

$\text{ACC}=\frac{TP+TN}{TP+TN+FP+FN},$ (8)

where TN denotes true negative. Sensitivity and Recall indicate the proportion
of all positive pixels that are correctly segmented as true positive:

$\text{Sensitivity}=\text{Recall}=\frac{TP}{TP+FN}.$ (9)

Precision indicates the proportion of all predicted positive pixels that are
TP pixels:

$\text{Precision}=\frac{TP}{TP+FP}.$ (10)

There is usually a trade-off between precision and sensitivity. Specificity is
the proportion of all negative pixels that are correctly segmented as true
negative:

$\text{Specificity}=\frac{TN}{TN+FP}.$ (11)

Larger values of these classification-based metrics indicate better
segmentation.

###  2.2 Prostate Image Registration

Image registration is the process of aligning two or more images acquired at
varying times, viewpoints, or by different sensors. Taking registering a pair
of images as an example, one is denoted as fixed image and the other as moving
image. The objective is to estimate the optimal deformation field between the
fixed and moving images, thus the warped moving image can be matched with the
fixed image, enabling the alignment of the regions of interest. Prostate image
registration plays a crucial role in assisting surgeons during preoperative
planning and intraoperative surgery, with TRUS being a frequently utilized
modality in this context. Its purpose is to provide surgeons with more
complementary and valuable information. This task often involves analyzing
multi-modality images (e.g., MR to TRUS, see Fig. 1(b)), thus remaining
challenging in clinical practice. We classify and summarize relevant
registration methods in Section 3.2, according to modality, dimension,
deformation type and technical route.

####  2.2.1 Evaluation Metrics

Considering the purpose of image registration, the numerical evaluation
metrics are mainly aimed at quantifying the similarity of corresponding
regions between the fixed image and the warped moving image. Therefore the
metrics like DSC, HD, MAD, and MAXD can also be used to evaluate the
registration performance, by calculating the overlap of corresponding regions
or the distance of corresponding boundaries/surfaces.

Target registration error (TRE) is one of the most commonly used metrics in
medical image registration. It involves measuring the Euclidean distance of
the manually identified corresponding landmarks in the fixed and moving
images:

$\text{TRE}=\frac{1}{N}\sum_{i=1}^{N}\left\|L_{I_{f}}^{i},L_{I_{m}}^{i}\circ\phi\right\|_{2},$
(12)

where $L$ denotes the landmark set, and $N$ is the total number of
corresponding landmark pair in $L$. $I_{f}$ and $I_{m}$ are the fixed and
moving images, respectively. $\phi$ is the estimated deformation field, and
$\circ$ denotes the warping operation. $\left\|\cdot\right\|_{2}$ is the L2
norm.

Surface registration error (SRE) has the similar computing method with TRE,
but the difference is SRE measures the similarity of surface points between
fixed and moving images:

$\text{SRE}=\frac{1}{N}\sum_{i=1}^{N}\left\|S_{I_{f}}^{i},S_{I_{m}}^{i}\circ\phi\right\|_{2},$
(13)

where $S$ denotes the prostate surface point set, and $N$ is the total number
of corresponding point pair in $S$. Smaller values of TRE and SRE indicate
better registration accuracy.

In addition to assess registration accuracy using above metrics, some studies
also evaluate the quality of the estimated deformation field $\phi$. The
Jacobian matrix $J_{\phi}(p)=\nabla\phi(p)$ evaluates the regularity of the
deformation field by capturing the local gradient of $\phi$ around pixel $p$.
Smaller value of the percentage of pixels with non-positive Jacobian
determinant (%$\left|J_{\phi}\right|\leq 0$) indicates smoother $\phi$.

###  2.3 PCa Classification and Detection

Early and accurate diagnosis, along with precise staging, effectively improves
the success rate of treatment. The standard method for diagnosing and grading
PCa is the histopathological analysis of prostate tissue samples, typically
obtained via TRUS-guided biopsy. Consequently, the accurate identification of
target lesions during the biopsy procedure has been a long-standing and active
research task. To better support TRUS-guided targeted biopsies, automated
detection methods have been developed to predict PCa based on TRUS images, see
Fig. 1(c). Most of these methods approach cancer detection as a classification
problem, where small regions of interest or whole images are categorized as
benign or malignant. We classify and introduce these methods in Section 3.3,
according to TRUS modality and technical route.

####  2.3.1 Evaluation Metrics

The evaluation metrics for classification task mainly include area under the
receiver operating characteristic curve (AUC), accuracy (8), sensitivity (9),
specificity (11), precision (10), and F-score.

The AUC is a measure of the diagnostic ability of a binary classifier, which
computes the area under the receiver operating characteristic (ROC) curve. The
ROC curve is a curve drawn with the true positive rate (TPR, namely
sensitivity) as the Y-axis and the false positive rate (FPR, namely
specificity) as the X-axis. The area under ROC curve can be calculated as:

$\text{AUC}=\int_{0}^{1}\text{TPR}(fpr)\,dfpr.$ (14)

It represents the probability that a randomly chosen positive instance is
ranked higher than a randomly chosen negative instance.

The F-score is a weighted harmonic mean of precision and recall:

$\text{F-score}=2\times\frac{\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}}.$
(15)

###  2.4 Needle Detection

Prostate needles are primarily used during surgical procedures for various
purposes, including biopsy, brachytherapy, HIFU, cryotherapy, and so on.
Automated detection and segmentation of interventional needles are crucial in
surgical settings because this ensures precise localization and safe
interventions, as illustrated in Fig. 1(d). We summarize needle detection
methods in Section 3.4, according to clinical scenario and technical route.

####  2.4.1 Evaluation Metrics

The numerical metrics for evaluating the needle detection/segmentation mainly
include shaft error and tip error. Shaft error measures the deviation between
the predicted and actual axes of the needle, usually expressed in terms of
distance or angle:

$\text{E}_{shaft}^{distance}=\frac{1}{N}\sum_{i=1}^{N}\left\|\boldsymbol{o}_{i},\hat{\boldsymbol{o}_{i}}\right\|_{2},$
(16)

$\text{E}_{shaft}^{angle}=arccos\left(\frac{\boldsymbol{v}\cdot\hat{\boldsymbol{v}}}{\|\boldsymbol{v}\|\|\hat{\boldsymbol{v}}\|}\right),$
(17)

where $N$ is the number of points in a needle, $\hat{\boldsymbol{o}_{i}}$
represents the predicted position for the i-th point, and $\boldsymbol{o}_{i}$
is the ground truth position for the of the i-th point. $\boldsymbol{v}$ and
$\hat{\boldsymbol{v}}$ represent the actual and predicted axis direction
vectors.

Tip error measures the difference between the predicted and actual tip
positions of the needle:

$\text{E}_{tip}=\left\|\boldsymbol{t},\hat{\boldsymbol{t}}\right\|_{2},$ (18)

where $\boldsymbol{t}$ and $\hat{\boldsymbol{t}}$ represent coordinates of the
actual and predicted needle tip.

$\raisebox{-30.13885pt}{\rotatebox{90.0}{{Dimensions
(Segmentation)}}}\begin{cases}\text{2D: }\begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg1}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg2}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg3}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg4}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg6}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg7}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg11}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg13}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg15}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg16}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg18}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg19}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg20}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg21}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg22}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg23}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg24}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg25}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg27}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg28}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg32}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg33}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg34}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg35}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg36}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg37}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg38}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg39}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg40}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg41}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg42}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg44}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg45}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg46}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg49}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg52}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg53}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg54}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg55}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg56}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg57}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg62}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg63}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg69}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg70}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg71}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg74}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg75}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg77}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg78}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg79}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg80}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg81}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg82}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg85}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg90}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg91}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg92}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg93}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg94}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg96}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg99}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg100}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg101}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg102}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg103}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg104}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg105}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg106}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg107}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg108}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg111}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg112}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg113}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg114}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg115}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg116}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg117}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg118}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg121}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg122}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg123}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\
\end{aligned}\\\ \\\ \text{2.5D: }\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg86}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg87}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg97}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\ \\\
\text{3D: }\begin{aligned} &\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg8}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg9}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg10}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg12}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg14}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg17}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg26}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg29}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg30}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg31}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg38}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg43}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg47}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg48}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg51}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg58}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg59}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg60}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg61}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg64}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg65}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg66}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg67}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg68}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg72}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg73}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg76}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg83}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg84}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg88}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg89}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg95}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg98}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg109}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg110}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg119}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg124}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\
\end{aligned}\\\ \end{cases}$ Figure 2: Prostate segmentation approaches
categorized as 2D, 2.5D, and 3D methods.

##  3 Method Overview

Figure 3: Key Technology Word Cloud in TRUS Prostate Imaging Analysis. The
quadrants depict: (A) upper left, prostate gland segmentation; (B) upper
right, prostate registration; (C) lower left, prostate cancer detection and
classification; (D) lower right, needle segmentation.

In this section, we provide an overview of the key technologies involved in
the four tasks of transrectal ultrasound (TRUS) prostate imaging analysis. To
visually demonstrate the key terms and methods involved in these tasks, we
present a comprehensive word cloud at the beginning of this section (see Fig.
3). The word cloud visually highlights the terms that frequently appear and
are of significant research importance in each task, with the font size of the
terms reflecting their prominence and widespread application in the
literature.

###  3.1 Prostate Gland Segmentation

####  3.1.1 Modalities

Multi-modality TRUS images including B-mode, color Doppler, elastography, and
micro-ultrasound have been utilized for the diagnosis of PCa. Each modality
offers unique insights into the physiological and pathological states of the
diseases. Among the TRUS modalities, the B-mode ultrasound is predominantly
employed for prostate segmentation. The B-mode TRUS affords a morphological
representation of the prostate, thereby facilitating the accurate delineation
of the prostate’s contour.

Almost all segmentation studies mentioned in this survey used B-mode TRUS for
the prostate segmentation. Few studies employed multi-modality TRUS or high-
resolution micro-ultrasound. Mahdavi et al., (2012) boosted the performance of
B-mode TRUS segmentation by offering supplementary information from the vibro-
elastography images. Jiang et al., (2024) and Zhang et al., (2024) segmented
prostate gland from the micro-ultrasound images. Their research has
underscored the potential of this high-resolution modality in the precise
segmentation of the prostate.

####  3.1.2 Dimensions

We categorize prostate segmentation approaches as 2D, 2.5D, 3D methods,
according to the TRUS dimension they processed. As depicted in Fig. 2, the
majority of researches have concentrated on 2D and 3D techniques, with a
comparatively modest exploration of the 2.5D methodology.

$\raisebox{-30.13885pt}{\rotatebox{90.0}{{Segmentation
Methods}}}\begin{cases}\raisebox{-30.13885pt}{\rotatebox{90.0}{{Traditional
Methods}}}\begin{cases}\text{Contour/Shape-based Methods:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg2}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg3}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg4}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg5}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg6}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg7}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg8}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg9}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg12}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg13}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg15}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg16}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg19}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg20}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg24}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg25}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg26}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg27}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg32}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg33}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg35}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg36}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg41}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg43}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg46}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg49}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg57}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg59}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg60}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg53}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg65}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg69}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg71}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg74}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg93}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg106}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{Region-based Methods:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg10}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg37}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg52}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg48}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg62}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg61}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg64}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg66}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg67}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg68}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg73}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{Conventional Machine Learning Methods:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg1}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg21}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg28}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg34}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg45}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg47}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg58}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg63}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg70}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg72}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg96}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg102}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg103}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg104}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg105}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg107}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg108}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg101}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg114}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg115}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg117}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg121}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg123}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{Hybrid Methods:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg11}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg14}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg17}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg18}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg21}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg22}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg23}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg29}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg30}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg31}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg38}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg42}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg39}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg40}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg39}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg40}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg44}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg48}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg54}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg55}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg80}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg89}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\end{cases}\\\ \\\ \raisebox{-30.13885pt}{\rotatebox{90.0}{{Deep Learning
Methods}}}\begin{cases}\text{Convolutional Neural Network (CNN)}\\\
\begin{aligned} &\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg79}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg82}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg84}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg85}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg86}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg87}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg88}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg91}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg90}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg95}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg98}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg99}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg100}{\@@citephrase{(}}{\@@citephrase{)}}}},\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg109}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg110}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg111}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg113}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg116}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg118}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg124}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\
\end{aligned}\\\ \\\ \text{Recurrent Neural Network (RNN)}\\\
\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg75}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg78}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\ \\\
\text{Attention Enhanced Network:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg77}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg83}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg92}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg94}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg97}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg119}{\@@citephrase{(}}{\@@citephrase{)}}}},\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{seg122}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\end{cases}\end{cases}$
Figure 4: Traditional segmentation methods and deep learning (DL) methods.
Traditional methods can be categorized as contour/shape based methods, region
based methods, conventional machine learning methods, and hybrid methods. DL
methods mainly apply convolutional neural network (CNN), recurrent neural
network (RNN), and attention enhanced networks.

The 2D segmentation is executed on an individual slice of the prostate image.
This approach is frequently employed for intraoperative real-time navigation,
serving to assist clinicians in the precise identification of the prostate
gland’s location and boundary. The merits of 2D segmentation include its
simplicity and high computational efficiency. Nonetheless, it falls short in
capturing the 3D morphology of the prostate, potentially leading to
inaccuracies in the volumetric assessment of the prostate, particularly
concerning the apex and base regions.

In contrast to 2D segmentation, three-dimensional segmentation is conducted on
the entire volumetric TRUS data. This technique is predominantly utilized in
preoperative planning, aiding physicians in delineating the extent and
trajectory for surgical excision or in radiation therapy planning by enabling
radiation oncologists to pinpoint the target region for irradiation. 3D
segmentation affords an exhaustive and accurate quantitative analysis of the
prostate’s shape, volume, and boundaries. However, it is encumbered by high
computational demands and necessitates the processing of a vast array of
voxels and intricate 3D structures.

Beyond the 2D and 3D approaches, the 2.5D segmentation technique operates on
multiple contiguous slices of the prostate image. By integrating information
from multiple slices, 2.5D segmentation can partially capture the volumetric
information. It offers an advantage over 2D segmentation in better handling
the 3D morphological variations of the prostate, albeit with limitations
imposed by the selection and thickness of the slices. Compared with 3D
segmentation, 2.5D segmentation can alleviate the computational load,
presenting a more efficient compromise between computational demands and
segmentation accuracy.

####  3.1.3 Segmentation Methods

We categorize prostate segmentation methods into two ways: traditional methods
and deep learning (DL) methods. Fig. 4 shows the categorization in detail. It
can be seen that before 2016 almost all of the studies were about traditional
methods, and after that DL methods dominated this area.

##### Traditional Methods.

We present the traditional methods for prostate segmentation in four
categories: contour/shape-based methods, region-based methods, conventional
machine learning methods, and hybrid methods. Such categorization is
determined on the underlying theoretical computational approaches employed to
address the segmentation task. Table 1 gives a comprehensive summary of the
representative studies using traditional methods for TRUS segmentation.

(a) Contour/Shape-based Methods. These methods use prostate contour/edge
information to guide the segmentation. Contour-based methods typically
initiate with edge detection. Pathak et al., (2000) introduced a Sticks
algorithm to enhance contrast and reduce speckle noise in TRUS, thus
facilitating the rendering of prostate edges as a visual guide for subsequent
edge detection and manual refinement. Shen et al., (2003) expanded on this by
employing a Gabor filter bank to characterize prostate boundary features
across multiple scales and orientations, followed by a hierarchical
deformation strategy to adaptively focus on feature similarity. This approach,
while more time-consuming, captured a broader range of boundary directions. Wu
et al., (2015) diverged from the traditional view by leveraging speckles’
intrinsic properties to aid segmentation, arguing that speckle characteristics
provide useful cues for prostate boundary delineation.

Active shape models (ASM), as pioneered by Cootes et al., (1995), offer a
statistical shape-based method that has been further developed by researchers
such as Cosío, (2008), who presented a robust automatic initialization of the
ASM for prostate. This method used pixel classification for the initialization
of prostate region and a multi-population genetic algorithm for ASM pose
adjustment. Mahdavi et al., (2012) and Bi et al., (2020) extended the ASM by
incorporating high-contrast vibro-elastography images and clustering
techniques using a Rayleigh mixture model, respectively, to enhance the
initialization and deformation guidance of the ASM model.

Beyond ASM, other shape-prior-based segmentation techniques have been
explored. Badiei et al., (2006) proposed an image warping technique to impose
an elliptical shape on the prostate, with subsequent optimization of the
elliptical fitting using boundary points from an edge detector. Mahdavi et
al., (2011) further advanced this by suggesting twisted and tapered ellipsoids
as suitable a priori 3D shapes, transforming the shape fitting into a convex
problem for efficient solution. Yan et al., 2010a  proposed adaptive learning
of local neighborhood shape statistics to capture patient-specific variations.
Qiu et al., (2012) introduced a semi-automatic method integrating shape
constraints with a modified level set formulation for 3D segmentation. These
methods emphasized the importance of integrating shape priors with
computational strategies to improve accuracy and efficiency.

(b) Region-based Methods. These methods harness local intensity values or
statistical measures within an energy minimization framework to achieve
segmentation. Fan et al., (2002) initially developed a fast discrimination
technique for extracting the prostate region and subsequently addressed the
“boundary leaking” issue by integrating regional information, rather than
gradient information, into the level set method. Yuan et al., (2013) employed
rotationally re-sliced images around a specified axis to delineate 3D prostate
boundaries, a technique that effectively capitalized on the inherent
rotational symmetry of prostate shapes to adjust a series of 2D slice
segmentation in a unified 3D context. The work (Yuan et al.,, 2013) was
further extended to (Qiu et al., 2014b, ), with more validation data. Qiu et
al., 2014a  introduced a convex optimization strategy for extracting the
prostate surface, while maintaining the global volume size prior. Moreover,
they implemented an efficient numerical solver on GPUs. Additionally, some
methods accomplish the extraction of the prostate region through template
matching. Nouranian et al., (2013) designed a multi-atlas fusion framework to
segment TRUS images using pairwise atlas shape similarity. Nouranian et al.,
(2014) further introduced the atlas agreement factor to boost the robustness
of the previous work (Nouranian et al.,, 2013). This factor was integrated
into an atlas selection algorithm to refine the dataset prior to merging the
atlas contours, yielding consistent segmentation.

Table 1: Overview of traditional methods for prostate gland segmentation in TRUS. Traditional Methods | References | Dimension | Auto/Semi-auto | Methods | Samples | Results
---|---|---|---|---|---|---
Metrics | Values
Contour and Shape | Pathak et al., (2000) | 2D | Semi-auto | Probabilistic filters, edge detection, manual editing | 125 |  MAD HD |  $0.7\pm 0.4$ mm $1.8\pm 1.0$ mm
Shen et al., (2003) | 2D | Automatic | Gabor filter bank, statistical shape model | 8 |  1-DSC 1-IoU MAD |  $1.66\pm 1.68\%$ $3.98\pm 0.97\%$ $3.20\pm 0.87$ pixels
Badiei et al., (2006) | 2D | Semi-auto | Image warping, edge detection, ellipse fitting | 17 |  MAD MAXD |  $0.67\pm 0.18$ mm $2.25\pm 0.56$ mm
Cosío, (2008) | 2D | Automatic | Pixel classification, active shape model | 22 |  MAD MAXD |  $1.74\pm 0.69$ mm $4.51\pm 2.42$ mm
Mahdavi et al., (2011) | 3D | Semi-auto | Shape fitting, convex optimization | 40 |  1-DSC |  $5.82\pm 4.15\%$
Qiu et al., (2012) | 3D | Semi-auto | Shape constraint, level set | 20 |  DSC MAD |  $93.39\pm 1.26\%$ $1.16\pm 0.34$ mm
Mahdavi et al., (2012) | 3D | Automatic | Elastography, active shape model | 11 |  DSC |  $87\pm 7\%$
Wu et al., (2015) | 2D | Automatic | Rotation-invariant texture feature, graph cut, level set | 598 |  MAD |  $1.06\pm 0.53$ mm
Bi et al., (2020) | 2D | Automatic | Rayleigh mixture model, active shape model | 30 |  DSC MAD |  $95\pm 0.81\%$ $1.86\pm 0.02$ pixels
Region | Fan et al., (2002) | 3D | Semi-auto | Region information, level set | 8 | Visual Comparison | Visual Comparison
Yuan et al., (2013) | 3D | Automatic | Rotational symmetry, convex optimization | 20 |  DSC MAD MAXD |  $93.7\pm 2.1\%$ $1.12\pm 0.4$ mm $3.15\pm 0.65$ mm
Nouranian et al., (2013) | 3D | Automatic | Multi-atlas fusion, pairwise shape similarity | 50 |  DSC MAD MAXD |  $92.08\pm 1.96\%$ $1.22\pm 0.57$ mm $3.47\pm 1.48$ mm
Qiu et al., 2014a  | 3D | Automatic | Volume-preserving prior, convex optimization | 12 |  DSC MAD MAXD |  $89.5\pm 2.4\%$ $1.4\pm 0.6$ mm $5.2\pm 3.2$ mm
Qiu et al., 2014b  | 3D | Automatic | Axial symmetry, convex optimization | 25 |  DSC |  $93.2\pm 2.0\%$
Nouranian et al., (2014) | 3D | Automatic | Multi-atlas fusion, atlas agreement factor | 280 |  DSC MSD |  $89.75\pm 3.75\%$ $1.33\pm 0.60$ mm
Machine Learning | Ghose et al., (2013) | 2D | Automatic | Statistical shape and appearance model, random forest | 126 |  DSC MAD HD |  $91\pm 4\%$ $1.30\pm 0.3$ mm $3.62\pm 0.56$ mm
Nouranian et al., (2015) | 2D | Automatic | Sparse dictionary learning | 590 |  MSD HD |  $0.98\pm 0.39$ mm $5.4\pm 1.38$ mm
Yang et al., (2016) | 3D | Automatic | Kernel support vector machine | 10 |  DSC |  $89.7\pm 2.3\%$
Peng et al., 2022b  | 2D | Semi-auto | Improved closed principal curve | 50 |  DSC Jaccard ACC |  $96.5\%$ $95.1\%$ $96.3\%$
Peng et al., 2024b  | 2D | Semi-auto | Adaptive polygon tracking model, principal curve | 1320 |  DSC Jaccard ACC |  $96.4\pm 2.4\%$ $95.5\pm 2.9\%$ $96.1\pm 2.5\%$
Hybrid | Zhan and Shen, (2003) | 3D | Automatic | Kernel support vector machine, statistical shape model | 3 |  1-IoU MAD |  $3.9\%$ $0.81$ voxel
Gong et al., (2004) | 2D | Automatic | Deformable superellipses, Bayesian model | 125 |  MAD HD |  $1.36\pm 0.58$ mm $3.42\pm 1.52$ mm
Tutar et al., (2006) | 3D | Semi-auto | Spherical harmonics, statistical analysis | 30 |  MAD MAXD |  $1.26\pm 0.41$ mm $4.06\pm 1.25$ mm
Zhan and Shen, (2006) | 3D | Automatic | Gabor-support vector machine, statistical texture matching | 5 |  1-IoU MAD |  $4.31\pm 0.40\%$ $1.07\pm 0.10$ voxels
Yan et al., 2010b  | 2D | Automatic | Partial active shape model, region model | 301 |  MAD |  $2.01\pm 1.02$ mm
Garnier et al., (2010) | 3D | Semi-auto | Discrete dynamic contour, optimal surface detection | 28 |  ASD |  $0.77\pm 0.28$ mm
Jaouen et al., (2018) | 3D | Automatic | Hybrid edge/region-based parametric deformable model | 36 |  DSC MAD HD |  $92\pm 2\%$ $1.42\pm 0.40$ mm $4.03\pm 1.22$ mm

(c) Conventional Machine Learning Methods. Early prostate segmentation methods
also encompass machine learning approaches. These techniques leverage features
such as intensity or higher-dimensional attributes like filter responses to
cluster and/or classify the image into prostate and background regions. Ghose
et al., (2013) introduced a method for prostate segmentation that constructed
a multiple average parameter model derived from the principal component
analysis of shape and a posterior probabilities within a multi-resolution
framework. The model parameters were subsequently refined using a priori
knowledge of the optimization space to achieve optimal prostate segmentation.
Nouranian et al., (2015) proposed a learning-based multi-label segmentation
algorithm. A sparse representation method was integrated into the algorithm to
learn a dictionary encoding information of images as well as clinical and
planning target volume segmentation. Yang et al., (2016) developed a block-
based feature learning framework, where patient-specific anatomical features
extracted from aligned training images served as signatures for each voxel. A
feature selection process identified the most robust and informative features
to train a kernel support vector machine (KSVM).

Additionally, some methods have employed the principal curve, which provided a
means of representing low-dimensional data manifolds in high-dimensional
spaces, thereby effectively capturing the complexity of prostate shapes. Peng
et al., 2022b  proposed an improved closed principal curve approach, utilizing
only a few seed points defined by radiologists as an approximate
initialization. Peng et al., 2024b  further developed an adaptive polygon
tracking model to alleviate the issue of determining the segment number of the
principal curve automatically. In addition, an interpretable mathematical
function was formulated to constrain a smooth boundary.

(d) Hybrid Methods. The objective of hybrid methods is to leverage advantages
from contour, shape, region, and conventional machine learning to achieve
better segmentation results. Zhan and Shen, (2003) combined kernel support
vector machine and statistical shape model (SSM) to segment 3D TRUS. The KSVM
was employed to classify the prostate from surrounding tissues, and then the
SSM was used to refine the contour. Later, Zhan and Shen, (2006) improved
(Zhan and Shen,, 2003) by using a set of Gabor-support vector machines
(G-SVMs) and the statistical texture matching model. Gong et al., (2004)
emphasized that the prior knowledge of prostate shapes played an important
role for reliable segmentation. They used deformable superellipses to model
the shape of prostate and further developed a robust Bayesian segmentation
approach. Tutar et al., (2006) employed the eighth spherical harmonic function
to depict the shape of the prostate, and statistically analyzed the shape
parameters. Yan et al., 2010b  employed partial active shape model and region
model to address the issue of missing boundaries in shadow areas. Garnier et
al., (2010) sequentially combined discrete dynamic contour and optimal surface
detection for semi-automatic 3D TRUS segmentation in HIFU Therapy. Jaouen et
al., (2018) proposed a hybrid edge and region-based parametric deformable
model. The edge-based model was derived from the radial bas-relief approach,
while the region-based model was regarded as a parametric active surface
driven by the Bhattacharyya gradient flow.

##### Deep Learning Methods.

We categorize deep learning methods for TRUS segmentation into three classes:
convolutional neural network (CNN), recurrent neural network (RNN), and
attention enhanced networks, according to their network structure. Due to
limited studies on Transformer currently in this field, relevant researches
are included in the attention enhanced networks section. Table 2 summarizes
representative studies related to the DL-based segmentation in TRUS.

(a) Convolutional Neural Network. CNNs are among the most successful and
widely used architectures in DL, particularly for computer vision tasks.
Following the advent of U-net Ronneberger et al., (2015), a plethora of CNNs
with encoder-decoder structures have been employed for medical image
segmentation. Several methods have sought to enhance segmentation performance
by improving loss functions. For instance, Lei et al., 2019a  developed a
multi-directional DL method to segment TRUS for US-guided radiation therapy.
They combined binary cross-entropy (BCE) loss and batch-based Dice loss into a
staged hybrid loss function for deeply-supervised training. Karimi et al.,
(2019) improved the uncertain segmentation by incorporating prior shape
information in the form of a statistical shape model. Karimi and Salcudean,
(2019) presented a loss function for training the segmentation network with
the goal of directly reducing Hausdorff distance.

Additionally, other methods have focused on the improvement of the network
structures. Lei et al., 2021b  developed an anchor-free mask CNN that
consisted of three subnetworks: a backbone, a fully convolutional one-state
object detector, and a mask head. Xu et al., (2022) proposed a novel polar
transform network to address the issue of difficult segmentation in the
challenging base and apex regions. The proposed network represented the
prostate shape in the polar coordinate space. This representation provided
much denser samples of the prostate volume, thus facilitating the learning of
discriminative features for accurate prostate segmentation.

Furthermore, some studies have focused on fully utilizing data information. Xu
et al., (2021) proposed a shadow-consistent semi-supervised segmentation
method that included shadow augmentation (Shadow-AUG) and shadow dropout
(Shadow-DROP). Shadow-AUG enlarged training samples by including simulated
shadow artifacts to the images, while Shadow-DROP enhanced the network to
leverage neighboring high-quality pixels to infer boundaries. Orlando et al.,
(2022) developed an automatic 3D TRUS segmentation network, trained on a
large, clinically diverse dataset with variable image quality.

(b) Recurrent Neural Network. RNNs are commonly utilized for processing
sequential data, such as speech, text, videos, and time-series data. Unlike
unidirectional feedforward neural networks, RNNs are bidirectional neural
networks, allowing the outputs of certain nodes to influence subsequent inputs
from the same nodes. Their capability to process arbitrary input sequences
using internal state, or memory, makes them suitable for tasks such as
unsegmented, connected handwriting recognition or speech recognition.

In the context of TRUS segmentation, Yang et al., (2017) first serialized
static 2D TRUS images into dynamic sequences and then predicted prostate shape
by sequentially exploring the shape prior. Subsequently, to mitigate the bias
caused by different serialization approaches, they employed a multi-view
fusion strategy to merge shape predictions obtained from various perspectives.
Furthermore, they integrated the RNN core into a multi-scale automatic context
scheme to continuously refine the details of the shape prediction graph.
Similarly, Anas et al., 2018a  not only utilized CNNs to extract spatial
features but also employed RNNs to leverage the temporal information among a
series of TRUS images. Additionally, they utilized cyclic connections within
and between different layers of the network to maximize the utilization of
temporal information.

Table 2: Overview of deep learning methods for prostate gland segmentation in TRUS. Deep Learning Methods | References | Dimension | Methods | Samples | Results
---|---|---|---|---|---
Metrics | Values
CNN | Lei et al., 2019a  | 3D |  1) Multi-directional deeply supervised V-Net 2) A hybrid loss with binary cross-entropy and batch-based Dice | 44 |  DSC MSD HD |  $92\pm 3\%$ $0.60\pm 0.23$ mm $3.94\pm 1.55$ mm
Karimi et al., (2019) | 2D |  1) An adaptive sampling strategy to address difficult cases 2) A CNN ensemble to identify uncertain segmentation | 675 |  DSC HD |  $93.9\pm 3.5\%$ $2.7\pm 2.3$ mm
Karimi and Salcudean, (2019) | 2D |  1) A loss function to reduce the Hausdorff distance (HD) 2) Three methods to estimate HD | 2625 |  DSC ASD HD |  $94.6\pm 4.1\%$ $1.05\pm 0.46$ mm $2.6\pm 1.8$ mm
Lei et al., 2021b  | 3D |  1) An anchor-free mask CNN | 83 |  DSC 95HD |  $94\pm 3\%$ $2.27\pm 0.79$ mm
Xu et al., (2022) | 3D |  1) Represent prostate shape in the polar coordinate space 2) A centroid perturbed test-time augmentation strategy | 315 |  DSC ASD HD |  $89.97\pm 3.47\%$ $1.30\pm 0.61$ mm $7.07\pm 3.19$ mm
Xu et al., (2021) | 3D |  1) Shadow augmentation and shadow dropout 2) Semi-supervised Learning | 2423 |  DSC ASD HD |  $91.60\pm 2.37\%$ $1.02\pm 0.34$ mm $6.37\pm 2.36$ mm
Orlando et al., (2022) | 2D |  1) Modified U-Net and U-Net$++$ | 6721 |  DSC MSD HD |  $94.0\%$ $0.90$ mm $3.27$ mm
RNN | Yang et al., (2017) | 2D |  1) Serialize static TRUS images into dynamic sequences 2) A multi-view shape fusion strategy 3) A multiscale auto-context scheme | 530 |  DSC Recall Precision |  $92.33\%$ $89.76\%$ $95.19\%$
Anas et al., 2018a  | 2D |  1) Residual convolution in recurrent networks 2) Recurrent connections for deep network layers | 3892 |  DSC HD |  $92.91\pm 2.74\%$ $2.97\pm 1.96$ mm
Attention | Wang et al., 2018a  | 2D |  1) Select useful complementary information from multi-level features to refine the features at each individual layer | 530 |  DSC Recall Precision |  $95.27\%$ $96.98\%$ $93.69\%$
Wang et al., (2019) | 3D |  1) Selectively leverage multi-level features from different layers | 40 |  DSC Recall Precision |  $90\pm 3\%$ $91\pm 4\%$ $90\pm 6\%$
Girum et al., (2020) | 2D |  1) A channel-wise attention strategy 2) Learning-based prior shape knowledge | 145 |  DSC 95HD |  $88\pm 3\%$ $2.01\pm 0.54$ mm
Vesal et al., (2022) | 2.5D |  1) Feature positioning information 2) A training scheme that utilizes knowledge distillation | 2892 |  DSC 95HD |  $94.0\pm 3.0\%$ $2.29\pm 1.45$ mm
Peng et al., 2024a  | 2D |  1) Attention gate modules with squeeze-and-excitation U-net 2) An improved polygon searching model 3) A storage-based quantum evolution model | 945 |  DSC ACC |  $94.3\pm 3.8\%$ $93.9\pm 4.1\%$

(c) Attention Enhanced Network. In DL methods, an attention block can
selectively modify inputs or assign different weights to input variables
according to their varying importance. Attention mechanisms in DL are
fundamentally analogous to the selective visual attention mechanisms in
humans, with the core objective being to select the most critical information
from a vast amount of data for the current task’s goal.

In the context of TRUS segmentation, Wang et al., 2018a  firstly developed a
network equipped with deep attentional feature (DAF) modules to enhance
segmentation by fully exploiting the complementary information encoded in
different layers of the CNN. The DAF module selectively utilized multi-level
features integrated across different layers to refine each individual layer,
suppressing non-prostatic noise in shallow layers of the CNN, and enriching
deeper features with more prostatic detail. Further, Wang et al., (2019)
extended (Wang et al., 2018a, ) to 3D TRUS segmentation, by using more
powerful backbone architecture. Girum et al., (2020) presented a multi-task DL
method for boundary detection in intraoperative TRUS, by leveraging both low-
level and high-level information. The channel-wise attention strategy was
employed for low-level feature refinement, and the learning-based prior
knowledge was utilized for high-level feature modeling. Vesal et al., (2022)
addressed the limitations of transfer learning and fine-tuning methods by
introducing a 2.5D deep neural network, which further relied on an attention
module that considered feature positioning information to improve segmentation
accuracy. Peng et al., 2024a  developed a coarse-to-fine architecture for TRUS
segmentation. They employed the attention gate module with squeeze-and-
excitation U-net to conduct initial segmentation, and then jointly used a
modified polygon-searching model and a storage-based quantum evolution model
to refine the segmentation.

###  3.2 Prostate Image Registration

####  3.2.1 Modalities

Prostate image registration is useful for assisting surgeons during
preoperative planning and intraoperative surgery. It can provide surgeons with
more complementary information, especially the multi-modality registration.
Fig. 5 lists the categorization of registration approaches according to the
image modality. The majority of studies focused on MRI-TRUS registration,
which registered the preoperative MRI scans to the TRUS images, providing
better resolution and soft tissue contrast to assist targeted biopsy. In
addition, CT-TRUS registration methods have also been investigated, mainly in
the scenarios of treatment planning and prognosis evaluation. Other studies
payed attention to the registration of preoperative and intraoperative TRUS
images.

$\raisebox{-30.13885pt}{\rotatebox{90.0}{{Modalities
(Registration)}}}\begin{cases}\text{TRUS-TRUS: }\begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg6}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg9}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg13}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg22}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg28}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg40}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg44}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg53}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg56}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg64}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg65}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{MRI-TRUS: }\begin{aligned} &\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg1}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg4}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg5}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg8}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg10}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg14}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg15}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg16}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg17}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg18}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg19}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg20}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg23}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg21}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg24}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg25}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg26}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg27}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg29}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg31}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg33}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg34}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg35}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg36}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg37}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg38}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg41}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg42}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg45}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg46}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg47}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg48}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg50}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg49}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg51}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg52}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg70}{\@@citephrase{(}}{\@@citephrase{)}}},\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg54}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg55}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg57}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg59}{\@@citephrase{(}}{\@@citephrase{)}}},\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg61}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg63}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg67}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg68}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg69}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\
\end{aligned}\\\ \\\ \text{CT-TRUS: }\begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg3}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg7}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg32}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg39}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg58}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\
\end{aligned}\\\ \end{cases}$ Figure 5: Prostate image registration studies
categorized as TRUS-TRUS, MRI-TRUS, and CT-TRUS.
$\raisebox{-30.13885pt}{\rotatebox{90.0}{{Dimensions
(Registration)}}}\begin{cases}\text{2D-2D: }\begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg8}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{2D-3D: }\begin{aligned} &\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg4}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg11}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg13}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg22}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg28}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg30}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg45}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg46}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg50}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg53}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg56}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg64}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{3D-3D: }\begin{aligned} &\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg1}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg3}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg5}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg6}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg7}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg9}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg10}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg14}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg15}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg16}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg17}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg18}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg19}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg21}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg20}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg23}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg24}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg25}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg26}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg27}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg29}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg31}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg32}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg33}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg34}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg35}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg36}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg37}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg38}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg39}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg40}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg41}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg42}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg44}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg47}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg48}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg49}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg51}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg52}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg70}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg54}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg55}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg57}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg58}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg59}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg61}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg63}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg65}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg67}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg68}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg69}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\
\end{aligned}\end{cases}$ Figure 6: Prostate image registration studies
categorized as 2D-2D, 2D-3D, and 3D-3D.

####  3.2.2 Dimensions

We classify registration methods into three categories based on
dimensionality: 2D-2D, 2D-3D, and 3D-3D. Images acquired intraoperatively are
predominantly 2D TRUS, whereas those acquired preoperatively are mostly 3D
(MRI, CT, 3D TRUS). Ultimately, the registration methods intended for prostate
biopsy would necessitate the registration between intraoperative 2D TRUS and
preoperative 3D images.

The process of registering 2D TRUS to 3D images from different modalities
presents two main challenges: differing dimensions and distinct modalities.
There was a scarcity of literature that addressed both of these issues
simultaneously. Consequently, most 2D-3D studies concentrated on registration
between TRUS images to bridge the dimensional gap, while 3D-3D studies
primarily focused on registration between TRUS and other modalities to
overcome the challenge posed by different imaging modalities.

####  3.2.3 Deformation Types

According to different deformation types, we categorize the registration
methods into linear and non-linear registration, as shown in Fig. 7. Linear
transformation, also referred to as isometric transformation, involves only
translation and rotation. Affine transformation includes scaling and shear in
addition to translation and rotation, making linear transformation a special
case of affine transformation. In this survey, both affine and linear
transformation are uniformly classified as the category of linear
registration. In contrast, non-linear registration refers to methods that go
beyond linear registration, primarily involving deformable transformation.
Unlike linear registration, deformable registration methods employ non-linear
dense transformation to align images. Most of prostate image registration
methods focused on non-linear registration to address the issue of large
deformation in TRUS images.

$\raisebox{-42.1944pt}{\rotatebox{90.0}{{Deformation Types
(Registration)}}}\begin{cases}\text{Linear:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg1}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg3}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg4}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg13}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg28}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg30}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg31}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg41}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg47}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg53}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg55}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg56}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg63}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg64}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg65}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg67}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg69}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\
\end{aligned}\\\ \\\ \text{Non-Linear:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg5}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg6}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg7}{\@@citephrase{(}}{\@@citephrase{)}}},\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg8}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg9}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg10}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg11}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg14}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg15}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg16}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg17}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg18}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg19}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg20}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg21}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg22}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg23}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg24}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg25}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg26}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg27}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg29}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg32}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg33}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg34}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg35}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg36}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg37}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg38}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg39}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg40}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg42}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg44}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg45}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg46}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg48}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg49}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg50}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg51}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg70}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg52}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg54}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg57}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg58}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg59}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg61}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg68}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\
\end{aligned}\end{cases}$ Figure 7: Prostate image registration studies
categorized as linear registration and non-linear registration.
$\raisebox{-30.13885pt}{\rotatebox{90.0}{{Registration
Methods}}}\begin{cases}\raisebox{-30.13885pt}{\rotatebox{90.0}{{Traditional
Methods}}}\begin{cases}\text{Intensity-based Methods:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg3}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg4}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg9}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg11}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg13}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg14}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg17}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg28}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\
\end{aligned}\\\ \\\ \text{Shape-based Methods:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg1}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg5}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg6}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg7}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg8}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg10}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg15}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg16}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg18}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg19}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg20}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg21}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg22}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg23}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg24}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg25}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg26}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg27}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg30}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg29}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg32}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg35}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg38}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg39}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg40}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg42}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg45}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\end{cases}\\\ \\\ \raisebox{-30.13885pt}{\rotatebox{90.0}{{Deep Learning
Methods}}}\begin{cases}\text{Image-based Network:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg31}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg41}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg44}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg46}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg47}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg53}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg55}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg56}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg58}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg59}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg61}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg63}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg64}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg69}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{Label-driven Network:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg33}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg34}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg36}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg37}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg48}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg50}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg51}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg54}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg57}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg65}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg67}{\@@citephrase{(}}{\@@citephrase{)}}}}\\\
\end{aligned}\\\ \\\ \text{Point Set Network:}\\\ \begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg49}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg52}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg70}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{reg68}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
\end{aligned}\end{cases}\par\end{cases}$ Figure 8: Traditional registration
methods and deep learning (DL) methods. Traditional methods can be categorized
as intensity-based methods and shape-based methods. DL methods mainly include
image-based networks, label-driven networks, and point set networks.

####  3.2.4 Registration Methods

We categorize registration methods into two main types: traditional methods
and DL methods, as shown in Fig. 8. Traditional methods, while often time-
consuming due to the iterative estimation of the deformation field, are known
for their stability and reliability across various scenarios. In contrast, DL
methods have gained popularity due to advancements in GPU computing power,
enabling them to achieve more efficient registration than traditional methods.

##### Traditional Methods.

Traditional registration methods mainly investigated the following two
aspects: matching criteria and deformation models. The matching criterion
defines the similarity of the image pair, with intensity-based methods relying
on the similarity of intensities or intensity-related features of images, and
shape-based methods focusing on the similarity of prostate’s region/boundary.
The deformation model is also critical, as the registration process involves
optimizing the parameters of the deformation fields to warp the moving image
to match with the fixed image. We summarize the traditional registration
methods on TRUS images according to different matching criteria.

(a) Intensity-based Methods. Table 3 summarizes some representative
traditional registration methods, including intensity-based methods and shape-
based methods. Xu et al., (2007) employed spatial tracking information to
initialize the registration between intraoperative 2D TRUS and preoperative 3D
TRUS. They then optimized the registration by calculating the grayscale
similarity to eliminate spatial tracking errors. Additionally, they manually
registered 3D TRUS and 3D MRI preoperatively, achieving a 2D TRUS-3D TRUS-3D
MRI registration process. Hungr et al., (2012) used the correlation
coefficient as a similarity metric to register 3D TRUS, while Baumann et al.,
(2012) employed the shift correlation model combined with the inverse
consistency constraint, performing linear registration separately from
deformable registration. Sun et al., (2014) used the modality independent
neighborhood descriptor (MIND) as the local similarity feature for MRI-TRUS
registration. They also developed a duality-based convex optimization
algorithm to compute MIND similarity. De Silva et al., (2013) used Powell’s
method to optimize the normalized cross-correlation (NCC) metric between
images, completing the linear registration of 2D TRUS to 3D TRUS. Later,
Gillies et al., (2017) also employed Powell’s method to optimize the NCC
metric and realized real-time registration of 2D and 3D TRUS.

Table 3: Overview of traditional methods for prostate TRUS registration. Traditional Methods | References |  Dimensions Modalities | Deformation | Methods | Samples | Results
---|---|---|---|---|---|---
Metrics | Values: final (initial)
Intensity | Baumann et al., (2012) |  2D TRUS 3D MRI | Non-linear |  1) Shift correlation and inverse consistency constraint 2) Local rigid and linear elastic deformation | 40 | TRE |  $0.8\pm 0.5$ ($13.8\pm 7.9$) mm
De Silva et al., (2013) |  2D TRUS 3D TRUS | Linear |  1) Optimize the normalized cross-correlation (NCC) metric using Powell’s method | 8 | TRE |  $1.87\pm 0.81$ ($4.75\pm 2.62$) mm
Sun et al., (2014) |  3D TRUS 3D MRI | Non-linear |  1) Modality independent neighborhood descriptor 2) Duality-based convex optimization algorithm | 20 |  TRE DSC MAD MAXD |  $1.93\pm 0.73$ ($3.37\pm 1.23$) mm $85.7\pm 4.7$ (n/a) $\%$ $1.84\pm 0.52$ (n/a) mm $6.90\pm 2.07$ (n/a) mm
Gillies et al., (2017) |  2D TRUS 3D TRUS | Linear |  1) Optimize NCC using Powell’s method 2) Golden section search technique | 14 | TRE |  $1.40\pm 1.18$ ($3.41\pm 3.26$) mm
Shape | Mitra et al., (2012) |  2D TRUS 2D MRI | Non-linear |  1) Point correspondences obtained from a statistical measure of shape-contexts 2) The radial-basis function of thin-plate splines | 20 |  TRE DSC 95HD |  $1.60\pm 1.17$ (n/a) mm $98.0\pm 0.4$ (n/a) $\%$ $1.63\pm 0.48$ (n/a) mm
Hu et al., (2012) |  3D TRUS 3D MRI | Non-linear |  1) Statistical motion model 2) Expectation maximization algorithm for model-to-image registration | 8 | TRE |  $2.40$ ($8.13$) mm
Zettinig et al., (2015) |  3D TRUS 3D MRI | Non-linear |  1) Coherent point drift algorithm 2) Thin-plate splines | 13 | TRE |  $2.00$ ($2.60$) mm
Hu et al., (2015) |  3D TRUS 3D MRI | Non-linear |  1) Statistical motion model 2) Kernel regression analysis | 44 | TRE |  $2.40$ ($6.19$) mm
Khallaghi et al., 2015c  |  3D TRUS 3D MRI | Non-linear |  1) Finite element model with biomechanical constraint 2) Gaussian mixture model | 19 | TRE |  $2.72\pm 1.15$ ($4.01\pm 1.45$) mm
Khallaghi et al., 2015b  |  3D TRUS 3D MRI | Non-linear |  1) Statistical shape model 2) Finite element model | 19 | TRE |  $2.35\pm 0.81$ ($4.01\pm 1.45$) mm
Wang et al., (2016) |  3D TRUS 3D MRI | Non-linear |  1) Personalized statistical deformable model 2) Hybrid point matching | 18 | TRE |  $1.44\pm 0.38$ ($6.78\pm 1.52$) mm
Wang et al., 2018b  |  3D TRUS 3D MRI | Non-linear |  1) Robust projective dictionary learning 2) Personalized statistical deformable model | 18 | TRE |  $1.51\pm 0.71$ ($6.78\pm 1.52$) mm
Samei et al., (2018) |  2.5D TRUS 3D MRI | Non-linear |  1) Linear stress-strain quasi-static FEM 2) 2.5D-3D non-rigid registration | 11 | TRE |  $0.72\pm 0.45$ ($4.62\pm 3.38$) mm

(b) Shape-based Methods. It remains challenging to use intensity-based metrics
to evaluate the similarity of multi-modality images. Therefore, most of MRI-
TRUS and CT-TRUS registration approaches sought the shape-based similarities
as the solution. Although the shape-based methods required additional steps to
obtain the prostate region or corresponding landmarks, they offered more
reliable and robust registration for multi-modality tasks. Mitra et al.,
(2012) introduced a non-linear thin-plate spline (TPS) regularized 2D MRI-TRUS
registration method. The boundary point correspondences were estimated based
on a statistical measure of shape-contexts. Zettinig et al., (2015) applied
the coherent point drift (CPD) algorithm to elastically align surfaces
extracted from TRUS and MRI. They then propagated the surface deformation to
the whole gland using the TPS interpolation. Lee and Song, (2018) proposed a
deformable CT-TRUS registration method based on the structural descriptor map
(SDM), calculated by solving the Laplace’s equation using the segmented
prostate surface. The SDM represented the surface distribution and could be
employed for deformable registration. Samei et al., (2018) proposed an
efficient algorithm to extract arbitrary slices in real-time from deformed 3D
images represented by a discrete grid. They simulated tissue deformation using
a patient-specific finite element model (FEM) with boundary conditions for
image-guided, robot-assisted laparoscopic radical prostatectomy (RALRP).
Later, Samei et al., (2020) incorporated this technology into a virtual
reality-based RALRP system.

Certain methods (Hu et al.,, 2009, 2012) employed both statistical shape
modeling (SSM) and biomechanical modeling to capture the motion of the
prostate, which has been proved as a robust solution for MRI-TRUS
registration. leveraging the SSM and biomechanical modeling, Hu et al., (2012)
further calculated the normal vector field of the prostate surface in TRUS and
used the expectation maximization (EM) algorithm to maximize the log-
likelihood function for model-to-image registration. Hu et al., (2015) later
used kernel regression analysis to express the multivariate subject-specific
probability density function for the construction of subject-specific
statistical motion model. Khallaghi et al., 2015c  used a probabilistic
framework based on Gaussian mixture model (GMM) to combine FEM with
biomechanical prior knowledge for extrapolating and constraining deformation
fields during MRI-TRUS registration. Khallaghi et al., 2015b  employed
probabilistic SSM-FEM to map two surfaces and their interiors to a common
intermediate SSM instance, aligning MRI to 3D TRUS through this intermediate
shape. Wang et al., (2016) introduced a hybrid point matching method to
establish reliable surface point correspondences. A series of FEM were
conducted using personalized anatomical and biomedical information. And a
personalized statistical deformable model was constructed by the principal
component analysis (PCA) of FEMs to estimate the deformation. To enhance the
efficacy of statistical shape modeling in (Wang et al.,, 2016), Wang et al.,
2018b  developed a robust projective dictionary learning algorithm to
integrate dimensionality reduction and dictionary learning into a unified
shape prior modeling framework, and applied it for MRI-TRUS registration.

##### Deep Learning Methods.

With the rapid development of GPUs, using DL methods to solve non-linear
registration with high computational complexity has become mainstream. These
DL methods for TRUS image registration can be categorized into image-based
networks, label-driven networks, and point set networks. The first two types
often take images as input, while the last one uses surface point clouds.
Image-based networks mainly rely on the similarity between moving and fixed
images as the loss function to update network parameters. In contrast, label-
driven networks directly use prostate annotations or landmarks to aid
registration, either by incorporating them into the loss function or by
directly registering them. Point set networks transform the image registration
problem in the form of point cloud matching and train the network using the
point-based loss function. Table 4 introduces the representative methods in
each category.

(a) Image-based Network. Some methods have focused on the development of novel
network architectures. Yan et al., (2018) proposed an adversarial MRI-TRUS
registration framework, which trained two CNNs concurrently - a generator for
image registration and a discriminator to assess the registration quality.
This method only considered linear transformation and required ground truth
transformation for the supervised learning. Guo et al., (2021) proposed a
2D-3D TRUS registration network. This network used a dual-branch encoding
module to extract 2D and 3D features for the estimation of linear
transformation parameters. And a 2D plane sampling module was designed to
enable efficient training and inference. This approach was subsequently
extended to (Guo et al.,, 2023), by newly designing a frame-to-slice
correction network to refine the transformation estimation and a similarity
filtering mechanism to assess the registration quality. Song et al., (2021)
introduced a cross-modal attention module to explicitly utilize spatial
correspondence for a linear transformation in 3D MRI-TRUS registration, and
further extended this approach to other applications (Song et al., 2022b, ).

Other methods have explored various training strategies and loss functions.
Haskins et al., (2019) employed a CNN to learn similarity metrics for the
representation of MRI-TRUS, and then used the learned similarity metric to
optimize the registration. Guo et al., (2020) utilized data generation methods
to create diverse training data distributions, allowing the network to perform
well for MRI-TRUS registration. Song et al., 2022a  introduced the transformed
grid distance loss, which addressed the issue of rotation representation and
bridged the gap between translation and rotation.

(b) Label-driven Network. These networks leveraged prostate annotations or
landmarks to evaluate the similarity of multi-modality prostate images. Hu et
al., 2018a  optimized a discriminator network to identify the deformation
field predicted by registration network from FE-simulated motion data. The
MRI-TRUS registration network simultaneously aimed to maximize similarity
between anatomical labels to drive image alignment and minimize an adversarial
generator loss that measured divergence between the predicted and simulated
deformation. Hu et al., 2018b  also investigated using various types of
identifiable corresponding anatomical structures to train the network for the
prediction of dense voxel correspondence. These anatomical structures could be
prostate region, vessels, point landmarks or other ad hoc structures. Later,
Hu et al., 2018c  extended the work (Hu et al., 2018b, ), by designing a new
multiscale Dice for the network training and an improved memory-efficient
network architecture. Hu et al., (2019) applied segmentation concepts to
registration, predicting the transformed region-of-interest (ROI) directly
without predicting the deformation field. Zeng et al., 2020b  employed a CNN
for segmentation and then supervised the training of registration network
using a surface loss. Chen et al., (2021) also used segmentation-based
learning to train the MRI-TRUS registration network for the application of
prostate brachytherapy. Song et al., (2023) trained a CNN to extract landmarks
and then leveraged these landmarks for affine registration.

(c) Point Set Network. Baum et al., (2020) described a point set registration
algorithm based on a free point transformer network for points extracted from
segmented MRI-TRUS surfaces. Based on the preliminary work in (Baum et al.,,
2020), Baum et al., (2021) developed a partial point-set registration network,
which allowed non-linear matching without known correspondence or heuristic
constraints. Fu et al., (2021) utilized tetrahedron meshing to generate
volumetric prostate point clouds from segmented masks, and trained a point
cloud matching network using deformation fields generated by FE analysis. Min
et al., (2023) developed a physically-informed neural network to impose
elastic constraints on the estimated displacements, and formulated an end-to-
end registration network training algorithm by minimizing surface distances as
estimated boundary conditions in partial differential equations.

Table 4: Overview of deep Learning methods for prostate TRUS registration. Deep Learning Methods | References |  Dimensions Modalities | Deformation | Methods | Samples | Results
---|---|---|---|---|---|---
Metrics | Values: final (initial)
Image-based Network | Yan et al., (2018) |  3D TRUS 3D MRI | Linear |  1) Adversarial learning 2) Supervised-learning using ground truth transformation | 763 | TRE |  $3.48$ ($6.11$) mm
Haskins et al., (2019) |  3D TRUS 3D MRI | Linear |  1) Use CNNs to learn similarity metrics for MRI–TRUS 2) Composite optimization strategy | 679 | TRE |  $3.86$ ($16$) mm
Guo et al., (2020) |  3D TRUS 3D MRI | Linear |  1) Coarse-to-fine multi-stage framework 2) Generate different training data distributions | 679 | TRE |  $3.57\pm 2.03$ ($8$) mm
Song et al., (2021) |  3D TRUS 3D MRI | Linear |  1) Cross-modal attention mechanism | 662 | TRE |  $3.63\pm 1.86$ ($8$) mm
Guo et al., (2021) |  2D TRUS 3D TRUS | Linear |  1) Dual-branch feature extraction module 2) Differentiable 2D slice sampling module | 619 | TRE |  $2.73$ ($5.86$) mm
Guo et al., (2023) |  2D TRUS 3D TRUS | Linear |  1) Frame-to-slice correction network 2) Similarity filtering mechanism | 618 | TRE |  $6.95\pm 8.03$ (n/a) mm
Label-driven Network | Hu et al., 2018a  |  3D TRUS 3D MRI | Non-linear |  1) Weakly-supervised anatomical-label-driven network 2) Adversarial deformation regularization | 108 |  TRE DSC |  $6.3$ (n/a) mm $82$ (n/a) $\%$
Hu et al., 2018b  |  3D TRUS 3D MRI | Non-linear |  1) Label-driven weakly-supervised learning | 111 |  TRE DSC |  $4.2$ (n/a) mm $88$ (n/a) $\%$
Hu et al., 2018c  |  3D TRUS 3D MRI | Non-linear |  1) Multiscale Dice for weakly-supervised training 2) Memory-efficient network architecture | 108 |  TRE DSC |  $3.6$ (n/a) mm $87$ (n/a) $\%$
Hu et al., (2019) |  3D TRUS 3D MRI | Non-linear |  1) Conditional image segmentation paradigm | 115 |  TRE DSC |  $2.1$ (n/a) mm $92$ (n/a) $\%$
Point Set Network | Baum et al., (2020) |  3D TRUS 3D MRI | Non-linear |  1) Free point transformer network | 108 |  TRE HD |  $4.7\pm 1.8$ ($5.1\pm 1.7$) mm $6.3\pm 1.6$ ($10.7\pm 2.6$) mm
Baum et al., (2021) |  3D TRUS 3D MRI | Non-linear |  1) Data-driven learning without heuristic constraints 2) Variable number of points in training and inference | 108 |  TRE HD |  $4.75\pm 1.45$ ($5.1\pm 1.7$) mm $6.18\pm 1.36$ ($10.7\pm 2.6$) mm
Fu et al., (2021) |  3D TRUS 3D MRI | Non-linear |  1) Biomechanical constraints via finite element analysis 2) Surface Chamber distance loss | 50 |  TRE DSC HD |  $1.57\pm 0.77$ (n/a) mm $94\pm 2$ (n/a) $\%$ $2.96\pm 1.00$ (n/a) mm
Min et al., (2023) |  3D TRUS 3D MRI | Non-linear |  1) Biomechanical constraints exerted from soft-tissue-modeling partial differential equations | 77 |  TRE |  $1.90\pm 0.52$ (n/a) mm

###  3.3 Prostate Cancer Classification and Detection

$\raisebox{-36.16663pt}{\rotatebox{90.0}{{Modalities
(Classification)}}}\begin{cases}\raisebox{-30.13885pt}{\rotatebox{90.0}{{Single-
modality}}}\begin{cases}\text{B-mode TRUS: }\begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{beerlage2001correlation}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{mohamed2003prostate}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{scheipers2003ultrasonic}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{feleppa2004ultrasonic}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{mohamed2005computer}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{llobet2007computer}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{han2008computer}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{moradi2008augmenting}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{mohamed2008prostate}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{maggio2009predictive}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{ \cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{glotsos2010multi}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{fei2012molecular}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{pareek2013prostate}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{shi2016stacked}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{huang2020texture}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{liu2021deep}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{akatsuka2022data}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{hassan2022prostate}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{sun2023three}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{huang2023transfer}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{choi2023mic}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{CEUS: }\begin{aligned} &\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{wildeboer2016multiparametric}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{azizi2017transfer}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{azizi2017detection}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{feng2018deep}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{javadi2020multiple}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{sedghi2020improving}{\@@citephrase{(}}{\@@citephrase{)}}},}\\\
&\textit{ \cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{fooladgar2022uncertainty}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{Micro-US: }\begin{aligned} &\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{shao2020improving}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{gilany2022towards}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{wilson2023self}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{gilany2023trusformer}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \end{cases}\\\ \\\ \raisebox{-30.13885pt}{\rotatebox{90.0}{{\quad Multi-
modality}}}\begin{cases}\text{TRUS+CEUS: }\begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{sun2024machine}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{TRUS+SWE: }\begin{aligned} &\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{wu2024multi}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{wu2024miccai}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \text{TRUS+SWE+CEUS: }\begin{aligned}
&\textit{\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{wildeboer2019machine}{\@@citephrase{(}}{\@@citephrase{)}}},
\cite[cite]{\@@bibref{Authors
Phrase1YearPhrase2}{wildeboer2020automated}{\@@citephrase{(}}{\@@citephrase{)}}}}\end{aligned}\\\
\\\ \end{cases}\\\ \end{cases}$ Figure 9: Prostate cancer classification
studies categorized as using single-modality and multi-modality.

####  3.3.1 Modalities

As depicted in Fig. 9, the predominant modalities for TRUS-based PCa
classification and detection methods are B-mode TRUS, as well as contrast-
enhanced ultrasound (CEUS) and micro-ultrasound (Micro-US). With the advent of
multi-modality imaging techniques, such as shear wave elastography (SWE),
there has been a growing interest in multi-modality TRUS-based PCa diagnosis
in recent years.

####  3.3.2 Classification and Detection Methods

We categorize the TRUS-based PCa diagnosis methods into two classes:
traditional methods and deep learning methods. Some representative studies are
summarized in Table 5.

##### Traditional Methods.

These methods have payed attention to analyzing the hand-crafted features
extracted from images to conduct diagnosis. Mohamed and Salama, (2005)
extracted texture features from TRUS, including the gray level dependence
matrix and gray level difference vector. These features were then ranked and
selected using a mutual information based selection algorithm to ensure that
only relevant features were used for classification using SVMs. Han et al.,
(2008) presented a method that combined multi-resolution auto-correlation
texture features with clinical features, such as tumor location and shape,
using a SVM for classification. Moradi et al., (2008) searched for the best-
performing feature subsets from texture features, radio frequency (RF) time
series features, and Lizzi-Feleppa (LF) features, and combined these optimal
subsets to construct a SVM classifier. Maggio et al., (2009) designed a multi-
feature kernel classification model based on generalized discriminant
analysis. They further employed the predictive deconvolution to reduce system-
dependent effects. Glotsos et al., (2010) developed a multi-classifier system
for discriminating between normal, infectious, and cancerous prostate based on
the texture analysis of TRUS. A two-level hierarchical decision tree was
designed to first distinguish between normal and abnormal cases, and then to
further classify the abnormal cases into infectious and cancerous categories.
Classification at each level was conducted using three classifiers - cubic
least square mapping probabilistic neural network, quadratic Bayesian, and
SVM. Wildeboer et al., (2020) constructed a classification model that utilized
TRUS, SWE, and CEUS. Initially, features concerning contrast perfusion and
dispersion were extracted from CEUS videos. Subsequently, radiomics features
were collected from all imaging modalities, and a random forest model was
employed for classification. Sun et al., (2024) collected clinical risk
factors, including age, PSA density, serum total PSA, rise time, as well as
radiomics features from target lesions identified in B-mode and CEUS. These
features were then used to develop a risk factors-radiomics combined model
through multivariable logistic regression analysis.

Table 5: Overview of TRUS-based prostate cancer classification and detection
methods.

Types | References | Methods | Samples | Results
---|---|---|---|---
Acc (%) | Sen (%) | Spe (%) | AUC (%)
Traditional Methods | Mohamed and Salama, (2005) | Texture features, support vector machine (SVM) | 33 | 87.75 | 83.33 | 90.00 | -
Han et al., (2008) | Texture features, clinical features, SVM | 51 | 96.40 | 92.00 | 95.90 | -
Moradi et al., (2008) | Texture features, radio frequency (RF) features, SVM | 35 | - | - | - | 95.00
Maggio et al., (2009) | Multi-feature kernel classification, predictive deconvolution | 37 | 89.00 | 79.00 | 89.00 | 93.00
Glotsos et al., (2010) | Texture features, multi-classifier system, hierarchical decision tree | 165 | 88.70 | 86.70 | 89.80 | -
Wildeboer et al., (2020) | Contrast enhanced features, radiomics features, random forest | 50 | - | - | - | 75.00
Sun et al., (2024) | Clinical risk factors, radiomics, multivariable logistic regression | 166 | 82.00 | 80.00 | 83.00 | 89.00
Deep Learning Methods | Shi et al., (2016) | Stacked deep polynomial network, representation learning | 70 | 86.11 | 76.40 | 91.45 | -
Azizi et al., 2017a  | Deep belief network, tissue mimicking, Gaussian mixture model | 197 | 68.00 | 63.00 | 67.00 | 72.00
Feng et al., (2018) | 3D convolutional neural network (CNN), spatial-temporal features | 27841 | 90.18 | 82.83 | 91.45 | -