# Morphology-based non-rigid registration of coronary computed tomography and
intravascular images through virtual catheter path optimization

Karim Kadry
Institute of Medical Engineering and Science
Massachusetts Institute of Technology
Cambridge, MA 02139
<EMAIL_ADDRESS>
Abhishek Karmakar
Meinig School of Biomedical Engineering
Cornell University
Ithaca, NY 14850
<EMAIL_ADDRESS>
Andreas Schuh
Biomedical Image Analysis Group
Imperial College London
HeartFlow, Inc., USA
London, UK
<EMAIL_ADDRESS>
Kersten Peterson
HeartFlow, Inc., USA
Redwood City, CA, 94063, USA
<EMAIL_ADDRESS>
Michiel Schaap
HeartFlow, Inc., USA
Redwood City, CA, 94063, USA
<EMAIL_ADDRESS>
David Marlevi
Department of Molecular Medicine and Surgery
Karolinska Institute
Stockholm, Sweden
<EMAIL_ADDRESS>
Charles Taylor
Department of Electrical Engineering
HeartFlow, Inc., USA
Redwood City, CA, 94063, USA
<EMAIL_ADDRESS>
Elazer Edelman
Institute of Medical Engineering and Science
Massachusetts Institute of Technology
Cambridge, MA 02139
<EMAIL_ADDRESS>
Farhad Nezami
Department of Surgery
Brigham and Women’s Hospital Harvard Medical School
Boston, MA 02115
<EMAIL_ADDRESS>
Corresponding Author

###### Abstract

Coronary Computed Tomography Angiography (CCTA) provides information on the
presence, extent, and severity of obstructive coronary artery disease. Large-
scale clinical studies analyzing CCTA-derived metrics typically require
ground-truth validation in the form of high-fidelity 3D intravascular imaging.
However, manual rigid alignment of intravascular images to corresponding CCTA
images is both time consuming and user-dependent. Moreover, intravascular
modalities suffer from several non-rigid motion-induced distortions arising
from distortions in the imaging catheter path. To address these issues, we
here present a semi-automatic segmentation-based framework for both rigid and
non-rigid matching of intravascular images to CCTA images. We formulate the
problem in terms of finding the optimal _virtual catheter path_ that samples
the CCTA data to recapitulate the coronary artery morphology found in the
intravascular image. We validate our co-registration framework on a cohort of
$n=40$ patients using bifurcation landmarks as ground truth for longitudinal
and rotational registration. Our results indicate that our non-rigid
registration significantly outperforms other co-registration approaches for
luminal bifurcation alignment in both longitudinal (mean mismatch: 3.3 frames)
and rotational directions (mean mismatch: 28.6 degrees). By providing a
differentiable framework for automatic multi-modal intravascular data fusion,
our developed co-registration modules significantly reduces the manual effort
required to conduct large-scale multi-modal clinical studies while also
providing a solid foundation for the development of machine learning-based co-
registration approaches.

##  1 Introduction

Coronary computed tomography angiography (CCTA) is a three dimensional image
modality that provides information on the presence, extent and severity of
obstructive coronary artery disease (CAD) (Tzimas et al. (2022)). As such,
CCTA allows for the detection of stenotic atherosclerotic sections and assists
clinicians in diagnosing CAD and planning treatment. CCTA Images can also be
used to create computational models of coronary blood flow, allowing for the
non-invasive estimation of fractional flow reserve (FFR-CT); a key diagnostic
parameter in assessing functional impairment (Uzu et al. (2019)).

Albeit widespread in use, CCTA provides primary information on luminal
anatomy, with limited capacity in assessing soft-tissue intraplaque tissue
components. CCTA also suffers from blooming artifacts in the presence of
highly attenuating calcium deposits (Kim et al. (2015); Budoff et al. (2008)),
which, combined with comparably low image resolution, creates difficulties in
resolving highly calcified arteries. Multiple studies have also been conducted
to quantify the degree to which CCTA can accurately assess CAD-related
diagnostic metrics such as luminal area (Uzu et al. (2019)), calcium
morphology (Takahashi et al. (2021)), and plaque burden (Fischer et al.
(2013); De Graaf et al. (2013); Brodoefel et al. (2009)). The majority of such
studies (Takahashi et al. (2021); Fischer et al. (2013); Uzu et al. (2019);
Brodoefel et al. (2009)) validate the performance of CCTA by manually co-
registering image slices taken along the CCTA artery to intravascular imaging
modalities such as intravascular ultrasound (IVUS) and optical coherence
tomography (OCT); both providing higher-fidelity visualization of the lumen
and surrounding tissue. There is also an increasing interest in validating
CCTA-derived segmentation algorithms against co-registered intravascular
imaging frames, again necessitating such multimodal image assessment (Lin et
al. (2021); van Assen et al. (2019)).

Manual co-registeration of CCTA and intravascular images is, however, a
challenging and time consuming task. Typically, cross-sectional frames of the
artery of interest are extracted from the CCTA images which then have to be
matched with corresponding frames from an intravascular acquisition through an
imaging catheter pullback procedure. Rigid registration in the longitudinal
and rotational directions is usually achieved by matching single landmarks in
both modalities, such as a large bifurcation (Takahashi et al. (2021)).
However, the beating of the heart, the irregular motion of the imaging
catheter, and the rotation of the catheter about its own axis create non-rigid
distortions that accumulate along the length of the pullback (Tsiknakis et al.
(2021)). Manually correcting for such artifacts is prohibitively time-
consuming, requiring a cardiologist to manually mark fiduciary points in both
images and shift images such that the annotated points sufficiently align
(Carlier et al. (2014); Tu et al. (2011); Hebsgaard et al. (2015)). Although
such techniques are accurate up to rigid translation, they require time
investment from a trained expert to find matching features in both modalities,
creating a need for computational algorithms that non-rigidly register CCTA
images to corresponding intravascular data in an automatic fashion.

Automatic co-registration techniques typically consist of discretely
optimizing a constructed cost function over a set of longitudinal or
rotational image shifts, where the cost function varies depending on the
modalities being registered. Some proposed cost functions include metrics such
as lumen diameters (Qin et al. (2021)), lumen contours (Molony et al. (2016);
Karmakar et al. (2020)), calcium thickness (Gharaibeh et al. (2020); Molony et
al. (2016)), and image pixel intensities (Tsiknakis et al. (2021)). Similarly,
rigid rotational registration for intravascular pullbacks has also been based
on extracted features such as luminal contours (Karmakar et al. (2020)), and
calcium angle (Molony et al. (2016)). However, the registration accuracy of
all rigid registration methods is compromised by inconsistent motor pullback
speeds and rotational drift, which introduce non-rigid longitudinal and
rotational distortions that misalign image features such as diseased plaque
and bifurcations.

To compensate for the longitudinal, rotational, and transverse motion of the
catheter, several non-rigid registration approaches have been proposed,
typically to be employed after initial rigid alignment. Currently, non-rigid
registration of multiple intravascular imaging datasets has been predominantly
performed through Dynamic Time Warping (DTW) and Dynamic Programming (DP)
(Tsiknakis et al. (2021); Molony et al. (2016)). However, DTW introduces non-
physiological assumptions into the registration process by discretely skipping
or repeating intravascular frames, assumed to be evenly spaced along the
longitudinal direction. As a result, DTW is not well suited for use for
intravascular images, with pullback acquisitions sometimes rendering up to 10
repeated intravascular imaging frames at a time (Molony et al. (2016)). On the
contrary, continuous non-rigid registration methods have been developed to
model the longitudinal stretch and rotational drift between intravascular
imaging frames using affine transforms and spline interpolation (Zhang et al.
(2014); Uzu et al. (2019)). While such continuous non-rigid methods are more
realistic, they extensively rely on manual annotations of all bifurcation
zones for image registration, severely limiting their scalability. As such,
there is no continuous non-rigid registration method as of yet that does not
explicitly require fiduciary landmarks for rotational and longitudinal
alignment. Further, there has been an increasing interest in machine learning
approaches to image co-registration in which a neural network is trained to
predict a spatial transform that maps a moving image onto a static target
image (Balakrishnan et al. (2019); Fu et al. (2020)). Such approaches
critically rely on a differentiable and continuous spatial transform allowing
for back-propagation of gradients to adjust the neural network weights
(Jaderberg et al. (2015)). While such continuous and differential spatial
transforms are available for co-registration of 3D and 2D medical images, a
similar framework that accounts for the unique variation in intravascular
catheter motion has not been developed.

Given the previous limitations noted in prior co-registration algorithms, we
here propose a novel semi-automatic framework that takes as input an
intravascular imaging pullback and a CCTA 3D image and aligns each
intravascular image frame along the artery to the equivalent frame in the CCTA
image. The proposed continuous registration methodology does not require
manual matching of landmarks, with the only manual effort being the selection
of viable intravascular imaging frames and the provision of a rough centerline
within the CT image. Specifically, we explore the problem of reconstructing
the path of a _virtual catheter_ moving through and sampling from a 3D CCTA
image such that the set of frames produced by the motion of the catheter
optimally reflect the equivalent target intravascular pullback. Key
contributions of this framework include:

  * •

We present the first continuous co-registration framework for rigid and non-
rigid matching of CCTA images and intravascular images up to pixelwise
alignment, with segmentations of the lumen and vessel wall as sole input.

  * •

We introduce a rigid registration approach that consists of our published
longitudinal rigid registration algorithm, which uses lumen area in a multi-
step decision process, and a rotational registration step that leverages the
segmentation of the vessel wall to produce an initial rotational configuration
for subsequent registration.

  * •

We introduce a novel non-rigid registration step, based only on the lumen
segmentation, which is robust to physiological catheter motions. The
registration is formulated in terms of finding the path of a virtual catheter,
which translates the CCTA image into an intravascular-like image by sampling
the segmentation along the virtual catheter path. The virtual catheter path is
reconstructed by spatially deforming the CCTA centerline by B-spline
deformations formulated in the longitudinal, rotational, and transverse
directions, ensuring a smooth and physiological reconstruction of catheter
motion.

  * •

Our non-rigid registration module being both continuous and differentiable,
allows for easy integration into future machine-learning-based approaches for
intravascular image registration.

  * •

We validate in a direct clinical setting, evaluating performance across a
multimodal cohort of cardiac patients($n=40$) and benchmarking performance
against previously developed state-of-the-art approaches.

##  2 Methodology

An overview of the co-registration pipeline is detailed in Figure 1. In brief,
bi-modality images are processed to produce binary segmentations of the lumen
and vessel wall (section 2.1.1), which are first used in a rigid registration
step, involving both longitudinal and rotational alignment (section 2.1.2).
The rigid registration is then used as an initial estimate of a virtual
catheter path forming the basis for a non-rigid registration (section 2.1.3).
The virtual catheter path initially samples the geometry of the CT lumen to
produce a virtual imaging pullback that is then compared to a Signed Distance
Field (SDF) derived from the intravascular equivalent. A non-rigid
transformation for the longitudinal, rotational, and transverse motion
distortions is applied on the virtual catheter path and optimized to align the
SDF’s in both modalities. The performance of our proposed co-registration
algorithm is then validated on a clinical cohort of relevant cardiac patients
(section 2.2.2)

Figure 1: Overview of the proposed registration pipeline. The imaging
modalities are rigidly co-registered in the longitudinal and rotational
directions, serving as the basis for the initialization of the virtual
pullback trajectory. The virtual pullback trajectory is then used to sample a
CT lumen signed distance field (SDF), used in direct comparison to the
equivalent OCT SDF.

###  2.1 Co-registration framework

####  2.1.1 Preprocessing

As the basis for our co-registration pipeline, luminal segmentations from the
two different image modalities are provided. Starting with the intravascular
image set, luminal frame-by-frame segmentations are used to produce an SDF
using a fast marching method (Treister and Haber (2016)), clamped to only have
negative values (indicating that a pixel is inside the lumen). Further, the
SDF is smoothed in the axial direction with a Gaussian convolutional kernel of
size 3 and standard deviation 0.1 in order to regularize the optimization
process.

Coupled to the intravascular image set, a corresponding 3D SDF from the CCTA
images is generated. Although several methods could potentially be applied for
such, a convenient approach is to derive the SDF from a computational mesh of
the coronary tree. Herein, to create an SDF a narrow band is defined within
the object mesh boundary, subsequently used to compute exact Euclidean
distances from each voxel center to the boundary. Outside the object boundary,
the distance field values are then set to zero. Corresponding binary
segmentations can then be produced by simple thresholding operations. Using
these computational meshes, vessel centerlines are obtained using VMTK (Antiga
et al. (2008)), generating an array $\bar{r}$ representing n spatial positions
with an axial spacing of 0.2mm. A spatial derivative is then applied to the
centerline points $\bar{r}$, defining a tangent vector T for each point. The
two vectors U and V that are orthogonal to the tangent vector can then be
obtained through the parallel transport method (Guo et al. (2013)), ensuring
that the vectors V and U remain stable between frames placed along the axial
direction. The centerline points and the orthogonal vectors hence define a set
of frames($\bar{r}$,T,U,and V) in 3D space that are used to sample the CCTA
SDF along an equivalent _virtual catheter_ pullback, with dimensions equalling
the intravascular dataset (in our case: 96x96x$N_{frames}$ with an in-plane
resolution of 80 micrometers), all using a curved-planar reformation procedure
(Kanitsar et al. (2002)). The resulting SDF is then smoothed in the axial
direction with a Gaussian convolutional kernel of size 3 and standard
deviation of 0.1. Through this method, virtual pullbacks of both the lumen and
vessel segmentations were produced.

Figure 2: Overview of the proposed rigid registration pipeline. The lumen
segmentation area vectors from both modalities are used to rigidly register
the modalities in the longitudinal direction using a sliding window approach.
The longitudinal registration is then used to match each equivalent frame for
the rotational registration. The vessel wall segmentations are then converted
to vessel thickness-arc angle plots and are used to determine an optimal rigid
rotation.

####  2.1.2 Rigid registration

An overview of the rigid registration step can be seen in Figure 2. For the
rigid longitudinal registration, the processed lumen segmentations are used to
create an area vector of equal lengths, sampling the CT virtual pullback to
correspond to the acquired intravascualr set. Here, We leverage our previous
work to rigidly align the pullbacks using a multi-step sliding window method,
minimizing the difference in area vectors (for details see (Karmakar et al.
(2020))). Before registration, continuous segments of the OCT pullback with
poor lumen segmentations due to residual blood or catheter housing were
manually excluded.

For rigid rotational registration, the luminal profiles were deemed unreliable
for producing good alignment. Therefore, the vessel border segmentations were
instead used for rotationally aligning the pullbacks. For each CT and
intravascular frame, respectively, a thickness-arc angle vector is extracted
by tracing a set of radial rays from the centroid of the vessel segmentation
in increments of 12 degrees. The thickness vectors are then matched according
to the result of the longitudinal registration, with non-overlapping frames
subsequently cropped. The optimal rigid rotation angle is then obtained by
sliding the set of CT thickness vectors over each equivalent intravascular
image vector and minimizing the mean squared error across all frames.

####  2.1.3 Non-rigid registration

The non-rigid registration process (Figure 3) consists of optimizing a set of
frame variables ($\bar{r}$, T, U, and V) representing a virtual catheter path
moving through the CCTA image. The loss function to be optimized is defined as
the mean squared error between the 3D SDF generated from the two image sets,
with the CCTA-SDF sampled along the aforementioned virtual catheter path. The
virtual pullback is initialized as the centerline that was calculated from the
CCTA 3D model and longitudinally cropped and rotated according to the output
of the rigid registration. After rigid registration a spline is defined based
on the centerline points $\bar{r}$ where the centerline points are fully
described by their arclength values $\bar{s}$ along the spline. Accordingly,
every i-th frame can be manipulated by 4 variables, representing the arclength
along the virtual catheter path $s_{i}$, the rotation angle of the frame
$\theta_{i}$ about the catheter path T, and the in-plane transverse
displacements $d_{u}$ and $d_{v}$ along the frame vectors U and V respectively
(see Figure 3).

To regularize the motion of the virtual catheter to be smooth and
physiological, the 4 frame manipulation variable sets are parametrized by a
sparse set of control points controlling a B-spline deformation (Rueckert et
al. (1999)) independently acting on 4 nx1 vectors representing the frame
manipulation variables $\bar{s},\bar{\theta},\bar{d_{u}},$ and $\bar{d_{v}}$.
Thus, for a 1D control point grid of size N, the relation between a frame
manipulation variable v and the control points p can be described by:

$\textbf{v}(s)=\sum^{N}_{i=0}\textbf{B}_{i}(s)\textbf{p}_{i},$ (1)

where $B_{i}(s)$ is a polynomial basis function of order 2. In matrix form,
the same can be represented by:

$\textbf{V}=\textbf{B}\textbf{P},$ (2)

in which $\textbf{V}\in\mathbb{R}^{n\times 1},\textbf{B}\in\mathbb{R}^{n\times
N},\textbf{P}\in\mathbb{R}^{N\times 1}$ where n is the number of frames and N
is the number of control points. $B$ is the univariate B-spline tensor and can
be pre-computed from the initial frame manipulation variable vectors, while P
is the deformed control point grid vector that is optimized during co-
registration.

Figure 3: Overview of the spatial deformation acting on the virtual catheter
path. The longitudinal transform stretches and compresses the space between
adjacent frames, at which point the frame vectors (T,U, and V) are
recalculated. The rotational transform is then applied to the frame vectors
orthogonal to the tangent (U and V) about T, and the transverse transform is
then applied to shift the centerline points in the direction of the new frame
vectors (U and V).

Instead of directly optimizing for the set of $N_{s}=30$ control points
$P^{s}$ controlling the arclength variables $s$ for each frame, the control
point deformations $\Delta P^{s}_{i}$ can be parametrized by a deformation
vector $X^{s}$ of size $N_{s}-1$. dictating the relative displacement of each
control point from its proximal neighbor, with the most proximal control point
being fixed. This is done to account for the cumulative effect of catheter
motor speed variations on the rest of the pullback. Therefore, the deformation
of each control point can be defined as the cumulative sum of the relative
deformations along the proximal control points. Moreover, to regularize the
catheter motion and prevent backwards movement, the relative deformation of
each control point is limited to a fraction (0.35) of the distance between
control points.

$\Delta P^{s}_{i}=X^{s}_{i}+\sum^{i-1}_{j=0}X^{s}_{j}$ (3)

Once the control points are deformed into a new configuration, the new
arclength values for each frame $\bar{s}$ is calculated through equation 2 and
the frame vectors (T,U, and V) are then recalculated.

$\bar{s}=B^{s}P^{s}$ (4)

####  2.1.4 Non-rigid rotational registration

Similar to the longitudinal registration, the set of $N_{\theta}=20$ control
points $P^{\theta}$ controlling the rotation of each frame about the catheter
axis can be parameterized by a relative rotation vector $X^{\theta}$ of size
$N_{\theta}$. The rotation value for each control points is defined by:

$\Delta P^{\theta}_{i}=X^{\theta}_{i}+\sum^{i-1}_{j=0}X^{\theta}_{j}$ (5)

The rotation correction for each frame is applied after the non-rigid
longitudinal transformation but before the non-rigid transverse
transformation. Once the control points are deformed into a new configuration,
the new rotation values for each frame $\bar{\theta}$ can be calculated
through equation 2 and used to rotate frame vectors U and V about the tangent
vectors T.

$\bar{\theta}=B^{\theta}P^{\theta}$ (6)

####  2.1.5 Non-rigid transverse registration

The virtual catheter was biased to stay close to the centerline by optimizing
the $N_{d}=60$ control points determining the in-plane transverse
displacements $d_{u}$ and $d_{v}$ directly. Thus the 2 orthogonal transverse
displacements for each frame was calculated from the matrix relation:

$\bar{d}=B^{d}P^{d}$ (7)

Where for each frame the displacements along the vectors U and V were applied
as a final step after the non-rigid longitudinal and rotational transforms.

###  2.2 Performance evaluation

####  2.2.1 Image data

To evalute our proposed co-registration framework, a dataset consisting of
$n=40$ matched OCT and CT image pairs from 5 different clinical centers were
selected, all originating from the Precise Percutaneous Coronary Intervention
Plan (P3) study (Nagumo et al. (2021)). As each OCT pullback image consisted
of 375 frames, the intravascular imaging dataset comprised of approximately
15,000 image frames. The OCT lumen in every frame was manually annotated by
trained cardiologists. Further, the vessel wall borders in every OCT frame
were segmented using a convolutional neural network, using the previously
published U-net as base architecture (Ronneberger et al. (2015)). Details of
the network, training, and validation can be found in Supplementary Material
A. The lumen and vessel wall segmentations were then re-sampled to represent a
3D image of dimensions 96x96x$N_{frames}$ with an in-frame resolution of 80
micrometers and an out-of-frame resolution of 0.4 mm. All utilized
intravascular pullbacks were manually deemed as of sufficient image quality,
with appropriate quality lumen segmentations. For the CCTA data, a 3D model of
the coronary tree for each patient was produced by HeartFlow using the CCTA
image (Sonck et al. (2022)). The 3D model was then used to produce a 3D SDF
with a resolution of 0.25mm along each axis with an image dimension of
768x768x482.

####  2.2.2 Co-registration accuracy

In order to evaluate the performance of the non-rigid registration, 114
bifurcations were manually marked in the OCT pullback as well as in the rigid
and non-rigid virtual pullback segmentations generated from the CCTA data.
Bifurcations were defined as the last image frame before a visual coronary
artery split into two branches could be seen. Bifurcations that were common to
both modalities had their frame numbers recorded for validation of the non-
rigid registration algorithm. Longitudinal validation was conducted by
comparing the frame number of a bifurcation in the OCT data with the
equivalent bifurcation frame number in the virtual pullback before and after
non-rigid registration. In order to validate the non-rigid rotational
registration, the bifurcation angle difference between OCT pullback and the
virtual pullback was compared before and after rotational registration. As the
bifurcation angle between bifurcation sections that were not longitudinally
matched is expected to be uncorrelated, a separate analysis was conducted to
characterize how angular mismatch varies when the bifurcations are
longitudinally matched. Furthermore, only bifurcations that had a frame
mismatch below 6 frames were considered for extensive analysis of rotational
accuracy.

####  2.2.3 Comparison to alternative approaches

The most common co-registration methodology employed for coronary artery
registration has been discrete optimization approaches such as DTW and Dynamic
Programming. Therefore, in order to evaluate the performance of our
longitudinal and rotational co-registration framework against state-of-the-art
discrete approaches, we applied the methodology described in Karmakar et. al
(Karmakar et al. (2022)) on the same dataset used in this study. The approach
utilizes DTW to longitudinally align two coronary imaging modalities and
Dynamic Programming to rotationally align each frame. We utilized a window
length of 4 (0.8mm) as implemented in the previous study and recorded
identical alignment metrics for 114 matched bifurcations in the dataset. The
non-rigid registration algorithm was applied after the rigid longitudinal
registration step described in section 2.1.2. A substudy was also conducted in
which the angular alignment of all bifurcations was compared to the angular
alignment of longitudinally matched bifurcations.

####  2.2.4 Optimization details

The gradient descent-based optimization procedure was implemented in PyTorch
with the Adam optimizer (Kingma and Ba (2014)). A learning rate of 0.001 was
used for the non-rigid longitudinal parameters and a rate of 0.01 was used for
both the rotational and transverse parameters. Each co-registration procedure
was run for a minimum of 200 iterations to ensure convergence.

##  3 Results

###  3.1 Longitudinal Registration

Longitudinal registration plots in Figures 4 and 7A1-2 show that using rigid
registration alone (Figure 7A1), few bifurcations were longitudinally aligned
within 6 (dotted line), 4 (dashed line), or 2 (solid line) frame distances.
However, after non-rigid alignment (Figure 7A2), distinct improvement can be
observed with a majority of bifurcations are aligned within 6 frames. These
results are visualized by the longitudinal mismatch plot (Figure 6A),
revealing that after rigid alignment, the percentage of bifurcations matched
within 2, 4, and 6 frames are 26.3, 42.1, and 57.9%, respectively, while after
non-rigid alignment, these values increase to 60.5, 78.9, and 86.8%. Moreover,
the scatterplot for non-rigid registration (A2) demonstrates that the majority
of bifurcations (86% shown in green) were enhanced in terms of frame
alignment, while a negligible number of bifurcations had slightly (11.4 %
shown in orange) or significantly (2.6 % shown in red) worse alignment after
non-rigid registration. Table 2 further demonstrates the effect of non-rigid
registration, in which the mean frame difference after rigid registration was
7.9 frames and subsequently decreased to 3.3 frames after non-rigid
registration.

Figure 4: Qualitative results for a single co registered case. Top row shows
area plot along the artery for the non-rigidly registered CT (green) and the
OCT images. The bifurcation zones (Sections A-G) are marked and labeled for
further analysis. Bifurcation frames from the CT, OCT, and overlapped
segmentation maps are presented in the bottom row for qualitative analysis of
the rotational and transverse co-registration.

Figure 5: Quantitative results comparing the quality of rigid and non-rigid co
registration in longitudinal and rotational directions with varying degrees of
misalignment. The mismatch plots exhibit the % of matched bifurcations with
increasing longitudinal (A) and rotational (B) alignment mismatch criteria
(x-axis).

Figure 6: Grid plot showing multiple aligned bifurcation segmentations using
an SDF-based loss. Angular mismatches up to 10, 20, and 30 degrees are shown
in the first, second, and third columns respectively.

###  3.2 Rotational Registration

Examination of the individual bifurcating frames in figure 4 for the CT (row
1) and OCT (row 2) frames indicates excellent rotational and transverse
alignment between both imaging modalities as evident from the raw images and
the overlapped segmentations (row 3). Rotational registration plots in figure
7B1-2 demonstrate that few bifurcations are rotationally aligned within 30
(dotted line), 20 (dashed line), or 10 (solid line) degrees after rigid
alignment (B1). After non-rigid alignment (Figure 7B2), a majority of
bifurcations were aligned within 30 degrees, with a significant amount aligned
within 20 and 10 degrees. Examination of the rotational mismatch plot (Figure
6B) quantitatively demonstrates an increase in the percentage of bifurcations
aligned up to an angular mismatch of 10, 20, and 30 degrees from % values of
25.3, 40.4, and 52.3 to 51.5, 69.7, and 79.8% respectively. Similarly, the
non-rigid registration scatterplot demonstrates that the majority of
bifurcations had their angular mismatch decreased after non-rigid alignment
(66% shown in green) and only a minority had their angular mismatch values
slightly (22% shown in orange) or significantly (12% shown in red) increased.
The mean value of the angular mismatch before and after non-rigid alignment is
reported in Table 2, in which the mean angular mismatch decreases from 36.0 to
28.6 degrees.

Figure 7: Quantitative results comparing the quality of rigid and non-rigid
co-registration in longitudinal and rotational directions. The first row
compares bifurcation frame mismatch before (A1) and after (A2) non-rigid
registration in the form of scatterplots. The second row compares bifurcation
angular mismatch before (B1) and after (B2) non-rigid registration in the form
of scatterplots. The scatterplot for the longitudinal and rotational non-rigid
registration (A2 and B2) are color-coded to exhibit the change in alignment
metric after non-rigid registration, where green represents an increase in
alignment, orange represents a mild decrease in alignment, and red represents
a strong decrease in alignment. Only bifurcations that were longitudinally
matched within 6 OCT frames were analyzed for rotational alignment.

###  3.3 Comparison with previous approaches

A direct comparison of the virtual catheter method with state-of-the-art
discrete optimization approaches can be seen in Tables 2 and 3. Comparing the
virtual catheter method to a discrete optimization approach for longitudinal
registration, it was shown that DTW produces significantly poorer results in
longitudinal registration, with the longitudinal mismatch of 11.7 frames being
higher than rigid longitudinal registration average of 7.9 frames. Comparing
the virtual catheter method to using Dynamic Programming for rotational
registration, it was shown that such discrete optimization algorithms exhibit
poor performance for CT-OCT rotational registration (angular mismatch of 77.9
degrees) which is higher than the angular mismatch after rigid rotational
registration alone. Table 3 quantifies the angular mismatch in the case where
non-rigid longitudinal registration is successful. For bifurcations with a
maximum frame mismatch of 6 after non-rigid registration, the angular mismatch
decreases from 77.9 to 65.2 for the Dynamic Programming approach, while for
the virtual catheter method, the angular mismatch decreases from 28.6 to 24.8.
In contrast, the angular mismatch for the rigid registration is unchanged
after excluding non-matching bifurcations.

Figure 8: Qualitative results comparing the alignment of calcium annotations
between OCT (first row) and CT (third row) for selected frames with good
luminal alignment. The middle row shows the calcium annotations for OCT (red)
and CT (green) superimposed on each other. Table 1: Accuracy of alternative
co-registration approaches, proposed for intravascular-intravascular image
registration. Data is presented from left to right including evaluated co-
registered modalities, dataset size, and overall methodological approach.
Further, average errors are presented in both longitudinal (frames) and
rotational (degree) directions.

Ref. | Modalities | Dataset Size | Methodology | Longitudinal mismatch | Angular mismatch
---|---|---|---|---|---
Karmakar et al. (2022) | OCT-OCT | 9 patients | DTW + Dynamic Programming | $0.9\pm 0.8$ | $7.7\pm 6.7$
Tsiknakis et al. (2023) | OCT-OCT | 21 patients | DTW + Harmony Search | $5.6\pm 6.7$ | $1.2\pm 0.81$
Karmakar et al. (2022) | OCT-IVUS | 7 patients | DTW + Dynamic Programming | $1.45\pm 0.7$ | $29.1\pm 23.2$
Molony et al. (2016) | OCT-IVUS | 12 patients | DTW + Dynamic Programming | $5.0\pm 6.2$ | $17.8\pm 21.9$

Table 2: Accuracy of co-registration approaches applied to CT-OCT image
registration. Data is presented from left to right including evaluated co-
registered modalities, dataset size, and overall methodological approach.
Further, average errors are presented in both longitudinal (frames) and
rotational (degree) directions. All approaches in this table have been
evaluated on the same dataset

Ref. | Modalities | Dataset Size | Methodology | Longitudinal mismatch | Angular mismatch
---|---|---|---|---|---
Karmakar et al. (2022) | CT-OCT | 40 patients | DTW + Dynamic Programming | $11.7\pm 12.1$ | $77.9\pm 61.0$
Ours (Rigid) | CT-OCT | 40 patients | Virtual Catheter Method | $7.9\pm 7.1$ | $36.0\pm 31.9$
Ours (Rigid+Non-rigid) | CT-OCT | 40 patients | Virtual Catheter Method |  $3.3\pm 3.9$ |  $28.6\pm 40.9$

Table 3: Accuracy of co-registration approaches applied to CT-OCT image
registration for bifurcations that are longitudinally matched. Data is
presented from left to right including methodological approach and number of
longitudinally matched bifurcations. Bifurcations are considered
longitudinally matched when they have a maximum frame difference of 6 after
non-rigid longitudinal registration. Further, average errors are presented for
rotational direction in degrees.

Ref. | Methodology | Matched Bifurcations | Angular Mismatch
---|---|---|---
Karmakar et al. (2022) | DTW + Dynamic Programming | 52/114 | $65.2\pm 72.9$
Ours (Rigid) | Virtual Catheter Method | 99/114 | $36.0\pm 33.0$
Ours (Rigid+Non-rigid) | Virtual Catheter Method | 99/114 | $24.8\pm 39.0$

##  4 Discussion

The aim of the current study was to develop a fully automatic registration
algorithm to align CCTA and intravascular images. Specifically, we propose a
novel registration process finding the optimal rigid and non-rigid spatial
transforms using a _virtual catheter_ path in the CCTA data, aligning the non-
invasive modality to its invasive counterpart. Our results indicate excellent
co-registration accuracy, with excellent agreement with reference manual
landmark annotations (Figure 4). Further, our results underline the critical
importance of a non-rigid registration step, with significant enhancement in
both longitudinal and rotational alignments observed when comparing rigid vs.
non-rigid alignments in Table 2. We demonstrate that for the vast majority of
bifurcations, our framework is able to improve the longitudinal and rotational
alignment of common bifurcations within the CT and OCT images. Lastly, we
demonstrate the added value of our approach as compared to state-of-the-art
alternatives, with a head-to-head comparison to previously developed discrete
optimization alignment algorithms (Table 1). A head-to-head comparison
demonstrates that discrete optimization approaches for longitudinal and
rotational alignment suffer a significant drop in alignment quality when
applied for the task of CT-OCT co-registration. Meanwhile, our approach
maintains performance accuracy in line with simpler tasks such as
intravascular-intravascular image registration.

###  4.1 Related work

Currently, a majority of CCTA studies that validate their CT findings with
intravascular images have used rigid manual registration based on fiduciary
landmarks such as bifurcations or large calcifications (Carlier et al. (2014);
Tu et al. (2011); Hebsgaard et al. (2015)). One of the few studies that
attempted to align CT and intravascular data up to a non-rigid level is by Uzu
et al. (2019) in which a B-spline deformation model was used to optimize the
alignment of manually annotated bifurcation landmarks. Though powerful, such
an approach is time-consuming due to the significant amount of manual
processing required to process the OCT images, rigidly align the geometric
models, and mark bifurcations within every artery. In comparison, our approach
implicitly matches nearby bifurcations using longitudinally smoothed SDFs
representing the CT and OCT lumens, respectively. Other approaches that
register intravascular-to-intravascular modalities have in the past relied on
DTW (Molony et al. (2016); Karmakar et al. (2022)), discretely optimizing the
frame-wise progression of one intravascular pullback to maximize longitudinal
and rotational alignment with another. Such methods, nevertheless, attempt to
recapitulate the continuous motion distortions introduced by the catheter path
with discrete non-physiological frames or repeats (Molony et al. (2016)).
However, skipping or repeating several frames that the catheter motion is not
smooth or continuous, which is an unrealistic assumption about the catheter
path.

Direct numerical comparison of reported co-registration accuracy across
published approaches is inherently difficult as co-registration accuracy is
highly dependent on the specific dataset explored as well as which modalities
are being coregistered. For example, the simplest co-registration task would
be represented by the alignment of same-modality images such as OCT-OCT image
pairs. For such tasks, our previously published DTW and dynamic programming
approach (Karmakar et al. (2022)) exhibits similar perfomance compared to
other state-of-the-art algorithms (Tsiknakis et al. (2023)) (See Table 1).
When applied to multimodality datasets, such as IVUS-OCT image pairs, our
previously developed approach (Karmakar et al. (2022)) suffers distinctive
drops in both longitudinal and rotational accuracy (see Table 1), however,
still maintains comparative performance to similar Dynamic Programming
approaches (Molony et al. (2016)). Thus, in order to facilitate a head-to-head
comparison on the more challenging task of registering CT-OCT image pairs, we
applied our previously developed discrete optimization algorithm (Karmakar et
al. (2022)) on our multi-modal dataset of 40 patients. Doing so, we found that
our previous approach produced significantly worse longitudinal and rotational
alignment compared to the virtual catheter method, with both longitudinal and
angular alignment being worse than simple rigid registration (Table 2). In
contrast, our developed methodology achieves competitive results with even
intravascular-intravascular registration studies (Table 1 and Table 2).

###  4.2 Methodological Adaptations

From the above it can be seen that the task of co-registering CT and OCT
images presents several unique difficulties for discrete registration
algorithms. Our framework has several features that were designed to mitigate
such challenges. First, the low resolution of CT images induces a circular
bias in the lumen segmentations (see Figure 4), as well as a tendency to miss
small bifurcations. Such circularly symmetric regions hence create zones of
longitudinal and rotational ambiguity along the pullback. Our approach tries
to minimize the dependency of such by formulating the longitudinal and
rotational transforms acting on the virtual path in terms of a regularized and
smooth B spline deformation. As such, the optimization procedure is mainly
dominated by the alignment of prominent non-symmetric features such as
bifurcations, rather than the circularly symmetric lumen segments. This
ensures that the rotational alignment of all non-bifurcating lumen frames that
are in proximity to matched bifurcations are properly matched due to B spline
interpolation (Figure 4). Another significant issue faced in previous
rotational co-registration algorithms (Karmakar et al. (2022); Molony et al.
(2016)) is that lumen bifurcations are only able to contribute to rotational
alignment if they exist within the same frame. As such, poor longitudinal
alignment of bifurcations was a significant contributing factor to the poor
performance of our previously developed dynamic programming algorithm for
rotational co-registration (Table 2). Our framework, in contrast, minimizes
this dependency through the use of a Gaussian smoothing kernel applied
longitudinally over the SDF. Longitudinal smoothing allows single-frame
bifurcations to appear in adjacent frames and smooths the loss surface such
that bifurcations in the different modalities can be better aligned (Figures 4
and 7). Another design choice that was found to increase training stability
and co-registration quality was the use of SDF’s to determine alignment, as
opposed to using a segmentation loss such as cross-entropy or Dice. This was
due to the fact that when the lumen segmentations were fully overlapping,
multiple rotational and transverse configurations contribute equally to a
segmentation loss function, preventing the algorithm from making fine
adjustments in the spatial transform. Figure 6 further demonstrates how the
quality of rotational registration varies with angular mismatch, where the
angular mismatch tends to occur due to the limitations of local optimization
of pixel alignment. At less than 10 degrees mismatch, the difference in
alignment is minimal, while under 30 degrees, the difference in alignment can
be attributed to differing lumen bifurcation shapes in the CT and OCT data.
Lastly, many co-registration methods normalize the position of the lumen by
the artery centroid (Uzu et al. (2019); Karmakar et al. (2020, 2022); Molony
et al. (2016)). While such an approach manages to align CT and OCT frames with
circularly symmetric lumen, it fails to effectively align equivalent frames
with bifurcations as the segmentation can have different maximum diameters
between the modalities and thus different centroids. Moreover, centering the
image around the lumen centroids can cause the algorithm to align bifurcations
180-degrees from the correct orientation. It was empirically found that this
phenomenon was found to be a significant contributing factor to the
degradation of the co-registration performance of our previously developed
discrete co-registration algorithm. In this framework, we instead choose to
jointly optimize for the transverse displacements of the virtual path in
addition to the longitudinal and rotational displacements, which allows for
the bifurcations in both modalities to be anchored around the OCT catheter
location and enables near pixelwise alignment of the lumen (Figures 4,6) and
plaque constituents such as calcium (Figure 8).

###  4.3 Limitations

Though very promising for clinical applications, our developed approach has a
number of limitations. First, the non-rigid spatial transform acting on the
virtual catheter path is found through gradient-based optimization, requiring
that landmarks lie sufficiently close such that proper matching is ensured.
For example, common bifurcations that have a frame mismatch of more than 6
frames (corresponding to the longitudinal smoothing kernel) are expected to be
uncorrelated in terms of orientation. This issue can be mitigated by
integrating deep learning networks which can accurately predict the spatial
transform needed to align the two modalities. Another limitation is the
dependence of non-rigid registration on the lumen. The lumen estimation is
expected to be accurate for both modalities and as such, ensures good
registration accuracy for regions that include many bifurcations. However, due
to the poor resolution of CCTA images, the lumen estimation tends to be highly
circular. Accordingly, it is expected that rotational co-registration
certainty increases with bifurcation proximity but decreases in stenotic
regions that contain highly circular CT luminal profiles. In the future, co-
registration accuracy can likely be improved by including contextual
information relating to the vessel wall such as lesion content and morphology
as a supervisory signal in the loss function. Third, the use of a pixel-wise
loss as a surrogate for luminal alignment may not necessarily result in
optimal alignment of lumen bifurcations. As seen in Figure 6, a pixel-wise
loss function can occasionally bias the spatial transform to align the central
lumen body over aligning the bifurcation in scenarios where the bifurcation
shapes are not perfectly matching. In the future, this issue can be mitigated
by introducing an orientation loss to bias the spatial transform to
rotationally align bifurcations. Lastly, regularizing the spatial transform
and smoothing the SDF’s can create difficulties in localizing landmarks up to
frame-wise precision. This can be seen in the area curve in Figure 4 section B
with the slightly mismatched bifurcation and in Figure 7 A2 and B2 with a
minor amount of bifurcations with increased frame and angular mismatch values.
The localization capabilities of the algorithm can be improved by introducing
multi-scale deformation steps where finer control point grids can be
recursively used as the basis for the spatial transform.

###  4.4 Translational Benefits

The development of automatic frame-wise matching algorithms for CT-OCT data
fusion would enable the development of several research-based applications.
First, intravascular imaging data can act as ground truth to validate the
reliability of CT in delineating several morphological metrics of
atherosclerosis, such as luminal area, lipid content, and calcium volume.
Understanding when CT-derived morphological metrics are reliable is critical
for both therapy planning and deciding when to rely on intravascular imaging.
For example, studying the interaction between calcium blooming and measured
lumen size in CT images necessitates that ground truth lumen measurements be
available, which can only be provided by frame-wise co-registration
algorithms. Figure 8 demonstrates that such a frame-by-frame comparison can be
done provided that longitudinal and rotational co-registration is of
sufficient quality. Second, multi-modal data fusion would allow for the
enhanced generation of patient-specific digital twins from coronary images.
There has been an increasing interest in the use of intravascular-images to
create computational digital twins for the prediction of coronary
pathophysiology and clinical decision-making (Kadry et al. (2021)). However,
intravascular images, while providing excellent resolution within the imaging
frame, do not provide sufficient information to create a fully physiological
artery model. Intravascular images typically suffer from intra-frame motion
drift artifacts in the longitudinal and rotational directions and cannot
capture information on the three-dimensional centerline of the artery. On the
other hand, CT suffers from poor resolution but is able to capture the three-
dimensional nature of the artery with high accuracy. Combining both modalities
would allow researchers to investigate the importance of longitudinal and
rotational distortions, as well as modeling arterial tortuosity.

## References

  * Tzimas et al. [2022] Georgios Tzimas, Gaurav S Gulsin, Hidenobu Takagi, Niya Mileva, Jeroen Sonck, Olivier Muller, Jonathon A Leipsic, and Carlos Collet.  Coronary ct angiography to guide percutaneous coronary intervention.  _Radiology: Cardiothoracic Imaging_ , 4(1):e210171, 2022.
  * Uzu et al. [2019] Kenzo Uzu, Hiromasa Otake, Gilwoo Choi, Takayoshi Toba, Hyun Jin Kim, Arjun Roy, Michiel Schaap, Leo Grady, Masahito Kawata, Toshiro Shinke, et al.  Lumen boundaries extracted from coronary computed tomography angiography on computed fractional flow reserve (ffrct): validation with optical coherence tomography.  _Eurointervention: Journal of Europcr in Collaboration with the Working Group on Interventional Cardiology of the European Society of Cardiology_ , 14(15):e1609–e1618, 2019.
  * Kim et al. [2015] Choongki Kim, Sung-Jin Hong, Dong-Ho Shin, Jung-Sun Kim, Byeong-Keuk Kim, Young-Guk Ko, Donghoon Choi, Yangsoo Jang, and Myeong-Ki Hong.  Limitations of coronary computed tomographic angiography for delineating the lumen and vessel contours of coronary arteries in patients with stable angina.  _European Heart Journal-Cardiovascular Imaging_ , 16(12):1358–1365, 2015.
  * Budoff et al. [2008] Matthew J Budoff, David Dowe, James G Jollis, Michael Gitter, John Sutherland, Edward Halamert, Markus Scherer, Raye Bellinger, Arthur Martin, Robert Benton, et al.  Diagnostic performance of 64-multidetector row coronary computed tomographic angiography for evaluation of coronary artery stenosis in individuals without known coronary artery disease: results from the prospective multicenter accuracy (assessment by coronary computed tomographic angiography of individuals undergoing invasive coronary angiography) trial.  _Journal of the American College of Cardiology_ , 52(21):1724–1732, 2008.
  * Takahashi et al. [2021] Yu Takahashi, Takayoshi Toba, Hiromasa Otake, Yusuke Fukuyama, Shinsuke Nakano, Yoichiro Matsuoka, Kosuke Tanimura, Yu Izawa, Hiroyuki Kawamori, Atsushi K Kono, et al.  Feasibility of morphological assessment of coronary artery calcification with electrocardiography-gated non-contrast computed tomography: a comparative study with optical coherence tomography.  _The International Journal of Cardiovascular Imaging_ , 37(4):1445–1453, 2021.
  * Fischer et al. [2013] Collin Fischer, Edward Hulten, Pallavi Belur, Ryan Smith, Szilard Voros, and Todd C Villines.  Coronary ct angiography versus intravascular ultrasound for estimation of coronary stenosis and atherosclerotic plaque burden: a meta-analysis.  _Journal of cardiovascular computed tomography_ , 7(4):256–266, 2013.
  * De Graaf et al. [2013] Michiel A De Graaf, Alexander Broersen, Pieter H Kitslaar, Cornelis J Roos, Jouke Dijkstra, Boudewijn PF Lelieveldt, J Wouter Jukema, Martin J Schalij, Victoria Delgado, Jeroen J Bax, et al.  Automatic quantification and characterization of coronary atherosclerosis with computed tomography coronary angiography: cross-correlation with intravascular ultrasound virtual histology.  _The international journal of cardiovascular imaging_ , 29(5):1177–1190, 2013.
  * Brodoefel et al. [2009] H Brodoefel, C Burgstahler, M Heuschmid, A Reimann, F Khosa, A Kopp, S Schroeder, CD Claussen, and ME Clouse.  Accuracy of dual-source ct in the characterisation of non-calcified plaque: use of a colour-coded analysis compared with virtual histology intravascular ultrasound.  _The British journal of radiology_ , 82(982):805–812, 2009.
  * Lin et al. [2021] A Lin, N Manral, P McElhinney, A Killekar, H Matsumoto, S Cadet, S Achenbach, SJ Nicholls, DT Wong, D Berman, et al.  Deep learning-based plaque quantification from coronary computed tomography angiography: external validation and comparison with intravascular ultrasound.  _European Heart Journal_ , 42(Supplement_1):ehab724–0161, 2021.
  * van Assen et al. [2019] Marly van Assen, Akos Varga-Szemes, U Joseph Schoepf, Taylor M Duguay, H Todd Hudson, Svetlana Egorova, Kjell Johnson, Samantha St Pierre, Beatrice Zaki, Matthijs Oudkerk, et al.  Automated plaque analysis for the prognostication of major adverse cardiac events.  _European Journal of Radiology_ , 116:76–83, 2019.
  * Tsiknakis et al. [2021] Nikos Tsiknakis, Constantinos Spanakis, Panagiota Tsompou, Georgia Karanasiou, Gianna Karanasiou, Antonis Sakellarios, George Rigas, Savvas Kyriakidis, Michael Papafaklis, Sotirios Nikopoulos, et al.  Ivus longitudinal and axial registration for atherosclerosis progression evaluation.  _Diagnostics_ , 11(8):1513, 2021.
  * Carlier et al. [2014] Stéphane Carlier, Rich Didday, Tristan Slots, Peter Kayaert, Jeroen Sonck, Mike El-Mourad, Nicolas Preumont, Dany Schoors, and Guy Van Camp.  A new method for real-time co-registration of 3d coronary angiography and intravascular ultrasound or optical coherence tomography.  _Cardiovascular Revascularization Medicine_ , 15(4):226–232, 2014.
  * Tu et al. [2011] Shengxian Tu, Niels R Holm, Gerhard Koning, Zheng Huang, and Johan HC Reiber.  Fusion of 3d qca and ivus/oct.  _The international journal of cardiovascular imaging_ , 27(2):197–207, 2011.
  * Hebsgaard et al. [2015] Lasse Hebsgaard, Troels Munck Nielsen, Shengxian Tu, Lars Romer Krusell, Michael Maeng, Karsten Tange Veien, Bent Raungaard, Christian Juhl Terkelsen, Anne Kaltoft, Johan HC Reiber, et al.  Co-registration of optical coherence tomography and x-ray angiography in percutaneous coronary intervention. the does optical coherence tomography optimize revascularization (doctor) fusion study.  _International journal of cardiology_ , 182:272–278, 2015\.
  * Qin et al. [2021] Hui Qin, Chunming Li, Yingguang Li, Jiayue Huang, Fan Yang, Takashi Kubo, Takashi Akasaka, Changyan Xiao, Juan Luis Gutiérrez-Chico, and Shengxian Tu.  Automatic coregistration between coronary angiography and intravascular optical coherence tomography: Feasibility and accuracy.  _JACC: Asia_ , 1(2):274–278, 2021.
  * Molony et al. [2016] David S Molony, Lucas H Timmins, Emad Rasoul-Arzrumly, Habib Samady, and Don P Giddens.  Evaluation of a framework for the co-registration of intravascular ultrasound and optical coherence tomography coronary artery pullbacks.  _Journal of biomechanics_ , 49(16):4048–4056, 2016.
  * Karmakar et al. [2020] Abhishek Karmakar, Max L Olender, Farhad Rikhtegar Nezami, David Marlevi, Evan Shlofmitz, Richard A Shlofmitz, and Elazer R Edelman.  Detailed investigation of lumen-based tomographic co-registration.  In _2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)_ , pages 1038–1042. IEEE, 2020.
  * Gharaibeh et al. [2020] Yazan Gharaibeh, Juhwan Lee, David Prabhu, Pengfei Dong, Vladislav N Zimin, Luis A Dallan, Hiram Bezerra, Linxia Gu, and David Wilson.  Co-registration of pre-and post-stent intravascular oct images for validation of finite element model simulation of stent expansion.  In _Medical Imaging 2020: Biomedical Applications in Molecular, Structural, and Functional Imaging_ , volume 11317, pages 306–316. SPIE, 2020\.
  * Zhang et al. [2014] Ling Zhang, Richard Downe, Zhi Chen, Shanhui Sun, T Masiarov, Tomas Kovarnik, John Lopez, Milan Sonka, and Andreas Wahle.  Side-branch guided registration of intravascular ultrasound pullbacks in coronary arteries.  In _MICCAI Workshop in Computing and Visualization for IntraVascular Imaging and Computer Assisted Stenting (CVII-STENT)_ , pages 44–51, 2014.
  * Balakrishnan et al. [2019] Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca.  Voxelmorph: a learning framework for deformable medical image registration.  _IEEE transactions on medical imaging_ , 38(8):1788–1800, 2019.
  * Fu et al. [2020] Yabo Fu, Yang Lei, Tonghe Wang, Walter J Curran, Tian Liu, and Xiaofeng Yang.  Deep learning in medical image registration: a review.  _Physics in Medicine & Biology_, 65(20):20TR01, 2020.
  * Jaderberg et al. [2015] Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al.  Spatial transformer networks.  _Advances in neural information processing systems_ , 28, 2015.
  * Treister and Haber [2016] Eran Treister and Eldad Haber.  A fast marching algorithm for the factored eikonal equation.  _Journal of Computational physics_ , 324:210–225, 2016\.
  * Antiga et al. [2008] Luca Antiga, Marina Piccinelli, Lorenzo Botti, Bogdan Ene-Iordache, Andrea Remuzzi, and David A Steinman.  An image-based modeling framework for patient-specific computational hemodynamics.  _Medical & biological engineering & computing_, 46(11):1097–1112, 2008.
  * Guo et al. [2013] Jixiang Guo, Shun Li, Yim Pan Chui, Jing Qin, and Pheng Ann Heng.  Mesh quality oriented 3d geometric vascular modeling based on parallel transport frame.  _Computers in Biology and Medicine_ , 43(7):879–888, 2013.
  * Kanitsar et al. [2002] Armin Kanitsar, Dominik Fleischmann, Rainer Wegenkittl, Petr Felkel, and Eduard Groller.  _CPR-curved planar reformation_.  IEEE, 2002.
  * Rueckert et al. [1999] Daniel Rueckert, Luke I Sonoda, Carmel Hayes, Derek LG Hill, Martin O Leach, and David J Hawkes.  Nonrigid registration using free-form deformations: application to breast mr images.  _IEEE transactions on medical imaging_ , 18(8):712–721, 1999.
  * Nagumo et al. [2021] Sakura Nagumo, Carlos Collet, Bjarne L Norgaard, Hiromasa Otake, Brian Ko, Bon-kwon Koo, Jonathon Leipsic, Daniele Andreini, Ward Heggermont, Jesper M Jensen, et al.  Rationale and design of the precise percutaneous coronary intervention plan (p3) study: Prospective evaluation of a virtual computed tomography-based percutaneous intervention planner.  _Clinical cardiology_ , 44(4):446–454, 2021.
  * Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In _International Conference on Medical image computing and computer-assisted intervention_ , pages 234–241. Springer, 2015.
  * Sonck et al. [2022] Jeroen Sonck, Sakura Nagumo, Bjarne L Norgaard, Hiromasa Otake, Brian Ko, Jinlong Zhang, Takuya Mizukami, Michael Maeng, Daniele Andreini, Yu Takahashi, et al.  Clinical validation of a virtual planner for coronary interventions based on coronary ct angiography.  _Cardiovascular Imaging_ , 15(7):1242–1255, 2022\.
  * Karmakar et al. [2022] Abhishek Karmakar, Max L Olender, David Marlevi, Evan Shlofmitz, Richard A Shlofmitz, Elazer R Edelman, and Farhad R Nezami.  Framework for lumen-based nonrigid tomographic coregistration of intravascular images.  _Journal of Medical Imaging_ , 9(4):044006, 2022\.
  * Kingma and Ba [2014] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Tsiknakis et al. [2023] Nikos Tsiknakis, Constantinos Spanakis, Panagiota Tsoumpou, Georgia Karanasiou, Gianna Karanasiou, Antonis Sakellarios, George Rigas, Savvas Kyriakidis, Michail I Papafaklis, Sotirios Nikopoulos, et al.  Oct sequence registration before and after percutaneous coronary intervention (stent implantation).  _Biomedical Signal Processing and Control_ , 79:104251, 2023\.
  * Kadry et al. [2021] Karim Kadry, Max L Olender, David Marlevi, Elazer R Edelman, and Farhad R Nezami.  A platform for high-fidelity patient-specific structural modelling of atherosclerotic arteries: from intravascular imaging to three-dimensional stress distributions.  _Journal of the Royal Society Interface_ , 18(182):20210436, 2021.