# Improve Cross-Modality Segmentation by Treating MRI Images as Inverted CT
Scans

Hartmut Häntze Department of Radiology, Charité - Universitätsmedizin Berlin
corporate member of Freie Universität Berlin and Humboldt Universität zu
Berlin, Hindenburgdamm 30, 12203 Berlin, Germany  Diagnostic Image Analysis
Group, Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA
Nijmegen, the Netherlands  Lina Xu  Department of Radiology, Charité -
Universitätsmedizin Berlin corporate member of Freie Universität Berlin and
Humboldt Universität zu Berlin, Hindenburgdamm 30, 12203 Berlin, Germany
Leonhard Donle  Department of Radiology, Charité - Universitätsmedizin Berlin
corporate member of Freie Universität Berlin and Humboldt Universität zu
Berlin, Hindenburgdamm 30, 12203 Berlin, Germany  Felix J. Dorfner  Department
of Radiology, Charité - Universitätsmedizin Berlin corporate member of Freie
Universität Berlin and Humboldt Universität zu Berlin, Hindenburgdamm 30,
12203 Berlin, Germany  Athinoula A. Martinos Center for Biomedical Imaging,
Massachusetts General Hospital and Harvard Medical School, 149 Thirteenth St,
Charlestown, MA 02129, USA  Alessa Hering  Diagnostic Image Analysis Group,
Radboud University Medical Center, Geert Grooteplein Zuid 10, 6525 GA
Nijmegen, the Netherlands  Lisa C. Adams  Department of Radiology, Klinikum
rechts der Isar, Technical University Munich, Ismaninger Str. 22, 81675 Munich
Keno K. Bressem$\ddagger$ Department of Radiology, Klinikum rechts der Isar,
Technical University Munich, Ismaninger Str. 22, 81675 Munich  German Heart
Center Munich, Technical University Munich, Lazarethstr. 36, 80636, Munich

(04.05.2024)

###### Abstract

Computed tomography (CT) segmentation models frequently include classes that
are not currently supported by magnetic resonance imaging (MRI) segmentation
models. In this study, we show that a simple image inversion technique can
significantly improve the segmentation quality of CT segmentation models on
MRI data, by using the TotalSegmentator model, applied to T1-weighted MRI
images, as example. Image inversion is straightforward to implement and does
not require dedicated graphics processing units (GPUs), thus providing a quick
alternative to complex deep modality-transfer models for generating
segmentation masks for MRI data.

$\dagger$Corresponding author<EMAIL_ADDRESS>

_Keywords_ MRI, CT, Segmentation, Cross-Modality, Radiology

##  1 Introduction

Segmentation of medical images plays a vital role in many automatic image
analysis tools. While segmentation has been well established for computed
tomography (CT) scans, with multiple open source models available [1, 2],
multi-class segmentation of magnetic resonance imaging (MRI), especially
outside the brain, is lacking behind. The main reason for this challenge is
that training segmentation models requires a large number of annotated images,
and the more classes involved, the greater the annotation effort needed. While
this problem can be partially alleviated by using augmented CT scans with
existing labels for retraining a model [3], implementing and training an
augmented model is resource-intensive, time-consuming, and technically
challenging. In this short paper, we demonstrate that image augmentation,
specifically inversion, can be sufficient to bridge the gap between MRI and CT
segmentation performance and CT segmentation model can be used to generate
masks for MR images. One key difference between MRI and CT images is that
dense tissue, such as bones, appears bright (hyperdense) in CT scans but dark
(hypointense) in MRI images. We attempt to minimize this difference by using
negatives of MRI images and analyze whether it has an effect on the semantic
segmentation performance of models trained solely on CT data.

##  2 Methods

This study was approved by the local ethics committee (EA4/062/20). Due to the
retrospective nature of the study, patient consent was waived. We utilized an
in-house dataset consisting of 90 T1-weighted and 62 T2-weighted MRI
sequences. All scans depict the abdominal region. As preprocessing we clip all
intensities to a value range between 0 and 3000 and create negatives within
their original intensity range. Then, we set all intensities within the first
percentile to zero (Equation 1). This step ensures that the surrounding area
around the patient remains black (Figure 1). Although this process may produce
some artifacts in the air-filled lungs, it proved to be very stable within the
abdominal region. We then run the TotalSegmentator model [2] on both the
original MRI images and their inverted versions. To evaluate the results, a
radiologist (LX) manually refined 20 labels to create a groundtruth. We
compare the model’s output to these ground truth labels and calculate the Dice
Similarity Coefficient (DSC). Since there is no training step involved in this
study, we evaluate the model’s performance on the entire dataset.

$INV(x)=\begin{dcases*}0&x $\in percentile_{1}(X)$\\\ max(X)-x+min(X)&x
else\end{dcases*}$ (1)

Figure 1: (a) Unprocessed T1 image, (b) inverted image, (c) inverted + black
background

##  3 Results

Table 1: Average Dice Similarity Coefficient and 95% Confidence Interval over 20 Classes Sequence | Unprocessed | Inverted | Inverted (black background)
---|---|---|---
T1 | 0.06 (0.02, 0.10) | 0.29 (0.23, 0.35) | 0.61 (0.53, 0.68)
T2 | 0.43 (0.32, 0.53) | 0.03 (0.01, 0.05) | 0.08 (0.04, 0.12)

Without preprocessing, TotalSegmentator fails to detect any classes in the
T1-weighted sequences, with the exception of the colon (DSC=0.40). For
T2-weighted sequences, TotalSegmentator can partially segment large organs but
struggles to segment blood vessels and muscles. Inverting both T1 and T2
sequences leads to improved segmentation quality for T1-weighted images but
decreases the performance for T2-weighted images (Table 1). Setting the
background intensity to zero significantly enhances the segmentation quality
for T1-weighted sequences. Improvements are observed across all classes,
including small vessels and organs such as the adrenal glands (Figure 2).

Figure 2: (a) Mean Dice of segmentation with TotalSegmentator for original and
inverted T1-weighted MRI images. Segmentation before (b) and after (c)
inversion.

##  4 Discussion

We demonstrate that image inversion can substantially improve the segmentation
quality of models trained on CT data for T1-weighted MRI images. This approach
can be used to quickly obtain initial segmentations that can then be refined.
Given sufficient MRI sequences it may even be enough to train an nnUnet
without any new or little annotation effort [4]. The results emphasize the
importance of color gradients for the TotalSegmentator model. In particular,
the contrast between the patient and the background appears to be crucial.
Consistently setting the background in an inverted image to black improved the
segmentation of all classes, even for organs located in the center of the
body. Although the results were promising for T1-weighted images, we could not
demonstrate improvements for T2-weighted images. This is likely due to the
increased intensity of water in T2 images, which causes most organs to appear
brighter than surrounding tissue. Extending the preprocessing pipeline with
other image augmentations, such as histogram equalization, might make MRI
images appear more similar to CT scans and further improve segmentation
results. More sophisticated modality-transfer methods could potentially
increase generalizability, however, implementing these complex models can be
challenging and may be disproportionate for small-scale projects. For certain
sequences, image inversion alone can be sufficient to achieve satisfactory
results.

## Acknowledgements

The authors acknowledge the Scientific Computing of the IT Division at the
Charité - Universitätsmedizin Berlin for providing computational resources
that have contributed to the research results reported in this paper
(https://www.charite.de/en/research/research_support_services/research_infrastructure/science_it/#c30646061).
We acknowledge financial support from the Wilhelm Sander Foundation.

## References

  * Hofmanninger et al. [2020] Johannes Hofmanninger, Florian Prayer, Jeanny Pan, Sebastian Röhrich, Helmut Prosch, and Georg Langs.  Automatic lung segmentation in routine imaging is primarily a data diversity problem, not a methodology problem.  _European Radiology Experimental_ , 2020.  doi:10.1186/s41747-020-00173-2.  URL https://doi.org/10.1186/s41747-020-00173-2.
  * Wasserthal et al. [2023] Jakob Wasserthal, Manfred Meyer, Hanns-Christian Breit, Joshy Cyriac, Shan Yang, and Martin Segeroth.  Totalsegmentator: robust segmentation of 104 anatomical structures in CT images.  _Radiology: Artificial Intelligence_ , 2023.  doi:10.1148/ryai.230024.  URL https://doi.org/10.1148/ryai.230024.
  * Geißler et al. [2024] Kai Geißler, Daniel Mensing, Markus Wenzel, Jochen G. Hirsch, and Stefan Heldmann.  Towards TotalSegmentator for MRI data leveraging GIN data augmentation.  In Olivier Colliot and Jhimli Mitra, editors, _Medical Imaging 2024: Image Processing_ , page 1292604. International Society for Optics and Photonics, SPIE, 2024.  doi:10.1117/12.3006351.  URL https://doi.org/10.1117/12.3006351.
  * Philipp et al. [2024] Lena Philipp, Maarten de Rooij, John Hermans, Matthieu Rutten, Horst Karl Hahn, Bram van Ginneken, and Alessa Hering.  Annotation-efficient strategy for segmentation of 3D body composition.  In _Submitted to Medical Imaging with Deep Learning_ , 2024.  URL https://openreview.net/forum?id=OzhonBbVAn.