Medical images may contain various types of artifacts with different patterns and mixtures, which depend on many factors such as scan setting, machine condition, patients’ characteristics, surrounding environment, etc. However, existing deep learning based artifact reduction methods are restricted by their training set with specific predetermined artifact type and pattern. As such, they have limited clinical adoption. In this paper, we introduce a “Zero-Shot” medical image Artifact Reduction (ZSAR) framework, which leverages the power of deep learning but without using general pre-trained networks or any clean image reference. Specifically, we utilize the low internal visual entropy of an image and train a light-weight image-specific artifact reduction network to reduce artifacts in an image at test-time. We use Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) as vehicles to show that ZSAR can reduce artifacts better than state-of-the-art both qualitatively and quantitatively, while using shorter execution time. To the best of our knowledge, this is the first deep learning framework that reduces artifacts in medical images without using a priori training set. Deep learning has demonstrated its great power in artifact reduction, a fundamental task in medical image analysis to produce clean images for clinical diagnosis, decision making, and accurate quantitative image analysis. Existing deep learning based frameworks Jiang et al., 2018; Yuan et al., 2019; Yi & Babyn, 2018) use training data sets that contain paired images (same images with and without artifacts) to learn the artifact features. Simulations are often needed to generate the data set for these methods, which may be different from clinical situations and lead to biased learning Veraart et al., 2016) . To address this issue, Kang et al. (2018) used cycle-consistent adversarial denoising network (CCADN) which no longer requires paired data. However, all these methods still suffer from two mainstays: First, they require clean image references, which can be hard to obtain clinically. For example, motion artifacts in Magnetic Resonance Imaging (MRI) are almost always present due to the lengthy acquisition process (Zaitsev et al., 2015) . In such situations, simulation is still the only way to generate the data set. Second, although the trained networks outperform non-learning based algorithms such as Block Matching 3D (BM3D) (Dabov et al., 2007) , they can only be applied to scenarios where the artifacts resemble what are in the training set, lacking the versatility that non-learning based methods can offer. To attain the performance of deep learning based methods and the versatility of non-learning based ones, we introduce a "Zero-Shot" image-specific artifact reduction network (ZSAR), which builds upon deep learning yet does not require any clean image reference or a priori training data. Based on the key observation that most medical images have areas that contain artifacts on a relatively uniform background and that the internal visual entropy of an image is much lower than that among images (Zontak & Irani, 2011) . At test-time, ZSAR extracts an artifact pattern directly and synthesizes paired image patches from input image to iteratively train a light-weight image-specific autoencoder for artifact reduction. Experimental results on clinical CT and MRI data with a variety of artifacts show that it outperforms the state-of-the-art methods both qualitatively and quantitatively, using shorter execution time. To the best of our knowledge, ZSAR is the first deep learning based method that reduces artifacts in medical images without a priori training data. In this paper, we introduced ZSAR, a "Zero-Shot" medical image artifact reduction framework, which exploits the power of deep learning to suppress artifacts in a medical image without using general pre-trained networks. Unlike previous state-of-the-art methods which are restricted by the training data, our method can be adapted for almost any medical images that contain varying or unknown artifacts. Experimental results on cardiac CT and MRI images have shown that our framework can reduce noises and motion artifacts qualitatively and quantitatively better than the state-of-the-art, using shorter execution time.
We introduce a “Zero-Shot” medical image Artifact Reduction framework, which leverages the power of deep learning but without using general pre-trained networks or any clean image reference. 