#  Subs: Subtree Substitution for Compositional Semantic Parsing

Jingfeng Yang†⋆ Le Zhang‡⋆ Diyi Yang†
† Georgia Institute of Technology
‡ Fudan University
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

Although sequence-to-sequence models often achieve good performance in
semantic parsing for i.i.d. data, their performance is still inferior in
compositional generalization. Several data augmentation methods have been
proposed to alleviate this problem. However, prior work only leveraged
superficial grammar or rules for data augmentation, which resulted in limited
improvement. We propose to use subtree substitution for compositional data
augmentation, where we consider subtrees with similar semantic functions as
exchangeable. Our experiments showed that such augmented data led to
significantly better performance on Scan and GeoQuery, and reached new SOTA on
compositional split of GeoQuery. We have publicly released our code at
https://github.com/GT-SALT/SUBS .

##  1 Introduction

††footnotetext: $\star$Equal Contribution. Jingfeng Yang proposed subtree
substitution data augmentation for compositional semantic parsing, implemented
augmentation and LSTM/BART parsers, and ran Scfg/Geca baselines. Le Zhang
induced span trees and ran span-based semantic parsing baseline.

Semantic parsing transforms natural language utterances to formal language.
Because meaning representations or programs are essentially compositional,
semantic parsing is an ideal testbed for compositional generalization.
Although neural seq2seq models could achieve state-of-the-art performance in
semantic parsing for i.i.d. data, they failed at compositional generalization
due to lack of reasoning ability. That is, they do not generalize well to
formal language structures that were not seen at training time. For example, a
model that observes at training time the questions “What is the population of
the largest state?” and “What is the largest city in USA?” may fail to
generalize to questions such as “What is the population of the largest city in
USA?”. This leads to large performance drops on data splits designed to
measure compositional generalization (compositional splits), in contrast to
the generalization abilities of humans.

To improve compositional generalization in semantic parsing (compositional
semantic parsing), prior work focused on incorporating inductive biases
directly to models or data augmentation. From the model perspective, some work
used neural-symbolic models Chen et al. (2020), generated intermediate
discrete structures Herzig and Berant (2020); Zheng and Lapata (2020), or
conducted meta-learning Lake (2019). From the data perspective, Jia and Liang
(2016) proposed to recombine data with simple synchronous context-free grammar
(Scfg), despite not for compositional generalization. Andreas (2019) used some
simple rules for data augmentation, where tokens with the same context were
considered as exchangeable. Such techniques are still limited since they only
leveraged superficial grammars or rules, and failed when there are
linguistically rich phrases or clauses.

Figure 1: Subtree substitution results in an augmented example. Natural
Language: What is the population of the largest city in the smallest state in
the USA ? Formal Language: answer ( population_1 ( largest ( city ( loc_2 (
smallest ( state ( loc_2 ( countryid ( usa ) ) ) ) ) ) ) ) ).

To fill this gap, we propose to augment the training data of semantic parsing
with diverse compositional examples based on induced or annotated (semantic
and syntactic) trees. Specifically, we propose to exchange subtrees where
roots have similar meaning functions. Since we consider all hierarchies in all
trees, deep structures and complex phrases or clauses are considered for data
augmentation, which is key for compositional generalization. For instance, in
Figure 1, if we exchange subtrees with “largest” as meaning function of its
root, composition of “population of the” and “largest city in the smallest
state in the USA” results in a new augmented structure “population of the
largest city in the smallest state in the USA”. Although certain substructure
substitution methods were explored in other NLP tasks Shi et al. (2021),
subtree substitution with fine-grained meaning functions has been under-
explored. Our experiments showed that such augmented data led to significantly
better performance on Scan Lake and Baroni (2018) and GeoQuery, and reached
new SOTA on compositional split of GeoQuery.

##  2 Methods

#### Span trees

Suppose training set is $\\{(x^{i},z^{i})\\}_{i=1}^{N}$, where $x_{i}$ is a
natural language utterance and $z_{i}$ is the corresponding program. An
utterance $x$ can be mapped to a span tree $T$, such that program($T$)= $z$,
where the deterministic function program($\cdot$) maps span trees to programs
Herzig and Berant (2020).

As shown in Figure 1, a span tree $T$ is a tree where each node covers a span
($i$, $j$) with tokens $x_{i:j}$ = ($x_{i},x_{i+1},\cdots,x_{j}$). A span
subtree can be viewed as a mapping from every span ($i$, $j$) to a single
category $c\in C$, where $C$ is a set of domain-specific categories
representing domain constants, which include entities (e.g. countryid#usa in
Figure 1) and predicates (e.g. loc_2 in Figure 1). The final program can be
computed from the span tree deterministically by the function program(·).
Concretely, program($T$) iterates over the nodes in $T$ bottom-up, and
generates a program $z_{i:j}$ for each node covering the span ($i$, $j$). For
a terminal node, $z_{i:j}=c$. For an internal node, $z_{i:j}$ is determined by
composing the programs of its children, $z_{i:s}$ and $z_{s:j}$ where $s$ is
the split point. As in Combinatory Categorical Grammar, composition is simply
function application, where a domain-specific type system is used to determine
which child is the function and which is the argument. Span trees can be
induced by a hard-EM algorithm or semi-automatically annotated. We refer the
reader to Herzig and Berant (2020) to see how to obtain span-trees.

###  2.1 Subtree Substitution (Subs)

As shown in Figure 1, we consider span subtrees with similar semantic
functions as exchangeable. Formally, func(·) maps a subprogram to a semantic
category, and subtrees with the same semantic categories have similar semantic
functions. For two data points ($x^{1},z^{1}$) and ($x^{2},z^{2}$), if
func($z^{1}_{i_{1}:j_{1}}$) = func($z^{2}_{i_{2}:j_{2}}$), we obtain a new
augmented ($x^{\prime},z^{\prime}$):

$x^{\prime}=x^{1}_{:i_{1}}+x^{2}_{i_{2}:j_{2}}+x^{1}_{j_{1}:},z^{\prime}=z^{1}\backslash
z^{1}_{i_{1}:j_{1}}/z^{2}_{i_{2}:j_{2}}$

Definition of func(·) may vary in different dataset. One straightforward way
is to extract the outside predicate in $z_{i:j}$ as its semantic category,
which is used on GeoQuery, such as func(largest ( state ( all ) ) )) =
largest.

###  2.2 Semantic Parsing

After getting augmented data by subtree substitution, we then combine
augmented data and the original training data to train a seq2seq semantic
parser, where we choose LSTM models with attention Luong et al. (2015) and
copying mechanism Gu et al. (2016), or pretrained $\text{BART}_{\text{Large}}$
Lewis et al. (2020) as the seq2seq model architecture.

| Right | AroundRight
---|---|---
LSTM | 0.00 | 1.00 (2800 updates)
LSTM + Subs | 1.00 | 1.00 (800 updates)
Table 1:  Accuracy of diagnostic experiments on Scan. | Question | Query
---|---|---
Herzig and Berant (2020) | 0.78 | 0.75
LSTM | 0.75 | 0.58
\+ Scfg (Jia et al., 2016) | 0.80 | 0.68
\+ Geca (Andreas, 2019) | 0.77 | 0.60
\+ Subs (ours, induced tree) | 0.79 | 0.72
\+ Subs (ours, gold tree) | 0.81 | 0.79
BART | 0.91 | 0.85
\+ Subs (ours, induced tree) | 0.91 | 0.85
\+ Subs (ours, gold tree) | 0.93 | 0.88
Table 2:  Exact-match accuracy on i.i.d. (Question) and compositional (Query)
splits of GeoQuery dataset.

##  3 Experiments and Results

Dataset We first use Scan Lake and Baroni (2018) as a diagnostic dataset to
test the performance of subtree substitution in compositional semantic
parsing. Scan is a synthetic dataset, which consists of simple English
commands paired with sequences of discrete actions. We use the program version
of Herzig and Berant (2020). For instance, “run right after jump” corresponds
to the program “i_after ( i_run ( i_right ) , i_jump )”. Also, semi-
automatically annotated span trees from Herzig and Berant (2020) are used for
subtree substitution. To test compositional semantic parsing, we use the
Primitive right (Right) and Primitive around right (AroundRight) compositional
splits from Loula et al. (2018), where templates of the form Primitive right
and Primitive around right (respectively) appear only in the test set. In
these templates, Primitive stands for jump, walk, run, or look. For
simplicity, func(·) is defined only on i_right and i_left, where func(i_right)
= func(i_left) = direction. That is, all “i_right” and “i_left” appear as leaf
nodes in span trees and they are exchangeable.

We use GeoQuery dataset to test the performance of subtree substitution in
both i.i.d. and compositional generalization for semantic parsing. GeoQuery
contains 880 questions about US geography Zelle and Mooney (1996). Following
Herzig and Berant (2020), we use the variable-free FunQL formalism from Kate
et al. (2005). The i.i.d. split (Question), which is randomly sampled from the
whole dataset, contains 513/57/256 instances for train/dev/test set. The
compositional split (Query) contains 519/54/253 examples for train/dev/test
set, where templates created by anonymizing entities are used to split the
dataset, to make sure that all examples sharing a template are assigned to the
same set Finegan-Dollak et al. (2018). As for span trees, we use semi-
automatically annotated span trees (gold tree) released by Herzig and Berant
(2020). Alternatively, we use the span trees induced by Herzig and Berant
(2020)’s span-based semantic parsing, without any human labour.

| training instances | augmented instances | avg att l | max att l | avg prg l | max prg l
---|---|---|---|---|---|---
Geca | 519 | 804 | 8.85 | 18 | 15.96 | 29
Subs | 519 | 29039 | 10.43 | 26 | 19.33 | 43
| avg seg l | max seg l | avg att seg l | max att seg l | avg prg seg l | max prg seg l
Geca | 1.93 | 4 | - | - | - | -
Subs | 5.99 | 25 | 3.98 | 13 | 8.01 | 25
Table 3: Complexity of augmented examples on the Query split of GeoQuery dataset, which is measured by maximal (max) and average (avg) lengths (l) of exchanged segments (seg) and resulted utterances(att)/programs(prg).  | 50 | 100 | 200 | 519
---|---|---|---|---
BART | 0.64 | 0.72 | 0.79 | 0.85
BART + Subs | 0.67 | 0.79 | 0.85 | 0.88
Table 4:  Effect of numbers of training examples on compositional split of
GeoQuery.

###  3.1 Diagnostic Results

Results of diagnostic experiments on Scan dataset are shown in Table 1, where
we use LSTM parser without data augmentation as the baseline. We can see that
on the Right split, LSTM seq2seq semantic parser could only achieve zero
exact-match accuracy without any data augmentation techniques, which means
that the model’s compositional generalizibility on the Right split is very
poor. After adding our augmented data with subtree substitution, we achieve an
exact-match accuracy of 100%. Actually, we got 6660 augmented examples besides
the original 12180 training examples. Among all augmented examples, 3351
examples are in the test set, which means 74.87% of 4476 test examples are
recovered by subtree substitution. On the AroundRight split, using LSTM
seq2seq semantic parser could already achieve 100% exact-match accuracy, which
means that the model learned from Primitive right and Primitive opposite right
generalize to Primitive around right well in our program format “i_primitive (
i_around ( i_right ) )”. After adding our augmented examples, the parser
converged to 100% exact-match accuracy faster, where our method requires
around 800 updates to converge while baseline model requires 2800 updates with
the same batch size 64.

###  3.2 Main Results

Table 2 shows the results of experiments on GeoQuery dataset, where we
examined both seq2seq LSTM and $\text{BART}_{\text{Large}}$ parsers. LSTM and
BART parsers without any data augmentation are simplest baselines. We also
compare with other two data augmentation methods as additional baselines,
recombining data with simple Scfg Jia and Liang (2016) or using simple rules
for Good Enough Data Augmentation (Geca) Andreas (2019), which were proven
useful for compositional semantic parsing. We can see that on the Question
split, adding augmented data from (gold) subtree substitution leads to
improvements for both LSTM and BART seq2seq models, suggesting that subtree
substitution as data augmentation helps i.i.d generalization for semantic
parsing. On the Query split, (gold) subtree substitution achieves more
substantial improvements over seq2seq baseline models (absolute 21% and 3%
improvements of the exact-match accuracy for LSTM and BART respectively),
achieving state-of-the-art results. Moreover, our methods are also better than
the two data augmentation baselines. Therefore, subtree substitution is a
simple yet effective compositional data augmentation method for compositional
semantic parsing. With (induced) subtree substitution, Subs still achieves
improvements for LSTM models.

Analysis of Augmented Data We further examine why subtree substitution could
achieve much better performance by analyzing its augmented data. As shown in
Table 3, Geca only identifies and exchanges very simple structures, where the
average and maximal length of exchanged segments are 1.93 and 4. A closer look
at these augmented data shows that nearly all of these segments are simple
entities (e.g. State: “Illinois”, “Arizona” etc.) or other Nouns (e.g. “area”,
“population” etc.). In contrast, subtree substitution can identify and
exchange much more complex structures, where the average and maximal length of
exchanged segments are 5.99 and 25. For example, largest city in the smallest
state in the USA and largest state are identified as exchangeable. As a
result, subtree substitution could produce more complex utterance and program
pairs, where the average and maximal length of these resulted utterances are
10.43 and 26, compared with the average (8.53) and maximal (18) length of
utterances returned by Geca. Moreover, subtree substitution could generate
much more augmented instances, because it can identify more complex structures
besides those simple ones identified by Geca. Compared with Scfg, Subs could
also identify complex structures automatically with subtrees, while Scfg only
handle simple phrases defined by rules.

Effect of Training Data Size Table 4 shows that with more training examples,
models’ performances improve. In all settings, using (gold) subtree
substitution boosts the performance of BART. When there are 100 and 200
training examples, the improvement is more significant, demonstrating the
effectiveness of Subs in the few-shot setting.

##  4 Related Work

Several data augmentation methods have been introduced for (compositional)
semantic parsing. Jia and Liang (2016) recombined data by Scfg, and Andreas
(2019) used some simple rules to exchange tokens with the same context.
However, they leveraged only superficial grammars or rules, which has limited
capacity to identify complex structures. Akyürek et al. (2020) learned to
recombine and resample data with a prototype-based generative model, instead
of using rules. Certain substructure substitution methods have been explored
for data augmentation in other NLP tasks Shi et al. (2021). Dependency tree
cropping and rotation within sentence was used in low-resource language POS
tagging Şahin and Steedman (2019) and dependency parsing Vania et al. (2019).
Dependency tree swapping was explored in low-resource language dependency
parsing Dehouck and Gómez-Rodríguez (2020), and Universal Dependency features
was used for zero-shot cross-lingual semantic parsing Yang et al. (2021).
However, subtree substitution with fine-grained meaning functions has not been
examined. Some rule-based data augmentation methods were also explored in
table semantic parsing Eisenschlos et al. (2020); Yang et al. (2022). To the
best of our knowledge, we are the first to explore tree manipulation for
semantic parsing.

##  5 Conclusion

This work proposed to use subtree substitution to compositionally augment the
data of semantic parsing to help the compositional generalization. Our method
achieved significant improvements over seq2seq models, other data augmentation
methods and span-based semantic parsing.

## Acknowledgements

The authors would like to thank reviewers for their helpful insights and
feedback. This work is funded in part by a grant from Amazon and Salesforce.

## References

  * Akyürek et al. (2020) Ekin Akyürek, Afra Feyza Akyürek, and Jacob Andreas. 2020.  Learning to recombine and resample data for compositional generalization.  _arXiv preprint arXiv:2010.03706_.
  * Andreas (2019) Jacob Andreas. 2019.  Good-enough compositional data augmentation.  _arXiv preprint arXiv:1904.09545_.
  * Chen et al. (2020) Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, and Denny Zhou. 2020.  Compositional generalization via neural-symbolic stack machines.  _arXiv preprint arXiv:2008.06662_.
  * Dehouck and Gómez-Rodríguez (2020) Mathieu Dehouck and Carlos Gómez-Rodríguez. 2020.  Data augmentation via subtree swapping for dependency parsing of low-resource languages.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 3818–3830.
  * Eisenschlos et al. (2020) Julian Martin Eisenschlos, Syrine Krichene, and Thomas Müller. 2020.  Understanding tables with intermediate pre-training.  _arXiv preprint arXiv:2010.00571_.
  * Finegan-Dollak et al. (2018) Catherine Finegan-Dollak, Jonathan K Kummerfeld, Li Zhang, Karthik Ramanathan, Sesh Sadasivam, Rui Zhang, and Dragomir Radev. 2018.  Improving text-to-sql evaluation methodology.  _arXiv preprint arXiv:1806.09029_.
  * Gu et al. (2016) Jiatao Gu, Zhengdong Lu, Hang Li, and Victor OK Li. 2016.  Incorporating copying mechanism in sequence-to-sequence learning.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics_ , pages 1631–1640.
  * Herzig and Berant (2020) Jonathan Herzig and Jonathan Berant. 2020.  Span-based semantic parsing for compositional generalization.  _arXiv preprint arXiv:2009.06040_.
  * Jia and Liang (2016) Robin Jia and Percy Liang. 2016.  Data recombination for neural semantic parsing.  _arXiv preprint arXiv:1606.03622_.
  * Kate et al. (2005) Rohit J Kate, Yuk Wah Wong, and Raymond J Mooney. 2005.  Learning to transform natural to formal languages.  In _AAAI_ , volume 5, pages 1062–1068.
  * Klein et al. (2017) Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M Rush. 2017\.  Opennmt: Open-source toolkit for neural machine translation.  _arXiv preprint arXiv:1701.02810_.
  * Lake and Baroni (2018) Brenden Lake and Marco Baroni. 2018.  Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks.  In _International Conference on Machine Learning_ , pages 2873–2882. PMLR.
  * Lake (2019) Brenden M Lake. 2019.  Compositional generalization through meta sequence-to-sequence learning.  _arXiv preprint arXiv:1906.05381_.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880.
  * Loula et al. (2018) Joao Loula, Marco Baroni, and Brenden M Lake. 2018.  Rearranging the familiar: Testing compositional generalization in recurrent networks.  _arXiv preprint arXiv:1807.07545_.
  * Luong et al. (2015) Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015.  Effective approaches to attention-based neural machine translation.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 1412–1421.
  * Ott et al. (2019) Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019.  fairseq: A fast, extensible toolkit for sequence modeling.  _arXiv preprint arXiv:1904.01038_.
  * Şahin and Steedman (2019) Gözde Gül Şahin and Mark Steedman. 2019.  Data augmentation via dependency tree morphing for low-resource languages.  _arXiv preprint arXiv:1903.09460_.
  * Shi et al. (2021) Haoyue Shi, Karen Livescu, and Kevin Gimpel. 2021.  Substructure substitution: Structured data augmentation for nlp.  _arXiv preprint arXiv:2101.00411_.
  * Vania et al. (2019) Clara Vania, Yova Kementchedjhieva, Anders Søgaard, and Adam Lopez. 2019.  A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages.  _arXiv preprint arXiv:1909.02857_.
  * Yang et al. (2021) Jingfeng Yang, Federico Fancellu, Bonnie Webber, and Diyi Yang. 2021.  Frustratingly simple but surprisingly strong: Using language-independent features for zero-shot cross-lingual semantic parsing.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 5848–5856.
  * Yang et al. (2022) Jingfeng Yang, Aditya Gupta, Shyam Upadhyay, Luheng He, Rahul Goel, and Shachi Paul. 2022.  Tableformer: Robust transformer modeling for table-text encoding.  _arXiv preprint arXiv:2203.00274_.
  * Zelle and Mooney (1996) John M Zelle and Raymond J Mooney. 1996.  Learning to parse database queries using inductive logic programming.  In _Proceedings of the national conference on artificial intelligence_ , pages 1050–1055.
  * Zheng and Lapata (2020) Hao Zheng and Mirella Lapata. 2020.  Compositional generalization via semantic tagging.  _arXiv preprint arXiv:2010.11818_.

##  Appendix A Training Details

We adapted OpenNMT Klein et al. (2017) for LSTM model with attention and
copying mechanism, while used fairseq Ott et al. (2019) to implement BART
model.

We manually tune the hyper-parameters. For LSTM models, we use one-layer
bidirectional LSTM in the encoder side and one-layer unidirectional LSTM in
the decoder side. We use dropout with 0.5 as dropout rate and Adam optimizer
with a learning rate of 0.001. We use MLP attention and reuse attention scores
as copying scores. On GeoQuery, the batch size is set to 1 sentence without
augmented data and set to 64 sentences with augmented data. On Scan, all batch
sizes are 64 sentences. For BART models, we use BART large models. We use Adam
as optimizer with a learning rate 1e-5. We use dropout and attention dropout
with 0.1 as dropout rate. Also, we use label smoothing with a rate 0.1. Batch
sizes are 1024 tokens. Besides, we employ a weight-decay rate 0.01. All the
parameters are manually tuned based on the dev performance.

We train all models on NVIDIA A100 SXM4 40 GB GPU. We set the max training
epoch to be 100 and select the best performed epoch according to dev
performance. Training process on each clause or whole sequence could be
finished within 3 hours.

For baselines with other data augmentation methods, we reran Geca and Scfg on
this FunQL formalism of GeoQuery and these splits with annotated span trees.
That’s why our results are a little different from the reported results in the
original paper. We got similar results with their source code and our code on
our data, in order to make sure that there is no problem with our results and
code.

We got the same denotation accuracy as reported by Herzig and Berant (2020),
but we reported exact-match accuracy on Table 2 for fair comparison.


# Robust Cross-domain CT Image Reconstruction via Bayesian Noise Uncertainty
Alignment

Kecheng Chen, Haoliang Li, Renjie Wan, Hong Yan This work is supported by the
Hong Kong Innovation and Technology Commission (InnoHK Project CIMDA).K. Chen,
H. Li, and H. Yan are with the Department of Electrical Engineering and the
Center for Intelligent Multidimensional Data Analysis, City University of Hong
Kong, Hong Kong, China. (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>h.yan@cityu.edu.hk)R. Wan is with the Department of Computer Science, Hong
Kong Baptist University, Hong Kong, China. (e-mail<EMAIL_ADDRESS>
###### Abstract

In this work, we tackle the problem of robust computed tomography (CT)
reconstruction issue under a cross-domain scenario, i.e., the training CT data
as the source domain and the testing CT data as the target domain are
collected from different anatomical regions. Due to the mismatches of the scan
region and corresponding scan protocols, there is usually a difference of
noise distributions between source and target domains (a.k.a. noise
distribution shifts), resulting in a catastrophic deterioration of the
reconstruction performance on target domain. To render a robust cross-domain
CT reconstruction performance, instead of using deterministic models (e.g.,
convolutional neural network), a Bayesian-endowed probabilistic framework is
introduced into robust cross-domain CT reconstruction task due to its
impressive robustness. Under this probabilistic framework, we propose to
alleviate the noise distribution shifts between source and target domains via
implicit noise modeling schemes in the latent space and image space,
respectively. Specifically, a novel Bayesian noise uncertainty alignment
(BNUA) method is proposed to conduct implicit noise distribution modeling and
alignment in the latent space. Moreover, an adversarial learning manner is
imposed to reduce the discrepancy of noise distribution between two domains in
the image space via a novel residual distribution alignment (RDA). Extensive
experiments on the head and abdomen scans show that our proposed method can
achieve a better performance of robust cross-domain CT reconstruction than
existing approaches in terms of both quantitative and qualitative results.

###### Index Terms:

Robust CT reconstruction, probabilistic model, noise modeling, adversarial
learning

##  I Introduction

Low-dose computed tomography (LDCT) image reconstruction is a very popular
research orientation recently. Usually, researchers aim to reconstruct
potential normal-dose CT (NDCT) images from LDCT images based on the image
domain [1], projection domain [2], or dual domains [3]. This is highly
beneficial to the patients and radiologists. On the one hand, compared with
NDCT imaging mode, the patients, especially for the children, will suffer from
lower radiation risk under the LDCT imaging mode [4]. On the other hand, by
the virtue of LDCT image reconstruction technique, it is sufficient for the
radiologists to leverage reconstructed LDCT images with reduced noise to make
clinical decisions. Among various approaches, deep learning (DL)-based LDCT
reconstruction methods are dominant recently [5], due to their impressive
capacities of noise suppression and structural information retaining. By
utilizing LDCT/NDCT images pairs in a specific anatomical region (e.g., the
abdomen or the head), researchers devote huge efforts to elaborate powerful
reconstructed models with different perspectives, such as edge preservation
[6] and texture transfer [7]. Moreover, unpaired CT reconstruction approaches,
as more challenging settings, are also developed to tackle the problem of
content mismatch between LDCT and NDCT images. To this end, a serial of
CycleGAN-based methods [8, 9] are proposed by the means of self-supervised and
adversarial schemes. More works about DL-based CT reconstruction can be found
in this review [5].

Figure 1: Examples of head and abdomen scans. (a) Abdomen LDCT scans. (b) Head
LDCT scans. (c) Probability density functions of noise distributions (the
subtraction of the LDCT image from the NDCT image) in selected ROIs from (a)
and (b), (d) Illustration of dynamic changes of CT scan protocol (i.e., mAs)
in two different patients with abdomen scans.

Usually, most existing DL-based CT reconstruction methods implicitly assume
that the training CT data (a.k.a. source domain $D_{S}$) and testing CT data
(a.k.a. source domain $D_{T}$) have a similar noise distribution [6, 7, 8, 9].
However, the noise distribution between $D_{S}$ and $D_{T}$ may be different
due to the mismatch of CT scan protocol (e.g., kVp and mAs of the CT scan
device) and different anatomical regions (e.g., $D_{S}$ and $D_{T}$ from head
and abdomen, respectively). This noise distribution shift between $D_{S}$ and
$D_{T}$ will result in a deterioration of the reconstruction performance,
which is recently referred to as robust CT reconstruction problem [10, 11,
12]. Specifically, Lee et al. [10] proposed to model the noise distribution of
target domain via a simple Poisson noise inserting model, such that the
pretrained convolutional neural networks (CNNs)-based model on $D_{S}$ can be
transferred to target domain. By leveraging unpaired sinogram data on $D_{T}$,
Li et al. [11] explicitly model the noise distribution using Gaussian mixture
model (GMM) on the target domain to fine-tune the pretrained CNNs-based model
on $D_{S}$ towards target domain.

However, one limitation of abovementioned approaches is that direct explicit
noise modeling in the target domain using model-based schemes usually may not
have sufficient capacity to represent complex and realistic noise, as
discussed by [13, 14], due to the restriction of human knowledge of noise
priors (e.g., the assumption of Poisson or Gaussian distribution). This
further results in a negative impact of noise transfer from source domain to
target domain. Moreover, we can observe that the dynamic changes (as the
perturbation) of CT scan protocol (i.e., mAs) in different slices of a patient
(see the Figure 1(d)) may cause nonuniform noise distributions in the intra-
domain CT images (see the Figure 1(c)), as the tube current modulation is
usually on to avoid excessive radiation doses [15]. For this perturbation,
existing deterministic neural network (e.g., CNNs)-based CT reconstruction
models show insufficient robustness due to their fixed model weights [16, 17,
18].

To address these problems, we propose to alleviate the noise distribution
shifts between source and target domains via implicit noise modeling schemes
under a probabilistic framework. Note that we focus more on the robust cross-
domain (i.e., $D_{S}$ and $D_{T}$ are collected from different anatomical
regions) CT reconstruction problem in this paper, as the difference of
anatomical region together with scan protocol-related perturbation will lead
to more representative and challenging noise distribution shifts (see the
Figure 1(c)). Specifically, we introduce a Bayesian-endowed probabilistic
framework into the robust cross-domain CT reconstruction problem as
illustrated in Figure 2. By doing so, the perturbations of cross-domain
samples can be implicitly considered by the distribution of the weights for
better robustness. By the virtue of this probabilistic framework, we propose
to alleviate the noise distribution shifts between source and target domains
in the latent space and image space, respectively. First, we propose to
leverage the Bayesian uncertainty of the latent feature endowed by the
probabilistic framework to implicitly model the noise distributions of two
domains in latent space such that the discrepancy of noise distributions can
be reduced. Second, by utilizing the residual information (as a proxy of the
noise) between LDCT images and estimated NDCT images in two domains, an
adversarial learning manner is imposed to alleviate the discrepancy of noise
distribution between source and target domains in the image space.

The contributions of this work can be summarized as following:

  * •

A Bayesian-endowed probabilistic model is introduced into the robust cross-
domain CT reconstruction problem for a better robustness.

  * •

A novel Bayesian noise uncertainty alignment (BNUA) method is proposed to
conduct implicit noise distribution modeling and alignment in the latent
space.

  * •

The discrepancy of noise distribution shifts between $D_{S}$ and $D_{T}$ is
reduced by a novel adversarial residual distribution alignment (RDA) scheme in
the image space.

##  II Methodology

###  II-A Problem Statement

Here, the source domain with LDCT/NDCT image pairs can be represented as
$D_{S}=\\{(\mathbf{x}_{1}^{S},\mathbf{y}_{1}^{S}),\cdots,(\mathbf{x}_{N}^{S},\mathbf{y}_{N}^{S}))\\}$,
where $N$ denotes that the number of pairs. The target domain has LDCT images
only, which can be represented as
$D_{T}=\\{\mathbf{x}_{1}^{T},\cdots,\mathbf{x}_{L}^{T}\\}$, where $L$ denotes
the number of LDCT images. Suppose that $D_{S}$ and $D_{T}$ are available and
collected from different anatomical regions with diverse kVp and mAs (as scan
protocol-related perturbations), e.g., $D_{S}$ ($D_{T}$) is either the head
(abdomen) or the abdomen (head) are different between $D_{S}$ and $D_{T}$.
There exists noise distribution shifts between $D_{S}$ and $D_{T}$ as shown in
Figure 1. By using paired LDCT/NDCT images on $D_{S}$ and LDCT images on
$D_{T}$, the objective of robust cross-domain CT reconstruction is to learn a
robust model $F$, which can generalize well on the target domain $D_{T}$ such
that the high-quality reconstructed NDCT image
$\mathbf{\hat{y}}^{T}=F(\mathbf{x^{T}})$ on the target domain can be obtained.

Figure 2: The overall framework of the proposed method. The CT reconstruction
model is disentangled into a BNN-based encoder for the feature extraction and
a BNN-based decoder for the content reconstruction from extracted features. By
conducting MC sampling, the Bayesian noise uncertainty alignment can be
imposed for the alleviation of noise distribution in the latent space. The
residual distribution alignment is to align noise distribution of two domains
in the image space using the residual information.

###  II-B Robust CT reconstruction under a probabilistic framework

While widely-used deterministic models (e.g., CNNs [19] and self-attention
module [20]) have achieved promising performance on the CT reconstruction
problem, they show insufficient robustness to input samples that suffer from
diverse perturbations (e.g., the dynamic changes of kVs and mAs) even for
intra-domain CT construction problem (i.e., both of source and target domains
are from the same anatomical region), as discussed by [17, 16]. The underlying
reason is that these deterministic models can not incorporate these
perturbations as the uncertainty into the prediction directly owing to their
fixed model weights. As such, they may be over-confident and produce
instability predictions for the perturbation. To address this issue, recent
work [17] utilized probabilistic models to render more robust performance for
intra-domain CT construction problem. It is no doubt that these scan protocol-
related perturbations are more prominent on cross-domain CT reconstruction
scenario, as these device parameters (e.g., kVs and mAs) will be modulated
dynamically for different anatomical regions [21].

In this paper, we propose to introduce a probabilistic framework into the
robust cross-domain CT reconstruction problem. By doing so, the perturbations
of cross-domain samples can be implicitly considered by the distribution of
the weights as the uncertainty. Specifically, compared with deterministic
models, probabilistic models turn to learn a distribution over model weights
$\mathbf{w}$ as below,

$\displaystyle\mathbf{w}$ $\displaystyle=$
$\displaystyle\arg\max_{\mathbf{w}}\log P(\mathbf{w}|D)$ (1) $\displaystyle=$
$\displaystyle\arg\max_{\mathbf{w}}[\log P(D|\mathbf{w})+\log P(\mathbf{w})],$

where $D$ denotes the training data. The first term is the complexity term,
and the second term is a prior distribution of the weight that can play a role
of implicit regularization for better robustness.

Here, Bayesian neural network (BNN) with variational inference [22], a
representative probabilistic model, is introduced to approximate the posterior
distribution $P(\mathbf{w}|D)$ of the weight, i.e.,

$\theta^{*}=\arg\min_{\theta}\operatorname{KL}[q(\mathbf{w}|\theta)||p(\mathbf{w}|D)].$
(2)

The rationale of variational inference in Eq. 2 is to find the parameters
$\theta$ of a distribution over the weight in order to minimize the Kullback-
Leibler (KL) divergence with its true posterior distribution over the weight.
Usually, Eq. 2 can be reformulated as

$\displaystyle\theta^{*}$ $\displaystyle=$
$\displaystyle\arg\min_{\theta}\operatorname{KL[q(\mathbf{w}|\theta)||p(\mathbf{w})]}$
(3) $\displaystyle-$ $\displaystyle\mathbb{E}_{\mathbf{w}\sim
p(\mathbf{w}|\theta)}[\log P(D|\mathbf{w})],$

where the first term can balance the model complexity with its prior
distribution. Moreover, the second term acts an analogous way to the first
term in Eq. (1), which aims to fit the training data well as the same as
deterministic models (as a reconstrution loss). This objective is so-called
evidence lower bound (ELBO) loss. By incorporating the re-parameter trick [23]
and stochastic gradient descent method, the variational parameter $\theta$ can
be solved flexibly.

Different from most existing encoder-decoder-based CT reconstruction models
[24], we propose to disentangle a CT reconstruction model into two decoupled
probabilistic components (as shown in Figure 2), including a BNN-based encoder
(for the feature extraction) and a BNN-based decoder (for the content
reconstruction from extracted features), respectively, i.e., $F=E\circ G$. In
the next sections, we will describe how to address the noise distribution
shifts between $D_{S}$ and $D_{T}$ under this probabilistic framework.

###  II-C Bayesian Noise Uncertainty Modeling

For existing cross-domain classification problems, the differences of the
image style, texture, and illumination between source and target domains will
result in the domain shift problem[25], leading to unfavorable testing
performance on target domain. To address this problem, a widely-adopted scheme
is to reduce the discrepancy of latent embeddings of two domains by
calculating the distance between latent embeddings [26], which can derive
robust classification results [27].

Motivated by this strategy, an intuitive idea for robust CT reconstruction is
to reduce the noise distribution discrepancy between source and target domains
in the latent space, as the domain shift of the robust cross-domain CT
reconstruction mainly results from the difference of noise distribution.
However, the representation of the noise distribution of LDCT images in the
latent space is more difficult [28] as the noise appeared in LDCT images is
coupled with its content information.

To tackle this issue, we propose to leverage the Bayesian uncertainty to
characterize the noise distribution in the latent space, which can in turn to
implicitly reflect the noise distribution in the image space. Before
introducing our proposed method, the concepts of Bayesian uncertainty and
Bayesian uncertainty-induced noise encoding [29] derived in the label space
need to be reviewed. Specifically, by calculating the variance of different
predictions (obtained by multiple Monte Carlo samples) of a same input,
Bayesian uncertainty (i.e., the computed variance) can encode the inherent
noise on the training data $D$ through the learned distribution of the weight
[29]. More importantly, these testing data, which are not consistent with the
inherent noise distribution on training data $D$ during inference, will lead
to a high uncertainty [29].

Thus, it is reasonable to introduce Bayesian uncertainty from label space into
the latent space for latent features (extracted by the BNN-based encoder $E$)
to characterize the noise distribution in the latent space. The rationality of
this scheme also can be observed from its probabilistic property, i.e., the
posterior distribution $P(\mathbf{w}_{E})$ of the weight over the
parameterized encoder $E$ is a conditional distribution over
$P(\mathbf{w}_{E}|D)$ given by training data $D$ as Eq. 1. Naturally, Bayesian
uncertainty of latent features, i.e., the probabilistic embeddings,
$P(\mathbf{z}|\mathbf{x},\mathbf{w}_{E})$, can implicitly encode the noise on
$D$.  Our empirical analysis in the experiment section 7 also shows the
rationality of the Bayesian uncertainty.

Furthermore, we can derive that the encoder $E$ trained on the source domain
$D_{S}$ will inherently encode the noise on LDCT images $\mathbf{x}^{S}$ of
the source domain $D_{S}$, due to its conditional distribution property. When
there exists noise distribution shifts between $D_{S}$ and $D_{T}$, the
encoder $E$ trained on source domain $D_{S}$ will not be generalizable enough
to extract the latent features for the LDCT images $\mathbf{x}^{T}$ on target
domain $D_{T}$. As a result, it is indeed necessary to reduce the noise
distribution discrepancy between source and target domains by minimizing
Bayesian uncertainty of the two domains. By doing so, a domain-invariant
encoder can be obtained, which means that the latent features $\mathbf{z}$
extracted by this domain-invariant encoder can be fed into the decoder $G$
safely and unbiasedly, leading to better robust cross-domain CT reconstruction
results.

Here, how to reduce the noise distribution discrepancy in the latent space via
Bayesian uncertainty will be described in detail. Specifically, by feeding two
batches of LDCT samples from source and target domains, i.e.,
$\\{\mathbf{x}_{i}^{S}\\}_{i=1}^{B}$ and $\\{\mathbf{x}_{i}^{T}\\}_{i=1}^{B}$
(where $\mathbf{x}_{i}^{S(T)}\in\mathbb{R}^{W\times H\times 1}$ denotes a LDCT
sample with spatial size of $W\times H$ and the channel number of 1. $B$
denotes the number of images in a mini-batch), into the probabilistic encoder
$E$, the output is a probabilistic embedding for each sample, i.e.,
$P(\mathbf{z}_{i}^{S(T)}|\mathbf{x}_{i}^{S(T)})=P(\mathbf{z}_{i}^{S(T)}|\mathbf{x}_{i}^{S(T)},\mathbf{w}_{E})$,
where $\mathbf{w}_{E}\sim P(\mathbf{w}_{E})$. The predictive distribution of
$P(\mathbf{z}^{S(T)}|\mathbf{x}^{S(T)})$ can be unbiased approximation using
Monte Carlo (MC) estimators with $M$ stochastic sampling operations over the
$\mathbf{w}_{E}$, i.e.,

$\mathbf{z}_{i}^{S(T)}=\mathbb{E}[P(\mathbf{z}_{i}^{S(T)}|\mathbf{x}_{i}^{S(T)},\mathbf{w}_{E})]=\frac{1}{M}\sum_{j=1}^{M}E_{\mathbf{w}_{E}^{j}}(\mathbf{x}_{i}^{S(T)}),$
(4)

where $E_{\mathbf{w}_{E}}(\cdot)$ denotes the parameterized probabilistic
encoder, and $\mathbf{z}_{i}^{S(T)}\in\mathbb{R}^{W^{{}^{\prime}}\times
H^{{}^{\prime}}\times C}$ with $C$ feature channels. By imposing the virtue of
probabilistic framework in Eq. 4, we can derive different Bayesian sampling
embeddings of a same input $\mathbf{x}_{i}$ to render Bayesian uncertainty
estimation,
i.e.,$\mathbf{Z}_{i}^{S(T)}=\\{\mathbf{z}_{i,j}^{S(T)}\\}_{j=1}^{M}$, where
the subscript $j$ denotes the index of MC sampling and
$\mathbf{Z}_{i}^{S(T)}\in\mathbb{R}^{M\times W^{{}^{\prime}}\times
H^{{}^{\prime}}\times C}$. Here, it is necessary to attain more compact latent
feature representation for Bayesian uncertainty estimation, as the information
on spatial dimensions $H^{{}^{\prime}}\times W^{{}^{\prime}}$ are redundant as
discussed by [30]. Usually, the channel-wise statistics can reflect more
expressive and representative latent features by shrinking the spatial
dimensions of latent embeddings, which is observed by previous lectures [31,
32].

By doing so, we utilize the squeeze module in [30] to conduct global
information embedding via global average pooling. As such, we can derive a
more compact and representative latent features for Bayesian uncertainty
estimation in latent space. Specifically, the global average pooling can be
imposed for each channel element of each sampling embedding, i.e.,
$\mathbf{z^{\prime}}^{S(T)}\in\mathbb{R}^{W^{{}^{\prime}}\times
H^{{}^{\prime}}}$ as following

$u^{S(T)}=F_{sq}(\mathbf{z^{\prime}}^{S(T)})=\frac{1}{W^{{}^{\prime}}\times
H^{{}^{\prime}}}\sum_{t=1}^{W^{{}^{\prime}}}\sum_{q=1}^{H^{{}^{\prime}}}z_{t,q}^{S(T)},$
(5)

where $F_{sq}(\cdot)$ denotes the global average pooling. Finally, the compact
latent feature representation can be calculated as
$\mathbf{U}_{i}^{S(T)}=F_{sq}(\mathbf{Z}_{i}^{S(T)})$, where
$\mathbf{U}_{i}^{S(T)}\in\mathbb{R}^{M\times C}$.

More importantly, we aim to minimize the Bayesian uncertainty discrepancy of
obtained latent features $\mathbf{U}_{i}^{S(T)}$ between source and target
domains such that the noise distribution between the two domains can be
reduced implicitly. To this end, we propose to leverage the covariance matrix
of latent features, as second-order statistics of latent features, to quantify
the uncertainty among features. Specifically, we propose to calculate the
covariance matrix over different Bayesian sampling embeddings of a same input,
i.e.,$\mathbf{U}_{i}^{S(T)}$ as following

$\displaystyle\mathbf{C}_{i}^{S}$ $\displaystyle=$
$\displaystyle\frac{1}{M-1}\sum_{j=1}^{M}(\mathbf{u}_{i,j}^{S}-\bm{\mu}_{i}^{S})^{t}(\mathbf{u}_{i,j}^{S}-\bm{\mu}_{i}^{S}),$
$\displaystyle\mathbf{C}_{i}^{T}$ $\displaystyle=$
$\displaystyle\frac{1}{M-1}\sum_{j=1}^{M}(\mathbf{u}_{i,j}^{S}-\bm{\mu}_{i}^{T})^{t}(\mathbf{u}_{i,j}^{T}-\bm{\mu}_{i}^{T}),$
(6)

where $\bm{\mu}_{i}^{S}$ and $\bm{\mu}_{i}^{T}$ denote the mean of the
$\mathbf{U}_{i}^{S}$ and $\mathbf{U}_{i}^{T}$, respectively.
$\mathbf{u}_{i,j}^{S}$ and $\mathbf{u}_{i,j}^{T}$ represent the $j$-th row of
the $\mathbf{U}_{i}^{S}$ and $\mathbf{U}_{i}^{T}$. The size of
$\mathbf{C}_{i}^{S}$ and $\mathbf{C}_{i}^{T}$ is $C\times C$.

Note that the calculation in Eq. II-C is different from existing covariance
matrix alignment of latent features (e.g., CORAL[33]) in cross-domain
classification problems, as the covariance matrix in Eq. II-C is derived from
the different Bayesian sampling embeddings of a same sample rather than
different samples in the source (or target) domain, which means that our
proposed method can ignore the interference of content information (as the
content information are same for different sampling embeddings of a same
input), and then focus more on the uncertainty estimate of latent features.

Finally, the Bayesian uncertainty discrepancy as a $\mathcal{L}_{BNUA}$
objective in a batch of samples between source and target domains can be
minimized as below,

$\mathcal{L}_{BNUA}=\frac{1}{B\times
B}\sum_{i=1}^{B}\sum_{j=1}^{B}\|\mathbf{C}_{i}^{S}-\mathbf{C}_{j}^{T}\|_{F}^{2},$
(7)

where $\|\cdot\|_{F}$ denotes the Frobenius norm. In this paper, the objective
$\mathcal{L}_{BNUA}$ is called as Bayesian Noise Uncertainty Alignment (BNUA)
loss due to its Bayesian property and implicit noise modeling via uncertainty
estimation. Our proposed Bayesian noise uncertainty alignment has several
advantages as following. First, it is an unsupervised manner as the
computation in Eq. 7 does not refer to any NDCT images on the target domain
$D_{T}$. Second, it is a parameter-free manner, as there are no additional
parameters in the BNUA module need to be learned. Instead, existing noise
modeling methods (e.g., GMM) need to solve extra parameters (e.g., mixing
coefficients via EM algorithm). Third, compared with existing explicit noise
distribution representation methods on image domain, the proposed implicit
noise modeling in latent space via Bayesian uncertainty may be more powerful.

###  II-D Residual Distribution Alignment via Adversarial Training

In the previous section, we figure out how to conduct the implicit noise
distribution alignment via minimizing Bayesian uncertainty discrepancy in the
latent space. Then, the encoder may be regularized to render the domain-
invariant feature embeddings between source and target domains. However, it is
also important to reduce the discrepancy of noise distribution between source
and target domains in the image space, as this discrepancy may affect the
quality of reconstructed CT images (i.e., estimated NDCT images) directly.

How to characterize the noise distribution in the image space is an urgent
issue. Here, we can first recall the widely-used LDCT image degraded model,
i.e., $\mathbf{x}=\mathbf{y}+\mathbf{n}$, where $\mathbf{x}$, $\mathbf{y}$ and
$\mathbf{n}$ denote the LDCT image, the NDCT image and the noise in the image
space, respectively. According to the degraded model, we can derive a simple
yet effective noise estimation method through the residual information (as a
proxy of the noise), i.e., $\hat{\mathbf{n}}=\mathbf{x}-\hat{\mathbf{y}}$,
where $\hat{\mathbf{y}}=F(\mathbf{x})$ denotes the estimated NDCT images by
the reconstruction model. By recalling the residual inforamtion, the noises on
source and target domains, i.e., $\hat{\mathbf{n}}^{S}$ and
$\hat{\mathbf{n}}^{T}$, can be approximated though their corresponding
residual information. Note that the supervised information on the source
domain (e.g., $\mathbf{x}^{S}$ and $\mathbf{y}^{S}$) can guarantee the
relative usability of estimated noises on the source and target domains.
Moreover, quite a few cross-domain classification methods also leverage the
predictions on the target domain (during training) as a complementary
information (a.k.a. pseudo label) to guide a promising cross-domain
performance [34, 35]. Similarly, the estimated noise on target domain
$\hat{\mathbf{n}}^{T}$ benefits from estimated (pseudo) NDCT images
$\hat{\mathbf{y}^{T}}$ on target domain during training, i.e.,
$\hat{\mathbf{n}}^{T}=\mathbf{x}^{T}-\hat{\mathbf{y}}^{T}$. One can observe
that this scheme is also decoupled with the content information existed on the
LDCT image as much as possible, as the residual information is derived from a
same LDCT image $\mathbf{x}$.

According to obtained estimated noises on source and target domains, reducing
noise distribution discrepancy between the two domains may be feasible in the
image space. It should be noted that the distributions of NDCT images on the
source and target domain may be different depending on their specific
anatomical regions, which means $P(\mathbf{y}^{T}|\mathbf{x}^{T})\neq
P(\mathbf{y}^{S}|\mathbf{x}^{S})$. Instead, forcing the noise distribution of
source domain $P(\mathbf{n}^{S})$ to converge towards the noise distribution
of target domain $P(\mathbf{n}^{T})$ becomes more reasonable. To this end, we
utilize an adversarial learning manner to reduce the noise distribution
discrepancy between source and target domains (namely residual distribution
alignement (RDA)).

Specifically, a parameterized discriminator $D$ can be utilized to distinguish
whether the estimated noise $\hat{\mathbf{n}}^{S}$ on source domain is similar
with the estimated noise $\hat{\mathbf{n}}^{T}$ on target domain. The
significant discrepancy of noise distribution between the two domains will
induce a high discriminator loss, which would push the reconstruction model
$F$ to reduce the noise distribution gap. Here, we adopt the LSGAN[36]-based
adversarial learning process due to its more stable convergence, which can be
formulated as following,

$\displaystyle\min_{\theta_{D}}\mathcal{L}(D)$ $\displaystyle=$
$\displaystyle\mathbb{E}_{\mathbf{x}^{T}\sim
P(\mathbf{x}^{T})}[D[\mathbf{x}^{T}-G(E(\mathbf{x}^{T}))]-1]$ $\displaystyle+$
$\displaystyle\mathbb{E}_{\mathbf{x}^{S}\sim
P(\mathbf{x}^{S})}[D[\mathbf{x}^{S}-G(E(\mathbf{x}^{S}))]-0)],$
$\displaystyle\min_{\mathbf{w}_{E},\mathbf{w}_{G}}\mathcal{L}(E,G)$
$\displaystyle=$ $\displaystyle\mathbb{E}_{\mathbf{x}^{S}\sim
P(\mathbf{x}^{S})}[D[\mathbf{x}^{S}-G(E(\mathbf{x}^{S}))]-1].$

One can observe from Eq II-D that the noise distribution discrepancy between
source and target domains will be reduced gradually though optimizing two
adversarial losses for the discriminator $D$ and the reconstruction model
$F=E\circ G$. Moreover, the reconstruction loss between the estimated NDCT
images $\hat{\mathbf{y}}^{S}$ and corresponding ground-truth NDCT images
$\mathbf{y}^{S}$ on the source domain would avoid an excessive noise
distribution transfer from source domain to target one. By balancing the
adversarial loss and reconstruction loss, our proposed method can minimize the
noise distribution discrepancy between domains to obtain a transferable noise
distribution. By doing so, a robust cross-domain CT reconstruction model is
prone to be learned.

###  II-E Model Training and Overall Framework

Our proposed robust cross-domain CT reconstruction framework consists of three
modules, including a probabilistic encoder, a probabilistic decoder, and a
discriminator. For the probabilistic encoder and decoder, we follow the
structure of the popular CT reconstruction backbone, i.e., a widely-adopted
CPCE model [24]. To balance the computational consumption with the sufficient
probabilistic property, we follow [37] to only replace the last layer of the
encoder and the decoder with its Bayesian neural network version.
Specifically, the last layer of the encoder is a Bayesian-based CNN layer with
32 channels and $3\times 3$ kernel size. The last layer of the decoder is a
Bayesian-based deconvolutional layer with 1 channel and $3\times 3$ kernel
size. The structure of the discriminator in the RDA module is the same as that
of CycleGAN [38]. The overall objective of our proposed framework consists of
three components, including a ELBO loss, a Bayesian noise uncertainty
alignment loss, and a distributional alignment loss. Here, the reconstruction
term of ELBO loss is guided from both image space (i.e., the mean absolute
error) and feature space (i.e., the perceptual loss [39] $PL(\cdot,\cdot)$).
The overall objective can be computed as following,

$\displaystyle\mathcal{L}$ $\displaystyle=$
$\displaystyle\sum_{i}[\|\mathbf{y}_{i}^{S}-\hat{\mathbf{y}}_{i}^{S}\|_{1}+PL(\mathbf{y}_{i}^{S},\hat{\mathbf{y}}_{i}^{S})]+\operatorname{KL}[q_{\theta}(Q_{\phi})\|p(Q_{\phi})]$
$\displaystyle+$
$\displaystyle\operatorname{KL}[q_{\theta}(C_{\omega})\|p(C_{\omega})]+\beta_{1}\mathcal{L}_{BNUA}+\beta_{2}\mathcal{L}_{RDA},$

where the variance of log likelihood is set to 1 for simplification computed
by the ground-truth $\mathbf{y}_{i}$ and its estimation
$\hat{\mathbf{y}}_{i}$. The third and fourth terms aim to learn a variational
distribution $q_{\theta}(\cdot)$ to approximate the Bayesian posterior
distribution on the weights, while minimizing the KL divergence with its prior
distribution $p(\cdot)$. $\beta_{1}$ and $\beta_{2}$ control the influence of
the BNUA module and RDA module, respectively.

TABLE I: The details of used datasets for the validation of the robust cross-
domain CT reconstruction task.

Dataset Domain A Domain B Scaning parts Abdomen Head kVs 100 120 mAs Dynamic
Dynamic Resolution $512\times 512$ $512\times 512$ Training Images 3952 (7
patients) 1476 (49 patients) Validation Images 850 (1 patient) 35 (1 patient)
Testing Images 1108 (2 patients) 131 (4 patients)

##  III Experiments and Analyses

###  III-A Datasets and Training Protocol

In this paper, the performance of the proposed robust cross-domain CT
reconstruction is validated on public clincal dataset, i.e, the Low Dose CT
Image and Projection Data (LDCT-and-Projection-
data)111https://wiki.cancerimagingarchive.net/pages/, which is released by
Mayo Clinic, where abdomen (namely Domain A) and head (namely Domain B) CT
scans are utilized as the cross-domain CT reconstruction task. In the LDCT-
and-Projection-data dataset, both normal-dose and simulated low (quarter)-dose
CT image are reconstructed using commercial CT system with a filtered back
projection method. There are 10 patients’ abdomen CT scans, where 7 patients
for training, 1 patient for validation, and 2 patient for testing. Moreover,
there are 54 patients’ head CT scans, where 49 patients for training, 1
patient for validation, and 4 patient for testing. As shown in Figure 3, the
significant noise distribution shifts between the abdomen and head scans can
be observed (see the third column of Figure 3 and corresponding probability
density function (PDF). As a result, the robust cross-domain CT reconstruction
is a challenging and meaningful task. More details about the used datasets can
be found in Table I. For data preprocessing, all CT scans are normalized into
$0\sim 1$ with anatomical-specific window widths (head: 90 and abdomen: 400)
and window levels (head: 35 and abdomen: 40). Each of CT scans is randomly
split into 8 patches with the size of $64$ for training.

Here, $A\rightarrow B$ denotes a robust cross-domain CT reconstruction task,
where the abdomen and head CT images as the source and target domains,
respectively, and vice versa. Thus, there are two robust cross-domain CT
reconstruction tasks.

###  III-B Implement Details

The proposed robust cross-domain CT reconstruction model is implemented by
Pytorch and BayesianTorch222https://github.com/IntelLabs/bayesian-torch. We
adopt mean-field variational inference (MFVI)[40] for the analytical
approximation of the Bayesian layer, where the parameters are characterized by
fully factorized Gaussian distribution endowed by variational parameters $\mu$
and $\sigma$, i.e. $q_{\theta}(w):=\mathcal{N}(w|\mu,\sigma).$ By using
BayesianTorch, deterministic neural networks can be transformed easily into
their Bayesian versions. For the parameter settings of Bayesian weight priors,
empirical Bayes using DNN (MOPED) [41] method is used, where the initial
perturbation factor $\delta$ for the weight to 0.1. The number of MC sampling
during training is set to 10 empirically.

The parameters of the overall model are updated by Adam optimizer with the
learning rate as $1\times 10^{-4}$. The number of the mini-batch is set to 32.
For each robust cross-domain task, the training set consists of the paired
LDCT/NDCT data in the training set of the source domain and the LDCT data in
the training set of the target domain. The important hyperparameters of our
proposed method, including $\beta_{1}$ and $\beta_{2}$, are selected on the
validation set of the source domain. Specifically, $\beta_{1}$ and $\beta_{2}$
are set to 10 and 0.001 for two robust cross-domain tasks, respectively.
Moreover, the robust cross-domain performance of the model is evaluated on the
testing set of the target domain.

Figure 3: Example samples of the used abdomen and head datasets. The noise map
of each LDCT image (in the third column) is obtained by the subtraction
between the LDCT image and corresponding NDCT image. The display windows for
the head scan and the abdomen scan are [-10,80] and [-160,240], respectively.

###  III-C Baseline Methods and Evaluation Metrics

In this paper, we compare our proposed method with some representative CT
reconstruction approaches, including Block-matching and 3D filtering (BM3D)
method [42], optimized non-local means (ONLM) method for CT reconstruction
[43], Noiser2noise [44], CycleGAN [38], unsupervised domain adaptation method
for LDCT denoising (UDA-LDCT) [10], semantic information alignment-based
cross-domain LDCT denoising (SIA-LDCT) method [12]. Note that BM3D and ONLM
are traditional model-based LDCT image reconstruction method. Noiser2noise
only refers to the LDCT images on target domain to conduct unsupervised image
denoising. By utilizing the LDCT images on target domain and the NDCT images
on source domain, CycleGAN can learn a transferable style from target domain
to source domain [38]. Compared with our proposed method, UDA-LDCT and SIA-
LDCT aim to address the noise distribution shifts between source and target
domains with a similar anatomical region. Note that we also introduce a MFVI-
based Bayesian neural network (MF-BNN)[45] as a baseline model, which is only
trained by paired source domain data. Except for imposing additional
$\mathcal{L}_{BNUA}$ and $\mathcal{L}_{RDA}$ modules, the network structure of
our proposed method is equal to that of the MF-BNN. The hyperparameters of
baseline methods are tuned in a wide range on the validation set of the source
domain.

Figure 4: The visual comparison among different baseline methods for
$\mathcal{A}\rightarrow\mathcal{B}$ robust cross-domain CT reconstruction
task. The green box on the left bottom of each image is the zoomed-in view of
the selected ROI. The quantitative results of each image are listed in the top
right corner. The display window is [-160,240].

To comprehensively evaluate the performance of different models, we introduce
two kinds of evaluation approaches, including image-based evaluation metrics
and perception-based evaluation metrics. The former includes the peak signal-
to-noise ratio (PSNR) and the structural similarity index measure (SSIM) [46].
The latter includes the gradient magnitude similarity deviation (GMSD) [47]
and discrete cosine transform-based sub-bands similarity index (DSS) [48],
which both derive from human visual system (as an efficient perceptual image
quality tool).

TABLE II: The experimental results on the robust cross-domain CT
reconstruction task from $\mathcal{A}\rightarrow\mathcal{B}$, i.e., the head
CT images as the source domain and the abdomen CT images as the target domain.
The average value and standard deviation are reported by running each method
with five times. For PSNR and SSIM, the higher, the better. For GMSD and DSS,
the lower, the better.

Method PSNR $\uparrow$ SSIM $\uparrow$ GMSD $\downarrow$ DSS $\downarrow$ MF-
BNN 19.2601$\pm$0.1769 0.4892$\pm$0.0121 0.1789$\pm$0.0012 0.3556$\pm$0.0011
BM3D 20.2815$\pm$0.1976 0.5824$\pm$0.0201 0.2001$\pm$0.0101 0.4618$\pm$0.0009
ONLM 20.0401$\pm$0.1622 0.5741$\pm$0.0321 0.1677$\pm$0.0154 0.3101$\pm$0.0003
Noiser2Noise 19.4195$\pm$0.1893 0.5412$\pm$0.0211 0.1775$\pm$0.0178
0.3795$\pm$0.0011 CycleGAN 19.3777$\pm$0.2001 0.5436$\pm$0.0134
0.1887$\pm$0.0221 0.3863$\pm$0.0010 SIA-LDCT 19.9717$\pm$0.1799
0.5011$\pm$0.0982 0.1499$\pm$0.0120 0.3316$\pm$0.0004 UDA-LDCT
20.2965$\pm$0.1828 0.5894$\pm$0.0114 0.1693$\pm$0.0203 0.3312$\pm$0.0027 Ours
22.7201$\pm$0.1782 0.6137$\pm$0.0213 0.1269$\pm$0.0104 0.2519$\pm$0.0003

###  III-D Experimental Results

Source: Head, Target: Abdomen. First, we utilize the head scans as the source
domain and the abdomen scans as the target domain. We can analyze from Figure
3 that the noise distribution shifts between source and target domains mainly
result from the mismatch of the noise level (e.g., the variance of the noise
distribution). Specifically, the noise level of the head scans is roughly
lower than that of the abdomen scans. As a result, the model trained on head
scans may not be powerful to remove the noise from the abdomen scans, which is
a challenging domain gap.

Figure 5: The visual comparison among different baseline methods for
$\mathcal{B}\rightarrow\mathcal{A}$ robust cross-domain CT reconstruction
task. The green and red boxes on the left and right bottom of each image are
the zoomed-in view of the selected ROIs. The quantitative results of each
image are listed in the top right corner. The display window is [-10,80].

The quantitative results of different baseline methods are reported in the
Table II. We have some observations as follows: First, our proposed method
outperforms all baseline methods with a clear margin. Second, UDA-LDCT, SIA-
LDCT and our proposed method roughly achieve better peroformance (especially
for the PSNR, GMSD and DSS) than Noiser2noiser and CycleGAN, which is
reasonable as these cross-domain CT reconstruction methods can leverage extra
supervised information from the source domain compared with
unsupervised/unpaired schemes. However, compared with UDA-LDCT and SIA-LDCT,
our proposed method not only can alleviate the noise distribution shifts
between the source and target domains via the noise distribution alignment in
the latent and image space, but also can avoid the negatively transferable
effect resulting from the difference of anatomical structure. Third, although
BM3D and ONLM can obtain acceptable performance in terms of PSNR and SSIM, the
perceptual quantitative results (such as GMSD and DSS) are significantly worse
than what our proposed method achieves.

The visual comparison among different baseline approaches is shown in Figure
4. We can observe from the zoomed-in view that our proposed method not only
can suppress the noise well (please see the region of low resistance in the
top right corner), but also can have a more natural texture to the NDCT images
(compared with BM3D, ONLM, Noiser2noise, and CycleGAN). Moreover, our proposed
method has a good contrast compared with SIA-LDCT and MF-BNN.

TABLE III: The experimental results on the robust cross-domain CT
reconstruction task from $\mathcal{B}\rightarrow\mathcal{A}$, i.e., the
abdomen CT images as the source domain and the head CT images as the target
domain. The average value and standard deviation are reported by running each
method with five times. For PSNR and SSIM, the higher, the better. For GMSD
and DSS, the lower, the better.

Method PSNR $\uparrow$ SSIM $\uparrow$ GMSD $\downarrow$ DSS $\downarrow$ MF-
BNN 24.3351$\pm$0.1336 0.6386$\pm$0.0075 0.1439$\pm$0.0026 0.2349$\pm$0.0041
BM3D 25.7475$\pm$0.1801 0.6525$\pm$0.0112 0.1534$\pm$0.0092 0.3530$\pm$0.0009
ONLM 26.6574$\pm$0.3231 0.6466$\pm$0.0282 0.1501$\pm$0.1231 0.3090$\pm$0.0213
Noiser2Noise 23.8407$\pm$0.1589 0.5792$\pm$0.0313 0.1628$\pm$0.0091
0.3132$\pm$0.0233 CycleGAN 23.7163$\pm$0.4285 0.6160$\pm$0.0382
0.1537$\pm$0.0093 0.3180$\pm$0.0224 SIA-LDCT 23.0560$\pm$0.1961
0.6403$\pm$0.0218 0.1375$\pm$0.0031 0.2738$\pm$0.0103 UDA-LDCT
22.4791$\pm$0.3118 0.6192$\pm$0.0435 0.1773$\pm$0.0022 0.3421$\pm$0.0226 Ours
25.4989$\pm$0.1478 0.6642$\pm$0.0345 0.1253$\pm$0.0017 0.1950$\pm$0.0196

Source: Abdomen, Target: Head. The second robust cross-domain CT
reconstruction task is from abdomen scans (as the source domain) to head scans
(as the target domain). The quantitative results of different baseline methods
are reported in the Table III. some observations can be noticed as following:
First, except for the PSNR, our proposed method outperforms the best
performance compared with other baseline methods, in terms of SSIM, GMSD, and
DSS. Instead, although ONLM and BM3D achieves the best and the second-best
performances for PSNR and SSIM, they do not have competitive perceptual scores
(such as GMSD and DSS) compared with our proposed method. This phenomenon can
be reflected by the visualized comparison in Figure 5. Specifically, it seems
that BM3D and ONLM have better capacity for noise removal compared with our
proposed method (see the zoomed-in green box). However, we can observe that 1)
BM3D suffers from the over-smooth problem (see the green circle of the zoomed-
in red box), leading to the unnatural texture compared with NDCT image. 2)
ONLM losses the tiny structure due to the excessive noise removal (see the
green circle of the zoomed-in red box). Instead, our proposed method achieves
better noise suppression compared with other deep learning-based method (see
the red arrow of the zoomed-in green box). Moreover, a more natural texture
and good structure retaining can be realized compared with traditional model-
based approaches.

TABLE IV: Ablation study on different components on $A\rightarrow B$ robust
cross-domain CT reconstruction task.

Setting Deterministic Bayesian BNUA RDA PSNR $\uparrow$ SSIM $\uparrow$
GMSD$\downarrow$ DSS$\downarrow$ 1 ✓ 18.9901$\pm$0.4311 0.4501$\pm$0.0202
0.1923$\pm$0.0101 0.3723$\pm$0.0013 2 ✓ 19.2601$\pm$0.2112 0.4892$\pm$0.0121
0.1789$\pm$0.0012 0.3556$\pm$0.0011 3 ✓ ✓ 21.7201$\pm$0.1988 0.6097$\pm$0.0210
0.1409$\pm$0.0211 0.2909$\pm$0.0011 4 ✓ ✓ 21.1312$\pm$0.1622 0.6001$\pm$0.0112
0.1298$\pm$0.0201 0.2592$\pm$0.0014 5 ✓ ✓ ✓ 22.7201$\pm$0.2342
0.6137$\pm$0.0213 0.1269$\pm$0.0104 0.2519$\pm$0.0003

Figure 6: The convergence curves of different losses during the training.

###  III-E Further analysis

In this section, we first explore the effectiveness of each component in our
proposed framework on robust cross-domain CT reconstruction task.
Deterministic Framework v.s. Probabilistic Framework. To validate the
effectiveness of the probabilistic framework, we compare the deterministic
CPCE model with our adopted Bayesian version. The quantitative results on the
$A\rightarrow B$ task are reported in Table IV. From the first and the second
rows of Table IV, we can observe the probabilistic framework outperforms the
deterministic framework, especially for the SSIM score, which is reasonable as
the probabilistic property of the weights in Bayesian network can act an
implicit regularization for better robustness.

Figure 7: The relationship between Bayesian uncertainty discrepancy (between
source and target domains) and quantitative results on validation set. The
Bayesian uncertainty discrepancy can be computed using Eq. 7. The average
value is reported by running each model with 10 times. Please zoom in for
better view.

Effectiveness of Bayesian Noise Uncertainty Alignment (BNUA). BNUA module aims
to alleviate the noise distribution discrepancy between the source and target
domains in the latent space. Here, we can notice from the third row of Figure
IV that the PSNR and SSIM scores are improved significantly via equipping the
$\mathcal{L}_{BNUA}$, compared with Bayesian neural network (in the second
row, as a baseline). Furthermore, by testing on the validation set, we compute
the Bayesian uncertainty discrepancy between source and target domains (as
described in Eq. 7) of different models (including the baseline model and the
baseline + $\mathcal{L}_{BNUA}$ model). The quantitative results can be found
in Figure 7. As we can see, the baseline model, who does not adopt any noise
distribution alignment strategy, suffers from a bigger Bayesian uncertainty
discrepancy for both two tasks (see the blue star on the bold black line),
which reflects the Bayesian uncertainty discrepancy can implicitly represent
the noise distribution gap of two domains in the latent space. Instead, by
imposing the $\mathcal{L}_{BNUA}$, the model has a smaller Bayesian
uncertainty discrepancy (see the red pentagram on the bold black line), which
also contributes to better quantitative results.

Effectiveness of Residual Distribution Alignment (RDA). RDA aims to reduce
noise distribution discrepancy between source and target domains in the image
space through the residual information (as the proxy of noise). By introducing
an adversarial manner, we can observe from the fourth row of the Figure IV
that the quantitative results (especially for the GMSD and DSS) are improved
with a clear margin compared with the baseline method. In the Figure 8, we
plot the probability density functions (PDFs) of selected ROIs in the region
of low resistance (which can reflect the noise distribution as much as
possible). We can notice that the baseline method and corresponding RDA
version can achieve an acceptable transfer from LDCT image to reconstructed
image. However, the RDA version (see the red curve) is closer to the NDCT
image (see the purple curve), which is reasonable as the discriminator in the
RDA scheme can force the model to reduce the noise distribution (using the
residual information as a proxy) discrepancy between source and target domains
such that a better robust cross-domain CT reconstruction performance can be
realized.

Figure 8: The probability density functions (PDFs) of selected ROIs in the
region of low resistance.

Empirical Convergence Analysis of Different Losses. Our proposed method
includes the ELBO loss, BNUA loss and RDA loss. Here, we conduct an empirical
convergence analysis of different losses. As shown in Figure 6, we can observe
that the ELBO loss and RDA loss have a faster convergence speed than the BNUA
loss during training.

##  IV Conclusion

In this paper, we address the problem of robust computed tomography (CT)
reconstruction issue under a cross-domain scenario. A Bayesian-endowed
probabilistic framework is introduced into a robust cross-domain CT
reconstruction task. Under this probabilistic framework, we propose to
alleviate the noise distribution shifts between source and target domains via
implicit noise modeling schemes in the latent space and image space.
Specifically, A novel Bayesian noise uncertainty alignment (BNUA) method is
proposed to conduct implicit noise distribution modeling and alignment in the
latent space. Moreover, the discrepancy of noise distribution shifts between
$D_{S}$ and $D_{T}$ is reduced by a novel adversarial residual distribution
alignment (RDA) scheme in the image space. Extensive experiments show that our
proposed method can achieve a better performance of robust cross-domain CT
reconstruction than existing methods.

## References

  * [1] Z. Yin, K. Xia, S. Wang, Z. He, J. Zhang, and B. Zu, “Unpaired low-dose ct denoising via an improved cycle-consistent adversarial network with attention ensemble,” _The Visual Computer_ , pp. 1–22, 2022.
  * [2] L. Yang, Z. Li, R. Ge, J. Zhao, H. Si, and D. Zhang, “Low-dose ct denoising via sinogram inner-structure transformer,” _IEEE Transactions on Medical Imaging_ , 2022.
  * [3] C. Niu, M. Li, X. Guo, and G. Wang, “Self-supervised dual-domain network for low-dose ct denoising,” in _Developments in X-Ray Tomography XIV_ , vol. 12242\. SPIE, 2022.
  * [4] A. Bonney, R. Malouf, C. Marchal, D. Manners, K. M. Fong, H. M. Marshall, L. B. Irving, and R. Manser, “Impact of low-dose computed tomography (ldct) screening on lung cancer-related mortality,” _Cochrane Database of Systematic Reviews_ , no. 8, 2022.
  * [5] G. Wang, J. C. Ye, and B. De Man, “Deep learning for tomographic image reconstruction,” _Nature Machine Intelligence_ , vol. 2, no. 12, pp. 737–748, 2020.
  * [6] Z. Guo, F. Zhou, Y. Chen, and J. Yuan, “A low-dose ct image denoising method combining multistage network and edge protection,” _Tehnički vjesnik_ , vol. 29, no. 3, pp. 1059–1067, 2022.
  * [7] H. Wang, X. Zhao, W. Liu, L. C. Li, J. Ma, and L. Guo, “Texture-aware dual domain mapping model for low-dose ct reconstruction,” _Medical Physics_ , 2022.
  * [8] J. Gu and J. C. Ye, “Adain-based tunable cyclegan for efficient unsupervised low-dose ct denoising,” _IEEE Transactions on Computational Imaging_ , vol. 7, pp. 73–85, 2021.
  * [9] L. Shi-min, L. Yang-chuan, Z. Ye-chen, W. Yan-ling, and G. Xin, “An improved low-dose ct image enhancement network based on cyclegan,” _Journal of Graphics_ , vol. 43, no. 4, p. 570, 2022.
  * [10] J.-Y. Lee, W. Kim, Y. Lee, J.-Y. Lee, E. Ko, and J.-H. Choi, “Unsupervised domain adaptation for low-dose computed tomography denoising,” _IEEE Access_ , 2022.
  * [11] D. Li, Z. Bian, S. Li, J. He, D. Zeng, and J. Ma, “Noise characteristics modeled unsupervised network for robust ct image reconstruction,” _IEEE Transactions on Medical Imaging_ , 2022.
  * [12] J. Huang, K. Chen, J. Sun, X. Pu, and Y. Ren, “Cross domain low-dose ct image denoising with semantic information alignment,” in _2022 IEEE ICIP_. IEEE, 2022, pp. 4228–4232.
  * [13] H. Lu, H. Han, and S. K. Zhou, “Dual-gan: Joint bvp and noise modeling for remote physiological measurement,” in _CVPR_ , 2021, pp. 12 404–12 413.
  * [14] J. Chen, J. Chen, H. Chao, and M. Yang, “Image blind denoising with generative adversarial network based noise modeling,” in _CVPR_ , 2018, pp. 3155–3164.
  * [15] M. K. Kalra, M. M. Maher, T. L. Toth, B. Schmidt, B. L. Westerman, H. T. Morgan, and S. Saini, “Techniques and applications of automatic tube current modulation for ct,” _Radiology_ , vol. 233, no. 3, pp. 649–657, 2004.
  * [16] V. Antun, F. Renna, C. Poon, B. Adcock, and A. C. Hansen, “On instabilities of deep learning in image reconstruction and the potential costs of ai,” _Proceedings of the National Academy of Sciences_ , vol. 117, no. 48, pp. 30 088–30 095, 2020.
  * [17] M. Tölle, M.-H. Laves, and A. Schlaefer, “A mean-field variational inference approach to deep image prior for inverse problems in medical imaging,” in _Medical Imaging with Deep Learning_. PMLR, 2021, pp. 745–760.
  * [18] Z. Eaton-Rosen, F. Bragman, S. Bisdas, S. Ourselin, and M. J. Cardoso, “Towards safe deep learning: accurately quantifying biomarker uncertainty in neural network predictions,” in _MICCAI_. Springer, 2018, pp. 691–699.
  * [19] H. Shan, A. Padole, F. Homayounieh, U. Kruger, R. D. Khera, C. Nitiwarangkul, M. K. Kalra, and G. Wang, “Competitive performance of a modularized deep neural network compared to commercial algorithms for low-dose ct image reconstruction,” _Nature Machine Intelligence_ , vol. 1, no. 6, p. 269, 2019\.
  * [20] M. Li, W. Hsu, X. Xie, J. Cong, and W. Gao, “Sacnn: Self-attention convolutional neural network for low-dose ct denoising with self-supervised perceptual loss network,” _IEEE transactions on medical imaging_ , vol. 39, no. 7, pp. 2289–2301, 2020.
  * [21] Y. Inoue, K. Nagahara, H. Kudo, and H. Itoh, “Ct dose modulation using automatic exposure control in whole-body pet/ct: effects of scout imaging direction and arm positioning,” _American Journal of Nuclear Medicine and Molecular Imaging_ , vol. 8, no. 2, p. 143, 2018.
  * [22] M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley, “Stochastic variational inference,” _Journal of Machine Learning Research_ , 2013.
  * [23] R. Tian, Y. Mao, and R. Zhang, “Learning vae-lda models with rounded reparameterization trick,” in _EMNLP_ , 2020, pp. 1315–1325.
  * [24] H. Shan, Y. Zhang, Q. Yang, U. Kruger, M. K. Kalra, L. Sun, W. Cong, and G. Wang, “3-d convolutional encoder-decoder network for low-dose ct via transfer learning from a 2-d trained network,” _IEEE transactions on medical imaging_ , vol. 37, no. 6, pp. 1522–1534, 2018.
  * [25] H. Li, S. J. Pan, S. Wang, and A. C. Kot, “Domain generalization with adversarial feature learning,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 5400–5409.
  * [26] M. Wang and W. Deng, “Deep visual domain adaptation: A survey,” _Neurocomputing_ , vol. 312, pp. 135–153, 2018.
  * [27] W. M. Kouw and M. Loog, “A review of domain adaptation without target labels,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 43, no. 3, pp. 766–785, 2019.
  * [28] K. Chen, J. Huang, J. Sun, Y. Ren, and X. Pu, “Task-driven deep learning for ldct image denoising,” in _The Fourth International Symposium on Image Computing and Digital Medicine_ , 2020, pp. 35–39.
  * [29] D. Hafner, D. Tran, A. Irpan, T. Lillicrap, and J. Davidson, “Reliable uncertainty estimates in deep neural networks using noise contrastive priors,” _stat_ , vol. 1050, p. 24, 2018.
  * [30] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _CVPR_ , 2018, pp. 7132–7141.
  * [31] J. Yang, K. Yu, Y. Gong, and T. Huang, “Linear spatial pyramid matching using sparse coding for image classification,” in _CVPR_. IEEE, 2009, pp. 1794–1801.
  * [32] L. Shen, G. Sun, Q. Huang, S. Wang, Z. Lin, and E. Wu, “Multi-level discriminative dictionary learning with application to large scale image classification,” _IEEE Transactions on Image Processing_ , vol. 24, no. 10, pp. 3109–3123, 2015.
  * [33] B. Sun, J. Feng, and K. Saenko, “Return of frustratingly easy domain adaptation,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 30, no. 1, 2016.
  * [34] Q. Wang and T. Breckon, “Unsupervised domain adaptation via structured prediction based selective pseudo-labeling,” in _AAAI_ , vol. 34, no. 04, 2020, pp. 6243–6250.
  * [35] C.-X. Ren, Y.-H. Liu, X.-W. Zhang, and K.-K. Huang, “Multi-source unsupervised domain adaptation via pseudo target domain,” _IEEE Transactions on Image Processing_ , vol. 31, pp. 2122–2135, 2022.
  * [36] X. Mao, Q. Li, H. Xie, R. Y. Lau, Z. Wang, and S. Paul Smolley, “Least squares generative adversarial networks,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2794–2802.
  * [37] Anonymous, “Domain generalization with small data,” in _Submitted to The Eleventh International Conference on Learning Representations_ , 2023, under review. [Online]. Available: https://openreview.net/forum?id=RKiWwhocuiU
  * [38] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2223–2232.
  * [39] J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in _European conference on computer vision_. Springer, 2016, pp. 694–711.
  * [40] A. Graves, “Practical variational inference for neural networks,” _Advances in neural information processing systems_ , vol. 24, 2011.
  * [41] R. Krishnan, M. Subedar, and O. Tickoo, “Specifying weight priors in bayesian deep neural networks with empirical bayes,” in _AAAI_ , vol. 34, no. 04, 2020, pp. 4477–4484.
  * [42] K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising with block-matching and 3d filtering,” in _Image processing: algorithms and systems, neural networks, and machine learning_ , vol. 6064. SPIE, 2006, pp. 354–365.
  * [43] Z. S. Kelm, D. Blezek, B. Bartholmai, and B. J. Erickson, “Optimizing non-local means for denoising low dose ct,” in _2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro_. IEEE, 2009, pp. 662–665.
  * [44] N. Moran, D. Schmidt, Y. Zhong, and P. Coady, “Noisier2noise: Learning to denoise from unpaired noisy data,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 12 064–12 072.
  * [45] M.-H. Laves, M. Tolle, A. Schlaefer, and S. Engelhardt, “Posterior temperature optimized bayesian models for inverse problems in medical imaging,” _Medical Image Analysis_ , vol. 78, 2022.
  * [46] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” _IEEE transactions on image processing_ , vol. 13, no. 4, pp. 600–612, 2004.
  * [47] W. Xue, L. Zhang, X. Mou, and A. C. Bovik, “Gradient magnitude similarity deviation: A highly efficient perceptual image quality index,” _IEEE transactions on image processing_ , vol. 23, no. 2, pp. 684–695, 2013.
  * [48] L. Zhang and H. Li, “Sr-sim: A fast and high performance iqa index based on spectral residual,” in _2012 19th IEEE international conference on image processing_. IEEE, 2012, pp. 1473–1476.