# Can We Reliably Improve the Robustness to Image Acquisition of Remote
Sensing of PV Systems?

Gabriel Kasmi1,2 Laurent Dubus2,3 Yves-Marie Saint Drenan1 Philippe Blanc1
1MINES Paris, Université PSL Centre Observation Impacts Energie (O.I.E.)
2RTE France
3WEMC (World Energy & Meteorology Council, UK)
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

Photovoltaic (PV) energy is crucial for the decarbonization of energy systems.
Due to the lack of centralized data, remote sensing of rooftop PV
installations is the best option to monitor the evolution of the rooftop PV
installed fleet at a regional scale. However, current techniques lack
reliability and are notably sensitive to shifts in the acquisition conditions.
To overcome this, we leverage the wavelet scale attribution method (WCAM)
[23], which decomposes a model’s prediction in the space-scale domain. The
WCAM enables us to assess on which scales the representation of a PV model
rests and provides insights to derive methods that improve the robustness to
acquisition conditions, thus increasing trust in deep learning systems to
encourage their use for the safe integration of clean energy in electric
systems.

##  1 Introduction

Photovoltaic (PV) energy grows rapidly and is crucial for the decarbonization
of electric systems [12]. The rapid growth of rooftop PV makes the estimation
of the global PV installed capacity challenging as centralized data is often
lacking [22]. Remote sensing rooftop PV on orthoimagery with computer vision
models is a promising solution for mapping rooftop PV installations. However,
current approaches lack reliability and generalize poorly from one image
provider to the other [42, 19]. Improving generalizability and reliability
requires improving the robustness to acquisition conditions and proposing
methods to assess the reliability of the decision process of the PV
classifiers [5].

Deep learning-based pipelines became the standard method for remote sensing PV
systems. DeepSolar [45] paved the way for country-wide mapping of PV systems
using deep learning and overhead imagery. While several works discuss the poor
generalizability of current methods [42, 7, 19], these works do not tackle the
robustness to varying acquisition conditions, which can be assimilated to
image corruptions [14].

In this work, we analyze the robustness to heterogeneous acquisition
conditions of models for remote sensing of PV installations using the wavelet
scale attribution method (WCAM, [23]). The WCAM assess the reliability of a
model’s decision by decomposing it into the scale-space domain. We analyze the
model’s sensitivity to acquisition conditions using the WCAM and derive a
principled method for improving the robustness to these acquisition
conditions. Our work shows that the WCAM provides a finer understanding of
what the model sees as a PV panel and guides us to improve the robustness to
acquisition conditions. By improving the reliability and robustness of deep
learning models for rooftop PV mapping, we aim to facilitate the mapping and,
thus, the integration into the electric grid of rooftop PV.

##  2 Related works

#### Remote sensing of PV installations

Many works leveraged overhead imagery and deep learning methods to map PV
installations [30, 29, 9, 46]. The DeepSolar [45] method marked a significant
milestone with mapping distributed and utility-scale installations over the
continental US using state-of-the-art deep learning models. Many works built
on DeepSolar to map regions or countries, especially in Europe [25, 1, 22, 7,
31, 32]. However, current methods cannot be transposed from one region to
another without incurring accuracy drops, thus limiting their practical
usability [19] due to a lack of reliability of the generated data [5]. To
address this gap, we propose to study and mitigate the impact of acquisition
conditions, ubiquitous with overhead imagery, which prevents reusing trained
models for registry updates.

#### Frequency-centric explanations

A line of works aimed at explaining the behavior of neural networks through
the lenses of frequency analysis. Several works showed that convolutional
neural networks (CNNs) are biased towards high frequencies [41, 44] and that
robust methods tend to limit this bias [48, 2]. Other works highlighted a so-
called spectral bias [35, 43, 21], showing that CNNs learn frequencies of the
input from the lowest to the highest. More recently, using wavelet transforms,
[23] expanded attribution methods from the pixel to the space-scale (wavelet)
domain. This work connects the fields of interpretability and robustness and
enables understanding what models see on images. It has not yet been applied
to orthoimagery, where scales are explicitly indexed.

##  3 Data and methods

#### Data

We consider the crowdsourced training dataset BDAPPV [24]. This dataset
contains annotated images of 28,000 PV panels in France and neighboring
countries. This dataset also proposes annotations of images that depict the
same PV panel but from two different image providers: images coming from the
Google Earth Engine (hereafter referred to as "Google") [10] and from the IGN,
the French public operator for geographic information. We have double
annotations for more than 8,000 PV systems. This allows us to assess the
impact of the acquisition conditions as the only change factor between two
images is the varying acquisition condition: the semantic content (the PV
panel and its surroundings) remains unchanged. The native GSD of Google images
is 10 cm/pixel and 20 cm/pixel for IGN images. We define the acquisition
conditions as the properties of the technical infrastructure (airborne or
spaceborne, camera type, image quantization, and postprocessing) and the
atmospheric and meteorological conditions the day the image was taken. Figure
3 in the appendix A presents images samples of the BDAPPV dataset.

#### A framework for measuring the impact of acquisition conditions

To assess the impact of acquisition conditions, we consider a ResNet-50 model
[13]. We train this model on Google images using standard augmentation methods
(random crops, rotations, and channel-wise normalization using the ImageNet
coefficients). Following [11], we refer to this method as "Empirical Risk
Minimization" (ERM [40]). Then, we evaluate the accuracy on a test set
containing the same images, one coming from Google and the other from IGN.
Table 1 presents the results. The surge in false negatives causes the decrease
in the F1 score. In other words, a classifier trained on Google images hardly
recognizes PV panels on IGN images.

Table 1: F1 Score and decomposition in true positives, true negatives, false positives, and false negatives computed on the test set for Google images (same provider as during training) and IGN images (same PV panels, different provider). | F1 Score | True positives | True negatives | False positives | False negatives
---|---|---|---|---|---
Google | 0.98 | 1891 | 2355 | 36 | 39
IGN | 0.44 | 566 | 2321 | 99 | 1335

#### Improving robustness to acquisition conditions

The literature on robustness to image corruptions [14] proposed numerous
approaches to improve the robustness of classification models [15, 16, 4, 3,
8]. We consider the well-established AugMix method [15] and the recently-
proposed RandAugment [4] and AutoAugment [3] methods. These methods apply a
random composition of perturbations to images during training to learn an
invariance against these perturbations. Since we know that varying acquisition
conditions mainly alter high-frequency components, we introduce two data
augmentation techniques that aim at reducing the reliance on high-frequency
components: Gaussian blurring ("Blurring") and Blurring + wavelet perturbation
(WP). Blurring consists of a fixed image blur, while the Blurring + WP also
perturbs the wavelet coefficients of the image to force the model to rely on
several rather than one scale for prediction. We refer the reader to the
appendix D for more details on the data augmentation strategies. We expect
these data augmentation techniques to reduce the accuracy drop highlighted in
Table 1.

#### Understanding the sensitivity to acquisition conditions with the Wavelet
sCale Attribution Method

Attribution [38, 37, 34] indicates the important regions for prediction, i.e.,
decomposes the prediction in the pixel (spatial) domain. The WCAM [23]
generalizes attribution to the wavelet (space-scale domain). The WCAM provides
us with two pieces of information: where the model sees and what scale (i.e.,
frequency) it sees at this location. Therefore, we can see if a prediction
relies on robust or fragile frequencies. Additionally, the decomposition of
the prediction in terms of scales is interpretable, particularly in the case
of orthoimagery. For example, on Google images, details at the 1-2 pixel scale
correspond to physical objects with a size between 0.1 and 0.2 m on the
ground. Thus, we know what the model sees as a panel; we can interpret it and
assess whether it is sensitive to varying acquisition conditions. The
decomposition brought by the WCAM enables a fine interpretation of the model’s
decision process. Appendix C provides additional background on reading and
interpreting WCAMs.

##  4 Results

Figure 1: Predictions on Google image (left, upper row) and IGN image (right,
upper row) and associated WCAMs (bottom row, displayed in logscale and with
the same color scale). The brighter, the more important the highlighted region
for the prediction

#### When important factors disappear

Changing the provider (i.e., altering the AC) alters the scales describing the
image of PV panels. If the model relied on a scale that is no longer on the
image, it can no longer recognize the PV panel. On Figure 1, we can see that
on google, the important factor was the factor (b) (on the left most image),
which is no longer important on the IGN image (on the right). On the IGN
image, the model instead relied on the factor (a) as the factor (b) is no
longer visible. This change in the important factor (at the same scale in this
example) seems to have driven the shift from predicting to not predicting the
PV panel. Interestingly, we can see that the scales highlighted in (c) are
visible on both images, but not important for the prediction in the IGN image:
the model no longer "sees" these details. We refer the reader to the appendix
C.2 for further guidance on how to interpret a WCAM.

#### Blurring and wavelet perturbation improve accuracy

Table 2 reports the results of various data augmentation that mitigate the
effect of acquisition conditions. Augmentations that explicitly discard small
scales (high frequencies) information perform the best. However, the blurring
method sacrifices the recall (which drops to 0.6) to improve the F1 score. On
Table 2, this can be seen by the increase in the number of false positives.
This method is therefore not very reliable to improve the accuracy and
robustness to acquisition conditions. On the other hand, adding wavelet
perturbation yields an improvement without sacrificing precision or recall.

Table 2: F1 Score and decomposition in true positives, true negatives, false
positives, and false negatives for models trained on Google with different
mitigation strategies. Evaluation on IGN images. The oracle corresponds to a
model trained on IGN images with standard augmentations. Best results are
bolded and second best underlined.

| F1 Score ($\uparrow$) | True positives | True negatives | False positives | False negatives
---|---|---|---|---|---
Oracle | 0.88 | 1818 | 1992 | 428 | 83
ERM [40] | 0.44 | 566 | 2321 | 99 | 1335
AutoAugment [3] | 0.46 | 598 | 2318 | 102 | 1303
AugMix [15] | 0.48 | 624 | 2318 | 102 | 1277
RandAugment [4] | 0.51 | 707 | 2280 | 140 | 1194
Blurring | 0.74 | 1855 | 1196 | 1224 | 46
Blurring + WP | 0.58 | 896 | 2114 | 306 | 1005

#### Relying on consistent scales

Figure 2 compares the scales on which the best-performing methods rely. We
want our models to rely on the largest scales (i.e., lowest frequencies) to
entail robustness [47], in our case against image quality alterations. We can
see that the blurring and wavelet perturbation enforces this property better
than other data augmentation techniques. Indeed, the model relies on coarser
scales (which are more robust) and on scales on which the ERM also relies.
More generally, the WCAM lets us compare methods that perform quantitatively
similarly.

Figure 2: WCAMs on IGN of models trained on Google with different augmentation
techniques.

##  5 Conclusions and future work

We explain why models are sensitive to acquisition conditions using the
wavelet scale attribution method (WCAM, [23]): depending on the AC, which
alter the scales, models leverage different scales to predict a PV panel. The
model no longer sees the panel if important scales (identified by the WCAM)
disappear. We show that models biased towards low frequencies are more robust
to acquisition conditions. This work paves the way for finer model monitoring
and in operational settings, increasing trust in these systems and
demonstrates the benefits of XAI in the energy field [26].

#### Future works

We plan to improve the set of data augmentation by incorporating target domain
knowledge (e.g., using the transfer function between the source and target
images) and study how the model accounts for this additional knowledge using
the WCAM.

##  6 Acknowledgements

This work is funded by RTE France, the French transmission system operator,
and benefited from CIFRE funding from the ANRT. The authors gratefully
acknowledge the support of this project.

## References

  * [1] Edoardo Arnaudo, Giacomo Blanco, Antonino Monti, Gabriele Bianco, Cristina Monaco, Paolo Pasquali, and Fabrizio Dominici.  A Comparative Evaluation of Deep Learning Techniques for Photovoltaic Panel Detection from Aerial Images.  IEEE Access, pages 1–1, 2023.
  * [2] Yiting Chen, Qibing Ren, and Junchi Yan.  Rethinking and Improving Robustness of Convolutional Neural Networks: a Shapley Value-based Approach in Frequency Domain.  October 2022.
  * [3] Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le.  AutoAugment: Learning Augmentation Policies from Data, April 2019.  arXiv:1805.09501 [cs, stat].
  * [4] Ekin D. Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le.  RandAugment: Practical automated data augmentation with a reduced search space, November 2019.  arXiv:1909.13719 [cs].
  * [5] Tim De Jong, Stefano Bromuri, Xi Chang, Marc Debusschere, Natalie Rosenski, Clara Schartner, Katharina Strauch, Marion Boehmer, and Lyana Curier.  Monitoring Spatial Sustainable Development: semi-automated analysis of Satellite and Aerial Images for Energy Transition and Sustainability Indicators.  arXiv preprint arXiv:2009.05738, 2020.
  * [6] Thomas Fel, Remi Cadene, Mathieu Chalvidal, Matthieu Cord, David Vigouroux, and Thomas Serre.  Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis, November 2021.  arXiv:2111.04138 [cs].
  * [7] Âzeddine Frimane, Robert Johansson, Joakim Munkhammar, David Lingfors, and Johan Lindahl.  Identifying small decentralized solar systems in aerial images using deep learning.  Solar Energy, 262:111822, September 2023.
  * [8] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel.  ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.  April 2023.
  * [9] Vladimir Golovko, Alexander Kroshchanka, Sergei Bezobrazov, Anatoliy Sachenko, Myroslav Komar, and Oleksandr Novosad.  Development of Solar Panels Detector.  In 2018 International Scientific-Practical Conference Problems of Infocommunications. Science and Technology (PIC S&T), pages 761–764, Kharkiv, Ukraine, October 2018. IEEE.
  * [10] Noel Gorelick, Matt Hancher, Mike Dixon, Simon Ilyushchenko, David Thau, and Rebecca Moore.  Google Earth Engine: Planetary-scale geospatial analysis for everyone.  Remote sensing of Environment, 202:18–27, 2017.  Publisher: Elsevier.
  * [11] Ishaan Gulrajani and David Lopez-Paz.  In search of lost domain generalization.  arXiv preprint arXiv:2007.01434, 2020.
  * [12] Nancy M Haegel, Robert Margolis, Tonio Buonassisi, David Feldman, Armin Froitzheim, Raffi Garabedian, Martin Green, Stefan Glunz, Hans-Martin Henning, Burkhard Holder, and others.  Terawatt-scale photovoltaics: Trajectories and challenges.  Science, 356(6334):141–143, 2017.  Publisher: American Association for the Advancement of Science.
  * [13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [14] Dan Hendrycks and Thomas Dietterich.  Benchmarking Neural Network Robustness to Common Corruptions and Perturbations, March 2019.  arXiv:1903.12261 [cs, stat].
  * [15] Dan Hendrycks, Norman Mu, Ekin D. Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan.  AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty, February 2020.  arXiv:1912.02781 [cs, stat].
  * [16] Dan Hendrycks, Andy Zou, Mantas Mazeika, Leonard Tang, Bo Li, Dawn Song, and Jacob Steinhardt.  PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures, March 2022.  arXiv:2112.05135 [cs].
  * [17] Wassily Hoeffding.  A Class of Statistics with Asymptotically Normal Distribution.  In Samuel Kotz and Norman L. Johnson, editors, Breakthroughs in Statistics: Foundations and Basic Theory, Springer Series in Statistics, pages 308–334. Springer, New York, NY, 1992.
  * [18] Toshimitsu Homma and Andrea Saltelli.  Importance measures in global sensitivity analysis of nonlinear models.  Reliability Engineering & System Safety, 52(1):1–17, April 1996\.
  * [19] Wei Hu, Kyle Bradbury, Jordan M. Malof, Boning Li, Bohao Huang, Artem Streltsov, K. Sydny Fujita, and Ben Hoen.  What you get is not always what you see—pitfalls in solar array assessment using overhead imagery.  Applied Energy, 327:120143, December 2022.
  * [20] Michiel J. W. Jansen.  Analysis of variance designs for model output.  Computer Physics Communications, 117(1):35–43, March 1999.
  * [21] Jason Jo and Yoshua Bengio.  Measuring the tendency of CNNs to Learn Surface Statistical Regularities, November 2017.  arXiv:1711.11561 [cs, stat].
  * [22] Gabriel Kasmi, Laurent Dubus, Philippe Blanc, and Yves-Marie Saint-Drenan.  Towards unsupervised assessment with open-source data of the accuracy of deep learning-based distributed PV mapping.  In Workshop on Machine Learning for Earth Observation (MACLEAN), in Conjunction with the ECML/PKDD 2022, 2022.
  * [23] Gabriel Kasmi, Laurent Dubus, Yves-Marie Saint Drenan, and Philippe Blanc.  Scale Matters: Attribution Meets the Wavelet Domain to Explain Model Sensitivity to Image Corruptions, May 2023.  arXiv:2305.14979 [cs, stat].
  * [24] Gabriel Kasmi, Yves-Marie Saint-Drenan, David Trebosc, Raphaël Jolivet, Jonathan Leloux, Babacar Sarr, and Laurent Dubus.  A crowdsourced dataset of aerial images with annotated solar photovoltaic arrays and installation metadata.  Scientific Data, 10(1):59, January 2023.
  * [25] Bala Bhavya Kausika, Diede Nijmeijer, Iris Reimerink, Peter Brouwer, and Vera Liem.  GeoAI for detection of solar photovoltaic installations in the Netherlands.  Energy and AI, 6:100111, December 2021.
  * [26] R. Machlev, L. Heistrene, M. Perl, K. Y. Levy, J. Belikov, S. Mannor, and Y. Levron.  Explainable Artificial Intelligence (XAI) techniques for energy and power systems: Review, challenges and opportunities.  Energy and AI, 9:100169, August 2022.
  * [27] S.G. Mallat.  A theory for multiresolution signal decomposition: the wavelet representation.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7):674–693, July 1989.  Conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence.
  * [28] Stéphane Mallat.  A wavelet tour of signal processing.  Elsevier, 1999.
  * [29] Jordan M. Malof, Kyle Bradbury, Leslie M. Collins, and Richard G. Newell.  Automatic detection of solar photovoltaic arrays in high resolution aerial imagery.  Applied Energy, 183:229–240, December 2016.
  * [30] Jordan M. Malof, Rui Hou, Leslie M. Collins, Kyle Bradbury, and Richard Newell.  Automatic solar photovoltaic panel detection in satellite imagery.  In 2015 International Conference on Renewable Energy Research and Applications (ICRERA), pages 1428–1431, Palermo, Italy, November 2015. IEEE.
  * [31] Kevin Mayer, Benjamin Rausch, Marie-Louise Arlt, Gunther Gust, Zhecheng Wang, Dirk Neumann, and Ram Rajagopal.  3D-PV-Locator: Large-scale detection of rooftop-mounted photovoltaic systems in 3D.  Applied Energy, 310:118469, March 2022.
  * [32] Kevin Mayer, Zhecheng Wang, Marie-Louise Arlt, Dirk Neumann, and Ram Rajagopal.  DeepSolar for Germany: A deep learning framework for PV system mapping from aerial imagery.  In 2020 International Conference on Smart Energy Systems and Technologies (SEST), pages 1–6, Istanbul, Turkey, September 2020. IEEE.
  * [33] William J. Morokoff and Russel E. Caflisch.  Quasi-Monte Carlo Integration.  Journal of Computational Physics, 122(2):218–230, December 1995\.
  * [34] Vitali Petsiuk, Abir Das, and Kate Saenko.  RISE: Randomized Input Sampling for Explanation of Black-box Models, September 2018.  arXiv:1806.07421 [cs].
  * [35] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred A. Hamprecht, Yoshua Bengio, and Aaron Courville.  On the Spectral Bias of Neural Networks, May 2019.  arXiv:1806.08734 [cs, stat].
  * [36] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  ImageNet Large Scale Visual Recognition Challenge, January 2015.  arXiv:1409.0575 [cs].
  * [37] Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization.  International Journal of Computer Vision, 128(2):336–359, February 2020.  arXiv:1610.02391 [cs].
  * [38] Karen Simonyan and Andrew Zisserman.  Very Deep Convolutional Networks for Large-Scale Image Recognition, April 2015.  arXiv:1409.1556 [cs].
  * [39] Ilya Meerovich Sobol.  On sensitivity estimation for nonlinear mathematical models.  Matematicheskoe modelirovanie, 2(1):112–118, 1990.  Publisher: Russian Academy of Sciences, Branch of Mathematical Sciences.
  * [40] Vladimir Vapnik.  The nature of statistical learning theory.  Springer science & business media, 1999.
  * [41] Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P. Xing.  High-Frequency Component Helps Explain the Generalization of Convolutional Neural Networks.  In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8681–8691, Seattle, WA, USA, June 2020\. IEEE.
  * [42] Rui Wang, Joseph Camilo, Leslie M. Collins, Kyle Bradbury, and Jordan M. Malof.  The poor generalization of deep convolutional networks to aerial imagery from new geographic locations: an empirical study with solar array detection.  In 2017 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), pages 1–8, Washington, DC, October 2017. IEEE.
  * [43] Zhi-Qin John Xu, Yaoyu Zhang, Tao Luo, Yanyang Xiao, and Zheng Ma.  Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks.  Communications in Computational Physics, 28(5):1746–1767, June 2020\.  arXiv:1901.06523 [cs, stat].
  * [44] Dong Yin, Raphael Gontijo Lopes, Jonathon Shlens, Ekin D. Cubuk, and Justin Gilmer.  A Fourier Perspective on Model Robustness in Computer Vision, September 2020.  arXiv:1906.08988 [cs, stat].
  * [45] Jiafan Yu, Zhecheng Wang, Arun Majumdar, and Ram Rajagopal.  DeepSolar: A Machine Learning Framework to Efficiently Construct a Solar Deployment Database in the United States.  Joule, 2(12):2605–2617, December 2018.
  * [46] Jiangye Yuan, Hsiu-Han Lexie Yang, Olufemi A. Omitaomu, and Budhendra L. Bhaduri.  Large-scale solar panel mapping from aerial images using deep convolutional networks.  In 2016 IEEE International Conference on Big Data (Big Data), pages 2703–2708, Washington DC,USA, December 2016. IEEE.
  * [47] Jiajin Zhang, Hanqing Chao, Amit Dhurandhar, Pin-Yu Chen, Ali Tajer, Yangyang Xu, and Pingkun Yan.  When Neural Networks Fail to Generalize? A Model Sensitivity Perspective, December 2022.  arXiv:2212.00850 [cs].
  * [48] Zhuang Zhang, Dejian Meng, Lijun Zhang, Wei Xiao, and Wei Tian.  The range of harmful frequency for DNN corruption robustness.  Neurocomputing, 481:294–309, April 2022.

##  Appendix A Additional figures

#### Examples of images

Figure 3 plots examples of images coming from BDAPPV. These images depict the
same scene for two different providers.

Figure 3: Examples of test of the same PV panels but with different providers
(Up Google, down: IGN).

##  Appendix B Training results on the Google test set

#### Training results

Table 3 reports the training results of our methods on the source (Google)
test set. We can see that a small margin outperforms our spectral method
compared to other methods on the source dataset.

Table 3: F1 Score and decomposition in true positives, true negatives, false
positives, and false negatives for models trained on Google images with
different strategies to mitigate the sensitivity to acquisition conditions.
Evaluation computed on the Google (source) dataset.

| F1 Score ($\uparrow$) | True positives | True negatives | False positives | False negatives
---|---|---|---|---|---
ERM [40] | 0.98 | 1891 | 2355 | 36 | 39
AutoAugment [3] | 0.98 | 1906 | 2340 | 51 | 24
AugMix [15] | 0.98 | 1894 | 2354 | 37 | 36
RandAugment [4] | 0.98 | 1907 | 2342 | 49 | 23
Blurring | 0.82 | 1636 | 1958 | 433 | 294
Blurring + WP | 0.90 | 1798 | 2135 | 256 | 132

##  Appendix C Reading WCAMs

###  C.1 Identification of the important scales in the input image

The content of this section comes from section 2.2. of [23]. Our only addition
is the Figure 4 and its comment (between "Figure 4 presents" and "larger than
8 pixels on the image."

#### Dyadic wavelet transform

A wavelet is an integrable function $\psi\in L^{2}(\mathbb{R})$ with zero
average, normalized and centered around 0. Unlike a sinewave, a wavelet is
localized in space and in the Fourier domain. It implies that dilatations of
this wavelet enable to scrutinize different frequencies (scales) while
translations enable to scrutinize spatial location. To compute an image’s
(continuous) wavelet transform (CWT), one first defines a filter bank
$\mathcal{D}$ from the original wavelet $\psi$ with the scale factor $s$ and
the 2D translation in space $u$. We have

$\mathcal{D}=\left\\{\psi_{s,u}(x)=\frac{1}{\sqrt{s}}\psi\left(\frac{x-u}{s}\right)\right\\}_{u\in\mathbb{R}^{2},\;s\geq
0},$ (1)

where $|\mathcal{D}|=J$, and $J$ denotes the number of levels. The computation
of the wavelet transform of a function $f\in L^{2}(\mathbb{R})$ at location
$x$ and scale $s$ is given by

$\mathcal{W}(f)(x,s)=\int_{-\infty}^{+\infty}f(u)\frac{1}{\sqrt{s}}\psi^{*}\left(\frac{x-u}{s}\right)\mathrm{d}u,$
(2)

which can be rewritten as a convolution [28]. Computing the multilevel
decomposition of $f$ requires applying Equation 2 $J$ times with all dilated
and translated wavelets of $\mathcal{D}$. [27] showed that one could implement
the multilevel dyadic decomposition of the discrete wavelet transform (DWT) by
applying a high-pass filter $H$ to the original signal $f$ and subsampling by
a factor of two to obtain the detail coefficients and applying a low-pass
filter $G$ and subsampling by a factor of two to obtain the approximation
coefficients. Iterating on the approximation coefficients yields a multilevel
transform where the $j^{th}$ level extracts information at resolutions between
$2^{j}$ and $2^{j-1}$ pixels. When dealing with images, the detail
coefficients can be decomposed into horizontal, vertical, and diagonal
components.

Figure 4 presents an example of a two-level dyadic wavelet transform. The
leftmost image is the input image. On the center-left image, the northwest,
southwest, and southeast panels represent the detail coefficients (horizontal,
diagonal, and vertical, respectively) at the smallest scale obtained from the
high pass filtering of the input image. The northeast panel represents the
approximation coefficients obtained from the low-pass filtering of the input
image. On the center-right image, in the northeast panel, the localization of
the detail and approximation coefficients is analogous to the decomposition
from stage 1; instead, the reference is the approximation from stage 1, not
the input image. On the rightmost image, the third decomposition level is
obtained from the high pass filtering of the approximation coefficients from
stage 2.

The detail coefficients at stage 1 show details at the 1-2 pixel scale. The
detail coefficients at stage 2 are the 2-4 pixel scale details. At stage 3, we
can see the detailed coefficients at the 4-8 pixel scale. The approximation
coefficients contain everything larger than 8 pixels on the image.

Figure 4: Example of a multilevel dyadic wavelet transform

#### Sobol sensitivity analysis

Let $(X_{1},\dots,X_{K})$ be independent random variables and
$\mathcal{K}=\\{1,\dots,K\\}$ denote the set of indices. Let $f$ be a model,
$X$ an input, and $f(X)$ the model’s decision (e.g., the output probability).
We denote $f_{\kappa}=f_{\kappa}(X_{\kappa})$ the partial contributions of the
variables $(X_{k})_{k\in\kappa}$ to the score $f(X)$. The Sobol-Hoeffding
decomposition [17] decomposes the decision score $f(X)$ into summands of
increasing dimension

$f(X)=f_{\emptyset}+\sum_{\kappa\in\mathcal{P}\left(\mathcal{K}\right)\backslash\\{\emptyset\\}}f_{\kappa}(X_{\kappa}),$
(3)

Where $f_{\emptyset}$ denotes the prediction with no features. Under the
orthogonality condition $\forall(u,v)\in\mathcal{K}^{2}$ such that $u\neq v$,
$\mathbb{E}\left[f_{u}(X_{u})f_{v}(X_{v})\right]=0$, we derive from Equation 3
the variance of the model’s score

$Var(f(X))=\sum_{\kappa\in\mathcal{P}\left(\mathcal{K}\right)}Var(f_{\kappa}(X_{\kappa})),$
(4)

Equation 4 enables us to describe the influence of a subset $\kappa$ of
features as the ratio between its own and total variance. This corresponds to
the first order Sobol index given by

$S_{\kappa}=\frac{Var(f_{\kappa}(X_{\kappa}))}{Var(f(X))}.$ (5)

$S_{\kappa}$ measures the proportion of the output variance $Var(f(X))$
explained by the subset of variables $X_{\kappa}$ [39]. In particular, $S_{k}$
only captures the direct contribution of the feature $X_{k}$ to the model’s
decision. To capture the indirect effect, due to the effect of $X_{k}$ on the
other variables, total Sobol indices $S_{T_{k}}$ [18] can be computed as

$S_{T_{k}}=\sum_{\kappa\in\mathcal{P}\left(\mathcal{K}\right),\,k\in\kappa}S_{\kappa}.$
(6)

Total Sobol indices (TSIs) measure the contribution of the $k^{th}$ feature,
taking into account both its direct effect and its indirect effect through its
interactions with the other features.

#### Efficient estimation of Sobol indices

As seen from Equation 5, estimating the impact of a feature $k$ on the model’s
decision requires recording the partial contribution $f_{k}(X_{k})$. This
partial contribution corresponds to a forward. Estimating Sobol indices
requires computing variances by drawing at least $N$ samples and computing $N$
forwards to estimate a first-order Sobol index $S_{k}$ of a single feature
$k$. As we are interested in the TSI of a feature $k$, we need to estimate the
Sobol index of all sets of features $\kappa\in\mathcal{K}$ such that
$k\in\kappa$. To minimize the computational cost of this computation, [6]
introduced an efficient sampling strategy based on Quasi-Monte Carlo methods
[33] to generate the $N$ perturbations of dimension $K$ applied to the input
and used Jansen’s estimator [20] to estimate the TSIs given the models’
outputs and the quasi-random perturbations. Their approach requires $N(K+2)$
forwards [6].

To estimate the TSIs, they draw two matrices from a Quasi-Monte Carlo sequence
of size $N\times K$ and convert them into perturbations, which they apply to
$X$. The perturbated input yields two matrices, $A$ and $B$. $a_{jk}$ (resp.
$b_{kj}$) is the element of $A$ (resp. $B$) corresponding to the $k^{th}$
feature and the $j^{th}$ sample. For the $k^{th}$ feature, they define
$C^{(k)}$ in the same way as $A$, except that the column corresponding to
feature $k$ is replaced by the column of $B$. They then derive an empirical
estimator for the Sobol index and TSI as

$\hat{S}_{k}=\frac{\hat{V}-\frac{1}{2N}\sum_{j=1}^{N}\left[f(B_{j})-f\left(C_{j}^{(k)}\right)\right]^{2}}{\hat{V}},\;\;\;\;\;\;\;\hat{S}_{T_{k}}=\frac{\frac{1}{2N}\sum_{j=1}^{N}\left[f(A_{j})-f\left(C_{j}^{(k)}\right)\right]^{2}}{\hat{V}},$
(7)

where $f_{\emptyset}=\displaystyle{\frac{1}{N}\sum_{j=1}^{N}f(A_{j})}$ and
$\hat{V}=\displaystyle{\frac{1}{N-1}\sum_{j=0}^{N}\left[f\left(A_{j}\right)-f_{\emptyset}\right]^{2}}$.
Further implementational details can be found in [6]. Figure 5 summarizes the
Wavelet sCale Attribution Method of [23].

Figure 5: Flowchart of the wavelet scale attribution method (WCAM). [23]
generate a set of perturbed images by perturbing the image’s discrete wavelet
transform (DWT). Expanding [6], they apply masks generated from a Quasi Monte-
Carlo sequence to the DWT of the image. They evaluate the model on the altered
samples reconstructed from the perturbed DWT. They compute the Sobol index for
each mask component using the predicted probabilities and the perturbation
masks using the Jansen estimator [20]

###  C.2 Decomposing the scales of the PV panels

On orthoimagery, PV panels are located in space and in scale. Figure 6 shows
an example. Depending on the scale of interest, the PV panel will show
different characteristics. At the smallest scales (i.e., high frequencies),
the PV panel corresponds to small details within the individual PV modules. On
the opposite, if we consider the PV system as a whole, it has a size of about
10m, i.e., 100 pixels in this image.

Figure 6: Decomposition of the scales of a PV panel

The WCAM enables us to see the important scales for the prediction and where
they are located. It lets us know what the model sees from the panel: the
overall system, the cluster, or individual modules.

Besides it also illustrates the impact of background noise on the detection.
If small scales not located on the panel are important for the prediction, it
indicates that the model relies on background noise for the prediction. For
our example image of Figure 6 could be the grass around the house.

###  C.3 Some examples

Figure 7 presents some examples of predictions made by a PV classifier. We can
see that for our illustration image (leftmost image on Figure 7), the grass,
but also the shape of the garden on the upper left of the image, play a small
role in the prediction of the PV panel. On the other hand, in the two
subsequent images, the prediction relies on factors at different scales
located around the PV panel.

The WCAM lets us see that when a model focuses on a PV panel, its focus can be
decomposed into different scales. Besides, these scales do not always have the
same importance. For instance, the importance of the 1-2 pixel details is more
important on the third image from the left than for the other images. It could
be because the panel is frameless, so the model has to rely on other factors,
as this one is missing for this image.

Figure 7: Decomposition in the space-scale domain of PV panel predictions

##  Appendix D Data augmentation strategies

###  D.1 Description of the data augmentations

#### AugMix [15]

The data augmentation strategy "Augment-and-Mix" (AugMix) consists in
producing a high diversity of augmented images from an input sample. A set of
operations (perturbations) to be applied to the images are sampled, along with
sampling weights. The image resulting $x_{aug}$ is obtained through the
composition $x_{aug}=\omega_{1}op_{1}\circ\dots\omega_{n}op_{n}(x)$ where $x$
is the original image. Then, the augmented image is interpolated with the
original image with a weight $m$ that is also randomly sampled. We have
$x_{augmix}=mx+(1-m)x_{aug}$.

#### AutoAugment [3]

This strategy aims at finding the best data augmentation for a given dataset.
The authors determined the best augmentations strategy $S$ as the outcome of a
reinforcement learning problem: a controller predicts an augmentation policy
from a search space. Then, the authors train a model, and the controller
updates its sampling strategy $S$ based on the train loss. The goal is that
the controller generates better policies over time. The authors derive optimal
augmentation strategies for various datasets, including ImageNet [36], and
show that the optimal policy for ImageNet generalizes well to other datasets.

#### RandAugment [4]

This strategy’s main goal is to remove the need for a computationally
expansive policy search before model training. Instead of searching for
transformations, random probabilities are assigned to the transformations.
Then, each resulting policy (a weighted sequence of $K$ transformations) is
graded depending on its strength. The number of transformations and the
strength are passed as input when calling the transformation.

#### Blurring

We apply a nonrandom Gaussian blur to the image. The value is set by comparing
visually Google and IGN images and trying to remove details from Google images
that are not visible on IGN images.

#### Blurring + Wavelet perturbation (WP)

We first blur the image. Then, for each color channel, we compute the dyadic
wavelet transform of the image and randomly perturb the coefficients (we
randomly set some coefficients to 0). The set of coefficients set to 0 is
determined with uniform sampling. This results in a random perturbation that
removes information for some precise scales and locations. We then reconstruct
the image from its perturbed wavelet coefficients.

##  Appendix E Complementary results

#### Probability shift

Figure 8 shows the change in predicted probabilities for positive Google
images. We can see that when the classifier no longer recognizes the PV panel,
the probability shift is large, suggesting that the important factor for
prediction disappeared from the image. These results provide more systematic
evidence of the qualitative pattern highlighted by Figure 1: if a critical
component is no longer depicted due to the change in the acquisition
condition, then the model no longer sees a PV panel (despite having other
information about it).

Figure 8: Evolution of the predicted probabilities for images depicting a PV
panel on the Google test set and the corresponding images on the IGN test set.
The predicted probability completely flips over when the model no longer
recognizes the PV panel.

#### Understanding false positives

On Figure 9, we plot some examples of false positives and their associated
WCAM. We can see that the WCAM helps us understand why the model misleads an
element of the image with a PV panel.

In the leftmost image, we can see that the shape of the roof mainly causes the
false alarm is reminiscent of the delineation of a PV system on a roof. On the
other hand, in the fourth image (from the left), multiple factors at different
scales lead to a wrong prediction.

Figure 9: Examples of false positives on IGN and corresponding WCAM.


# Deep Injective Prior for Inverse Scattering

AmirEhsan Khorashadizadeh, Sepehr Eskandari, Vahid Khorashadi-Zadeh and Ivan
Dokmanić  Manuscript received December 29, 2022.
AmirEhsan Khorashadizadeh and Ivan Dokmanić were supported by the European
Research Council Starting Grant 852821–SWING. AmirEhsan Khorashadizadeh is
with the Department of Mathematics and Computer Science of the University of
Basel, 4001 Basel, Switzerland (e-mail: amir.kh@unibas.ch). Sepehr Eskandari
is with Microwave Systems, Sensors, and Imaging Lab (MiXIL), University of
Southern California, Los Angeles, CA 90089 USA (e-mail: sepehres@usc.edu).
Vahid Khorashadi-Zadeh is with School of Electric and Computer Engineering,
University of Tehran, Tehran 14399-57131, Iran (e-mail:
v.khorashadi@ut.ac.ir). Ivan Dokmanić is with the Department of Mathematics
and Computer Science of the University of Basel, 4001 Basel, Switzerland, and
also with the Department of Electrical, Computer Engineering, the University
of Illinois at Urbana-Champaign, Urbana, IL 61801 USA (e-mail:
ivan.dokmanic@unibas.ch).
Our implementation is available at https://github.com/swing-
research/scattering_injective_prior. 0000-0003-0660-6823 0000-0003-2831-6288
0000-0001-9786-7812 0000-0001-7132-5214

###### Abstract

In electromagnetic inverse scattering, we aim to reconstruct object
permittivity from scattered waves. Deep learning is a promising alternative to
traditional iterative solvers, but it has been used mostly in a supervised
framework to regress the permittivity patterns from scattered fields or back-
projections. While such methods are fast at test-time and achieve good results
for specific data distributions, they are sensitive to the distribution drift
of the scattered fields, common in practice. If the distribution of the
scattered fields changes due to changes in frequency, the number of
transmitters and receivers, or any other real-world factor, an end-to-end
neural network must be re-trained or fine-tuned on a new dataset. In this
paper, we propose a new data-driven framework for inverse scattering based on
deep generative models. We model the target permittivities by a low-
dimensional manifold which acts as a regularizer and learned from data. Unlike
supervised methods which require both scattered fields and target signals, we
only need the target permittivities for training; it can then be used with any
experimental setup. We show that the proposed framework significantly
outperforms the traditional iterative methods especially for strong scatterers
while having comparable reconstruction quality to state-of-the-art deep
learning methods like U-Net.

##  I Introduction

electromagnetic inverse scattering is the problem of determining the
electromagnetic properties of unknown objects from how they scatter incident
fields. As it is non-destructive, it has a variety of applications in
different areas, such as early detection of breast cancer [1], mineral
prospecting [2], detecting defects and cracks inside objects [3], imaging
through the wall [4] and remote sensing [5].

We consider the reconstruction of the finite number of parameters of the
object from the scattered fields. While inverse scattering is well-posed and
Lipschitz stable in theory, when full-aperture continuous measurements are
available [6], it is a severely ill-posed inverse problem for a finite number
of measurements. This means that a small perturbation in the scattered fields
may lead to a large error in the reconstructed permittivity pattern [7].
Moreover, the forward operator from the permittivity to the scattered fields
is nonlinear, which further complicates the inversion. The nonlinearity is due
to the multiple scattering; the problem becomes more nonlinear as the
permittivity contrast increases [7]. All these together make the inverse
scattering challenging, especially for strong scatterers (objects with large
permittivity) and noisy measurements, which require an effective
regularization to restrict the search space and enable accurate
reconstruction.

A number of optimization-based methods are proposed to address nonlinearity
and ill-posedness of the inverse scattering including Born iterative [8],
distorted Born iterative method (DBIM) [9], contrast source inversion (CSI)
[10], and subspace-based optimization (SOM) [11]. Although these methods have
been shown to be effective for objects with small permittivity, they do not
give an accurate reconstruction for large permittivity. All the above methods
iteratively optimize a regularized objective, with a hand-crafted
regularization term.

Recently, thanks to the significant increase in computing power and the
availability of sufficient data, data-driven approaches to inverse scattering
have started receiving attention. Most deep learning models for inverse
scattering use a supervised learning approach, which takes the scattered
fields or a simple back-projection as input and trains a deep neural network
to produce the target permittivity pattern. The authors of [12, 13, 14] used
scattered fields as input. While this approach provides good reconstructions
even for objects with strong scatterers [14], it is sensitive to the
experiment configuration such as the frequency or the number of incident waves
and receivers; if the distribution of the scattered fields in test-time
slightly changes, the quality of reconstructions remarkably degrades; the
model requires new training data which is too costly. One strategy to tackle
this issue is to use back-projections as input instead of the raw scattered
fields, thus enabling the use of convolutional neural networks [15, 16, 17].
While this approach is shown to be effective for objects with small and
moderate permittivity, the quality of the back-projections significantly drops
in large permittivity (Figure 3), which leads to a drop in the reconstruction
quality [14]. On the other hand, supervised learning methods are also
potentially vulnerable to adversarial attacks [18], which is problematic in
medical applications [19]. Moreover, incorporating the well-known physics of
the scattering problem (forward operator), which can strikingly improve the
accuracy of the reconstructions, is not easy in supervised learning models
[20].

To tackle these issues, we propose a deep learning approach to inverse
scattering using injective generative models. The proposed method benefits
from an unsupervised learning framework—the training phase uses only the
target permittivity patterns and the physics of scattering is fully
incorporated into the solution. Deep generative models are a class of
unsupervised learning methods that model the probability distribution of data
by using deep neural networks. Latent-variable generative models such as
generative adversarial networks (GAN) [21, 22], variational autoencoders [23],
normalizing flows [24, 25, 26] and diffusion models [27] train a deep neural
network to transform the samples of a simple (Gaussian) distribution to those
of target data distribution. We expect a trained generator produce plausible
samples similar to the training data when taking samples of a Gaussian
distribution as input.

Bora et al. [28] used GANs to constrain the solution domain to the range of
the trained generator for solving compressed sensing inverse problems. The
strength of this idea is that it requires only the target signals for training
and does not know anything about the inverse problem is going to be solved. As
soon as the generator is trained, it can be used to solve any inverse problem
with a known forward operator. This property makes the model robust to the
distribution shift of the measurements and adversarial attack.

Several methods have tried to improve the quality of reconstructions; Kelkar
et al. [29] used the popular style-GAN generator [30] while Hussein et al.
[31] jointly optimize the generator weights with latent codes to further
reduce the reconstructions error. However, GANs are known to be unstable in
training, and the optimization process over latent space sticks in the local
minimum which requires many restarts [28]. Recently, normalizing flows as a
generator instead of GANs have been shown to have a stable optimization
landscape [32, 33]. Nevertheless, normalizing flows have their own drawbacks;
having a latent code with the same dimension as data space makes them too
expensive in training and does not provide an appropriate constraint in the
generator output, leading to poor reconstruction for ill-posed inverse
problems. More recently, injective normalizing flows [34, 35] are proposed to
alleviate these issues. Injective flows are a class of deep generative models
that are well-suited for solving ill-posed inverse problems. These models
unlike GANs give us access to the approximate likelihood of the generated
samples, which provides a likelihood-based regularization. Moreover, injective
flows unlike regular normalizing flows benefit from a low-dimensional latent
space which provides an additional effective regularizer for ill-posed inverse
problems. Injective flows have been shown in [35] to effectively solve linear
inverse problems.

In this paper, we use injective flows as the generator to solve full-wave
inverse scattering. we will show that the proposed method effectively solves
inverse scattering even for objects with large permittivity for which
traditional methods fail. Moreover, we use a data-driven initial guess for
starting the iterative optimization, which significantly outperforms the
simple back-projection initialization used in the traditional methods.
Finally, we show that the proposed framework yields reconstructions of
comparable or better quality to highly successful supervised methods like the
U-Net [36].

##  II Forward and Inverse Scattering

Figure 1: The setup for the inverse scattering problem, red arrows show the
incident plane waves; the green circles are the receivers.

We begin our discussion with equations governing the 2D forward and inverse
scattering problem. We consider two-dimensional transverse magnetic
scattering, where the longitudinal direction is along the $z$ direction
($TM_{z}$). As shown in Figure 1, non-magnetic scatterers with permittivity
$\epsilon_{r}$ in a vacuum background with permittivity $\epsilon_{0}$ and
permeability $\mu_{0}$ are located in investigation domain $\mathcal{D}_{inv}$
which is a $D{\times}D$ square, and are illuminated by $N_{i}$ plane waves
with equispaced directions. We have $N_{r}$ receivers placed uniformly on a
circle $S$ with radius $R$ which measure the scattered fields. The forward
scattering problem can be derived from the time-harmonic form of Maxwell’s
equations and stated as[37],

$\nabla\times(\nabla\times
E^{t}(r))-k_{0}^{2}\epsilon_{r}(r)E^{t}(r)=i\omega\mu_{0}J(r)$ (1)

where $E^{t}$ is the total electric field,
$k_{0}=\omega\sqrt{\mu_{0}\epsilon_{0}}$ is the wavenumber of the homogeneous
background and $J$ is the contrast current density and can be calculated using
equivalence theorem [38] as $J(r)=\chi(r)E^{t}(r)$ where
$\chi(r)=\epsilon_{r}(r)-1$ and is called the contrast. The time-dependence
factor $\exp(i\omega t)$ with angular working frequency $\omega$ is assumed
and will be suppressed throughout this paper. By using Dyadic Green’s function
[39], Equation (1) can be formalized by two coupled integral equations. The
first one, called Lippmann–Schwinger equation, relates total electric fields
$E^{t}$ in an unknown domain to contrast current density $J$,

$E^{t}(r)=E^{i}(r)+k_{0}^{2}\int\displaylimits_{\mathcal{D}_{inv}}g(r,r^{\prime})J(r^{\prime})dr^{\prime},$
(2)

where $r\in\mathcal{D}_{inv}$, and

$g(r,r^{\prime})=\dfrac{1}{4i}H^{(2)}_{0}(k_{0}|r-r^{\prime}|),$

and $H^{(2)}_{0}$ is the Hankel function of the second kind, denotes 2D free
space Green’s function. The second equation, referred to as the data equation,
maps the contrast current density $J$ to the scattered electric fields $E^{s}$
at the receivers locations,

$E^{s}(r)=k_{0}^{2}\int\displaylimits_{\mathcal{D}_{inv}}g(r,r^{\prime})J(r^{\prime})dr^{\prime},\textbf{
}r\in S.$ (3)

We discretize the investigation domain $\mathcal{D}_{inv}$ with $N\times N$
units and rewrite Equations (2) and (3) as

$\displaystyle E^{t}$ $\displaystyle=E^{i}+G_{d}\chi E^{t}$ (4) $\displaystyle
E^{s}$ $\displaystyle=G_{s}\chi E^{t}+\delta$ (5)

where $G_{d}\in\mathbb{R}^{N^{2}\times N^{2}}$ and
$G_{s}\in\mathbb{R}^{N_{r}\times N^{2}}$ have analytical closed form [7].
Moreover, $E^{t}$, $E^{i}$ and $E^{s}$ respectively correspond to total,
incident and scattered electric fields and ${\chi}$ is a diagonal matrix with
the diagonal elements ${\chi}(n,n)=\epsilon_{r}(r_{n})-1$. We also consider
additive noise $\delta$ to the measurements.

We merge Equations (4) and (5) to make a single expression for the forward
equation [7],

$E^{s}=G_{s}\chi(I-G_{d}\chi)^{-1}E^{i}+\delta,$ (6)

which is a nonlinear mapping $\chi\mapsto E^{s}$. It is convenient to define a
forward operator $A$ mapping $\chi$ to $E^{s}$,

$y=A(x)+\delta,$ (7)

where $A(\cdot)$ is the nonlinear forward scattering operator

$A(\chi)=G_{s}\chi{(I-G_{d}\chi)}^{-1}E^{i},$ (8)

$y=E^{s}$ and $x=\chi$. The task of inverse scattering is to reconstruct the
contrast signal $\chi$ from the scattered fields $E^{s}$ where we assume
$G_{d}$, $G_{s}$, incident electric waves $E^{i}$ and consequently the forward
operator $A(\cdot)$ are known. In the next section, we briefly review deep
generative models as the prior model for inverse problems.

##  III Deep Generative Models

One major division in machine learning is generative and discriminative
models. In discriminative models, one aims to learn a direct mapping from the
measurements to the target signals. Discriminative approaches have been
extensively used in solving inverse problems, specifically U-Net [36] has
shown great success in many applications such as computed tomography (CT)
[40], magnetic resonance imaging (MRI) [41], optoacoustic tomography [42] and
electromagnetic inverse scattering [16]. The key idea of this success might be
ascribed to having a multiscale architecture [43]. Recently, the variational
version of U-Net has shown excellent performance for posterior sampling and
uncertainty quantification of inverse scattering problem [44].

On the other hand, the task of generative models is to learn the probability
distribution of data. Latent-variable generative models including generative
adversarial networks (GANs) [21, 22], variational autoencoders [23] and
normalizing flows [24, 25, 26] are a subcategory of deep generative models
(DGMs) which train a deep neural network, called the generator, to transform
the samples of a simple and known distribution (often Gaussian) to data
distribution samples. DGMs have many applications such as image
generation[22], image-to-image translation [45], density estimation [46] and
variational inference [47, 48]. Recently, DGMs have been used as a prior for
solving inverse problems [28, 49, 35, 50, 51, 52]; consider a DGM trained on a
training set of target signals (the solutions of a given inverse problem), one
can search in the latent space of the trained generator for the latent code
yielding a solution aligns with the given measurements. The pre-trained
generator plays the role of an effective regularizer for generating plausible
target signals. As discussed earlier, the key advantage of this approach is
that the measurements are not used in the training phase. As a result, once
the DGM is trained, it can be used for solving any inverse problems.

However, the choice of DGM is of paramount importance to provide stable
training, high-quality sample generation, and an effective regularizer for
solving ill-posed inverse problems. While modern GANs with the various
innovations in architectures and training protocols exhibit high-quality
samples, they suffer from training instability [53, 54] and have shown poor
reconstructions when used as a prior for solving ill-posed inverse problems
[55, 35]. Normalizing flows alleviates many drawbacks of GAN; they are stable
in training and can produce high-quality samples. Normalizing flows comprise a
set of bijective transformations which have tractable inverse and $\log\det$
Jacobian computations. They give access to the likelihood of the generated
samples and can be trained based on maximum likelihood. Normalizing flows as a
prior were shown to be more effective than GANs for solving inverse problems
[32, 33]. However unlike GANs, normalizing flows are bijective mappings,
having the same dimension in the latent space and data space, consequently,
the network output is not constraint and leading to poor regularization for
solving ill-posed inverse problems [14].

More recently, the authors of [35] showed that injective normalizing flows are
so effective for solving ill-posed inverse problems. Injective normalizing
flows are an extension of normalizing flows which have a low-dimensional
latent space. Injective flows provide a low-dimensional representation of the
data (like GANs) which perform as a strong regularization for solving inverse
problems while giving access to the likelihood of the generated samples (like
normalizing flows) which can be seen as the second regularization. In the next
section, we briefly review injective flows called Trumpets.

##  IV Injective Normalizing Flows

Figure 2: Injective normalizing flows [35] comprise two submodules, a low-
dimensional bijective flow $h_{\eta}$ and an injective network with expansive
layers $g_{\gamma}$. The MOG initialization is the mean of the Gaussian in the
latent space $z_{init}=\mu_{z}$ (red circle).

Injective normalizing flows [35] map a low-dimensional latent space to the
high-dimensional data space using a set of invertible neural networks.
Injective normalizing flows have a fast inverse on the range and give access
to the likelihood of the generated samples. As shown in Figure 2,
$f_{\theta}(z)=g_{\gamma}(h_{\eta}(z))$ with weights $\theta=(\gamma,\eta)$
comprises two subnetworks: an injective part (with expansive layers)
$g_{\gamma}$ that maps a low-dimensional space $\mathbb{R}^{d}$ to high-
dimensional data space $\mathbb{R}^{D}$ where $d\ll D$, and a bijective
mapping $h_{\eta}$ that keeps the dimension in low dimensional space
$\mathbb{R}^{d}$. The training of the injective normalizing flows consists of
two phases, at first, the range of the injective generator must be adjusted to
lie on the training data by optimizing over the weights of the injective
subnetwork $g_{\gamma}$, when we ensure the training data are close to the
range of the generator, in the second phase the likelihood of the pre-image of
the training data should be maximized over the weights of the bijective
subnetwork $h_{\eta}$. Further details are explained in [35]. When the network
is trained, we can generate random samples similar to the training data

$x_{\text{gen}}=f(z_{\text{gen}})$ (9)

where $z_{\text{gen}}\sim\mathcal{N}(\mu_{z},\sigma_{z}^{2}I)$. Further
information about the universality of density and manifold approximation of
injective normalizing flows are studied in [56].

Due to having a low-dimensional latent space, injective flows provide a low-
dimensional manifold in the high-dimensional data space. When they are
trained, the manifold would contain plausible samples which can be used as an
effective regularizer for solving ill-posed inverse problems. The injective
part provides a projection operator $g_{\gamma}(g_{\gamma}^{\dagger}(x))$
which maps the data samples $x$ to the intermediate space by
$z^{\prime}=g_{\gamma}^{\dagger}(x)$ and projects them back to the data space
by $g_{\gamma}^{\dagger}(z^{\prime})$. The authors of [35] used the projection
operator to project a sample on the manifold to suppress the noise and
artifacts, as the learned manifold contains high-quality samples. In the next
section, we will present our method of solving inverse scattering problems
using injective normalizing flows.

##  V Injective Flows for Inverse Scattering

Inverse scattering is a severely ill-posed inverse problem, which means a
small perturbation in the scattered fields leads to an exponentially large
error in the contrast [7]. This makes the inversion unstable even for a small
amount of noise. As discussed in section II, inverse scattering is a nonlinear
inverse problem whose nonlinearity heavily relies on the maximum contrast
value. For objects with large contrasts, the problem becomes highly nonlinear,
which makes the inversion extremely difficult. In such a scenario, the
significance of having a strong regularizer to restrict the search domain is
crucially important.

We consider the contrast signal $\chi=x\in\mathcal{X}$ and the scattered
fields $E^{s}=y\in\mathcal{Y}$, described by forward operator (7), as random
vectors. While our framework admits other distributions beyond Gaussian for
the additive noise $\delta$ in (7), in this paper we assume that $\delta$ is a
random vector with Gaussian distribution
$\delta\sim\mathcal{N}(0,{\sigma}^{2}I)$ yielding

$Y|X\sim\mathcal{N}(A(X),{\sigma}^{2}I).$ (10)

One effective approach for solving ill-posed inverse problems is computing MAP
estimate, where we look for the solution $x$ that has the highest posterior
likelihood for a given measurement $y$,

$\begin{array}[]{rcl}x_{\text{MAP}}=\operatorname*{arg\,max}_{x}~{}\log(p_{X|Y}(x|y)),\end{array}$
(11)

where $p_{X|Y}(x|y)$ is the posterior distribution for the given measurement
$y$. Using Bayes theorem yields,

$\begin{array}[]{rcl}x_{\text{MAP}}&=&\operatorname*{arg\,min}_{x}~{}-\displaystyle\log(p_{X|Y}(x|y))\\\
&=&\operatorname*{arg\,min}_{x}~{}-\log(\dfrac{p_{Y|X}(y|x)p_{X}(x)}{p_{Y}(y)})\\\
&=&\operatorname*{arg\,min}_{x}~{}-\log(p_{Y|X}(y|x))-\log(p_{X}(x)).\end{array}$
(12)

The first term can be obtained from (10),

$\begin{array}[]{rcl}x_{\text{MAP}}=\operatorname*{arg\,min}_{x}~{}\dfrac{1}{2}\|y-A(x)\|_{2}^{2}-\lambda\log(p_{X}(x)),\end{array}$
(13)

where the first term is the data-consistency loss while $\log(p_{X}(x))$ is
the prior distribution of the contrast and plays the role of the
regularization. We also have $\lambda$ which is a hyperparameter to adjust the
amount of regularization term (although it comes from the noise standard
deviation which we assume is unknown). In principle, the prior distribution
$p_{X}(x)$ is not available and must be estimated. For example, traditionally
$p_{X}(x)$ is approximated with a Gaussian distribution with zero mean leading
to the Tikhonov regularization. However, a Gaussian distribution is often too
far from the true prior distribution and leads to poor reconstructions.

In this paper, we study a new regularization family for inverse scattering
based on deep generative models. We consider a training set of contrast
signals $\\{x^{(i)}\\}_{i=1}^{N}$ is available, and we train a deep generative
model $x=f(z)$ to produce high-quality contrast samples. we expect the trained
generator $f$ to produce high-quality contrast samples when it takes random
samples from the Gaussian distribution in the latent space $z\in\mathcal{Z}$.
As discussed in section III, this property of deep generative models makes
them an effective regularizer for solving inverse problems [28].

We use injective normalizing flows as the generator of the contrast signal
since they are well-suited for solving ill-posed inverse problems [35]. We
perform an optimization in the latent space to find the latent code that
aligns with scattered fields $y$,

$z_{\text{MAP}}=\operatorname*{arg\,max}_{z}~{}\dfrac{1}{2}\|y-A(f(z))\|_{2}^{2}+\lambda\log(p_{X}(f(z))),$
(14)

Where an approximation to $p_{X}$ is also provided by the injective flows and
acts as the second regularizer. The reconstructed contrast signal can be
obtained as $x_{\text{MAP}}=f(z_{\text{MAP}})$. We call this method latent
space optimization (LSO). It is worth mentioning that (14) has been previously
proposed by [32, 33] for solving compressed sensing inverse problems using
regular normalizing flows.

Unlike the supervised learning methods for inverse scattering [15, 16, 17, 13]
which use a paired training set of contrast and scattered fields
$\\{(x^{(i)},y^{(i)})\\}_{i=1}^{N}$, our framework benefits from an
unsupervised learning paradigm where scattered fields are not used in the
training of injective flows. This means, if the distribution of the scattered
fields changes (due to the change in experimental configuration), the
generative network is not required to be retrained unlike the supervised
methods; as soon as the injective generator is trained over the contrast
signals, we can optimize (14) for each new scattered fields to reconstruct the
corresponding contrast. Apart from the advantages of an unsupervised learning
paradigm, the proposed method fully exploits the underlying physics of the
scattering problem by optimizing over the complex-valued scattered fields in
(14). Kothari et al. [57] have shown incorporating wave physics in the neural
network architecture can significantly improve the quality of reconstructions,
especially for out-of-distribution data.

We solve the optimization problem (14) by Adam optimizer [58] and compute the
gradients with respect to $z$ by automatic differentiation provided by
TensorFlow [59] in Python. We also use an alternative method for Equation (14)
proposed by [35] which performs the optimization in the data space,

$x_{\text{MAP}}=\operatorname*{arg\,min}_{x}~{}\dfrac{1}{2}\|y-A(g(g^{\dagger}(x)))\|_{2}^{2}-\lambda\log(p_{X}(x))$
(15)

where $g(g^{\dagger}(x))$ is the projection operator explained in section IV.
We call this method data space optimization (DSO). This method is shown in
[35] to be effective for solving linear inverse problems. While in each of the
LSO iterations the reconstructed point $x=f(z)$ is always on the learned
manifold, this is not the case for the DSO method, where the reconstructed
image might be off the manifold. Moreover, the DSO method requires more
computations than LSO as we need to take derivatives over the reverse
direction of the injective subnetwork $g^{\dagger}(x)$.

Figure 3: Performance analysis of back-propagation (BP) and Born approximation
(BA) methods for different $\epsilon_{r}$; while BA and BP reconstructions are
visually meaningful for small $\epsilon_{r}$, their performance sharply drop
for objects with large $\epsilon_{r}$. Figure 4: Illustration of the MOG
initializer in the data space $f(\mu_{z})$ for random ellipses and MNIST
datasets Figure 5: Performance evaluation of the trained injective normalizing
flows for random ellipses dataset; ground truth contrasts, their projections
on the learned manifold and some randomly generated samples.

The choice of initial guess is crucially important in inverse scattering
solvers. While a poor initialization misleads the optimization process, a good
initial step helps the algorithm to converge more accurately and faster. The
authors of [9] used Born approximation as the initialization for the distorted
Born iterative method (DBIM). A back-propagation (BP) solution was also used
in [10, 60] as an initial guess of the contrast source inversion (CSI) method.
Figure 3 shows the ground truth, back-propagation (BP) and Born approximations
(BA) for an object with different maximum $\epsilon_{r}$ values. While BP and
BA can present fuzzy reconstructions for objects with small permittivity,
their performance sharply drops for large $\epsilon_{r}$ (especially
numerically) which makes them a poor initialization for strong scatterers.

In order to circumvent this issue, we use a data-driven initialization
suggested in [32]; mean of the Gaussian distribution (MOG) in the latent space
as shown in Figure 2. The MOG initializer $z_{\text{init}}=\mu_{z}$ provides a
fixed initialization with respect to the measurements (scattered fields);
thereby being independent of the maximum contrast value and the problem
configuration. This property helps (14) and (15) to converge better even for a
large $\epsilon_{r}$. In section VI, we will show that MOG initialization
significantly improves the quality of the reconstructions compared to BP. To
our best knowledge, this is the first time that we propose a data-driven
initializer for inverse scattering.

##  VI Computational Experiments

Figure 6: Performance comparison of different methods over random ellipses
dataset in resolution $64\times 64$ Figure 7: Performance comparison of
different methods over MNIST dataset in resolution $32\times 32$

In the following, we will evaluate the performance of DSO and LSO for inverse
scattering. We consider the MOG and BP initializations for DSO while using
just MOG initialization for LSO. We compare the performance of our proposed
methods with a traditional iterative method DBIM [9]. While our method is
based on an unsupervised-learning paradigm; scattered fields are not used
during training, we also compare its performance with a successful supervised
learning method, U-Net [36] which has shown great success for inverse
scattering [16], to show the effectivity of our method. U-Net takes the back-
propagation (BP) as input and returns the permittivity pattern of the object
in the output.

We use MNIST [61] with 60000 training samples in the resolution $N=32$. We
also use a custom dataset of 60000 training samples with resolution $N=64$ of
overlapping ellipses used in [14] to have a more challenging task for accurate
reconstructions.

We use $N_{i}=12$ incident plane waves and $N_{r}=12$ receivers, uniformly
distributed on a circle with radius $R=20$ cm around the object with maximum
permittivity $\epsilon_{r}$ and dimension $D=20$ cm. The working frequency is
3 GHz and we added 30 dB noise to the scattered fields.

We used the injective normalizing flows with the same architecture described
in [35]. We trained the injective subnetwork $g_{\gamma}$ for 150 epochs to
ensure the training samples (contrast signals) are close to the range of the
generator. Figure 5 shows some test samples and their projections on the
learned manifold. Then we trained the bijective subnetwork $h_{\eta}$ for 150
epochs to maximize the likelihood of the pre-image of the training samples in
the intermediate space. Figure 5 illustrates some randomly generated samples
which confirms that the model can produce plausible samples to be used as an
effective prior for solving inverse scattering.

We optimize (14) and (15) by using Adam optimizer with a learning rate of 0.05
for 300 iterations. We use $\lambda=0.01$ for BP and $\lambda=0$ for MOG
initialization. It is worth mentioning that when we use MOG initializer, we
start from high-likelihood regions (mean of the Gaussian) which can be viewed
as a hidden regularizer, we thus use $\lambda=0$ in this case. Figure 4 shows
the MOG initialization for MNIST and random ellipses datasets.

TABLE I: Performance of different methods for solving inverse scattering ($\epsilon_{r}=4$) averaged over 25 test samples | PSNR | SSIM
---|---|---
| MNIST | Ellipses | MNIST | Ellipses
BP | 7.75 | 7.00 | 0.01 | 0.01
DBIM [9] | 5.77 | 4.67 | 0.01 | 0.01
U-Net [36] | 24.26 | 21.94 | 0.90 | 0.82
DSO (BP) | 8.73 | 7.89 | 0.16 | 0.16
DSO (MOG) | 21.50 | 14.56 | 0.56 | 0.44
LSO (MOG) | 25.22 | 20.50 | 0.89 | 0.85

Figures 6 and 7 show the performance of different methods for inverse
scattering for $\epsilon_{r}=4$ over five test samples from random ellipses
and MNIST datasets. While the traditional iterative method (DBIM) meets with
complete failure in this challenging task ($\epsilon_{r}=4$ and 30 dB noise),
DSO and LSO have strikingly more accurate reconstructions, which clearly shows
the significance of a data-driven regularization. Moreover, the MOG
initialization, as we expected, exhibits remarkably better reconstructions
compared to BP for this high permittivity ($\epsilon_{r}=4$). Furthermore,
both figures show that LSO outperforms DSO; running optimization in the latent
space results in better reconstructions as discussed in section V. Finally,
while LSO does not use scattered fields in the training phase, it could
present the reconstructions with comparable quality (or even better) to the
highly successful supervised method U-Net. Table I shows the numerical results
in PSNR and SSIM averaged over 25 test samples.

As discussed in section V, the maximum $\epsilon_{r}$ of the object plays a
significant role in the performance of inverse scattering solvers; objects
with large $\epsilon_{r}$ are more difficult to be reconstructed. Figure 8
demonstrates the performance of different methods per $\epsilon_{r}$ over the
MNIST dataset. This figure discovers that LSO with MOG initialization is
effective even for objects with large, $\epsilon_{r}$ which clearly signifies
the power of a data-driven initialization and performing the optimization in
the latent space.

Figure 8: The performance of different methods for different $\epsilon_{r}$
over MNIST dataset; SSIM is computed over the normalized signals between $-1$
and $1$.

##  VII Limitations and Conclusions

We proposed a learning-based framework for inverse scattering using an
injective prior. The proposed method fully exploits the physical domain of the
scattering problem while benefiting from a data-driven initialization which
makes a powerful solver even for objects with a large contrast. The invertible
generator admits performing optimization in both latent and data space and
uses a data-driven or back-projection as the initializer. We showed that
performing optimization in the latent space and using the mean of the Gaussian
as the initial guess significantly outperforms the traditional iterative
methods and even gives reconstructions comparable to the successful supervised
learning method, U-Net.

_Limitations:_ The proposed framework has several limitations and entails
discussion. It requires running an iterative method in test-time, which is
slow and cannot be used for real-time applications. To speed up the
convergence, one may consider a more accurate initial guess by exploiting the
physical domain in the data-driven initializer; a combination of traditional
back-projection (like BP) and the data-driven initializers (like MOG).
Recently, Hussein et all. [31] optimized the generator weights with a small
rate after finding the optimal latent code in Equation (14) to further improve
the reconstructions. This idea might be taken in our framework to go beyond
the quality of the generator, but we leave it for future works.

## References

  * [1] N. K. Nikolova, “Microwave imaging for breast cancer,” _IEEE microwave magazine_ , vol. 12, no. 7, pp. 78–94, 2011.
  * [2] A. Friedman, “Application of inverse scattering to oil field evaluation problems,” in _Mathematics in Industrial Problems_. Springer, 1998, pp. 169–178.
  * [3] R. Zoughi, _Microwave non-destructive testing and evaluation principles_. Springer Science & Business Media, 2000, vol. 4.
  * [4] V. Khorashadi-Zadeh and M. Dehmollaian, “Through a cinder block wall refocusing using sar back projection method,” _IEEE Transactions on Antennas and Propagation_ , vol. 67, no. 2, pp. 1212–1222, 2018.
  * [5] Y.-Q. Jin, _Electromagnetic scattering modelling for quantitative remote sensing_. World Scientific, 1993.
  * [6] A. I. Nachman, “Global uniqueness for a two-dimensional inverse boundary value problem,” _Annals of Mathematics_ , pp. 71–96, 1996.
  * [7] X. Chen, _Computational methods for electromagnetic inverse scattering_. John Wiley & Sons, 2018.
  * [8] Y. Wang and W. C. Chew, “An iterative solution of the two-dimensional electromagnetic inverse scattering problem,” _International Journal of Imaging Systems and Technology_ , vol. 1, no. 1, pp. 100–108, 1989.
  * [9] W. C. Chew and Y.-M. Wang, “Reconstruction of two-dimensional permittivity distribution using the distorted born iterative method,” _IEEE transactions on medical imaging_ , vol. 9, no. 2, pp. 218–225, 1990.
  * [10] P. M. Van Den Berg and R. E. Kleinman, “A contrast source inversion method,” _Inverse problems_ , vol. 13, no. 6, p. 1607, 1997.
  * [11] X. Chen, “Subspace-based optimization method for solving inverse-scattering problems,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 48, no. 1, pp. 42–49, 2009.
  * [12] Y. Khoo and L. Ying, “Switchnet: a neural network model for forward and inverse scattering problems,” _SIAM Journal on Scientific Computing_ , vol. 41, no. 5, pp. A3182–A3201, 2019.
  * [13] P. Ran, Y. Qin, and D. Lesselier, “Electromagnetic imaging of a dielectric micro-structure via convolutional neural networks,” in _2019 27th European Signal Processing Conference (EUSIPCO)_. IEEE, 2019, pp. 1–5.
  * [14] A. Khorashadizadeh, K. Kothari, L. Salsi, A. A. Harandi, M. de Hoop, and I. Dokmanić, “Conditional Injective Flows for Bayesian Imaging,” _arXiv preprint arXiv:2204.07664_ , 2022.
  * [15] L. Li, L. G. Wang, F. L. Teixeira, C. Liu, A. Nehorai, and T. J. Cui, “DeepNIS: Deep neural network for nonlinear electromagnetic inverse scattering,” _IEEE Transactions on Antennas and Propagation_ , vol. 67, no. 3, pp. 1819–1825, 2018.
  * [16] Z. Wei and X. Chen, “Deep-learning schemes for full-wave nonlinear inverse scattering problems,” _IEEE Transactions on Geoscience and Remote Sensing_ , vol. 57, no. 4, pp. 1849–1860, 2018.
  * [17] J. E. Fajardo, J. Galván, F. Vericat, C. M. Carlevaro, and R. M. Irastorza, “Phaseless microwave imaging of dielectric cylinders: An artificial neural networks-based approach,” _arXiv preprint arXiv:1908.10424_ , 2019.
  * [18] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” _arXiv preprint arXiv:1706.06083_ , 2017.
  * [19] V. Antun, F. Renna, C. Poon, B. Adcock, and A. C. Hansen, “On instabilities of deep learning in image reconstruction and the potential costs of AI,” _Proceedings of the National Academy of Sciences_ , vol. 117, no. 48, pp. 30 088–30 095, 2020.
  * [20] X. Chen, Z. Wei, M. Li, and P. Rocca, “A review of deep learning approaches for inverse scattering problems (invited review),” _Progress In Electromagnetics Research_ , vol. 167, pp. 67–81, 2020.
  * [21] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” _Advances in neural information processing systems_ , vol. 27, 2014.
  * [22] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” _arXiv preprint arXiv:1511.06434_ , 2015.
  * [23] D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” _arXiv preprint arXiv:1312.6114_ , 2013.
  * [24] L. Dinh, D. Krueger, and Y. Bengio, “Nice: Non-linear independent components estimation,” _arXiv preprint arXiv:1410.8516_ , 2014.
  * [25] L. Dinh, J. Sohl-Dickstein, and S. Bengio, “Density estimation using real nvp,” _arXiv preprint arXiv:1605.08803_ , 2016.
  * [26] D. P. Kingma and P. Dhariwal, “Glow: Generative flow with invertible $1\times 1$ convolutions,” _Advances in neural information processing systems_ , vol. 31, 2018.
  * [27] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 6840–6851, 2020.
  * [28] A. Bora, A. Jalal, E. Price, and A. G. Dimakis, “Compressed sensing using generative models,” in _International Conference on Machine Learning_. PMLR, 2017, pp. 537–546.
  * [29] V. A. Kelkar and M. Anastasio, “Prior image-constrained reconstruction using style-based generative models,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 5367–5377.
  * [30] T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 4401–4410.
  * [31] S. A. Hussein, T. Tirer, and R. Giryes, “Image-adaptive gan based reconstruction,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 04, 2020, pp. 3121–3129.
  * [32] M. Asim, M. Daniels, O. Leong, A. Ahmed, and P. Hand, “Invertible generative models for inverse problems: mitigating representation error and dataset bias,” in _International Conference on Machine Learning_. PMLR, 2020, pp. 399–409.
  * [33] J. Whang, Q. Lei, and A. Dimakis, “Compressed sensing with invertible generative models and dependent noise,” in _NeurIPS 2020 Workshop on Deep Learning and Inverse Problems_ , 2020.
  * [34] J. Brehmer and K. Cranmer, “Flows for simultaneous manifold learning and density estimation,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 442–453, 2020.
  * [35] K. Kothari, A. Khorashadizadeh, M. de Hoop, and I. Dokmanić, “Trumpets: Injective flows for inference and inverse problems,” in _Uncertainty in Artificial Intelligence_. PMLR, 2021, pp. 1269–1278.
  * [36] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in _International Conference on Medical image computing and computer-assisted intervention_. Springer, 2015, pp. 234–241.
  * [37] P. Y. Chen, D. J. Bergman, and Y. Sivan, “Spectral decomposition of the lippmann-schwinger equation applied to cylinders,” _arXiv preprint arXiv:1705.01747_ , 2017.
  * [38] S. R. Rengarajan and Y. Rahmat-Samii, “The field equivalence principle: Illustration of the establishment of the non-intuitive null fields,” _IEEE Antennas and Propagation Magazine_ , vol. 42, no. 4, pp. 122–128, 2000\.
  * [39] H. Levine and J. Schwinger, “On the theory of electromagnetic wave diffraction by an aperture in an infinite plane conducting screen,” _Communications on Pure and Applied Mathematics_ , vol. 3, no. 4, pp. 355–391, 1950.
  * [40] K. H. Jin, M. T. McCann, E. Froustey, and M. Unser, “Deep convolutional neural network for inverse problems in imaging,” _IEEE Transactions on Image Processing_ , vol. 26, no. 9, pp. 4509–4522, 2017.
  * [41] C. M. Hyun, H. P. Kim, S. M. Lee, S. Lee, and J. K. Seo, “Deep learning for undersampled mri reconstruction,” _Physics in Medicine & Biology_, vol. 63, no. 13, p. 135007, 2018.
  * [42] N. Davoudi, X. L. Deán-Ben, and D. Razansky, “Deep learning optoacoustic tomography with sparse data,” _Nature Machine Intelligence_ , vol. 1, no. 10, pp. 453–460, 2019.
  * [43] T. Liu, A. Chaman, D. Belius, and I. Dokmanic, “Learning multiscale convolutional dictionaries for image reconstruction,” _IEEE Transactions on Computational Imaging_ , 2022.
  * [44] A. Khorashadizadeh, A. Aghababaei, T. Vlašić, H. Nguyen, and I. Dokmanić, “Deep variational inverse scattering,” _arXiv preprint arXiv:2212.04309_ , 2022.
  * [45] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2223–2232.
  * [46] Q. Liu, J. Xu, R. Jiang, and W. H. Wong, “Density estimation using deep generative neural networks,” _Proceedings of the National Academy of Sciences_ , vol. 118, no. 15, 2021.
  * [47] D. Rezende and S. Mohamed, “Variational inference with normalizing flows,” in _International conference on machine learning_. PMLR, 2015, pp. 1530–1538.
  * [48] G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan, “Normalizing flows for probabilistic modeling and inference,” _Journal of Machine Learning Research_ , vol. 22, no. 57, pp. 1–64, 2021.
  * [49] G. Ongie, A. Jalal, C. A. Metzler, R. G. Baraniuk, A. G. Dimakis, and R. Willett, “Deep learning techniques for inverse problems in imaging,” _IEEE Journal on Selected Areas in Information Theory_ , vol. 1, no. 1, pp. 39–56, 2020.
  * [50] B. Kawar, M. Elad, S. Ermon, and J. Song, “Denoising diffusion restoration models,” _arXiv preprint arXiv:2201.11793_ , 2022.
  * [51] T. Vlašić, H. Nguyen, A. Khorashadizadeh, and I. Dokmanić, “Implicit neural representation for mesh-free inverse obstacle scattering,” _arXiv preprint arXiv:2206.02027_ , 2022.
  * [52] A. Khorashadizadeh, A. Chaman, V. Debarnot, and I. Dokmanić, “Funknn: Neural interpolation for functional generation,” _arXiv preprint arXiv:2212.14042_ , 2022.
  * [53] H. Thanh-Tung and T. Tran, “Catastrophic forgetting and mode collapse in gans,” in _2020 International Joint Conference on Neural Networks (IJCNN)_. IEEE, 2020, pp. 1–10.
  * [54] M. Arjovsky and L. Bottou, “Towards principled methods for training generative adversarial networks,” _arXiv preprint arXiv:1701.04862_ , 2017.
  * [55] J. Whang, Q. Lei, and A. Dimakis, “Solving inverse problems with a flow-based noise model,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 11 146–11 157.
  * [56] M. Puthawala, M. Lassas, I. Dokmanić, and M. de Hoop, “Universal joint approximation of manifolds and densities by simple injective flows,” _arXiv preprint arXiv:2110.04227_ , 2021.
  * [57] K. Kothari, M. de Hoop, and I. Dokmanić, “Learning the geometry of wave-based imaging,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 8318–8329, 2020.
  * [58] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [59] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard _et al._ , “Tensorflow: A system for large-scale machine learning,” in _12th $\\{$USENIX$\\}$ symposium on operating systems design and implementation ($\\{$OSDI$\\}$ 16)_, 2016, pp. 265–283.
  * [60] P. M. van den Berg, A. Van Broekhoven, and A. Abubakar, “Extended contrast source inversion,” _Inverse problems_ , vol. 15, no. 5, p. 1325, 1999.
  * [61] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” _Proceedings of the IEEE_ , vol. 86, no. 11, pp. 2278–2324, 1998.