# Galaxy Distribution Incompleteness Testing Using Self-Organizing Maps

Isaac McMahon  McWilliams Center for Cosmology, Department of Physics,
Carnegie Mellon University, Pittsburgh, PA, 15213  Markus Michael Rau
McWilliams Center for Cosmology, Department of Physics, Carnegie Mellon
University, Pittsburgh, PA, 15213  High Energy Physics Division, Argonne
National Laboratory, Lemont, IL 60439  Rachel Mandelbaum  McWilliams Center
for Cosmology, Department of Physics, Carnegie Mellon University, Pittsburgh,
PA, 15213

##  1 Abstract

The calibration of redshift distributions for photometric samples using
spectroscopic surveys is plagued by the difficulty in modelling the selection
functions of spectroscopic surveys. In this work, we analyse how these
selection functions impact redshift inference and quantify the induced biases
using local calibration tests in photometry space. The study is carried out
using simulations that mimic the radial selection function of a spectroscopic
survey and an accompanying mock catalog of a photometric galaxy survey
catalog. We use a self-organizing map to partition the photometry space and
perform a local $\chi^{2}$ test to study the probability calibration of
redshift inferences that use the spectroscopic data for calibration. The goal
of this work is to investigate the effect of uncorrected selection functions
in the calibration data on redshift prediction accuracy and critically discuss
mitigation methods. In particular we test culling-based bias correction
techniques, which aim to remove redshift calibration biases by identifying
regions in photometry with few spectroscopic calibration data, and propose
avenues for future research. We found that removing regions in color-magnitude
space that are underpopulated with spectroscopic calibration data does not
remove all biases in redshift inference induced by the selection function.

##  2 Introduction

With the advent of precision cosmology driven by imaging surveys like the Dark
Energy Survey (DES; e.g., Abbott et al., 2018), the Kilo-Degree Survey (KiDS;
e.g., Hildebrandt et al., 2017), the Hyper Suprime-Cam survey (HSC; e.g.,
Aihara et al., 2018), the Rubin Observatory Legacy Survey of Space and Time
(LSST; e.g., Ivezić et al., 2019), the Roman Space Telescope High Latitude
Imaging Survey (HLIS; e.g. Spergel et al., 2015) and Euclid (e.g. Laureijs et
al., 2011), the accurate estimation and calibration of the galaxies’ redshifts
using photometric information becomes increasingly important. In particular,
while measures of structure growth such as weak gravitational lensing do not
require precise estimates of redshift for each galaxy, they do require
accurate estimation of the redshift distributions of galaxy samples
(Kilbinger, 2015; Mandelbaum, 2018).

Traditionally these photometric redshifts have been calibrated by direct
comparison with spatially overlapping spectroscopic samples. Photometric
redshifts are calibrated using either spatial information via cross-
correlation methods (e.g. Newman, 2008; Ménard et al., 2013; McQuinn & White,
2013; Scottez et al., 2016; Raccanelli et al., 2017; Morrison et al., 2017;
Davis et al., 2017; Gatti et al., 2018; van den Busch et al., 2020;
Hildebrandt et al., 2021), by direct comparison with spatially overlapping
spectroscopic samples (e.g. Coupon et al., 2009; Hildebrandt et al., 2010;
Dahlen et al., 2013; Tanaka et al., 2018; Bonnett et al., 2016) or by
comparison/inference using narrow-band photometric data (see e.g. Buchs et
al., 2019). The reliance on spectroscopic calibration data in both prediction
and calibration has been called into question (e.g. Bonnett et al., 2016;
Schmidt et al., 2020; Hartley et al., 2020); however, spectroscopic
observations are still crucial for the calibration of the color-redshift
relation for current and upcoming photometric survey programs (see e.g. Saglia
et al., 2022). Spectroscopic surveys have complex selection functions that
result in different magnitude-redshift mappings in comparison with photometric
surveys. While it is possible to identify and correct for selection functions
in magnitude space, this might not be the case for pure line-of-sight
selection functions111See Bonnett et al. (2016) Appendix D for an extreme
case.. If unaccounted for, both types of selection function can bias ensemble
redshift estimation and validation using spectroscopic samples. As shown in
Schmidt et al. (2020), Zhao et al. (2021), and Dey et al. (2022), performing
local tests for redshift calibration in magnitude space is essential, as
selection function-induced redshift calibration biases can ‘cancel out’ in
magnitude space.

In this work we study this effect using simulations of spectroscopic
incompleteness by Hartley et al. (2020). We add to Hartley et al. (2020) a
statistical significance test for incompleteness localized in magnitude space,
and a global test that combines the local tests to obtain a single $p$-value.
We also test the ability to correct for selection effects in redshift by
excluding parts of the magnitude space, similar to the approach of Masters et
al. (2015). To this end, we partition the magnitude space of the mock catalog
of the photometric survey photometry using a self-organizing map (SOM). We
then predict photometric redshift distributions of individual galaxies using
the simulated spectroscopic calibration dataset. After populating the SOM with
these predictions, we apply a local $\chi^{2}$ test and consider several
calibration scenarios to investigate the aforementioned effects. The research
note concludes with a discussion of areas for future work.

##  3 Dataset

Hartley et al. (2020) simulates a realistic selection function of a
spectroscopic survey to investigate the impact of spectroscopic incompleteness
on Dark Energy Survey science. The mock catalogs consist of a simulated
photometric sample that mimicks DES Y1 survey data (Hoyle et al., 2018; Abbott
et al., 2019), based on the Buzzard simulations (DeRose et al., 2019). The
spectroscopic mock catalog mimicks the four spectroscopic datasets VVDS-Deep,
VVDS-Wide, VIPERS and zCosmos. The spectroscopic selection functions of these
surveys are mimicked by simple color cuts and a spatial selection that matched
the real spectroscopic surveys. To mimick a realistic redshifting procedure,
Hartley et al. (2020) determine the redshifts by a combination of a cross
correlation technique and manual redshifting. The manual redshifting is
performed on a small subsample of galaxies by a set of human experts that
determine quality flags for the spectral fits. These flags are then mapped to
the full sample using a Machine Learning (ML) algorithm. For further details
on the data generation process, we refer readers to Hartley et al. (2020). The
simulated mock catalog provided to us contains spectroscopic redshifts,
subject to incompleteness, as well as the true redshifts from the simulations.
Furthermore we have the photometry in the DES filter set for all galaxies. In
the following discussion, we denote the dataset (with/without) the
spectroscopic selection function as (spectroscopic/target) samples. The
(spectroscopic/target) samples contain (73318/134155) galaxies, respectively.
The faintest magnitudes in the $i$-band in the (spectroscopic/target) samples
are (24.8/24.7), with median magnitude (21.8/23.0). The spectroscopic and
target samples have nearly identical maximum (1.98/2.00) and median
(0.58/0.58) redshifts.

##  4 Methodology

###  4.1 Photometric Redshift Estimation

We use the classification-based conditional density estimate described in Rau
et al. (2015) to estimate individual galaxy redshift distributions. The
individual galaxy redshift distribution is parametrized using a histogram
model with $M$ histogram bins.

$\hat{p}(z|\mathbf{m})=\sum_{i=1}^{M}\pi_{i}(\mathbf{m})\mathbf{1}(z\in[z_{L}^{i},z_{R}^{i}])\,,$
(1)

where $\mathbf{m}$ denotes the photometry (the magnitudes of the galaxy in a
filter set), $[z_{L}^{i},z_{R}^{i}]$ denotes the redshift bin edges of the
parametrization, and $\pi_{i}(\mathbf{m})$ denotes the magnitude-dependent
histogram heights. The term $\mathbf{1}(z\in[z_{L}^{i},z_{R}^{i}])$ is zero if
the redshift $z$ lies in the interval defined by $[z_{L}^{i},z_{R}^{i}]$ and
zero otherwise. The magnitude dependence of $\pi_{i}(\mathbf{m})$ is learned
using a Random Forest Classifier with cross-entropy loss function using a
training set with accurate redshift information. If not stated otherwise, we
will train the aforementioned Random Forest Classifier using the spectroscopic
mock simulation data and apply the model to the simulated photometric dataset.
The classification is done with respect to the redshift bins defined in Eq.
(1). The classifier used is implemented using the sklearn package (Pedregosa
et al., 2011). Here we use it in its default configuration with 100 training
bins. We define $z_{\rm phot}$ of each galaxy to be the mean of the
conditional distribution estimate derived for that galaxy. We obtain a mean
(standard deviation) of the residual $z_{\rm phot}-z_{\rm spec}$ of 0.0039
(0.14), which is consistent with the performance of comparative methods in
Sánchez et al. (2014) listed in Tables 6-8. We note that we do not use outlier
removal in the quoted metrics.

###  4.2 Selection Functions

#### Random Subsampling

In ideal random subsampling, the training and test sets are drawn from the
same parent population. The joint distribution $p(z,\mathbf{m})$ of redshift
$z$ and magnitude $\mathbf{m}$ are therefore equal between the training and
test set and we would have

$\begin{split}p_{\rm test}(z|\mathbf{m})&=p_{\rm train}(z|\mathbf{m})\\\
p_{\rm test}(\mathbf{m})&=p_{\rm train}(\mathbf{m})\,.\end{split}$ (2)

In practise this ideal setup is often not achievable in the context of
redshift estimation due to instrumental limitations and targeting strategies
of the spectroscopic survey. As a result, these selection functions often lead
to biases in parameter inference and interpretation in the context of
photometric redshift inference. In this context we identify two relevant types
of selection functions:

#### Covariate Shift

If the test set differs from the training set in the marginal distribution of
its covariates, i.e. photometry222In this note, the term ‘photometry’ can
refer to colors, magnitudes or fluxes. Our inference is based on the galaxy
magnitudes, so we use the more general term ‘photometry’ and magnitude
interchangeably in this note., but not in the conditional distribution of
redshift given photometry, i.e.

$\begin{split}p_{\rm test}(z|\mathbf{m})&=p_{\rm train}(z|\mathbf{m})\\\
p_{\rm test}(\mathbf{m})&\neq p_{\rm train}(\mathbf{m})\,,\end{split}$ (3)

we refer to this as a covariate shift scenario. For more details we refer to
the substantial statistics literature on covariate shift corrections (for a
review see, e.g., Sugiyama & Kawanabe, 2012). Spectroscopic surveys observe
the galaxy spectrum with much larger resolution than their photometric
counterparts. Thus, observing galaxies to faint magnitudes in a spectroscopic
survey requires much longer exposure times as compared with photometric
surveys. The spectroscopic sample, therefore, ‘thins out’ at faint magnitudes,
which implies a covariate shift between the training sample, that is matched
to the spectroscopic sample, and the test sample that is given by the
photometric sample.

#### Line-of-sight selection function

If the conditional distributions of redshift given the photometry of the
training and test sets differ, i.e.

$p_{\rm test}(z|\mathbf{m})\neq p_{\rm train}(z|\mathbf{m})\,,$ (4)

we are considering a line-of-sight selection effect. For example, this effect
can be introduced due to instrumental effects, like a wavelength-dependent
signal-to-noise ratio in spectrographs, which can deteriorate spectroscopic
redshift measurements if the characteristic features of the spectrum move
outside of the instrumental sensitivity window. As a result there may not be
accurate spectroscopic redshift measurements available for higher redshifts. A
line-of-sight selection is often coupled with a covariate shift. Hartley et
al. (2020) forward model both the spectroscopic target function and selection
functions induced by redshift failures. The final catalogs show both a
covariate shift and a line-of-sight selection with respect to the test sample
that mimics a ‘DES-like’ photometric survey. We refer the interested reader to
Hartley et al. (2020) for more details on these simulations.

###  4.3 Local Test of Selection Effects

One of the goals of photometric redshift validation is to measure how well the
conditional density predictions of redshift given photometry are calibrated
for the galaxies in the sample. Good calibration is to be understood as
correctly predicted frequentist coverage probability, i.e. within a given
credibility interval obtained from the $p(z|\mathbf{m})$ estimate, we would
expect the predicted number of true redshifts under data replication. In the
more traditional approach, this photometric redshift calibration is not
quantified as a function of magnitude or photometry. As shown in Schmidt et
al. (2020) this can be especially problematic, since miscalibrations across
photometry can balance out, leading to wrong conclusions about the actual
quality of photometric redshift estimates. Tests for calibration accuracy
therefore need to be performed as a function of location in magnitude space.
There exists work in the statistics literature that propose tests for this
scenario; we would like to especially highlight Zhao et al. (2021) and Dey et
al. (2022). Specifically, Dey et al. (2022) proposed methodology to calibrate
individual galaxy redshift estimates as a function of color space if a
representative spectrophotometric calibration dataset is available. We
complement this work by studying the impact of non-representative
spectrophotometric calibration data, i.e., data that is subject to a line-of-
sight selection function, on sample redshift distribution inference.

In this research note, we take a simple approach and train a self-organizing
map (SOM), a popular dimensionality reduction technique in cosmology and in
particular photometric redshift estimation (e.g. Masters et al., 2015; Wright
et al., 2020; Myles et al., 2021), on the magnitude space of the spectroscopic
and photometric samples. The SOM is a data-driven way of dividing up a sample
based on its high-dimensional distribution in magnitude space into 2D cells,
each with galaxies of similar photometry.

We train a SOM using the training set data that consists of the simulated
four-band DES photometry, where we use the MiniSOM python package (Vettigli,
2018). The SOM is initialized with random values, then trained on the four
DES-Y1 magnitudes to arrange the galaxies by similarity to each other on the
map. We estimate the conditional density estimates for each galaxy in the test
set and then populate the trained SOM using the test set galaxies and their
estimated conditional density estimates. Each SOM cell has then a
corresponding training and test set sample as well as a set of conditional
density estimates for each test set galaxy in the cell.

On each of these cells we test if the conditional distribution estimates are
well calibrated, where we define correct calibration as uniformly distributed
$\text{PIT}(z)=\int_{0}^{z}\mathrm{d}z^{{}^{\prime}}\hat{p}(z^{{}^{\prime}}|\mathbf{m})$
with respect to the true redshifts $z$. To test the quality of the
calibration, we perform a $\chi^{2}$ test between the distribution of PIT
values obtained by evaluating all estimated conditional distributions at the
correct redshift, and the uniform distribution. This test is performed for all
galaxies in each SOM cell. We obtain a $p$-value for each SOM cell that we
compare with a Bonferroni-corrected significance level of $\alpha=0.05/N_{\rm
SOM}$. The Bonferroni correction adapts the significance level for the number
of simultaneous hypotheses that are tested to counteract the increased chance
to reject the null hypothesis by random chance when carrying out multiple
comparisons. Here $N_{\rm SOM}$ is the number of populated cells and therefore
the number of tension tests we perform between the calibration data and the
estimated individual galaxy photometric redshift distributions
$\hat{p}(z|\mathbf{m})$.

###  4.4 Bias Mitigation using SOM-Cell Culling

Following Masters et al. (2015), we now remove a given fraction of SOM cells
with mean spectroscopic redshift values in the calibration sample that have a
large statistical error $\sigma_{\rm\left\langle z_{\rm
spec}\right\rangle}=\frac{\sigma_{\rm spec}}{\sqrt{N_{\rm spec,cell}}}$
starting with cells that have the largest error. Here, $\sigma_{\rm spec}$
denotes the standard deviation of true redshift values. We hold the SOM
partition and the respective data and $\hat{p}(z|\mathbf{m})$ estimates
attached to the SOM cells constant. Given sufficient flexibility of the
conditional density estimates and a proportionality between the number of
spectroscopic populated SOM cells and the severity of the selection effect,
one could expect this procedure to remove cells that would yield especially
biased predictions. We note that this is a criterion based on how well the
spectroscopic calibration data ‘populates’ the SOM. During SOM-cell culling,
‘goodness-of-fit’ criteria constructed using photometric redshift estimates
are not used.

##  5 Results

We consider three scenarios that illustrate the effect of spectroscopic
incompleteness on photometric redshift predictions:

#### Random Subsampling

We construct a disjunct randomly selected subsample by randomly splitting the
full test set in half. One part is assigned as the training set, the other as
the new test set. We now quantify the performance of the photometric redshift
estimates on the test set following the procedures described in the previous
sections. Fig. 1 top left shows the photometric redshift distributions of the
sample of galaxies estimated using the ‘stacked’ conditional photometric
redshift estimates with the distribution of true redshift. The stacked
distribution is constructed as $\hat{p}(z)=\sum_{i=1}^{N_{\rm
gal}}\hat{p}(z_{i}|\mathbf{m}_{i})$. While we note that stacking is not
appropriate in general (see e.g. Malz, 2021; Rau et al., 2021), we use it for
illustration, so the methodological error can therefore be neglected here.
Both distributions visually agree well. To quantify the agreement in magnitude
space, we perform the statistical tests described in the previous sections and
list the results in Tab. 1. We report the failure rates for the different
scenarios in the first column. The number of populated cells after culling is
shown in the second column and the failure rates (with Bonferroni corrected
$\alpha$) in the third column. The failure rate for ‘Random Subsampling’, as
listed in Tab. 1, is $3.25\%$, which means that our test detects a significant
miscalibration in only $3.25\%$ of the SOM cells, which we interpret as model
error, i.e. the intrinsic deficiencies in the model training. This failure
rate serves as the lower bound on the accuracy for the following discussion
and will be compared with other results, listed in the first column of Tab. 1,
that are additionally impacted by the line-of-sight and covariate shift
selection functions. We will consider these scenarios in the following
paragraphs.

Figure 1: Sample photometric redshift distributions obtained using the true
redshifts (orange) and by stacking conditional density estimates (blue). The
(top left/top right/bottom left/bottom right) subpanels show the scenarios of
a (Random Subsampling/Spectroscopic Training set without culling/Spectroscopic
Training set with 95% culling/Spectroscopic Training set with 80% culling).
The x-axis shows the redshift, the y-axis the number of galaxies in each
redshift bin.

#### Spectroscopic Training Set without culling

We use the simulated spectroscopic data as a training set to train a
conditional density estimator to predict conditional density estimates for the
full test set. Fig. 1 top right panel shows the corresponding photometric
redshift distributions of the stacked predicted sample redshift distribution
and the photometric redshift distribution of the sample of true redshifts. We
see that the covariate shift and line-of-sight selection effects in the
training data deteriorate the agreement between the sample photometric
redshift distribution of true redshifts and the predicted photometric sample
redshift distribution. Specifically, we see a peaked structure in the latter
that is indicative of projection effects, where galaxies are systematically
placed in the wrong redshift intervals in undercovered, non-representative
areas of magnitude space. The failure rate of SOM cells increases to $21.5\%$
(4th row Tab. 1), as compared with the baseline value for the representative
training sample of $3.25\%$ (1st row Tab. 1). This indicates that the
selection functions that are present in the training set increase the number
of cells that show significant miscalibration.

#### Spectroscopic Training Set with culling

Fig. 1 (lower left/lower right) shows the sample redshift distributions
estimated on the test sample with (95%/80%) fraction of culled SOM cells,
which corresponds to a fraction of (47%/19%) remaining galaxies. In both
cases, the peaked features persist in the sample redshift distribution
estimates, similar to the upper right panel. We can clearly distinguish these
effects from the corresponding scenario when we consider a representative
training set, also with 95% culling fraction as shown in Fig. 2. We clearly
see that the estimates that use a representative training set, while not
optimal, lack large projection peaks in the sample redshift distribution. They
can therefore be attributed to the line-of-sight and covariate shift selection
functions present in the spectroscopic data that cannot be corrected by
culling procedures. This is in agreement with the statistical analysis
reported in Tab. 1, where we see an increase in the failure rate when applying
culling from $21.5\%$ for the Spectroscopic Training Set with $0\%$ cells
culled to ($45\%$/$75\%$) for ($80\%$/$95\%$) cells culled.

We now study the impact of the culling procedure, that itself is a selection
function, on the calibration of the conditional density estimates. Applying
SOM-cell culling to the ‘Random Subsampling’ scenario, we report an increase
in the failure rates (see Tab. 1) from $3.25\%$ to ($15\%$/$40\%$) for
($80\%$/$95\%$). We interpret this increase in failure rate even for this
ideal scenario as follows: The SOM-cell culling procedure imposes a selection
function that is not present in the training sample which can lead to mild
biases in the trained model. Comparing the ‘Random Subsampling’ scenarios with
the ‘Spectroscopic Training Set’ scenarios, we note that the selection
functions in the spectroscopic training set further deteriorate the
probability calibration. This can be seen by comparing the failure rates
between the ($80\%$/$95\%$) ’Random Subsampling’ with the ($80\%$/$95\%$)
’Spectroscopic Training Set’ scenarios in Tab. 1. We can therefore conclude
that the culling procedure is not able to correct for the degradation of
redshift performance due to the aforementioned line-of-sight and covariate
shift selection functions.

Figure 2: Sample photometric redshift distributions obtained using the true redshifts (orange) and by stacking conditional density estimates (blue). Both panels show the scenario of ‘Random Subsampling’. The (left/right) panels show the results (without/95%) culling. The x-axis shows the redshift, the y-axis the number of galaxies in each redshift bin.  Sample | Cells Culled | Galaxies Remaining | Populated Cells | Failure Rate
---|---|---|---|---
| 0% | 100% | 400 | 3.25%
Random Subsampling | 80% | 50% | 80 | 15%
| 95% | 20% | 20 | 40%
| 0% | 100% | 400 | 21.5%
Spectroscopic Training Set | 80% | 47% | 80 | 45%
| 95% | 19% | 20 | 75%
Table 1: Failure rates of the various analysis scenarios discussed in the
test. The (first/second/third/fourth/fifth) columns list the analysis
scenario, percentage of SOM cells culled, the percentage of galaxies remaining
after culling, the number of populated cells in the SOM, and the rate of
galaxies that fail the $\chi^{2}$ test (Failure Rate) with the Bonferroni
correction.

##  6 Conclusions and Future Work

As discussed in this note, spectroscopic calibration data is subject to
complex selection functions that limit their usefulness as a reliable source
of photometric redshift calibration. We have shown that for broad band
photometry, culling techniques in magnitude space are not a reliable way to
mitigate these biases. This is likely due to the line-of-sight selection
functions that are not necessarily identifiable using photometry alone. We
note that this result is clearly survey specific and additional photometric
bands might change this assessment. Besides the dependency of our results on
the specific set of photometric filters, our methodology to test for
incompleteness is not directly applicable to real data, since the true
redshift values are unknown. It should be seen as a way to forecast the
performance of photometric redshift calibration methodology such as culling
using simulated data. This is not a limitation of our testing methodology in
particular, as it will similarly apply to all statistical calibration tests
for conditional probability density function estimates that utilize a
spectrophotometric calibration dataset. In future work, it will be interesting
to extend this model testing framework towards photometry space in a posterior
predictive testing framework applied to invertible ML models. This can then be
applied in conjunction with forward modelling of the galaxy population and
photometry and will be interesting for future work.

We thank Will Hartley and Chihway Chang for the use of their simulated galaxy
catalog, and Brett Andrews for his feedback.

## References

  * Abbott et al. (2018) Abbott T. M. C., et al., 2018, ApJS, 239, 18
  * Abbott et al. (2019) Abbott T. M. C., et al., 2019, Phys. Rev. D, 99, 123505
  * Aihara et al. (2018) Aihara H., et al., 2018, PASJ, 70, S4
  * Bonnett et al. (2016) Bonnett C., et al., 2016, Phys. Rev. D, 94, 042005
  * Buchs et al. (2019) Buchs R., et al., 2019, MNRAS, 489, 820
  * Coupon et al. (2009) Coupon J., et al., 2009, A&A, 500, 981
  * Dahlen et al. (2013) Dahlen T., et al., 2013, ApJ, 775, 93
  * Davis et al. (2017) Davis C., et al., 2017, arXiv e-prints, p. arXiv:1710.02517
  * DeRose et al. (2019) DeRose J., et al., 2019, arXiv e-prints, p. arXiv:1901.02401
  * Dey et al. (2022) Dey B., Zhao D., Newman J. A., Andrews B. H., Izbicki R., Lee A. B., 2022, arXiv e-prints, p. arXiv:2205.14568
  * Gatti et al. (2018) Gatti M., et al., 2018, MNRAS, 477, 1664
  * Hartley et al. (2020) Hartley W. G., et al., 2020, MNRAS, 496, 4769
  * Hildebrandt et al. (2010) Hildebrandt H., et al., 2010, A&A, 523, A31
  * Hildebrandt et al. (2017) Hildebrandt H., et al., 2017, MNRAS, 465, 1454
  * Hildebrandt et al. (2021) Hildebrandt H., et al., 2021, A&A, 647, A124
  * Hoyle et al. (2018) Hoyle B., et al., 2018, MNRAS, 478, 592
  * Ivezić et al. (2019) Ivezić Ž., et al., 2019, ApJ, 873, 111
  * Kilbinger (2015) Kilbinger M., 2015, Reports on Progress in Physics, 78, 086901
  * Laureijs et al. (2011) Laureijs R., et al., 2011, arXiv e-prints, p. arXiv:1110.3193
  * Malz (2021) Malz A. I., 2021, Phys. Rev. D, 103, 083502
  * Mandelbaum (2018) Mandelbaum R., 2018, ARA&A, 56, 393
  * Masters et al. (2015) Masters D., et al., 2015, The Astrophysical Journal, 813, 53
  * McQuinn & White (2013) McQuinn M., White M., 2013, MNRAS, 433, 2857
  * Ménard et al. (2013) Ménard B., Scranton R., Schmidt S., Morrison C., Jeong D., Budavari T., Rahman M., 2013, arXiv e-prints, p. arXiv:1303.4722
  * Morrison et al. (2017) Morrison C. B., Hildebrandt H., Schmidt S. J., Baldry I. K., Bilicki M., Choi A., Erben T., Schneider P., 2017, MNRAS, 467, 3576
  * Myles et al. (2021) Myles J., et al., 2021, MNRAS, 505, 4249
  * Newman (2008) Newman J. A., 2008, ApJ, 684, 88
  * Pedregosa et al. (2011) Pedregosa F., et al., 2011, Journal of Machine Learning Research, 12, 2825
  * Raccanelli et al. (2017) Raccanelli A., Rahman M., Kovetz E. D., 2017, MNRAS, 468, 3650
  * Rau et al. (2015) Rau M. M., Seitz S., Brimioulle F., Frank E., Friedrich O., Gruen D., Hoyle B., 2015, MNRAS, 452, 3710
  * Rau et al. (2021) Rau M. M., Morrison C. B., Schmidt S. J., Wilson S., Mandelbaum R., Mao Y. Y., 2021, arXiv e-prints, p. arXiv:2101.01184
  * Saglia et al. (2022) Saglia R., et al., 2022, arXiv e-prints, p. arXiv:2206.01620
  * Sánchez et al. (2014) Sánchez C., et al., 2014, MNRAS, 445, 1482
  * Schmidt et al. (2020) Schmidt S. J., et al., 2020, MNRAS, 499, 1587
  * Scottez et al. (2016) Scottez V., et al., 2016, MNRAS, 462, 1683
  * Spergel et al. (2015) Spergel D., et al., 2015, arXiv e-prints, p. arXiv:1503.03757
  * Sugiyama & Kawanabe (2012) Sugiyama M., Kawanabe M., 2012, Machine Learning in Non-Stationary Environments: Introduction to Covariate Shift Adaptation. The MIT Press
  * Tanaka et al. (2018) Tanaka M., et al., 2018, PASJ, 70, S9
  * Vettigli (2018) Vettigli G., 2018, MiniSom: minimalistic and NumPy-based implementation of the Self Organizing Map, https://github.com/JustGlowing/minisom/
  * Wright et al. (2020) Wright A. H., Hildebrandt H., van den Busch J. L., Heymans C., 2020, A&A, 637, A100
  * Zhao et al. (2021) Zhao D., Dalmasso N., Izbicki R., Lee A. B., 2021, arXiv e-prints, p. arXiv:2102.10473
  * van den Busch et al. (2020) van den Busch J. L., et al., 2020, A&A, 642, A200


# StarUnLink: identifying and mitigating signals from communications
satellites in stellar spectral surveys

Spencer Bialek1, Sara Lucatello2,3, Sebastien Fabbro1,4, Kwang Moo Yi5, Kim A.
Venn1

1 Department of Physics and Astronomy, University of Victoria, Victoria, BC,
V8W 3P2, Canada
2 INAF-Osservatorio Astronomico di Padova, Vicolo Osservatorio 5, 35122,
Padova, Italy
3 Institute for Advanced Studies, Technische Universität München,
Lichtenbergstraße 2 a, 85748, Garching bei München, Germany
4 National Research Council Herzberg Astronomy and Astrophysics, Victoria, BC,
Canada
5 Department of Computer Science, University of British Columbia, Vancouver,
BC, V6T 1Z4, Canada Victoria, BC, Canada
E-mail<EMAIL_ADDRESS>
(Accepted XXX. Received YYY; in original form ZZZ)

###### Abstract

A relatively new concern for the forthcoming massive spectroscopic sky surveys
is the impact of contamination from low earth orbit satellites. Several
hundred thousand of these satellites are licensed for launch in the next few
years and it has been estimated that, in some cases, up to a few percent of
spectra could be contaminated when using wide field, multi-fiber
spectrographs. In this paper, a multi-staged approach is used to assess the
practicality and limitations of identifying and minimizing the impact of
satellite contamination in a WEAVE-like stellar spectral survey. We develop a
series of convolutional-network based architectures to attempt identification,
stellar parameter and chemical abundances recovery, and source separation of
stellar spectra that we artificially contaminate with satellite (i.e. solar-
like) spectra. Our results show that we are able to flag 67% of all
contaminated sources at a precision level of 80% for low-resolution spectra
and 96% for high-resolution spectra. Additionally, we are able to remove the
contamination from the spectra and recover the clean spectra with a $<$1%
reconstruction error. The errors in stellar parameter predictions reduce by up
to a factor of 2-3 when either including contamination as an augmentation to a
training set or by removing the contamination from the spectra, with overall
better performance in the former case. The presented methods illustrate
several machine learning mitigation strategies that can be implemented to
improve stellar parameters for contaminated spectra in the WEAVE stellar
spectroscopic survey and others like it.

###### keywords:

methods: data analysis, techniques: spectroscopic, surveys

††pubyear: 2023††pagerange: StarUnLink: identifying and mitigating signals
from communications satellites in stellar spectral surveys–A

##  1 Introduction

The number of satellites in low Earth orbit (LEO hereafter) has dramatically
expanded since the turn of the century, increasing by at least an order of
magnitude in the last decade. Commercial enterprises have become major players
in this context, aiming at building infrastructure for global low-latency
broadband Internet and other telecommunications needs. Several tens of
thousands of LEO satellites from multiple companies have been licensed for
launch by the end of the decade, and several hundred thousands more have been
announced.

Besides potential implications for debris, there is a growing concern of LEO
satellites impacting our observations of astronomical sources in the night
sky. A recent study by Lawler et al. (2022) modeled the orbits of 65,000
artificial satellites (e.g. Starlink, OneWeb, Kuiper, and StarNet/GW) soon to
be inhabiting our night sky and found that, at certain points throughout the
night, there can be several hundred (or $\sim$7% of the total visible point
sources in the sky at $g<7$ within a couple of hours of sunrise or sunset)
satellites bright enough to interfere with astronomical observations. The
impact of LEO satellite on astronomy has been studied by several working
groups supported by the National Science Foundation, American Astronomical
Society, Canadian Astronomical Society, and the European Astronomical Society.
These studies have led to the SATCON1 and SATCON2 reports (Walker & Hall,
2020, 2021), the Report on Mega-Constellations to the Government of Canada and
the Canadian Space Agency (Boley et al., 2021), and the Dark and Quiet Skies
for Science and Society reports111The reports can be found at this website
(Walker et al., 2020, 2021)

In the SATCON1 report, the impacts on the data collected at large
spectroscopic facilities were noted to be larger than anticipated due to
several factors: the size of the primary mirrors, the long integration times,
and the brightness of a satellite when observing faint targets. In particular,
the contamination from a bright satellite would likely go unnoticed until
after the spectra are recorded, thus can only be detected in post-processing.
This could be a significant problem for the forthcoming sky surveys.

Here we will focus on spectroscopic surveys that will make use of upcoming and
planned wide-field, high multiplexing spectrographs, because of their
requirements for extreme precision and accuracy in understanding the dynamics
and formation history of the Milky Way. In particular we will assess the
impacts of LEO satellites on the European WEAVE Survey (Dalton et al., 2018),
with implications that carry over to 4MOST (de Jong et al., 2019) (both are on
4m telescopes and are similarly impacted due to having similar characteristics
such as diameter, average fiber density on the field-of-view, and spectral
range/resolution) and to a lesser extent e.g. the 8.2m Subaru Telescope’s
Prime Focus Spectrograph (Tamura et al., 2018), and the 11.25m Maunakea
Spectroscopic Explorer (e.g., McConnachie et al., 2016).

The expected impact on spectroscopic observations is difficult to quantify,
but Hainaut & Williams (2020) estimate that, for wide field, multi-fiber
spectrographs, a few percent of spectra collected during the first and last
hours of the night will be impacted by LEO satellite contamination, while
Bassa et al. (2022) estimate that up to 0.8% of 4MOST spectra will be affected
– though we note these estimates were based on several tens of thousands of
LEO satellites which, as previously mentioned, could be an underestimate for
when 4MOST and WEAVE are collecting most of their data, and thus are likely
lower limits. These surveys are designed to observe thousands of spectra per
setting over a large field of view, with typically 10+ settings per night.
Over the lifetime of these surveys ($\sim$10 years), then 0.8% per setting x
1000 stars per setting x 10 settings per night x 300 nights per year x 10
years = 240,000 contaminated spectra (out of 30 million spectra in the total
survey).

Because satellites will be abundant, physical mitigation strategies such as
avoiding observations cannot be an option – they will sacrifice too much in
terms of observing time (Hu et al., 2022). Thus, alternative strategies are
necessary, which is the aim of this paper. As the light from a bright
satellite would primarily be reflected sunlight, then it is easy to simulate a
contaminated observation of a stellar target by modelling it as an additive
mix of stellar and solar-like spectra. If there are solar spectra of varying
S/N observed with the same instrument collecting stellar spectra, it is
possible to create a large data set of artificially contaminated spectra. One
can therefore attempt post-processing algorithms to identify which features of
the spectra correspond to either solar or stellar light, and thereafter used
for contamination mitigation strategies.

In this paper, we study the practicality of detecting and mitigating satellite
contamination using a convolutional neural network and U-Net on post-processed
spectra. In Section 2, we clarify how we use the Gaia-ESO FLAMES-UVES spectra
as an appropriate post-processed test dataset. In Section 3, we describe the
ML methods used here. In Section 4, we apply our ML methods to contaminated
Gaia-ESO FLAMES-UVES spectra to detect and remove the contamination and
mitigate the errors in stellar parameter estimation. We discuss our results in
Section 5, including the limitations and best strategies going forward, and
further applications of our method for bright sky subtraction and detecting
spectroscopic binary stars more generally.

##  2 Spectral Data

A set of 3100 R$\sim$47,000 FLAMES-UVES spectra (Pasquini et al., 2002) from
the Gaia-ESO Survey (GES) DR4 (Sacco et al., 2014), in the wavelength range
$483-530$ nm and with no continuum normalization applied, is used in this
study. The GES targeted dwarfs and giants with metallicities between -2.5 and
+0.5 dex and spectral types from O to M, so our sample contains a large
variety of stars. These spectra were chosen as they cover a broad spectral
region at optical-IR wavelengths, with high SNR, and high resolution. To match
the resolution and wavelength sampling of the WEAVE stellar survey, they were
degraded and re-sampled to R=20,000 for the high-resolution (HR) mode, and
R=5000 for the low-resolution (LR) mode, which we respectively refer to as the
WEAVE-GES-HR sample and WEAVE-GES-LR sample hereafter. Note that this spectral
range corresponds to the ‘green’ arm of the WEAVE HR mode and a portion of the
‘blue’ arm of the WEAVE LR mode; the red ($595-685$ nm) and blue ($404-465$
nm) arms of the HR mode and the red ($579-959$ nm) and entire blue ($366-606$
nm) arms of the LR mode will be used in future studies.

Additionally, the spectra are supplemented with a data set of solar/sky
spectra222A set of 232 FLAMES-UVES solar spectra were collected from this
website. A communications satellite is visible in the sky only because it is
reflecting sunlight, thus a collection of solar spectra is crucial for
simulating what a contaminated observation would look like. The details of the
simulated contamination will be explained in Section 3.

##  3 Machine Learning Methods

The application of ML methods for analyzing astrophysical datasets has
increased in popularity, particularly as computing power and complex ML
algorithms become more powerful and accessible (Fluke & Jacobs, 2020). Large
observational surveys, as well as simulations, have provided massive datasets
for developing ML tools with astrophysical applications. Large datasets are
useful because they help to identify patterns and relationships in the data
that might not be apparent with smaller datasets. With more data of high
quality, models can learn to generalize better due to the diversity of
examples, meaning they can make accurate and robust predictions about new,
unseen data.

While other tools could be used to identify solar contamination, e.g. a
traditional matched filter with solar spectrum template, we decided to use ML
because of its computational efficiency (a dataset of millions of spectra
could be analyzed in a couple of minutes) and history of successful
implementations in astronomy (see Sen et al. 2022 for a recent review).

A neural network (NN) is a function that transforms an input into a desired
output. The function is composed of many parameters, arranged in layers with
non-linear activation functions, which form a highly non-linear combination of
the input features, and allows for complex mappings to be represented with
high precision. A convolutional neural network (CNN), in which a sequence of
learned filters followed by a series of learned inter-connected nodes, can
transform a stellar spectrum into a prediction of associated stellar
parameters. NNs and CNNs have proven to be effective at deconstructing and
modelling the components of stellar spectra (e.g., Fabbro et al., 2018; Leung
& Bovy, 2019; Ting et al., 2019; Guiglion et al., 2020; O’Briain et al., 2020;
Bialek et al., 2020; Sharma et al., 2020; Zhao et al., 2022).

In this section, we describe the ML methods we have built to identify and
demix contaminated spectra in the post-processed spectra in the Gaia-ESO
FLAMES-UVES survey.

###  3.1 Creating the training and test sets

The sample of 3100 FLAMES-UVES spectra was split 80%/20% into reference/test
sets, with a further 85%/15% split of the reference set to acquire
training/validation sets. Within each set, a random stellar spectrum was
chosen, contaminated to a random fraction (between 0-50% of the median
intensity value) with a randomly chosen solar spectrum from a set of 232 solar
spectra, degraded from a resolution of 47,000 to 20,000 for WEAVE-GES-HR and
to 5000 for WEAVE-GES-LR, and finally re-binned to the respective HR or LR
WEAVE wavelength grid. Spectra in the reference set were additionally
uniformly radial velocity shifted (|$v_{rad}|<200$km/s) before the
contamination was applied. The boundaries on the contamination were chosen
based on Figure 6 from Bassa et al. (2022) which shows the effective magnitude
of LEO satellites to be around 21 – given the normal exposure times expected
for WEAVE ($\sim$600-1200s) – and on the expected limiting magnitude of WEAVE
to be G=20.7 in the low-resolution mode (Jin et al., 2022). Note the expected
limiting magnitude for the high-resolution spectra is G=16 so the
contamination problem is far more problematic in the low-resolution spectra.
The contaminated spectrum was obtained with the formula

$\mathbf{x}_{\mathrm{contaminated}}=\mathbf{x}_{\mathrm{stellar}}+c\times\frac{\tilde{\mathbf{x}}_{\mathrm{stellar}}\mathbf{x}_{\mathrm{solar}}}{\tilde{\mathbf{x}}_{\mathrm{solar}}}$

where $\mathbf{x}_{\mathrm{stellar}}$ is a radial velocity shifted stellar
spectrum, $\mathbf{x}_{\mathrm{solar}}$ is a solar spectrum, $c$ is the
contamination fraction, and $\tilde{\mathbf{x}}$ refers to the median. The
process was repeated until the final training/validation/test sets contained
35000/5000/10000 spectra (the performance did not increase when training on
more samples). All data sets retained the original, clean spectra, such that
e.g. the training set included 35000 contaminated and their corresponding
35000 clean spectra, important for corrections when imbalanced training was
taking place.

###  3.2 Neural networks

####  3.2.1 Classification network for detection

A standard CNN was implemented for the classification task (i.e., our StarNet
architecture, Fabbro et al. 2018; Bialek et al. 2020). The input was an
individual stellar spectrum and the output prediction was a score between 0
(clean) and 1 (contaminated). A binary cross-entropy (BCE) loss function was
used to compare the CNN’s output prediction with the target label of the input
spectrum, such that the output of the trained CNN can be thought of as a
prediction of the probability that the spectrum is contaminated (though it is
not guaranteed to be a probability - i.e. calibrated). The Adam algorithm
(Kingma & Ba, 2014) was used for the gradient-based optimization of the BCE,
with a learning rate that was decreased when the performance on the validation
set plateaued. The model that scored the lowest BCE on the validation set was
saved, to ensure the final model did not overfit to the training data, and
used for all tests.

####  3.2.2 Regression network for stellar parameters

The same CNN and training scheme was used as in the classification task;
however, the last layer was changed to linear outputs, and the loss function
was changed to the mean-squared error (MSE). These were then used to regress
on the stellar parameters of the individual stellar spectra. The mean and
standard deviation of the distributions of stellar parameters were used to
normalize the labels (for zero mean and unit variance), and then to de-
normalize the inferred stellar parameters.

####  3.2.3 U-Net for removing contamination

Figure 1: A schematic illustration of the Wave U-Net removing satellite
contamination from a stellar spectrum. The details of each block can be found
in the project’s Github repository.

U-Nets are CNNs that were originally created for biomedical imaging
segmentation tasks (Ronneberger et al., 2015). These include downsampling and
upsampling layers that encode then decode data, and they combine high-
resolution information from downsampled layers with subsequent upsampling
layers via skip connections which helps to prevent information loss and
vanishing gradients.

After a few trials on U-Net architectures, a particular adaptation of the
U-Net to one-dimensional domains, Wave U-Net (Stoller et al., 2018), was
adopted as a base architecture. Wave U-Net was created for audio source
separation and introduced architectural improvements such as an upsampling
technique and context-aware prediction framework to reduce artifacts in its
output. Empirical results of Wave U-Net showed its capacity in isolating raw
wave forms from various instruments (e.g. bass, guitar, drums, voice). In a
similar manner, Wave U-Net can be given a raw spectrum with LEO satellite
contamination and tasked with inferring – and therefore isolating – the pure
stellar spectrum. Figure 1 shows schematically how Wave U-Net works on a
stellar spectrum. We note an improved version of Wave U-Net was used in this
study333details of the architecture improvements can be found at
https://github.com/f90/Wave-U-Net-Pytorch.

A similar training scheme as the previous methods was used but a mean absolute
error (MAE) regression loss was used instead of an MSE loss because the MSE
led to an undesired amount of smoothing in the reconstructed spectra.

####  3.2.4 An ensemble of models

If an ensemble of NNs with random initializations are independently trained on
the same data and given the same task, then every NN will settle to a
different minimum of the loss function owing to the stochastic nature of the
training procedure. As such, each NN will have learned something unique about
the training data, and no single NN in the ensemble can necessarily be
considered the best. It is therefore an advantage to utilize each NN in the
ensemble to make the most robust predictions. Following our similar procedure
as in Bialek et al. (2020), an ensemble of CNNs was trained for the detection
and probability of an individual spectrum being contaminated.

##  4 Results

The WEAVE-GES-HR and WEAVE-GES-LR spectra were used to assess the performance
of the CNN and Wave U-Net and showed the following results.

###  4.1 Detection

Figure 2: A confusion matrix showing information about the rates of true
positives, true negatives, false positives, and false negatives of the WEAVE-
GES-LR sample. The test set of 10,000 spectra was divided such that 10% were
contaminated spectra and 90% were uncontaminated (clean) spectra. The model
had a true positive detection rate of 85.42% and false positive detection rate
of 2.33%. A decision boundary of 0.7 was used for this plot; the false
positive rate can be decreased by raising the decision boundary (i.e.
accepting positive predictions only when the model is very confident) at the
cost of decreasing the true positive rate. Note: similar results were found
with the WEAVE-GES-HR sample.

For the detection task, a spectrum was considered to be contaminated if its
contamination level was $>$1%. In the test set of 10,000 spectra, 1,000 were
contaminated and 9,000 were clean, uncontaminated spectra. Figure 2 shows a
confusion matrix for the contamination detection on the WEAVE-GES-LR test set
with the decision threshold set to 0.7, i.e. spectra were considered
contaminated if the output prediction of the CNN was $>$0.7. The true negative
detection rate was an impressive 97.67%, meaning that, in this particular
case, only 2.33% of clean spectra (210 total) were misclassified. Meanwhile,
the true positive detection rate was 85.42% such that 14.58% of contaminated
spectra (143 total) were misclassified as clean spectra. The results on the
WEAVE-GES-HR sample were similar. Depending on the decision threshold chosen,
these variables can change based on the desired optimization; the next section
explores this.

####  4.1.1 Fine-tuning the decision threshold

Since the CNN outputs a prediction between 0 and 1, there must be a choice
made about when to consider a prediction to be positive or negative. A natural
choice for this decision threshold might be 0.5 if calibrated, i.e.
predictions above and below this value are the contaminated and clean spectra,
respectively. If the goal is to detect contaminated spectra, the threshold can
be lowered to capture fainter contaminations, or raised for only the most
confident predictions for contaminated spectra.

Figure 3: A precision-recall plot showing how effectively two CNNs separately
trained to identify contaminated WEAVE-GES-HR and WEAVE-GES-LR spectra were
able to identify contaminated spectra depending on the decision threshold
chosen. As an example, for the WEAVE-GES-HR spectra, the threshold could be
chosen such that the precision and recall of the identification procedure are
96% and 67% respectively, i.e. extremely precise yet missing 33% of the true
contaminants.

To illustrate the prediction dynamics as the decision boundary is modified, a
Precision-Recall analysis was conducted. Mathematically, precision and recall
are defined as:

$Precision=\frac{TP}{TP+FP}$ $Recall=\frac{TP}{TP+FN}$

where TP is the true positives, FP is the false positives, and FN is the false
negatives. Precision is a measure of the quality of the CNN’s predictions,
whereas recall is a measure of the fraction of contaminated sources that were
identified. Figure 3 shows how the precision and recall change as a function
of decision threshold for the WEAVE-GES-HR and WEAVE-GES-LR samples: generally
as the decision boundary increases, the precision increases and the recall
decreases. The performance on the WEAVE-GES-HR is better overall, likely
because the increased resolution provides more information to distinguish
stellar features from solar features.

In the case of identifying contaminated spectra, it is probably desirable to
optimize the decision threshold such that there is a high precision while
minimally sacrificing the recall, e.g. the values can be optimized such that
the model achieves 96% precision and 67% recall if a decision threshold of
0.96 is chosen: while the model misses 33% of the total contaminated sources
with this threshold, the trade-off is that there is trust when it does
identify contamination. The precision drops to 80% at the same recall level
for the WEAVE-GES-LR sample.

####  4.1.2 Undetected contamination

To understand how the model was failing at properly identifying all of the
contaminated sources, we examined how well the model recovered them depending
on their contamination level and stellar parameters.

Figure 4: Two CNNs were separately trained to identify contaminated WEAVE-GES-
HR and WEAVE-GES-LR spectra and the fraction of sources properly identified as
being contaminated (i.e. the recall) was computed in bins of contamination
level for decision thresholds of 0.95 (solid line) and 0.80 (dotted line). The
majority of false negatives reside in the low ($<$5%) contamination regime.
Figure 5: The recovery fraction (i.e. recall) of identified contaminated
sources as a function of stellar parameter for the WEAVE-GES-HR and WEAVE-GES-
LR samples, split into sources with high contamination (between 5% and 50%;
dashed lines) and low contamination ($<5$%; solid lines), using a decision
threshold of 0.95. Also shown are the histograms of stellar parameters; low
number statistics in certain parameter ranges cause spurious features.

The recall as a function of contamination gives a more detailed understanding
of the completeness of the model and is shown in Figure 4 for two CNNs trained
separately on the WEAVE-GES-HR and WEAVE-GES-LR samples. Using a decision
threshold of 0.95, the WEAVE-GES-HR CNN recovered approximately 20% of
contaminated sources at a level of 2-3% contamination, but the recovery
fraction increases rapidly to 60-80% above 10% contamination. When lowering
the decision threshold to 0.80, the recovery rate approximately doubles in the
low contamination regime and rises to 80-90% for higher levels of
contamination. The WEAVE-GES-LR CNN showed similar trends albeit with a
decreased performance overall. The false negatives are concentrated in the low
($<$5%) contamination regime.

To uncover any potential biases in detection with respect to the stellar
parameters, the recall for the WEAVE-GES-HR and WEAVE-GES-LR CNNs was computed
as a function of the parameters $T_{\textrm{eff}}$, log$g$, [Fe/H],
$v_{\textrm{rad}}$, and SNR, using a decision threshold of 0.95, as seen in
Figure 5. One would expect the recovery fraction to decrease around solar
values ($T_{\textrm{eff}}$ = 5777 K, log$g$ = 4.44, and [Fe/H] = 0, de Mello
et al. 2014) since it would be difficult to distinguish a solar spectrum mixed
with a solar-like spectrum. There indeed appears to be a local dip at solar
log$g$ and a decreasing trend towards solar metallicity. The most obvious
trend occurs with radial velocities: the recovery rate increases substantially
at higher absolute velocities. Interestingly no clear trends are seen with
SNR, perhaps because the SNR of each spectrum is sufficiently high. Lastly,
there are several anomalous or sporadic points where e.g. the recovery
fraction is zero or nearly zero, but this is likely due to low numbers of
samples in those regions (creating problems for both training the model and
testing the model with small number statistics).

Taken together, Figures 4 and 5 imply that (1) a spectrum with solar-like
surface gravity and metallicity presents an increased difficulty for the CNN
classification scheme, (2) there is no obvious correlation between the
recovery rate and effective temperature or SNR, and (3) the most significant
factors in detectability are contamination levels and the relative difference
between the radial velocity of the stellar and contaminating source.

###  4.2 Stellar parameters prediction

The stellar parameters $T_{\textrm{eff}}$, log$g$, and [Fe/H], along with the
absolute abundances A(Ca), A(Mg), A(O), A(S), and A(Ti), were predicted with a
CNN. For the WEAVE-GES-LR sample, two models were trained: one on 35,000
spectra with contamination between 0% and 50% (CNN+) and one on 35,000 spectra
without contamination (CNN-), and were used in the following tests. Note: the
results on WEAVE-GES-HR can be seen in Appendix A.

####  4.2.1 Removing contamination first

Figure 6: A Wave U-Net architecture was tasked with inferring (pink dashed
line) the pure stellar spectrum (black solid line) from spectra contaminated
by $\sim$40% with a solar spectrum (grey solid line). Shown here are several
examples, in the Mg I b triplet region and regions with strong Ti I and Ca I
absorption, of the inferred WEAVE-GES-LR stellar spectra with their
contamination removed. The last two examples are of the same low-metallicity
star at different SNR, which highlights the difficulty Wave U-Net has with
low-metallicity and low SNR spectra. Aside from that, the predictions closely
match the ground truth, though some residuals can be seen in the deeper
absorption features. Figure 7: Wave U-Net was used to remove solar
contamination from the test set of WEAVE-GES-LR stellar spectra, and shown
here are the distributions of residuals between the predicted spectra and true
spectra split into five groups of increasing levels of contamination. The
bias, $\overline{x}$, and 1-sigma error, s, were calculated for each group of
spectra; the error in all cases is $<$1% and gradually increases with the
level of contamination. The residuals around strong absorption features (e.g.
the Mg I b triplet region around 5175 Å) are more severe. Figure 8: The
residuals between stellar parameter and abundance predictions and GES catalog
values, with the mean (solid lines) and standard deviation (dashed lines) of
the residuals binned according to the amount of contamination in the original
spectra. Two CNN models were separately trained on WEAVE-GES-LR spectra: one
with a training set that included contamination (CNN+), and one that did not
(CNN-). Each model was used to predict the stellar parameters $T$eff, logg,
[Fe/H], and elemental abundances A(Ca), A(Mg), A(O), A(S), A(Ti) on a test set
of WEAVE-GES-LR contaminated spectra. Additionally, CNN- was used for
predictions on a test set of WEAVE-GES-LR spectra with their contamination
removed by Wave U-Net. Overall better predictions were achieved with the CNN+.
Predictions on the abundances O and S had large errors in all cases because of
the lack of strong absorption features in the limited wavelength range
studied. Figure 9: The residuals between stellar abundance predictions and GES
catalog values. The mean (solid lines) and standard deviation (dashed lines)
of the residuals were binned according to the abundance value, and each row
corresponds to increasing levels of contamination in steps of 10%. Two CNN
models were separately trained on WEAVE-GES-LR spectra: one with a training
set that included contamination (CNN+), and one that did not (CNN-). Each
model was used to predict the stellar elemental abundances A(Ca), A(Mg), A(O),
A(S), A(Ti) on a test set of WEAVE-GES-LR contaminated spectra. Additionally,
CNN- was used for predictions on a test set of WEAVE-GES-LR spectra with their
contamination removed by Wave U-Net. At higher abundance values and lower
contamination levels, there is almost no difference between the three models.
The differences become pronounced at lower abundance values and higher
contamination levels, where CNN+ can be seen to perform best.

One method for dealing with the satellite contamination explored in this study
was to attempt removing its signature from the observed spectra completely – a
process analogous to subtracting off a sky spectrum to isolate the pure
stellar spectrum. Once the spectra are cleaned of their contamination, they
can be sent to any downstream pipeline to estimate stellar parameters. As
described in Section 3.2.3, the Wave U-Net architecture was used for this
task.

Figure 6 shows several examples of Wave U-Net recovering WEAVE-GES-LR stellar
spectra in regions with strong Mg, Ti, and Ca features with $\sim$40% solar
contamination: in most cases, the contamination was almost completely removed
and the stellar spectra were recovered but with some more obvious errors
around the deeper absorption features, especially for stars with low
metallicity and low SNR. To see the overall performance of Wave U-Net the
relative residuals of its predictions on the entire test set of WEAVE-GES-LR
spectra were calculated, and the distributions are shown in Figure 7. The
1-sigma errors were under 1% across the entire test set, with a minimum of
0.46% for the spectra with 0-10% contamination, though it was confirmed that
the model has larger residuals in the deeper absorption features.

With Wave U-Net operational, we obtained a test set of WEAVE-GES-LR spectra
with their contamination removed to test the performance of CNN-. Figure 8
shows the mean and standard deviation of residuals of CNN-’s stellar parameter
predictions on the test set of WEAVE-GES-LR contaminated spectra before and
after being processed by Wave U-Net. The residuals on the contaminated spectra
are nearly identical to those on the Wave U-Net spectra in the low
contamination regime, $\sim$2-3x higher in the high contamination regime, and
significantly increase for the abundances around the 25% contamination level.
The residuals on the Wave U-Net spectra are relatively constant across most
contamination levels, though a slight increase can be seen at the highest
levels.

####  4.2.2 Training a CNN on contaminated spectra

Instead of identifying and throwing away a contaminated spectrum, or
attempting to remove its contamination, one might be interested in extracting
accurate information despite the contamination. One way of accomplishing this
task is again to use ML.

A major benefit of the feed-forward CNN architecture used in this study is
that the spectra can be augmented with features (e.g. solar contamination) to
be identified or, in this case, ignored. By feeding artificially contaminated
spectra through the NN and giving it the task of predicting stellar parameters
of uncontaminated spectra, the NN learns to be robust to contamination in its
predictions.

To demonstrate the ability of a CNN to predict through contamination, the CNN+
model was given a test set of contaminated spectra, and Figure 8 shows the
residuals of its predictions on the WEAVE-GES-LR spectra. The CNN+ residuals
are constant and overall lower than CNN-’s residuals on both the Wave U-Net
spectra and contaminated spectra over the full range of contamination and
across all parameters. Even with 0-2% contamination levels, the residuals for
$T_{\textrm{eff}}$ and log$g$ are slightly larger in CNN- than for CNN+. Note
that in the wavelength range studied, there are limited oxygen and sulfur
features and thus the residuals remain large and somewhat constant for both
CNN+ and CNN-.

Figure 9 shows a similar comparison but the residuals were instead binned
according to the stellar abundances, and split into 5 groups of increasing
contamination. At low (0-10%) contamination levels, the results are nearly
identical. At lower abundance levels, it can be seen that Wave U-Net has
difficulty with properly recovering the spectra even at lower (10-20%)
contamination levels, leading to worse precision than even the CNN- on
contaminated spectra. The performance of the CNN+ becomes especially apparent
at higher contamination levels and low abundance levels, whereas all three
models perform equally well at higher abundance levels.

####  4.2.3 Removing contamination for downstream tasks

Figure 10: $The~{}Cannon$ was used to predict the stellar parameters $T$eff,
log$g$, and [Fe/H] on three test sets of WEAVE-GES-LR spectra: contaminated
(by $<$50%) spectra, clean spectra, and spectra with contamination removed by
Wave U-Net. The residuals between the predictions on the contaminated spectra
and clean spectra, as well as the residuals between the predictions on Wave
U-Net spectra and clean spectra, were computed. The histograms show the
improvements in stellar parameter estimates when Wave U-Net is used to remove
the contamination, confirming that Wave U-Net can be used in conjunction with
other stellar parameter estimation pipelines.

To show the utility of removing contamination outside the scope of neural
networks and our pipeline, $The~{}Cannon$ (Ness et al., 2015) was used to
predict the stellar parameters $T_{\textrm{eff}}$, log$g$, and [Fe/H] on
contaminated, clean, and Wave U-Net WEAVE-GES-LR spectra. Figure 10 shows the
substantial reduction in prediction residuals – the standard deviation reduces
by a factor of $\sim$2 – when the contaminated spectra are cleaned by Wave
U-Net first.

##  5 Discussion

###  5.1 Limitations

####  5.1.1 Low levels of contamination

The results from our study show that the majority of low-contamination ($<$5%
contamination) spectra are not detected with the presented framework. Since
low contamination will likely be the most prevalent level of contamination
(especially in the WEAVE-HR spectra), this could mean that many contaminated
spectra would pass through our current pipeline undetected. This creates a
problem if the goal is simply to flag the contaminated spectra for removal or
further processing, but there isn’t necessarily a problem for stellar
parameter and abundance predictions. Indeed, Figures 8 and 13 show that the
errors on abundance predictions due to satellite contamination are
insignificant compared to typical spectroscopic uncertainties (e.g., Recio-
Blanco et al., 2022; Accetta et al., 2022; Steinmetz et al., 2020; Aguado et
al., 2019; Smiljanic et al., 2016) when contamination is $\lesssim$20% for
WEAVE-GES-LR and $\lesssim$10% for WEAVE-GES-HR.

For temperature and surface gravity predictions, however, Figures 8 and 13
show that even low levels of contamination can introduce noticeable errors.
When Wave U-Net is used to remove the low-level contamination, the resulting
predictions are at best marginally better and sometimes marginally worse than
those on contaminated spectra; removing very low levels of contamination is
difficult for a Wave U-Net. By including contamination as an augmentation to a
training set instead of attempting to remove it, such that a CNN learns to
bypass (or ignore) the contamination when making stellar parameter
predictions, the errors on temperature and surface gravity estimates are
reduced. Contamination as an augmentation therefore appears to be the best
strategy for dealing with spectra containing low (and high) levels of
contamination, though Wave U-Net is only marginally worse.

####  5.1.2 Realistic training set

In this study it was assumed the spectrum reflected by the satellites would be
an unmodified solar spectrum, but in reality the satellites from each company
will have different albedos and thus reflect a slightly modified solar
spectrum. The results in this paper should therefore be interpreted as a best-
case scenario of knowing the characteristics of the contamination completely.
There are two options to include more realistic contamination information in
the pipeline: (1) use the published materials, if available, from each company
regarding the reflection properties of their satellites and include these as a
wavelength-dependent data augmentation in the training set, and/or (2) the
telescope could intentionally observe stars that either have previously had
confirmed satellite contamination in their spectra or are predicted to have a
satellite crossing their path, to collect real examples for the training set.
Ideally both options would be pursued to curate a realistic training set,
however it is not possible – with current data that each company supplies – to
predict satellite positions with enough accuracy to know whether they have
contaminated or will contaminate a star spectrum at a given time in a given
place. We do expect that there will be some studies (both from industry and
from astronomers) aimed at characterizing the reflectivity of various classes
of satellites.

####  5.1.3 Solar twins

Figure 11: A set of WEAVE-GES-LR solar twins (defined as having
$T_{\textrm{eff}}$ within 100 K, $\textrm{log}g$ within 0.1 dex, and [Fe/H]
within 0.04 dex of the solar values) was collected and CNN+ was used to
predict their stellar parameters and abundances and compare to GES catalog
values. Shown here are the normalized distributions of residuals (predicted
minus catalog values) on the solar twins (orange) and the rest of the test set
(grey). The residuals are mostly compatible with the overall distribution, but
there is a consistent over-prediction of magnesium abundance and a small
under-prediction of temperature. Note: only five solar twins were found in the
test set, each with multiple instances of contamination and all with
|$v_{\textrm{rad}}|<15$km/s.

A star with solar-like stellar parameters presents an added difficulty for a
NN since the contaminating and true spectra are similar and thus harder to
disentangle. This was indeed the case for the identification task, where the
CNN had a lower recall around solar-like surface gravity and metallicity. For
the task of predicting stellar parameters and abundances, however, CNN+ can
help overcome this limitation. A set of five solar twins – defined as having
$T_{\textrm{eff}}$ within 100 K, $\textrm{log}g$ within 0.1 dex, and [Fe/H]
within 0.04 dex of the solar values (a slightly narrower definition than used
by Ramírez et al. 2009) – was used to check if their predicted parameters were
significantly affected by contamination. Figure 11 shows that the
distributions of the stellar parameters and abundances for the solar twins are
compatible with the overall distributions from the test set with two
exceptions: (1) a consistent and significant over-prediction of 0.1 dex for
the magnesium abundances, and (2) an insignificant under-prediction of $<$100
K for temperature. We suggest that even low levels of satellite contamination
can impact magnesium abundance predictions in solar twins. With a sample size
of five, however, the biases we see could also come from an under-
representation of solar twins in the test set; a larger test set is required
to make statistically significant conclusions.

###  5.2 Further applications

Another way to view the results in this paper is the application of a CNN to
bright sky subtraction. Typically high resolution spectroscopy is carried out
at observatories during "bright time", when the face of the Moon is $>50\%$
illuminated, leading to stellar spectra with higher levels of solar
contamination. There is nothing fundamentally different about a satellite
reflecting solar light and the moon reflecting solar light (aside from
wavelength-dependent reflectance properties) so the methods outlined in this
paper could be directly applied to bright time stellar spectra.

A similar method could also be used to detect whether a measured spectrum
contains light from a binary stellar system. A training data set could be
created in which stellar spectra are "contaminated" with a library of
artificial binary counterparts; a CNN could be trained to detect the binary
systems while a U-Net could be trained to separate the two spectra. One would
need to carefully consider realistic values for the relative differences in
radial velocity, temperature, surface gravity, abundances, and brightness
between the artificially created binary stellar spectra.

##  6 Conclusions

The detection and mitigation of contaminated spectra are important tasks in
astronomical research, as contamination can lead to inaccurate conclusions and
bias in data analysis. In this study, we applied a convolutional neural
network (CNN) for both detection and stellar parameter prediction
(specifically stellar parameters $T_{\textrm{eff}}$, log$g$, and [Fe/H], and
the abundances A(Ca), A(Mg), A(O), A(S), and A(Ti)), and a Wave U-Net for
removal of satellite contamination in a WEAVE-like spectral survey. We
determined the limitations and feasibility of a machine learning-based
pipeline for these tasks.

In a test set of 10,000 artificially contaminated WEAVE-GES-HR spectra, the
decision threshold of a CNN was adjusted such that a detection precision of
96% and a detection recall of 67% were achieved, which means that while the
model misses 33% of the total contaminated sources (mostly those with $<$10%
contamination), it can confidently identify the contamination it does detect.
The results on WEAVE-GES-LR spectra were slightly poorer, with a precision of
80% at the same recall level. It was also found that a contaminated spectrum
is harder to identify if it has solar-like surface gravity, solar-like
metallicity, or low radial velocity.

A Wave U-Net model was trained to recover clean spectra from contaminated
spectra and was able to recover them with 1-$\sigma$ errors of 0.46% in the
low ($<10\%$) contamination regime and $<$1% in the high (40-50%)
contamination regime. Two CNNs were separately trained on contaminated (CNN+)
and clean (CNN-) spectra, and it was shown that the accuracy in predictions,
when compared to CNN- predictions on contaminated spectra, was improved by up
to a factor of 2-3 when using CNN- on the Wave U-Net spectra and when using
CNN+ on contaminated spectra, with overall better results from the CNN+.
Including an expected contamination as an artificially augmented training set
therefore appears to be the best strategy for mitigating the effects of
contamination in our pipeline.

## Acknowledgements

We thank Federico Sestito, John Pazder, and Ted Grosson for the helpful
comments and discussions about bright time sky subtraction, satellite
coatings, and LSST satellite contamination. We also thank the anonymous
referee for comments which helped strengthen and clarify our results. SB and
KAV thank the Natural Sciences and Engineering Research Council for funding
through the Discovery Grants program and the CREATE program in New
Technologies for Canadian Observatories. SL acknowledges the support by PRIN
INAF 2019 grant ObFu 1.05.01.85.14 (“Building up the halo: chemo-dynamical
tagging in the age of large surveys”, PI. S. Lucatello).

Software: astropy (Robitaille et al., 2013), matplotlib (Hunter, 2007), numpy
(Harris et al., 2020), pytorch (Paszke et al., 2019), scipy (Virtanen et al.,
2020), seaborn (Waskom, 2021).

## Code availability

All code for this project can be found at the project’s Github repository:
https://github.com/astroai/StarUnLink.

## Data availability

All data underlying this article will be shared on reasonable request to the
corresponding author.

## References

  * Accetta et al. (2022) Accetta K., et al., 2022, The Astrophysical Journal Supplement Series, 259, 35
  * Aguado et al. (2019) Aguado D. S., et al., 2019, Monthly Notices of the Royal Astronomical Society, 490, 2241
  * Bassa et al. (2022) Bassa C. G., Hainaut O. R., Galadí-Enríquez D., 2022, A&A, 657, A75
  * Bialek et al. (2020) Bialek S., Fabbro S., Venn K. A., Kumar N., O’Briain T., Yi K. M., 2020, MNRAS, 498, 3817
  * Boley et al. (2021) Boley A., Lawler S., Barmby P., Di Francesco J., Falle A., Howse J., Kavelaars J., 2021, Technical report, Report on Mega-Constellations to the Government of Canada and the Canadian Space Agency. https://arxiv.org/abs/2104.05733
  * Dalton et al. (2018) Dalton G., et al., 2018, in Proc. SPIE. p. 107021B, doi:10.1117/12.2312031
  * Fabbro et al. (2018) Fabbro S., Venn K. A., O’Briain T., Bialek S., Kielty C. L., Jahandar F., Monty S., 2018, MNRAS, 475, 2978
  * Fluke & Jacobs (2020) Fluke C. J., Jacobs C., 2020, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10, e1349
  * Guiglion et al. (2020) Guiglion G., et al., 2020, Astronomy & Astrophysics, 644, A168
  * Hainaut & Williams (2020) Hainaut O. R., Williams A. P., 2020, A&A, 636, A121
  * Harris et al. (2020) Harris C. R., et al., 2020, Nature, 585, 357
  * Hu et al. (2022) Hu J. A., Rawls M. L., Yoachim P., Ivezić Ž., 2022, The Astrophysical Journal Letters, 941, L15
  * Hunter (2007) Hunter J. D., 2007, Computing in science & engineering, 9, 90
  * Jin et al. (2022) Jin S., et al., 2022, arXiv preprint arXiv:2212.03981
  * Kingma & Ba (2014) Kingma D. P., Ba J., 2014, arXiv preprint arXiv:1412.6980
  * Lawler et al. (2022) Lawler S. M., Boley A. C., Rein H., 2022, AJ, 163, 21
  * Leung & Bovy (2019) Leung H. W., Bovy J., 2019, MNRAS, 483, 3255
  * McConnachie et al. (2016) McConnachie A. W., et al., 2016, arXiv e-prints, p. arXiv:1606.00060
  * Ness et al. (2015) Ness M., Hogg D. W., Rix H.-W., Ho A. Y., Zasowski G., 2015, The Astrophysical Journal, 808, 16
  * O’Briain et al. (2020) O’Briain T., Ting Y. S., Fabbro S., Moo K. Y., Venn K. A., Bialek S., 2020, MNRAS
  * Pasquini et al. (2002) Pasquini L., et al., 2002, The Messenger, 110, 1
  * Paszke et al. (2019) Paszke A., et al., 2019, Advances in neural information processing systems, 32
  * Ramírez et al. (2009) Ramírez I., Melendez J., Asplund M., 2009, Astronomy & Astrophysics, 508, L17
  * Recio-Blanco et al. (2022) Recio-Blanco A., et al., 2022, arXiv preprint arXiv:2206.05541
  * Robitaille et al. (2013) Robitaille T. P., et al., 2013, Astronomy & Astrophysics, 558, A33
  * Ronneberger et al. (2015) Ronneberger O., Fischer P., Brox T., 2015, in International Conference on Medical image computing and computer-assisted intervention. pp 234–241
  * Sacco et al. (2014) Sacco G. G., et al., 2014, Astronomy & Astrophysics, 565, A113
  * Sen et al. (2022) Sen S., Agarwal S., Chakraborty P., Singh K. P., 2022, Experimental Astronomy, 53, 1
  * Sharma et al. (2020) Sharma K., Kembhavi A., Kembhavi A., Sivarani T., Abraham S., Vaghmare K., 2020, Monthly Notices of the Royal Astronomical Society, 491, 2280
  * Smiljanic et al. (2016) Smiljanic R., et al., 2016, Astronomy & Astrophysics, 589, A115
  * Steinmetz et al. (2020) Steinmetz M., et al., 2020, The Astronomical Journal, 160, 83
  * Stoller et al. (2018) Stoller D., Ewert S., Dixon S., 2018, arXiv preprint arXiv:1806.03185
  * Tamura et al. (2018) Tamura N., et al., 2018, in Evans C. J., Simard L., Takami H., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 10702, Ground-based and Airborne Instrumentation for Astronomy VII. p. 107021C, doi:10.1117/12.2311871
  * Ting et al. (2019) Ting Y.-S., Conroy C., Rix H.-W., Cargile P., 2019, The Astrophysical Journal, 879, 69
  * Virtanen et al. (2020) Virtanen P., et al., 2020, Nature methods, 17, 261
  * Walker & Hall (2020) Walker C., Hall J., 2020, Technical report, Impact of Satellite Constellations on Optical Astronomy and Recommendations Toward Mitigations. NSF’s NOIRLab (Communications, Education & Engagement division) Workshop Report
  * Walker & Hall (2021) Walker C., Hall J., 2021, Technical report, SATCON2 Executive Summary Report. NSF’s NOIRLab (Communications, Education & Engagement division) Workshop Report
  * Walker et al. (2020) Walker C., et al., 2020, Dark & Quiet Skies I (2020), Dark & Quiet Skies I (2020), Report of the conference held 5-9 October, 2020., doi:10.5281/zenodo.5898785
  * Walker et al. (2021) Walker C., et al., 2021, Dark & Quiet Skies II (2021), Dark & Quiet Skies II (2021), Report of the conference held 3-7 October, 2021., doi:10.5281/zenodo.5874725
  * Waskom (2021) Waskom M. L., 2021, Journal of Open Source Software, 6, 3021
  * Zhao et al. (2022) Zhao Z., Wei J., Jiang B., 2022, Advances in Astronomy, 2022, 1
  * de Jong et al. (2019) de Jong R. S., et al., 2019, The Messenger, 175, 3
  * de Mello et al. (2014) de Mello G. P., Da Silva R., Da Silva L., De Nader R., 2014, Astronomy & Astrophysics, 563, A52

##  Appendix A Results on high-resolution spectra

The stellar parameters $T_{\textrm{eff}}$, log$g$, and [Fe/H], along with the
absolute abundances A(Ca), A(Mg), A(O), A(S), and A(Ti), were predicted with a
CNN. For the WEAVE-GES-HR sample, two models were trained: one on 35,000
spectra with contamination between 0% and 50% (CNN+) and one on 35,000 spectra
without contamination (CNN-). Additionally, a Wave U-Net was used to remove
the contamination from the WEAVE-GES-HR sample. Figure 12 shows the residuals
in the reconstructed Wave U-Net spectra, and Figure 13 shows the residuals in
the stellar parameter and abundance predictions of CNN+ and CNN-.

Figure 12: Wave U-Net was used to remove solar contamination from the test set
of WEAVE-GES-HR stellar spectra, and shown here are the distributions of
residuals between the predicted spectra and true spectra split into five
groups of increasing levels of contamination. The bias, $\overline{x}$, and
1-sigma error, s, were calculated for each group of spectra; the error in all
cases is $<$1% and gradually increases with the level of contamination. The
residuals around strong absorption features (e.g. the Mg I b triplet region
around 5175 Å) are more severe. Figure 13: The residuals between stellar
parameter and abundance predictions and GES catalog values, with the mean
(solid lines) and standard deviation (dashed lines) of the residuals binned
according to the amount of contamination in the original spectra. Two CNN
models were separately trained on WEAVE-GES-HR spectra: one with a training
set that included contamination (CNN+), and one that did not (CNN-). Each
model was used to predict the stellar parameters $T$eff, logg, [Fe/H], and
elemental abundances A(Ca), A(Mg), A(O), A(S), A(Ti) on a test set of WEAVE-
GES-HR contaminated spectra. Additionally, CNN- was used for predictions on a
test set of WEAVE-GES-HR spectra with their contamination removed by Wave
U-Net. Overall better predictions were achieved with the CNN+. Predictions on
the abundances O and S had large errors in all cases because of the lack of
strong absorption features in the limited wavelength range studied.