# Bayesian Flow Networks in Continual Learning

Mateusz Pyla
IDEAS NCBR
Jagiellonian University, Faculty of Mathematics and Computer Science
Jagiellonian University, Doctoral School of Exact and Natural Sciences
<EMAIL_ADDRESS>
&Kamil Deja
IDEAS NCBR
Warsaw University of Technology &Bartłomiej Twardowski
IDEAS NCBR
Autonomous University of Barcelona &Tomasz Trzciński
IDEAS NCBR
Warsaw University of Technology
Jagiellonian University, Faculty of Mathematics and Computer Science

###### Abstract

Bayesian Flow Networks (BFNs) has been recently proposed as one of the most
promising direction to universal generative modelling, having ability to learn
any of the data type. Their power comes from the expressiveness of neural
networks and Bayesian inference which make them suitable in the context of
continual learning. We delve into the mechanics behind BFNs and conduct the
experiments to empirically verify the generative capabilities on non-
stationary data.

##  1 Introduction

Diffusion models (Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and
Surya Ganguli., ) have been progressively advancing the state-of-the-art in
generative modelling, especially in the field of image processing (Prafulla
Dhariwal, Alexander Nichol., ; Jonathan Ho, Chitwan Saharia, William Chan,
David J Fleet, Mohammad Norouzi, and Tim Salimans., ; Diederik Kingma, Tim
Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models., ). This
is thanks to the usage of the diffusion processes that allows to learn complex
data distributions (Jonathan Ho, Ajay Jain, and Pieter Abbeel., ; Chin-Wei
Huang, Jae Hyun Lim, and Aaron C Courville., ; Diederik Kingma, Tim Salimans,
Ben Poole, and Jonathan Ho. Variational diffusion models., ; Yang Song,
Stefano Ermon., ; Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma,
Abhishek Kumar, Stefano Ermon, and Ben Poole., ).

However, diffusion models tend to struggle when in comes to non-continuous
data. This is mostly because of the fact that denoising process for discrete,
discretised or tabular data is not easy to define. Therefore, Alex Graves et
al. recently introduced Bayesian Flow Networks (BFNs) (Alex Graves, Rupesh
Kumar Srivastava, Timothy Atkinson, and Faustino Gomez., ) to efficiently
train the model and iteratively update the data parameters without forward
pass. The general idea behind this technique is to change the way in which we
model training data where instead of modelling a single instance, authors
propose to output the parameters of the distribution that best fits the
training data. The main motivation behind this concept is that by doing so,
authors introduce an elegant way to directly model the discrete data
distribution.

In this work, we argue, that with direct modelling of parameters describing
the original training data, BFNs can also be used to efficiently consolidate
portions of separate data chunks. Therefore, we relate to the problem of
continual learning, which tackles the ability of ML models to learn
progressively as new data arrive. Bayesian update is an elegant way to manage
prior belief and information from new observations. However, in Bayesian
learning we often face the issue of turning theory into practical
implementations, limiting the use of the Bayesian learning paradigm (Nan Ding,
Xi Chen, Tomer Levinboim, Sebastian Goodman, and Radu Soricut., ; Hao Wang,
Dit-Yan Yeung., ; Hao Wang, Dit-Yan Yeung., ).

In this preliminary studies, we show the first benchmark of Bayesian Flow
Networks in Continual Learning setup. We show how we can adapt several known
techniques that prevent catastrophic forgetting in neural networks to
continually train BFNs. We highlight their strengths and drawbacks and discuss
future directions on how to employ BFNs to continually consolidate knowledge.

##  2 Related Work

##### Continual Learning (CL)

gathers various approaches in machine learning that aim at reducing
catastrophic forgetting, a phenomenon where models suffer from abrupt loss in
performance when retrained with additional data. Usually there are three
standard group of approaches that try to mitigate this issue: (i)
architectural approaches – methods that focus on the structure of the model
itself that adds task-specific submodules to the architecture; (ii) memory
approaches, methods involve storing some extra information in memory which are
then used to rehearse knowledge during training in subsequent task; (iii)
Regularization Approaches: that identify the important weights for the learned
tasks and penalise the large updates on those weights when learning a new task
(Gido M Van de Ven, Andreas S Tolias., ; Marc Masana, Xialei Liu, Bartłomiej
Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost van de Weijer., ).

Several of those approaches were applied in generative continual learning. In
particular, (Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner.,
) adapt several regularisation-based methods and introduce Variational
Continual Learning where additional architectural change is added with each
task. In BooVAE (Evgenii Egorov, Anna Kuzina, and Evgeny Burnaev., ) an
additive aggregated posterior expansion technique is used to continually
trained Variational Autoencoders, while (Alessandro Achille, Tom Eccles, Loic
Matthey, Chris Burgess, Nicholas Watters, Alexander Lerchner, and Irina
Higgins., ) propose to continually disentangle data representations with VAE.
Several methods train GANs in CL scenarios, e.g. using memory buffer (Chenshen
Wu, Luis Herranz, Xialei Liu, Yaxing Wang, Joost van de Weijer, and Bogdan
Raducanu., ). Most similarly to this work, in (Michał Zając, Kamil Deja, Anna
Kuzina, Jakub M. Tomczak, Tomasz Trzciński, Florian Shkurti, and Piotr Miłoś.,
) authors benchmark the possibilities of continual-learning of diffusion
models with recent CL strategies.

Figure 1: BFNs: Our goal is to model one dimensional data distribution
controlled by unknown $\xi$, which we can only sample from. We start off with
some initial prior belief (0 in this case) that we are very uncertain of (blue
prior). We sample an observation $y$ and add noise to obtain what we call a
sender distribution – Gaussian centered at $y$. We pass the parameters of
input to the neural network obtaining output distribution, improved version
enriched by jointly processes of all the variables. We comply with the noise
scheduler to obtain orange receiver. We minimise the KL divergence between the
sender and receiver so that our output gets closes to the samples from true
data distribution.

##  3 Method

Although BFNs work on different type of data, both discrete and continuous
time steps, the most approachable way to understand the dynamics is through
continuous data in the discrete setting and extending it as in 6.1.

###  3.1 Inference

We use neural network $\psi$ parameterised by $\theta$ to learn the parameters
$\xi$ controlling the data distribution. The underlying distribution is
complex, however we can sample from it. The general idea is to start with
uneducated guess of normal distribution with high variance, and iteratively
improve the estimation of data distribution with the help of the network as we
sample more and more data. We assume that we model the data only with
Gaussians.

As in diffusion models, we set the number of steps and we establish the
accuracy scheduler managing the usefulness of noised samples for various time
steps. We start from the prior belief, for instance centered at 0 with huge
standard deviation. We want our network to predict better data parameters from
the current estimate. To update network weights, we calculate the gradient as
a KL divergence between the predicted and original data in their noised forms.
While explaining the mathematical formulation behind the process, we point out
concrete example in 1.

More rigorously, we threat each data variable separately. Due to the
independence, the input distribution can be expressed as the product of one
dimensional distributions.

$\displaystyle
p_{{}_{I}}(\operatorname{\mathbf{x}}\mid\xi)=\Pi_{d=1}^{D}p_{{}_{I}}(x^{(d)}\mid\xi^{(d)})$
(1)

Rather than data points, we receive the noisy samples forming the normal
distribution centered at the true values with variance purely depended on the
accuracy scheduler.

$\displaystyle
p_{{}_{S}}\left(\operatorname{\mathbf{y}}\mid\operatorname{\mathbf{x}};\alpha\right)=\Pi_{d=1}^{D}p_{{}_{S}}\left(y^{(d)}\mid
x^{(d)};\alpha\right)$ (2)

Since we proceed each dimension independently, we need global feedback coming
from the interactions between variables. The role of neural network is to
update the guess knowing the previous belief, so that we can better decode the
sended sample.

$\displaystyle
p_{{}_{O}}(\operatorname{\mathbf{x}}\mid\xi,t)=\Pi_{d=1}^{D}p_{{}_{O}}(x^{(d)}\mid\Psi^{(d)}(\xi,t))$
(3)

Since we are not aware of the true $x^{(d)}$, knowing
$p_{{}_{S}}\left(\cdot\mid x^{(d)};\alpha\right)$ we can only marginalise over
all possible values $x^{\prime(d)}$ weighted by the output probability,
obtaining the receiver:

$\displaystyle p_{{}_{R}}(\operatorname{\mathbf{y}}\mid\xi;t,\alpha)$
$\displaystyle=\operatorname*{\mathbb{E}}_{p_{{}_{O}}(\operatorname{\mathbf{x}}^{\prime}\mid\xi;t)}p_{{}_{S}}\left(\operatorname{\mathbf{y}}\mid\operatorname{\mathbf{x}}^{\prime};\alpha\right)$
(4)

Iteratively, for the next time steps, we apply Bayesian updates to improve the
input distribution (that accumulates only local knowledge about a single
dimension) by the acquired knowledge from receiver (that encodes global
knowledge about interactions between dimensions). Both input and sender are
Gaussian distributions and factorised independently, hence the update is
straightforward: $\rho_{i+1}=\rho_{i}+\alpha$ and
$\mu_{i+1}=\frac{\rho_{i}\mu_{i}+\alpha y}{\rho_{i+1}}$ Once we set the number
of steps to infinity, under mild conditions for the scheduler, we are able to
efficiently compute the dynamics:

$\displaystyle
p_{{}_{F}}(\xi\mid\operatorname{\mathbf{x}};t)=p_{{}_{U}}(\xi\mid\xi_{0},\operatorname{\mathbf{x}};\beta(t)).$
(5)

There is a freedom in choosing the underlying network, as long as it returns
the new parameters of data (U-Net (Olaf Ronneberger, Philipp Fischer, and
Thomas Brox., ), Transformers (Ashish Vaswani, Noam Shazeer, Niki Parmar,
Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia
Polosukhin., ), TabTransformer (Xin Huang, Ashish Khetan, Milan Cvitkovic, and
Zohar Karnin., )) and inference conditions on the time step.

The proposed scheduler is of form $\beta(t)\doteq\sigma_{1}^{-2t}-1$ for
$t\in[0,1]$ yielding accuracy rate
$\alpha(t)=\frac{2\log\sigma_{1}}{\sigma_{1}^{2t}}$. When $\alpha$ is 0, the
model is uninformative of samples and confidence increases with the higher
values. $\sigma_{1}$ is the hyperparameter standing for standard deviation at
the final time.

###  3.2 Training

Loss function can be intuitively understood as costs of revealing the
underlying data distribution with the least possible effort or information.
Our objective is to match output distribution to the data distribution
indirectly by optimizing KL divergence between their noisy versions.
Specifically, we minimise KL divergence between sender and receiver
distributions: $D_{KL}\left(p_{{}_{S}}\parallel p_{{}_{R}}\right)$ in order to
bring output predictions closer and closer to the true data values.

$\displaystyle
L^{n}(\operatorname{\mathbf{x}})\doteq\operatorname*{\mathbb{E}}_{p(\xi_{1},\dots,\xi_{n-1})}\sum_{i=1}^{n}D_{KL}\left(p_{{}_{S}}\left(\cdot\mid\operatorname{\mathbf{x}};\alpha_{i}\right)\parallel
p_{{}_{R}}(\cdot\mid\xi_{i-1};t_{i-1},\alpha_{i})\right),$ (6)

This loss indirectly optimises our true goal:

$\displaystyle
L^{r}(\operatorname{\mathbf{x}})=-\operatorname*{\mathbb{E}}_{p_{{}_{F}}(\xi\mid\operatorname{\mathbf{x}},1)}\ln
p_{{}_{O}}(\operatorname{\mathbf{x}}\mid\xi;1).$ (7)

Let us note that this kind of form follows information-theory interpretation:
we minimise the number of nats required to transmits a sample between two
distributions.

###  3.3 BFN in Continual Learning

We propose to extend the basic idea of BFNs in order to benchmark it with
several known continual-learning strategies. In particular, we start with a
simple regularisation strategy, where we prevent model in subsequent tasks to
diverge from the previous by penalising $\mathcal{L}_{1}$ or $\mathcal{L}_{2}$
norm.

We compare the regularisation approach with two rehearsal-based methods. In
the first one we employ a simple buffer-based rehearsal where we store a
subset of previous data examples in a buffer and use them together with new
data samples when retraining a model on new tasks. In the second one, taking
advantage of the generative model we continually train, we propose to generate
examples from previous tasks in order to use them as rehearsal samples in a
generative replay approach.

##  4 Experiments

We evaluate the performance of BFNs in Continual Learning using the standard
MNIST dataset and our new scenario with tabular data on US flights connections
in 2013 (CC0:Public Domain,, 2013). One of the most common setting in which we
are able to assess the continual learning capabilities of the proposed model
is to split the training dataset into disjoint chunks and perform the training
in a sequential way. In Class-Incremental Learning set up, each task often
contains the same . Each task $\tau_{i}$ is associated with a dataset
$\mathcal{D}_{i}$, and the objective is to model the distribution of
$\mathcal{D}_{i}$.

In particular, we split the MNIST in CIL setting $5\times 2$ by dividing it
into 5 tasks, each binary classification of two consecutive digits. Following
(Alex Graves, Rupesh Kumar Srivastava, Timothy Atkinson, and Faustino Gomez.,
), we also binarise the images. In flight dataset we divide group flights by
month of the journey obtaining 12 tasks.

###  4.1 Image dataset

In Figure 2, we present the results of our experiments with MNIST dataset. To
measure the catastrophic forgetting, after each task, we generate 1000
examples and report the share of each class as measured by the externally
trained classifier. As visible, in finetuning (without any CL strategy), we
can observe catastrophic forgetting as with each new task model abruptly
forgets how to generate examples from the previous classes. On the other hand,
both: buffer-based and generative-based replay prevent catastrophic
forgetting, as even after the last task, we can observe some generations of
classes from the first task. For qualitative analysis we provide some samples
in Figure 4.

Figure 2: Results of classification of the generated images (a) left: using
finetuning (b) middle: memory-based (c) right: generative-based method. (d)
The colours corresponds to percentage share of consecutive digits across the
sequential training.

##  5 Tabular data

To evaluate the performance of BFNs in modelling categorical data in the
continual learning scenario, we refer to the problem of tabular data
modelling. The results are presented in Figure 3. We inspect the test loss
metric as a proxy for model surprise of provided data.

Figure 3: Results of loss applied on test images (a) left: using finetuning
(b) right: generative-replay method. Loss induced by BFNs measures bits by
dimension and offers (negative log-) likelihood estimation interpretation. On
$Y$ axis incrementally we are proceed with tasks, whereas values on $X$ axis
corresponds to data batches from indicated tasks.

##  6 Conclusion, Limitations and Future work

Bayesian Flow Networks (BFNs) are exciting family of generative models that
are able to deal with various type of data. In this work we highlight that
modelling data distribution parameters does not prevent those models from
catastrophic forgetting. However, BFNs can benefit from known CL strategies
such as rehearsal and generative replay. In our future works, we plan to
explore further how we can benefit from the sweet combination of Bayesian
update, with neural modelling in order to continually adjust parameters of the
data distrubution.

## References

  * [1] Alessandro Achille, Tom Eccles, Loic Matthey, Chris Burgess, Nicholas Watters, Alexander Lerchner, and Irina Higgins. Life-long disentangled representation learning with cross-domain latent homologies. Advances in Neural Information Processing Systems, 31, 2018.
  * [2] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis.Advances in Neural Information Processing Systems, 34, 2021.
  * [3] Nan Ding, Xi Chen, Tomer Levinboim, Sebastian Goodman, and Radu Soricut. Bridging the gap between practice and pac-bayes theory in few-shot meta-learning, 2021.
  * CC0:Public Domain, [2013] CC0: Public Domain. Flights dataset in 2013. https://www.kaggle.com/datasets/mahoora00135/flights/data, 2013.
  * [5] Evgenii Egorov, Anna Kuzina, and Evgeny Burnaev. Boovae: Boosting approach for continual learning of vae. Advances in Neural Information Processing Systems, 34, 2021.
  * [6] Alex Graves, Rupesh Kumar Srivastava, Timothy Atkinson, and Faustino Gomez. Bayesian flow networks, 2023.
  * [7] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  * [8] Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation.Journal of Machine Learning Research, 23(47):1–33, 2022.
  * [9] Chin-Wei Huang, Jae Hyun Lim, and Aaron C Courville. A variational perspective on diffusion-based generative models and score matching. Advances in Neural Information Processing Systems, 34, 2021.
  * [10] Xin Huang, Ashish Khetan, Milan Cvitkovic, and Zohar Karnin.  Tabtransformer: Tabular data modeling using contextual embeddings, 2020.
  * [11] Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 21696–21707. Curran Associates, Inc., 2021.
  * [12] Marc Masana, Xialei Liu, Bartłomiej Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost van de Weijer. Class-incremental learning: survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  * [13] Cuong V Nguyen, Yingzhen Li, Thang D Bui, and Richard E Turner.  Variational continual learning.  In International Conference on Learning Representations, 2018.
  * [14] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation, 2015.
  * [15] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256–2265. PMLR, 2015.
  * [16] Yang Song and Stefano Ermon.  Generative modeling by estimating gradients of the data distribution.  Advances in Neural Information Processing Systems, 32, 2019.
  * [17] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.  Score-based generative modeling through stochastic differential equations.  In International Conference on Learning Representations, 2020.
  * [18] Gido M Van de Ven and Andreas S Tolias.  Three scenarios for continual learning.  arXiv preprint arXiv:1904.07734, 2019.
  * [19] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need, 2023.
  * [20] Hao Wang and Dit-Yan Yeung.  Towards bayesian deep learning: A framework and some existing methods.  IEEE Transactions on Knowledge and Data Engineering, 28(12):3395–3408, 2016.
  * [21] Hao Wang and Dit-Yan Yeung.  A survey on bayesian deep learning.  ACM computing surveys (csur), 53(5):1–37, 2020.
  * [22] Chenshen Wu, Luis Herranz, Xialei Liu, Yaxing Wang, Joost van de Weijer, and Bogdan Raducanu.  Memory replay gans: Learning to generate new categories without forgetting.  In NeurIPS, 2018.
  * [23] Michał Zając, Kamil Deja, Anna Kuzina, Jakub M. Tomczak, Tomasz Trzciński, Florian Shkurti, and Piotr Miłoś.  Exploring continual learning of diffusion models, 2023.

## Appendix

###  6.1 Extension

For discrete or discretised case, we assume that we model the distributions by
$\alpha$ parameters, which is the vector of length equal to the number of
possible values, i.e. $\alpha_{i}$ is the probability of generating $i^{th}$
value. For more technicalities, we refer to chapter 5 in the original paper.

In order to obtain closed-form update formulas, under mind conditions we set
the accuracy schedule $\beta$ which is just compound sum of accuracy rates,
i.e. $\beta(t)\doteq\int_{t^{\prime}=0}^{t}\alpha_{t^{\prime}}dt^{\prime}$ as
in diffusion models. Then, through the series of mathematical derivations, we
are able to calculate the overall Bayesian Flow update given the time step as
in equation 205 in the original paper.

###  6.2 Additional results - image generations

Figure 4: Results of image generations for (a) finetuning; (b) memory-based
approach; (c) generative-based approach. Each row corresponds to the
consecutive task.

####  6.2.1 Additional results – regularisation-based approach.

We also notice the quality degradation for regularisation-based approaches. We
observed two cases, either the $\lambda$ coefficient corresponding to the
weight of regularisation component is too small and therefore not significant,
or there is quality degradation for image generation.

Figure 5:  Results of image generation for regularisation-based approach. For
each task, two samples were generated.

##### Experiment details

For work on MNIST, we apply standard U-Net, widely applied for diffusion
models with 64 model channels, 2 resnet blocks and attention resolution
32,16,8. The size of the model is 6.1M parameters.

For work on tabular data (5 categorical, 9 numerical columns), we use
TabTransformer with dimension 32, depth 6 and 8 heads. Overall models has
around 600k learnable parameters. There are roughly 320k data points.


# SAR Image Change Detection Based on Multiscale Capsule Network

Yunhao Gao, Feng Gao, Junyu Dong, Heng-Chao Li This work was partially
supported by the National Natural Science Foundation of China (No. 41606198,
41576011).Y. Gao, F. Gao, and J. Dong are with the Qingdao Key Laboratory of
Mixed Reality and Virtual Ocean, School of Information Science and
Engineering, Ocean University of China, Qingdao 266100, China. H.-C. Li is
with the Sichuan Provincial Key Laboratory of Information Coding and
Transmission, Southwest Jiaotong University, Chengdu 610031, China.

###### Abstract

Traditional change detection methods based on convolutional neural networks
(CNNs) face the challenges of speckle noise and deformation sensitivity for
synthetic aperture radar images. To mitigate these issues, we proposed a
Multiscale Capsule Network (Ms-CapsNet) to extract the discriminative
information between the changed and unchanged pixels. On the one hand, the
capsule module is employed to exploit the spatial relationship of features.
Therefore, equivariant properties can be achieved by aggregating the features
from different positions. On the other hand, an adaptive fusion convolution
(AFC) module is designed for the proposed Ms-CapsNet. Higher semantic features
can be captured for the primary capsules. Feature extracted by the AFC module
significantly improves the robustness to speckle noise. The effectiveness of
the proposed Ms-CapsNet is verified on three real SAR datasets. The comparison
experiments with four state-of-the-art methods demonstrated the efficiency of
the proposed method. Our codes are available at
https://github.com/summitgao/SAR_CD_MS_CapsNet.

###### Index Terms:

Change detection, multiscale capsule network, synthetic aperture radar, deep
learning.

##  I Introduction

Synthetic aperture radar (SAR) imaging acquisition technologies have been
developed rapidly. Plenty of multitemporal SAR images are available to monitor
the changed information of the earth. Therefore, SAR image change detection
has drawn increasing attention recently. Researchers have designed a variety
of SAR change detection methods for ecological surveillance, disaster
monitoring [1], and urban planning [2].

Although plenty of techniques have been proposed [3], SAR image change
detection is a still challenging task. Image quality is deteriorated by
speckle noise which hinders the meticulous interpretation of SAR data. Many
methods are implemented to address the issue of speckle noise. They are
usually comprised of three steps: 1) image coregistration, 2) difference image
(DI) generation, and 3) DI classification [4]. Image coregistration is a
fundamental task for SAR image change detection. The spatial correspondences
between multitemporal SAR images can be established. In the second step, the
DI is commonly generated by the log-ratio, Gauss-ratio [5], and neighborhood-
ratio [6] operators. For the DI classification step, most researches are
devoted to building a robust classifier. It is a non-trivial task since a
powerful classifier directly determines the precision of change detection.

Many researchers are dedicated to developing powerful classifiers for change
detection. Li et al. [7] designed two-level clustering algorithm for
unsupervised change detection. In [8], local-neighborhood information is
embedded in the clustering objective function to improve the change detection
performance. Gong et al. [9] developed an improved Markov random field (MRF)
based on fuzzy $c$-means (FCM) clustering to suppress the speckle noise. In
[4], a deep belief network (DBN) was employed for SAR image change detection.
Some of the most noteworthy approaches also achieved breakthrough relying on
the level-set algorithm [10], stacked autoencoder (SAE) [11], PCANet [12].

In recent years, the convolutional neural network (CNN) has greatly boosted
the performance of many visual tasks [13]. It is widely acknowledged that CNN
is capable of robust feature learning. Inspired by these achievements, CNN has
been successfully applied in SAR change detection [14]. Zhan et al. [15]
refined a deep siamese CNN to measure the similarity of patch-pair from SAR
images. In [16], transferred deep learning was applied to sea ice SAR image
change detection based on CNN. Liu et al. [17] proposed an elegant local
restricted CNN (LR-CNN) for polarimetric SAR change detection. Although CNN-
based methods have achieved excellent performance in SAR image change
detection, the accuracy sometimes deteriorates under the case of
transformation, such as tilts and rotations. Specifically, CNN is incapable of
modeling the positional relationship among ground objects.

More recently, the capsule network (CapsNet) was designed for assigning parts
to wholes [18]. An activity vector from capsules represents the entity
instantiation parameters such as pose, texture, and deformation. The existence
of entities is expressed by the length of instantiation parameters. To
propagate information, dynamic routing mechanism is utilized to send activity
vectors to the appropriate layer above. It is empirically verified that the
CapsNet is effective for image segmentation and classification tasks. A few
attempts based on the CapsNet have been applied for remote sensing images
analysis, and obtain promising results. In [19], a new CNN architecture based
on capsule networks is proposed for hyperspectral image (HSI) classification.
Zhu et al. [20] proposed a 3D deep capsule network for HSI classification. The
local connection and weight sharing strategy greatly reduces the number of
parameters. As far as we know, the literature on the CapsNet-based SAR change
detection is very sparse.

We argue that the weakness of existing SAR image change detection approaches
mainly comes from two aspects: One is the correlation of features from
different positions fails to be modeled effectively. The other one lies in the
intrinsic speckle noise in SAR images. With the aforementioned challenges, we
proposed a Multiscale Capsule Network (Ms-CapsNet) to extract the
discriminative information between multitemporal SAR images. To enhance the
feature correlations, we introduced the capsule module to achieve the
equivariant properties. The proposed Ms-CapsNet provides a group of
instantiation parameters which can effectively capture features from different
positions. To tackle the problem of speckle noise, an adaptive fusion
convolution (AFC) module is designed to convert pixel intensities to
activities of the local features. Accordingly, local features are noise
robustness. Extensive experiments on three real datasets are conducted to show
the superiority of our proposed method over four state-of-the-art works.

For clarity, the main contributions are summarized as follows:

  * •

The multiscale capsule network (Ms-CapsNet) is designed to extract the
features from different positions. And equivariant properties can be achieved
by capsule module. Logically, the demand for training samples is reduced by
the correlative and complete information to some extent.

  * •

A simple yet effective AFC module is designed, which can effectively convert
pixel intensities to activities of local features. The AFC module extracts
higher semantic features and emphasizes the meaningful one through attention-
based strategy. Therefore, the activity local features become more noise
robustness and immediately accepted as the input of the primary capsule.

  * •

Extensive experiments have been implemented on three SAR datasets to validate
the effectiveness of the proposed method. Moreover, we have released the codes
and setting to facilitate future researches in multitemporal remote sensing
image analysis.

##  II Methodology

Figure 1: Illustration of the proposed change detection method based on
multiscale capsule network (Ms-CapsNet). First, image patches are fed into AFC
module for higher semantic features. Then, multiscale primary capsules layers
are adopted with kernel $3\times 3$ and $5\times 5$ to obtain primary capsules
$v_{p1}$ and $v_{p2}$. Later $v_{p1}$ and $v_{p2}$ are input to conv-capsule
layer and class capsule layer, respectively. Finally, fuse the output of class
capsule layers to calculate the final result.

The proposed method is illustrated in Fig. 1. A difference image (DI) is first
generated by the log-ratio operator. Hierarchical FCM clustering [12] is
employed to select reliable training samples for Ms-CapsNet. Finally, pixels
in the DI are classified by the trained Ms-CapsNet to obtain the final change
map.

In our implementations, image patches are extracted from DI with the size of
$r\times r$. The proposed Ms-CapsNet is comprised of AFC and capsule modules.
The AFC module is used to convert pixel intensities to high semantic features
through which the speckle noise is suppressed to some extent. The capsule
module is utilized to activate high semantic features. In the following
subsections, we will describe both modules in detail.

###  II-A Adaptive Fusion Convolution Module

As shown in Fig. 2, the proposed AFC module is utilized to encode the input.
Recently, the self-attention mechanism has been employed for visual structure
understanding. The long-range dependencies can be captured by the self-
attention mechanism. Inspired by Hu’s work [21], we design a simple yet
effective AFC module based on channel-wise attention (CA). First, atrous
convolution (Conv 1-1, Conv 1-2 and Conv 1-3 with kernel size $3\times 3$) is
adopted with different dilation rates, which are set to $1$, $2$ and $3$ to
capture multiscale features. Then, the multiscale features are aggregated by
CA-based feature fusion.

Figure 2: Illustration of the Adaptive Fusion Convolution (AFC) module.

The input features $\textbf{F}_{in}\in\mathbb{R}^{w_{0}\times w_{0}\times
c_{0}}$ from atrous convolution are fed into CA. Then, global average pooling
(GAP) squeezes $\textbf{F}_{in}$ in the spatial domain to obtain
$\textbf{F}_{avg}\in\mathbb{R}^{1\times 1\times c_{0}}$. Then, 1D-Convolution
(1D-Conv) is employed to explore the channel relationship of
$\textbf{F}_{avg}$. After the $Sigmoid$ function, a channel weighting-based
vector $M$ can be obtained. Finally, the channel weighting-based feature
$\textbf{F}_{out}$ can be computed as $\textbf{F}_{out}=M\otimes F_{in}$,
where $\otimes$ denotes channel-wise multiplication. Therefore, the channel
weighting-based features from Conv 1-1, Conv 1-2 and Conv 1-3 are
$\textbf{F}_{1}$, $\textbf{F}_{2}$ and $\textbf{F}_{3}$. We fused the features
by pixel-wise summation:

$\textbf{F}=D_{1}(\textbf{F}_{1})+D_{2}(\textbf{F}_{2})+D_{3}(\textbf{F}_{3}),$
(1)

where F represents the fused features, $D_{1}$, $D_{2}$, and $D_{3}$ are
dimension matching functions which are operated by $1\times 1$ convolution.

###  II-B Capsule Module

The capsule module is a neural network comprised of the primary capsule layer,
the conv-capsule layer, and the fully-connected layer, as illustrated in Fig.
1.

####  II-B1 Primary Capsule Layer

This layer is employed to extract the low-level features from multi-
dimensional entities through convolutional-like operation with kernel size
$k\times k$. Different from traditional convolution, multiple feature maps
will be obtained instead of one. The primary capsule layer first receive the
feature map $F\in\mathbb{R}^{w\times w\times c}$ from the AFC module. Then
convolutional-like operation and $squashing$ activation function are employed
to obtain the output capsules $v_{p}$. The $squashing$ activity function is
denoted as:

$v=\frac{\|s\|^{2}}{1+\|s\|^{2}}\frac{s}{\|s\|},$ (2)

where $s$ is the total input and $v$ is the vector output of capsule. In the
primary capsule layer, the size of the output capsules $v_{p}$ is $w_{1}\times
w_{1}\times n\times d$, where $n$ is the number of feature maps, $n\times d=c$
and $d=8$. The $[w_{1}\times w_{1}]$ grid is shared weights. In other words,
we obtain $[w_{1}\times w_{1}\times n]$ 8D vectors in total primary capsules.
In our implementations, multi-scale information is taken into account. Two
primary capsule layers are employed with kernel size $k=3$ and $k=5$,
respectively. Therefore, multi-scale feature representation can be obtained.
Feature vectors from two scales are denoted by $v_{p1}$ and $v_{p}$,
respectively.

####  II-B2 Conv-Capsule Layer

This layer uses local connections and the shared transformation matrix to
reduce the number of parameters to some extent [20]. Conv-capsule layer uses
the dynamic routing strategy to update the coupling coefficient $c$. The
connection (transformation matrix) between the primary capsule layer and the
conv-capsule layer is $W$, and the transformation matrix $W$ is also shared in
each grid. Therefore, the output $v_{c}$ of the conv-capsule layer can be
expressed as:

$v_{c}=squashing(\sum c\cdot u),$ (3)

where $c$ is the coupling coefficient, $u=W\cdot v_{p}$. $v_{p}$ is the output
of the primary capsule layer. For dynamic routing, we first set the agreement
$b$ to $0$. The coupling coefficient $c$ can be calculated by $c=softmax(b)$.
That is to say, we update $b$ to calculate the latest coupling coefficient
$c$. In addition, the update process of $b$ can be expressed as $b\leftarrow
b+u\cdot v_{c}$. The detailed descriptions of the dynamic routing can be found
in [18].

####  II-B3 Class Capsule Layer

The class capsule layer can be considered as a fully connected layer. Dynamic
routing mechanism is still used for coupling coefficient updating. In this
layer, multiscale activity vectors $v_{o1}\in\mathbb{R}^{2\times 16}$ and
$v_{o2}\in\mathbb{R}^{2\times 16}$ from class capsule layer are fused by
summation $v_{o}=v_{o1}\oplus v_{o2}$. Then the vector norm is calculated to
measure the probability of classes. The loss function of Ms-CapsNet can be
defined as:

$\begin{split}L=T_{k}\max(0,m^{+}-&\|v_{o}\|)^{2}+\\\
&\lambda(1-T_{k})\max(0,\|v_{o}\|-m^{-})^{2}.\end{split}$ (4)

Here $T_{k}=1$ when the label $k$ is presented ($k=0$ means the unchanged
class, $k=1$ means the changed class). $\lambda=0.5$ is used to constrain the
length of the activity vector of the initial class capsule. If there is a
changed class object in the image, the class capsule of the changed class
should output a vector with a length of at least $m^{+}=0.9$. On the contrary,
if there is no object of the changed class, a vector with a length less than
$m^{-}=0.1$ will be output from the class capsule. Then, the final change map
can be calculated by pixel-wise classification.

##  III Experimental Results and Analysis

In this section, we first describe the datasets and evaluation criteria in our
experiments. Next, an exhaustive investigation of several vital parameters on
the change detection performance is presented. Finally, we conduct extensive
experiments to verify the effectiveness of the proposed method.

Figure 3: Visualized results of different change detection methods on three
datasets. (a) Image captured at $t_{1}$. (b) Image captured at $t_{2}$. (c)
Ground truth change map. (d) Result by PCANet. (e) Result by MLFN. (f) Result
by DCNN. (g) Result by LR-CNN. (h) Result by the proposed Ms-CapsNet.

###  III-A Dataset and Evaluation Criteria

To verify the effectiveness of the proposed Ms-CapsNet, we employed Ms-CapsNet
on three multitemporal SAR datasets acquired by different sensors. The first
dataset is the Sulzberger dataset. It is captured at Sulzberger Ice Shelf by
Envisat satellite of the European Space Agency on March 11 and 16, 2011,
respectively. The size of the dataset is $256\times 256$ pixels, as
illustrated in the first row of Fig. 3 (a)-(c). The other two datasets named
Yellow River I and Yellow River II datasets, are captured at the Yellow River
Estuary by Radarsat-2 in June 2008 and June 2009, respectively. Their sizes
are $257\times 289$ and $306\times 291$ pixels in the second and third rows of
Fig. 3 (a)-(c), respectively. It is very challenging to perform accuracy
change detection on the Yellow River dataset since the speckle noise is much
stronger. Geometric corrections have been performed on these datasets, and the
ground truth images were manually annotated carefully with expert knowledge.

In the following experiments, the proposed Ms-CapsNet is compared with four
closely related methods, including the PCANet [12], the transferred multilevel
fusion network (MLFN) [16], the deep convolutional neural networks (DCNN)
[22], the CNN with local spatial restrictions (LR-CNN) [17]. To verify the
effectiveness of the proposed Ms-CapsNet, false positives (FP), false
negatives (FN), percentage correct classification (PCC), overall errors (OE),
and Kappa coefficient (KC) are adopted as the evaluation criteria.

###  III-B Parameters Analysis of the Proposed Ms-CapsNet

####  III-B1 Analysis of the Patch Size

The patch size $r$ represents the scale of spatial neighborhood information.
Fig. 4 shows the relationship between $r$ and PCC, where $r$ is changed from 5
to 17. According to Fig. 4, the PCC values increase first and then tend to be
stable. It is evident that the contextual information is important for change
detection. However, a large patch size will increase the computational cost.
Therefore, we choose $r=9$ for the Sulzberger and Yellow River I datasets, and
$r=11$ for the Yellow River II dataset.

Figure 4: Relationship between the PCC value and patch size.

####  III-B2 Analysis of The Training Sample Numbers

TABLE I: Relationship between the PCC value and the number of training samples. Method | PCC of different training samples number (%)
---|---
200 |  400 |  600 |  800 |  1000
PCANet [12] |  91.44 |  92.12 |  93.34 |  93.88 |  94.26
MLFN [16] |  94.20 |  94.72 |  95.01 |  95.24 |  95.48
DCNN [22] |  92.51 |  92.58 |  92.66 |  93.07 |  93.81
LR-CNN [17] |  93.81 |  94.24 |  94.68 |  95.00 |  95.32
Ms-CapsNet |  94.54 |  94.98 |  95.28 |  95.45 |  95.78

Table I compares the proposed Ms-Capsule with other methods when considering
different number of training samples, i.e, $200$, $400$, $600$, $800$, and
$1000$. We can observe that the accuracy of other methods drops sharply when
the number of samples is less than $600$. Especially, DCNN and LR-CNN depend
heavily on large volumes of training data, and few training samples will lead
to overfitting which degrades the performance. In summary, the PCC value of
the proposed method is less afflicted with the training sample numbers. It is
because the feature spatial correlations can reduce the dependence on training
samples to some extent.

###  III-C Change Detection Results on Three Datasets

TABLE II: Change detection results on three datasets. Method | Sulzberger dataset | Yellow River I dataset | Yellow River II dataset
---|---|---|---
FP | FN | OE | PCC | KC | FP | FN | OE | PCC | KC | FP | FN | OE | PCC | KC
PCANet [12] | 1410 | 1437 | 2847 | 95.66 | 88.63 | 1063 | 362 | 1425 | 98.40 | 86.47 | 2435 | 1533 | 3968 | 94.66 | 82.43
MLFN [16] | 616 | 664 | 1280 | 98.05 | 94.89 | 721 | 863 | 1584 | 98.22 | 83.82 | 1544 | 1972 | 3516 | 95.27 | 83.82
DCNN [22] | 312 | 1467 | 1779 | 97.29 | 92.74 | 698 | 922 | 1620 | 98.18 | 83.33 | 1231 | 2370 | 3601 | 95.15 | 83.08
LR-CNN [17] | 1198 | 680 | 1878 | 97.13 | 92.58 | 1118 | 423 | 1541 | 98.27 | 85.36 | 1923 | 1460 | 3383 | 95.45 | 84.83
Ms-CapsNet | 425 | 779 | 1204 | 98.16 | 95.16 | 468 | 407 | 875 | 99.02 | 91.22 | 1173 | 1798 | 2971 | 96.00 | 86.25

In this subsection, the proposed method is compared with four closely related
methods. The quantitative results of different methods on three datasets are
displayed in Table II. Fig. 3 exhibits the visual results of the proposed Ms-
CapsNet together with all competitors.

Fig. 3(d)-(h) present the change maps corresponding to the experiments
reported in Table II. On the Sulzberger dataset (the first row of Fig. 3), the
result of PCANet tends to be rather noisy, and it is afflicted with high FP
value. Although other methods generally performed well, the results are
deteriorated due to higher OE values. The proposed Ms-CapsNet exhibits less
misclassified pixels and obtains the best PCC and KC values.

On the Yellow River I and II datasets (the second and third rows of Fig. 3),
we can observe that the proposed Ms-CapsNet achieves at least 0.5% improvement
over other compared methods. Considering that the interference of different
characteristics of speckle noise weakens the model performance, the proposed
method is relatively noise robust. The PCANet suffers from high FP value, and
there are many noisy regions in the generated change maps. LR-CNN performs
better since local spatial restrictions can balance the influence of local
noise. CNN-based methods can suppress noise interference to some extent
through deep feature representation. However, relatively high OE values are
still obtained. In general, the proposed Ms-CapsNet exhibits the best
performance according to Table II and Fig. 3. It reveals that the proposed Ms-
CapsNet benefits from the spatial relation exploration.

##  IV Conclusion

In this paper, we proposed a new multiscale capsule network (Ms-CapsNet) for
SAR image change detection. The proposed method benefits from two aspects:
First, in order to enhance the spatial feature correlations, multiscale
capsule module is utilized to model the spatial relationship of features
between one object and another. Equivariant properties can be achieved by
aggregating the feature from different positions. Further, we design an AFC
module to alleviate the interference of speckle noise. The module can
effectively convert pixel-wise intensities to activity local features.
Extensive experiments are conducted on three real SAR datasets, and the
experimental results demonstrated the superior performance of the proposed Ms-
CapsNet.

## References

  * [1] D. Burnner, G. Lemonie, and L. Bruzzone, “Earthquake damage assessment of buildings using VHR optical and SAR imagery,” _IEEE Trans. Geosci. Remote Sens._ , vol. 48, no. 5, pp. 2403–2420, May 2010.
  * [2] S. Quan, B. Xiong, D. Xiang, L. Zhao, S. Zhang, and G. Kuang, “Eigenvalue-based urban area extraction using polarimetric SAR data,” _IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens._ , vol. 11, no. 2, pp. 458–471, Feb. 2018.
  * [3] R. J. Radke, S. Andra, O. Al-Kofahi, and B. Roysam, “Image change detection algorithms: A systematic survey,” _IEEE Trans. Image Process._ , vol. 14, no. 3, pp. 294–307, Mar. 2005.
  * [4] M. Gong, J. Zhao, J. Liu, Q. Miao, and L. Jiao, “Change detection in synthetic aperture radar images based on deep neural networks,” _IEEE Trans. Neural Netw. Learn. Syst._ , vol. 27, no. 1, pp. 125–138, Jan. 2016.
  * [5] B. Hou et al., “Unsupervised change detection in SAR image based on gauss-log ratio image fusion and compressed projection,” _IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens._ , vol. 7, no. 8, pp. 3297–3317, 2014.
  * [6] M. Gong, Y. Cao, and Q. Wu, “A neighborhood-based ratio approach for change detection in SAR images,” _IEEE Geosci. Remote Sens. Lett._ , vol. 9, no. 2, pp. 307–311, 2012.
  * [7] H. Li, T. Celik, N. Longbotham, and W. J. Emery, “Gabor feature based unsupervised change detection of multitemporal SAR images based on two-level clustering,” _IEEE Geosci. Remote Sens. Lett._ , vol. 12, no. 12, pp. 2458–2462, Dec. 2015.
  * [8] L. Jia, M. Li, P. Zhang, Y. Wu, and H. Zhu, “SAR image change detection based on multiple kernel k-means clustering with local-neighborhood information,” _IEEE Geosci Remote Sens. Lett._ , vol. 13, no. 6, pp. 856–860, Jun. 2016.
  * [9] M. Gong, Z. Zhou, and J. Ma. “Change detection in synthetic aperture radar images based on image fusion and fuzzy clustering,” _IEEE Trans. Image Process._ , vol. 21, no. 4, pp. 2141–2151, Apr. 2012.
  * [10] Y. Bazi, F. Melgani, and H. D. Al-Sharari, “Unsupervised change detection in multispectral remotely sensed imagery with level set methods,” _IEEE Trans. Geosci. Remote Sens._ , vol. 48, no. 8, pp. 3178–3187, Aug . 2010.
  * [11] P. Planinsic and D. Gleich, “Temporal change detection in SAR images using log cumulants and stacked autoencoder,” _IEEE Geosci. Remote Sens. Lett._ , vol. 15, no. 2, pp. 297–301, Feb. 2018.
  * [12] F. Gao, J. Dong, B. Li, and Q. Xu, “Automatic change detection in synthetic aperture radar images based on PCANet,” _IEEE Geosci. Remote Sens. Lett._ , vol. 13, no. 12, pp. 1792–1796, Dec. 2016.
  * [13] R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in _Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit. (CVPR)_ , Jun. 2014, pp. 580–587.
  * [14] Q. Liu, R. Hang, H. Song, and Z. Li, “Learning multiscale deep features for high-resolution satellite image scene classification,” _IEEE Tran. Geosci. Remote Sens._ , vol. 56, no. 1, pp. 117–126, Jan. 2018.
  * [15] Y. Zhan, K. Fu, M. Yan, X. Sun, H. Wang, and X. Qiu, “Change detection based on deep siamese convolutional network for optical aerial images,” _IEEE Geosci. Remote Sens. Lett._ , vol. 14, no. 10, pp. 1845–1849, Oct. 2017.
  * [16] Y. Gao, F. Gao, J. Dong, and S. Wang. “Transferred deep learning for sea ice change detection from synthetic aperture radar Images,” _IEEE Geosci. Remote Sens. Lett._ , vol. 16, no.10, pp. 1655–1659, Oct. 2019.
  * [17] F. Liu, L. Jiao, X. Tang, S. Yang, W. Ma, and B. Hou, “Local restricted convolutional neural network for change detection in polarimetric SAR images,” _IEEE Trans. Neural Netw. Learn. Syst._ , vol. 30, no. 3, pp. 1–16, Mar. 2019.
  * [18] S. Sabour, N. Frosst, and G. E. Hinton, “Dynamic routing between capsules,” _in Proc. Adv. Neural Inf. Process. Syst._ , 2017, pp. 3859–-3869.
  * [19] M. E. Paoletti et al., “Capsule networks for hyperspectral image classification,” _IEEE Trans. Geosci. Remote Sens._ , vol. 57, no. 4, pp. 2145–2160, Apr. 2019.
  * [20] K. Zhu et al., “Deep convolutional capsule network for hyperspectral image spectral and spectral-spatial classification,” _Remote Sens._ , vol. 11, no. 3, pp. 1–28, Mar. 2019, Art. no. 223.
  * [21] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” _in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR)_ , Jun. 2018, pp. 7132–7141.
  * [22] W. Song, S. Li, L. Fang, and T. Lu, “Hyperspectral image classification with deep feature fusion network,” _IEEE Trans. Geosci. Remote Sens._ , vol. 56, no. 6, pp. 3173–3184, Jun. 2018.


$T_\text{eff,~iso}$     $\mathcal{U}_\text{MIST}(3500~\text{K}, ~6000~\text{K})$

$\log g_\text{iso}$     $\mathcal{U}_\text{MIST}(0.0, ~0.4)$

$\text{[Fe/H]}_\text{iso}$     $\mathcal{U}_\text{MIST}(-4.0, ~-1.0)$

2lFinal Samples with Scatter

$T_\text{eff}$     $\mathcal{N}(T_\text{eff,~iso}, ~250~\text{K})$

$\log g$     $\mathcal{N}(\log g_\text{iso}, ~0.25)$

$\text{[Fe/H]}$     $\mathcal{N}(\text{[Fe/H]}_\text{iso}, ~0.25)$

$v_\text{micro}$     $\mathcal{N}(2.478 - 0.325\log g, ~0.25)$

$\text{[X}_1\text{/Fe]}$     $\mathcal{U}(-1.00, ~1.00)$

$\text{[X}_2\text{/Fe]}$     $\mathcal{U}(-0.50, ~0.50)$

$\text{[X}_3\text{/Fe]}$     $\mathcal{U}(-0.25, ~1.00)$

Distributions from which the training label sets are drawn. , , and [Fe/H] are drawn initially from the MIST isochrone set described in Table <ref> before additional scatter is applied.
$\text{X}_1$ includes C, N, and O. $\text{X}_2$ includes Na, Sc, V, Cr, Mn, Co, Ni, Cu, Zn, Ga, Sr, Y, Zr, Ba, and La. $\text{X}_3$ includes Mg, Al, Si, K, Ca, Ti, Ce, Pr, Nd, Sm, Eu, Gd, Dy, Ho, Er, Os, and Th.

Ab initio spectra are generated using the same method described in Ting et al., 2019, which we summarize here. For each of the 25000 sets of stellar labels, we compute 1D LTE model atmospheres using the  code maintained by R. Kurucz [Kurucz, 1970, Kurucz, 1993, Kurucz, 2005, Kurucz, 2013, Kurucz, 2017, Kurucz & Avrett, 1981]. We adopt Solar abundances from Asplund et al., 2009 and the standard mixing length theory with a mixing length of 1.25 and no overshoot for convection. After the model atmosphere converges, we use the  radiative transfer code (also maintained by R. Kurucz) to produce its normalized spectrum at a nominal resolution of $R=300000$.

For a little less than $\sim$20% of the labels, the stellar atmosphere and/or spectrum fails to converge. These failed models predominantly belong to stellar atmospheres with very low metallicities ($\text{[Fe/H]}\lesssim-3.0$). It is possible that better initialization of low-metallicity atmospheres might improve convergence, but we leave this to a future study.

The $\sim$20500 successfully generated spectra are then continuum normalized using the theoretical continua from . Lastly, the normalized spectra are convolved and sub-sampled down to the highest spectral resolution and wavelength sampling present in our archival data ($R=86600$ and d$v=1.17$km/s ).

§.§ Training Procedure

We implement our adoption of  using , a powerful and flexible Python machine learning framework, and  [<https://www.pytorchlightning.ai/>], a lightweight wrapper designed to streamline the development and training of PyTorch models.

As with many machine learning techniques, it is helpful to scale the input labels so that they all share a similar dynamic range of order unity with zero mean. To do so, we normalize all stellar labels according to

\begin{equation}
\theta_{*,i}^\prime = \frac{\theta_{*,i}-\theta_{*,i,\text{min}}}{\theta_{*,i,\text{max}}-\theta_{*,i,\text{min}}} - 0.5,
\end{equation}

where $\theta_{*,i,\text{min}}$ and $\theta_{*,i,\text{max}}$ are the minimum and maximum values of each label, $i$, included in the training set.
For clarity, we drop the prime notation throughout the rest of this work and convert back to physical units when reporting results.

For each model/training set, we train directly on 80% of the successfully generated spectra ($\sim$16000) and validate with the remaining 20% ($\sim$4000). Training is performed iteratively in batches of 512 spectra using a rectified Adam optimizer (with a learning rate of 0.0001) to minimize the model's L1 loss (i.e., the mean absolute error). Though unknown to the optimization, the L1 loss is also calculated on the cross-validation dataset each epoch. Training is halted after 2000 epochs without improvement of the best L1 validation loss, at which point the model that minimized the L1 validation loss is chosen as the final model. Training of the model was completed in $\sim$150 GPU hours after $\sim$24000 epochs.

§.§ Accuracy

We determine the internal accuracy (i.e., the median interpolation error; MIE) of  as is in [Ting et al., 2019]. Using the trained neural networks, we generate spectra for each set of stellar labels in the cross-validation dataset and compare to the original ab initio spectra generated with  and . The median interpolation error is thus

\begin{equation}
\sigma_\text{MIE} = \text{Med}(|f_\lambda(\theta_{*,\text{valid}})-f_{\lambda, \text{valid}}|).
\end{equation}

Figure <ref> graphically presents how accurately  interpolates the synthetic spectra. In the top left panel, we show the distribution of interpolation errors for our cross-validation set, taking the median over all wavelength pixels. We find that for $\sim$85% of spectra, the MIE is $\lesssim$0.1%, though the long tail of the distribution indicates that some spectra have errors as high as $\sim$1%. Unlike Ting et al., 2019 who find larger MIE for cooler stars, we find that this long tail towards higher errors corresponds to stars with higher metallicities ($\text{Fe/H} > -2$; red histogram). Because the stars analyzed in this paper all have $\text{[Fe/H]}\lesssim-2.4$, the adopted MIE is likely on the conservative end.

In the bottom panel, we show the pixel-by-pixel MIE for the entire wavelength range of the model, taking the median over all cross-validation spectra; this is the $\sigma_\text{MIE}$ adopted in Equation <ref>. Typical pixel-by-pixel MIEs for the  are 0.01–0.1% for $\lambda>4500$ and 0.1–0.5% for $\lambda<4500$. The MIE is generally larger in the blue due to the higher density of absorption lines and the presence of complicated molecular features. The MIE is also larger in the proximity of strong absorption features like the the Balmer lines. We believe this to be the reason why higher-metallicity stars in our cross-validation set have larger MIE than the lower-metallicity stars. The results over all wavelength pixels is summarized in the top right panel, which shows the cumulative number of wavelength pixels as a function of MIE. Roughly 80% of pixels have $\sigma_\text{MIE} < 0.001$, and 95% of pixels have $\sigma_\text{MIE} < 0.006$.

Top Left: Histogram of the median interpolation error of each model in the cross validation set. The median error is consistently larger for higher-metallicity stars ($\text{[Fe/H]}>-2$; red) compared to lower-metallicity stars ($\text{[Fe/H]}>-2$; blue). Top Right: Cumulative percentage of pixels in each spectrum as a function of the median interpolation error. Approximately 80% of pixels have $\sigma_\text{MIE} < 0.001$, and 95% of pixels have $\sigma_\text{MIE} < 0.006$. Bottom: The median interpolation error across the cross-validation set as a function of wavelength. Errors are largest in the proximity of strong H lines and complicated molecular features.

§ FITTING ROUTINES

§.§ Optimization

Using 's automatic differentiation engine and the Adam optimization algorithm, we minimize the negative log-posterior, which is equivalent to maximizing Equation <ref>. The optimization is performed 10 times with unique initializations. The parameters from the trial with the highest log-posterior value after convergence are taken as the best-fit optimization values. In the rest of this section, we present our choices of initialization, learning rates, and convergence criteria. These choices are summarized in Table <ref>.

For each initialization, we begin by defining a fiducial model spectrum with the mean stellar labels of the training set (i.e., $\theta_{*,i}^\prime = 0$) and the appropriate resolution for the observations. No Doppler shift, macroturbulent broadening, or continuum correction (other than the blaze function) is applied . Using this fiducial spectrum, the radial velocity is then initialized via a grid search from -300 to 300 km/s in steps of 2 km/s to the value that minimizes the negative log-posterior. The polynomial continuum coefficients are then initialized by performing a polynomial fit with  to the ratio of the observed spectrum and the (now Doppler-shifted) fiducial spectrum. All other labels are initialized by randomly sampling from their priors.

Learning Rates
We find learning rates of 0.1 to work well for all labels but the radial velocity, which requires a much smaller learning of 0.001 due to the sensitivity of $P(\Theta|D)$ to $v_r$ at high resolution.
To improve convergence, the learning rates are decayed every 10 step by a multiplicative factor of 0.9 for $v_r$ and 0.99 for all other labels.

Convergence of the optimization is achieved when the change in all model parameters is below a given threshold. We define this tolerance to be $10^{-5}$ for the scaled stellar labels, $\theta_{*}^\prime$, and $10^{4}$ for both $\log_{10}v_\text{macro}$ and $v_r$.

We define the convergence criteria for the continuum coefficients slightly differently. Instead of imposing a threshold on the change in $c_{n,o}$, we require that at every pixel (excluding masked pixels) the value of the continuum polynomial changes by less than 5% of the observed flux uncertainty.

Treatment of labels in the optimizer

Label     Initialization
Learning Rate      Decay     Timescale

$\theta_*^\prime$     Prior     1e-1     0.99     10     1e-5

$\log_{10}v_\text{macro}$     Prior     1e-1     0.99     10     1e-4

$v_r$     Grid Search     1e-3     0.9     10     1e-4

$c_{o,n}$     np.polyfit     1e-1     0.99     10     5e-2

Initialization procedure and optimization hyper-parameters for the stellar labels and nuisance parameters of our model.

§.§ MCMC Sampling

Using the affine invariant MCMC methods introduced by Goodman & Weare, 2010 and implemented in [<https://emcee.readthedocs.io/>], we sample our log-posterior distribution (Equation <ref>). As in the optimization routine, the log-posterior is evaluated using scaled stellar labels, $\theta_{*}^\prime$, which are converted to physical units when results are reported. In the rest of this section, we present the specifics of our sampling routine.

Before sampling the posterior in earnest, we begin by initializing 128 walkers at the maximum a posteriori value of $\Theta$ found via our optimization algorithm. Gaussian scatter of 0.1 is applied to all labels except the continuum coefficients, which are held constant throughout MCMC sampling. During this burn-in phase, the walkers sample the posterior until 1) the mean value for each label of the walkers changes less than 0.5% over the previous 100 steps and 2) the mean $\log P$ of the walkers has changed by less than 0.00001%. After the burn-in phase is complete, 512 walkers are initialized around the location of the walker with the highest $\log P$ with a Gaussian scatter in each label equal to half the standard deviation of the burn-in walkers for that label. Now that the initialization is complete, and the walkers have had a chance to settle around the maximum a posteriori, we begin the production run of our posterior sampling.

We sample the posterior distribution until the following two convergence criteria have been met: 1) the auto-correlation time, $\tau$, has changed by $<$1% over the previous 100 steps and 2) the sampler has run for $>$30$\tau$ steps. If these criteria have not been met after 15000 steps, walkers are re-initialized around the location of the walker with the highest $\log P$, and the sampling is restarted. Once convergence has been reached, we discard the first $5\tau$ samples from each walker and  thin each chain's samples by $\sim\tau/2$ to remove any residual effects of burn-in or correlated samples. Unthinned chains can be made available upon request.

Move Proposal
The default move proposal of  is the “stretch move" method of Goodman & Weare, 2010, which is not well suited for the dimensional of our problem. Instead, we adopt a weighted mixture of 80% differential evolution proposals (; Braak & F, 2006, Nelson et al., 2014) and 20% differential evolution snooker proposals (; ter Braak & Vrugt, 2008). We find that this combination of move proposals improves the convergence time by more than an order of magnitude over the default move proposal.

§ COMPARISON WITH LITERATURE VALUES

As a check on our fitting procedure, we compare our default high-resolution high-S/N stellar label measurements to those measured from previous stellar spectral analyses of the same stars. Our high-resolution measurements and those included in the literature comparison are presented in Table <ref>. Their chemical abundances have been adjusted to place them on the Asplund et al., 2009 Solar abundance scale adopted in our analysis. This collection of literature measurements is a non-exhaustive, but representative sample of previous spectroscopic studies in M15 across a wide range of spectroscopic configurations, fitting techniques, and measured elements.

Literature Stellar Parameters

Reference     [K]        [Mg/Fe]    [Ca/Fe]    [Fe/H]     ...

This Study   4415	   0.78	   0.37	    0.04	    -2.47	   ...

S+97	   4275	   0.45	   0.72	    0.32	    -2.35	   ...

S+00b	   4275	   0.45	   ...		    0.56	    -2.45	   ...

S+06	   4275	   0.45	   ...		    ...		    -2.46	   ...

R+09  	   ...		   ... 		   ... 		    ...		    -2.32	   ...

C+09b 	   4324	   0.69 	   0.49 	    ...		    -2.23	   ...

S+11   	   4343 	   0.88 	   0.60 	    0.22	    -2.53	   ...

W+13  	   4324 	   0.69 	   ... 		    0.16	    -2.32	   ...

K+18   	   4253 	   0.67 	   0.66 	    0.28	    -2.49	   ...

M+19   	   4545 	   0.80 	   0.27	    0.17	    -2.08	   ...

J+20    	   4377 	   0.64 	   0.39 	    0.28	    -2.30	   ...

This Study   4390	   0.54	   0.24	   0.01	    -2.49	   ...

S+97    	   4200	   0.15 	   ...		   0.19	    -2.43	   ...

S+00b 	   4200	   0.15 	   ...		   0.19	    -2.35	   ...

O+06   	   4200 	   0.35 	   ...		   ...		    -2.40	   ...

S+06   	   4200 	   0.15 	   ...		   ...		    -2.51	   ...

C+09a   	   4313 	   0.65 	   ...		   ...		    -2.33	   ...

W+13    	   4313 	   0.65 	   ...		   0.10	    -2.33	   ...

K+18    	   4263 	   0.65 	   0.15	   0.19	    -2.50	   ...

M+19    	   4548 	   0.81 	   0.28	   0.51	    -2.14	   ...

J+20    	   4449 	   0.90 	   ...		   ...		    -2.08	   ...

This Study   4489	   0.78	   0.16	    0.15	    -2.45	   ...

S+97    	   4375	   0.50 	   0.38	    0.28	    -2.43	   ...

L+06   	   4350 	   0.50 	   0.33	    0.32	    -2.36	   ...

S+06   	   4375 	   0.50 	   ...		    ...		    -2.50	   ...

W+13    	   4377 	   0.77 	   ...		    ...		    -2.34	   ...

K+18    	   4351 	   0.84 	   ...		    0.12	    -2.49	   ...

M+19    	   4670 	   1.09 	   0.22	    0.15	    -2.20	   ...

J+20    	   4543 	   1.02 	   ...		    ...		    -2.09	   ...

Non-exhaustive compilation of literature measurements of stellar parameters for stars in our sample. All chemical abundances have been scaled to the Asplund et al., 2009 Solar abundance scale for consistency in comparison. Reference abbreviations are as follows: S+97 = Sneden et al., 1997, S+00b = Sneden et al., 2000, L+06 = Letarte et al., 2006, O+06 = Otsuki et al., 2006, S+06 = Sobeck et al., 2006, R+09 = Roederer et al., 2009, C+09a = Carretta et al., 2009, C+09b = Carretta et al., 2009, S+11 = Sobeck et al., 2011, W+13 = Worley et al., 2013, K+18 = Kirby et al., 2018, M+19 = Masseron et al., 2019, J+20 = Jönsson et al., 2020.

(This table is available in its entirety in machine-readable form online.)

In Figure <ref> we present our measurements of , , and [Fe/H] (black stars) alongside literature measurements (colored circles) for the 8 stars in our sample. Error bars represent $1\sigma$ uncertainties where available and are too small to be visible for the statistical uncertainty of our own measurements.

Overall, our measurements fall nicely amidst the locus of literature measurements. On average, our measurements differ from the median literature value for , , and [Fe/H] by roughly 100 K, $-0.1$ dex, and $-0.15$ dex respectively. The most outlying atmospheric parameters are recovered for star K934, for which we recover values of , , and [Fe/H] to be $\sim$300 K, 0.5 dex, and 0.3 dex lower respectively than measured from APOGEE spectra by Masseron et al., 2019 and Jönsson et al., 2020. We believe that the proximity of K934 to K731 in the CMD (Figure <ref>) justify our measurements, which would place the two stars similarly close in - space. We also note that two APOGEE studies also consistently recover [Fe/H] higher than most other studies, perhaps owing to the difference in wavelength coverage (NIR vs. optical).

The  (top),  (middle), and [Fe/H] (bottom) measured for each of the 8 stars in our sample using the full-spectrum fitting techniques presented in this paper (black stars). For comparison, we also plot the values for , , and [Fe/H] reported in a representative sample of literature studies of the same stars (colored circles). In instances where studies report separate values for neutral and ionized atomic species, the ionized value is represented by an open circle. Error bars represent $1\sigma$ uncertainties when provided and are too small to be visible for our own measurement uncertainties. Scatter is added in the x-dimension for clarity; points are ordered from left to right in order of increasing mean observed wavelength. A key to the abbreviated references is provided in Table <ref>.

In Figure <ref>, we present a comparison of our measurements with measurements from the same literature studies for the remaining 35 chemical abundances. The same symbol schema is adopted as in Figure <ref>. When separate abundances are provided for neutral and ionized atomic species (e.g., [Ti I/Fe I] vs. [Ti II/Fe II]), the abundances for the ionized species are represented with open circles.

The detailed chemical composition of the 8 M15 stars as measured in this study (black stars) and as reported in the literature (colored circles; the same as in Figure <ref>). 95% upper and lower limits are plotted where appropriate (see <ref>) and when reported by the literature. As in Figure <ref>, when separate abundances are provided for neutral and ionized atomic species, the ionized values are represented with open circles. Scatter is added in the x-dimension for clarity; points are ordered from left to right in order of increasing mean observed wavelength.

In brief, we find generally good agreement with the literature for measurements of C, N, O, Mg, Ca, Ti, V, Cr, Mn, Co, Ni, Cu, Y, Zr, La, Ce, Pr, Nd, Gd, Dy, Er, and Th with a few caveats. For 5 elements, our measurements are systematically offset $\sim$0.25 dex higher (Sc, Ba, Sm, and Eu) or lower (Zn) compared to literature measurements. Mixed or poor agreement with values was found for (Na, Al, Si, K, and Ho). No literature measurements were found to compare to for Ga and Os.

In general, differences between our abundance measurements and those from the literature can be attributed to differences in the stellar models, line lists, oscillator strengths, and/or wavelength coverage employed.

Across all elements, our measurements agree best with literature measurements made over similar wavelength ranges (e.g., the optical; Sneden et al., 1997, Sobeck et al., 2011) and less well with those made over non-overlapping spectral coverage (e.g., the NIR; Mészáros et al., 2015, Masseron et al., 2019, Jönsson et al., 2020).

As in <ref>, we present a detailed comparison with the literature for each element loosely grouped by nucleosynthetic origin.

§.§ Literature Comparison by Element Group

Iron-Peak Elements

In addition to Fe, we find generally good agreement with the literature for iron-peak elements V, Cr, Co, and Ni. While we recover only upper limits on Mn, these upper limits are generally consistent with literature values.

The one exception to our general agreement with the literature is for K934, for which Jönsson et al., 2020 reports $\text{[Mn/Fe]} = 0.68\pm0.15$. Because this is the only reported measurement of [Mn/Fe] in K934, and Jönsson et al., 2020 do not report [Mn/Fe] for any of the other stars in this sample, it is difficult to identify the source of this discrepancy. Given the ubiquity of low [Mn/Fe] abundances in metal-poor stars <cit.>, we are inclined to trust our measurement in this instance. We note that large NLTE offsets ($\sim$0.2–0.4 dex) have been calculated for [Mn/Fe] in low-metallicity stars <cit.>, which have not been accounted for in either this study or any of the referenced studies.

Only three stars in our sample have literature [Cu/Fe] measurements and all come from one of two studies, Sobeck et al., 2011 and Jönsson et al., 2020. The values reported by these two studies are discrepant by as much as 1 dex, an effect we attribute to the difference in wavelength coverage of the two surveys: Sobeck et al., 2011 used optical Keck/HIRES spectra (the same archival spectra, in fact, as analyzed in this paper), and Jönsson et al., 2020 used NIR APOGEE spectra. Jönsson et al., 2020 also urges caution in adopting the [Cu/Fe] measurements for metal-poor stars with $\text{[Fe/H]} < -1$ as they find systematically higher [Cu/Fe] at lower [Fe/H] in contrast to previous studies and the expectations of Cu's nucleosynthetic origin <cit.>. It is possible, however, that NLTE effects are responsible for the low [Cu/Fe] values found from optical spectroscopy <cit.>.

For these three stars, we recover upper limits on [Cu/Fe] that are consistent with the lower abundances of Sobeck et al., 2011. This is in line with expectations given the same underlying observations and LTE assumptions.

We routinely recover [Zn/Fe] to be $\sim$0.25 dex smaller than reported in the literature by Letarte et al., 2006 and Sobeck et al., 2011. We believe this offset is driven by the Zn I line at 4681.4, which is blended with a poorly modelled Fe I line at 4681.6 and excluded from the analysis of Letarte et al., 2006 and Sobeck et al., 2011. The other two Zn lines at 4723.5, 4811.9 are slightly underestimated in our fits, consistent with the $\sim$0.25 dex lower measurement of [Zn/Fe].

Owing to the paucity of quality Ga lines in the archival spectra—there are only two ($\lambda\lambda$40341a and 4173.2) and both are weak, heavily blended, and within NLTE masks—we are unable to precisely constrain [Ga/Fe] within our model bounds for all but two stars, K462 and K479. Because no literature values of [Ga/Fe] exist for these stars, we cannot confirm the fidelity of these measurements and urge caution in their adoption.

$\alpha$ Elements

In general, we find good agreement with the literature for $\alpha$ elements Mg, Ca, and Ti, though [Ca/Fe] measurements are on the low end of reported values for a few stars. This is most likely a result of differences in the handling of NLTE effects.

We find good agreement with the literature for Si for the C147Hr and C316Hr programs, but the blue-only U09H program observations yield somewhat more mixed agreement, in some cases off by 0.25–0.5 dex. This can be traced to Si's role as an electron donor in stellar atmospheres. When the full optical spectrum is available, Si is primarily constrained through its isolated absorption lines near 5700 and 7400. However, when only the blue-optical spectrum is available, Si is primarily constrained through its indirect influence on other absorption features through changes to the atmospheric structure. While indirect measurement of elements is feasible <cit.>, it relies heavily on the accuracy of the stellar atmospheric models. We know the 1D-LTE models employed in this work to be imperfect, thus explaining the inconsistencies with the literature seen for the blue-only [Si/Fe] abundances.

C, N, O

There exists large (0.5–1.0 dex) scatter in the literature values measured for C, N, and O abundances, owing to the complicated nature of their molecular features and the varying wavelength coverage and methods of these studies (see for example the analysis of [C/Fe] measurements across surveys by Arentsen et al., 2022). We find good agreement with the literature for [C/Fe] with the exception of few measurements from Masseron et al., 2019, which are substantially higher than our values.

Only three of our stars have previously measured [N/Fe]. For K386, our measurement agrees with the measurement of Masseron et al., 2019, but for K341 and K462, we measure [N/Fe] to be $>$0.5 dex larger than reported by Jönsson et al., 2020.

For nearly all stars in our sample, literature values span a large range from $\text{[O/Fe]}\sim0.0$–1.0. Our measurements, either lower limits of $\text{[O/Fe]}\gtrsim1.0$ or in the range of $\text{[O/Fe]}\sim0.75$–1.0, are most consistent with the high end of the reported literature values.

Light-Odd Elements

The light-odd elements Na, Al, and K, similar to C, N, and O, exhibit large (0.5–1.0 dex) scatter in the reported literature values, which is due to the combination of the limited absorption features available for these elements as well as their sensitivity to NLTE effects <cit.>. We recover [Na/Fe] values that either fall among the literature values or lie slightly above the literature measurements except for stars K462, for which we measure [Na/Fe] $\sim$ 0.25 lower than previously reported.

For roughly half of the stars in our sample we measure [Al/Fe] in agreement with literature values, while we recover substantially lower [Al/Fe] for the others.

In general, measurements of [K/Fe] (or lower limits) are in coarse agreement with measurements (or lower limits) from Masseron et al., 2019. As with Na, the exceptions to this is K462, for which we recover much lower [K/Fe].

Literature measurements of [Sc/Fe] come from Sneden et al., 1997 and Sobeck et al., 2011, which largely analyzed the same archival spectra as in this paper. Our measurements of [Sc/Fe] are in good agreement with those from Sobeck et al., 2011 except for the sole Sc I measurement in K583, which is itself highly discrepant from the Sc II measurement from the same study. Agreement is good with Sneden et al., 1997 for the 4 stars observed as part of the U09H program, while for the remaining stars our values are higher by 0.25–0.5 dex. Not coincidentally, the 4 stars for which agreement is best are the 4 stars for which the same spectra are analyzed in both studies.

Neutron-Capture Elements

We find good agreement for neutron-capture elements Y, Zr, La, Pr, Nd, Gd, Dy, Er, and Th. We recover values for [Sr/Fe] that are in good agreement with the values reported by Sobeck et al., 2011, but are $\sim$0.5 dex higher than reported by Otsuki et al., 2006. This discrepancy was previously identified by Sobeck et al., 2011 and attributed to uncertainties in measuring abundances from the Sr II resonance lines.

Similarly, we find good agreement in [Ce/Fe] with Sobeck et al., 2011, but mixed agreement with Masseron et al., 2019. For stars K341 and K462, we recover [Ce/Fe] values that match Masseron et al., 2019 quite well, but for stars K386, K431, K583, and K969, we recover [Ce/Fe] values that are $\sim$0.25–0.50 dex smaller. This is of order the systematic error that Masseron et al., 2019 reports between [Ce/Fe] values measured using differently derived atmospheric parameters.

Our recovered abundances for Ba, Sm, and Eu are consistently $\sim$0.25–0.50 dex larger than the values reported in the literature. We link these offsets to a combination of factors, including line saturation, NLTE effects, and hyperfine splitting that result in inaccurately modeled line profile shapes (see Roederer et al., 2008, Eitner et al., 2019).

In the case of Ho, our measurements disagree by $>$0.5 dex from the only available measurements of Sobeck et al., 2011. Four of the five Ho II lines in our spectrum (3797.8, 3811.8, 3892.1, 4046.6) are heavily blended in poorly modelled portions of the the spectrum while the fifth (4153.8) is quite weak. As a result, we believe our [Ho/Fe] measurements to be in err.

Owing to the dearth of Os lines in the archival spectra, we are unable to precisely constrain [Os/Fe] within our model bounds for any star in our sample. Because no literature values of [Os/Fe] exist for this star, we cannot confirm the fidelity of these measurements and urge caution in their adoption.

[Arentsen et al., 2022]
Arentsen, A., Placco, V. M., Lee, Y. S., et al. 2022, Monthly Notices of the
Royal Astronomical Society, 515, 4082, 10.1093/mnras/stac2062

[Arp, 1955]
Arp, H. C. 1955, The Astronomical Journal, 60, 317, 10.1086/107232

[Asplund, 2005]
Asplund, M. 2005, Annual Review of Astronomy & Astrophysics, vol. 43,
Issue 1, pp.481-530, 43, 481, 10.1146/annurev.astro.42.053102.134001

[Asplund et al., 2009]
Asplund, M., Grevesse, N., Sauval, A. J., & Scott, P. 2009, Annual Review of
Astronomy and Astrophysics, 47, 481,

[Astropy Collaboration et al., 2013]
Astropy Collaboration, Robitaille, T. P., Tollerud, E. J., et al. 2013,
Astronomy and Astrophysics, 558, A33, 10.1051/0004-6361/201322068

[Astropy Collaboration et al., 2018]
Astropy Collaboration, Price-Whelan, A. M., Sipőcz, B. M., et al.
2018, The Astronomical Journal, 156, 123, 10.3847/1538-3881/aabc4f

[Baumgardt & Vasiliev, 2021]
Baumgardt, H., & Vasiliev, E. 2021, Monthly Notices of the Royal Astronomical
Society, 505, 5957, 10.1093/mnras/stab1474

[Bedell et al., 2014]
Bedell, M., Meléndez, J., Bean, J. L., et al. 2014, The Astrophysical
Journal, 795, 23, 10.1088/0004-637X/795/1/23

[Bergemann, 2011]
Bergemann, M. 2011, Monthly Notices of the Royal Astronomical Society, 413,
2184, 10.1111/j.1365-2966.2011.18295.x

[Bergemann & Cescutti, 2010]
Bergemann, M., & Cescutti, G. 2010, A&A, 522, A9,

[Bergemann et al., 2017]
Bergemann, M., Collet, R., Amarsi, A. M., et al. 2017, The Astrophysical
Journal, 847, 15, 10.3847/1538-4357/aa88cb

[Bergemann & Gehren, 2008]
Bergemann, M., & Gehren, T. 2008, Astronomy and Astrophysics, Volume 492,
Issue 3, 2008, pp.823-831, 492, 823, 10.1051/0004-6361:200810098

[Bergemann et al., 2013]
Bergemann, M., Kudritzki, R.-P., Würl, M., et al. 2013, The Astrophysical
Journal, 764, 115, 10.1088/0004-637X/764/2/115

[Bergemann et al., 2012]
Bergemann, M., Lind, K., Collet, R., Magic, Z., & Asplund, M. 2012, Monthly
Notices of the Royal Astronomical Society, 427, 27,

[Bergemann et al., 2010]
Bergemann, M., Pickering, J. C., & Gehren, T. 2010, Monthly Notices of the
Royal Astronomical Society, 401, 1334,

[Blanco-Cuaresma, 2019]
Blanco-Cuaresma, S. 2019, Monthly Notices of the Royal Astronomical Society,
486, 2075, 10.1093/mnras/stz549

[Braak & F, 2006]
Braak, T., & F, C. J. 2006, Statistics and Computing, 16, 239,

[Bundy et al., 2019]
Bundy, K., Westfall, K., MacDonald, N., et al. 2019, Bulletin of the American
Astronomical Society, 51, 198

[Cargile et al., 2020]
Cargile, P. A., Conroy, C., Johnson, B. D., et al. 2020, The Astrophysical
Journal, 900, 28, 10.3847/1538-4357/aba43b

[Carlberg et al., 2011]
Carlberg, J. K., Majewski, S. R., Patterson, R. J., et al. 2011, The
Astrophysical Journal, 732, 39, 10.1088/0004-637X/732/1/39

[Carnall, 2017]
Carnall, A. C. 2017, arXiv e-prints, arXiv:1705.05165

[Carretta et al., 2009]
Carretta, E., Bragaglia, A., Gratton, R., & Lucatello, S. 2009a,
Astronomy and Astrophysics, 505, 139, 10.1051/0004-6361/200912097

[Carretta et al., 2009]
Carretta, E., Bragaglia, A., Gratton, R. G., et al. 2009b,
Astronomy and Astrophysics, 505, 117, 10.1051/0004-6361/200912096

[Cayrel et al., 2004]
Cayrel, R., Depagne, E., Spite, M., et al. 2004, A&A, 416, 1117,

[Choi et al., 2016]
Choi, J., Dotter, A., Conroy, C., et al. 2016, The Astrophysical Journal,
823, 102, 10.3847/0004-637X/823/2/102

[Collaboration et al., 2018]
Collaboration, G., Babusiaux, C., van Leeuwen, F., et al. 2018, A&A, 616,
A10, 10.1051/0004-6361/201832843

[Cooper et al., 2022]
Cooper, A. P., Koposov, S. E., Allende Prieto, C., et al. 2022, Overview of
the DESI Milky Way Survey

[Cramer, 1946]
Cramer, H. 1946, Mathematical Methods of Statistics, Princeton Mathematical
Series No. 9 (Princeton: Princeton University Press)

[Cui et al., 2012]
Cui, X.-Q., Zhao, Y.-H., Chu, Y.-Q., et al. 2012, Research in Astronomy and
Astrophysics, 12, 1197, 10.1088/1674-4527/12/9/003

[Czekala et al., 2015]
Czekala, I., Andrews, S. M., Mandel, K. S., Hogg, D. W., & Green, G. M. 2015,
The Astrophysical Journal, 812, 128, 10.1088/0004-637X/812/2/128

[Dalton et al., 2016]
Dalton, G., Trager, S., Abrams, D. C., et al. 2016, in Ground-Based and
Airborne Instrumentation for Astronomy VI, Vol. 9908 (SPIE),
394–403, 10.1117/12.2231078

[Darmois, 1945]
Darmois, G. 1945, Revue de l'Institut International de Statistique / Review of
the International Statistical Institute, 13, 9, 10.2307/1400974

[de Jong et al., 2019]
de Jong, R. S., Agertz, O., Berbel, A. A., et al. 2019, The Messenger, 175,
3, 10.18727/0722-6691/5117

[De Silva et al., 2015]
De Silva, G. M., Freeman, K. C., Bland-Hawthorn, J., et al. 2015, Monthly
Notices of the Royal Astronomical Society, 449, 2604,

[Den Hartog et al., 2019]
Den Hartog, E. A., Lawler, J. E., Sneden, C., Cowan, J. J., & Brukhovesky, A.
2019, The Astrophysical Journal Supplement Series, 243, 33,

[Dotter, 2016]
Dotter, A. 2016, The Astrophysical Journal Supplement Series, 222, 8,

[Eitner et al., 2019]
Eitner, P., Bergemann, M., & Larsen, S. 2019, A&A, 627, A40,

[El-Badry et al., 2018]
El-Badry, K., Rix, H.-W., Ting, Y.-S., et al. 2018, Monthly Notices of the
Royal Astronomical Society, 473, 5043, 10.1093/mnras/stx2758

[Escala et al., 2019]
Escala, I., Kirby, E. N., Gilbert, K. M., Cunningham, E. C., & Wojno, J. 2019,
The Astrophysical Journal, 878, 42, 10.3847/1538-4357/ab1eac

[Falcon et al., 2020]
Falcon, W., Borovec, J., Wälchli, A., et al. 2020,
PyTorchLightning/Pytorch-Lightning: 0.7.6 Release, Zenodo,

[Fanson et al., 2020]
Fanson, J., Bernstein, R., Angeli, G., et al. 2020, in Ground-Based and
Airborne Telescopes VIII, Vol. 11445 (SPIE), 295–314,

[Foreman-Mackey et al., 2013]
Foreman-Mackey, D., Hogg, D. W., Lang, D., & Goodman, J. 2013, Publications
of the Astronomical Society of the Pacific, 125, 306, 10.1086/670067

[Fréchet, 1943]
Fréchet, M. 1943, Revue de l'Institut International de Statistique / Review
of the International Statistical Institute, 11, 182, 10.2307/1401114

[Gaia Collaboration et al., 2022]
Gaia Collaboration, Vallenari, A., Brown, A. G. A., et al. 2022, Gaia
Data Release 3: Summary of the Content and Survey Properties

[Gardner et al., 2006]
Gardner, J. P., Mather, J. C., Clampin, M., et al. 2006, Space Science
Reviews, 123, 485, 10.1007/s11214-006-8315-7

[Gilbert et al., 2019]
Gilbert, K. M., Kirby, E. N., Escala, I., et al. 2019, The Astrophysical
Journal, 883, 128, 10.3847/1538-4357/ab3807

[Gilmore et al., 2012]
Gilmore, G., Randich, S., Asplund, M., et al. 2012, The Messenger, 147, 25

[Gilmozzi & Spyromilio, 2007]
Gilmozzi, R., & Spyromilio, J. 2007, The Messenger, 127, 11

[Goodman & Weare, 2010]
Goodman, J., & Weare, J. 2010, Communications in Applied Mathematics and
Computational Science, 5, 65, 10.2140/camcos.2010.5.65

[Gray, 2021]
Gray, D. F. 2021, The Observation and Analysis of Stellar Photospheres, 4th
edn. (Cambridge: Cambridge University Press),

[Griffith et al., 2022]
Griffith, E. J., Johnson, J. A., Weinberg, D. H., et al. 2022, Untangling the
Sources of Abundance Dispersion in Low-Metallicity Stars

[Harris et al., 2020]
Harris, C. R., Millman, K. J., van der Walt, S. J., et al. 2020, Nature,
585, 357, 10.1038/s41586-020-2649-2

[Heiter et al., 2014]
Heiter, U., Soubiran, C., Netopil, M., & Paunzen, E. 2014, Astronomy and
Astrophysics, 561, A93, 10.1051/0004-6361/201322559

[Hinkel et al., 2016]
Hinkel, N. R., Young, P. A., Pagano, M. D., et al. 2016, The Astrophysical
Journal Supplement Series, 226, 4, 10.3847/0067-0049/226/1/4

[Holtzman et al., 2015]
Holtzman, J. A., Shetrone, M., Johnson, J. A., et al. 2015, The Astronomical
Journal, 150, 148, 10.1088/0004-6256/150/5/148

[Hunter, 2007]
Hunter, J. D. 2007, Computing in Science & Engineering, 9, 90,

[Ishigaki et al., 2013]
Ishigaki, M. N., Aoki, W., & Chiba, M. 2013, The Astrophysical Journal, 771,
67, 10.1088/0004-637X/771/1/67

[Jofré et al., 2019]
Jofré, P., Heiter, U., & Soubiran, C. 2019, Annual Review of Astronomy and
Astrophysics, 57, 571, 10.1146/annurev-astro-091918-104509

[Jofré et al., 2017]
Jofré, P., Heiter, U., Worley, C. C., et al. 2017, A&A, 601, A38,

[Jönsson et al., 2020]
Jönsson, H., Holtzman, J. A., Allende Prieto, C., et al. 2020, The
Astronomical Journal, 160, 120, 10.3847/1538-3881/aba592

[Kirby et al., 2011]
Kirby, E. N., Cohen, J. G., Smith, G. H., et al. 2011, The Astrophysical
Journal, 727, 79, 10.1088/0004-637X/727/2/79

[Kirby et al., 2009]
Kirby, E. N., Guhathakurta, P., Bolte, M., Sneden, C., & Geha, M. C. 2009, The
Astrophysical Journal, 705, 328, 10.1088/0004-637X/705/1/328

[Kirby et al., 2018]
Kirby, E. N., Xie, J. L., Guo, R., Kovalev, M., & Bergemann, M. 2018, The
Astrophysical Journal Supplement Series, 237, 18,

[Kirby et al., 2010]
Kirby, E. N., Guhathakurta, P., Simon, J. D., et al. 2010, The Astrophysical
Journal Supplement Series, 191, 352, 10.1088/0067-0049/191/2/352

[Kollmeier et al., 2017]
Kollmeier, J. A., Zasowski, G., Rix, H.-W., et al. 2017, SDSS-V:
Pioneering Panoptic Spectroscopy

[Kovalev et al., 2019]
Kovalev, M., Bergemann, M., Ting, Y.-S., & Rix, H.-W. 2019, Astronomy and
Astrophysics, 628, A54, 10.1051/0004-6361/201935861

[Kurucz, 1970]
Kurucz, R. L. 1970, SAO Special Report, 309

[Kurucz, 1993]
—. 1993, SYNTHE Spectrum Synthesis Programs and Line Data, CD-ROM

[Kurucz, 2005]
—. 2005, Memorie della Societa Astronomica Italiana Supplementi, 8, 14

[Kurucz, 2013]
—. 2013, Astrophysics Source Code Library, ascl:1303.024

[Kurucz, 2017]
—. 2017, Astrophysics Source Code Library, ascl:1710.017

[Kurucz & Avrett, 1981]
Kurucz, R. L., & Avrett, E. H. 1981, SAO Special Report, 391

[Kustner, 1921]
Kustner, F. 1921, Veroeffentlichungen des Astronomisches Institute der
Universitaet Bonn, 15, 1

[Larsen et al., 2022]
Larsen, S. S., Eitner, P., Magg, E., et al. 2022, A&A, 660, A88,

[Lawler et al., 2013]
Lawler, J. E., Guzman, A., Wood, M. P., Sneden, C., & Cowan, J. J. 2013, The
Astrophysical Journal Supplement Series, 205, 11,

[Letarte et al., 2006]
Letarte, B., Hill, V., Jablonka, P., et al. 2006, Astronomy & Astrophysics,
453, 547, 10.1051/0004-6361:20054439

[Majewski et al., 2017]
Majewski, S. R., Schiavon, R. P., Frinchaboy, P. M., et al. 2017, The
Astronomical Journal, 154, 94, 10.3847/1538-3881/aa784d

[Mashonkina & Gehren, 2000]
Mashonkina, L., & Gehren, T. 2000, Astronomy and Astrophysics, 364, 249

[Mashonkina et al., 2007]
Mashonkina, L., Korn, A. J., & Przybilla, N. 2007, Astronomy and Astrophysics,
461, 261, 10.1051/0004-6361:20065999

[Masseron et al., 2019]
Masseron, T., García-Hernández, D. A., Mészáros, S., et al.
2019, A&A, 622, A191, 10.1051/0004-6361/201834550

[McKinney, 2010]
McKinney, W. 2010, in Proceedings of the 9th Python in Science Conference, ed.
S. van der Walt & J. Millman, 51–56

[Mészáros et al., 2015]
Mészáros, S., Martell, S. L., Shetrone, M., et al. 2015, The
Astronomical Journal, 149, 153, 10.1088/0004-6256/149/5/153

[Nelson et al., 2014]
Nelson, B., Ford, E. B., & Payne, M. J. 2014, The Astrophysical Journal
Supplement Series, 210, 11, 10.1088/0067-0049/210/1/11

[Nissen & Gustafsson, 2018]
Nissen, P. E., & Gustafsson, B. 2018, Astronomy and Astrophysics Review, 26,
6, 10.1007/s00159-018-0111-3

[Ochsenbein et al., 2000]
Ochsenbein, F., Bauer, P., & Marcout, J. 2000, Astronomy and Astrophysics
Supplement, v.143, p.23-32, 143, 23, 10.1051/aas:2000169

[Otsuki et al., 2006]
Otsuki, K., Honda, S., Aoki, W., Kajino, T., & Mathews, G. J. 2006, The
Astrophysical Journal Letters, 641, L117, 10.1086/504106

[Paszke et al., 2019]
Paszke, A., Gross, S., Massa, F., et al. 2019, arXiv e-prints, 1912,

[Paxton et al., 2011]
Paxton, B., Bildsten, L., Dotter, A., et al. 2011, The Astrophysical Journal
Supplement Series, 192, 3, 10.1088/0067-0049/192/1/3

[Paxton et al., 2013]
Paxton, B., Cantiello, M., Arras, P., et al. 2013, The Astrophysical Journal
Supplement Series, 208, 4, 10.1088/0067-0049/208/1/4

[Paxton et al., 2015]
Paxton, B., Marchant, P., Schwab, J., et al. 2015, The Astrophysical Journal
Supplement Series, 220, 15, 10.1088/0067-0049/220/1/15

[Paxton et al., 2018]
Paxton, B., Schwab, J., Bauer, E. B., et al. 2018, The Astrophysical Journal
Supplement Series, 234, 34, 10.3847/1538-4365/aaa5a8

[Prochaska et al., 2020]
Prochaska, J., Hennawi, J., Westfall, K., et al. 2020, The Journal of Open
Source Software, 5, 2308, 10.21105/joss.02308

[Rao, 1945]
Rao, C. R. 1945, Bulletin of the Calcutta Mathematical Society, 37, 81

[Reback et al., 2022]
Reback, J., jbrockmendel, McKinney, W., et al. 2022, Pandas-Dev/Pandas:
Pandas 1.4.3, Zenodo, 10.5281/zenodo.6702671

[Recio-Blanco et al., 2022]
Recio-Blanco, A., de Laverny, P., Palicio, P. A., et al. 2022, Gaia
Data Release 3: Analysis of RVS Spectra Using the General
Stellar Parametriser from Spectroscopy

[Roederer & Barklem, 2018]
Roederer, I. U., & Barklem, P. S. 2018, The Astrophysical Journal, 857, 2,

[Roederer et al., 2009]
Roederer, I. U., Kratz, K.-L., Frebel, A., et al. 2009, The Astrophysical
Journal, 698, 1963, 10.1088/0004-637X/698/2/1963

[Roederer et al., 2008]
Roederer, I. U., Lawler, J. E., Sneden, C., et al. 2008, The Astrophysical
Journal, 675, 723, 10.1086/526452

[Ryabchikova et al., 2015]
Ryabchikova, T., Piskunov, N., Kurucz, R. L., et al. 2015, Physica Scripta,
90, 054005, 10.1088/0031-8949/90/5/054005

[Sandage, 1970]
Sandage, A. 1970, The Astrophysical Journal, 162, 841, 10.1086/150715

[Sandford, 2020]
Sandford, N. 2020, Zenodo, 10.5281/zenodo.3924672

[Sandford et al., 2022]
Sandford, N. R., Weinberg, D. H., Weisz, D. R., & Fu, S. W. 2022, Strong
Outflows and Inefficient Star Formation in the Reionization-era
Ultra-faint Dwarf Galaxy Eridanus II, 10.48550/arXiv.2210.17045

[Sandford et al., 2020]
Sandford, N. R., Weisz, D. R., & Ting, Y.-S. 2020, The Astrophysical Journal
Supplement Series, 249, 24, 10.3847/1538-4365/ab9cb0

[Schlafly & Finkbeiner, 2011]
Schlafly, E. F., & Finkbeiner, D. P. 2011, The Astrophysical Journal, 737,
103, 10.1088/0004-637X/737/2/103

[Simon, 2019]
Simon, J. D. 2019, Annual Review of Astronomy and Astrophysics, 57, 375,

[Sitnova et al., 2013]
Sitnova, T. M., Mashonkina, L. I., & Ryabchikova, T. A. 2013, Astronomy
Letters, 39, 126, 10.1134/S1063773713020084

[Skidmore et al., 2015]
Skidmore, W., TMT International Science Development Teams, & Science
Advisory Committee, T. 2015, Research in Astronomy and Astrophysics, 15,
1945, 10.1088/1674-4527/15/12/001

[Skrutskie et al., 2006]
Skrutskie, M. F., Cutri, R. M., Stiening, R., et al. 2006, The Astronomical
Journal, 131, 1163, 10.1086/498708

[Smiljanic et al., 2014]
Smiljanic, R., Korn, A. J., Bergemann, M., et al. 2014, A&A, 570, A122,

[Smith et al., 2021]
Smith, V. V., Bizyaev, D., Cunha, K., et al. 2021, The Astronomical Journal,
161, 254, 10.3847/1538-3881/abefdc

[Sneden et al., 1991]
Sneden, C., Gratton, R. G., & Crocker, D. A. 1991, Astronomy and Astrophysics,
Vol. 246, p. 354 (1991), 246, 354

[Sneden et al., 2000]
Sneden, C., Johnson, J., Kraft, R. P., et al. 2000a, The
Astrophysical Journal, 536, L85, 10.1086/312742

[Sneden et al., 1997]
Sneden, C., Kraft, R. P., Shetrone, M. D., et al. 1997, The Astronomical
Journal, 114, 1964, 10.1086/118618

[Sneden et al., 2000]
Sneden, C., Pilachowski, C. A., & Kraft, R. P. 2000b, The
Astronomical Journal, 120, 1351, 10.1086/301509

[Sobeck et al., 2006]
Sobeck, J. S., Ivans, I. I., Simmerer, J. A., et al. 2006, The Astronomical
Journal, 131, 2949, 10.1086/503106

[Sobeck et al., 2011]
Sobeck, J. S., Kraft, R. P., Sneden, C., et al. 2011, The Astronomical
Journal, 141, 175, 10.1088/0004-6256/141/6/175

[Straumit et al., 2022]
Straumit, I., Tkachenko, A., Gebruers, S., et al. 2022, The Astronomical
Journal, 163, 236, 10.3847/1538-3881/ac5f49

[Tamura et al., 2018]
Tamura, N., Takato, N., Shimono, A., et al. 2018, in Ground-Based and
Airborne Instrumentation for Astronomy VII, Vol. 10702
(International Society for Optics and Photonics), 107021C,

[Taylor et al., 2018]
Taylor, W., Cirasuolo, M., Afonso, J., et al. 2018, in Ground-Based and
Airborne Instrumentation for Astronomy VII, Vol. 10702 (SPIE),
421–440, 10.1117/12.2313403

[ter Braak & Vrugt, 2008]
ter Braak, C. J. F., & Vrugt, J. A. 2008, Stat Comput, 18, 435,

[Ting et al., 2018]
Ting, Y.-S., Conroy, C., Rix, H.-W., & Asplund, M. 2018a, The
Astrophysical Journal, 860, 159, 10.3847/1538-4357/aac6c9

[Ting et al., 2017]
Ting, Y.-S., Conroy, C., Rix, H.-W., & Cargile, P. 2017a, The
Astrophysical Journal, 843, 32, 10.3847/1538-4357/aa7688

[Ting et al., 2019]
—. 2019, The Astrophysical Journal, 879, 69, 10.3847/1538-4357/ab2331

[Ting et al., 2018]
Ting, Y.-S., Hawkins, K., & Rix, H.-W. 2018b, The Astrophysical
Journal, 858, L7, 10.3847/2041-8213/aabf8e

[Ting et al., 2017]
Ting, Y.-S., Rix, H.-W., Conroy, C., Ho, A. Y. Q., & Lin, J.
2017b, The Astrophysical Journal Letters, 849, L9,

[Ting & Weinberg, 2022]
Ting, Y.-S., & Weinberg, D. H. 2022, The Astrophysical Journal, 927, 209,

[Tolstoy et al., 2009]
Tolstoy, E., Hill, V., & Tosi, M. 2009, Annual Review of Astronomy and
Astrophysics, 47, 371, 10.1146/annurev-astro-082708-101650

[van der Walt et al., 2011]
van der Walt, S., Colbert, S. C., & Varoquaux, G. 2011, Computing in Science
& Engineering, 13, 22, 10.1109/MCSE.2011.37

[Vasiliev & Baumgardt, 2021]
Vasiliev, E., & Baumgardt, H. 2021, Monthly Notices of the Royal Astronomical
Society, 505, 5978, 10.1093/mnras/stab1475

[Vogt et al., 1994]
Vogt, S. S., Allen, S. L., Bigelow, B. C., et al. 1994, Instrumentation in
Astronomy VIII, 2198, 362, 10.1117/12.176725

[Wang et al., 2022]
Wang, Z., Hayden, M. R., Sharma, S., et al. 2022, Monthly Notices of the
Royal Astronomical Society, 514, 1034, 10.1093/mnras/stac1425

[Wenger et al., 2000]
Wenger, M., Ochsenbein, F., Egret, D., et al. 2000, Astronomy and
Astrophysics Supplement, v.143, p.9-22, 143, 9, 10.1051/aas:2000332

[Worley et al., 2013]
Worley, C. C., Hill, V., Sobeck, J., & Carretta, E. 2013, A&A, 553, A47,

[Xiang et al., 2019]
Xiang, M., Ting, Y.-S., Rix, H.-W., et al. 2019, The Astrophysical Journal
Supplement Series, 245, 34, 10.3847/1538-4365/ab5364

[Xiang et al., 2022]
Xiang, M., Rix, H.-W., Ting, Y.-S., et al. 2022, Astronomy and Astrophysics,
662, A66, 10.1051/0004-6361/202141570

[Xiong et al., 2021]
Xiong, J.-P., Zhang, B., Liu, C., et al. 2021, Research in Astronomy and
Astrophysics, 21, 265, 10.1088/1674-4527/21/10/265