# Fairness-aware Class Imbalanced Learning

Shivashankar Subramanian♣ Afshin Rahimi♠
Timothy Baldwin♣ Trevor Cohn♣ Lea Frermann♣
♣School of Computing and Information Systems, University of Melbourne
♠School of Information Technology and Electrical Engineering, University of
Queensland
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Class imbalance is a common challenge in many NLP tasks, and has clear
connections to bias, in that bias in training data often leads to higher
accuracy for majority groups at the expense of minority groups. However there
has traditionally been a disconnect between research on class-imbalanced
learning and mitigating bias, and only recently have the two been looked at
through a common lens. In this work we evaluate long-tail learning methods for
tweet sentiment and occupation classification, and extend a margin-loss based
approach with methods to enforce fairness. We empirically show through
controlled experiments that the proposed approaches help mitigate both class
imbalance and demographic biases.111Code available at:
https://github.com/shivashankarrs/classimb_fairness

##  1 Introduction

Class imbalance is common in many NLP tasks, including machine reading
comprehension Li et al. (2020), authorship attribution Caragea et al. (2019),
toxic language detection Breitfeller et al. (2019), and text classification
Tian et al. (2020). A skewed class distribution hurts the performance of deep
learning models Buda et al. (2018), and approaches such as instance weighting
Lin et al. (2017); Cui et al. (2019); Li et al. (2020), data augmentation
Juuti et al. (2020); Wei and Zou (2019), and weighted max-margin Cao et al.
(2019) are commonly used to alleviate the problem.

Bias in data often also manifests as skewed distributions, especially when
considered in combination with class labels. This is often referred to as
“stereotyping” whereby one or more private attributes are associated more
frequently with certain target labels, for instance more men being employed as
surgeons than women. Prior work has identified several classes of bias,
including bias towards demographic groups based on gender, disability, race or
religion Caliskan et al. (2017); May et al. (2019); Garimella et al. (2019);
Nangia et al. (2020), and bias towards individuals Prabhakaran et al. (2019).
Methods to mitigate these biases include data augmentation Badjatiya et al.
(2019), adversarial learning Li et al. (2018), instance weighting based on
group membership Kamiran and Calders (2011), regularization Wick et al.
(2019); Kennedy et al. (2020), and explicit subspace removal Bolukbasi et al.
(2016); Ravfogel et al. (2020).

Figure 1: Example of a two-class problem where grey and blue points denote
majority and minority classes, respectively, and circles and squares denote
two sub-groups. Imbalanced learning methods such as LDAM Cao et al. (2019)
maximise the (soft-)margin for minority classes and do not consider sub-groups
within each class.

This paper draws a connection between class-imbalanced learning and
stereotyping bias. Most work has focused on class-imbalanced learning and bias
mitigation as separate problems, but the unfairness caused by social biases is
often aggravated by the presence of class imbalance Yan et al. (2020). Class-
imbalanced learning approaches improve the performance of minority classes at
some cost to the performance of majority classes. A common approach re-weights
instances in the training objective to be proportional to the inverse
frequency of their class. Approaches such as FOCAL (Lin et al., 2017) and DICE
(Li et al., 2020) extend this approach by down-weighting “easy” instances.
Label-Distribution-Aware Margin Loss (“LDAM”: Cao et al. (2019)) is an
alternative approach, which encourages a larger margin for the minority class,
but it does not consider sub-group proportions (see Figure 1). On the other
hand, debiasing approaches do not typically focus on class imbalance
explicitly. For instance, in toxicity classification, certain sub-groups are
often predicted more confidently for toxicity (encouraging false negatives for
the majority sub-group), which tend to be close to the margin for the non-
toxic class (encouraging false positives; Borkan et al. (2019)).

In this work, we modify the training objective of LDAM as a state-of-the-art
approach for imbalanced learning so that margins depend not just on class-
imbalance, but also on the subgroup distribution within each class.
Specifically, we extend LDAM with popular debiasing strategies. We show the
effectiveness of our approach through several controlled experiments on two
text classification data sets.

##  2 Proposed Approaches

Let $(\mathbf{x},y,g)$ denote a training instance, comprising an input, label,
and group identifier, respectively. LDAM Cao et al. (2019) addresses class
imbalance by enforcing a larger margin for minority classes:

$\displaystyle\mathcal{L}_{\textup{LDAM}_{\mathrm{}}}(\mathbf{x},y;f)$
$\displaystyle=-\log\frac{e^{z_{y}-\Delta_{y}}}{e^{z_{y}-\Delta_{y}}+\sum_{j\neq
y}e^{z_{j}}}$ $\displaystyle\Delta_{j}$
$\displaystyle=\frac{C}{n_{j}^{1/4}}\textup{ for }j\in\\{1,\dots,k\\}$

where $\mathbf{z}=f(\mathbf{x})$ are the model outputs, $k$ is the number of
classes, $n_{j}$ is the number of instances in class $j$, and $C$ is a
hyperparameter. Smaller classes are associated with a larger $\Delta_{y}$,
which is subtracted from the model output $z_{y}$, thus enforcing a larger
margin.

We propose three extensions to LDAM, each of which takes into account
imbalance in the distribution of private attributes across classes:

#### LDAMiw

adds instance re-weighting, based on groups within each class:

$\displaystyle\mathcal{L}_{\textup{LDAM}_{\mathrm{iw}}}(\mathbf{x},y,g;f)=\mathrm{\omega}_{y,g}\mathcal{L}_{\textup{LDAM}_{\mathrm{}}}(x,y;f)$

where $g$ is the group of instance $\mathbf{x}$; and
$\mathrm{\omega}_{y,g}=\frac{1-\beta}{1-\beta^{N_{y,g}}}$ weights each
class–group combination based on its smoothed inverse frequency. $\beta$ is a
constant set to 0.9999 Cui et al. (2019) and $N_{y,g}$ is the number of
instances belonging to class $y$ and group $g$. Mistakes on minority groups
within minority classes are penalised most.

#### LDAMadv

adds an adversarial term, so that the learnt representations are a poor
predictor of group membership Li et al. (2018):

$\displaystyle\mathcal{L}_{\textup{LDAM}_{\mathrm{adv}}}$
$\displaystyle(\mathbf{x},y,g;f)=$
$\displaystyle\mathcal{L}_{\textup{LDAM}_{\mathrm{}}}(\mathbf{x},y;f)-\lambda_{\text{adv}}\mathrm{CE}(g,l(\mathbf{x}))$

where $f$ shares the lower layers of the network with the adversary $l$,
$\mathrm{CE}$ denotes cross-entropy loss, and $\lambda_{\text{adv}}$ is a
hyperparameter. This objective $\mathcal{L}_{\textup{LDAM}_{\mathrm{adv}}}$ is
jointly minimised _wrt_ $f$ and maximised _wrt_ $l$. The penalty results in
hidden representations that are informative for the main classification task
($f$), but uninformative for the adversarial group membership prediction task
($l$). The adversarial loss is implemented using gradient reversal Ganin and
Lempitsky (2015).

#### LDAMreg

adds a soft regularization term which encourages fairness as equalised odds by
reducing maximum mean discrepancy Gretton et al. (2012) across groups. The
probability of predicting some class $k$ for any individual group $g$ should
be close to $k$’s probability over the whole data set:

$\displaystyle\mathcal{L}_{\textup{LDAM}_{\mathrm{reg}}}(\mathbf{X},\mathbf{y},\mathbf{g};f)=\mathcal{L}_{\textup{LDAM}_{\mathrm{}}}(\mathbf{X},\mathbf{y};f)+$
$\displaystyle\quad\rho\sum_{g}\left\|\frac{1}{N_{g}}\sum_{i:g_{i}=g}f(\mathbf{x}_{i})-\frac{1}{N}\sum_{i}f(\mathbf{x}_{i})\right\|^{2}$

where we have moved from single instance loss to the loss over the full
training set, $N_{g}$ denotes the number of training instances in group $g$,
and hyper-parameter $\rho$ controls the trade-off between performance and
fairness.

##  3 Experimental Results

We perform experiments on the two tasks of emoji prediction and occupation
classification, both of which are binary classification tasks with binary
protected attributes.

Emoji prediction: We use the Twitter dataset of Blodgett et al. (2016), where
tweets are associated with the private attribute race (black/white), and
sentiment labels are derived from emoji usage (happy/sad) Elazar and Goldberg
(2018). We experiment with different levels of class and stereotyping
imbalance in the Emoji dataset, including its original distribution (see
Section 3.1).

Occupation classification: This data set consists of short biographies scraped
from the web, annotated for private attribute gender (male/female) and target
occupation labels De-Arteaga et al. (2019). We focus on two occupations with
well-documented gender stereotypes – surgeon and nurse. The resulting dataset
is mildly class-imbalanced (59% surgeon: 41% nurse), with roughly symmetric
natural gender splits (90% male for surgeon and 90% female for nurse).

For emoji prediction we follow Ravfogel et al. (2020) and use the DeepMoji
encoder, which was trained on millions of tweets and is known to encode
demographic information Elazar and Goldberg (2018). For occupation
classification, we use the BERT-base uncased model and classify via the last
hidden state of the CLS token Devlin et al. (2019). Both encoders are followed
by a single hidden layer MLP.

We evaluate classification performance based on macro-averaged F-score (to
account for class imbalance), and evaluate fairness using performance GAP: the
average of the true positive rate (TPR) and true negative rate (TNR)
differences between the two subgroups De-Arteaga et al. (2019); Ravfogel et
al. (2020). Note that a wide variety of fairness measures (both on the group-
and individual levels) have been proposed, which are impossible to satisfy
simultaneously Garg et al. (2020). Often, a suitable measure is chosen based
on the target application. Here we use the popular equalised odds measure
considering both TPR and TNR of classifiers, in order to address scenarios
where certain sub-groups are predicted more often with some classes (see
Section 1). We report fairness as $1-$GAP, such that higher numbers are
better, and a perfectly fair model achieves $1{-}\operatorname{GAP}=1$. We
compare our methods against the following benchmarks:

vanilla:

unweighted cross-entropy loss.

FOCAL:

re-weights easy examples during training Lin et al. (2017).

CW:

instance re-weighting based on the inverse class proportion and cross-entropy.

IW:

instance re-weighting based on the combination of inverse class and group
proportions, and cross-entropy Kamiran and Calders (2011).

INLP:

Iterative null-space projection Ravfogel et al. (2020): in each iteration, we
learn a SVM classifier $W$ using hidden representations ($X_{h}$) as the
independent variables to predict the protected attribute, where $X_{h}$ is
projected onto the nullspace of $W$ to remove the protected information.

LDAM:

the original LDAM model Cao et al. (2019).

LDAMcw:

a variant of LDAM with instance re-weighting by inverse class proportion Cao
et al. (2019).

.4.6.8.8.911 -
GAP(a)LDAMregLDAMadvINLPLDAMLDAMcwLDAMiwFOCALvanillaIWCW.5.6.7.8.4.6.81F-score(b).5.6.7.8.4.6.81(c)
Figure 2: F-score vs. fairness ($1{-}\operatorname{GAP}$) across the (a)
original, (b) 90/90 and (c) 95/95 setting on the emoji prediction task. Models
which balance fairness vs. performance through hyperparameters are shown as
Pareto frontiers, while the others are reported as single points.
.4.6.81.91F-score1 - GAPLDAMregLDAMadvINLPLDAMLDAMcwLDAMiwFOCALvanillaIWCW
Figure 3: F-score vs. $1{-}\operatorname{GAP}$ on occupation classification
(original class balance and stereotyping) for the same set of models as in
Figure 1.

###  3.1 Model Comparison

| F-score |  | $1{-}\operatorname{GAP}$
---|---|---|---
Ratio | vanilla | INLP | LDAMiw | LDAMreg | LDAMadv | ADV |  | vanilla | INLP | LDAMiw | LDAMreg | LDAMadv | ADV
0.5 | 0.76 | 0.75 | 0.76 | 0.75 | 0.75 | 0.76 |  | 0.87 | 0.88 | 0.89 | 0.92 | 0.91 | 0.83
0.6 | 0.76 | 0.71 | 0.76 | 0.74 | 0.74 | 0.75 |  | 0.79 | 0.82 | 0.80 | 0.92 | 0.91 | 0.80
0.7 | 0.74 | 0.65 | 0.75 | 0.74 | 0.73 | 0.75 |  | 0.70 | 0.84 | 0.73 | 0.93 | 0.89 | 0.78
0.8 | 0.72 | 0.62 | 0.74 | 0.73 | 0.73 | 0.74 |  | 0.61 | 0.84 | 0.67 | 0.93 | 0.72 | 0.76

Table 1: Performance and Fairness on the Emoji data set with fixed balanced
class-distribution, but varying the stereotyping ratio (black:white) per
class. The ratio column denotes the % of black instances relative to white.
The test-set is stereotype balanced (50:50).

| F-score |  | $1{-}\operatorname{GAP}$
---|---|---|---
Ratio | vanilla | INLP | LDAMcw | LDAMiw | LDAMreg | LDAMadv |  | vanilla | INLP | LDAMcw | LDAMiw | LDAMreg | LDAMadv
0.7 | 0.83 | 0.80 | 0.83 | 0.82 | 0.80 | 0.80 |  | 0.62 | 0.73 | 0.60 | 0.75 | 0.84 | 0.83
0.8 | 0.80 | 0.77 | 0.83 | 0.80 | 0.77 | 0.78 |  | 0.64 | 0.80 | 0.61 | 0.76 | 0.84 | 0.85
0.9 | 0.74 | 0.72 | 0.79 | 0.75 | 0.72 | 0.74 |  | 0.70 | 0.79 | 0.62 | 0.84 | 0.85 | 0.82

Table 2: Performance and Fairness on the Emoji data set, fixing the
stereotyping ratio to 0.8:0.2 (black:white) per class, and varying the class-
balance ratio (proportion of positive class is shown). The test sets in each
row are different, and mimic the class-imbalance and stereotyping of the
training data (i.e. results across rows are not comparable).

We include simulated experimental settings with the emoji dataset following
Ravfogel et al. (2020) where they keep the class proportions balanced, but
vary group proportions (stereotyping). In our work, we systematically vary
both class imbalance and stereotyping, in order to assess the robustness of
the models wrt class imbalance and fairness individually. We explore three
settings: varying both dimensions at the same time (Figure 2), controlling for
class imbalance and vary stereotyping (Table 1), and controlling for
stereotyping while varying class imbalance (Table 2).

We simultaneously vary stereotyping and class imbalance in the emoji dataset,
exploring several settings:

  * •

Original: the dataset is sampled based on the natural class distribution (70%
positive; Blodgett et al. (2016)), and within each class the black:white ratio
is set to 18:82, based on US census estimates.

  * •

90/90: the class distribution is skewed (90% positive), and black:white ratio
is set to 90:10 for positive tweets and 10:90 for negative tweets (i.e.
“stereotyping” the classes).

  * •

95/95: as per the above, but with class skew and stereotyping ratios set to
95:5.

For the occupation classification task, the original data is used as is
(Figure 3).

#### Model Selection.

For models with hyper-parameters which trade off performance and fairness, the
optimal balance of F-score and fairness is not clear, so we adopt the concept
of Pareto optimality Godfrey et al. (2007) and present the Pareto frontier in
the graphs. In particular, for LDAMreg and LDAMadv, we perform a hyper-
parameter search over $C$ ($10^{-2}$ to 30), $\rho$ ($10^{-4}$ to $10^{2}$),
and $\lambda$ ($10^{-4}$ to $10^{2}$). In general, a higher $C$ prioritises
F-score over fairness, and a higher $\rho$ and $\lambda$ prioritise fairness.
For INLP, we tune the number of iterations as a hyper-parameter. The remaining
models don’t have trade-off hyper-parameters, so we report a single-point best
model: for LDAM and LDAMcw we tune $C$ by choosing the best-performing model
over the dev set. For LDAMiw, we set $\beta$ to 0.9999 following Cui et al.
(2019), and tune $C$ to identify the fairest model on the dev set.

The results in Figure 2 (a)–(c) show that LDAMreg is overall superior to the
other approaches, especially for higher F-scores. For increasingly extreme
levels of class imbalance and stereotyping (as we move to the right in the
figure), the advantage of LDAMreg over LDAMadv and INLP decreases
substantially. Across all the settings, LDAMcw has the highest bias (is least
fair). With higher class imbalance and stereotyping, most class-imbalanced
learning methods—FOCAL, CW and LDAMcw—exhibit high bias. In the stereotyping
settings, LDAMiw reduces bias compared to IW.

Analogous results on the occupation data (original class proportions) are in
Figure 3. Once again, the proposed LDAM extensions perform the best overall,
with LDAMreg achieving the best trade-off in performance. Class-imbalanced
learning approaches (FOCAL, LDAM, and LDAMcw) are most biased on this dataset,
with IW improving fairness over vanilla cross-entropy training, and LDAMiw
providing large improvements in fairness.

###  3.2 Stereotyping-Class balance Trade-off

In addition to comparing models based on the trade-off between performance and
fairness in class-imbalanced learning, we wish to disentangle the effect of
stereotyping from class imbalance. We do so by: (a) fixing class balance to
50:50 and varying stereotyping (Table 1); and (b) fixing stereotyping to a
symmetric 0.8 while varying class imbalance (Table 2). A stereotyping level of
symmetric 0.8 means 80:20 black:white for positive and 20:80 black:white for
negative tweets. We perform model selection by choosing the INLP model with
best harmonic mean of performance and fairness for Table 2, and use the
results from the original paper for Table 1. We select all other models by
first selecting from models with F-score at least as high as INLP, and then
selecting the one with the lowest GAP. We include a recent adversarial model
in the varying stereotyping experiments, which performed strongly on the
class-balanced emoji data (ADV: Han et al. (2021)).

Our results on varying stereotyping levels in Table 1 show that the vanilla
baseline drops in performance more sharply than most proposed models, and
results in the most unfair predictions by a large margin. LDAMiw, LDAMadv, and
ADV retain high F-scores but drop in fairness with increasing stereotyping,
while INLP exhibits the opposite pattern. LDAMreg achieves the best balance of
F-score and fairness. Table 2 presents results for fixed stereotyping and
varying class imbalance (0.7–0.9 positive). We include LDAMcw for handling
class imbalance but exclude ADV, which does not address class-imbalance
directly. We observe that LDAMcw has the highest F-score, but scores poorly
for fairness. LDAMiw achieves the best trade-off with high class-imbalance,
but shows large variation across settings. LDAMreg appears more stable,
exhibiting a good performance–fairness trade-off.

##  4 Conclusion and Future Work

We explored the interplay of class-imbalance and stereotyping in two language
classification data sets. We showed that vanilla class-imbalanced learning
(IW, CW, FOCAL, LDAM and LDAMcw) can exacerbate unfairness. We extended class-
imbalanced learning approaches to handle fairness under stereotyping, and
showed that our models provide consistent gains in fairness without
sacrificing accuracy. Both LDAMreg which uses maximum mean discrepancy
regularizer Tzeng et al. (2014) and LDAMadv with adversarial loss Ganin and
Lempitsky (2015) are different ways to make the text representation
independent of demographic attributes. Consistent with previous work Louizos
et al. (2016) we find that LDAMreg is robust and performs best across several
test scenarios, except in extremely skewed (or stereotyped) settings where the
gains of LDAMreg over its adversarial counterpart (LDAMadv) diminishes. In
addition, LDAMadv introduces more parameters into the model, and is in general
hard to train, hence LDAMreg is more preferable overall. In the future, we
plan to extend our methods to more complex tasks and multiple private
attributes Subramanian et al. (2021).

##  5 Acknowledgement

We thank Xudong Han for the discussions and inputs. This work was funded in
part by the Australian Government Research Training Program Scholarship, and
the Australian Research Council.

## References

  * Badjatiya et al. (2019) Pinkesh Badjatiya, Manish Gupta, and Vasudeva Varma. 2019.  Stereotypical bias removal for hate speech detection task using knowledge-based generalizations.  In _The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019_ , pages 49–59. ACM.
  * Blodgett et al. (2016) Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016.  Demographic dialectal variation in social media: A case study of African-American English.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1119–1130, Austin, Texas. Association for Computational Linguistics.
  * Bolukbasi et al. (2016) Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam Tauman Kalai. 2016.  Man is to computer programmer as woman is to homemaker? debiasing word embeddings.  In _Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain_ , pages 4349–4357.
  * Borkan et al. (2019) Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019\.  Nuanced metrics for measuring unintended bias with real data for text classification.  In _Companion Proceedings of The 2019 World Wide Web Conference_ , WWW ’19, page 491–500, New York, NY, USA. Association for Computing Machinery.
  * Breitfeller et al. (2019) Luke Breitfeller, Emily Ahn, David Jurgens, and Yulia Tsvetkov. 2019.  Finding microaggressions in the wild: A case for locating elusive phenomena in social media posts.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 1664–1674, Hong Kong, China. Association for Computational Linguistics.
  * Buda et al. (2018) Mateusz Buda, Atsuto Maki, and Maciej A. Mazurowski. 2018.  A systematic study of the class imbalance problem in convolutional neural networks.  _Neural Networks_ , 106:249–259.
  * Caliskan et al. (2017) Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017.  Semantics derived automatically from language corpora contain human-like biases.  _Science_ , 356(6334):183–186.
  * Cao et al. (2019) Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Aréchiga, and Tengyu Ma. 2019.  Learning imbalanced datasets with label-distribution-aware margin loss.  In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , pages 1565–1576.
  * Caragea et al. (2019) Cornelia Caragea, Ana Uban, and Liviu P. Dinu. 2019.  The myth of double-blind review revisited: ACL vs. EMNLP.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2317–2327, Hong Kong, China. Association for Computational Linguistics.
  * Cui et al. (2019) Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge J. Belongie. 2019.  Class-balanced loss based on effective number of samples.  In _IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019_ , pages 9268–9277. Computer Vision Foundation / IEEE.
  * De-Arteaga et al. (2019) Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019.  Bias in bios: A case study of semantic representation bias in a high-stakes setting.  In _Proceedings of the Conference on Fairness, Accountability, and Transparency_ , FAT* ’19, page 120–128, New York, NY, USA. Association for Computing Machinery.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Elazar and Goldberg (2018) Yanai Elazar and Yoav Goldberg. 2018.  Adversarial removal of demographic attributes from text data.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 11–21, Brussels, Belgium. Association for Computational Linguistics.
  * Ganin and Lempitsky (2015) Yaroslav Ganin and Victor S. Lempitsky. 2015.  Unsupervised domain adaptation by backpropagation.  In _Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015_ , volume 37 of _JMLR Workshop and Conference Proceedings_ , pages 1180–1189. JMLR.org.
  * Garg et al. (2020) Pratyush Garg, John Villasenor, and Virginia Foggo. 2020.  Fairness metrics: A comparative analysis.  In _2020 IEEE International Conference on Big Data (Big Data)_ , pages 3662–3666. IEEE.
  * Garimella et al. (2019) Aparna Garimella, Carmen Banea, Dirk Hovy, and Rada Mihalcea. 2019.  Women’s syntactic resilience and men’s grammatical luck: Gender-bias in part-of-speech tagging and dependency parsing.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3493–3498, Florence, Italy. Association for Computational Linguistics.
  * Godfrey et al. (2007) Parke Godfrey, Ryan Shipley, and Jarek Gryz. 2007.  Algorithms and analyses for maximal vector computation.  _The VLDB Journal_ , 16(1):5–28.
  * Gretton et al. (2012) Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. 2012.  A kernel two-sample test.  _The Journal of Machine Learning Research_ , 13(1):723–773.
  * Han et al. (2021) Xudong Han, Timothy Baldwin, and Trevor Cohn. 2021.  Diverse adversaries for mitigating bias in training.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 2760–2765, Online. Association for Computational Linguistics.
  * Juuti et al. (2020) Mika Juuti, Tommi Gröndahl, Adrian Flanagan, and N. Asokan. 2020.  A little goes a long way: Improving toxic language classification despite data scarcity.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 2991–3009, Online. Association for Computational Linguistics.
  * Kamiran and Calders (2011) F. Kamiran and T. Calders. 2011.  Data preprocessing techniques for classification without discrimination.  _Knowledge and Information Systems_ , 33:1–33.
  * Kennedy et al. (2020) Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Davani, Morteza Dehghani, and Xiang Ren. 2020.  Contextualizing hate speech classifiers with post-hoc explanation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5435–5442, Online. Association for Computational Linguistics.
  * Li et al. (2020) Xiaoya Li, Xiaofei Sun, Yuxian Meng, Junjun Liang, Fei Wu, and Jiwei Li. 2020.  Dice loss for data-imbalanced NLP tasks.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 465–476, Online. Association for Computational Linguistics.
  * Li et al. (2018) Yitong Li, Timothy Baldwin, and Trevor Cohn. 2018.  Towards robust and privacy-preserving text representations.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 25–30, Melbourne, Australia. Association for Computational Linguistics.
  * Lin et al. (2017) Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár. 2017.  Focal loss for dense object detection.  In _IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017_ , pages 2999–3007. IEEE Computer Society.
  * Louizos et al. (2016) Christos Louizos, Kevin Swersky, Yujia Li, Max Welling, and Richard S. Zemel. 2016\.  The variational fair autoencoder.  In _4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings_.
  * May et al. (2019) Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019\.  On measuring social biases in sentence encoders.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 622–628, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Nangia et al. (2020) Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020.  CrowS-pairs: A challenge dataset for measuring social biases in masked language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1953–1967, Online. Association for Computational Linguistics.
  * Prabhakaran et al. (2019) Vinodkumar Prabhakaran, Ben Hutchinson, and Margaret Mitchell. 2019.  Perturbation sensitivity analysis to detect unintended model biases.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5740–5745, Hong Kong, China. Association for Computational Linguistics.
  * Ravfogel et al. (2020) Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020\.  Null it out: Guarding protected attributes by iterative nullspace projection.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7237–7256, Online. Association for Computational Linguistics.
  * Subramanian et al. (2021) Shivashankar Subramanian, Xudong Han, Timothy Baldwin, Trevor Cohn, and Lea Frermann. 2021.  Evaluating debiasing techniques for intersectional biases.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_.
  * Tian et al. (2020) Jiachen Tian, Shizhan Chen, Xiaowang Zhang, and Zhiyong Feng. 2020.  A graph-based measurement for text imbalance classification.  In _24th European Conference on Artificial Intelligence_.
  * Tzeng et al. (2014) Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014.  Deep domain confusion: Maximizing for domain invariance.  _arXiv e-prints_ , pages arXiv–1412.
  * Wei and Zou (2019) Jason Wei and Kai Zou. 2019.  EDA: Easy data augmentation techniques for boosting performance on text classification tasks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 6382–6388, Hong Kong, China. Association for Computational Linguistics.
  * Wick et al. (2019) Michael L. Wick, Swetasudha Panda, and Jean-Baptiste Tristan. 2019.  Unlocking fairness: a trade-off revisited.  In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , pages 8780–8789.
  * Yan et al. (2020) Shen Yan, Hsien-Te Kao, and Emilio Ferrara. 2020.  Fair class balancing: Enhancing model fairness without observing sensitive attributes.  In _CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020_ , pages 1715–1724. ACM.


# Insights into neutron star equation of state by machine learning

Ling-Jun Guo  College of Physics, Jilin University, Changchun, 130012, China
School of Fundamental Physics and Mathematical Sciences, Hangzhou Institute
for Advanced Study, UCAS, Hangzhou, 310024, China  Jia-Ying Xiong  School of
Fundamental Physics and Mathematical Sciences, Hangzhou Institute for Advanced
Study, UCAS, Hangzhou, 310024, China  Institute of Theoretical Physics,
Chinese Academy of Sciences, Beijing 100190, China  Yao Ma<EMAIL_ADDRESS>School of Fundamental Physics and Mathematical Sciences, Hangzhou Institute
for Advanced Study, UCAS, Hangzhou, 310024, China  University of Chinese
Academy of Sciences, Beiing 100049, China  TaiJi Laboratory for Gravitational
Wave Universe (Beijing/Hangzhou), University of Chinese Academy of Sciences,
Beijing, 100049, China  Yong-Liang Ma<EMAIL_ADDRESS>School of Fundamental
Physics and Mathematical Sciences, Hangzhou Institute for Advanced Study,
UCAS, Hangzhou, 310024, China  TaiJi Laboratory for Gravitational Wave
Universe (Beijing/Hangzhou), University of Chinese Academy of Sciences,
Beijing, 100049, China  International Center for Theoretical Physics Asia-
Pacific (ICTP-AP) , UCAS, Beijing, 100190, China

###### Abstract

Due to its powerful capability and high efficiency in big data analysis,
machine learning has been applied in various fields. We construct a neural
network platform to constrain the behaviors of the equation of state of
nuclear matter with respect to the properties of nuclear matter at saturation
density and the properties of neutron stars. It is found that the neural
network is able to give reasonable predictions of parameter space and provide
new hints into the constraints of hadron interactions. As a specific example,
we take the relativistic mean field approximation in a widely accepted
Walecka-type model to illustrate the feasibility and efficiency of the
platform. The results show that the neural network can indeed estimate the
parameters of the model at a certain precision such that both the properties
of nuclear matter around saturation density and global properties of neutron
stars can be saturated. The optimization of the present modularly designed
neural network and extension to other effective models are straightforward.

##  I Introduction and Motivation

With the development of technologies, the amount of experimental data is
increasing rapidly. These massive data are usually from different experimental
targets and provide information on different aspects of the same system.
Therefore, in order to get a complete understanding of the physics involved,
one needs to combine these data and analyze them systematically. However, it’s
usually a challenge for researchers to handle this kind of process due to the
complexity of the data and the massive parameters in the model. The recently
developed machine learning (ML) or artificial intelligence(AI)-driven
technologies provide a way out. The ML methods have already earned credits in
the fields of big data analysis due to their advantages of efficiency and
adaptivity Krizhevsky  _et al._ (2012); LeCun  _et al._ (2016); Jordan and
Mitchell (2015); Li  _et al._ (2019a), and they have already been applied in
many different fields of physics, e.g., Refs. Mehta  _et al._ (2019); Abraham
_et al._ (2018); Niu  _et al._ (2019); Gu  _et al._ (2020); Brady  _et al._
(2021); Boehnlein  _et al._ (2022); Oala  _et al._ (2023); He  _et al._
(2023a, b) and references therein.

In nuclear physics, the properties of nuclear matter (NM) have been
investigated for a long period but no consensus has been arrived at. Several
fundamental questions are waiting for clarification, for example, what are the
constituents of the NM, whether is a phase transition involved in the dense
compact star matter or not, is dense nuclear matter in other states than
Fermi-liquid, et al., (see, e.g., reviews Refs. Brown and Rho (2002); Hayano
and Hatsuda (2010); Holt  _et al._ (2016); Baym  _et al._ (2018); Ma and Rho
(2020); Li  _et al._ (2019b); Ma and Rho (2021); Burgio  _et al._ (2021); Ma
and Yang (2023) and references therein). To resolve these questions, all the
existing information from both experiments and theories should be combined in
the corresponding analysis and a reliable technique, like ML developed here,
is necessary.

The constraints on nuclear matter come from both terrestrial experiments and
astrophysical observations. Owing to the analysis of the structures of heavy
nuclei, e.g., ${}^{24}\mathrm{Mg},{}^{90}\mathrm{Zr},{}^{116}\mathrm{Sn}$ and
${}^{208}\mathrm{~{}Pb}$, and the data from heavy-ion collisions, one can
provide information about NM properties around nuclear saturation density
($n_{0}\approx 0.16\mathrm{fm}^{-3}$) Youngblood  _et al._ (1999); Steiner and
Li (2005); Karataglidis  _et al._ (2002); Brown (2000), such as the binding
energy of nucleon $e_{0}$, the symmetry energy $E_{\mathrm{sym}}$, the
incompressibility coefficient $K_{0}$, the skewness coefficient $L_{0}$, and
so on. Besides the above information obtained from terrestrial experiments,
there are also constraints from astrophysics. The signals from, e.g., PSR
J1614-2230, J0348+043, PSR J0740+6620, J0030+0451 and PSR J0740+662 constrain
the mass-radius (MR) relation of neutron stars (NSs) Demorest  _et al._
(2010); Antoniadis  _et al._ (2013); Fonseca  _et al._ (2016); Özel and Freire
(2016); Cromartie  _et al._ (2019); Fonseca  _et al._ (2021). The observation
of gravitational waves (GWs) from the binary neutron star (BNS) merger,
GW170817 Abbott  _et al._ (2017, 2018), yields a new independent
constraints—tidal deformations—on the MR relations more rigorously Annala  _et
al._ (2018, 2022) and the multi-messenger era of NSs has begun Hartley (2017);
Collaboration  _et al._ (2018). Therefore, in the study on NM, both the data
from terrestrial experiments and astrophysical abservations should be
considered and these massive data increase the difficulty of the analysis
process.

On the other hand, the equation of state (EoS) plays a key role in the studies
on NM, and it’s usually parameterized by effective models or theories, e.g.,
one boson exchange (OBE) model Machleidt  _et al._ (1987) and chiral effective
field theory ($\chi{\mathrm{EFT}}$) Epelbaum  _et al._ (2009); Ma and Rho
(2019), due to the nonperturbative nature of quantum chromodynamics (QCD). The
information of QCD is encoded into the low energy constants (LECs) of these
models and theories, which are determined by fitting the experimental data,
but the fitting process is somehow fine-tuning because of the cancellation
mechanism between attractive and repulsive terms. It’s easy to imagine that
this kind of searching of parameter spaces can be quite a labor- and time-
consuming work.

In practice, the simultaneous treatment of the massive data and fine-tuning of
the LECs to obtain a physical EoS is very difficult if not unfeasible. With
the help of ML methods, the cost of the labor- and time-consuming work can be
reduced. The recently developed artificial intelligence (AI) technologies
Vaswani  _et al._ (2017); Long  _et al._ (2015), applying ML methods simplify
such process and some works have already been done along this line recently,
see Refs. Drischler  _et al._ (2020); Ferreira and Providência (2021); Annala
_et al._ (2022) for details.

The purpose of this work is to propose a ML platform to carry out the above
idea. We shall construct a neural network (NN) with respect to the properties
of NM around saturation density and the MR relations of NSs to constrain the
behaviors of EoS. A specific but widely accepted nucleon force model is
adopted as a preliminary illustration of the efficiency of this NN. The
platform developed can be easily extended to other complex, general and
practical models.

##  II Neural network framework and its application

The structure of our NN platform for the parameter-searching processes is
shown in Fig. 1.

Figure 1: Structure of NN platform.

Concretlty, the strategy of the NN is the following: At first, a model or
theory with its pre-chosen parameter space should be given to train the
computation module of NN, and it doesn’t matter whether the pre-chosen
parameter space is a physical one or not. Then, the trained computation module
predicts possible values of parameters according to the Bayesian module
calculation by experimental data from NM and NS. If the self-supervised module
judges that the data set indicates a parameter set beyond the training set, it
will make possible predictions and generate a corresponding training set. In
this situation, the new training set and the previous one will be combined by
an adaptive algorithm to train the computation module again, and this process
will repeat till the probability from Bayesian analysis converges to a fixed
value, which represents the confidence level of the input model.

###  II.1 Application in a specific model

Afrer a discussion on the basic idea of the NN, we illustrate its application
in this part by using a widely used nuclear force model Sugahara and Toki
(1994); Shen  _et al._ (1998). The model takes the following form:

$\displaystyle\mathcal{L}_{\mathrm{RMF}}$ $\displaystyle=$
$\displaystyle\bar{\psi}\left[i\gamma_{\mu}\partial^{\mu}-M-g_{\sigma}\sigma-
g_{\omega}\gamma_{\mu}\omega^{\mu}-g_{\rho}\gamma_{\mu}\tau_{a}\rho^{a\mu}\right]\psi$
(1)
$\displaystyle{}+\frac{1}{2}\partial_{\mu}\sigma\partial^{\mu}\sigma-\frac{1}{2}m_{\sigma}^{2}\sigma^{2}-\frac{1}{3}g_{2}\sigma^{3}-\frac{1}{4}g_{3}\sigma^{4}$
$\displaystyle{}-\frac{1}{4}W_{\mu\nu}W^{\mu\nu}+\frac{1}{2}m_{\omega}^{2}\omega_{\mu}\omega^{\mu}+\frac{1}{4}c_{3}\left(\omega_{\mu}\omega^{\mu}\right)^{2}$
$\displaystyle{}-\frac{1}{4}R_{\mu\nu}^{a}R^{a\mu\nu}+\frac{1}{2}m_{\rho}^{2}\rho_{\mu}^{a}\rho^{a\mu},$

where $\psi$ is the iso-doublet of nucleon field with mass $M$, $\sigma$ is
the iso-scalar scalar meson field and, $\omega^{\mu}$ and $\rho^{\mu}$ are,
respectively, the iso-scalar and iso-vector meson fields with $W^{\mu\nu}$ and
$R^{a\mu\nu}$ being their field strength tensors

$\displaystyle W^{\mu\nu}$ $\displaystyle=$
$\displaystyle\partial^{\mu}\omega^{\nu}-\partial^{\nu}\omega^{\mu}\ ,$
$\displaystyle R^{a\mu\nu}$ $\displaystyle=$
$\displaystyle\partial^{\mu}\rho^{a\nu}-\partial^{\nu}\rho^{a\mu}+g_{\rho}\epsilon^{abc}\rho^{b\mu}\rho^{c\nu}\
.$ (2)

In this model, the nucleon-meson coupling terms are the most simple one-boson-
exchange type interactions, the non-linear $\sigma$ terms are crucial to
describe incompressibility and, the non-linear $\omega$ terms are added in
order to reproduce the density dependence of nucleon self-energy.

In practice, to calculate the nuclear matter properties using model (1), some
approximations should be applied. In the relativistic mean field (RMF)
approximation which is widely used, the problem is reduced to solve the
following coupled equations of motion (EoMs):

$\displaystyle m_{\sigma}^{2}\sigma+g_{2}\sigma^{2}+g_{3}\sigma^{3}$
$\displaystyle=$
$\displaystyle{}-g_{\sigma}\left(\rho_{n,s}+\rho_{p,s}\right)\ ,$
$\displaystyle m_{\omega}^{2}\omega+c_{3}\omega^{3}$ $\displaystyle=$
$\displaystyle g_{\omega}\left(\rho_{p}+\rho_{n}\right)\ ,$ $\displaystyle
m_{\rho}^{2}\rho$ $\displaystyle=$ $\displaystyle
g_{\rho}\left(\rho_{p}-\rho_{n}\right)\ ,$ (3)

where $\rho_{n(p)}$ and $\rho_{n,s(p,s)}$ are, respectively, the density and
scalar density of neutron (proton)

$\displaystyle\rho_{n(p),s}$ $\displaystyle=$
$\displaystyle\frac{m_{N}^{*3}}{\pi^{2}}\int_{0}^{t_{n(p)}}{\rm{d}}x\frac{x^{2}}{\sqrt{1+x^{2}}}$
(4) $\displaystyle=$
$\displaystyle\frac{m_{N}^{*3}}{\pi^{2}}\left[\frac{1}{2}\left(t_{n(p)}\sqrt{1+t_{n(p)}^{2}}-{\rm{arcsinh}}{t_{n(p)}}\right)\right]\
,$

with $m_{N}^{*}=M+g_{\sigma}\sigma$ being the effective mass of nucleons and
$t_{n(p)}=\frac{(3\pi^{2}\rho_{n(p)})^{1/3}}{m_{N}^{*}}$. After determining
the expectations of meson fields by solving the EoSs (3), the energy density
$\mathcal{E}$ can be obtained by calculating Hamiltonian and the pressure $P$
can be obtained via
$P=-\mathcal{E}+\rho_{N}\frac{\rm{d}\mathcal{E}}{\rm{d}\rho_{N}}$.

Usually, in a model of nuclear matter, the solution of the EoMs and the
prediction of the EoS suffer from the constraints from the nuclear matter
properties around saturation density and the mathematical structure of the
EoMs. From the mathematical structure of the EoMs (3), it can be seen that
even in the simple Model (1), the EoMs have the multi-roots problem, which
seemingly lead to the NN convergence problem. But one can expect the results
given by the NN are smooth and physical due to the mathematical reasons: (i)
Identity theorem of analytic functions; (ii) The full rank properties of a
equation group coefficient matrix. Therefore, the solution planes of Eq. (3)
will only intersect or be tangent if the natural condition is assumed. In
either case, the solution plane given by NN will be defined uniquely by
choosing a starting point, which will be the origin point in field expectation
solution space in the following discussions.

In the explicit calculation, the computation module of NN #1#1#1The NN in this
work is built on PyTorch platform. Paszke  _et al._ to solve above equation
group are based on forty fully connected layers, where there is a batch
normalization layer between each two fully connected layers to accelerate the
training Ioffe and Szegedy (2015). The fully connected layers are constructed
as Fig. 2.

Figure 2: Layers to solve EoMs of meson fields. ReLU is used as activation
function Glorot  _et al._ (2011). The layers construction for $\rho$ field is
ignored in current work for simplicity, since its EoM in Eq. (3) is just a
linear function of nucleon density.

The loss function in above NN is defined as

$\displaystyle f_{\rm{loss},\sigma}$ $\displaystyle=$
$\displaystyle\left[m_{\sigma}^{2}\sigma+g_{2}\sigma^{2}+g_{3}\sigma^{3}+g_{\sigma}\left(\rho_{n,s}+\rho_{p,s}\right)\right]^{2}\
,$ $\displaystyle f_{\rm{loss},\omega}$ $\displaystyle=$
$\displaystyle\left[m_{\omega}^{2}\omega+c_{3}\omega^{3}-g_{\omega}\left(\rho_{p}+\rho_{n}\right)\right]^{2}\
,$ (5)

which enhances the ability of NN to give a smooth and physical prediction. The
pre-training set is generated randomly at regions where $\left\\{\rho_{n},\
\rho_{p}\right\\}\in\left[\left\\{0,0\right\\},\left\\{10n_{0},10n_{0}\right\\}\right]$
with distribution function
$\mathcal{P}(\rho_{n},\rho_{p})=\frac{\sqrt{10}}{20}\sqrt{\frac{n_{0}}{\rho_{n}+\rho_{p}}}$
in order to improve the ability to describe low density regions. The learning
process is done with the help of Adam algorithm Kingma and Ba (2014). Based on
the results of so solved EoMs, the EoS can be obtained from Lagrangian (1) and
star properties can be calculated by solving the Tolman-Oppenheimer-Volkoff
(TOV) equation Tolman (1939); Oppenheimer and Volkoff (1939).

In the numerical calculation by using the NN constructed above, we impose the
constraints from the neutron star properties and the nuclear matter properties
around saturation density. Our choices are list in Table 1 which are estimated
from Abbott  _et al._ (2018, 2019); Dutra  _et al._ (2012, 2014); Choi  _et
al._ (2021); Colo  _et al._ (2004)

Table 1:  $M$ and $R$ is the mass and radius of the primary star in GW170817. $e_{0}$ is the binding energy of nucleon, $E_{\mathrm{sym}}(n)=\left.\frac{1}{2}\frac{\partial^{2}E(n,\alpha)}{\partial\alpha^{2}}\right|_{\alpha=0}$ is the symmetry energy and $K_{0}=\left.9n_{0}^{2}\frac{\partial^{2}E(n,0)}{\partial n^{2}}\right|_{n=n_{0}}$ is the incompressibility coefficient at $n_{0}$, respectively. Normal distribution is assumed for data error bars with $95\%$ confidence level. |  $M$ ($M_{\odot}$) |  $R$ (km) |  $n_{0}$ (fm-3) |  $e_{0}$ (MeV) |  $E_{\rm sym}$ (MeV) |  $K_{0}$ (MeV)
---|---|---|---|---|---|---
Constraints |  $1.48\pm 0.12$ |  $11.9\pm 1.4$ |  $0.155\pm 0.05$ | $-16\pm 1.0$ |  $32.0\pm 2.0$ |  $250\pm 50$
Optimal |  |  |  $0.161$ | $-15.7$ |  $28.6$ |  $266$

With respect to the constraints shown in Table 1 , it’s ready to calculate the
parameters of Model (1) using the NN constructed in this work. Our results of
the optimal solutions are shown in Table 2 #2#2#2 The weight of parameter
point is defined as $\mathcal{W}=\prod\mathcal{P}_{i}$, where $\mathcal{P}$
refers to probability density and $i$ represents physical quantities.. The
optimal values of the parameter set lead to the NM properties shown in Table 1
and the corresponding MR relation illustrated in Fig. 3. From the NM
properties and the MR relation, one can see that the optimal values from NN
are globally consistent with the empirical values and the observations from
GW170817. This demonstrates the rationality of the NN approach.

Table 2:  Optimal values of the parameters calculated from NN. The emprical values $M_{N}=938$ MeV, $m_{\omega}=782$ MeV and $m_{\rho}=765$ are physical values at vacuum taken as inputs for simplicity. $g_{\sigma}$ | $g_{\omega}$ | $g_{\rho}$ | $g_{3}$ | $c_{3}$ | $g_{2}$ | $m_{\sigma}$
---|---|---|---|---|---|---
$9.82$ |  $11.8$ |  $3.42$ |  $1.26$ |  $72.6$ |  $-1550$ MeV |  $531$ MeV
Figure 3: MR relation obtained by the optimal solution.

(a) $e_{0}$ with NM property constraints.

(b) $E_{\rm{sym}}$ with NM property constraints.

(c) MR relations with MR constraints.

(d) MR relations without MR constraints.

Figure 4: Regions of the NM properties and MR relations predicted by NN.

We next plot the regions of the NM properties and the MR relations from the
predicted parameter space with $10.1\%$ confidence level within the data error
band #3#3#3 The confidence level is calculated by
$P_{c}=\int_{V}J\mathcal{W}{\mathrm{d}}V$, where the measure $V$ is the
parameter space region with $O$ being the optimal point and $J$ refers to the
determinant of Jacobian matrix mapping data space to parameter space. . In
Fig. 4, the bands of the NM properties with and without GW170817 data are
presented. One can see that the inclusion of the constraints from GW170817
shrinks the bands and softens the EoS globally. This observation shows the
necessity to analyze both the constraints from NS relations and NM properties
simultaneously.

From Fig. 3 and Table 1, one can see an interesting fact that predicted
quantities by NN with $7$ free parameters are not located at the center values
of 6 constraints. The possible reasons are: (i) The parameter space is
redundant; (ii) The degree of data’s freedom exceeds $7$. The redundancy of
parameter space means that at least two parameters in this system can be
expressed by one parameter, which indicates the optimal solution is actually
an open area not a specific point in the parameter space. After scanning the
area around the optimal solution listed in Table 2, we find that it’s indeed a
point, which excludes the possibility of the redundancy of the parameter
space. Then the left reason is just that the dimension of data space is larger
than $7$. This is imaginable since the MR relation is obtained via the
integral of EoS over the corresponding density interval, which means that NS
properties are affected by the whole EoS line shape, not some specific points.
Meanwhile, from NM properties and MR relations obtained in this work shown in
Fig. 4, it’s found that the statistical significance of NM properties in the
Bayesian analysis is reduced by including the constraints of MR relation
because the area obtained from the case with MR constraint deviates more from
the $e_{0}$ and $E_{\rm sym}$ constraints but meets the MR constraint better.
It again shows the need to take care of both NM and NS properties,
simultaneously, in nuclear force study. Besides, it’s found that MR lines only
lie at the right side of the constraint in Fig. 4, which provides evidence of
the NN’s ability to identify data-favored models.

Since parameter space is high dimensional, it’s hard to fully describe the
error bands of parameters. To have an idea of the distribution of a certain
parameter, we vary it in the parameter space while fixing the others at the
optimal values #4#4#4The number of events is set to be 10000.. The
distributions of the parameters are shown in Fig. 5 and it can be seen that
the probability density distributions of parameters are not regular normal
distributions, which indicates the correlations between parameters, especially
those multi-meson couplings.

Figure 5: Distributions of the parameters estimated by NN.

##  III Summary and Outlook

An NN platform is constructed for analysis on bands of strong interaction
parameters with respect to both NS properties and NM properties. It has three
main modules:

  1. 1.

The computation module is used to solve the coupled EoMs to obtain the
corresponding physical quantities, and the LOSS function used during the
training process is the EoM itself so that this kind of process can be applied
to other physical problems straightforwardly;

  2. 2.

The Bayesian module is used to calculate the confidence level of the parameter
space given by the computation module with reasonable physical error bars;

  3. 3.

The self-supervised module, the most important part, gives the platform the
ability to search parameter space automatically and adaptively with the
advantages of NN, and will try to find the optimal solutions of parameters.

As an example, we apply the constructed NN to a specific nucleon force model
to show that the platform is able to give a reasonable prediction of parameter
space. The numerical results suggest the necessity to analyze NS properties
with the NM properties simultaneously since the predicted parameter space with
only NM properties usually makes it hard to describe the MR relations. The
results also indicate the NN platform’s ability to identify data favored
theoretical parametrization with the help of the optimal solution’s confidence
level, $P_{c}$. The NN applied in this example is modularly designed so that
the computation module can be adjusted to solve other RMF EoS and the other
two modules can be applied directly to find the optimal parameter space for
corresponding physical data.

In our future works, other NS and NM properties, such as tidal deformations,
the symmetry energy density slope, and the skewness coefficient, will be added
to the Bayesian module to provide more rigorous constraints on parameters, and
the algorithms involved will be changed into the AI governed ones to improve
the efficiency and adaptivity. The correlation analysis on parameter space
will be carried out to distinguish the leading-order physical contribution
parameters from the ones causing noise. The existing algorithms built in this
work will be applied to other $\chi{\mathrm{EFTs}}$, such as the setup in Ref.
Li  _et al._ (2017), which are nonlinear realizations of QCD symmetry. The
constraints on these LECs of the $\chi{\mathrm{EFTs}}$ can be of great help to
understanding non-perturbative properties of QCD, such as quark condensate and
trace anomaly at low energy and dense regions.

In modern physics, multi-source data handling and error propagations are
becoming more and more important to help one understand EFTs and provide
rigorous constraints on possible new physics. The statistical analysis built
in this platform can be extended to other model verification processes,
especially those with massive data or complex mappings between physical
quantities and parameters. It’s hoped to provide a heuristic way for
theoretical physics research.

## Acknowledgments

The work of Y. L. M. is supported in part by the National Key R&D Program of
China under Grant No. 2021YFC2202900 and the National Science Foundation of
China (NSFC) under Grant No. 11875147 and No. 12147103.

## References

  * Krizhevsky  _et al._ (2012) A. Krizhevsky, I. Sutskever, and G. E. Hinton, Advances in neural information processing systems 25 (2012).
  * LeCun  _et al._ (2016) Y. LeCun, Y. Bengio, and G. Hinton, Nature 521, 436 (2016).
  * Jordan and Mitchell (2015) M. I. Jordan and T. M. Mitchell, Science 349, 255 (2015).
  * Li  _et al._ (2019a) Y. Li, C. Huang, L. Ding, Z. Li, Y. Pan, and X. Gao, Methods 166, 4 (2019a).
  * Mehta  _et al._ (2019) P. Mehta, M. Bukov, C.-H. Wang, A. G. R. Day, C. Richardson, C. K. Fisher, and D. J. Schwab, Phys. Rept. 810, 1 (2019), arXiv:1803.08823 [physics.comp-ph] .
  * Abraham  _et al._ (2018) S. Abraham, A. Aniyan, A. K. Kembhavi, N. Philip, and K. Vaghmare, Monthly Notices of the Royal Astronomical Society 477, 894 (2018).
  * Niu  _et al._ (2019) H. Niu, Z. Gong, E. Ozanich, P. Gerstoft, H. Wang, and Z. Li, The Journal of the Acoustical Society of America 146, 211 (2019).
  * Gu  _et al._ (2020) R. Gu, Z. Yang, and Y. Ji, Journal of Network and Computer Applications 157, 102576 (2020).
  * Brady  _et al._ (2021) J. Brady, P. Wen, and J. W. Holt, Phys. Rev. Lett. 127, 062701 (2021).
  * Boehnlein  _et al._ (2022) A. Boehnlein _et al._ , Rev. Mod. Phys. 94, 031003 (2022), arXiv:2112.02309 [nucl-th] .
  * Oala  _et al._ (2023) L. Oala, M. Aversa, G. Nobis, K. Willis, Y. Neuenschwander, M. Buck, C. Matek, J. Extermann, E. Pomarico, W. Samek, _et al._ , Transactions on Machine Learning Research (2023).
  * He  _et al._ (2023a) W. B. He, Y. G. Ma, L. G. Pang, H. C. Song, and K. Zhou, Nucl. Sci. Tech. 34, 88 (2023a).
  * He  _et al._ (2023b) W. He, Q. Li, Y. Ma, Z. Niu, J. Pei, and Y. Zhang, Sci. China Phys. Mech. Astron. 66, 282001 (2023b).
  * Brown and Rho (2002) G. E. Brown and M. Rho, Phys. Rept. 363, 85 (2002).
  * Hayano and Hatsuda (2010) R. S. Hayano and T. Hatsuda, Rev. Mod. Phys. 82, 2949 (2010).
  * Holt  _et al._ (2016) J. W. Holt, M. Rho, and W. Weise, Phys. Rept. 621, 2 (2016).
  * Baym  _et al._ (2018) G. Baym, T. Hatsuda, T. Kojo, P. D. Powell, Y. Song, and T. Takatsuka, Rept. Prog. Phys. 81, 056902 (2018).
  * Ma and Rho (2020) Y. L. Ma and M. Rho, Prog. Part. Nucl. Phys. 113, 103791 (2020).
  * Li  _et al._ (2019b) B.-A. Li, P. G. Krastev, D.-H. Wen, and N.-B. Zhang, Eur. Phys. J. A 55, 117 (2019b), arXiv:1905.13175 [nucl-th] .
  * Ma and Rho (2021) Y.-L. Ma and M. Rho, Symmetry 13, 1888 (2021), arXiv:2009.09219 [nucl-th] .
  * Burgio  _et al._ (2021) G. F. Burgio, H. J. Schulze, I. Vidana, and J. B. Wei, Prog. Part. Nucl. Phys. 120, 103879 (2021), arXiv:2105.03747 [nucl-th] .
  * Ma and Yang (2023) Y. L. Ma and W. C. Yang, Symmetry 15, 776 (2023).
  * Youngblood  _et al._ (1999) D. Youngblood, H. Clark, and Y. W. Lui, Physical Review Letters 82, 691 (1999).
  * Steiner and Li (2005) A. W. Steiner and B. A. Li, Physical Review C 72, 041601 (2005).
  * Karataglidis  _et al._ (2002) S. Karataglidis, K. Amos, B. A. Brown, and P. K. Deb, Physical Review C 65, 044306 (2002).
  * Brown (2000) B. A. Brown, Physical review letters 85, 5296 (2000).
  * Demorest  _et al._ (2010) P. Demorest, T. Pennucci, S. Ransom, M. Roberts, and J. Hessels, Nature 467, 1081 (2010).
  * Antoniadis  _et al._ (2013) J. Antoniadis _et al._ , Science 340, 6131 (2013).
  * Fonseca  _et al._ (2016) E. Fonseca _et al._ , Astrophys. J. 832, 167 (2016).
  * Özel and Freire (2016) F. Özel and P. Freire, Ann. Rev. Astron. Astrophys. 54, 401 (2016), arXiv:1603.02698 [astro-ph.HE] .
  * Cromartie  _et al._ (2019) H. T. Cromartie _et al._ (NANOGrav), Nature Astron. 4, 72 (2019).
  * Fonseca  _et al._ (2021) E. Fonseca _et al._ , Astrophys. J. Lett. 915, L12 (2021).
  * Abbott  _et al._ (2017) B. P. Abbott _et al._ (LIGO Scientific, Virgo), Phys. Rev. Lett. 119, 161101 (2017).
  * Abbott  _et al._ (2018) B. P. Abbott _et al._ (LIGO Scientific, Virgo), Phys. Rev. Lett. 121, 161101 (2018).
  * Annala  _et al._ (2018) E. Annala, T. Gorda, A. Kurkela, and A. Vuorinen, Phys. Rev. Lett. 120, 172703 (2018), arXiv:1711.02644 [astro-ph.HE] .
  * Annala  _et al._ (2022) E. Annala, T. Gorda, E. Katerini, A. Kurkela, J. Nättilä, V. Paschalidis, and A. Vuorinen, Phys. Rev. X 12, 011058 (2022), arXiv:2105.05132 [astro-ph.HE] .
  * Hartley (2017) W. Hartley, The Astrophysical Journal Letters 848, L12 (2017).
  * Collaboration  _et al._ (2018) I. Collaboration, MAGIC, AGILE, ASAS-SN, HAWC, HESS, INTEGRAL, Kanata, Kiso, Kapteyn, _et al._ , Science 361, eaat1378 (2018).
  * Machleidt  _et al._ (1987) R. Machleidt, K. Holinde, and C. Elster, Phys. Rept. 149, 1 (1987).
  * Epelbaum  _et al._ (2009) E. Epelbaum, H.-W. Hammer, and U.-G. Meissner, Rev. Mod. Phys. 81, 1773 (2009).
  * Ma and Rho (2019) Y. L. Ma and M. Rho,  _Effective Field Theories for Nuclei and Compact-Star Matter_ (WSP, 2019).
  * Vaswani  _et al._ (2017) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, in  _Proceedings of the 31st International Conference on Neural Information Processing Systems_ (2017).
  * Long  _et al._ (2015) J. Long, E. Shelhamer, and T. Darrell, in  _2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ (2015).
  * Drischler  _et al._ (2020) C. Drischler, R. Furnstahl, J. Melendez, and D. Phillips, Physical Review Letters 125, 202702 (2020).
  * Ferreira and Providência (2021) M. Ferreira and C. Providência, Journal of Cosmology and Astroparticle Physics 2021, 011 (2021).
  * Sugahara and Toki (1994) Y. Sugahara and H. Toki, Nucl. Phys. A 579, 557 (1994).
  * Shen  _et al._ (1998) H. Shen, H. Toki, K. Oyamatsu, and K. Sumiyoshi, Nucl. Phys. A 637, 435 (1998).
  * (48) A. Paszke _et al._ , arXiv:1912.01703 [cs.LG] .
  * Ioffe and Szegedy (2015) S. Ioffe and C. Szegedy, in  _Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37_ (2015).
  * Glorot  _et al._ (2011) X. Glorot, A. Bordes, and Y. Bengio, in  _International Conference on Artificial Intelligence and Statistics_ (2011).
  * Kingma and Ba (2014) D. P. Kingma and J. Ba, CoRR abs/1412.6980 (2014).
  * Tolman (1939) R. C. Tolman, Phys. Rev. 55, 364 (1939).
  * Oppenheimer and Volkoff (1939) J. R. Oppenheimer and G. M. Volkoff, Phys. Rev. 55, 374 (1939).
  * Abbott  _et al._ (2019) B. P. Abbott _et al._ (LIGO Scientific, Virgo), Phys. Rev. X 9, 011001 (2019).
  * Dutra  _et al._ (2012) M. Dutra, O. Lourenco, J. S. Sa Martins, A. Delfino, J. R. Stone, and P. D. Stevenson, Phys. Rev. C 85, 035201 (2012).
  * Dutra  _et al._ (2014) M. Dutra, O. Lourenço, S. S. Avancini, B. V. Carlson, A. Delfino, D. P. Menezes, C. Providência, S. Typel, and J. R. Stone, Phys. Rev. C 90, 055203 (2014).
  * Choi  _et al._ (2021) S. Choi, T. Miyatsu, M.-K. Cheoun, and K. Saito, Astrophys. J. 909, 156 (2021).
  * Colo  _et al._ (2004) G. Colo, N. Van Giai, J. Meyer, K. Bennaceur, and P. Bonche, Phys. Rev. C 70, 024307 (2004).
  * Li  _et al._ (2017) Y. L. Li, Y. L. Ma, and M. Rho, Phys. Rev. D 95, 114011 (2017).