# Understanding Out-of-distribution:A Perspective of Data Dynamics

Dyah Adila
Department of Computer Sciences
University of Wisconsin-Madison
<EMAIL_ADDRESS>
&Dongyeop Kang
Department of Computer Science and Engineering
University of Minnesota
<EMAIL_ADDRESS>
Corresponding author

###### Abstract

Despite machine learning models’ success in Natural Language Processing (NLP)
tasks, predictions from these models frequently fail on out-of-distribution
(OOD) samples. Prior works have focused on developing state-of-the-art methods
for detecting OOD. The fundamental question of how OOD samples differ from in-
distribution samples remains unanswered. This paper explores how data dynamics
in training models can be used to understand the fundamental differences
between OOD and in-distribution samples in extensive detail. We found that
syntactic characteristics of the data samples that the model consistently
predicts incorrectly in both OOD and in-distribution cases directly contradict
each other. In addition, we observed preliminary evidence supporting the
hypothesis that models are more likely to latch on trivial syntactic
heuristics (e.g., overlap of words between two sentences) when making
predictions on OOD samples. We hope our preliminary study accelerates the
data-centric analysis on various machine learning phenomena.

##  1 Introduction

Detecting out-of-distribution (OOD) has become one of the key bottlenecks in
building reliable open-world systems [3], leading to new SOTA approaches meant
to mitigate this problem [5, 16, 18, 29, 20, 23]. Previous works have explored
the discrepancy between OOD and in-distribution test performance from the
model and algorithm perspective. Le Lan & Dinh (2020) [17] observed the
limitation of density estimation for anomaly detection. Zhang et al. (2021)
[37] suggested that the cause of OOD failure is the combination of the model
architecture and maximum likelihood objective. Choi et al. (2018) [6], Just &
Ghosal (2019) [14], Fetaya et al. (2020) [9], Kirichenko et al. (2020) [15],
Zhang et al. (2020) [36], and Wang et al. (2020) [34] concluded that
significant mismatch between the real and estimated distribution gives rise to
the performance discrepancy. Nevertheless, what are the fundamental
differences between in-distribution and OOD data samples which cause models to
be especially brittle to the latter?

Our approach to studying this difference is inspired by emerging studies on
machine learning models adopting shallow heuristics (i.e., irrelevant
statistical patterns found in the majority of training examples), instead of
learning the underlying generalizations that they are intended to capture [22,
33, 1, 2]. For example, Beery et al. (2018) [2] demonstrated a network that
accurately recognizes cows in a typical context (e.g., pasture) consistently
misclassifies cows in a non-typical context (e.g., beach). Similar heuristics
also arise in visual question answering systems [1] and researchers proposed
graph generative modeling schemes [13] (inspired by graph convolutional
networks [30]) to handle the problem implicitly. In this paper, we study this
problem within the Natural Language Inference (NLI): the task of determining
whether a premise sentence entails (i.e., implies the truth of) a hypothesis
sentence [7, 8, 4]. McCoy et al. (2019) [22] exhaustively characterized
shallow heuristics that commonly appears in benchmark NLI datasets, which we
will refer to as syntactic heuristics.

To this end, we study the difference between in-distribution and OOD samples
which gives rise to models brittleness through two perspectives: (i) training
dynamics of the model, and (ii) syntactic heuristics of the data samples.
Specifically, we perform two types of data-dynamic analyses: We first examine
the difference between OOD and in-distribution samples distributions in the
data cartography space [28] at each training epoch. Cartography space allows
us to distinguish which samples are harder to learn (i.e., the model often
misclassifies them) based on training dynamics measures. Second, we mark each
data sample by their syntactic heuristic, which enables us to identify what
shallow characteristics tend to be harder to learn. For instance, we
hypothesize whether more word overlap between a premise and a hypothesis text
influences model’s ability to infer their label correctly.

Our analyses suggested that the syntactic heuristic that the model deems hard-
to-learn during training directly contradicts the characteristic of OOD
samples that are hard-to-learn during inference. We also found preliminary
evidence suggesting the model’s tendency to make inferences based on trivial
syntactic heuristic is higher in the OOD case. We hope that this study will
drive more effort to better understand the difference between in-distribution
and OOD samples and, consequently, develop more informed and data-centric OOD
detection and generalization methods.

##  2 Our Hypotheses

This section presents our two initial hypotheses to characterize OOD data
samples. Motivated by previous works [11, 12], we perform a comparative
analysis between OOD and in-distribution samples over dynamic information of
model training (e.g., confidence, prediction variance across epoch) per set of
train-test epoch. Moreover, to interpret the quantitative difference of these
measures, we mark data samples based on their syntactic heuristics. We
hypothesize that the difference between OOD and in-distribution samples can be
characterized in the combination of multiple training dynamics (§2.1) and
sample heuristics (§2.2).

###  2.1 H1: Are OOD and in-distribution samples different with their training
dynamics?

Data cartography [28] is a tool to group data samples into three regions:
easy-to-learn, hard-to-learn, and ambiguous (see Figure 1), enabled by
following two training dynamics as axes:

  1. 1.

Confidence: the mean model probability of the true label ($y_{i}^{*}$) across
epochs (equation 1). The term $p_{\theta}^{(e)}(y_{i}^{*}|x_{i})$ denotes the
model’s probability with parameters $\theta$ at the end of $e^{th}$ epoch.

  2. 2.

Variability: the spread of $p_{\theta}^{(e)}(y_{i}^{*}|x_{i})$ across epochs,
and is defined by equation 2.

$\hat{\mu}_{i}=\frac{1}{E}\sum_{e=1}^{E}p_{\theta}^{(e)}(y_{i}^{*}|x_{i})$ (1)

$\hat{\sigma_{i}}=\sqrt{\frac{\sum_{e=1}^{E}(p_{\theta}^{(e)}(y_{i}^{*}|x_{i})-\hat{\mu}_{i})^{2}}{E}}$
(2)

Intuitively, higher variability implies a high range of probability outputted
by the model for the same sample. As seen in Figure 1, the rightmost region
(i.e., high variability) is the ambiguous region. The easy-to-learn region is
characterized by high confidence and low variability (i.e., correct prediction
with high assigned probability across epochs), and hard-to-learn region
samples have low confidence and low variability (i.e., incorrect prediction
across epochs).

At each epoch $E$, we record the following four measurements for all train
samples $i\in N_{train}$ and test samples $j\in N_{test}$ ($N_{train}$ and
$N_{test}$ denote the size of train and test sets respectively). As $\theta$
is constant for all samples at each epoch, for conciseness sake, we abbreviate
$p_{\theta}^{(e)}(y_{i}^{*}|x_{i})$ as $p_{i}^{(e)}$.

$\hat{\mu}_{i}^{E}=\frac{1}{E}\sum_{e=1}^{E}p_{i}^{(e)}$ (3)

$\hat{\sigma}_{i}^{E}=\sqrt{\frac{\sum_{e=1}^{E}(p_{i}^{(e)}-\hat{\mu}_{i}^{E})^{2}}{E}}$
(4)

$\forall\quad i\in N_{train}$

$\hat{\mu}_{j}^{E}=\frac{1}{E}\sum_{e=1}^{E}p_{j}^{(e)}$ (5)

$\hat{\sigma_{j}}^{E}=\sqrt{\frac{\sum_{e=1}^{E}(p_{j}^{(e)}-\hat{\mu}_{j}^{E})^{2}}{E}}$
(6)

$\forall\quad j\in N_{test}$

In each epoch $E$, we then plot all train samples based on their respective
($\hat{\mu}_{i}^{E}$, $\hat{\sigma_{i}}^{E}$) resulting in train cartography
map $C_{train}^{E}$ (e.g., figure 1(a)). Similarly, we also plot all test
samples based on their ($\hat{\mu}_{j}^{E}$, $\hat{\sigma_{j}}^{E}$) resulting
in test cartography map $C_{test}^{E}$. (e.g., figure 1(c))

As models’ performance in inferring in-distribution samples far exceeds that
of the OOD case, we hypothesize that most OOD samples will stay in the hard-
to-learn regions in $C_{test}^{E}$ for all $E$ . In contrast, most in-
distribution samples will eventually move to the easy-to-learn region in
$C_{test}^{E}$ for larger $E$, which results in a noticeable difference
between the two distributions in the cartography space. We hope to interpret
this difference using syntactic heuristic (§2.2).

###  2.2 H2: Do models tend to adopt syntactic attributes of OOD samples more
readily?

Past works have shown that machine learning models often adopt ‘shortcut’ data
characteristics instead of the generalization that humans would learn and use
to perform the same task [22, 33, 1, 2]. More specifically, a model trained on
NLI might assign a label of contradiction to any input containing the word
“not” [24, 26], as this heuristic often applies in standard NLI training sets.
As OOD samples come from a different distribution, they will have less common
generalizable characteristics with the training set than the in-distribution
samples. Thus, we hypothesize that models will rely even more upon the
shortcut characteristics (i.e., heuristics) when making inferences on OOD
samples.

We focus on lexical overlap heuristic [22], as it is one of the simplest and
most common across NLI datasets. This heuristic assumes that a premise entails
all hypotheses constructed from words in the premise. Below are examples:
Premise Hypothesis Label Type The judge was paid by the actor. The actor paid
the judge. Entailed Support The actor was paid by the judge. The actor paid
the judge. Not entailed Contradict
A sample can either: (i) supporting, (ii) contradicting (i.e., possessing the
property of heuristic and labels entailed or not entailed respectively), or
(iii) having no heuristic. We mark each sample’s heuristics, as illustrated by
the different circle colors of Figure 1.

To quantify the degree of which each sample adopts lexical heuristic, we
calculate the percentage of words in the hypothesis (s2) which overlap with
premise (s1), which aligns with the lexical heuristic definition. More
formally, we measure $m2=\frac{\left|s1\bigcap s2\right|}{\left|s2\right|}$.

To measure the model’s tendency to adapt the heuristic, we calculate three
sets of correlations: (i) $\rho(m2,{\mu}_{i}^{E})$ (ii) $\rho(m2,{\mu}_{j(in-
dist)}^{E})$ (iii) $\rho(m2,{\mu}_{j(OOD)}^{E})$, and observe the trends
across epochs. We calculate these correlations for all samples and also for
samples from each class independently. Intuitively, a high heuristic adoption
is indicated by high absolute correlation values in the individual class
samples (e.g., the model learns that high words overlap means the label is
entailment, vice versa).

##  3 Experiments

(a) Training cartography map at $E=2$ ($C_{train}^{2}$)

(b) Training cartography map at $E=8$ ($C_{train}^{8}$)

(c) Evaluation cartography map at $E=2$ ($C_{test}^{2}$)

(d) Evaluation cartography map at $E=8$ ($C_{test}^{8}$)

Figure 1: Training and evaluation cartography maps. The different colors illustrate different sample heuristics and the different shapes in figures (c) and (d) illustrate OOD sample (’o’ for in-distribution and ’x’ for OOD). Table 1: Datasets combination in our experiments. Training | Eval (In-distribution) | Eval (OOD)
---|---|---
MNLI (train) | MNLI (dev matched) | WNLI (train)
MNLI (train) | MNLI (dev matched) | RTE (dev)
RTE (train) | RTE (dev) | WNLI (train)

Setup: In all experiments, we trained Roberta [21] with batch size 20 and
learning rate $1.1e^{-5}$, and initialized with the same random seed. All
experiments were carried out using PyTorch [25] with one NVidia Tesla K20X
GPU. Experiments were carried out using three datasets (MNLI [35], RTE [31],
WNLI [19]) which combinations are shown in table 1.

In our study, we define OOD samples as those with different textual genre and
sentence structure from the training. For instance, WNLI were derived by
taking sentences from fiction books. Each hypothesis-premise pair is a
sentence and it’s copy with the ambiguous pronouns replaced by each possible
referent. While RTE and MNLI’s hypothesis-premise pairs are different
sentences. RTE are texts from Wikipedia and MNLI were crowdsourced from
speech, fiction, and government report. Our extensive experiments results can
be found in Appendix A.3, Appendix A.1, and Appendix A.2. For conciseness
sake, we only present results with MNLI as train and in-distribution data, and
WNLI as OOD data.

###  3.1 OOD vs in-distribution on training dynamics and syntactic heuristic

Result for this section are the cartography maps $C_{train}^{2}$,
$C_{train}^{8}$, $C_{test}^{2}$, $C_{test}^{8}$ in Figure 1. For visualization
clarity, the points showed are a sampled fraction from the original set. We
observe the following:

  * •

Observation: A contradicting pattern between the trajectories of training and
OOD samples. By comparing figures (a) and (b) ($C_{train}^{2}$ and
$C_{train}^{8}$), we observed samples that contradict the heuristic (blue
circles) either stays in the hard-to-learn region or move towards the
ambiguous region. On contrary, the figures (c) and (d) ($C_{test}^{2}$ and
$C_{test}^{8}$) shows that OOD samples that support the heuristic (green ’x’s)
stay in the hard-to-learn region.
Conjecture: Samples heuristics (i.e., lexical overlap) that the model deems
hard/easy to learn during training completely flips in the OOD case.

  * •

Observation: At the end of epoch 8, we observe way less color mix at the
ambiguous region of $C_{test}^{8}$ (blue and green ’x’s at the ambiguous
region of map (d)). While in $C_{test}^{8}$ (map (b)), this mix is very
apparent in the easy-to learn region (blue and green circles).
Conjecture: a more generalizable knowledge is learned during training
(indicated by the samples mix). Although, this generalizable knowledge is
slower to be adopted by the model in inferring OOD samples.

###  3.2 OOD vs in-distribution on tendency to adopt syntactic heuristics

(a) Correlation in all samples

(b) Correlation in entailment samples

Figure 2: Hypothesis 2.2: $\rho(m2,{\mu}_{i}^{E})$, $\rho(m2,{\mu}_{j(in-
dist)}^{E})$, $\rho(m2,{\mu}_{j(OOD)}^{E})$ on all samples (a) and entailment
samples(b)

For the second hypothesis, we find the following observations (refer to figure
2):

  * •

Observation: Correlation values ($\rho(m2,{\mu}_{i}^{E})$,
$\rho(m2,{\mu}_{j(in-dist)}^{E})$, $\rho(m2,{\mu}_{j(OOD)}^{E})$) are
relatively low in the plot for all samples (a). This applies to all cases
(train, in-distribution, and OOD test). However, an obvious divergence is
observed in the $\rho(m2,{\mu}_{j(OOD)}^{E})$ plot for the entailment samples
(b).
Conjecture: This observation confirms our hypothesis in 2.2 that models are
more prone to adopt syntactic heuristics when making inference on OOD samples.

  * •

Observation: All samples’ trends adhere to the entailment trends in the train
and in-distribution test case (orange and blue lines in figures (a) and (b))
but contradict the OOD case (green lines in figures (a) and (b)). In the OOD
case, we can see that the negative and non-zero slopes are only found in all
samples trend, but the correlation keeps increasing (with a huge difference in
value) in the entailment samples trend.
Conjecture: This might indicate that knowledge used by the model to infer
class labels is consistent for all classes in train and in-distribution but
conflicts in the OOD case.

More supplementary plots of this analysis (more experiments on different
datasets and the trend plots for non-entailment samples) can be found in
appendix A.4.

##  4 Limitations and Future Directions

This data-centric approach to understanding the difference between OOD and in-
distribution samples offers an intuition of failure mode in OOD. However, the
results and approach presented in this paper have some limitations.

Although all NLI sets in our experiments came from different topic domains and
had distinct characteristics, a more careful selection is needed to ensure
maximum separation of in-distribution vs. OOD data distribution. Zhang et al.
(2021) [37] highlights the problem with experiments setup where OOD regions
can lie in support of the data distribution: high logit values assigned to
certain OOD samples due to model estimation error. By ensuring this criterion
is met, we can achieve a more accurate confidence score, which in turn will
improve our results in section 3.1.

More controlled experiment is needed for results in section 3.2 to ensure that
the correlation between confidence and m2 is not spurious (i.e., there exists
no unobserved confounding variable). One way to remedy this is by running the
same measurements on random subsets and see if the correlation pattern still
holds (i.e., randomized controlled experiments).

Future work may include remedies for the two mentioned limitations.
Furthermore, investigating the root cause of contradicting syntactic
heuristics of OOD and in-distribution samples that the model fails to predict
correctly (section 3.1) is also an exciting direction. It will also be
exciting to see the effect of correcting for distribution mismatches [27, 10,
32] to the observations presented in this paper. To further corroborate the
analysis and conjecture presented in this paper, the two possible directions
can be: (i) adding more syntactic heuristics for analysis (e.g., coreference,
negation), and (ii) setting up similar experiments on vision and other domains
datasets. Finally, we hope that the insights provided by this data-centric
view can motivate more informed development of OOD detection and
generalization methods.

## References

  * [1] Aishwarya Agrawal, Dhruv Batra, and Devi Parikh.  Analyzing the behavior of visual question answering models.  In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1955–1960, Austin, Texas, November 2016. Association for Computational Linguistics.
  * [2] Sara Beery, Grant Van Horn, and Pietro Perona.  Recognition in terra incognita.  In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision – ECCV 2018, pages 472–489, Cham, 2018\. Springer International Publishing.
  * [3] Abhijit Bendale and Terrance Boult.  Towards open world recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1893–1902, 2015.
  * [4] Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning.  A large annotated corpus for learning natural language inference.  In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal, September 2015\. Association for Computational Linguistics.
  * [5] Jiefeng Chen, Yixuan Li, Xi Wu, Yingyu Liang, and Somesh Jha.  Atom: Robustifying out-of-distribution detection using outlier mining.  In Proceedings of European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), 2021\.
  * [6] Hyun-Jae Choi, Eric Jang, and Alexander Amir Alemi.  Waic, but why? generative ensembles for robust anomaly detection.  arXiv: Machine Learning, 2018.
  * [7] Cleo Condoravdi, Dick Crouch, Valeria de Paiva, Reinhard Stolle, and Daniel G. Bobrow.  Entailment, intensionality and text understanding.  In Proceedings of the HLT-NAACL 2003 Workshop on Text Meaning, pages 38–45, 2003.
  * [8] Ido Dagan, Oren Glickman, and Bernardo Magnini.  The pascal recognising textual entailment challenge.  In Joaquin Quiñonero-Candela, Ido Dagan, Bernardo Magnini, and Florence d’Alché Buc, editors, Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, pages 177–190, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg.
  * [9] Ethan Fetaya, Jörn-Henrik Jacobsen, Will Grathwohl, and Richard Zemel.  Understanding the limitations of conditional generative models.  arXiv preprint arXiv:1906.01171, 2019.
  * [10] Alexander Fuchs, Christian Knoll, and Franz Pernkopf.  Distribution mismatch correction for improved robustness in deep neural networks.  arXiv preprint arXiv:2110.01955, 2021.
  * [11] Dan Hendrycks and Kevin Gimpel.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  Proceedings of International Conference on Learning Representations, 2017.
  * [12] Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song.  Pretrained transformers improve out-of-distribution robustness.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2744–2751, Online, July 2020. Association for Computational Linguistics.
  * [13] Jingjing Jiang, Ziyi Liu, Yifan Liu, Zhixiong Nan, and Nanning Zheng.  X-ggm: Graph generative modeling for out-of-distribution generalization in visual question answering.  In Proceedings of the 29th ACM International Conference on Multimedia, pages 199–208, 2021.
  * [14] John Just and Sambuddha Ghosal.  Deep generative models strike back! improving understanding and evaluation in light of unmet expectations for ood data.  arXiv preprint arXiv:1911.04699, 2019.
  * [15] Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson.  Why normalizing flows fail to detect out-of-distribution data, 2020.
  * [16] Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell.  Simple and scalable predictive uncertainty estimation using deep ensembles.  In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6405–6416, Red Hook, NY, USA, 2017. Curran Associates Inc.
  * [17] Charline Le Lan and Laurent Dinh.  Perfect density models cannot guarantee anomaly detection.  arXiv preprint arXiv:2012.03808, 2020.
  * [18] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  Advances in neural information processing systems, 31, 2018.
  * [19] Hector Levesque, Ernest Davis, and Leora Morgenstern.  The winograd schema challenge.  In Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning, 2012.
  * [20] Weitang Liu, Xiaoyun Wang, John D Owens, and Yixuan Li.  Energy-based out-of-distribution detection.  arXiv preprint arXiv:2010.03759, 2020.
  * [21] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  arXiv preprint arXiv:1907.11692, 2019.
  * [22] Tom McCoy, Ellie Pavlick, and Tal Linzen.  Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy, July 2019. Association for Computational Linguistics.
  * [23] Sina Mohseni, Mandar Pitale, JBS Yadawa, and Zhangyang Wang.  Self-supervised learning for generalizable out-of-distribution detection.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 5216–5223, 2020.
  * [24] Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig.  Stress test evaluation for natural language inference.  In Proceedings of the 27th International Conference on Computational Linguistics, pages 2340–2353, Santa Fe, New Mexico, USA, August 2018. Association for Computational Linguistics.
  * [25] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala.  Pytorch: An imperative style, high-performance deep learning library.  In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., 2019.
  * [26] Ivan Sanchez, Jeff Mitchell, and Sebastian Riedel.  Behavior analysis of NLI models: Uncovering the influence of three factors on robustness.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1975–1985, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
  * [27] Steffen Schneider, Evgenia Rusak, Luisa Eck, Oliver Bringmann, Wieland Brendel, and Matthias Bethge.  Improving robustness against common corruptions by covariate shift adaptation.  Advances in Neural Information Processing Systems, 33, 2020.
  * [28] Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, and Yejin Choi.  Dataset cartography: Mapping and diagnosing datasets with training dynamics.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9275–9293, Online, November 2020\. Association for Computational Linguistics.
  * [29] Kshitij Tayal, Rahul Ghosh, and Vipin Kumar.  Model-agnostic methods for text classification with inherent noise.  In Proceedings of the 28th International Conference on Computational Linguistics: Industry Track, pages 202–213, 2020.
  * [30] Kshitij Tayal, Nikhil Rao, Saurabh Agarwal, Xiaowei Jia, Karthik Subbian, and Vipin Kumar.  Regularized graph convolutional networks for short text classification.  In Proceedings of the 28th International Conference on Computational Linguistics: Industry Track, pages 236–242, 2020.
  * [31] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman.  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  2019\.  In the Proceedings of ICLR.
  * [32] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell.  Tent: Fully test-time adaptation by entropy minimization.  arXiv preprint arXiv:2006.10726, 2020.
  * [33] Jianyu Wang, Zhishuai Zhang, Cihang Xie, Yuyin Zhou, Vittal Premachandran, Jun Zhu, Lingxi Xie, and Alan Yuille.  Visual concepts and compositional voting.  arXiv preprint arXiv:1711.04451, 2017.
  * [34] Ziyu Wang, Bin Dai, David Wipf, and Jun Zhu.  Further analysis of outlier detection with deep generative models, 2020\.
  * [35] Adina Williams, Nikita Nangia, and Samuel Bowman.  A broad-coverage challenge corpus for sentence understanding through inference.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
  * [36] Hongjie Zhang, Ang Li, Jie Guo, and Yanwen Guo.  Hybrid models for open set recognition.  In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision – ECCV 2020, pages 102–117, Cham, 2020. Springer International Publishing.
  * [37] Lily Zhang, Mark Goldstein, and Rajesh Ranganath.  Understanding failures in out-of-distribution detection with deep generative models.  In International Conference on Machine Learning, pages 12427–12436. PMLR, 2021.

##  Appendix A Appendix

###  A.1 Supplementary plot: OOD vs in-distribution on training dynamics
information (Training and in-dis: RTE; OOD: WNLI)

(a) Training cartography map at epoch 2

(b) Training cartography map at epoch 8

(c) Evaluation cartography map at epoch 2

(d) Evaluation cartography map at epoch 8

Figure 3: Training cartography maps (training set: RTE). The number of
heuristics related samples in RTE is small.

###  A.2 Supplementary plot: OOD vs in-distribution on syntactic
characteristics (entailment)

(a) Correlation between m2 and all samples $\hat{\mu}_{i}$

(b) Correlation between m2 and entailment samples $\hat{\mu}_{i}$

Figure 4: Results for hypothesis 2.2. Training and in-distribution test
samples are RTE, and OOD samples are WNLI.

(a) Correlation between m2 and all samples $\hat{\mu}_{i}$

(b) Correlation between m2 and entailment samples $\hat{\mu}_{i}$

Figure 5: Results for hypothesis 2.2. Training and in-distribution test
samples are MNLI, and OOD samples are RTE.

Results presented are at the end of epoch 8 for MNLI training and the end of
epoch 50 for RTE training. This is based on the epoch in which the training
error has converged (around 0.02).

###  A.3 Supplementary plot: OOD vs in-distribution on training dynamics
information (Training and in-dis: MNLI; OOD: RTE)

(a) Training cartography map at epoch 2

(b) Training cartography map at epoch 8

(c) Evaluation cartography map at epoch 2

(d) Evaluation cartography map at epoch 8

Figure 6: Training and evaluation cartography maps (train: MNLI, evaluation:
RTE). The number of heuristics related samples in RTE is small.

###  A.4 Supplementary plot: OOD vs in-distribution on syntactic
characteristics (non-entailment)

This section shows plots for correlation between confidence scores
($\hat{\mu}_{i}$) of non entailment samples and m2

(a) Train & in-distribution: MNLI, OOD: WNLI

(b) Train & in-distribution: RTE, OOD: WNLI

(c) Train & in-distribution: MNLI, OOD: RTE

Figure 7: Supplementary results for 3.2. Correlation between $\hat{\mu}_{i}$
of non-entailment samples and m2

###  A.5 Supplementary material: Extra lexical overlap measure

We also added another measure to quantify tendency to adopt lexical overlap
heuristic. We calculated $m1=\frac{\left|s1\bigcap
s2\right|}{\left|s1\right|}$. Essentially, this measures how much percentage
of words found in the premise ($s1$) can also be found in the hypothesis
($s2$).

(a) Correlation between m2 and all samples $\hat{\mu}_{i}$

(b) Correlation between m2 and entailment samples $\hat{\mu}_{i}$

Figure 8: Results for hypothesis 2.2. Training and in-distribution test
samples are MNLI, and OOD samples are WNLI.

(a) Correlation between m2 and all samples $\hat{\mu}_{i}$

(b) Correlation between m2 and entailment samples $\hat{\mu}_{i}$

Figure 9: Results for hypothesis 2.2. Training and in-distribution test
samples are RTE, and OOD samples are WNLI.

(a) Correlation between m2 and all samples $\hat{\mu}_{i}$

(b) Correlation between m2 and entailment samples $\hat{\mu}_{i}$

Figure 10: Results for hypothesis 2.2. Training and in-distribution test
samples are MNLI, and OOD samples are RTE.


#  Axially symmetric particlelike solutions with the flux of a magnetic field
in the non-Abelian Proca-Higgs theory

Vladimir Dzhunushaliev<EMAIL_ADDRESS>Department of Theoretical
and Nuclear Physics, Al-Farabi Kazakh National University, Almaty 050040,
Kazakhstan  Institute of Nuclear Physics, Almaty 050032, Kazakhstan
Academician J. Jeenbaev Institute of Physics of the NAS of the Kyrgyz
Republic, 265 a, Chui Street, Bishkek 720071, Kyrgyzstan  Vladimir Folomeev
<EMAIL_ADDRESS>Institute of Nuclear Physics, Almaty 050032, Kazakhstan
Academician J. Jeenbaev Institute of Physics of the NAS of the Kyrgyz
Republic, 265 a, Chui Street, Bishkek 720071, Kyrgyzstan  International
Laboratory for Theoretical Cosmology, Tomsk State University of Control
Systems and Radioelectronics (TUSUR), Tomsk 634050, Russia

###### Abstract

Within the non-Abelian SU(2) Proca-Higgs theory, we study localised axially
symmetric solutions possessing a finite field energy. It is shown that in a
certain sense such solutions are analogues of the Nielsen-Olesen tube, since
they have a longitudinal magnetic field creating a flux of this field over the
central cross-section of the Proca tube. The main difference between the Proca
tube and the Nielsen-Olesen tube is that the Proca tube is described by a
topologically trivial solution and has finite size, since its energy density
decreases exponentially with distance. The dependence of the total field mass
of the Proca tube on the value of one of the parameters determining the
solution is examined in detail. The solutions are obtained both in the
presence and in the absence of external sources (charge and current
densities).

non-Abelian Proca theory, Higgs field, finite-size flux tube solutions

###### pacs:

11.90.+t

##  I Introduction

In recent years interest in systems involving various massive vector fields
has increased considerably. Such fields do emerge, in particular, when
considering Proca theories. The presence of a mass of vector fields results in
substantial differences in the behaviour of Proca fields compared with Yang-
Mills fields: the Proca fields decay exponentially fast at spatial infinity,
which is impossible in principle for the Yang-Mills fields.

At the present time, studies of Proca theories are performed in various
directions. In particular, within the past few years, there was prolific
activity in the field of physics of Proca stars [1, 2, 3, 4, 5] and black
holes modelled within generalized Proca theories [6]. The massive extension of
a SU(2) gauge theory is studied in Refs. [7, 8], the static and spherically
symmetric solutions in a class of the generalized Proca theory with the
nonminimal coupling to the Einstein tensor are found in Ref. [9], the
cosmological implications of generalized Proca theories are under
investigation in Ref. [10].

In Refs. [11, 12, 13], we have studied infinite cylindrically symmetric
solutions (tubes) containing a longitudinal electric field. The solutions have
been obtained within the non-Abelian Proca-Higgs theory. It is of great
interest to get similar solutions describing finite-size objects; this implies
that the field energy density should decrease away from the center of a
configuration sufficiently fast. In this connection, in the present study we
extend the results of Refs. [11, 12, 13] and examine axially symmetric
solutions in the non-Abelian Proca-Higgs theory with a longitudinal color
magnetic field directed along the symmetry axis and transverse electric fields
located in a plane perpendicular to the symmetry axis. As will be shown below,
the asymptotic behavior of the fields leads to the fact that the total energy
of such configurations is finite; this enables us to call such solutions as
particlelike solutions. At the same time, the field configurations obtained
have a non-Abelian longitudinal magnetic Proca field; this enables us to call
them tubes with a flux of the corresponding magnetic field over a plane
perpendicular to the tube axis and located at the center of the axis. The
latter means that such a tube can be regarded as an analogue of the Nielsen-
Olesen tube [14], since in both cases there is the flux of the magnetic field
over the transverse cross-section of the tube. On the other hand, there also
exist substantial differences: (i) the Nielsen-Olesen tube is a topologically
nontrivial configuration, whereas our field configuration is topologically
trivial; (ii) the Proca tube has finite size, since the fields decrease
exponentially with distance; and (iii) the Nielsen-Olesen tube contains an
Abelian magnetic field, whereas the Proca tube – a non-Abelian magnetic field.

It should be also noted that in our study we consider a system supported by
Proca fields together with a Higgs scalar field. The presence of the latter,
as will be shown below, leads to a very interesting result: it initiates the
process where the Proca fields are being pushed outside by the Higgs field.
This interesting result is an analogue of the Meissner phenomenon in
superconductivity and of the dual Meissner effect in QCD.

The field configurations under investigation can emerge as quasiparticles in
the hypothetical “quark-Proca-gluon-Higgs” plasma. In this connection we note
that, in QCD, it is assumed that in the quark-gluon plasma there exist
multiple bound states of quasiparticles (for details see Ref. [15] where the
existence of such quasiparticles has been suggested, as well as Ref. [16]
where studies in this direction have been continued and the role of multiple
(colored) bound states in the quark-gluon plasma phase at not too high
temperature is demonstrated).

Thus, we will study here particlelike solutions in the Proca-Higgs theory in
which there is a color longitudinal Proca field directed along the symmetry
axis; this enables us to call such solutions as Proca tubes with a flux of the
magnetic field. The paper is organized as follows. In Sec. II, we write down
the general field equations for the non-Abelian-Proca-Higgs theory. In Sec.
III, we obtain cylindrically symmetric solutions to the equations of Sec. II
describing infinite tubes with the flux of the longitudinal color magnetic
field. In Sec. IV, we find axially symmetric solutions to the equations of
Sec. II (both with and without charge and current densities) describing
finite-size tubes with the flux of the longitudinal chromomagnetic field.
Finally, in Sec. V, we summarize and discuss the results obtained in the
present paper.

##  II Non-Abelian SU(3) Proca-Higgs theory

The Lagrangian describing a system consisting of a non-Abelian SU(3) Proca
field $A^{a}_{\mu}$ interacting with nonlinear scalar field $\phi$ can be
taken in the form (hereafter, we work in units such that $c=\hbar=1$)

$\mathcal{L}=-\frac{1}{4}F^{a}_{\mu\nu}F^{a\mu\nu}-\frac{\left(\mu^{2}\right)^{ab,\mu}_{\phantom{ab,}\nu}}{2}A^{a}_{\mu}A^{b\nu}+\frac{1}{2}\partial_{\mu}\phi\partial^{\mu}\phi+\frac{\lambda}{2}\phi^{2}A^{a}_{\mu}A^{a\mu}-\frac{\Lambda}{4}\left(\phi^{2}-M^{2}\right)^{2}.$
(1)

Here
$F^{a}_{\mu\nu}=\partial_{\mu}A^{a}_{\nu}-\partial_{\nu}A^{a}_{\mu}+gf_{abc}A^{b}_{\mu}A^{c}_{\nu}$
is the field strength tensor for the Proca field, where $f_{abc}$ are the
SU(3) structure constants, $g$ is the coupling constant, $a,b,c=1,2,\dots,8$
are color indices, $\mu,\nu=0,1,2,3$ are spacetime indices. The Lagrangian (1)
also contains the arbitrary constants $M,\lambda,\Lambda$ and the Proca field
mass matrix $\left(\mu^{2}\right)^{ab,\mu}_{\phantom{ab,}\nu}$.

Using (1), the corresponding field equations can be written in the form

$\displaystyle D_{\nu}F^{a\mu\nu}-\lambda\phi^{2}A^{a\mu}=$
$\displaystyle-\left(\mu^{2}\right)^{ab,\mu}_{\phantom{ab,}\nu}A^{b\nu}+j^{a\mu},$
(2) $\displaystyle\Box\phi=$ $\displaystyle\lambda
A^{a}_{\mu}A^{a\mu}\phi+\Lambda\phi\left(M^{2}-\phi^{2}\right),$ (3)

where, for the sake of generality, we have also added the current four-vector
$j^{a\mu}$. For such systems, the field energy density is

$\begin{split}\varepsilon=&\frac{1}{2}\left(E^{a}_{i}\right)^{2}+\frac{1}{2}\left(H^{a}_{i}\right)^{2}-\left[\left(\mu^{2}\right)^{ab,\alpha}_{\phantom{ab,}0}A^{a}_{\alpha}A^{b}_{0}-\frac{1}{2}\left(\mu^{2}\right)^{ab,\alpha}_{\phantom{ab,}\beta}A^{a}_{\alpha}A^{b\beta}\right]+\frac{1}{2}\left(\partial_{t}\phi\right)^{2}+\frac{1}{2}\left(\nabla\phi\right)^{2}\\\
&+\lambda\phi^{2}\left[\left(A^{a}_{0}\right)^{2}-\frac{1}{2}A^{a}_{\alpha}A^{a\alpha}\right]+\frac{\Lambda}{4}\left(\phi^{2}-M^{2}\right)^{2},\end{split}$
(4)

where $i=1,2,3$ and $E^{a}_{i}$ and $H^{a}_{i}$ are the components of the
electric and magnetic field strengths, respectively.

In the present paper we will consider solutions belonging to the subgroup
$\text{SU(2)}\subset\text{SU(3)}$ spanned on the Gell-Mann matrices
$\lambda^{2,5,7}$.

##  III Infinite flux tube solutions

To begin with, it will be helpful to study a simpler system with an infinite
flux tube containing a flux of a longitudinal electric field [11, 12, 13]. To
describe such a tube, let us choose the ansatz for the field potentials in the
form

$A^{2}_{t}=\frac{f(\rho)}{g},\;A^{7}_{\varphi}=\frac{\rho
w(\rho)}{g},\;\phi=\phi(\rho)$ (5)

written in cylindrical coordinates $\\{t,\rho,z,\varphi\\}$. Such a tube
contains the following color electric and magnetic fields (physical
components):

$E^{2}_{\rho}=-\frac{f^{\prime}}{g},\quad E^{5}_{\varphi}=-\frac{fw}{2g},\quad
H^{7}_{z}=-\frac{\rho w^{\prime}+w}{g\rho}.$ (6)

(Henceforth in this section the prime denotes differentiation with respect to
$\rho$.) In this case the energy flux is absent, since all components of the
Poynting vector are zero,

$S^{i}=\frac{\epsilon^{ijk}}{\sqrt{\gamma}}E^{a}_{j}H^{a}_{k}=0,$ (7)

where $\epsilon^{ijk}$ is the completely antisymmetric Levi-Civita symbol and
$\gamma$ is the determinant of the space metric.

Substituting the potentials (5) in Eqs. (2) and (3), we get the following set
of equations (without the current):

$\displaystyle f^{\prime\prime}+\frac{f^{\prime}}{\rho}$ $\displaystyle=$
$\displaystyle f\left(\frac{w^{2}}{4}+\lambda\phi^{2}-\mu_{1}^{2}\right),$ (8)
$\displaystyle w^{\prime\prime}+\frac{w^{\prime}}{x}-\frac{w}{\rho^{2}}$
$\displaystyle=$ $\displaystyle
w\left(-\frac{f^{2}}{4}+\lambda\phi^{2}-\mu_{2}^{2}\right),$ (9)
$\displaystyle\phi^{\prime\prime}+\frac{\phi^{\prime}}{\rho}$ $\displaystyle=$
$\displaystyle\phi\left[\frac{\lambda}{g^{2}}\left(-f^{2}+w^{2}\right)+\Lambda\left(\phi^{2}-M^{2}\right)\right]$
(10)

with the following components of the Proca field mass matrix:
$\mu_{1}^{2}=\left(\mu^{2}\right)^{22,t}_{\phantom{ab,}t}$ and
$\mu_{2}^{2}=\left(\mu^{2}\right)^{77,\varphi}_{\phantom{ab,}\varphi}$. We
seek a solution to Eqs. (8)-(10) in the vicinity of the origin of coordinates
in the form

$\displaystyle f(\rho)$ $\displaystyle=$ $\displaystyle
f_{0}+f_{2}\frac{\rho^{2}}{2}+\dots\quad\text{with}\quad
f_{2}=\frac{f_{0}}{2}\left[\lambda\phi_{0}^{2}-\mu_{1}^{2}\right],$
$\displaystyle w(\rho)$ $\displaystyle=$ $\displaystyle w_{1}\rho+\dots,$
$\displaystyle\phi(\rho)$ $\displaystyle=$
$\displaystyle\phi_{0}+\phi_{2}\frac{\rho^{2}}{2}+\dots\quad\text{with}\quad\phi_{2}=\frac{\phi_{0}}{2g^{2}}\left[-\lambda
f_{0}^{2}+g^{2}\Lambda\left(\phi_{0}^{2}-M^{2}\right)\right],$

where the expansion coefficients $f_{0},\phi_{0}$, and $w_{1}$ are arbitrary.

The derivation of solutions to the set of equations (8)-(10) is an eigenvalue
problem for the parameters $\mu_{1},\mu_{2}$, and $M$. The numerical solution
describing the behavior of the Proca field potentials and of the corresponding
electric and magnetic fields is given in Figs. 4-4. In particular, Fig. 4
shows the energy density obtained from Eq. (4) using (5) and (6) in the form

$\varepsilon=\frac{1}{g^{2}}\left[\frac{f^{\prime
2}}{2}+\frac{ww^{\prime}}{\rho}+\frac{w^{\prime
2}}{2}+\frac{g^{2}}{2}\phi^{\prime
2}+\frac{w^{2}}{2\rho^{2}}+\frac{f^{2}w^{2}}{8}-\frac{\mu_{1}^{2}f^{2}}{2}-\frac{\mu_{2}^{2}w^{2}}{2}+\frac{\lambda}{2}\left(f^{2}+w^{2}\right)\phi^{2}+\frac{g^{2}\Lambda}{4}\left(\phi^{2}-M^{2}\right)^{2}\right].$
(11)

Figure 1: The graphs of the Proca field potentials $f$, $w$ and of the scalar
field $\phi$ for the following values of the system parameters:
$\lambda=10,\Lambda=1,g=1,f_{0}=0.2,\phi_{0}=1,w_{1}=1,M=1.44227,\mu_{1}=3.359585,\mu_{2}=3.7633.$

Figure 2: The profile of the flux tube energy density from Eq. (11) for the
solutions given in Fig. 4.

Figure 3: The profiles of the color electric fields $E^{2}_{\rho}$ and
$E^{5}_{\varphi}$ from Eq. (6) for the solutions given in Fig. 4.

Figure 4: The profile of the color magnetic field $H^{7}_{z}$ from Eq. (6) for
the solutions given in Fig. 4.

The asymptotic behavior of the functions $f,w$, and $\phi$, which follows from
Eqs. (8)-(10), is

$f(\rho)\approx f_{\infty}\frac{e^{-\rho\sqrt{\lambda
M^{2}-\mu_{1}^{2}}}}{\sqrt{\rho}},\quad w(\rho)\approx
w_{\infty}\frac{e^{-\rho\sqrt{\lambda
M^{2}-\mu_{2}^{2}}}}{\sqrt{\rho}},\quad\phi\approx
M-\phi_{\infty}\frac{e^{-\rho\sqrt{2\Lambda M^{2}}}}{\sqrt{\rho}},$ (12)

where $f_{\infty},w_{\infty}$, and $\phi_{\infty}$ are integration constants.

It is seen from Figs. 4-4 that the cylindrically symmetric solutions found can
be used to describe infinitely long tubes within the non-Abelian Proca-Higgs
theory under consideration. Such tubes possess the finite linear energy
density (see Fig. 4),

$\mathcal{E}=2\pi\int\limits_{0}^{\infty}\rho\,\varepsilon(\rho)d\rho<\infty,$

and the finite flux of the longitudinal color magnetic field $H^{7}_{z}$ (see
Fig. 4),

$\Phi_{z}^{H}=2\pi\int\limits_{0}^{\infty}\rho H^{7}_{z}d\rho<\infty.$

Note that, unlike the systems considered by us earlier in Refs. [11, 12, 13],
these tubes contain neither the flux of the longitudinal color electric field
$E^{a}_{z}$ nor the energy flux/momentum density (since the Poynting vector is
zero).

##  IV Finite flux tube solutions

Let us now extend the solutions of the previous section to the case of finite-
size configurations. To obtain static axially symmetric solutions, we employ
the ansatz (5), with an obvious generalization to the case where the
components of the four-potential depend also on the coordinate $z$. As will be
demonstrated below, in this case, it is possible to get solutions describing
configurations possessing finite sizes both along the $\rho$-axis and along
the $z$-axis.

For convenience of performing numerical computations, we employ spherical
coordinates $\\{t,r,\theta,\varphi\\}$ in which the ansatz (5) can be recast
in the form

$A^{2}_{t}=\frac{f(r,\theta)}{g},\;A^{7}_{\varphi}=r\sin{\theta}\frac{w(r,\theta)}{g},\;\phi=\phi(r,\theta),$
(13)

where all functions now depend on $r$ and $\theta$. For such ansatz, there are
the following nonzero physical components of color electric and magnetic
fields:

$\displaystyle E^{2}_{r}=$ $\displaystyle-\frac{f_{,r}}{g},\quad
E^{2}_{\theta}=-\frac{f_{,\theta}}{gr},\quad E^{5}_{\varphi}=-\frac{fw}{2g},$
(14) $\displaystyle H^{7}_{r}=$
$\displaystyle-\frac{w_{,\theta}+w\cot\theta}{gr},\quad
H^{7}_{\theta}=\frac{w+rw_{,r}}{gr}.$ (15)

(Henceforth a comma in lower indices denotes differentiation with respect to
the corresponding coordinate.) In turn, the field equations (2) and (3) yield

$\displaystyle
f_{,rr}+\frac{2}{r}f_{,r}+\frac{1}{r^{2}}\left(f_{,\theta\theta}+\cot\theta
f_{,\theta}\right)-\frac{1}{4}f\left(w^{2}+4\lambda\phi^{2}\right)+\mu_{1}^{2}f=$
$\displaystyle j^{2t},$ (16) $\displaystyle
w_{,rr}+\frac{2}{r}w_{,r}+\frac{1}{r^{2}}\left(w_{,\theta\theta}+\cot\theta
w_{,\theta}\right)-\frac{w}{r^{2}\sin^{2}\theta}+\frac{1}{4}\left(f^{2}-4\lambda\phi^{2}\right)w+\mu_{2}^{2}w=$
$\displaystyle-r\sin{\theta}j^{7\varphi},$ (17)
$\displaystyle\phi_{,rr}+\frac{2}{r}\phi_{,r}+\frac{1}{r^{2}}\left(\phi_{,\theta\theta}+\cot\theta\phi_{,\theta}\right)+\frac{\lambda}{g^{2}}\left(f^{2}-w^{2}\right)\phi-\Lambda\phi\left(\phi^{2}-M^{2}\right)=$
$\displaystyle 0.$ (18)

The results of numerical simulations for these equations are given in Sec.
IV.1 (in the absence of the currents) and in Sec. IV.2 (in the presence of the
currents).

###  IV.1 The case with no charges

Consider first the case where the currents $j^{a\mu}=0$ in Eqs. (16) and (17).
To solve the set of equations (16)-(18), it is necessary to impose appropriate
boundary conditions for the Proca and scalar fields at the origin ($r=0$), at
infinity ($r\to\infty$), on the positive $z$ axis ($\theta=0$), and, using the
reflection symmetry with respect to $\theta\to\pi-\theta$, in the $\\{x,y\\}$
plane ($\theta=\pi/2$). So we require

$\displaystyle\left.\frac{\partial f}{\partial
r}\right|_{r=0}=\left.\frac{\partial\phi}{\partial
r}\right|_{r=0}=0,\left.w\right|_{r=0}=0;\quad\left.f\right|_{r=\infty}=0,\left.w\right|_{r=\infty}=0,\left.\phi\right|_{r=\infty}=M;$
$\displaystyle\left.\frac{\partial
f}{\partial\theta}\right|_{\theta=0}=\left.\frac{\partial\phi}{\partial\theta}\right|_{\theta=0}=0,\left.w\right|_{\theta=0}=0;\quad\left.\frac{\partial
f}{\partial\theta}\right|_{\theta=\pi/2}=\left.\frac{\partial
w}{\partial\theta}\right|_{\theta=\pi/2}=\left.\frac{\partial\phi}{\partial\theta}\right|_{\theta=\pi/2}=0.$

The set of three coupled nonlinear elliptic partial differential equations
(16)-(18) has been solved numerically subject to the above boundary
conditions. Calculations are carried out using the package FIDISOL [17] with
typical errors less than $10^{-4}$. As in the case of an infinite tube of Sec.
III, the input parameters are eigenparameters $\mu_{1},\mu_{2}$, and $M$,
whose magnitudes determine the solution completely. For different values of
the aforementioned parameters, one can obtain qualitatively different field
distributions, some typical examples of which are depicted in Fig. 5.

Figure 5: Distributions of the fields $\phi$ (upper row), $f$ (second row),
$w$ (third row), and the total energy density of the system $\varepsilon$ from
Eq. (19) (lower row) for different values of $\mu_{2}$ and fixed values of the
parameters $M=1.4$, $\Lambda=1$, $\lambda=10$, $g=1$, and $\mu_{1}=3.3$.

In particular, Fig. 5 shows the total energy density of the system under
consideration obtained from (4) using (13)-(15) in the form

$\begin{split}\varepsilon=&\frac{1}{g^{2}}\left[\frac{f_{,\theta}^{2}}{2r^{2}}+\frac{f_{,r}^{2}}{2}+\frac{w_{,r}^{2}}{2}+\frac{w_{,\theta}^{2}}{2r^{2}}+\cot\theta\frac{ww_{,\theta}}{r^{2}}+\frac{ww_{,r}}{r}+\frac{g^{2}}{2}\left(\frac{\phi_{,\theta}^{2}}{r^{2}}+\phi_{,r}^{2}\right)+\frac{f^{2}w^{2}}{8}-\frac{\mu_{1}^{2}}{2}f^{2}-\frac{\mu_{2}^{2}}{2}w^{2}\right.\\\
&\left.+\frac{\lambda}{2}\phi^{2}\left(f^{2}+w^{2}\right)+\csc^{2}\theta\frac{w^{2}}{2r^{2}}+\frac{g^{2}\Lambda}{4}\left(\phi^{2}-M^{2}\right)^{2}\right].\end{split}$
(19)

It is seen from the graphs of Fig. 5 that the energy density is negative in
some regions. To see whether this will give a negative total energy of the
system or not, we calculate the total mass (energy) of the configurations
under consideration,

$M_{\text{tot}}=2\pi\int\varepsilon r^{2}\sin{\theta}drd\theta.$ (20)

The results of calculations are given in Fig. 6 for the range
$1.02\leq\mu_{2}\leq 4.2$ for fixed values of other system parameters. For
$\mu_{2}=\mu_{2(\text{crit})}\approx 3.73$, the function $f\to 0$, and with
further increase of $\mu_{2}$, it always remains equal to zero.
Correspondingly, the configurations with $\mu_{2}\lesssim\mu_{2(\text{crit})}$
possess both the magnetic and electric fields, but for
$\mu_{2}\gtrsim\mu_{2(\text{crit})}$ only the magnetic field is present.
Taking into account that the asymptotic expansions of the solutions at large
$r$ give the restriction on the mass $\mu_{2}<\sqrt{\lambda M^{2}}$ (see
below), the numerical calculations indicate that when $\mu_{2}$ goes to this
upper limit the functions $w\to 0$ (the magnetic field is switched off) and
$\phi\to M$ over all space. Correspondingly, the energy density and the total
mass of the system vanish, as is seen from Fig. 6.

Figure 6: The dependence of the total mass (20) of the configurations under
consideration on $\mu_{2}$ for fixed values of the parameters $M=1.4$,
$\Lambda=1$, $\lambda=10$, $g=1$, and $\mu_{1}=3.3$. The vertical dashed line
separates the systems containing the electric and magnetic fields (located to
the left of the line) from the configurations possessing only the magnetic
field (located to the right of the line).

Using the solutions obtained, in Figs. 7-9, we have plotted the distributions
of the color electric and magnetic fields for different values of $\mu_{2}$.
The analysis of these distributions indicates that

  * •

According to Fig. 7, there is a point lying on a circle in the $z=0$ plane
where the lines of force of the electric field $\vec{E}^{2}$ penetrate into.
Aside from this, for sufficiently small values of $\mu_{2}$, there is a
surface in the form of a torus where the lines of force of the electric field
penetrate into as well.

  * •

As is seen from Fig. 9, the magnetic field $\vec{H}^{7}$ possesses a vortex
centred at a circle in the $z=0$ plane. In this case, as $\rho\rightarrow 0$,
the component $H^{7}_{\rho}\rightarrow 0$, whereas the component $H^{7}_{z}$
remains nonzero; as a result, there is a flux of this field perpendicular to
the transverse cross-section of the tube.

  * •

The configurations obtained for $\mu_{2}\gtrsim\mu_{2(\text{crit})}$ resemble
somewhat the Nielsen-Olesen tube: both types of systems contain only a
magnetic field with the flux directed along the $z$-axis.

  * •

For all the configurations obtained, the electric field $\vec{E}^{5}$ is a
vortex field (see Fig. 8).

Figure 7: The color electric field $\vec{E}^{2}$ strength distributions for
different values of the Proca mass $\mu_{2}$ and a fixed value $\mu_{1}=3.3$.
The parameters $M=1.4$, $\Lambda=1$, $\lambda=10$, $g=1$.  Figure 8: The color
electric field $\vec{E}^{5}$ strength distributions for different values of
the Proca mass $\mu_{2}$ and a fixed value $\mu_{1}=3.3$. The parameters
$M=1.4$, $\Lambda=1$, $\lambda=10$, $g=1$ (the graphs are plotted in the
$\\{x,y\\}$ plane, i.e., when $\theta=\pi/2$).  Figure 9: The color magnetic
field $\vec{H}^{7}$ strength distributions for different values of the Proca
mass $\mu_{2}$ and a fixed value $\mu_{1}=3.3$. The parameters $M=1.4$,
$\Lambda=1$, $\lambda=10$, $g=1$.

Thus, as in the case with an infinite tube of Sec. III, the finite tube under
consideration does not contain the flux of the longitudinal chromoelectric
field $E^{2}_{z}$,

$\Phi_{z}^{E}=2\pi\int\limits_{0}^{\infty}\rho E^{2}_{z}(z=0,\rho)d\rho=0,$

but possesses the flux of the longitudinal chromomagnetic field $H^{7}_{z}$,

$\Phi_{z}^{H}=2\pi\int\limits_{0}^{\infty}\rho H^{7}_{z}(z=0,\rho)d\rho\neq
0.$

Also, the finite tube contains no energy flux density – the Poynting vector
(7) is zero.

Consider now the asymptotic behavior of the solutions found. As
$r\rightarrow\infty$, the fields $f,w\rightarrow 0$ exponentially fast; hence,
one can neglect the nonlinear terms in Eqs. (16)-(18) containing $f^{2}$ and
$w^{2}$. In turn, asymptotically, the field $\phi\approx M-\eta\rightarrow M$,
and the function $\eta$ decays exponentially; this permits us to replace the
term $\phi^{2}$ by $M^{2}$ in Eqs. (16) and (17). As a result, from Eqs.
(16)-(18), one can obtain the following asymptotic equations:

$\displaystyle\bigtriangleup_{r,\theta}f+\left(\mu_{1}^{2}-\lambda
M^{2}\right)f=$ $\displaystyle 0,$ (21)
$\displaystyle\bigtriangleup_{r,\theta}w-\frac{w}{r^{2}\sin^{2}{\theta}}+\left(\mu_{2}^{2}-\lambda
M^{2}\right)w=$ $\displaystyle 0,$ (22)
$\displaystyle\bigtriangleup_{r,\theta}\eta-2\Lambda M^{2}\eta=$
$\displaystyle 0,$ (23)

where $\bigtriangleup_{r,\theta}$ is the Laplacian operator. Eqs. (21) and
(23) have obvious solutions in the form

$\displaystyle f\approx$ $\displaystyle
C_{f}\left(Y\right)^{0}_{l_{f}}\frac{e^{-r\sqrt{\lambda
M^{2}-\mu_{1}^{2}}}}{r},$ (24) $\displaystyle\eta\approx$ $\displaystyle
C_{\eta}\left(Y\right)^{0}_{l_{\eta}}\frac{e^{-r\sqrt{2\lambda M^{2}}}}{r},$
(25)

where $\left(Y\right)^{0}_{l_{f,\eta}}$ are spherical functions and
$C_{f,\eta}$ are constants. In turn, Eq. (22) has a solution similar to (24),
but only with the angular part expressed in terms of special functions (we do
not show this expression here to avoid overburdening the text). It follows
from the above expressions that there are upper limits for the masses
$\mu_{1}$ and $\mu_{2}$ ensuring the exponential asymptotic decay of the
solutions: $\mu_{1}^{2},\mu_{2}^{2}<\lambda M^{2}$.

###  IV.2 The case with nonzero charges

In this subsection we consider the case where the right-hand sides of the
Proca equations (16) and (17) contain nonzero current densities. When
considering a self-consistent problem with a tube connecting quarks, such
currents must be created by spinor fields describing quarks. For the sake of
simplicity, here we consider a toy model where the currents are given by hand.
This means that the location of the quarks and the magnitude of color currents
and charges created by them are fixed.

In the simplest case the currents can be given, for instance, by the Gaussian
distribution,

$j^{a\mu}=\left(j_{0}\right)^{a\mu}e^{-\frac{R^{2}}{R^{2}_{0}}},$ (26)

where $\left(j_{0}\right)^{a\mu}$ is an arbitrary constant and $R$ is some
function of $z$ and $\rho$.

####  IV.2.1 The case of nonzero charge density $j^{2t}$

Consider first the case where there is only the charge density $j^{2t}$ in Eq.
(16). For this case, we choose $R^{2}=\left(z-l\right)^{2}+\rho^{2}$. Such a
choice implies that a “quark” is located on the tube axis at a distance $l$
from the origin of coordinates. Choosing different values of the constant
$(j_{0})^{2t}$, in Fig. 10, we have plotted the corresponding distributions of
the color electric field $\vec{E}^{2}$. As is seen from the figure, as
$(j_{0})^{2t}$ increases (in modulus), the behavior of the lines of force
changes qualitatively: if for small $(j_{0})^{2t}$ the electric field created
by the Proca field by itself is still comparable to the magnitude of the field
determined by the charge, it ceases to be so with increase of $(j_{0})^{2t}$
when the field determined by the charge becomes dominating; this results in
the aforementioned qualitative changes of the structure of the electric field.
This is illustrated by the appearance of a “fixed” point in the $z=0$ plane
and at some distance from the tube axis. The behavior of the lines of force is
similar here to the behavior of phase trajectories for autonomous differential
equations.

Figure 10: The color electric field $\vec{E}^{2}$ strength distributions for
different values of the parameter $(j_{0})^{2t}$ and fixed values
$\mu_{1}=3.3$ and $\mu_{2}=3.7$. The parameters $M=1.4$, $\Lambda=1$,
$\lambda=10$, $g=1$, $R_{0}=0.2$, and $l=1$.

Distributions of the electric, $\vec{E}^{5}$, and magnetic, $\vec{H}^{7}$,
fields change qualitatively only slightly compared with the case without
charge. Hence, to avoid overburdening the text, we do not show them here.

####  IV.2.2 The case of nonzero current density $j^{7\varphi}$

Figure 11: The electric and magnetic fields strength distributions for
different values of the parameter $(j_{0})^{7\varphi}$ and fixed values
$\mu_{1}=3.3$ and $\mu_{2}=3.7$. The parameters $M=1.4$, $\Lambda=1$,
$\lambda=10$, $g=1$, $R_{0}=0.2$, and $l=1$. For $(j_{0})^{7\varphi}=-15$ and
$10$, the electric field is absent.

Consider now the case where there is only the current density $j^{7\varphi}$
in Eq. (17). For this case, we take $R^{2}=\left(\rho-l\right)^{2}+z^{2}$ in
Eq. (26); this implies the presence of the current of quarks in the $z=0$
plane located on a circle of radius $l$. As in the case of the presence of the
charge density considered above, we will change the value of the arbitrary
constant $(j_{0})^{7\varphi}$ and keep track of the behavior of the electric
and magnetic fields. As a result, it turns out that both fields may exist
simultaneously only in a very restricted range of values of
$(j_{0})^{7\varphi}$. For example, for the same fixed values of the system
parameters as those used in the case with a charge, there is the following
range of possible values $-11\lesssim(j_{0})^{7\varphi}\lesssim 2$ for which
the system possesses both the electric and magnetic fields. When one goes
beyond this range, only the magnetic field remains. Thus there are some
critical values of $(j_{0})^{7\varphi}$ for which the electric field is
switched off. One might expect that for another values of the system
parameters, distinct from those for which the aforementioned range of
$(j_{0})^{7\varphi}$ has been found, there will take place a similar situation
where there will be some critical values of $(j_{0})^{7\varphi}$ determining
the range in which configurations both with electric and magnetic fields do
exist.

The corresponding field strength distributions are given in Fig. 11. It is
seen from them that the qualitative picture of the electric field strength
distribution changes only slightly (the same is true of the strength of the
field $\vec{E}^{5}$ that is still a vortex field). In turn, the distribution
of the magnetic field may depend considerably on the value of the constant
$(j_{0})^{7\varphi}$; this especially manifests itself for large negative
values of $(j_{0})^{7\varphi}$ when the influence of the external source of
the field becomes already comparable with the contribution caused by the
nonlinearity of the Proca fields.

##  V Discussion and conclusions

In the present paper, we have studied particlelike solutions with the flux of
a magnetic field in the non-Abelian Proca-Higgs theory. The crucial feature of
these solutions is that they are localised in space and have the flux of the
color magnetic Proca field passing through the central cross-section of the
Proca tube. This allows the possibility of comparing the field configurations
obtained with the Nielsen-Olesen tube. In both cases, there are common
features: (i) the presence of a longitudinal magnetic field creating the flux
of the magnetic field over the transverse cross-section of the tube; and (ii)
the scalar field pushes out the magnetic field (the analogue of the Meissner
effect). The main distinctions between the Proca tube and the Nielsen-Olesen
tube are: (i) the Proca tube has finite size, whereas the Nielsen-Olesen tube
is infinite; and (ii) the Proca tube is a topologically trivial object.

Natural questions arising in this context are: (i) Do Proca fields really
exist in nature? (ii) Where the tubes obtained by us could occur? Regarding
the first question, it should be pointed out that at the present time numerous
studies are carried out to find such fields in nature, see, e.g., Refs. [5,
18, 19, 20]. According to another point of view, Proca theories may be not
fundamental but some phenomenological approaches to other theories. For
instance, one might suppose that the SU(2) Proca-Higgs theory could serve as a
phenomenological description of some phenomena in SU(3) QCD. For example, this
could be in the case where some degrees of freedom belonging to the subgroup
$\text{SU(2)}\subset\text{SU(3)}$ acquire mass as a result of quantization,
and such degrees of freedom are approximately described by a SU(2) Proca
theory, whereas the remaining degrees of freedom are purely quantum and are
approximately described by a scalar field as a condensate. Of course, such a
description is approximate and is rather similar to a phenomenological
description of superconductivity by the Ginzburg-Landau equation.

Regarding the second question, one may suppose that if Proca fields do exist
in nature (it is unimportant whether they are fundamental or phenomenological
ones), at high temperatures, in non-Abelian Proca theories, there can exist a
“quark-Proca-gluon-Higgs” plasma; this is analogous to what happens in QCD
when the quark-gluon plasma occurs.

Summarizing the results obtained in the present paper:

  * •

We have studied finite-size tube solutions in the non-Abelian Proca-Higgs
theory both with and without external sources.

  * •

It is shown that such configurations have the finite total energy and the flux
of the magnetic field over the transverse cross-section.

  * •

It is demonstrated that there exist some critical values of the system
parameters $\mu_{2}$ and $(j_{0})^{7\varphi}$ for which the electric field is
switched off and only the magnetic field remains nonzero.

  * •

It is established that for the Proca tubes to exist, the presence of a scalar
field is necessary; such field acts as Cooper pairs in a superconductor and
pushes out the Proca field creating localised objects.

  * •

Comparison of properties of the solutions obtained with the properties of the
Nielsen-Olesen tube has been carried out.

  * •

It is assumed that such particlelike solutions may describe quasiparticles in
the hypothetical “quark-Proca-gluon-Higgs” plasma.

In conclusion, we would like to emphasize the rich internal structure of Proca
theories allowing the possibility of obtaining the particlelike solutions
studied in the present paper, whose existence is ensured by the presence of a
mass of the vector Proca bosons.

## Acknowledgements

We gratefully acknowledge support provided by the program No. BR10965191 of
the Ministry of Education and Science of the Republic of kazakhstan. We are
also grateful to the Research Group Linkage Programme of the Alexander von
Humboldt Foundation for the support of this research. We thank Jutta Kunz and
Burkhard Kleihaus for many useful discussions and helpful correspondence.

## References

  * [1] R. Brito, V. Cardoso, C. A. R. Herdeiro, and E. Radu, “Proca stars: Gravitating Bose-Einstein condensates of massive spin 1 particles,” Phys. Lett. B 752, 291 (2016).
  * [2] C. A. R. Herdeiro, A. M. Pombo, and E. Radu, “Asymptotically flat scalar, Dirac and Proca stars: discrete vs. continuous families of solutions,” Phys. Lett. B 773, 654 (2017).
  * [3] V. Dzhunushaliev and V. Folomeev, “Dirac star in the presence of Maxwell and Proca fields,” Phys. Rev. D 99, no. 10, 104066 (2019).
  * [4] C. Herdeiro, I. Perapechka, E. Radu, and Y. Shnir, “Asymptotically flat spinning scalar, Dirac and Proca stars,” Phys. Lett. B 797, 134845 (2019).
  * [5] J. C. Bustillo, N. Sanchis-Gual, A. Torres-Forné, J. A. Font, A. Vajpeyi, R. Smith, C. Herdeiro, E. Radu, and S. H. W. Leong, “GW190521 as a Merger of Proca Stars: A Potential New Vector Boson of $8.7\times 10^{-13}$ eV,” Phys. Rev. Lett. 126, no.8, 081101 (2021).
  * [6] L. Heisenberg, R. Kase, M. Minamitsuji, and S. Tsujikawa, “Hairy black-hole solutions in generalized Proca theories,” Phys. Rev. D 96, no.8, 084049 (2017).
  * [7] E. Allys, P. Peter, and Y. Rodriguez, “Generalized Proca action for an Abelian vector field,” JCAP 1602 no.02, 004 (2016).
  * [8] E. Allys, P. Peter, and Y. Rodriguez, “Generalized SU(2) Proca Theory,” Phys. Rev. D 94, no.8, 084041 (2016).
  * [9] M. Minamitsuji, “Solutions in the generalized Proca theory with the nonminimal coupling to the Einstein tensor,” Phys. Rev. D 94, no.8, 084039 (2016).
  * [10] A. De Felice, L. Heisenberg, R. Kase, S. Mukohyama, S. Tsujikawa, and Y. l. Zhang, “Cosmology in generalized Proca theories,” JCAP 06, 048 (2016).
  * [11] V. Dzhunushaliev and V. Folomeev, “Proca tubes with the flux of the longitudinal chromoelectric field and the energy flux/momentum density,” Eur. Phys. J. C 80, no.11, 1043 (2020).
  * [12] V. Dzhunushaliev, V. Folomeev, T. Kozhamkulov, A. Makhmudov, and T. Ramazanov, “Non-Abelian Proca theories with extra fields: particlelike and flux tube solutions,” Phys. Scripta 95, no.7, 074013 (2020).
  * [13] V. Dzhunushaliev, V. Folomeev, and A. Tlemisov, “Linear energy density and the flux of an electric field in Proca tubes,” Symmetry 13, no.4, 640 (2021).
  * [14] H. B. Nielsen and P. Olesen, “Vortex-line models for dual strings,” Nucl. Phys. B 61, 45 (1973).
  * [15] E. V. Shuryak and I. Zahed, “Rethinking the properties of the quark gluon plasma at T approximately T(c),” Phys. Rev. C 70, 021901 (2004).
  * [16] E. V. Shuryak and I. Zahed, “Towards a theory of binary bound states in the quark gluon plasma,” Phys. Rev. D 70, 054507 (2004).
  * [17] W. Schönauer and R. Weiß, “Efficient vectorizable PDE solvers,” J. Comput. Appl. Math. 27, 279 (1989).
  * [18] R. Kase, M. Minamitsuji, and S. Tsujikawa, “Neutron stars with a generalized Proca hair and spontaneous vectorization,” Phys. Rev. D 102, no.2, 024067 (2020).
  * [19] N. Sanchis-Gual, C. Herdeiro, J. A. Font, E. Radu, and F. Di Giovanni, “Head-on collisions and orbital mergers of Proca stars,” Phys. Rev. D 99, no.2, 024017 (2019).
  * [20] A. de Felice, L. Heisenberg, and S. Tsujikawa, “Observational constraints on generalized Proca theories,” Phys. Rev. D 95, no.12, 123540 (2017).