# Sem4SAP: Synonymous Expression Mining From Open Knowledge Graph For Language
Model Synonym-Aware Pretraining

Zhouhong Gu,1 Equal Contribution Sihang Jiang∗,1 Wenhao Huang1 Jiaqing Liang2
Hongwei Feng,1 Corresponding Authors Yanghua Xiao†,1,3 1Shanghai Key
Laboratory of Data Science, School of Computer Science, Fudan University,
China
2School of Data Science, Fudan University
3Fudan-Aishu Cognitive Intelligence Joint Research Center {zhgu20,
liangjiaqing<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The model’s ability to understand synonymous expression is crucial in many
kinds of downstream tasks. It will make the model to better understand the
similarity between context, and more robust to the synonym substitution
attack. However, many Pretrained Language Model (PLM) lack synonym knowledge
due to limitation of small-scale synsets and PLM’s pretraining objectives. In
this paper, we propose a framework called Sem4SAP to mine synsets from Open
Knowledge Graph (Open-KG) and using the mined synsets to do synonym-aware
pretraining for language models. We propose to coarsly filter the content in
Open-KG and use the frequency information to better help the clustering
process under low-resource unsupervised conditions. We expand the mined
synsets by migrating core semantics between synonymous expressions. We also
propose two novel and effective synonym-aware pre-training methods for
injecting synonym knowledge into PLMs. Extensive experiments demonstrate that
Sem4SAP can dramatically outperform the original PLMs and other baselines on
ten different tasks.

##  1 Introduction

Synonymous expressions are words, morphemes, or phrases that mean exactly or
nearly the same to each other Stanojević and others (2009). In the field of
Natural Language Process (NLP), synonymous expressions play an important role
in various applications. For example, many downstream tasks have suffered from
synonym substitution attack Chiang and Lee (2022), the attack brings in
imperceptible perturbations, while not changing human predictions, will make a
well-trained NLP model behave much worse than usual. And synonym knowledge
facilitates models to capture fine-grained semantic relations in some
similarity-oriented tasks in the field of Information Retrieval (IR) Li et al.
(2022). For the tasks like entity linking Hoffart et al. (2011) and knowledge
graph canonization Galárraga et al. (2014), the core challenge lies in
modeling the semantic similarity of entities in complex contexts, where
understanding the synonymous relationship between phrases is essential.

Figure 1: An example of how Sem4SAP works for synonymous expression mining in
Open-KG and improving model’s understanding of synonymous expressions.

However, most prevailing language models (LMs) have limited ability to
understand synonymous expressions. Statistical language models Cavnar et al.
(1994) is based on a probabilistic approach where they assign probabilities to
each word in a sentence given the context. Without directly using synonymous
expressions to do the augmentation, statistical language models will take
synonymous expressions only as common words with no similar semantics.
Pretrained Deep Language Models (PLMs) Peters et al. (2018) play an important
role in NLP nowadays, however, they are mainly trained by mask prediction task
Devlin et al. (2018), which primarily concern with the probability of a
certain sequence of words, but do not concern with the nuances of the word’s
semantics. As a result, PLMs, even large-scale PLMs, tend only to pick a word
that fits the context best but not understand the synonymous relation between
words.

Based on the above limitations of the existing models, we propose to improve
machine’s understanding to synonymous expressions from the following two
aspects:

  1. 1.

Construct large-scale explicit synsets. It is difficult for the models
themself to capture implicit synonymous relations directly from context, let
alone enhance the model’s understanding of synonymous expressions. Starting
from explicit synset, which is the set of synonymous expressions, is more
feasible at present.

  2. 2.

Develop synonym-aware pretraining tasks. The prevailing pretraining tasks
prevent PLMs from being aware of the synonymous expressions. Therefore, novel
pretraining tasks which directly inject synonym knowledge into LMs need to be
developed.

Although there have some explicit synsets, such as Wordnet Miller (1995) or
BigCilin Liu et al. (2022), it is difficult to directly apply them to
facilitate downstream tasks for their homogeneous expression and the limited
number of synonymous relation. We find it feasible to use Open Knowledge Graph
(Open-KG) Lehmann et al. (2015) to improve the diversity of the expressions
and to enlarge the number of synonymous relations of existing explicit
synsets. The Open-KG is constructed from the online encyclopedia, which is
contributed by a large number of netizens’ crowdsourcing. From the table
listed in Figure 1, it is intuitive to find that there are many diverse
synonymous expressions in Open-KG. For example, gender is generally be divided
into two categories, but there have 233 different expressions of gender in
Wiki-data Vrandečić and Krötzsch (2014), which is one of the most famous Open-
KG in the world, and even 534 different expressions of gender in CN-DBpedia Xu
et al. (2017), which is the largest Chinese Open-KG.

The limitation of the existing pretraining tasks lies in lacking of the
objectives for learning to represent the relation among synonymous
expressions. The challenge is that although synonymous expressions have the
same semantics most of time, they also have some specific meaning in some
sepcific aspects. So, the representations of two sentences with only
synonymous differences should be similar, but the ambiguity should also be
kept.

In this paper, we propose Sem4SAP. As shown in Figure 1, Sem4SAP is a
framework of Synonymous Expressions Mining in Open-KG for Synonym-Aware
Pretraining. In Synonymous Expression Mining, Sem4SAP leverages the frequency
information in Open-KG to detect the core semantic of a expression and uses
both textual and distributed representational features of the core semantic to
calculate the similarity between each expression for better clustering
expressions into synsets. Besides, by keeping track of the updated content in
Open-KG, synsets will always contain the newly emerged expressions, which
ensures the output of Sem4SAP is always closely related to the up-to-date
Internet. In Synonym-Aware Pretraining, Sem4SAP consists of two complementary
pretraining tasks, which inject synonymous expression knowledge and synonymous
context knowledge into models, and two tricks to keep the specific ambiguity
of the synonymous expressions.

In summary, we conclude that this paper makes the following contributions:

  1. 1.

We propose to acquire the up-to-date synsets with greatly diverse synonymous
expressions by using the crowd-sourcing content in continually updating Open-
KG.

  2. 2.

We propose a novel synonymous expressions mining method to cluster the content
in Open-KG into synsets, and we also propose two novel pretraining tasks to
inject the synonym knowledge into LMs.

  3. 3.

Extensive experiments have been carried out to verify that Sem4SAP outperforms
all the other baselines in improving the synonymous expressions understanding
to models, and every component in Sem4SAP is effective.

##  2 Preliminary

####  2.0.1 Entity, Predicate, Triple, Subject, Object,
Knowledge Graph.

An _entity_ can be a thing, person, place, e.g. Apple Inc., Donald Trump,
U.S., etc. A _predicate_ describes the characteristics of the entity. A
_triple_ $<s,p,o>$ is the fact that describes _subject_ $s$, while $p$ is a
predicate, and $o$ is _object_. For example, triple $<$U.S., President, Donald
Trump$>$ represents the fact that the president of the U.S. is Donald Trump.
Then a _Knowledge Graph (KG)_ $G=<E,T>$ consists of entities $E$ and triples
$T$. Specifically, entities and triples constitute the essential elements of a
KG Lehmann et al. (2015). And Open-KG, which differs from the Knowledge Bases
(KB), is the most primitive KG, while KB is a strictly processed KG.

####  2.0.2 Relation, Property, Value.

Predicate can be further divided into relation and property. A _relation_ is
generally used to represent the relationship between entities, while a
_property_ is used to describe some features of a subject. _Value_ is a
concrete description to the subject. So both entity and value can all be the
object to a triple, while only entity can be the subject to a triple. A
relation connects two entities, while a property connects an entity and a
value.

####  2.0.3 Synonym, Synonymous Expression, Synset.

_Synonym_ is a word or phrase that means exactly or nearly the same as another
word or phrase Stanojević and others (2009). For example, “Aspirin” and
“Acetylsalicylic Acid” refer to the same drug; “United States” and “USA”
represent the same country. _Synonymous expression_ is a relaxed form of
synonym, which is a sequence of characters that means exactly or nearly the
same as another sequence of characters in the same language. For example,
“gender male” and “man gender” refer to the same gender. And _synset_ is a set
of synonyms or synonymous expressions that are grouped together.

Figure 2: Sem4SAP consists of two main components and a optional component.
First, Sem4SAP conduct the Synonymous Values mining in Open-KG. Then Sem4SAP
use the mined synsets to do the expansion. Finally, Sem4SAP use two synonym-
aware pretraining methods to boost the PLMs’ understanding to synonym.

##  3 Synonymous Values Mining

In this section, we elaborate the Synonymous Expressions Mining method of
Sem4SAP. Our method targets the value objects in Open-KG. Since value object
is connected by property but not relation, we detailed our method of
distinguishing property from relation in subsection 3.1. The similarity
calculation method is in subsection 3.2. And we introduce the synsets
clustering method in subsection 3.3 Illustration of methods in this section is
shown in the upper of Figure 2

###  3.1 Property Selection

Since entity contains only specific semantics, while value contains more
general semantics, which is expected to facilitate more aspects than entity.
Value only appears as the object in triplet _$\langle$ s,p,o$\rangle$_, so the
mining targets at the tail of the triplet. However, predicate is a general
semantic constraint to the objects, we expect to find the predicate which has
the following characteristics to better find the most descriptive values:
First, predicate should describe some general features of an entity, or to say
it should be a property but not a relation. Second, predicate should describe
a categorical feature of an entity but not quantitative. The distribution of
objects has been used to distinguish whether the given predicate is a property
of relation Li et al. (2021). As shown in Figure 3, bigger the Shannon entropy
of the object distribution is, more possible the predicate is a relation
(predicate = Address) or quantitive property (predicate = Birthday), otherwise
it is more likely a property (predicate = Gender or Nationality).

Figure 3: The distribution of objects connected by some predicates and the
entropy of these distributions.

Besides, the precision of finding categorical properties can be further
improved. We observe that the value of categorical property tends to have
concentrated semantics, like the semantics of gender, which tend to
concentrate on the “male” or “female”. We propose to use the Shannon entropy
of the distribution to all possible word-pieces Wu et al. (2016) (for example,
all possible word-pieces to expression “man” are “m”, “a”, “n”, “ma”, “an”,
“man”) as well as the entropy of the distribution to all objects to
distinguish property. And the formula for calculating the Possibility of
Categorical Properties (PCP) is as follows:

$\small PCP(p_{i})=\frac{\\#c}{ent(Val(p_{i}))*ent(Str(p_{i}))}\ \forall c\in
C_{p_{i}}.$ (1)

$Val(p_{i})$ and $Str(p_{i})$ represent the value frequency distribution and
word-pieces frequency distribution of given predicate $p_{i}$ respectively.
$C_{p_{i}}$ denote all characters in the objects connected by the given
predicate $p_{i}$, and $\\#c$ denote the number of given character $c$ in all
objects of the given predicate, which is used as a penalty term in the
formula. And $ent(p_{i})$ represents the Shannon Entropy of the value
distribution of $p_{i}$.

We use Formula 1 to get possibilities for all predicates. Higher the
$PCP(p_{i})$ is, more possible $p_{i}$ is categorical.

###  3.2 Similarity Calculation

We propose to use the frequency information of every word-pieces to detect the
important part of the value objects. Then use textual features and distributed
representation features of these important parts to cluster value objects into
synsets.

For textual features, we count the number of occurrences of each word-piece of
all value objects as the frequency. And use the frequencies of every word-
pieces as the weight of importance. If two value objects have more high-
weighted word-pieces in common, they will share a high similarity for their
important part is similar. Specifically, we use the formula of TF-IDF Ramos
and others (2003) improved with the frequency of every character to calculate
the similarity of two value objects:

$TS_{\mathbb{TF-IDF}}(o_{m},o_{n})=\frac{f(o_{m}\cap o_{n};O_{p})}{f(o_{m}\cup
o_{n};O_{p})},$ (2)

where $\mathbb{TF-IDF}$ denote we are using the TF-IDF method to calculate the
Textual Similarity, $O_{p}$ denote all the value object $o$ of property $p$,
and $f(o;O)$ denote the sum of the frequencies from characters $o$ in $O$.

For distributed representation similarity, we mainly use the LM embedding as
the distributed representation of the value objects. Since LMs have limited
expressive ability in representing sequences with few tokens Batista (2018),
we input all word-pieces of a value object into LMs. The frequencies of each
word-pieces of a value object are used as the weights to calculate the
representation, which means the important word-piece of a value object will
contribute more to the embedding of a value object. We use cosine similarity
to calculate the distributed representation similarity between two value
objects, the specific formula is as follows:

$\small
DRS_{\mathbb{M}_{j}}(o_{m},o_{n})=\frac{LM_{\mathbb{M}_{j}}(o_{m};\Theta)\cdot
LM_{\mathbb{M}_{j}}(o_{n};\Theta)}{\parallel
LM_{\mathbb{M}_{j}}(o_{m};\Theta)\parallel\cdot\parallel
LM_{\mathbb{M}_{j}}(o_{n};\Theta)\parallel},$ (3)

where $LM$ denote the LM model, $\mathbb{M}_{j}$ denotes the different LM
type, and $\Theta$ denotes the different parameters of LM.
$LM_{\mathbb{M}_{j}}(o_{m};\Theta)$

Finally, we combine the above two formulas as follows to get the final
Semantic Similarity of two value objects:

$\displaystyle SS($ $\displaystyle\langle p_{i},o_{m}\rangle,\langle
p_{j},o_{n}\rangle)$ (4) $\displaystyle=\begin{cases}0&p_{i}\neq p_{j}\\\
\prod^{N_{TS}}_{i}TS_{\mathbb{S}_{i}}(o_{m},o_{n})*\prod^{N_{DRS}}_{j}DRS_{\mathbb{M}_{j}}(o_{m},o_{n})&p_{i}=p_{j}\end{cases}.$

###  3.3 Synsets Clustering

A weighted complete similarity network can be formed after calculating the
similarity between all value objects. The external synonym database can be
integrated by setting a new node and adjusting the edge weight in the network.
Some community detection methods can be used to cluster value objects into
synsets based on the similarity network. And in this paper, we propose to
remove edges between value objects which have a low similarity between each
other, and simply apply Louvain algorithm Blondel et al. (2008) to do the
clustering.

Model | GLUE | CLUE
---|---|---
SST-2 | MRPC | QQP | MNLI-mm | RTE | AFQMC | TNEWS | IFLYTEK | CMNLI | CSL
Bert-base | 89.36 | 85.14 | 67.33 | 83.10 | 62.85 | 71.59 | 54.74 | 56.48 | 78.43 | 78.73
-BT | 93.5 | 88.9 | 71.2 | 83.4 | 66.4 | 73.70 | 56.58 | 60.29 | 79.69 | 80.36
+EDA | 89.14 | 84.16 | 65.68 | 80.47 | 62.71 | 70.17 | 52.42 | 55.14 | 75.65 | 76.74
+LIBERT | 86.48 | 85.29$\uparrow$ | 69.71$\uparrow$ | 81.12 | 64.51$\uparrow$ | 70.29 | 53.27 | 56.74$\uparrow$ | 74.33 | 77.49
+PICSO | 88.47 | 84.69 | 70.34$\uparrow$ | 82.11 |  63.56$\uparrow$ | 71.43 | 55.48$\uparrow$ | 55.40 | 74.19 | 79.16$\uparrow$
+Sem4SAP $\uparrow$ | 92.89$\uparrow$ | 88.41$\uparrow$ |  69.15$\uparrow$ | 83.08 | 65.16$\uparrow$ | 73.54$\uparrow$ |  56.82$\uparrow$ |  60.38$\uparrow$ | 78.41 |  81.22$\uparrow$
Roberta-base | 92.36 | 86.41 | 68.45 | 84.78 | 64.37 | 70.87 | 55.53 | 56.52 | 79.45 | 78.20
-BT | 94.6 | 89.8 | 71.5 | 85.2 | 67.3 | 74.45 | 57.12 | 60.88 | 80.17 | 80.59
+EDA | 91.78 | 86.04 | 67.14 | 83.45 | 62.51 | 69.43 | 53.48 | 55.67 | 76.64 | 76.31
+LIBERT | 86.91 | 86.13 |  69.38$\uparrow$ | 82.47 | 63.35 | 70.05 | 53.44 | 56.91$\uparrow$ | 74.46 | 74.84
+PICSO | 91.56 | 85.43 | 72.22$\uparrow$ | 82.35 | 62.19 | 68.51 | 56.41$\uparrow$ | 56.17 | 75.43 | 79.14$\uparrow$
+Sem4SAP | 93.13$\uparrow$ | 87.17$\uparrow$ | 71.48$\uparrow$ | 84.96$\uparrow$ | 66.35$\uparrow$ | 73.51$\uparrow$ |  57.34$\uparrow$ | 60.70$\uparrow$ |  80.35$\uparrow$ |  81.28$\uparrow$
Roberta-large | 93.17 | 90.54 | 87.60 | 89.45 | 86.34 | 72.84 | 57.15 | 59.79 | 81.11 | 80.10
-BT | 96.7 | 92.3 | 90.2 | 90.2 | 88.2 | 74.02 | 57.86 | 62.55 | 81.70 | 81.36
+EDA | 93.25$\uparrow$ | 88.37 | 86.51 | 89.22 | 85.48 | 73.10$\uparrow$ | 55.42 | 59.64 | 80.32 | 78.61
+LIBERT | 89.64 | 87.14 | 85.79 | 86.97 | 85.29 | 71.26 | 56.43 | 58.32 | 78.44 | 79.71
+PICSO $\uparrow$ | 92.48 | 90.04 |  89.17$\uparrow$ | 83.56 | 85.46 | 72.43 | 57.32$\uparrow$ | 58.25 | 76.42 | 80.56$\uparrow$
+Sem4SAP | 96.14$\uparrow$ | 91.47$\uparrow$ | 89.02$\uparrow$ | 89.17 | 87.77$\uparrow$ | 74.25$\uparrow$ |  57.64$\uparrow$ | 62.46$\uparrow$ | 81.04 |  82.41$\uparrow$

Table 1: Overall Experiments on GLUE and CLUE datasets

##  4 Synsets Expansion

In this section, we propose an extension method towards adding redundancy to
the synsets even though the mined synsets are enough for doing synonym-aware
pretraining. Since there has much redundant information in synonymous
expressions, understanding synonymous expression is more difficult than
synonyms. Adding redundant information to the synsets will help models
identify the irrelevant information of the expressions, which will improve the
robustness of the models.

By adding redundancy, we aim to find the expression pattern, which is composed
of important and unimportant parts to deliver core semantic or auxiliary
semantics. We propose to use frequency $f$, Point-wise Mutual Information
(PMI) Church and Hanks (1990), and left-right neighborhood characters richness
of every word-piece to find its core part.

PMI is a measurement for the association of two given characters, referring to
the probability of a combination of characters occurring as a whole word. Left
and right neighborhood characters’ richness is also a metric to detect whether
the given sequence is a complete word or not, and it is often calculated by
the entropy of their left and right neighboring characters. The higher the
entropy of the left and right neighboring characters is, the richer the left
and right neighborhood characters’ richness is, which means the word-piece is
more likely to be used as a complete word.

We propose the following formula to calculate the Probability of Core Semantic
(PCS) of every word-pieces of an expression:

$\small PCS(x,y)=f_{xy}*PMI(x,y)*lrEnt(xy).$ (5)

$f_{xy}$ denote the importance of word-piece $xy$, which is concatenate by
word-piece $x$ and $y$. $PMI$ and $lrEnt$ denote PMI metric and left-right
entropy.

We choose word-pieces with higher $PCS$ scores to do the replacement with each
other within the same property just like the example in the middle of Figure
2. In this way, we will get expanded synsets.

##  5 Synonym-Aware Pretraining

In order to make better use of the mined synsets, we design a pretraining
method for synonym-aware LMs. The pretraining method consists of two
pretraining tasks to inject synonym knowledge into models, and two extra
tricks to prevent overfitting and semantic drifting of the models.

Token-level Synonym Boosting: We first design a token-level synonym boosting
task for injecting vocabulary synonym knowledge into LMs. This task allows the
LM to consider two synonymous expressions as having the same semantics. So the
objective of this task is to pull in the output of two expressions if they are
synonymous to each other, otherwise, remain unchanged.

Sentence-level Synonym Boosting: Then we design a sentence-level synonym
boosting task for injecting contextual synonym knowledge into LMs. This task
allows the LM to consider two sentences as having the same semantics if there
is only a synonym substitution difference between them. And the objective of
this task is just like the previous that pull in the output of two sentences
if they are synonymous.

Two extra tricks: We design two extra tricks to prevent overfitting and
semantic drifting, especially for continual pretraining based on other PLMs.

The first trick is to fix the output of the most common expressions and pull
in the outputs of the others in the token-level synonym pretraining task.
Since the most common expressions are trained most often, their
representations are more expressive than the other expressions. In this way,
the semantics of these most expressive expressions are delivered to the other
synonymous expressions.

The second trick is to stop pulling in when the distance between the outputs
of synonymous expressions is much shorter than the initial distance in both
synonym boosting pretraining tasks. It ensures that the model’s comprehension
of similar texts will not become identical, which remains the ambiguity of
different expressions in some aspects.

Figure 4: The ablation study of our synonym mining method. The left chart
shows the final results of downstream tasks. Middle’s shows the result on
Synset Mining on DBpedia. Right’s shows the result on Synset Mining on CN-
DBpedia.

##  6 Experiments

In this section, we evaluate the effectiveness of Sem4SAP. The setup of our
experiments is detailed in Sec. 6.1. The overall experiments of Sem4SAP and
other baselines are detailed in Sec. 6.2. The ablation study is shown in Sec.
6.3. And the case study is in Sec. 6.4.

Word | ENG Synset | CHN Synset | WordNet 3.0
---|---|---|---
Female | 11 | 16 | 3
U.S. | 7 | 8 | 11
Ph.D. | 10 | 12 | 2
♂ | 14 | 22 | 0
_#Words_ | 34,329 | 65,010 | 147,306
_#Expended_ | 272,745 | 2,719,864 | -

Table 2: Results and examples of synsets mined by Sem4SAP. Second row and
third row denote the number of synonymous expressions to the word in first row
in our mined synsets, and fourth row denote the number in Wordnet.

###  6.1 Experiments Setup

####  6.1.1 Datasets

We conduct synonymous expressions mining in English Open-KG Dbpedia Lehmann et
al. (2015) and Chinese Open-KG CN-DBpedia Xu et al. (2017), the result and
some examples of our mined synsets have listed in Table 2. Since these two
Open-KGs are composed of content in the general area, so we use General
Language Understanding Evaluation benchmark GLUE Wang et al. (2018), and
Chinese general Language Understanding Evaluation benchmark CLUE Xu et al.
(2020) to verify the effectiveness of Sem4SAP. These benchmarks consist of
many datasets, and we choose ten of them that we think best suit our
evaluation, the detailed description, and statistics will be listed in the
Appendix. Besides, since we aim at improving the model’s understanding of
synonymous expressions, so we add back translated data to the test set of
these datasets. Specifically, we use Youdao Translation API to generate new
data by translating the data in the test set into English or Chinese and back-
translating them, and adding new data to the test set. Back translation will
bring in many synonymous expressions, which is able to verify whether the
models will be affected by synonymous substitution attacks.

####  6.1.2 Baselines

We choose several works that can also enhance PLM’s understanding of
synonymous expressions as baselines. We also choose Bert and Roberta as the
backbone PLMs. The abbreviation and description of these works are as follows:

Bert Devlin et al. (2018) is one of the most famous PLMs which shows its great
ability in natural language understanding. Roberta Liu et al. (2019) has made
progress in the pretraining technique which performs better than Bert in
natural language understanding. -BT denotes the performance of the original
dataset. EDA Wei and Zou (2019) is one of the most famous text data
augmentation methods in using four text transformations, which are synonym
substitution, random insertion, random swapping, and random deletion, to
enhance the performance of the model. LIBERT Lauscher et al. (2019) trains
Bert from scratch with an auxiliary task that binary classifies whether entity
pairs are synonymous pairs. PICSO Li et al. (2022) use an adapter to store the
synonymous knowledge without undermining the raw knowledge in PLM. And Sem4SAP
is our proposed method.

####  6.1.3 Parameter Setting

In synonymous expressions mining phrase, we choose 5000 properties in Chinese
and English Open-KGs respectively with the lowest $PCP$ score, and clear 40%
edges with the lowest weight during the clustering. In Synonym-Aware
Pretraining, we take the expressions that match exactly the value objects of
the highest frequency in Open-KG as the most common expressions. We also
choose to stop pulling in the output of the synonymous expressions or
sentences when their distance has already been pulled in at least 60 % shorter
than the initial distance.

####  6.1.4 Fairness of the Experiments

To keep the fairness of the experiments, we first acquire the text from the
training data in all datasets listed in Table 1. Then, we conduct the synonym
substitution to the acquired text by replacing the words or phrases in the
text which match exactly to the expressions in mined synsets. Finally, we have
at least pretrain all baselines in the way how backbone PLMs are trained. So,
the pretraining corpus for all backbone PLMs is the same, the only difference
is the pretraining method.

###  6.2 Overall Experiments

We compare baselines on both GLUE and CLUE dataset, and the experimental
results are listed in Table 1. The _yellow_ cell denotes the result of the
original dataset without doing back translation, and most of these results are
cited from the paper of Robert, and Bert and from the leaderboard of GLUE and
CLUE. The _green_ cell denotes the best result, and the _lime_ cell denotes
the second-best result. The $\uparrow$ denotes whether the performance is
higher than the raw PLM’s. The double underline denotes whether the
performance on the back translation dataset is even higher than the
performance on the original dataset.

Robustness of PLMs: From the result, we can see that the performances of the
PLMs on original datasets are mostly better than the performances on the
dataset added back to translation data, which represents that the existing
PLMs are all suffering from substitution attacks.

Performance of Baselines: The EDA and LIBERT mostly lower the performance of
the raw PLM, this is because they introduce synonym knowledge improperly,
which impairs the existing knowledge in PLMs. PICSO perform well on QQP,
TNEWS, and CSL datasets, which correspond to the results in their paper.

Performance of Sem4SAP: Sem4SAP out-perform most other baselines in most
tasks. Sem4SAP is also a stable method that achieves better performance in all
the tasks than raw PLM except for some Natural Language Inference tasks (MNLI-
mm, CMNLI).

Performance of Large-scale PLM: For models with larger parameters (Roberta-
large), most baselines only inject harmful information to the model while ours
further improve the performance. Moreover, for large models, Sem4SAP still
makes the performance better than some of the original datasets, which means
using Sem4SAP to inject synonym knowledge is also an effective way of text
data augmentation to the large-scale PLMs.

###  6.3 Ablation Study

The result of ablation studies is presented in Figure 4. The left figure shows
the performances on different downstream applications by removing seven
components in Sem4SAP respectively. The middle and right figures show the
quality of mined synsets in Chinese and English language by removing four
components of our synonym mining method one by one, the results are annotated
by three graduate students. The label in the Figure 4 are listed as follows:
“PS: random” denote we randomly choose the properties in Property Selection
phrase. “SC: w/o f” denote we don’t use the frequency information in Synset
Clustering phrase. “SC: w/o ts” denote we remove the textual similarity in
Synset Clustering phrase. “SC: w/o drs” denote we remove the distributed
representational similarity in Synset Clustering phrase. “SE: w/o SE” denote
we don’t do the Synset Expansion. “SAP: w/o TSB” denote we remove the Token-
level Synonym Boosting method in Synonym-Aware Pretraining phrase. “SAP: w/o
SSB” denote we remove the Sentence-level Synonym Boosting method in Synonym-
Aware Pretraining phrase.

We use Bert-base model in all three figures. In the left figure, we use
accuracy metric to evaluate the result of ablation study. In the middle and
right figure, we use five additional metrics to present the result of the
clustering: “N_S” represents the number of Synsets mined by Sem4SAP. “N_sv”
represents the number of different value objects mined by Sem4SAP. “N_esv”
represents the number of synonymous expressions obtained by Sem4SAP after
doing synset expansion. “RI” indicate the Rand Index Rand (1971), which is a
metric to denote the clustering purification。 “w f” or “w/o f” denote
calculating RI with frequency of the value objects or not.

From Figure 4, we can find that the method of Property Selection and using
frequency to do the clustering are the two most important designs of our
synonymous expression mining method. Few synonyms and many synsets are
clustered by randomly selecting properties, which means only a few synonyms in
each synset, and the clustering purity is really low, resulting in the
smallest covering area in the left figure. When frequency information is not
used, the mined synsets will contain a large number of synonyms, but the
clustering purity is very low, resulting in the second smallest covering in
the left figure. And Sem4SAP will be better than bert-base when we use at
least the above two methods. Moreover, we found that removing the token-level
or sentence-level synonym boosting method will reduce the performance of the
model a lot, so we conclude that these two methods are complementary. The
token-level synonym boosting method enhances the ability of the model to
understand specific synonyms, while the sentence-level synonym boosting method
enhances the ability of the model to understand synonyms in context.

###  6.4 Case Study

Here we give some cases of Sem4SAP in Table 3. The first table is the cases of
our mined synsets. Although there have some symbols in Open-KG, “♂” for
example, Sem4SAP still understand its meaning by mining the co-occurrence of
other words, like “男性” (male) $\rightarrow$ “男♂” (male ♂) $\rightarrow$ “♂的”
(gender of ♂). Sem4SAP cluster “美国 (U.S.A)” and “鹰国 (Country of Eagle)” in the
same synset, which shows Sem4SAP has a good understanding of slangs. We think
this is because “美国 (U.S.A)” and “老鹰 (Eagle)” may have some co-occurrence in
pre-training data of PLMs, and Sem4SAP can well integrate the existing
knowledge in PLMs and also capture the important parts of the words to better
understanding them.

The second table gives some cases of how performance changes by using Sem4SAP.
But it still suffers from synonym substitution attacks. When the Chinese
phrase “找私房钱” (look for own money) changes into “找钱” (look for money, give
change), the model will not identify it as a sentence of amusement anymore. If
the English phrase “end raunchy” changes into “end crass”, the model may
identify that “crass” is a much more negative word than “raunchy”, then
identify the sentence as “Negative Emotion”. And Sem4SAP can make models
understand that these are synonyms and that they should have similar semantics
under the given context, which will improve the performance in downstream
tasks.

Mined Synset
---
Case 1 ✓
Gender
男性 (male), 男♂ (male ♂), 男孩子 (boy), ♂的 (of ♂)
Case 2 ✓
Status of the Novel
暂停 (pause), 持续暂停 (pausing), 正在休息 (resting)
Case 3 ✗
Nationality
美国 (U.S.A), 鹰国 (Country of Eagle), 鹰酱 (Cute Eagle)
Performance Changes in Downstream Tasks
Case 1
Dataset: TNews
Label: 休闲益智 (Amusing)
Roberta-large: 自己的私房钱还是得自己来找了。 ✓
Translate: I’m gonna have to find my own cash.
Roberta-large: 我得自己去找钱了。 ✗
Sem4SAP: 自己的私房钱还是得自己来找了。 ✓
Token-level boosting: 私房钱, 私房钞票, 钱 (money, hidded money)
Sentence-level boosting: 另类解谜游戏，可以教你如何藏钱哦。
Sentence-level boosting: 另类解谜游戏，可以教你如何藏私房钱哦。
(An alternative puzzle game that will teach you how to hide money)
Sem4SAP: 我得自己去找钱了。 ✓
Case 2
Dataset: SST
Label: Positive Emotion
Roberta-large: It’s is your ticket right here to end raunchy college humor. ✓
Translate: 这是你的一张票，你将结束粗俗的大学幽默。
Roberta-large: This is your ticket, you will end crass college humor. ✗
Sem4SAP: It’s your ticket right here to end raunchy college humor. ✓
Token-level boosting: raunchy, crass, raunchy life
Sentence-level boosting: Crass, then gasp for gas, verbal deportment
Sentence-level boosting: Raunchy, then gasp for gas, verbal deportment
Sem4SAP: This is your ticket, you will end crass college humor. ✓

Table 3: Case study of our proposed synonym mining method and text data
augmentation algorithms

##  7 Related Works

###  7.1 Synonym Detection

Early efforts on synonym detection focus on finding entity synonyms from a
semi-supervised situation such as query logs Ren and Cheng (2015), web tables
He et al. (2016), and synonymy dictionaries Ustalov et al. (2017). In
comparison, our work aims at developing a framework to cluster all chosen
vocabulary terms into synsets without any supervised signals. Modern synonym
detection combines entity set expansion task and synonym detection task which
are considered tightly coupled to each other Shen et al. (2020), and the
syntactic dependencies are more developed by using deep model to find
synonymous meaning Yang et al. (2022). There are other works who tries to find
synonyms in an Open-KG Qu et al. (2017) especially when there is a need to
canonize an Open-KG, but most them utilize side information to compensate for
the lack of semantic Vashishth et al. (2018); Lin and Chen (2019); Shen et al.
(2022), others detecting synonyms by jointly conducting entity linking Liu et
al. (2021). However, they ignore that not all the content in KG is suitable
for synonym detection, so their methods are too time-extensive and results are
noisy. Our methods first detect which terms are suitable to do the synonym
detection, and utilize three heterogeneous features to extract the semantics
of the terms and to cluster them into synsets.

###  7.2 Synonym Augmentation

There are many efforts have been made to use synonyms to enhance the effect of
downstream applications. LIBERT Lauscher et al. (2019) trains Bert from
scratch with an auxiliary task that binary classifies whether entity pairs are
synonymous pairs. SAPBERT pre-trains BERT with synsets from Unified Medical
Language System, which is a comprehensive collection of biomedical terms. Some
choose to use a synonym to construct sentences by simply using templates and
conducting Mask-Word-Predication task Yuan et al. (2022). And PICSO uses an
adapter to store the synonym knowledge to prevent the undermining of the
original PLMs Li et al. (2022). We aim to develop specific pre-training tasks
for injecting synonym knowledge from explicit synsets into LMs. The tasks we
developed not only ensures the enhancement of PLM’s ability to understand
synonyms, but also ensures that no semantic drift will be introduced.

##  8 Conclusion

In this paper, we aim to boost the performance of the models in synonym
understanding. We propose the Sem4SAP framework, which consists of three main
components. First, we do the Synonymous Values mining in Open-KG. Then we
expand the mined synsets. Finally, we use two synonym-aware pretraining
methods to boost the PLMs’ understanding of synonyms. In the future, we want
to use our method to mine more synsets and release them after manual
annotation.

## References

  * Batista [2018] D Batista.  Language models and contextualised word embeddings, 2018.
  * Blondel et al. [2008] Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre.  Fast unfolding of communities in large networks.  Journal of statistical mechanics: theory and experiment, 2008(10):P10008, 2008.
  * Cavnar et al. [1994] William B Cavnar, John M Trenkle, et al.  N-gram-based text categorization.  In Proceedings of SDAIR-94, 3rd annual symposium on document analysis and information retrieval, volume 161175. Citeseer, 1994.
  * Chiang and Lee [2022] Cheng-Han Chiang and Hung-yi Lee.  How far are we from real synonym substitution attacks?  arXiv preprint arXiv:2210.02844, 2022.
  * Church and Hanks [1990] Kenneth Ward Church and Patrick Hanks.  Word association norms, mutual information, and lexicography.  Computational Linguistics, 16(1):22–29, 1990.
  * Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * Galárraga et al. [2014] Luis Galárraga, Geremy Heitz, Kevin Murphy, and Fabian M Suchanek.  Canonicalizing open knowledge bases.  In Proceedings of the 23rd acm international conference on conference on information and knowledge management, pages 1679–1688, 2014.
  * He et al. [2016] Yeye He, Kaushik Chakrabarti, Tao Cheng, and Tomasz Tylenda.  Automatic discovery of attribute synonyms using query logs and table corpora.  In Proceedings of the 25th International Conference on World Wide Web, pages 1429–1439, 2016.
  * Hoffart et al. [2011] Johannes Hoffart, Mohamed Amir Yosef, Ilaria Bordino, Hagen Fürstenau, Manfred Pinkal, Marc Spaniol, Bilyana Taneva, Stefan Thater, and Gerhard Weikum.  Robust disambiguation of named entities in text.  In Proceedings of the 2011 conference on empirical methods in natural language processing, pages 782–792, 2011.
  * Lauscher et al. [2019] Anne Lauscher, Ivan Vulić, Edoardo Maria Ponti, Anna Korhonen, and Goran Glavaš.  Specializing unsupervised pretraining models for word-level semantic similarity.  arXiv preprint arXiv:1909.02339, 2019.
  * Lehmann et al. [2015] Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer, et al.  Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia.  Semantic web, 6(2):167–195, 2015.
  * Li et al. [2021] Chenguang Li, Jiaqing Liang, Yanghua Xiao, and Haiyun Jiang.  Towards fine-grained concept generation.  IEEE Transactions on Knowledge and Data Engineering, 2021.
  * Li et al. [2022] Yangning Li, Jiaoyan Chen, Yinghui Li, Tianyu Yu, Xi Chen, and Hai-Tao Zheng.  Embracing ambiguity: Improving similarity-oriented tasks with contextual synonym knowledge.  arXiv preprint arXiv:2211.10997, 2022.
  * Lin and Chen [2019] Xueling Lin and Lei Chen.  Canonicalization of open knowledge bases with side information from the source text.  In 2019 IEEE 35th International Conference on Data Engineering (ICDE), pages 950–961. IEEE, 2019.
  * Liu et al. [2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  arXiv preprint arXiv:1907.11692, 2019.
  * Liu et al. [2021] Yinan Liu, Wei Shen, Yuanfei Wang, Jianyong Wang, Zhenglu Yang, and Xiaojie Yuan.  Joint open knowledge base canonicalization and linking.  In Proceedings of the 2021 International Conference on Management of Data, pages 2253–2261, 2021.
  * Liu et al. [2022] Ming Liu, Yaojia LV, Jingrun Zhang, Ruiji Fu, and Bing Qin.  Bigcilin: An automatic chinese open-domain knowledge graph with fine-grained hypernym-hyponym relations.  arXiv preprint arXiv:2211.03612, 2022.
  * Miller [1995] George A Miller.  Wordnet: a lexical database for english.  Communications of the ACM, 38(11):39–41, 1995.
  * Peters et al. [2018] Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer.  Deep contextualized word representations.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237, 2018.
  * Qu et al. [2017] Meng Qu, Xiang Ren, and Jiawei Han.  Automatic synonym discovery with knowledge bases.  In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 997–1005, 2017.
  * Ramos and others [2003] Juan Ramos et al.  Using tf-idf to determine word relevance in document queries.  In Proceedings of the first instructional conference on machine learning, volume 242, pages 29–48. Citeseer, 2003.
  * Rand [1971] William M Rand.  Objective criteria for the evaluation of clustering methods.  Journal of the American Statistical association, 66(336):846–850, 1971.
  * Ren and Cheng [2015] Xiang Ren and Tao Cheng.  Synonym discovery for structured entities on heterogeneous graphs.  In Proceedings of the 24th International Conference on World Wide Web, pages 443–453, 2015.
  * Shen et al. [2020] Jiaming Shen, Wenda Qiu, Jingbo Shang, Michelle Vanni, Xiang Ren, and Jiawei Han.  Synsetexpan: An iterative framework for joint entity set expansion and synonym discovery.  arXiv preprint arXiv:2009.13827, 2020.
  * Shen et al. [2022] Wei Shen, Yang Yang, and Yinan Liu.  Multi-view clustering for open knowledge base canonicalization.  In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1578–1588, 2022.
  * Stanojević and others [2009] Maja Stanojević et al.  Cognitive synonymy: a general overview.  FACTA UNIVERSITATIS-Linguistics and Literature, 7(2):193–200, 2009\.
  * Ustalov et al. [2017] Dmitry Ustalov, Alexander Panchenko, and Chris Biemann.  Watset: Automatic induction of synsets from a graph of synonyms.  arXiv preprint arXiv:1704.07157, 2017.
  * Vashishth et al. [2018] Shikhar Vashishth, Prince Jain, and Partha Talukdar.  Cesi: Canonicalizing open knowledge bases using embeddings and side information.  In Proceedings of the 2018 World Wide Web Conference, pages 1317–1327, 2018.
  * Vrandečić and Krötzsch [2014] Denny Vrandečić and Markus Krötzsch.  Wikidata: a free collaborative knowledgebase.  Communications of the ACM, 57(10):78–85, 2014.
  * Wang et al. [2018] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman.  Glue: A multi-task benchmark and analysis platform for natural language understanding.  arXiv preprint arXiv:1804.07461, 2018.
  * Wei and Zou [2019] Jason Wei and Kai Zou.  Eda: Easy data augmentation techniques for boosting performance on text classification tasks.  arXiv preprint arXiv:1901.11196, 2019.
  * Wu et al. [2016] Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al.  Google’s neural machine translation system: Bridging the gap between human and machine translation.  arXiv preprint arXiv:1609.08144, 2016.
  * Xu et al. [2017] Bo Xu, Yong Xu, Jiaqing Liang, Chenhao Xie, Bin Liang, Wanyun Cui, and Yanghua Xiao.  Cn-dbpedia: A never-ending chinese knowledge extraction system.  In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, pages 428–438. Springer, 2017.
  * Xu et al. [2020] Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, et al.  Clue: A chinese language understanding evaluation benchmark.  arXiv preprint arXiv:2004.05986, 2020.
  * Yang et al. [2022] Dongqiang Yang, Pikun Wang, Xiaodong Sun, and Ning Li.  Synonym detection using syntactic dependency and neural embeddings.  arXiv preprint arXiv:2209.15202, 2022.
  * Yuan et al. [2022] Hongyi Yuan, Zheng Yuan, and Sheng Yu.  Generative biomedical entity linking via knowledge base-guided pre-training and synonyms-aware fine-tuning.  arXiv preprint arXiv:2204.05164, 2022.


# Latitudinal Propagation of Thermal Rossby Waves in Stellar Convection Zones

Rekha Jain  School of Mathematics and Statistics, University of Sheffield, S3
7RH UK<EMAIL_ADDRESS>Bradley W. Hindman  JILA, University of
Colorado, Boulder, CO 80309-0440, USA  Department of Applied Mathematics,
University of Colorado, Boulder, CO 80309-0526, USA

###### Abstract

Using an analytic model, we derive the eigenfrequencies for thermal Rossby
waves that are trapped radially and latitudinally in an isentropically
stratified atmosphere. We ignore the star’s curvature and work in an
equatorial f-plane geometry. The propagation of inertial waves is found to be
sensitive to the relative direction of the wave vector to the zonal direction.
Prograde propagating thermal Rossby waves are naturally trapped in the radial
direction for frequencies above a critical threshold, which depends on the
angle of propagation. Below the threshold frequency, there exists a continuous
spectrum of prograde and retrograde inertial waves that are untrapped in an
isentropic atmosphere, but can be trapped by gradients in the specific entropy
density such as occurs in a stellar convection zone. Finally, we discuss the
implications of these waves on recent observations of inertial oscillations in
the Sun, as well as in numerical simulations.

##  1 Introduction

A primary motivation for studying inertial oscillations of stars is their
implications in understanding the stellar interior structure. In particular,
observations of such oscillations may provide a strong constraint on
superadiabaticity and other thermodynamic variables within the star’s
convection zone (e.g., Gilman, 1987). Recent observations in the Sun of Rossby
waves and other inertial oscillations (e.g., Löptien et al., 2018; Hanasoge &
Mandal, 2019; Proxauf et al., 2020; Gizon et al., 2021; Hathaway & Upton,
2021; Hanson et al., 2022) have aroused interest in using these waves as
seismic probes of the solar interior and in the potential role that they play
in the Sun’s magnetic cycle (e.g., Dikpati & McIntosh, 2020). Through
exploitation of the Sun’s acoustic oscillations ($p$ modes), helioseismology
has successfully mapped the Sun’s differential rotation and its thermal
structure throughout the convection zone (see Christensen-Dalsgaard, 2002).
However, some quantities, such as the turbulent viscosity and radial entropy
gradient in the convection zone, are essentially invisible to the $p$ mode
oscillations. Further, the $p$ modes, as measured from the ecliptic, do not
sample high-latitudes well; therefore, flows and thermal structures in the
polar caps are poorly constrained. All of these missing pieces are important
elements in theories of stellar dynamics and the dynamo. Thus, there remains a
prominent gap in our understanding of the solar interior. The observation of
these new class of oscillations, which are likely to have sensitivity to many
of these parameters, could bridge that gap.

Thermal Rossby waves (e.g., Roberts, 1968; Busse, 1970; Hindman & Jain, 2022)
are a specific type of gravito-inertial wave that corresponds to convective
modes that have been partially or completely stabilized by rotation. While
they have yet to be detected in the Sun observationally, they are ubiquitous
in laboratory experiments of convection in a rotating fluid (e.g., Mason,
1975; Busse & Hood, 1982; Smith et al., 2014; Lonner et al., 2022). Further,
thermal Rossby waves are seen in numerical simulations of stellar convection
and are a crucial ingredient in the maintenance of a star’s differential
rotation (e.g., Brun et al., 2011; Miesch et al., 2000; Hindman et al., 2020).
These waves appear at convective onset and persist even when the fluid becomes
turbulent. In a Boussinesq fluid, the waves fill the spatial domain; but, in a
gravitationally stratified fluid, numerical simulations have shown that the
waves can be trapped in radius, being concentrated near the surface or near
the bottom of the convection zone depending on the stratification (Jones et
al., 2009; Hindman et al., 2020; Hindman & Jain, 2023). This strong variation
in the location of the wave cavity is a clear indication that these waves are
extremely sensitive to the stratification, and in particular the radial
entropy gradient. Thus, if detected, such waves would serve as an excellent
seismic probe.

While there is a rich literature on Rossby waves in stratified astrophysical
disks (e.g., Li et al., 2000; Lin, 2012), to date only a few studies have
explored how thermal Rossby waves propagate through a stratified star
(Glatzmaier & Gilman, 1981; Gilman, 1987; Hindman & Jain, 2022, 2023).
Further, for simplicity, all of these have ignored propagation and reflection
in the latitudinal direction. Studying the radial and latitudinal wave cavity
of low-mass stars with a near-surface convection zone will be our main aim
here. The present paper considers the propagation of thermal Rossby waves and
their kin in all three directions (zonal, latitudinal and radial) within an
adiabatically stratified background atmosphere. Section 2 describes a model to
this effect and in Section 3 we derive the resulting governing equation and
discuss the nature of the solutions. In section 4, we consider the eigenmodes
of a semi-infinite polytropic atmosphere and in Section 5 we consider finite
domains. Finally, we summarize and present our conclusions in Section 6.

##  2 The Model

Although most stars are nearly spherical, we consider a local Cartesian
coordinate system by defining a tangent plane at the star’s equator and
assuming that the rotation vector is uniform over the whole tangent plane.
This f-plane approximation simplifies the study of waves that have short
horizontal wavelengths. Therefore, we place the origin at the stellar surface
with the unit vectors $\hat{x}$, $\hat{y}$ and $\hat{z}$ pointing in the
longitudinal, latitudinal and radial directions, respectively. We adopt
uniform rotation to avoid singularities in the equations resulting from
critical layers where the local rotation rate equals a wave’s phase speed
(e.g., Gilman, 1987; Gizon et al., 2021).

We investigate the linearized fluid equations in the absence of a background
flow. We consider a steady-state background, denoted with subscript $0$, with
perturbations about that background indicated with subscript $1$. The
background atmosphere is assumed to be a plane-parallel atmosphere which is
gravitationally stratified with a constant gravitational acceleration
${\mbox{{\boldmath$g$}}}=-g{{\mbox{{\boldmath$\hat{z}$}}}}$. Thus, the
background pressure and density vary as a function of $z$ and are denoted by
$P_{0}(z)$ and $\rho_{0}(z)$, respectively. These quantities are related by
hydrostatic balance and the ideal gas law. The linearized equation of motion
that governs a rotating, inviscid fluid on an f-plane is given by:

$\displaystyle\frac{\partial{{{\mbox{{\boldmath$u$}}}}_{1}}}{\partial
t}=-2\left({\mbox{{\boldmath$\Omega$}}}\times{{\mbox{{\boldmath$u$}}}_{1}}\right)-\frac{1}{\rho_{0}}{\mbox{{\boldmath$\nabla$}}}P_{1}-\frac{g\rho_{1}}{\rho_{0}}{{\mbox{{\boldmath$\hat{z}$}}}}\;,$
(1)

where
${{\mbox{{\boldmath$u$}}}}_{1}=u{\mbox{{\boldmath$\hat{x}$}}}+v{\mbox{{\boldmath$\hat{y}$}}}+w{\mbox{{\boldmath$\hat{z}$}}}$
is the perturbed fluid velocity and $P_{1}$ and $\rho_{1}$ are the Eulerian
fluctuations of the pressure and density. The rotation vector points purely in
the latitudinal direction,
${\mbox{{\boldmath$\Omega$}}}=\Omega{\mbox{{\boldmath$\hat{y}$}}}$. The
linearized continuity equation for a compressible fluid is,

$\displaystyle\frac{\partial\rho_{1}}{\partial
t}+\rho_{0}({\mbox{{\boldmath$\nabla$}}}\cdot{\mbox{{\boldmath$u$}}}_{1})-\rho_{0}\frac{w}{H}=0\;,$
(2)

where

$\displaystyle H=-\left(\frac{d\ln\rho_{0}}{dz}\right)^{-1}\;,$

is the density scale height. We consider adiabatic motions such that

$\displaystyle\frac{\partial\delta P}{\partial t}=\frac{\partial
P_{1}}{\partial t}-g\rho_{0}w=c_{s}^{2}\left(\frac{\partial\rho_{1}}{\partial
t}+w\frac{d\rho_{0}}{dz}\right)$ (3)

where $\delta P$ is the Lagrangian pressure fluctuation and $c_{s}^{2}$ is the
square of the sound speed defined by

$\displaystyle c_{s}^{2}(z)=\frac{\gamma P_{0}}{\rho_{0}}$

with $\gamma$ as the ratio of specific heats.

##  3 The Governing Wave Equation

Since the background atmosphere is steady and invariant in the $\hat{x}$ and
$\hat{y}$ directions, we seek horizontal plane-wave solutions of the form

$f(x,y,z,t)=\tilde{f}(z)\,e^{i(k_{x}x+k_{y}y)}\,e^{-i\omega t}\;,$ (4)

where $f$ is any perturbed fluid variable. Furthermore, $k_{x}$ and $k_{y}$
are the wavenumbers in the $x$ and $y$ directions and $\omega$ is the temporal
frequency. Without loss of generality, we only consider positive longitudinal
wavenumbers, $k_{x}>0$. The waves propagate in the prograde direction if the
frequency is positive, $\omega>0$, and in the retrograde direction for
negative frequencies, $\omega<0$.

Using the above plane-wave form along with equations (1)–(3), we obtain the
following governing equation for $\delta P$

$\displaystyle\left\\{\frac{d^{2}}{dz^{2}}+\frac{1}{H}\frac{d}{dz}+\Lambda^{2}(z)\right\\}\delta
P=0\;,\qquad\qquad$ (5)
$\displaystyle\Lambda^{2}(z)\equiv\frac{\omega^{2}-4\Omega^{2}}{c_{s}^{2}}-k_{h}^{2}\left(1-\frac{N^{2}}{\omega^{2}}\right)+\frac{2\Omega
k_{x}}{\omega\cal H}+\frac{4k_{y}^{2}\Omega^{2}}{\omega^{2}}\;.$

In this Equation, $k_{h}$ is the total horizontal wavenumber, i.e.,
$k_{h}^{2}=k_{x}^{2}+k_{y}^{2}$. Further, the square of the buoyancy
frequency, $N^{2}$, is defined as follows

$\displaystyle N^{2}(z)={g}\left(\frac{1}{H}-\frac{g}{c_{s}^{2}}\right),$ (6)

and ${\cal H}$ is a scale height that depends on $H$ and $N^{2}$

$\displaystyle\frac{1}{\cal H}=\left(\frac{1}{H}-\frac{2N^{2}}{g}\right).$ (7)

The governing wave equation (5) can be written as a Helmholtz equation through
the substitution $\delta P=\sqrt{\rho_{0}}\,\Psi(z)$ ,

$\displaystyle\frac{d^{2}\Psi}{dz^{2}}+k_{z}^{2}\,\Psi=0,$ (8)

with

$\displaystyle
k_{z}^{2}=\frac{\omega^{2}-(\omega_{ac}^{2}+4\Omega^{2})}{c_{s}^{2}}-k_{h}^{2}\left(1-\frac{N^{2}}{\omega^{2}}\right)+\frac{2\Omega
k_{x}}{\omega\cal H}+\frac{4k_{y}^{2}\Omega^{2}}{\omega^{2}}.$ (9)

Here,

$\displaystyle\omega_{ac}^{2}\equiv\frac{c_{s}^{2}}{4H^{2}}\left(1-2\frac{dH}{dz}\right)$
(10)

is the square of the acoustic cut-off frequency, $\omega_{ac}$.

Equation (9) provides a local dispersion relation for both acoustic waves and
gravito-inertial waves. In the low-frequency limit, Equation (9) reduces to a
local dispersion relation for just gravito-inertial waves,

$\displaystyle k_{z}^{2}=\left[\frac{2\Omega k_{x}}{\omega{\cal
H}}+\frac{4k_{y}^{2}\Omega^{2}}{\omega^{2}}+k_{h}^{2}\frac{N^{2}}{\omega^{2}}\right]-\left(k_{h}^{2}+\frac{\omega^{2}_{ac}}{c_{s}^{2}}\right)\;.$
(11)

The two terms in the parentheses provide a negative contribution and lead to
vertical evanescence (i.e., $k_{z}^{2}<0$). Conversely, the terms inside the
square brackets can be positive, thereby leading to vertical propagation
($k_{z}^{2}>0$). The first two terms in the brackets arise from the Coriolis
force. As discussed in Hindman & Jain (2022), the first of these terms is
positive for prograde waves and can produce propagating thermal Rossby waves.
The second is a newly identified term that also leads to vertical propagation
and is responsible for the axisymmetric inertial waves in a sphere previously
studied by Guenther & Gilman (1985). As we will see in subsequent sections,
this term can in fact lead to vertical detrapping for waves of very low
frequency. Finally, the third term in the square brackets is the buoyancy term
responsible for internal gravity waves.

###  3.1 Neutrally Stable Atmosphere

Previous studies of latitudinal propagation in a compressible atmosphere have
been carried out by Thuburn et al. (2002) and Kasahara (2003); but these
efforts considered a stably-stratified isothermal atmosphere and investigated
the modifications to acoustic-gravity waves by the Coriolis force. Further,
since the atmosphere was isothermal, waves cannot be naturally trapped by the
stratification. Thus, it is important to explore the radial and latitudinal
propagation in an atmosphere where the density scale height varies with
height. Hindman & Jain (2022) demonstrated that inertial waves in a neutrally
stable convection zone can be trapped in the radial direction. In that paper
we considered waves that did not propagate latitudinally. Here we demonstrate
that radial trapping is still possible when latitudinal propagation is
allowed, but there is also a continuous spectrum of extremely low-frequency
inertial waves that are free to propagate to any depth.

We now consider an isentropic atmosphere such that $N^{2}=0$, i.e., the
buoyancy forces disappear and the Coriolis force is the only restoring force
for the low-frequency waves. Such a neutrally stable atmosphere is polytropic
and possesses a single height at which the pressure, density, and temperature
all vanish. We place the origin $z=0$ at this singular point and let the
atmosphere exist within the region below, for $z<0$. In an isentropic
polytrope the atmospheric profiles have the following power-law forms:

$\displaystyle\rho_{0}(z)$ $\displaystyle=$ $\displaystyle
A_{0}(-z)^{\alpha}\;,$ (12) $\displaystyle P_{0}(z)$ $\displaystyle=$
$\displaystyle\frac{gA_{0}}{\alpha+1}(-z)^{\alpha+1}\;,$ (13) $\displaystyle
c_{s}^{2}(z)$ $\displaystyle=$ $\displaystyle\frac{\gamma g}{\alpha+1}(-z)\;,$
(14) $\displaystyle H(z)={\cal H}(z)$ $\displaystyle=$
$\displaystyle\frac{(-z)}{\alpha}\;,$ (15)
$\displaystyle\frac{\omega_{ac}^{2}(z)}{c_{s}^{2}(z)}$ $\displaystyle=$
$\displaystyle\frac{\alpha(\alpha+2)}{z^{2}}$ (16)

where $A_{0}$ is an arbitrary scale factor and the dimensionless parameter
$\alpha$ is the polytropic index given by $\alpha=\left(\gamma-1\right)^{-1}$.

###  3.2 Propagation Diagram and Eigenmodes

For an isentropic stratification, where $N^{2}=0$, the low-frequency form of
the local dispersion relation (11) reduces to

$\displaystyle k_{z}^{2}=\left[\frac{2\alpha\Omega
k_{x}}{\omega(-z)}+4k_{y}^{2}\frac{\Omega^{2}}{\omega^{2}}\right]-\left(k_{h}^{2}+\frac{\alpha(\alpha+2)}{4z^{2}}\right)\;.$
(17)

Near the upper boundary of the atmosphere ($z\to 0$), the second term in the
parentheses (arising from the acoustic cut-off frequency) is large and leads
to reflection and vertical evanescence. Deep within the atmosphere
($z\to-\infty$), the dispersion relation reduces to

$\displaystyle k_{z}^{2}\approx k_{h}^{2}\left(\varpi^{2}-1\right)\;,$ (18)

where, for later convenience, we have defined

$\displaystyle\varpi^{2}\equiv\frac{k_{y}^{2}}{k_{h}^{2}}\frac{4\Omega^{2}}{\omega^{2}}=\frac{4\Omega^{2}}{\omega^{2}}\sin^{2}\chi\;.$
(19)

Here, $\chi$ indicates the direction of horizontal propagation, with $\chi=0$
corresponding to pure prograde propagation and $\chi=\pi/2$ indicating pure
northward propagation ($k_{x}=k_{h}\cos\chi$ and $k_{y}=k_{h}\sin\chi$). From
Equation (18), we can easily determine that the inertial waves can be either
vertically evanescent ($k_{z}^{2}<0$) or vertically propagating
($k_{z}^{2}>0$) depending on the frequency and the horizontal direction of
propagation. A wave cavity exists when the waves become evanescent, which
requires $\varpi^{2}<1$ or equivalently,

$\displaystyle\omega>2\frac{|k_{y}|}{k_{h}}\Omega=2\Omega|\sin\chi|\;.$ (20)

These waves are naturally trapped by the density stratification and form a
discrete spectrum of inertial eigenmodes that in the limit of $k_{y}=0$ become
the thermal Rossby waves of Hindman & Jain (2022) and the fast branch of
thermal Rossby waves as discussed by Hindman & Jain (2023). For frequencies
lower than the critical value given in Equation (20), the waves remain
vertically propagating to all depths and a downward propagating inertial wave
is never reflected back upwards in a semi-infinite domain: no cavity exists.
This family of solution corresponds to a continuous spectrum of untrapped
inertial eigenmodes.

This behavior is fully revealed in Figure 1 which provides a propagation
diagram as a function of height within an isentropic atmosphere. The lightly
shaded region indicates those frequencies that correspond to vertically
propagating waves. The green horizontal lines indicate the critical value (and
its negative) provided by Equation (20). The frequencies above the upper bound
correspond to trapped inertial waves, while frequencies between the two bounds
constitute the untrapped modes. As one can see, the trapped waves propagate
between two turning points and therefore form a discrete spectrum of normal
modes. Conversely, the untrapped inertial waves are screened from the origin
by an upper turning point, but lack a lower turning point. The trapped waves
all possess positive frequencies, and are hence prograde-propagating, whereas
the untrapped continuum modes can be prograde thermal Rossby waves or
retrograde inertial waves.

###  3.3 Vertical Wave Equation for an Isentropic Stratification

For the polytropic atmosphere, the Helmholtz equation with $k_{z}^{2}$ given
by Equation (17) can be transformed into the well-known Whittaker Equation by
making a change of variable $\zeta=-2\sqrt{1-\varpi^{2}}\,k_{h}z$,

$\displaystyle\frac{d^{2}\Psi}{d\zeta^{2}}+\left[\frac{\kappa}{\zeta}-\frac{1}{4}+\frac{1/4-\mu^{2}}{\zeta^{2}}\right]\Psi=0$
(21)

where $\mu\equiv(\alpha+1)/2$ is a constant. The parameter $\kappa$ is the
eigenvalue of the second-order ordinary differential equation (21) and it
depends on the frequency and direction of propagation of the wave, i.e.

$\displaystyle\kappa=\frac{\alpha\Omega}{\omega}\frac{\cos\chi}{\sqrt{1-\varpi^{2}}}\;.$
(22)

Notice, that for all of the trapped prograde waves (refer to Figure 2) we have
$\varpi^{2}<1$; hence, for these trapped waves, the parameter $\kappa$ and the
dimensionless depth $\zeta$ have real values. Conversely, the low-frequency
waves without a lower turning point have $\varpi^{2}>1$ and both $\kappa$ and
the dimensionless depth $\zeta$ are purely imaginary.

Whittaker’s Equation (21) has two solutions, the Whittaker functions
$M_{\kappa\mu}(\zeta)$ and $W_{\kappa\mu}(\zeta)$ (see Abramowitz & Stegun,
1968). These can be expressed with Kummer’s confluent hypergeometric functions
$M$ and $U$ as follows:

$\displaystyle
M_{\kappa\mu}(\zeta)=e^{-\zeta/2}\zeta^{\mu+\frac{1}{2}}M\left(-\eta,1+2\mu,\zeta\right)$
$\displaystyle
W_{\kappa\mu}(\zeta)=e^{-\zeta/2}\zeta^{\mu+\frac{1}{2}}U(-\eta,1+2\mu,\zeta)$

with

$\displaystyle\eta\equiv\kappa-\left(\mu+\frac{1}{2}\right)\;.$ (23)

##  4 Semi-Infinite Domain

In this section we consider an atmosphere of semi-infinite extent in radius.
The atmosphere has a physical upper surface (at $z=0$), but is infinitely deep
($z\to-\infty$). Such an atmosphere is valid to use if the waves are naturally
trapped and are confined over a region of finite radius. However, here we
primarily explore the eigenmodes for such an atmosphere because their
mathematics provides illumination for the behavior of waves in finite domains.

###  4.1 Naturally Trapped Modes in Radius

The radially trapped modes are acquired by demanding that the solutions vanish
and remain regular at the two singular points of Whittaker’s Equation (21),
$\zeta=0$ and $\zeta\rightarrow\infty$,

$\displaystyle\delta
P(z)=\rho_{0}^{1/2}\Psi(z)=C_{n}\,\zeta\,e^{-\zeta/2}\,M(-n,\alpha+2,\zeta)\;,$
(24)

where $C_{n}$ is an arbitrary constant and the parameter $\eta$ must be a non-
negative integer, $\eta=n\in[0,1,2,3,...]$, in order to avoid divergence of
the eigenfunction in the limit $z\to-\infty$. Therefore, the eigenvalue
$\kappa$ takes on discrete values that depend on the radial order $n$ and the
polytropic index $\alpha$,

$\displaystyle\kappa_{n}=n+1+\frac{\alpha}{2}\;.$ (25)

Since the frequency depends on the eigenvalue through Equation (22), the
discretization of the eigenvalue leads to discretization of the corresponding
frequencies. These eigenfrequencies depend on the direction of propagation
$\chi$, the radial order $n$, the rotation rate $\Omega$, and the polytropic
index $\alpha$,

$\displaystyle\frac{\omega_{n}}{\Omega}=\left[\frac{\alpha^{2}\cos^{2}\chi}{(n+1+\alpha/2)^{2}}+4\sin^{2}\chi\right]^{1/2}\;.$
(26)

The blue dotted horizontal lines in Figure 1 indicate these discrete
eigenfrequencies. There are a countable infinity of such modes with an
accumulation point at the critical frequency, i.e., $\omega\to
2\Omega\left|\sin\chi\right|$ as $n\to\infty$.

The expression for the eigenfrequency, $\omega_{n}$, in Equation (26) clearly
shows that the eigenfrequencies are unaffected by the magnitude of the
horizontal wavenumber, $k_{h}$, even though they do depend on the direction of
propagation $\chi$. This is a consequence of the self-similarity of a
polytropic atmosphere (for details, see Hindman & Jain, 2022). For purely
northward or southward propagation, i.e., $\chi=\pm\pi/2$, the
eigenfrequencies become independent of the radial order $n$ and on the
stratification $\alpha$ taking on the value $\omega_{n}=2\Omega$. We only
illustrate positive values of $\chi$ because the eigenfrequencies are
symmetric in $\chi$, i.e., $\omega_{n}(-\chi)=\omega_{n}(\chi)$.

The lower turning point can be calculated directly from Equation (17) by
solving for the two heights in the atmosphere where $k_{z}^{2}=0$. The deepest
of these two solutions corresponds to the lower turning point,

$\displaystyle-k_{h}z_{\rm
turn}=\frac{\kappa_{n}+\sqrt{\kappa_{n}^{2}-\alpha(\alpha+2)/4}}{\sqrt{1-\varpi^{2}}}.$
(27)

Note that the lower turning point for all radial orders deepens as the
propagation angle increases from zero. Hence, the cavity is shallowest for
purely zonal propagation ($\varpi^{2}=0$) and becomes infinitely deep for
purely latitudinal propagation ($\varpi^{2}=1$). Below the lower turning
point, the eigenfunction becomes evanescent and decays exponentially with
depth. This behavior is illustrated in Figure 2. The red curve illustrates the
eigenfunction for a $n=3$ mode with a lower turning point near $k_{h}z=-5$.

###  4.2 Untrapped Modes in Radius

Waves with frequencies that lie with the band

$\displaystyle|\omega|<2\Omega|\sin\chi|\;,$ (28)

do not possess a lower turning point. Hence, these waves continue to propagate
deep in the atmosphere. This behavior leads to a continuous spectrum of wave
modes that are regular at the origin, $z=0$, and oscillatory in the limit
$z\to-\infty$. The regularity condition at the origin is the only boundary
condition that we can physically enforce. This leads to eigenfunctions of the
form,

$\displaystyle\delta
P=C(\omega)\,\zeta\,e^{-\zeta/2}M\left(\mu+1/2-\kappa,\alpha+2,\zeta\right)\;,$
(29)

where $C(\omega)$ is an amplitude determined by initial conditions. The
dimensionless depth $\zeta$ and the eigenvalue $\kappa$ are both imaginary and
$\kappa$ is no longer discrete,

$\displaystyle\kappa=-i\frac{\alpha\Omega}{\omega}\frac{\cos\chi}{\sqrt{\varpi^{2}-1}}\;,$
(30) $\displaystyle\zeta=-2i\sqrt{\varpi^{2}-1}\,k_{h}z\;.$ (31)

The eigenfunction given by Equation (29) is a standing wave that is comprised
of a wave launched from infinite depth that travels upwards, reflects off the
acoustic cut-off frequency near the origin, and then travels downwards back to
infinite depth. Such an eigenfunction is illustrated in Figure 2, where the
blue curve shows a continuum mode that lacks a lower turning point. The wave
remains oscillatory in the limit $z\to-\infty$.

For special integer values of the polytropic index, $\alpha=2L$, where $L$ is
any nonnegative integer, solutions for the untrapped continuum modes can be
written in terms of real functions with real parameters and arguments using
the regular Coulomb wave function $F_{L}$ (see Abramowitz & Stegun, 1968),

$\delta P=C(\omega)\,z^{\alpha/2}F_{L}\left(q,{\cal Z}\right)\;,$ (32)

with

$\displaystyle q$ $\displaystyle\equiv$
$\displaystyle-i\kappa=-\frac{\alpha\Omega}{\omega}\frac{\cos\chi}{\sqrt{\varpi^{2}-1}}\;,$
(33) $\displaystyle{\cal Z}$ $\displaystyle\equiv$
$\displaystyle-\frac{i}{2}\zeta=-\sqrt{\varpi^{2}-1}\,k_{h}z\;.$ (34)

##  5 Finite Domain

###  5.1 Radial Trapping

In a solar-like star, the convection zone is approximately 200 Mm deep with
the stably-stratified radiative zone lying underneath. Within the transition
between the two layers, the buoyancy frequency jumps dramatically and this
large change should make the bottom of the convection zone an efficient
reflector of gravito-inertial waves. Therefore, an appropriate model of
inertial waves in a star’s convection zone is to apply a regularity condition
at the origin (as we did in the previous section) and a reflective lower
boundary condition at a finite depth of $D$. For this later condition we adopt
$\delta P(z=-D)=0$ for $D=200$ Mm. The global dispersion relation satisfying
these boundary conditions is

$\displaystyle
M\left(\mu+1/2-\kappa,\,\alpha+2,\,2\sqrt{1-\varpi^{2}}k_{h}D\right)=0\;.$
(35)

We note that the imposition of a lower boundary condition at a finite depth
converts the continuous spectrum of untrapped waves into a discrete spectrum.
In Figure 3, the dimensionless frequency $\omega/\Omega$ is plotted as a
function of $k_{h}R$ for four different angles of propagation. Note that
$k_{h}R$ corresponds to the harmonic degree of spherical harmonics in a
spherical geometry. In each panel, the uppermost curve with the highest
positive frequencies corresponds to modes that lack nodes in radius (i.e.,
radial order $n=0$). The second highest indicates modes with one radial node
($n=1$). Sequentially lower curves have one additional node and have an
accumulation point at zero frequency for an infinite number of nodes. All of
these modes have positive frequencies and are prograde propagating. The curve
with the largest negative frequency also lacks radial nodes ($n=0$), but
corresponds to retrograde-propagating inertial waves. Each subsequent curve
with smaller negative frequencies has an additional node with an accumulation
point at zero frequency as well. The horizontal green lines indicate the
frequency bounds that separate the modes that are naturally trapped by the
stratification ($\omega>2\Omega\left|\sin\chi\right|$) from those that would
have been untrapped but are now trapped by the lower boundary of the finite
domain.

###  5.2 Latitudinal Trapping

Since we have derived the inertial waves within an equatorial f-plane model
instead of spherical geometry, we have implicitly assumed that the waves are
confined near the equator and have short horizontal wavelengths, i.e.,
$k_{h}R\gg 1$ where $R$ is the star’s photospheric radius. The first of these
assumptions allows us to ignore the curvature terms in the fluid equations
that arise from the spherical geometry. The second can be justified by
examining the results of numerical simulations and eigenmode calculations in
spherical geometry (i.e., Jones et al., 2009; Hindman et al., 2020; Bekki et
al., 2022), which clearly indicate that thermal Rossby waves are indeed
confined or trapped near the equator.

The traditional way to capture latitudinal trapping in plane-parallel geometry
is to adopt an equatorial $\beta$-plane approximation, where all atmospheric
and geometric terms in the fluid equations are linearized with respect to the
latitudinal coordinate (i.e., one assumes $y/R\ll 1$). We will not do so here
to avoid the resulting complication of solving the wave equations in a truly
2D atmosphere. Instead, we will retain our f-plane geometry but make the
simple assumption that the waves are confined within a latitudinal band that
extends north and south of the equator by a fixed distance $L/2$. To enforce
reflection at $y=\pm L/2$, we impose Neumann boundary conditions on the
Lagrangian pressure fluctuation,

$\left.\frac{\partial\delta P}{\partial y}\right|_{y=\pm L/2}=0\;,$ (36)

which is equivalent to an impenetrable boundary condition ($v=0$). Such
boundary conditions are quite similar to those employed in the study of Rossby
waves in astrophysical disks (Lin, 2012).

To simplify comparison with waves in a spherical geometry, we will assume that
the longitudinal direction is periodic which quantizes the longitudinal
wavenumber, $k_{x}=m/R$, with $m$ being the azimuthal order of the concomitant
spherical harmonic. Thus, our spatial domain is shaped like a millstone, with
an outer annular radius of $R$, an inner radius of $R-D$, and a cylindrical
height of $L$ (see Figure 4). In numerical simulations, the latitudinal extent
of thermal Rossby wave eigenfunction varies as a function of horizontal
wavenumber (i.e., Hindman et al., 2020). But, in general, the waves often fill
the region outside the cylinder that is tangent to the base of the convection
zone at the equator. Hence, we choose the width of the latitudinal band $L$ to
be the length of the chord that is tangent to the bottom of the star’s
convection zone (see Figure 4),

$L=2\sqrt{R^{2}-(R-D)^{2}}\;.$ (37)

Using $R=700$ Mm and $D=200$ Mm, one obtains $L\approx 980$ Mm.

The latitudinal boundary conditions discretizes the latitudinal wavenumber,

$k_{y}=\frac{\lambda\pi}{L}\;,\quad\lambda=\left[0,1,2,3,...\right]\;,$ (38)

leading to eigenfunctions of the form,

$\delta P(x,y,z,t)=C\cos\left[\lambda(y-L/2)\right]\,e^{imx/R}\,\zeta
e^{-\zeta/2}\,M(-\eta,\alpha+2,\zeta)\,e^{-i\omega t}\;.$ (39)

The quantum number $\lambda$ can be any non-negative integer, with the value
of $\lambda$ indicating the number of latitudinal nodes that appear in the
eigenfunction. Modes with $\lambda=0$ correspond to $k_{y}=0$ and we have
explored these modes previously in Hindman & Jain (2022). We recognize that
our choice of $L$ results in large latitudinal wavelengths for low latitudinal
orders (small $\lambda)$; hence, they break the short wavelength
approximation. Our goal, however, is not to derive accurate quantitative
frequencies but to instead generate a general qualitative understanding of the
wave behavior. Hence, we carry on nonetheless.

The eigenfrequencies for this “millstone” model can be generated by solving
Equation (39) numerically, for the discrete values of $k_{x}$ and $k_{y}$ that
we just discussed. Figure 5 presents the results. The eigenfrequencies are
plotted as a function of azimuthal order $m$ for the five lowest latitudinal
orders. For clarity of presentation, only the radial fundamental modes,
lacking radial nodes ($n=0$), are illustrated. The solid curves show the
prograde-propagating thermal Rossby waves and the dashed lines correspond to
the retrograde-propagating inertial waves. The color of the curve indicates
the latitudinal order. The reader should note that for the $\lambda=0$ mode,
which propagates purely zonally ($k_{y}=0$), the retrograde solution is
missing because it becomes a zero-frequency geostrophic mode (Hindman & Jain,
2022). The dotted curves show the frequency for which the lower turning point
passes through the bottom boundary on the convection zone. Every frequency,
for the appropriate value of $\lambda$, that lies above the dotted curve
corresponds to a mode with two turning points in the radial domain; hence,
such a mode has a cavity that only partially fills the domain. Those
frequencies that lie below the dotted line correspond to modes with only one
turning point in the radial domain and that are trapped through reflection off
the lower boundary. Initially, the mode frequency increases as the azimuthal
order increases until the lower turning point crosses into the domain. At
larger azimuthal orders, the turning point continues to move upwards and the
depth of the wave cavity shrinks commensurately. In response, the mode
frequency asymptotes to a constant value that is independent of the lower
boundary condition. For the prograde thermal Rossby waves, the asymptotic
value can be obtained from the dispersion relation that applies for the semi-
infinite domain, Equation (26),

$\lim_{m\to\infty}\omega_{n}=\frac{2\alpha\Omega}{2n+\alpha+2}$ (40)

For the retrograde inertial waves, the asymptotic value is zero because the
angle of propagation $\chi$ approaches zero as the azimuthal order becomes
large, with the result that the retrograde wave cavity shrinks to zero
frequency (see Figure 1).

##  6 Conclusion

We have carried out a linear wave analysis for a compressible and stratified
atmosphere representing a stellar convection zone rotating at a constant rate.
The rotation axis is assumed to be perpendicular to the direction of
stratification. By adopting an f-plane approximation, we derive and solve
dispersion relations for waves propagating through a neutrally-stable
polytropic atmosphere in all three spatial directions: zonal, latitudinal and
radial.

The density stratification enables radial trapping of prograde-propagating
waves with frequencies above a threshold frequency—see Equation (28). Low-
frequency waves with frequencies below the threshold (both prograde and
retrograde) cannot be trapped by an isentropic density stratification.
However, the waves can reflect off of strong gradients in the buoyancy
frequency (as occurs at the base of the convection zone) and thereby become
radially trapped. If we consider the bottom of the convection zone to be
perfectly reflective, we obtain the eigenfrequencies illustrated in Figure 3.
If we further place impenetrable latitudinal boundaries, as we did for our
millstone shaped domain (see Figure 4), we obtain the eigenfrequencies shown
in Figure 5. As expected, the eigenfrequencies generally increase as the
horizontal wavenumber increases and decrease as the radial wavenumber (or
radial order) increases. In particular, we point to the shape of the
dispersion curves that appear in Figure 5, as these suggest that all of the
latitudinal overtones should have frequencies that initially rise as the
azimuthal order increases and eventually asymptote to a common value.

While observations have yet to directly detect thermal Rossby waves, numerical
simulations have long evinced such waves. For many years now thermal Rossby
waves—in their unstable, nonlinear form—have appeared as “banana cells” or
“Busse columns.” More recently, stable long-wavelength thermal Rossby waves
have been identified as well (Bekki et al., 2022). However, only the radial
and latitudinal fundamental have been reported. The first latitudinal overtone
was actually the first thermal Rossby wave to be discussed in the literature.
In a linear stability analysis, Roberts (1968) calculated the $\lambda=1$
thermal Rossby wave and demonstrated that this sort of wave represents the
convective modes in a rotating system at convective onset. However, this
antisymmetric mode turns out to be less unstable than the sectoral mode with
$\lambda=0$ and, hence, the nonlinear convective cells that appear in
numerical simulations of thermal convection in a spherical shell usually
possess rough symmetry across the equator. Our calculation here may suggest
the form of previously undetected tesseral modes that, due to being stable and
hence low amplitude, have been skulking around in numerical simulations for
many years. If observationally detected these modes can serve as seismic
probes for specific entropy density.

RJ would like to thank MSRC (University of Sheffield) for partial support. BWH
is supported by NASA through grants 80NSSC18K1125, 80NSSC19K0267,
80NSSC20K0193 and acknowledges collaboration with the COFFIES DSC.

##  Appendix A Nomenclature

There is often confusion concerning the names that are applied to the
different types of gravito-inertial waves, particularly thermal Rossby waves.
Beyond the fact that thermal Rossby waves have been called by a myriad of
names (e.g., low-frequency prograde waves, columnar convective modes,
overstable convective modes), part of the confusion arises because all of
these waves are in some sense related to each other and can transition from
one type of wave to another as various parameters vanish or become large.
Thermal Rossby waves are distinct from classical Rossby waves only through
geometry. The restoring force is essentially the same, arising from the
conservation of potential vorticity (or equivalently angular momentum).
Classical Rossby waves concern vortical motions that are largely horizontal,
either because the fluid layer is thin (such as the Earth’s atmosphere) or the
stratification is extremely stable, thus discouraging vertical motions. The
conservation principle therefore operates on 2D spherical surfaces and this
leads to retrograde propagation. Thermal Rossby waves usually reside in thick
atmospheres where fluid elements are free to move vertically without
inhibition. In fact, in an unstable stratification like that found in a
convection zone, such motions are reinforced. Without vertical constraint, the
vortex columns align instead with the rotation axis and prograde-propagating
waves are produced by conservation of potential vorticity in this rotationally
aligned geometry.

Even if we restrict our attention to only zonally-propagating waves in the
axially constrained geometry, there are two distinct wave modes (see Hindman &
Jain, 2022, 2023). In a stable stratification, with only weak rotation
influence, the two gravito-inertial wave solutions consist of the retrograde
and prograde branches of the internal gravity waves. However, in an atmosphere
of neutral stability, the two solutions correspond to pure inertial waves. The
prograde branch now transitions to thermal Rossby waves and the retrograde
branch has moved to zero frequency, becoming a stationary geostrophic mode.
Finally, in a weakly unstable stratification, both branches become prograde.
The branch with the faster zonal phase speed is easily identified as thermal
Rossby waves while the slow branch has been inconsistently named. Busse (1986)
called this branch the thermal mode while Hindman & Jain (2023) called them
the slow thermal Rossby wave branch.

Here we also consider propagation latitudinally and this complicates the
naming scheme further. As we stated previously, for zonal propagation in an
isentropic atmosphere, the two solution branches are prograde thermal Rossby
waves and zero-frequency geostrophic modes. When the waves are allowed to
propagate obliquely to the equator, the prograde branch remains a prograde
inertial wave that is firmly a thermal Rossby wave. The zero-frequency branch
becomes retrograde and we choose to call it a retrograde inertial wave for
lack of better choice.

Oblique propagation through a nonadiabatic stratification leads to an even
further loss of clarity. The local dispersion relation reveals why,

$\displaystyle k_{z}^{2}=\left[\frac{2\Omega k_{x}}{\omega{\cal
H}}+\frac{4k_{y}^{2}\Omega^{2}}{\omega^{2}}+k_{h}^{2}\frac{N^{2}}{\omega^{2}}\right]-\left(k_{h}^{2}+\frac{\omega^{2}_{ac}}{c_{s}^{2}}\right)\;.$
(A1)

There are three types of restoring forces that lead to propagation.
Stratification coupled with zonal propagation leads to the first term in the
square brackets. This term is positive only for waves with a prograde phase
speed. This term provides a compressional $\beta$-effect and leads to thermal
Rossby waves. The second term is always positive and leads to inertial waves
that propagate latitudinally. The third term arises from buoyancy and leads to
internal gravity waves. Generally, however, more than one of these terms will
be in operation and the wave is a three-way hybrid of internal gravity waves
and the two types of inertial waves. An obvious naming scheme becomes apparent
only when one, or possibly two, of the terms dominate.

## References

  * Abramowitz & Stegun (1968) Abramowitz, M., & Stegun, I. A. 1968, Handbook of mathematical functions with formulas, graphs and mathematical tables
  * Bekki et al. (2022) Bekki, Y., Cameron, R. H., & Gizon, L. 2022, A&A, 666, A135, doi: 10.1051/0004-6361/202244150
  * Brun et al. (2011) Brun, A. S., Miesch, M. S., & Toomre, J. 2011, ApJ, 742, 79, doi: 10.1088/0004-637X/742/2/79
  * Busse (1970) Busse, F. H. 1970, Journal of Fluid Mechanics, 44, 441, doi: 10.1017/S0022112070001921
  * Busse (1986) —. 1986, Journal of Fluid Mechanics, 173, 545, doi: 10.1017/S002211208600126X
  * Busse & Hood (1982) Busse, F. H., & Hood, L. L. 1982, Geophysical and Astrophysical Fluid Dynamics, 21, 59, doi: 10.1080/03091928208209005
  * Christensen-Dalsgaard (2002) Christensen-Dalsgaard, J. 2002, Reviews of Modern Physics, 74, 1073, doi: 10.1103/RevModPhys.74.1073
  * Dikpati & McIntosh (2020) Dikpati, M., & McIntosh, S. W. 2020, Space Weather, 18, e02109, doi: 10.1029/2019SW002109
  * Gilman (1987) Gilman, P. A. 1987, ApJ, 318, 904, doi: 10.1086/165422
  * Gizon et al. (2021) Gizon, L., Cameron, R. H., Bekki, Y., et al. 2021, A&A, 652, L6, doi: 10.1051/0004-6361/202141462
  * Glatzmaier & Gilman (1981) Glatzmaier, G. A., & Gilman, P. A. 1981, ApJS, 47, 103, doi: 10.1086/190753
  * Guenther & Gilman (1985) Guenther, D. B., & Gilman, P. A. 1985, ApJ, 295, 195, doi: 10.1086/163365
  * Hanasoge & Mandal (2019) Hanasoge, S., & Mandal, K. 2019, ApJ, 871, L32, doi: 10.3847/2041-8213/aaff60
  * Hanson et al. (2022) Hanson, C. S., Hanasoge, S., & Sreenivasan, K. R. 2022, Nature Astronomy, 6, 708, doi: 10.1038/s41550-022-01632-z
  * Hathaway & Upton (2021) Hathaway, D. H., & Upton, L. A. 2021, ApJ, 908, 160, doi: 10.3847/1538-4357/abcbfa
  * Hindman et al. (2020) Hindman, B. W., Featherstone, N. A., & Julien, K. 2020, ApJ, 898, 120, doi: 10.3847/1538-4357/ab9ec2
  * Hindman & Jain (2022) Hindman, B. W., & Jain, R. 2022, ApJ, 932, 68, doi: 10.3847/1538-4357/ac6d64
  * Hindman & Jain (2023) —. 2023, ApJ, 943, 127, doi: 10.3847/1538-4357/acaec4
  * Jones et al. (2009) Jones, C. A., Kuzanyan, K. M., & Mitchell, R. H. 2009, Journal of Fluid Mechanics, 634, 291, doi: 10.1017/S0022112009007253
  * Kasahara (2003) Kasahara, A. 2003, Journal of Atmospheric Sciences, 60, 1085, doi: 10.1175/1520-0469(2003)60<1085:TROTHC>2.0.CO;2
  * Li et al. (2000) Li, H., Finn, J. M., Lovelace, R. V. E., & Colgate, S. A. 2000, ApJ, 533, 1023, doi: 10.1086/308693
  * Lin (2012) Lin, M.-K. 2012, ApJ, 754, 21, doi: 10.1088/0004-637X/754/1/21
  * Lonner et al. (2022) Lonner, T. L., Aggarwal, A., & Aurnou, J. M. 2022, Journal of Geophysical Research (Planets), 127, e2022JE007356, doi: 10.1029/2022JE007356
  * Löptien et al. (2018) Löptien, B., Gizon, L., Birch, A. C., et al. 2018, Nature Astronomy, 2, 568, doi: 10.1038/s41550-018-0460-x
  * Mason (1975) Mason, P. J. 1975, Philosophical Transactions of the Royal Society of London Series A, 278, 397, doi: 10.1098/rsta.1975.0032
  * Miesch et al. (2000) Miesch, M. S., Elliott, J. R., Toomre, J., et al. 2000, ApJ, 532, 593, doi: 10.1086/308555
  * Proxauf et al. (2020) Proxauf, B., Gizon, L., Löptien, B., et al. 2020, A&A, 634, A44, doi: 10.1051/0004-6361/201937007
  * Roberts (1968) Roberts, P. H. 1968, Philosophical Transactions of the Royal Society of London Series A, 263, 93, doi: 10.1098/rsta.1968.0007
  * Smith et al. (2014) Smith, C. A., Speer, K. G., & Griffiths, R. W. 2014, Journal of Physical Oceanography, 44, 2273, doi: 10.1175/JPO-D-13-0255.1
  * Thuburn et al. (2002) Thuburn, J., Wood, N., & Staniforth, A. 2002, Quarterly Journal of the Royal Meteorological Society, 128, 1793, doi: 10.1256/003590002320603412

Figure 1: Propagation diagram for a neutrally-stable polytropic atmosphere in
a semi-infinite domain. The positive (negative) frequencies indicate waves
propagating in prograde (retrograde) direction in the rotating frame. The
lightly-shaded region indicates the frequencies of waves that are radially
propagating ($k_{z}^{2}>0$) at the indicated height in the atmosphere. The
purple curves denote those frequencies and radii for which a radial turning
point occurs (i.e., $k_{z}^{2}=0$). Thus, at a given frequency, an inertial
wave cavity exists in the shaded region between two turning points. Such a
cavity for thermal Rossby waves is formed purely by the stratification. We
have indicated the eigenfrequencies for such modes for a semi-infinite
atmosphere using the blue horizontal dotted lines. The green horizontal lines
denote the critical frequencies—see Equation (19)—which separate the low-
frequency waves that lack a lower turning point from the higher frequency
waves that have two turning points (an upper and a lower turning point).
Figure 2: Illustration of two types of eigenfunction. The two curves show the
normalized Lagrangian pressure fluctuation as a function of dimensionless
height for a naturally trapped (red curve) and an untrapped thermal Rossby
wave (blue curve). The trapped mode corresponds to a radial order of $n=3$ and
has two turning points in radius: an upper turning point that lies very near
the upper surface and a lower turning point near $k_{h}z\approx=-5$. The wave
cavity exists between these two turning points and the eigenfunction decays
exponentially with depth below the lower turning point. The untrapped wave has
only a single turning point. It corresponds to a continuum mode that is
launched from infinite depth, travels upwards, reflects off the upper turning
point near the origin, and then travels back downwards. This specific
continuum mode is prograde with a frequency $\omega/\Omega=0.5$. Both
eigenfunctions are computed for an angle of propagation of $\chi=45^{\circ}$.
Figure 3: Dimensionless eigenfrequencies as a function of harmonic degree
$k_{h}R$ for a neutrally-stable polytropic atmosphere with a finite depth of
$D=200$ Mm. The green solid line is the same as in Figure 1. For waves with
latitudinal propagation ($\chi>0$), the uppermost and lowermost curves are for
$n=0$ modes lacking radial nodes in their pressure eigenfunctions. Each
subsequent radial overtone ($n=1$, $n=2$, and so forth) lies slightly closer
to zero frequency. Thus, there is a retrograde and prograde branch for each
radial overtone. The solid cone of blue color is artificial and arises because
the curves become so close together that they cannot be resolved in the image.
The left-most panel, which corresponds to pure zonal propagation, lacks the
retrograde lower curves altogether. Figure 4: Sketch showing the finite domain
considered in Section 5.2. The outer surface of a star and the bottom of the
convection zone are indicated in the meridional plane as orange circles. The
waves are confined within the convection zone of depth $D$, and within a
latitudinal band of length $L$ which is the length of the chord that is
tangent to the bottom of the star’s convection zone. The domain is periodic in
the azimuthal direction with a circumference length of $2\pi R$. A meridional
slice through the domain is shown as the gray box. Figure 5: Eigenfrequencies
of modes trapped latitudinally and radially in the millstone domain illustrate
in Figure 4. The frequencies are plotted as a function of the azimuthal order.
Each distinct color corresponds to a different latitudinal order $\lambda$, as
indicated in the legend. The solid curves show the prograde-propagating
thermal Rossby waves, whereas the dashed curves are for the retrograde
inertial waves. The dotted curves indicate the frequency where the lower
turning point passes through the bottom boundary. Wave frequencies above the
appropriate dotted curve have two turning points in the radial domain and have
a wave cavity that is screened from the lower boundary condition. Frequencies
below the dotted curve possess only an upper turning point in the domain and
are reflected off the lower boundary.