* [9] T. Chang and and T. Skjelbred, _The topological Schur lemma and related results_ , Ann. Math. 100 (1974), 307–321.
  * [10] M. Crainic, _On the perturbation lemma, and deformations_ , preprint (2004), available at arXiv:math/0403266 [math.AT].
  * [11] R. N. Diethorn, _Generators of Koszul homology with coefficients in a $J$-closed module_, J. Pure Appl. Algebra 224(10) (2020).
  * [12] D. Eisenbud, _Homological algebra on a complete intersection, with an application to group representations_ , Trans. Amer. Math. Soc. 260(1) (1980), 35--64.
  * [13] M. Franz, _Homotopy Gerstenhaber formality of Davis--Januszkiewicz spaces_ , Homol. Homotopy Appl. 23(2) (2021), 325--347.
  * [14] M. Franz, _The cohomology rings of smooth toric varieties and quotients of moment-angle complexes_ , Geom. Topol. 25(4) (2021), 2109--2144.
  * [15] M. Franz and X. Fu, _Cohomology of smooth toric varieties: naturality_ , preprint (2021), available at arXiv:2104.03825 [math.AT].
  * [16] M. Franz and V. Puppe, _Exact sequences for equivariantly formal spaces_ , C. R. Math. Acad. Sci. Soc. R. Can. 33(1) (2011), 1--10.
  * [17] X. Fu, _Homotopy types of partial quotients for a certain case_ , preprint (2019), available at arXiv:1912.03574 [math.AT].
  * [18] M. Goresky, R. Kottwitz and R. MacPherson, _Equivariant cohomology, Koszul duality, and the localization theorem_ , Invent. Math. 131 (1997), 25--83.
  * [19] J. Grbić, T. E. Panov, S. Theriault and J. Wu, _The homotopy types of moment-angle complexes for flag complexes_ , Trans. Amer. Math. Soc. 368 (2016), 6663--6682.
  * [20] D. R. Grayson and M. E. Stillman, _Macaulay2, a software system for research in algebraic geometry_ , available at https://math.uiuc.edu/Macaulay2/.
  * [21] V. K. A. M. Gugenheim, _On the chain complex of a fibration_ , Illinois J. Math. 16(3) (1972), 398--414.
  * [22] A. Hattori, _Spectral sequence in the de Rham cohomology of fibre bundles_ , J. Fac. Sci. Univ. Tokyo Sect. 1 8 (1960), 289--331.
  * [23] M. Hochster, _Cohen--Macaulay rings, combinatorics, and simplicial complexes_ , Ring Theory II (Proc. Second Conf., Univ. Oklahoma, Norman, Okla., 1975), pp. 171--223, Lecture Notes in Pure and Appl. Math., vol. 26, Dekker, NY, (1977).
  * [24] L. Katthän, _Linear maps in minimal free resolutions of Stanley--Reisner rings_ , Mathematics 7(7), 605 (2019).
  * [25] Y. Li, _On Hochster’s formula for a class of quotient spaces of moment-angle complexes_ , Osaka J. Math. 56 (2019), 33--50.
  * [26] I. Y. Limonchenko and G. D. Solomadin, _On the homotopy decomposition for the quotient of a moment-angle complex and its applications_ , Proc. Steklov Inst. Math. 317 (2022), 117--140.
  * [27] J.-L. Loday and B. Vallette, _Algebraic Operads_ , Grundlehren der mathematischen Wissenschaften, vol. 346. Springer Berlin, Heidelberg, (2012).
  * [28] S. Luo, T. Matsumura and W. F. Moore, _Moment angle complexes and big Cohen--Macaulayness_ , Algebr. Geom. Topol. 14(1) (2014), 379--406.
  * [29] D. Notbohm and N. Ray, _On Davis--Januszkiewicz homotopy types I; formality and rationalisation_ , Algebr. Geom. Topol. 5 (2005), 31--51.
  * [30] T. E. Panov and I. K. Zeinikesheva, _Equivariant cohomology of moment-angle complexes with respect to coordinate subtori_ , Proc. Steklov Inst. Math. 317 (2022), 141--150.


# BRUMS at SemEval-2020 Task 3: Contextualised Embeddings for Predicting the
(Graded) Effect of Context in Word Similarity

Hansi Hettiarachchi1, Tharindu Ranasinghe2
1School of Computing and Digital Technology, Birmingham City University, UK
2Research Group in Computational Linguistics, University of Wolverhampton, UK
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

This paper presents the team BRUMS submission to SemEval-2020 Task 3: Graded
Word Similarity in Context. The system utilises state-of-the-art
contextualised word embeddings, which have some task-specific adaptations,
including stacked embeddings and average embeddings. Overall, the approach
achieves good evaluation scores across all the languages, while maintaining
simplicity. Following the final rankings, our approach is ranked within the
top 5 solutions of each language while preserving the 1st position of Finnish
subtask 2.

##  1 Introduction

00footnotetext:  This work is licensed under a Creative Commons Attribution
4.0 International Licence. Licence details:
http://creativecommons.org/licenses/by/4.0/.

In natural language, meaning of a word has an influence by its surrounding or
context words. This impact is mainly driven by the associated linguistic and
cognitive phenomena [Armendariz et al., 2020b]. Following these two phenomena,
it was found that word meanings have a continuous nature in addition to the
commonly known discrete nature.

From the cognitive perspective, word meanings can be assigned based on the
conceptual structures in human mind [Evans, 2006, Gärdenfors, 2014].
Therefore, meaning of a word can be varied according to the mental state of
the reader which is triggered with the contact of context words. Thus, word
meanings are not just limited to a discrete nature. From the linguistic
perspective, context words can modify the meaning of a word by contextual
selection and contextual modulation [Cruse et al., 1986]. Contextual selection
identifies the meanings of polysemous words (i.e. words with multiple senses)
such as ‘cell’, ‘bank’, etc. using the context words. In this case, the most
appropriate meaning will be picked from a set of discrete senses. For an
example, phrase ‘prison cell’ implies that the word ‘cell’ refers a room.
Contextual modulation modifies the meanings of single sense words by
highlighting their characteristics. Thus, unlike the contextual selection,
modulation makes a continuous effect on the meaning and it is widely
available, because majority of the words are general to a certain extent. For
an example, ‘butter’ is a single sense word. But, if it appears in the phrase
‘poured the butter’, context words reveal that the mentioned butter is in
liquid state. Focusing on the above-mentioned facts, it is important to
consider both discrete and continuous effects while predicting the meaning of
words in natural language text.

Even though there is a continuous effect on meaning, majority of the previous
research were only focused on the discrete effect. Considering this
limitation, SemEval-2020 Task 3 was designed by targeting the continuous
(graded) effects of context. With the involvement of continuous effects, the
goal of this task is predicting the effect of context in human perception of
similarity. For an example, given the phrases:

> ‘Her prison cell was almost an improvement over her room at the last
> hostel.’

> ‘His job as a biologist didn’t leave much room for a personal life. He knew
> much more about human cells than about human feelings.’

this task needs to predict that words ‘room’ and ‘cell’ have more similar
meaning in phrase 1 compared to phrase 2.

As participants of SemEval-2020 Task 3, we experimented the impact of
contextualised word embeddings extracted by different architectures and
learning methods on predicting the (graded) effect of context in word
similarity. Rather than focusing on the embeddings taken with default
settings, we evaluated stacked embeddings, different parameter settings and
average embeddings. Further, we experimented the embeddings generated with
improved known data rate. In this approach, model unknown words in the data
set are replaced with model known words to support more effective embedding
generation. The rest of this paper is organised as follows. Section 2
describes the task including its subtasks and data sets. Available methods
related to this research are summarised under Section 3 and our approaches are
described in Section 4. Following these, results are mentioned in Section 5
and paper is concluded with Section 6.

##  2 Task description and Data sets

SemEval-2020 Task 3 [Armendariz et al., 2020a] is focused on predicting the
(graded) effect of context in word similarity. There were two unsupervised
subtasks associated with this shared task as follows;

  * •

Subtask 1 - Predicting Changes: Predicting the degree and direction of change
in similarity of same word pair within two different contexts ( This task
targets the ability to identify continuous effects on meaning or effects made
by context in human perception of similarity )

  * •

Subtask 2 - Predicting Ratings: Predicting the similarity of same word pair
within two different contexts ( This task is more similar to the traditional
task which evaluates the similarity of words based on their contexts )

As the evaluation data set, CoSimLex [Armendariz et al., 2020b] was used. This
data set is newly created using human annotators, with the focus on graded
effect on word similarity. As the first version, 340 English pairs, 112
Croatian pairs, 111 Slovenian pairs and 24 Finnish pairs were released without
human annotated scores to use with the evaluation phase of SemEval-2020 Task
3. Among them, randomly selected 10 English pairs, 5 Croatian pairs and 5
Slovenian pairs were released including the human annotated scores as the
practice data for participants to evaluate their models.

##  3 Related Work

Effect of context was considered by previous research to predict the meaning
of words and their similarity. In order to include wider context knowledge,
Huang et al. [Huang et al., 2012] introduced a neural network-based language
model which considers both local and global contexts to learn word
representations. As local context, sequence of words is considered and as
global context, corresponding document is considered. To capture the multiple
senses of a word, they suggested to cluster learned context word vectors into
different meaning groups.

In later research, there was a tendency to focus on sense embeddings. Sense
embeddings are vectors generated to represent different senses of words. Chen
et al. [Chen et al., 2014] suggested to use words in WordNet [Miller, 1995]
gloss contents to produce sense embeddings. To obtain vector representations
of words, they used Skip-gram model [Mikolov et al., 2013]. For each
polysemous word, a sense was picked by comparing the similarity between its
context vector and available sense vectors. Given a sentence, average of its
word vectors was computed as the context vector. Similar approach was
suggested by another recent research using BERT embeddings [Devlin et al.,
2018] to incorporate more contextual features [Loureiro and Jorge, 2019]. In
this approach, sense embedding generation was initiated using SemCor corpus
[Miller et al., 1994]. Using the sentences available in the corpus for each
sense, BERT embeddings were generated and average of those embeddings were
taken as the sense embedding. To improve the sense coverage, WordNet’s
ontology and glosses also considered. Given a sentence, generated BERT
embedding corresponding to a polysemous word was compared with sense
embeddings to disambiguate it.

Using meaning groups and sense embeddings, above-mentioned approaches are only
looking for discrete word meanings with the effect of context and we could not
find any approach which considers both discrete and continuous effects. As we
are aware of CoSimLex is the first data set which is annotated by considering
the continuous effect of context and SemEval-2020 Task 3 is the first shared
task which is focused on predictions based on this effect. However, most of
the available research are based on prediction-based word embeddings and among
them contextualised word embeddings are found to be more capable in extracting
word meanings based on the context.

##  4 Methodology

This section describes the different approaches used to predict the (graded)
effect of context in word similarity. Since the similarity between a word pair
is measured using the cosine similarity between word vectors, we experimented
different methods to generate vector representations based on recently
published contextualised word embedding models as further described in
Sections; 4.1 \- 4.4. All the implementations are done in Python 111The code
is available on https://github.com/HHansi/Semeval-2020-Task3.

###  4.1 Contextualised Embeddings

Unlike the classic word embeddings; Word2Vec [Mikolov et al., 2013], GloVe
[Pennington et al., 2014], fastText [Bojanowski et al., 2017], etc.,
contextualised word embeddings capture the variations of word meanings based
on their context. Therefore, they were successfully applied to wide range of
NLP tasks such as text classification [Sun et al., 2019, Ranasinghe et al.,
2019], question answering [Devlin et al., 2018, Alloatti et al., 2019] and
machine translation [Imamura and Sumita, 2019, Zhu et al., 2020] with improved
performance.

Among the various contextualised embedding models available, we used ELMo
[Peters et al., 2018], Flair [Akbik et al., 2018], BERT [Devlin et al., 2018],
Transformer-XL [Dai et al., 2019] and XLNet [Yang et al., 2019] for this task
in order to evaluate the impact by different architectures and learning
methods on contextual word similarity prediction. Both ELMo and Flair are
based on bidirectional LSTM (Long Short-Term Memory) architecture [Sundermeyer
et al., 2012] and other models are based on bidirectional Transformer
architecture [Devlin et al., 2018]. ELMo is learned on sequence of tokens and
Flair is learned on sequence of characters. The Transformer architecture
introduced by BERT was extended as Transformer-XL by enabling the learning
dependencies beyond fixed length contexts. XLNet is a further improved version
of Transformer-XL considering the advantages in autoregressive and
autoencoding methods. All these Transformer-based models are learned on
sequence of tokens.

This research is supported by available pretrained embedding models. Fixed
size word vectors are generated for each of the target words using their given
context and pretrained model weights. The experimented models including their
language coverage are summarised in Table 1. We used the implementations by
FLAIR [Akbik et al., 2019]222Git repository of flair is available on
https://github.com/flairNLP/flair and Hugging Face 333All the models supported
by Hugging Face can be found on https://huggingface.co/models for embedding
generation. More details about the models including layers, parameters and
training corpora are available with FLAIR and Hugging Face documentation.

Model | en | hr | sl | fi
---|---|---|---|---
ELMo:large | x | - | - | -
Flair:multi-X | x | x | x | x
Flair:hr-X | - | x | - | -
Flair:sl-X | - | - | x | -
Flair:sl-v0-x | - | - | x | -
Flair:fi-X | - | - | - | x
BERT:large-cased | x | - | - | -
BERT:large-uncased | x | - | - | -
BERT:base-multilingual-cased | x | x | x | x
BERT:base-multilingual-uncased | x | x | x | x
BERT:base-finnish-cased-v1 | - | - | - | x
BERT:base-finnish-uncased-v1 | - | - | - | x
Transformer-XL:wt103 | x | - | - | -
XLNet:large-cased | x | - | - | -
Table 1:  Embedding models used by this research and their language coverage
based on the languages; English(en), Croatian(hr), Slovenian(sl) and
Finnish(fi) specific to this task

###  4.2 Stacked Embeddings

Secondly, we tried out the stacked embeddings generated using above-mentioned
(Section 4.1) contextualised word embeddings. Stacked embeddings combine each
vector by concatenating them to form the final vector [Akbik et al., 2018] as
shown in Equation 1. $v_{i}^{stk}$ represents the final or stacked word vector
corresponding to the word $i$ and $v_{i}^{model_{m}}$ represents the vector
obtained by using the embedding model $m$. Combining word embeddings from
different learning methods allows to combine their characteristics together.
For this research, we experimented the stacked embeddings by combining up to
three models only.

$\displaystyle v_{i}^{stk}$ $\displaystyle=\begin{bmatrix}v_{i}^{model_{1}}\\\
v_{i}^{model_{2}}\\\ \vdots\\\ v_{i}^{model_{m}}\end{bmatrix}$ (1)

###  4.3 Parameter Settings with BERT

As the next approach, we experimented the effectiveness of BERT embeddings
using different parameter settings for embedding extraction layers, sub-token
selection and scalar mix. Further, we tested the impact by average embeddings
also. BERT model was selected for this experiment, because we could obtain
good results for all languages using it.

Embedding extraction layers indicate from which layers of the learned model,
weights need to be taken to represent the word vectors. For all the above
experiments mentioned in Section 4.1 and 4.2, we used the concatenation of
last four layers, because it was found as the best approach to represent the
features in underlying text [Devlin et al., 2018].

Since Transformer-based models use sub-tokens to get the embeddings, we
analysed the impact by different sub-token selection techniques; first, last,
concatenation of first and last (first-last) and mean on predicting the word
similarity. Only the first sub-token was used for above experiments.

Scalar mix allows the computation of parameterised scalar mixture of the
defined layers [Tenney et al., 2019]. This technique was found to be useful,
because the best performing layer of a Transformer model could vary depending
on the task and it is unclear to the user. Further, Liu et al. [Liu et al.,
2019] found that scalar mix of Transformer layers has the ability in
outperforming the best individual layers. For above experiments, no scalar mix
was used.

Average Embeddings: As average embeddings, we considered the average of
weights in different layers in order to combine the information learned by
them together. For word $i$, by considering the last $k$ layers, average
embedding $v_{i}^{avg}$ is calculated by following the Equation 2. Weights in
the last layer are represented by the vector $v_{i}^{-1}$.

$v_{i}^{avg}=\frac{v_{i}^{-k}+...+v_{i}^{-1}}{k}$ (2)

###  4.4 Improved Known Data Rate

Since we used pretrained embedding models for this research, there were tokens
such as person names, organisations, locations, etc. in the data set which are
unknown to the vocabulary of the pretrained model. By replacing them with some
known tokens, we can provide more familiar data to the model during the
embedding generation. To convert these tokens into a known form automatically,
we used Named Entity Recognition (NER) [Nadeau and Sekine, 2007]. Named
entities which are identified using the models available with spaCy 444More
details about spaCy are available on https://spacy.io/ were used to replace
the unknown tokens. For an example, ‘…underground in the late 1960s, Sihanouk
had to make concessions…’ is converted as ‘…underground in the late 1960s,
person had to make concessions…’. But, considering the limited availability of
NER models, we conducted this experiment only on English data set.

##  5 Results

This section contains the results obtained for both subtasks using the
approaches described in Section 4. We evaluated the suggested methods using
practice data and used the best methods for submissions. The top three scores
obtained by submissions on both tasks for all four languages and baseline
scores are summarised under Section 5.1 and 5.2.

For all the reported Transformer-based models, as the default parameter
setting, concatenation of last four layers using first sub-token without
scalar mix is used. Any parameter change except the default setting is
mentioned within the brackets after the model name. Stacked models are
mentioned using $+$ symbol (e.g. $model_{1}+model_{2}$) and phrase with NE is
appended to model name if improved known data rate is used.

As the baseline model for both tasks, the multilingual BERT model released by
Gary Lai 555Git repository of baseline embedding model is available on
https://github.com/imgarylai/bert-embedding was used as informed by the task
organisers. This model was trained using an uncased multilingual data set
extracted from Wikipedia.

###  5.1 Subtask 1

To evaluate the subtask 1 results, Pearson correlation between the model
predicted values and gold standards was measured. Average values of scores
produced by human annotators are used as gold standards. Since this task is to
measure the change in similarity between two contexts, the sign of results is
also an important measure, because it indicates the direction of change.
Therefore, the uncentered variation of the Pearson correlation which is
calculated using the standard deviation from zero was used.

The top three results we obtained with the baseline results for English,
Croatian, Slovenian and Finnish are shown in Table 2. According to the
results, our approaches could outperform the baseline in all languages except
Finnish. English and Croatian used the average embeddings to obtain the best
score while Slovenian used the stacked embeddings.

Model |  Score
---|---
baseline |  0.713
BERT:large-cased+Transformer-XL :wt103 |  0.684
BERT:large-cased(k=14) |  0.754
BERT:large-cased(k=14):with NE |  0.753
(a) Final evaluation results-English

Model |  Score
---|---
baseline |  0.587
BERT:base-multilingual-uncased |  0.651
BERT:base-multilingual-cased(first- last) |  0.664
BERT:base-multilingual-cased(k=4) |  0.664
(b) Final evaluation results-Croatian

Model |  Score
---|---
baseline |  0.603
Flair:sl-forward+Flair:sl-backward+ BERT:base-multilingual-uncased |  0.648
BERT:base-multilingual-cased(k=4) |  0.608
BERT:base-multilingual-cased(k=6) |  0.621
(c) Final evaluation results-Slovenian

Model |  Score
---|---
baseline |  0.671
BERT:finnish-cased-v1 |  0.642
BERT:finnish-uncased-v1 |  0.594
BERT:finnish-cased-v1(k=6) |  0.626
(d) Final evaluation results-Finnish

Table 2: Subtask 1 results on final evaluation

###  5.2 Subtask 2

To evaluate the subtask 2 results, harmonic mean of Pearson and Spearman
correlations between model predicted values and gold standards was used
[Camacho-Collados et al., 2017].

The top three resutls we obtained with the baseline results for English,
Croatian, Slovenian and Finnish are shown in Table 3. Based on the results,
our approaches outperformed the baseline in all languages. The best score in
each language was obtained using BERT embeddings which used first-last sub-
token with scalar mix.

Model |  Score
---|---
baseline |  0.573
BERT:base-multilingual-uncased |  0.570
BERT:large-cased(first-last,scalar- mix) |  0.715
BERT:large-cased(first-last,scalar- mix):with NE |  0.713
(a) Final evaluation results-English

Model |  Score
---|---
baseline |  0.402
BERT:base-multilingual-cased |  0.482
BERT:base-multilingual-uncased (first-last) |  0.528
BERT:base-multilingual-uncased (first-last,scalar-mix) |  0.545
(b) Final evaluation results-Croatian

Model |  Score
---|---
baseline |  0.516
BERT:base-multilingual-cased |  0.524
BERT:base-multilingual-cased(k=4) |  0.524
BERT:base-multilingual-uncased(first- last,scalar-mix) |  0.573
(c) Final evaluation results-Slovenian

Model |  Score
---|---
baseline |  0.289
BERT:finnish-cased-v1 |  0.644
BERT:finnish-uncased-v1 |  0.636
BERT:finnish-uncased-v1(first-last, scalar-mix) |  0.645
(d) Final evaluation results-Finnish

Table 3: Subtask 2 results on final evaluation

##  6 Conclusion

In this paper, we presented different approaches used for SemEval-2020 Task 3:
Graded Word Similarity in Context. We mainly evaluated the recent
contextualised word embedding models with different embedding generation
techniques. Depending on the differences between languages, we could not find
any universal approach suitable for all languages to predict the effect of
context in word similarity. For subtask 1: predicting changes, best results
were obtained using average embeddings and stacked embeddings. It concludes
that by combining weights in different layers of same model or different
models, degree and direction of change in similarity can be predicted more
accurately. For subtask 2: predicting ratings, best results were obtained by
BERT embeddings which are generated using first-last sub-token with scalar
mix. Different pretrained models were performed well on each language. It
concludes that the capability of BERT embeddings on predicting the similarity
of words based on their contexts can be further improved using appropriate
parameter settings.

As future directions of this research, we hope to experiment the impact by
learning and fine tuning options of contextual embedding models on graded
effect of context in word similarity.

## References

  * [Akbik et al., 2018]  Alan Akbik, Duncan Blythe, and Roland Vollgraf.  2018\.  Contextual string embeddings for sequence labeling.  In Proceedings of the 27th International Conference on Computational Linguistics, pages 1638–1649.
  * [Akbik et al., 2019]  Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf.  2019\.  Flair: An easy-to-use framework for state-of-the-art nlp.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations), pages 54–59.
  * [Alloatti et al., 2019]  Francesca Alloatti, Luigi Di Caro, and Gianpiero Sportelli.  2019\.  Real life application of a question answering system using BERT language model.  In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue, pages 250–253, Stockholm, Sweden, September. Association for Computational Linguistics.
  * [Armendariz et al., 2020a]  Carlos S. Armendariz, Matthew Purver, Senja Pollak, Nikola Ljubešić, Matej Ulčar, Marko Robnik-Šikonja, Ivan Vulić, and Mohammad Taher Pilehvar.  2020a.  SemEval-2020 task 3: Graded word similarity in context (GWSC).  In Proceedings of the 14th International Workshop on Semantic Evaluation.
  * [Armendariz et al., 2020b]  Carlos S. Armendariz, Matthew Purver, Matej Ulčar, Senja Pollak, Nikola Ljubešić, and Mark Granroth-Wilding.  2020b.  CoSimLex: A resource for evaluating graded word similarity in context.  In Proceedings of The 12th Language Resources and Evaluation Conference, pages 5878–5886, Marseille, France, May. European Language Resources Association.
  * [Bojanowski et al., 2017]  Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov.  2017\.  Enriching word vectors with subword information.  Transactions of the Association for Computational Linguistics, 5:135–146.
  * [Camacho-Collados et al., 2017]  Jose Camacho-Collados, Mohammad Taher Pilehvar, Nigel Collier, and Roberto Navigli.  2017\.  Semeval-2017 task 2: Multilingual and cross-lingual semantic word similarity.  In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 15–26.
  * [Chen et al., 2014]  Xinxiong Chen, Zhiyuan Liu, and Maosong Sun.  2014\.  A unified model for word sense representation and disambiguation.  In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1025–1035.
  * [Cruse et al., 1986]  D Alan Cruse, David Alan Cruse, D A Cruse, and D A Cruse.  1986\.  Lexical semantics.  Cambridge university press.
  * [Dai et al., 2019]  Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G Carbonell, Quoc Le, and Ruslan Salakhutdinov.  2019\.  Transformer-xl: Attentive language models beyond a fixed-length context.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2978–2988.
  * [Devlin et al., 2018]  Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  2018\.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805.
  * [Evans, 2006]  Vyvyan Evans.  2006\.  Cognitive linguistics.  Edinburgh University Press.
  * [Gärdenfors, 2014]  Peter Gärdenfors.  2014\.  The geometry of meaning: Semantics based on conceptual spaces.  MIT Press.
  * [Huang et al., 2012]  Eric H Huang, Richard Socher, Christopher D Manning, and Andrew Y Ng.  2012\.  Improving word representations via global context and multiple word prototypes.  In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pages 873–882. Association for Computational Linguistics.
  * [Imamura and Sumita, 2019]  Kenji Imamura and Eiichiro Sumita.  2019\.  Recycling a pre-trained BERT encoder for neural machine translation.  In Proceedings of the 3rd Workshop on Neural Generation and Translation, pages 23–31, Hong Kong, November. Association for Computational Linguistics.
  * [Liu et al., 2019]  Nelson F Liu, Matt Gardner, Yonatan Belinkov, Matthew E Peters, and Noah A Smith.  2019\.  Linguistic knowledge and transferability of contextual representations.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1073–1094.
  * [Loureiro and Jorge, 2019]  Daniel Loureiro and Alipio Jorge.  2019\.  Liaad at semdeep-5 challenge: Word-in-context (wic).  In Proceedings of the 5th Workshop on Semantic Deep Learning (SemDeep-5), pages 1–5.
  * [Mikolov et al., 2013]  Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean.  2013\.  Efficient estimation of word representations in vector space.  arXiv preprint arXiv:1301.3781.
  * [Miller et al., 1994]  George A Miller, Martin Chodorow, Shari Landes, Claudia Leacock, and Robert G Thomas.  1994\.  Using a semantic concordance for sense identification.  In Proceedings of the workshop on Human Language Technology, pages 240–243. Association for Computational Linguistics.
  * [Miller, 1995]  George A Miller.  1995\.  Wordnet: a lexical database for english.  Communications of the ACM, 38(11):39–41.
  * [Nadeau and Sekine, 2007]  David Nadeau and Satoshi Sekine.  2007\.  A survey of named entity recognition and classification.  Lingvisticae Investigationes, 30(1):3–26.
  * [Pennington et al., 2014]  Jeffrey Pennington, Richard Socher, and Christopher Manning.  2014\.  Glove: Global vectors for word representation.  In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543.
  * [Peters et al., 2018]  Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer.  2018\.  Deep contextualized word representations.  In Proceedings of NAACL-HLT, pages 2227–2237.
  * [Ranasinghe et al., 2019]  Tharindu Ranasinghe, Marcos Zampieri, and Hansi Hettiarachchi.  2019\.  Brums at hasoc 2019: Deep learning models for multilingual hate speech and offensive language identification.  In Proceedings of the 11th annual meeting of the Forum for Information Retrieval Evaluation (December 2019).
  * [Sun et al., 2019]  Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang.  2019\.  How to fine-tune bert for text classification?  In Maosong Sun, Xuanjing Huang, Heng Ji, Zhiyuan Liu, and Yang Liu, editors, Chinese Computational Linguistics, pages 194–206, Cham. Springer International Publishing.
  * [Sundermeyer et al., 2012]  Martin Sundermeyer, Ralf Schlüter, and Hermann Ney.  2012\.  Lstm neural networks for language modeling.  In Thirteenth annual conference of the international speech communication association.
  * [Tenney et al., 2019]  Ian Tenney, Dipanjan Das, and Ellie Pavlick.  2019\.  Bert rediscovers the classical nlp pipeline.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593–4601.
  * [Yang et al., 2019]  Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le.  2019\.  Xlnet: Generalized autoregressive pretraining for language understanding.  In Advances in neural information processing systems, pages 5754–5764.
  * [Zhu et al., 2020]  Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, and Tieyan Liu.  2020\.  Incorporating bert into neural machine translation.  In International Conference on Learning Representations.


# Elliptic Flowers: simply connected billiard tables with chaotic or non-
chaotic flows moving around chaotic or non-chaotic cores

Leonid Bunimovich

(March 2021)

###### Abstract

We introduce a class of billiards with chaotic unidirectional flows (or non-
chaotic unidirectional flows with “vortices”) which go around a chaotic or
non-chaotic “core”, where orbits can change their orientation. Moreover, the
corresponding billiard tables are simply connected in difference with many
attempts to build billiards with interesting and/or exotic dynamics by putting
inside billiard tables various “scatterers” with funny shapes. Therefore the
billiards in this new class are amenable to experimental studies in physics
labs as well as to the rigorous mathematical ones, which may shed a new light
on understanding of classical and quantum dynamics of Hamiltonian systems.

##  1 Introduction

Among the main advances of the 20th century science was the discovery that
deterministic (dynamical) systems can demonstrate an enormously broad variety
of behaviors, from completely regular (integrable) to completely chaotic. Most
of the essential steps in this striking progress were based on finding various
classes of dynamical systems, which demonstrated new, often unexpected and
counter-intuitive, types of behavior. Billiards play a special role in the
list of basic classes of dynamical systems. While being arguably the most
ancient class of dynamical systems ever studied (what is called now
integrability of billiards in circles, was known for millennia) it remains on
a forefront of the theory of dynamical systems and their applications. This
special role of billiards is mainly due to two factors. Billiards form one of
the most visual classes of dynamical systems (if not just the most visual one)
and besides, billiards are natural models of many real life phenomena, first
of all in physics. The purpose of this paper is to introduce a new class of
billiards, which demonstrate various new surprising properties. At the same
time, these billiards are amenable to rigorous mathematical, as well as to
promising numerical, and even to experimental, studies. Formally these
billiards could be considered as generalizations of the first class of
billiards, which was presented as a justification of the discovery of a new
mechanism of chaos (hyperbolicity) in dynamical systems [3, 4, 5, 6], called
the mechanism of defocusing. Boundaries of the first billiard tables, which
demonstrated the mechanism of defocusing, consist of arcs of circles, so that
the arcs completing all components of the boundary to the full circles lie
within the corresponding billiard table. Such systems were called flower-like
billiards [5], and later referred to as Bunimovich flowers (see e.g. [2, 20]).
In what follows we will refer to this class of billiards as to circular
flowers. It is worthwhile to mention that so beloved, especially by the
physics community, stadium billiard is, in fact, a very special and even a
singular example in this class. Indeed, a stadium arose from a flower with two
circular petals by taking their common tangents. Such billiards were initially
called squashes. A squash becomes stadium, when two circular arcs have the
same radius.

The boundaries of the presented in this paper class of billiards consist of
pieces of ellipses. Therefore these billiards will be called elliptic flowers
(EF). However, in difference with circular flowers, which always have chaotic
and ergodic dynamics, this class of billiards demonstrates a large variety of
possible behaviors. For instance, a billiard is called a track billiard [11]
if almost all its orbits always move clockwise or counter-clockwise. Again, an
ancient example of such dynamics is given by an integrable billiard between
two concentric circles. A natural (although somewhat ambitious) question was
whether there exist track billiards with chaotic behavior. This question
arises in quantum chaos [14, 17] in search of examples/situations where
Shnirelman peak [19] in the spectrum of the Schroedinger operator may appear.
In a track billiard this peak naturally appears in the phase space at the
common boundary of tracks. Although existence of chaotic tracks sounded a
little bit too exotic, it was proved that there exist such billiards [11]. In
elliptic flowers, which the present paper introduces, always exist tracks,
which could be chaotic or non-chaotic. However, these tracks do not occupy the
entire phase space, as it was in [11]. Instead these tracks are going around a
core, which also is an invariant set with a positive measure. Dynamics in the
core also (as in tracks) can be chaotic or non-chaotic. A simple description
of dynamics in the elliptic flowers billiards is the existence of a core
surrounded by two flows (tracks) going in the opposite directions, which could
be chaotic or have internal “vortices”. Dynamics in the core could be chaotic
or be the one typical for a generic Hamiltonian system, i.e. a mixture of
elliptic islands and chaotic “seas”. It should be noted that an integrable
billiard in ellipse has two integrable tracks, consisting of orbits tangent to
the confocal smaller ellipses. The corresponding (integrable) core consists of
orbits which intersect the segment between the foci after any reflection off
the boundary. Therefore the new in spirit elliptic flowers are the ones where
either tracks or cores are chaotic, or the both are chaotic. Of course, if a
billiard has a caustic, then orbits which do not intersect the caustic form
two tracks, and the rest form a core. However, the issue is to investigate
dynamics in tracks and core. Examples with regular dynamics, e.g. ellipses are
well known.

An important property of the elliptic flowers is that these billiard tables
are simply connected. In other words, they do not have any internal
boundaries, which are often used in constructions of billiards with various
exotic behaviors. Therefore, elliptic flowers can be built as experimental
devices joining in the labs mushrooms [10], which actually have more
sophisticated, than the elliptic flowers, boundaries. In this respect, it is
also worthwhile to mention that by varying the natural parameters, which
describe the EFs, one can change dynamics of elliptic flowers in several ways.
Such studies may shed a new light on our understanding of dynamics of generic
Hamiltonian systems, both in the classical and quantum settings.

Another possible use of elliptic flowers comes from the theory of the slow-
fast systems, where a recent breakthrough established importance of non-
ergodicity for effective averaging over fast dynamics [16, 18]. Particularly,
mushroom billiards, which always have a divided into KAM islands and ergodic
chaotic “seas” phase space, were effectively used in these studies [16].
Indeed, the elliptic flowers, which we consider here, are always non-ergodic
because of existence of at least three ergodic components, which are tracks
and a core. Moreover, the boundaries of elliptic flowers can be smoothed, what
cannot be done for mushroom billiards and for circular flowers.

In the present paper the examples are given of elliptic flowers billiards with
chaotic and non-chaotic tracks, surrounding chaotic and non-chaotic cores.
Moreover, the first example of another phenomenon is given, which goes against
the developed community intuition about billiards. It is universally known
that, although mechanism of defocusing generates chaotic (hyperbolic)
dynamics, the correlations in billiards with at least one focusing component
of the boundary always decay slowly (power-like). On another hand, the
correlations decay exponentially in dispersing billiards ([1, 22]). We
demonstrate that in the chaotic cores of the elliptic flowers the correlations
may decay exponentially. This result once again shows that the mechanism of
dispersing is a special case of the mechanism of defocusing, not just
formally, what is obvious, but also factually. Indeed, the purely focusing
billiards can generate any type of behavior which dispersing billiards may
generate as well as quite different dynamics from the one of dispersing
billiards. (Defocusing is clearly more general because this mechanism allows
the orbits to converge and diverge in configuration space, while dispersing
allows only divergence of the orbits. Therefore dispersing is the limiting
case of defocusing when orbits never converge).

Our goal here is to present the simplest sub-classes of the elliptic flowers
billiards which demonstrate all the described above new types of dynamics.
Therefore in this paper we consider only the simplest elliptic flowers
billiards. It allows to provide either simple and visual proofs, or the
results immediately follow from the already existing ones in the mathematical
theory of billiards. These general results actually have quite sophisticated
and long proofs, but they allow almost immediately conclude our results here.
Already consideration of these simplest cases allows to get all the promised
results on coexistence of chaotic or non-chaotic tracks with chaotic or non-
chaotic cores. All four possible types of coexistence are demonstrated.

However, the dynamics of general elliptic flower billiards is much richer.
Therefore some natural questions, which arise for the future studies of the
EFs billiards, are discussed in the last section. Some conjectures on possible
existence of elliptic flowers with even more rich dynamics are also formulated
there.

##  2 Multilayered Elliptic Flowers Billiards

Construction of a multilayered elliptic flower billiard (MEF-billiard) starts
with a choice of any convex polygon on the Euclidean plane. Let
$A_{1},A_{2},...A_{n}$ be the vertices of a convex polygon $A$. This polygon
will be called a base for the corresponding family of elliptic flowers
billiards. The first elliptic layer over $A$ is formed by all ellipses, which
do not intersect $A$ and have focuses at the points $A_{i}$ and $A_{i+1}$,
where $i$ varies between $1$ and $n$, and $i+1$ is taken modulus $n$. In other
words, the first layer of ellipses over a polygon $A$ consists of all ellipses
with focuses at the ends of one and the same side of $A$. The second layer of
ellipses over $A$ is a union of all ellipses which have focuses at the points
$A_{i}$ and $A_{i+2}$, where again $i+2$ is taken mod($n$). Clearly the second
layer of ellipses over the polygon $A$ consists of all ellipses with focuses
at the ends of small diagonals of $A$, which connect the ends of two
neighboring sides of $A$. Analogously, a layer of ellipses number $m$ over a
given polygon is defined in the same way, where, instead of $(i+2)$ (mod $n$),
one should take $(i+m)$ (mod $n$). Therefore all ellipses have focuses at some
vertices of $A$. It is easy to see that a regular polygon with $n$ vertices
has $n/2$ layers, if $n$ is an even number, and $(n+1)/2$ layers, if $n$ is
odd.

######  Definition 2.1.

A billiard table is called an unstructured elliptic flower over a base polygon
$A$ if its entire boundary consists of pieces of ellipses which belong to
layers of ellipses over $A$.

Figure 1: An elliptic flower billiard table there the base polygon is a square

For example, in Fig. 1 the base polygon $A$ is a square. There are only two
layers in this case. Observe because the lines passing through the sides of
$A$ never intersect outside of the polygon $A$. It is easy to see that such
situation occurs only when $A$ is a triangle or a square.

We will consider in what follows a special subset of the set of all
unstructured elliptic flowers, which are constructed in the following way.

######  Definition 2.2.

Consider a base polygon $A$ with the vertices $A_{1},A_{2},...,A_{n}$. A
billiard table $Q$ is called a structural elliptic flower over $A$ if all
regular components of the boundary $\partial Q$ are the arcs of the ellipses
from the layers over $A$, and each arc does not intersect the straight lines
containing the sides of $A$.

Remark 1. It follows from the Definition 2.2 that the ends of the elliptic
arcs which form the boundary $\partial Q$ belong to the straight lines passing
through the sides of the base polygon $A$ (see Fig. 1).

Remark 2. Observe that all boundary components of a structural elliptic flower
can belong only to one layer of ellipses over the base polygon $A$ if their
ends are the points of intersection of the lines passing through the sides of
the base polygon $A$. Also there are no structural elliptic flowers with all
boundary components belonging to the first layer. (Fig. 2 (a),(b))

Figure 2: Structural elliptic flowers with a pentagon as a base polygon. (a)
All arcs of ellipses belong to the second layer. (b) Regular components of the
boundary belong to the 1st and 2nd layers in the smaller flower. In the larger
flower petals belong to the 2nd and 3rd layers. The lines like BD cut out from
the pentagon a smaller core.

Clearly, the ends of any regular (smooth) component of the boundary of a
structural elliptic flower belong to lines containing two different sides of
the base polygon $A$ (Fig. 1, Fig. 2). However, if $A$ is a polygon with
$n\geq 5$ sides then the lines containing the sides of $A$ do intersect
outside of this base polygon (Fig. 3). As a result, these lines form a finite
partition the plane into some compact closed sets and some infinite sets which
we will call zones.

It is worthwhile to mention that our construction is much more general than a
construction of a billiard table which contains a given caustic $K$. Recall
that a curve $K$ is a caustic for a billiard if from one link of a billiard
orbit being tangent to $K$ follows that all other links of this orbit are also
tangent to $K$. Given a convex curve $K$, one can obtain a billiard table
possessing this curve $K$ as a caustic by the following string construction.
Take a rope with the length larger that the length of $K$. Then go around $K$
tightly keeping the rope at its full length. The resulting closed curve is a
billiard table $Q$ which has $K$ as a caustics. This construction smoothens,
i.e. even if the curve $K$ is only piecewise $C^{1}$ the so constructed curve
(boundary of a billiard table $Q$) always will be globally $C^{1}$. For
instance, if $K$ is a triangle then the boundary of the resulting billiard
table, consisting of six elliptic arcs, is globally $C^{1}$, but the curvature
is discontinuous in the six points where the ellipses are glued together.

Figure 3: Partition of the plane into zones.

For the sake of simplicity and clarity we will ignore a detailed consideration
of elliptic flowers which can be constructed with a special procedure (choice
of ellipses and their focuses) for each zone. Also for the same reason we will
study in this paper only the simplest classes of the elliptic flower
billiards. Therefore in what follows we will consider elliptic flowers
billiards with the boundary components only from the first and/or the second
layers. Moreover, we will study here only regular polygons with $n\leq 6$ as
the bases of the EFs billiards. It is easy to see that in these cases there
are only two layers and two zones if $n=3$ or $4$, and two layers and three
zones if $n=5$ or $6$ (see Fig. 2). Remarkably, already these the simplest
elliptic flowers have all the main new features which were discussed in the
introduction. Besides, clearly only such elliptic flowers would be the most
easy to build in the experimental labs.

Recall that a billiard is a dynamical system generated by an uniform motion of
a point particle within a region $Q$ in the Euclidean plane. This region $Q$
is a configuration space of a billiard, which is called a billiard table. The
boundary $\partial Q$ of $Q$ consists of a finite number of smooth (at least
$C^{2}$) curves, which are called the regular components of $\partial Q$. A
point particle moves along the straight lines within $Q$, and it gets
elastically reflected off the boundary, i.e. the angle of incidence equals the
angle of reflection. These angles are formed by the internal unit normal
vector $n(q)$ at the point of reflection $q\in\partial Q$ and the vectors of
the particle’s velocities just before and just after a moment of the
reflection off the boundary of the billiard table. Therefore, the orbits of
billiards are broken lines, consisting of straight segments (links) which
connect two points at the boundary of a billiard table, where two consecutive
reflections occur. Without any loss of generality we may assume that a
constant speed of the particle equals one.

Billiards are Hamiltonian systems. Therefore the billiard dynamics preserves
the phase volume $\mu$. It is easy to see that the set of orbits which hit
singular points of the boundary of a billiard table has measure zero. The
major questions about billiards are related to ergodic and statistical
properties of these dynamical systems with respect to the invariant measure
$\mu$.

######  Definition 2.3.

A positive measure invariant set $Tr$ of a billiard dynamical system is called
track if all orbits in $Tr$ move either clockwise or counter-clockwise in a
billiard table.

Certainly, only positive measure tracks are of interest. For instance, a
billiard within any convex region $Q$ with smooth boundary has zero measure
“tracks” consisting of tangent to $\partial Q$ orbits.

It is well-known that a billiard in ellipse $E$ is an integrable dynamical
system. One positive measure family of orbits of such billiard consists of all
orbits which are tangent between any two consecutive reflections off $E$ to
some confocal with $E$ ellipse. Another positive measure family of orbits
consists of all such orbits that the straight line, which contain a segment
between any two consecutive reflections off $\partial E$, is tangent to some
confocal with $E$ hyperbola. The two families of orbits, tangent to the
confocal ellipses and hyperbolas, are separated in the phase space by the set
of orbits which pass through the focuses of $E$ between any two consecutive
reflections off the boundary. It is worthwhile to recall that, according to
the optical definition of ellipses, an ellipse $E$ is a such closed curve that
there are two points $F_{1}$ and $F_{2}$ with the property that a ray of light
passing through $F_{1}$ will pass after reflection off $\partial E$ through
$F_{2}$ , and vice versa. Certainly the orbits tangent to confocal ellipses
form tracks with regular (even integrable) dynamics.

In what follows the regular boundary components of the flower billiard tables
will be sometimes referred to as to petals.

##  3 Main Results

This section presents several statements, which show that structural elliptic
flowers always have tracks which could be chaotic (hyperbolic) or non-chaotic.
Likewise the dynamics in a core, which is the complement to tracks in the
phase space of a billiard, could be chaotic or not. A billiard in ellipse has
a core consisting of such orbit which between any two consecutive reflections
off the boundary intersect the segment connecting the focuses.

Denote by $M(A)$, $Q(A)$ the phase space and the configuration space (billiard
table) built over the base polygon $A$. A core $Co(M(A))$ is the complement to
the tracks $Tr(A)$ in the phase space $M(A)$. The projection of the core
$Co(M(A))$ to the billiard table $Q(A)$ will be called a base core and denoted
by $Co(Q(A))$.

We start with the following simple Proposition.

######  Proposition 3.1.

Let $A$ be a regular polygon with $n\leq 6$ sides. Then any structural
elliptic flower billiard with the base $A$ will have two tracks. The EF
billiards built over triangles and squares will always have a base core $A$.
The EF billiards built over pentagons and hexagons will have the same base
core $A$ if their boundaries belong to the zones 1 and 2. Otherwise, if the
boundary of an elliptic flower billiard table intersect zone 3, then a core is
a convex polygon, which is a proper subset of $A$.

###### Proof.

If the boundary of a billiard table $Q(A)$ does not intersect zone 3, then the
statement immediately follows from the construction of the structural
multilayered elliptic flowers and from the optical definition of ellipses.

Let now the boundary of $Q(A)$ intersects zone 3 (see Fig.2(b)). Connect then
each point, which is an end point of a boundary component (elliptic arc) in
this zone, by the straight segments to the another focus ($D$ in Fig. 2 (b))
of the corresponding ellipse. (Observe, that this end point $B$ is already
connected to one focus of this ellipse by the straight line which belongs to
the boundary of the zone 3). Then the collection of all straight lines, which
are going through all such segments, cut out a smaller convex polygon
$A^{\prime}$ from the $A$. Again, the construction of the structured
multilayered elliptic flowers, together with the optical definition of the
ellipses, ensures that $A^{\prime}$ is indeed the base core of the billiard in
$Q(A)$. ∎

Actually any structural elliptic flower billiard has two tracks of positive
measure. A proof of the corresponding general statement uses the same argument
as the Proposition above.

######  Theorem 3.2.

Let $A$ be a convex polygon in the Euclidean plane with the vertices
$A_{1},A_{2},...,A_{n}$. Consider any billiard table $Q(A)$, which is a
structured elliptic flower built over the base polygon $A$. Then the billiard
in $Q(A)$ has two positive measure tracks, consisting of orbits moving
clockwise and counter-clockwise respectively.

###### Proof.

The base polygon $A$ is convex. Therefore it lies on one side of a line going
through any side of $A$. Then by construction of the structural elliptic
flowers it follows from the optical definition of ellipses that if any link of
a billiard orbit goes through some vertex of $A$ then all other links of such
orbit also contain one vertex of $A$.

By making now an analogous consideration to the one in the proof of
Proposition 3.1, we obtain that a billiard in any structured elliptic flower
$Q(A)$ with the base $A$ has two tracks. Likewise, the corresponding core
coincides with $A$, if the boundary $\partial Q(A)$ belongs only to the zones
1 and 2. Otherwise, a core $Co(Q(A))$ of the corresponding billiard is a
convex polygon, which is a proper subset of the base polygon $A$. ∎

######  Corollary 3.3.

The core $Co(M(A))$ consists of all such billiard orbits that each their link
intersect $A$.

Observe that the Corollary 3.3 implies also that the core is an invariant
subset of the corresponding billiard dynamics.

This statement immediately follows from the construction of the structural
multilayered elliptic flowers and the proof of the Proposition 3.1 and Theorem
3.2.

######  Corollary 3.4.

The core $Co(M(A))$ of an elliptic flower billiard has a positive measure in
the phase space $M(A)$. That means $\mu(Co(M(A)))>0$.

######  Definition 3.5.

A core $Co(M(A))$ of an elliptic flower billiard will be called an ergodic
core if it is an ergodic component of this dynamical system. Analogously, a
track will be called ergodic if it is an ergodic component of the
corresponding elliptic flower billiard.

As was already mentioned, in what follows we will mainly concentrate on the
case when the base polygon $A$ is a regular polygon with $n\leq 6$ sides.

######  Proposition 3.6.

If the base polygon $A$ is a regular triangle or a square then the dynamics on
the corresponding core of a structural elliptic flower over $A$ cannot be
ergodic.

###### Proof.

Indeed, according to the Remark 1 (see also Fig. 1) the boundary $\partial
Q(A)$ of a structural elliptic flower over a regular triangle consists of six
regular components, which contain three pairs of arcs of two confocal
ellipses. Moreover, a period two orbit which connects the ends of the small
axes for any pair of these ellipses is a stable elliptic periodic orbit.
Therefore the dynamics on a core of EF billiards, built over a regular
triangles, is not ergodic. The same is true for the four pairs of the arcs of
confocal ellipses in case when $A$ is a square. It is easy to see that the
boundary of a structural elliptic flower built over square contains two pairs
of arcs of confocal ellipses (Fig. 1). Therefore, in this case there are at
least two elliptic periodic orbits (of period two) in the core. Therefore the
core is not an ergodic (chaotic) components of the corresponding elliptic
flower billiard. ∎

The large and the small axes of ellipses will be denoted by $a_{i}$ and
$b_{i}$ respectively, where the index $i$ refers to one of the ellipses which
contain the $i$-th boundary components of an elliptic flower billiard table.
Also $c^{2}=a^{2}-b^{2}$, where $c$ is a half-length of the segment connecting
the focuses of an ellipse.

In this paper our main goal is to demonstrate in the simplest examples the
richness of dynamics of the elliptic flowers billiards. Therefore in what
follows we will consider the simplest billiards in this class called special
one layer (SOL) elliptic flowers.

Construction of special one layer elliptic flowers: Let $A$ be an arbitrary
convex polygon with $n$ vertices. Consider the straight semi-lines which have
the ends at the centers of the sides of $A$, are orthogonal to the
corresponding sides, and do not intersect $A$. Then a special one layer
elliptic flower is a billiard table with $n$ boundary components, which are
the arcs of ellipses with the focuses located at the ends of the small
diagonals of $A$ and the ends on two semi-lines orthogonal to the
(intersecting) sides of $A$, which the corresponding small diagonal connects
(Fig. 4). If $A$ is a triangle, then the sides play also the role of small
diagonals.

Figure 4: Narcissus: structural one layer elliptic flower over a regular
hexagon.

The next statement shows that, there are structural elliptic flowers with an
ergodic core if a regular base polygon has at least five sides. (It should be
mentioned though that structural elliptic flowers built over non-regular
triangles and rectangulars may also have ergodic dynamics on their core).

######  Theorem 3.7.

If a number of sides of the base regular polygon $A$ is $n\geq 5$, then there
exist structured elliptic flowers $Q(A)$ such that dynamics of the
corresponding billiard, restricted to the core, is hyperbolic and ergodic.

###### Proof.

For the sake of clarity and brevity we will present a proof only for SOL
elliptic flowers.

Consider a hyperbolic and ergodic circular flower billiard [6], where all the
corresponding circles have the same radius $R$, and the centers of all these
circles are at the centers of the small diagonals of $A$. Recall that all
circular arcs, which complement regular components of an ergodic circular
flower to the entire circles, belong to the interior of the billiard table.

Recall now that the maximal radius of curvature of the boundary of an ellipse
with the parameters (semi-axes lengths) $a$ and $b$ equals

$\frac{a^{2}}{b}=b+\frac{c^{2}}{b}$ (1)

The osculating circle with this maximal radius is tangent to an ellipse at an
end of its small axis. We call such circle a maximal osculating circle.

Substitute now each of these $n$ circles by the tangent to them ellipses with
the focuses located at the centers of the corresponding shortest diagonals of
$A$ and the small axis orthogonal to these diagonals. As a result of we obtain
a special one layer elliptic flower $Q(A)$, where the ends of the regular
components of the boundary $\partial Q(A)$ lie on the lines orthogonal to the
sides of the regular polygon $A$ at their centers.

All $n$ regular components of the boundary of $Q(A)$ are the arcs of ellipses,
each of which has focuses at the ends of the shortest diagonals
$A_{i},A_{(i+2)}$ of $A$. In what follows we will refer to such diagonals of
any polygon as to small diagonals (just having in mind that in a general
convex polygon not all small diagonals could be also the shortest ones).
Observe that there are no ellipses among them with the same focuses because
$n\geq 5$.

Denote by $l$ the length of sides of the base polygon $A$. Then the lengths of
its small diagonals are equal to $2l\cos(\pi/n)$. So $c$ in the relation (1)
equals $l\cos(\pi/n)$. The centers of all ellipses are at the centers of the
corresponding shortest (small) diagonals of the basic polygon $A$. The
distances between these points and the center of the polygon $A$ equal

$\frac{l(\cos(2\pi/n))}{2\sin(\pi/n)}$ (2)

Consider now the maximal osculating circle tangent to a boundary component at
its point with the minimal curvature (i.e. at the end of the small axis of the
corresponding ellipse). Take sufficiently large radius $R$ for an ergodic
circular flower over $A$. Observe that $R=b$, where $b$ is the length of the
small semi-axis of the ellipse tangent to a regular component of the boundary
of the circular flower. It immediately follows from the relations (1) and (2)
that for any $n\geq 5$ the maximal osculating circle consists of two arcs. The
first arc goes outside of the elliptic flower billiard table, and the second
arc completely lies within the billiard table $Q(A)$ if $R$ is large enough or
if $c$ is small enough. In other words, both these conditions mean that the
base polygon should be sufficiently small in comparison with the size of the
elliptic flower billiard table $Q(A)$.

Take now any interior point $q$ of any regular component of the boundary
$\partial Q(A)$. Then any straight segment, which connects $q$ to any other
regular component of the boundary will intersect the osculating circle at the
point $q$ within the billiard table $Q(A)$.

Sufficient conditions of ergodicity of the two-dimensional billiards were
extensively studied (see e.g. [8, 13]). However, the cores of elliptic flower
billiards have the special property which allows to reduce these conditions
just to one. Namely, any orbit in the core cannot have two consecutive
reflections off one and the same regular component (an elliptic arc) of the
boundary $\partial Q(A)$. Therefore, it is enough for ergodicity that any link
of any orbit in the core intersect both osculating circles of two ellipses at
the points, which this link connect.

Recall now that any orbit of the core of the elliptic flower billiard
intersects the base polygon $A$ between any two consecutive reflections off
the boundary. Thus each link of such orbit intersect both osculating circles
tangent to the boundary $\partial Q(A)$ at the ends of this link.

Therefore the defocusing mechanism [3, 4, 6] ensures that dynamics in the core
of the elliptic flower $Q(A)$ is completely hyperbolic, i.e. it has
nonvanishing Lyapunov exponents. The ergodicity of the dynamics in the core
immediately follows from [8, 13]. ∎

Remark 3. Observe that the chaotic cores of the special one layer elliptic
flowers are nonempty subsets of the base polygon $A$. The sizes of these
subsets decrease when $b=R$ increases. These cores are regular convex
polygons, which, when $b$ tends to infinity, converge to the polygon formed by
the lines orthogonal to the sides of $A$ at their ends.

Remark 4. A necessary condition of ergodicity of a two-dimensional billiard
says that all focusing components of the boundary should be absolutely
focusing [12]. However, all orbits of the core cannot have two consecutive
collisions with one and the same regular component of the boundary. Therefore
the absolute focusing condition does not play any role here. Observe also that
in the papers dealing with general results on ergodicity of billiards only
ergodicity in the entire phase space was studied, rather than ergodicity of
dynamics on some subset of the phase space. However, the corresponding proofs
can be applied word by word for our case, when the core is an ergodic
component with a positive, but not full, measure.

We will consider now conditions of hyperbolicity and ergodicity of dynamics in
tracks. The orbits in tracks, in difference with the core, may have any number
of consecutive reflections off the one and the same regular boundary component
of an elliptic flower. Therefore, in dealing with tracks we will require all
regular components of the boundary to be absolutely focusing. Recall [7, 9]
that a $C^{2}$-smooth focusing curve $\Gamma$ is absolutely focusing if any
parallel beam of rays, which is fallen on $\Gamma$, will leave this curve
after any series of consecutive reflections off $\Gamma$ as a convergent
(focused) beam.

######  Theorem 3.8.

There exist elliptic flower billiards with hyperbolic and ergodic tracks.

###### Proof.

In what follows tracks, which are hyperbolic and ergodic, will be called
chaotic tracks. Again for the sake of clarity and simplicity we will consider
only special one layer flower billiards.

Consider now (the same as above) small perturbations of the hyperbolic and
ergodic circular flower, when all $n$ equal circles become equal ellipses with
focuses at the ends of the small diagonals of $A$. Let also their small axis
equals $2R$, and thus $b=R$. All $n$ regular components of the boundary of
$Q(A)$ are the arcs of ellipses, each of which has focuses at the ends of the
shortest diagonals $A_{i},A_{(i+2)}$ of $A$. Recall that there are no ellipses
with the same focuses because $n\geq 5$.

The polygon $A$ is regular and the corresponding circular flower has identical
arcs of a circle with radius $R$. Therefore each of the circles containing a
component of this circular flower intersects only two neighboring boundary
components. We can keep this property for an elliptic flower by choosing
sufficiently large (with respect to the size of the base polygon $A$) circles.
Indeed, it immediately follows from the relation (1).

Now, it follows from the symmetry of $A$ and the symmetry of our construction
that the maximal osculating circles to the ellipses, which form the boundary
of elliptic flower billiard table intersect only two neighboring osculating
circles. Besides any largest osculating circle does not contain the ends
(which belong to the boundary of the billiard table) of the short diagonals of
the neighboring osculating circles. Therefore divergence of the orbits beats
convergence at any link of an orbit which connects two points of different
boundary components.

However in tracks, in difference with core, orbits may have any number of
consecutive reflections off one and the same boundary components. Therefore
each regular boundary component should be absolutely focusing. The condition
for the symmetric arcs of ellipses with center at an end of the small diagonal
of ellipse to be absolutely focusing was found in [21]. This condition says
that projections of these arcs to the large axis of ellipse should not exceed
$a/\sqrt{2}$. It is easy to see that this condition is satisfied if we choose
sufficiently large $R=b$ (for a fixed base polygon $A$ with at least five
sides). Clearly for regular triangles and for the squares this condition never
holds.

A sufficient condition of the absolute focusing for petals of SOL elliptic
flowers can be obtained from an elementary geometric consideration. This
sufficient condition of absolute focusing says that the angle between the
minor axis of an ellipse and the line connecting the center of an ellipse to
the end point of the corresponding component of the boundary (petal) must be
smaller than $\pi/4$. Clearly this angle is a decreasing function of the
length $b$ of the small axis, and the condition of the absolute focusing holds
for sufficiently large $R=b$. ∎

By combining now the proofs of theorems and 3.2 and 3.7 we obtain

######  Theorem 3.9.

There exist hyperbolic elliptic flowers billiards with phase space consisting
of three ergodic components of positive measure, which are two chaotic tracks
and a chaotic core.

Exactly the same proof, as the one of Theorem 3.8 goes again for special one
layer elliptic flowers with at least five sided regular base polygon by
perturbing a hyperbolic and ergodic circular flower with large circles.

For general multilayered elliptic flowers a proof is quite analogous in logic
but contains various extra trigonometric formulas. Once again, our goal in
this paper is just to show that the dynamics described in theorems 3.7, 3.8
and 3.9 does exist in elliptic flowers. General multilayered elliptic flowers
provide for more possibilities, some of which will be described in the last
section.

Wild Rose and Narcissus: the simplest elliptic flower billiards with chaotic
tracks and chaotic core. It was shown above that, if an elliptic flower has
chaotic tracks and chaotic core, then a minimal number of sides of a basic
polygon $A$ is five. Besides, a general multilayered elliptic flower billiard
table, built over a 5-polygon, has more than five petals (regular components
of the boundary). We give now the simplest example of an elliptic flower with
all the properties considered in the theorems 3.7, 3.8 and 3.9. (It can be
done by choosing $b$ either large or small enough).

Consider a regular convex pentagon $A$. Construct now a special one layer
(SOL) elliptic flower over $A$. This flower has five petals, and therefore it
can be naturally called a wild rose elliptic flower billiard. Let also each
ellipse containing these petals have focuses at the ends of the corresponding
small diagonal of $A$. It was shown in Theorem 3.9 that this billiard has a
chaotic core surrounded by chaotic tracks if the size of the base pentagon $A$
is relatively small in comparison with the size of the corresponding billiard
table $Q(A)$. This wild rose billiard is the simplest one to built in a lab.
Observe also that an experimenter can choose a relevant size of this devise.
Another simple example is the Narcissus billiard, which is built over a
regular hexagons exactly at the same fashion as the wild rose was built over a
regular pentagon (Fig. 4).

##  4 Some Open Questions and Conjectures

As it was shown above, elliptic flower billiards demonstrate a large variety
of behaviors, some of which have never been observed before, or even
conjectured to exist. Therefore the first natural general question for the
future studies of elliptic flowers is to understand how the changes in the
dynamics of these billiards occur. Elliptic flower billiard can be specified
by several parameters, all of which have a clear and visual geometric meaning.
Therefore various types of bifurcations, which occur in these billiards, could
be described in some clear geometric terms.

Certainly, some trivial answers are immediately there, e.g. how a core may
loose ergodicity by acquiring elliptic periodic orbit(s). However, a question
on appearance of “vortices” with regular dynamics within chaotic flows in
tracks is, certainly, of interest. Another question is whether immediately
infinitely many vortices appear in chaotic tracks as a result of such
transition. A general intuition suggests that it is likely the case. However,
even in this situation it would be interesting to look at the structure of the
set of vortices, and determine whether or not there is some kind of
“coherence” between them.

Another interesting issue, although probably essentially a technical one, is
about construction of elliptic flowers with some specific properties for any
convex polygon taken as a base. The results of the present paper were
intentionally dealing with the simplest situation, when the base polygon is
regular.

There are still, at least theoretical, possibilities that there exist elliptic
flowers billiards, which have some interesting properties not considered in
the present paper. Recall that in all the examples of multilayered elliptic
flower billiards, studied in the present paper, all petals (regular components
of the boundary) belonged only to the first or to the second layer of the base
polygon.

Moreover, mostly even the simpler class of special one layer elliptic flowers
was analysed. One may expect that the studies of elliptic flowers with the
higher levels petals will bring some new examples of an unseen before
behaviors, which are amenable to rigorous studies.

Observe also that all elliptic flowers considered in our paper had $C^{0}$
boundary. However, it is easy to construct elliptic flowers with $C^{1}$ and
with $C^{2}$ boundaries. (See, for instance, the string construction described
in the section 2). Could such elliptic flowers billiards demonstrate analogous
properties to the ones studied here? Certainly the tracks will be there, but
it is not so clear about other features.

Intriguing could be also the future studies of quantum elliptic flowers
billiards. One may expect to see some new surprises there.

Another general question is whether it is possible to construct billiards with
similar dynamics when a core is any convex set on a plane rather than a
polygon.

Acknowledgements. I am indebted to Hassan Attarchi for help with figures and
useful comments.

## References

  * [1] Baladi V, Demers M and Liverani C 2018 Exponential Decay of Correlations for Finite Horizon Sinai Billiard Flows Invent. Math. 211 39-177
  * [2] Balint P and Melbourne I 2008 Decay of correlations and invariance principles for dispersing billiards with cusps, and related planar billiard flows J. Stat. Phys. 133 435-447
  * [3] Bunimovich L A 1973 On the Ergodic Properties of Billiards Close to Dispersing Academy of Sciences of the USSR Doklady 211 1024-1027
  * [4] Bunimovich L A 1974 On Ergodic Properties of Some Billiards,” Functional Analysis and its Applications 8 254-255
  * [5] Bunimovich L A 1974 On Billiards Close to Dispersing Ones,” Mathematical Sbornik 94 45-67
  * [6] Bunimovich L A 1979 On the Ergodic Properties of Nowhere Dispersing Billiards Commun.Math.Phys. 65 295-312
  * [7] Bunimovich L A 1988 Many-dimensional Nowhere Dispersing Billiards with Chaotic Behavior Physica D: Nonlinear Phenpmena 33 58-64
  * [8] Bunimovich L A 1990 Theorem on Ergodicity of Two-Dimensional Hyperbolic Billiards Commun. Math. Phys. 130 599-621
  * [9] Bunimovich L A 1992 On Absolutely Focusing Mirrors in Ergodic Theory and Related Topics” (ed. by U. Krengel et al), Springer Lect. Notes Math.1514 62-82
  * [10] Bunimovich L A 2001 Mushrooms and Other Billiards with Divided Phase Space Chaos 11 1-7
  * [11] Bunimovich L A and DelMagno G 2009 Track Billiards Comm. Math. Phys. 13 699-713
  * [12] Bunimovich L A and Grigo A 2010 Focusing components in typical chaotic billiards should be absolutely focusing Comm. Math. Phys. 293 127-143
  * [13] DelMagno G and Markarian R 2017 On the Bernoulli Property of Planar Hyperbolic Billiards Comm. Math. Phys. 350 917–955
  * [14] Chirikov B V and Shepelyansky D L 1995 Shnirelman Peak in Level Spacing Statistics Phys. Rev. Lett. 74 518-521
  * [15] Donnay V 1991 Using Integrability to Create Chaos: billiards with positive entropy Comm. Math. Phys. 141 225-257
  * [16] Gelfreich V Rom-Kedar V and Turaev D 2014 Oscillating mushrooms: adiabatic theory for a non-ergodic system, J. Phys. A: Mathematical and General 47 395101
  * [17] Horvat M and Prosen T 2004 Uni-directional transport properties of a serpent billiard J. Phys. A: Mathematical and General 37 (9), 313323
  * [18] Shah K, Turaev D, Gelfreich V and Rom-Kedar V 2017 Equilibration of energy in slow-fast systems, PNAS
  * [19] Shnirelman A I 1975 On asymptotic multiplicity of Laplace operator spectrum Russian Math. Surveys 30 265–266
  * [20] Vaienti S and Zhang Hong-Kun Optimal bounds for decay of correlations and $\alpha$-mixing for nonuniformly hyperbolic dynamical systems arXiv:1605.01793arXiv.org ¿ math ¿
  * [21] Wojtkowski M 1986 Principles for the design of billiards with nonvanishing Lyapunov exponents Comm. Math. Phys. 105 391–414
  * [22] Young L S 1998 Statistical Properties of Dynamical Systems with Some Hyperbolicity 1998 Annals Math. 147 585-650