# Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-
Resource Settings

Iker García-Ferrero Rodrigo Agerri German Rigau
HiTZ Basque Center for Language Technologies - Ixa NLP Group
University of the Basque Country UPV/EHU
{ iker.garciaf, rodrigo.agerri, german.rigau<EMAIL_ADDRESS>
###### Abstract

Zero-resource cross-lingual transfer approaches aim to apply supervised models
from a source language to unlabelled target languages. In this paper we
perform an in-depth study of the two main techniques employed so far for
cross-lingual zero-resource sequence labelling, based either on data or model
transfer. Although previous research has proposed translation and annotation
projection (data-based cross-lingual transfer) as an effective technique for
cross-lingual sequence labelling, in this paper we experimentally demonstrate
that high capacity multilingual language models applied in a zero-shot (model-
based cross-lingual transfer) setting consistently outperform data-based
cross-lingual transfer approaches. A detailed analysis of our results suggests
that this might be due to important differences in language use. More
specifically, machine translation often generates a textual signal which is
different to what the models are exposed to when using gold standard data,
which affects both the fine-tuning and evaluation processes. Our results also
indicate that data-based cross-lingual transfer approaches remain a
competitive option when high-capacity multilingual language models are not
available.

Figure 1: In the data-based transfer approach we translate and project the
labels of the gold data into the target language, and use the resulting silver
data to train a model for the target language. In the model-based transfer
approach we train a model with gold data in English and use it in a zero-shot
setting in the target language.

##  1 Introduction

Sequence labelling is the task of assigning a label to each token in a given
input sequence. Sequence labelling is a fundamental process in many downstream
NLP tasks. Currently, most successful approaches for this task apply
supervised deep-neural networks Lample et al. (2016); Akbik et al. (2018);
Devlin et al. (2019); Conneau et al. (2020). However, as it was the case for
supervised statistical approaches Agerri and Rigau (2016), their performance
still depends on the amount of manually annotated training data. Additionally,
deep-neural models still show a significant loss of performance when evaluated
in out-of-domain data Liu et al. (2021). This means that to improvie their
performance, it would therefore be necessary to develop very costly manually
annotated data for each language and domain of application. Thus, considering
that for most of the languages in the world the amount of manually annotated
corpora is simply nonexistent Joshi et al. (2020), then the task of developing
sequence labelling models for languages and domain-specific tasks, for which
supervised data is not available, remains a challenge of great interest. This
task is known as zero-resource cross-lingual sequence labelling.

Data-based cross-lingual transfer methods aim to automatically generate
labelled data for a target language. Previous works on data-based transfer
have proposed _translation and annotation projection_ as an effective
technique for zero-resource cross-lingual sequence labelling Jain et al.
(2019); Fei et al. (2020). In this setting, as illustrated in Figure 1, the
idea is to translate gold-labelled text into the target language and then,
using automatic word alignments, project the labels from the source into the
target language. The result is an automatically generated dataset in the
target language that can be used for training a sequence labelling model.

The emergence of multilingual language models Devlin et al. (2019); Conneau et
al. (2020) allows for model-based cross-lingual transfer. As Figure 1
illustrates, using labelled data in one source language (usually English), it
is possible to fine-tune a pre-trained multilingual model that is directly
used to make predictions in any of the languages included in the model. This
is also known as _zero-shot_ cross-lingual sequence labelling.

In this work we present an in-depth study of both approaches using the latest
advancements in machine translation, word aligners and multilingual language
models. We focus on two sequence labelling tasks, namely, Named Entity
Recognition (NER) and Opinion Target Extraction (OTE). In order to do so, we
present a data-based cross-lingual transfer approach consisting of translating
gold labeled data between English and 7 other languages using state-of-the-art
machine translation systems. Sequence labelling annotations are then
automatically projected for every language pair. Additionally, we also
produced manual alignments for those 4 languages for which we had expert
annotators. After translation and projection, for the data-transfer approach
we fine-tune multilingual language models using the automatically generated
datasets. We then compare the performance obtained for each of the target
languages against the performance of the zero-shot cross-lingual method,
consisting of fine-tuning the multilingual language models in the English gold
data and generating the predictions in the required target languages.

The main contributions of our work are the following: First, we empirically
establish the required conditions for each of these two approaches, data-
transfer and zero-shot model-based, to outperform the other. In this sense,
our experiments show that, contrary to what previous research suggested Fei et
al. (2020); Li et al. (2021), the zero-shot model-based approach obtains the
best results when high-capacity multilingual models including the target
language and domain are available. Second, when the performance of the
multilingual language model is not optimal for the specific target language or
domain (for example when working on a text genre and domain for which
available language models have not been trained), or when the required
hardware to work with high-capacity language models is not easily accessible,
then data-transfer based on _translate and project_ constitutes a competitive
option. Third, we observe that machine translation data often generates
training and test data which is, due to important differences in language use,
markedly different to the signal received when using gold standard data in the
target language. These discrepancies seem to explain the larger error rate of
the translate and project method with respect to the zero-shot technique.
Finally, we create manually projected datasets for four languages and
automatically projected datasets for seven languages. We use them to train and
evaluate cross-lingual sequence labelling models. Additionally, they are also
used to extrinsically evaluate machine translation and word alignment systems.
These new datasets, together with the code to generate them are publicly
available to facilitate the reproducibility of results and its use in future
research.111https://github.com/ikergarcia1996/Easy-Label-Projection
https://github.com/ikergarcia1996/Easy-Translate

##  2 Related work

###  2.1 Data-based cross-lingual transfer

Data-based cross-lingual transfer methods aim to automatically generate
labelled data for a target language. Some of these methods exploit parallel
data. Ehrmann et al. (2011) automatically annotate the English version of a
multi-parallel corpus and projects the annotations into all the other
languages using statistical alignments of phrases. Wang and Manning (2014)
project model expectations rather than labels, which facilities transfer of
model uncertainty across languages. Ni et al. (2017) use a heuristic scheme
that effectively selects good-quality projection-labeled data from noisy data.
They also project word embeddings from a target language into a source
language, so that the source-language sequence labelling system can be applied
to the target language without re-training. Agerri et al. (2018) use parallel
data from multiple languages as source to project the labelled data to a
target language, showing that the combination of multiple sources improves the
quality of the projections. Li et al. (2021) uses the XLM-R model Conneau et
al. (2020) for labelling sequences in the source part of the parallel data and
also for annotation projection.

Instead of relying on parallel data, Jain et al. (2019) and Fei et al. (2020),
use machine translation to automatically translate the sentences of a gold-
labelled dataset to the target languages. The translated data is then
annotated by projecting the gold labels from the source dataset. For this
purpose, Jain et al. (2019) first generate a list of projection candidates by
orthographic and phonetic similarity. They choose the best matching candidate
based on distributional statistics derived from the dataset. Fei et al. (2020)
leverages the word alignment probabilities calculated with FastAlign Dyer et
al. (2013) and the POS tag distributions of the source and target words.

High quality parallel data or machine translation systems are not always
available. Thus, Xie et al. (2018) proposes to find word translations based on
bilingual word-embeddings. Alternatively, Guo and Roth (2021) translate
labelled data in a word-by-word manner with a dictionary. Then, they the
construct target-language text from the source-language annotations with a
constrained pretrained language model.

###  2.2 Model-based transfer

Language models trained on monolingual corpora in many languages Devlin et al.
(2019); Conneau et al. (2020) allow zero-shot cross-lingual model transfer.
Task-specific data in one language is used to fine-tune the model for
evaluation in another language Pires et al. (2019). The zero-shot cross-
lingual capability can be improved for the sequence labelling task using
different techniques. The approaches of Wang et al. (2019) and Ouyang et al.
(2021) use monolingual corpora to improve the alignment of the language
representations within a multilingual model. Instead of using a single source
model, Rahimi et al. (2019) propose to use many models from many source
languages to improve the zero-shot transfer to a new language. They learn to
infer which are the most reliable models in an unsupervised manner. Wu et al.
(2020) take advantage of a Teacher-Student learning approach. NER models in
the source languages are used as teachers to train a student model on
unlabeled data in the target language. Bari et al. (2021) propose an
unsupervised data augmentation framework to improve the cross-lingual
adaptation of models using self-training. Hu et al. (2021) use the minimum
risk training framework to overcome the gap between the source and the target
languages/domains. They propose a unified learning algorithm based on the
expectation maximization.

Using low-capacity multilingual language models such as mBERT, Fei et al.
(2020) finds that their data-based cross-lingual transfer approach is superior
to the zero-shot transfer method. However, Li et al. (2021) when using XLM-
RoBERTa, a higher capacity multilingual model, obtain the best results for
German and Chinese applying the data-based cross-lingual transfer approach,
while the zero-shot approach is best for Spanish and Dutch. We extend their
research on zero-resource settings with two different Sequence Labelling
tasks, seven languages and three multilingual models of different capacity.
Our experiments and the error analysis carried out establish the required
conditions on which zero-shot and data-transfer approaches outperform each
other.

##  3 Translation and projection method

Our data-based cross-lingual transfer method to perform cross-lingual sequence
labelling is the following: we assume our source language to be English, for
which we have train and development data. Furthermore, we also assume that the
only gold-labelled data available for the target language is the evaluation
set. In this setting, we automatically generate data for the target language
by translating the gold-labelled English data. Then we project the gold labels
from the source sentences to the translated sentences by leveraging automatic
word alignments. Given a sentence $x=\left\langle
x_{1},...,x_{n}\right\rangle$ with length $n$ in the source language and a
translated sentence $y=\left\langle y_{1},...,y_{m}\right\rangle$ with length
$m$ in the target language, we use a word aligner to find a set of pairs
$A=\left\\{\left\langle
x_{i},y_{j}\right\rangle:x_{i}\in\mathbf{x},y_{j}\in\mathbf{y}\right\\}$ where
for each word pair $\left\langle x_{i},y_{j}\right\rangle$ $y_{i}$ is the
lexical translation of $x_{j}$. Next, given a sequence $s=\left\langle
x_{a},...,x_{b}\right\rangle\in\mathbf{x}$ labeled with a category $C$ we will
label the sequence $t=\left\langle y_{c},...,y_{d}\right\rangle\in\mathbf{y}$
with category $C$ if $\\{\forall y_{j}\in t\,\exists x_{i}\in s:(\left\langle
x_{i},y_{j}\right\rangle\in A)\\}$. In other words, if a word labelled with a
category in the source sentence is aligned to a word in the target sentence,
we label the target word with the category from the word in the source
sentence. Figure 2 illustrates our method.

Figure 2: Illustration of the translation and annotation projection method for
Opinion Target Extraction (OTE).

When projecting annotations we find two main problems: split annotations and
annotation collision. In the first case, a labeled sequence in the source
sentence is split into multiple sequences in the target sentence. This happens
when the alignment for a word is missing. In this case, we merge the sequences
in the target sentence if the gap between them is just one word. If we still
end up with multiple sequences, we choose the longest one. In the annotation
collision case, a word in the target sentence is aligned to two different
labelled sequences in the source language. If the two sequences are of the
same category, we merge them and we label the two sequences as a single one in
the target sequence. If they are of different category we just consider the
one with the longest length. Finally, if a punctuation symbol in the target
sequence is aligned to a labeled word in the source sentence we remove this
alignment.

##  4 Datasets

We conducted experiments in two sequence labelling tasks, namely, Opinion
Target Extraction (OTE) and Named Entity Recognition (NER). Figure 3
illustrates both tasks.

(a) Illustration of the Opinion Target Extraction task.

(b) Illustration of the Named Entity Recognition Task.

Figure 3: Sequence Labelling tasks used in our experiments.

Opinion Target Expression (OTE): Given a review, the task is to detect the
linguistic expression used to refer to the reviewed entity. We use the
SemEval-2016 Task 5 Aspect Based Sentiment Analysis (ABSA) datasets Pontiki et
al. (2016). We experiment with the English, Spanish, Dutch, French, Russian
and Turkish datasets from the restaurant domain.

Named Entity Recognition (NER): Given a text, the task is to detect named
entities and classify them in pre-defined categories. For Spanish and Dutch we
use the CoNLL-2002 datasets Tjong Kim Sang (2002). For English and German we
use the CoNLL-2003 datasets Tjong Kim Sang and De Meulder (2003) and for
Italian we use Evalita 2009 data Speranza (2009). We map the Geo-Political
Entities from Evalita 2009 to location labels to make them compatible with the
CoNLL data.

##  5 Experimental Setup

We perform 1-to-1 annotation projection in two directions:
Translate-Train: We translate the English train and development data to the
target language. We project the gold labels from the English data to the
translated dataset. We then train a sequence labelling model using only the
automatically generated dataset for the target language.
Translate-Test: We translate the target language test set to English. We then
use a model trained in the English gold-labelled data to label the translated
test set. Finally, we project the labelled sequences back to the target
language.

These two data-based cross-lingual transfer approaches are compared with the
zero-shot method in which a fine-tuned model using English gold-labelled data
is evaluated by generating predictions in the target language. Finally, we
also fine-tuned language models on the gold-labelled data, which would
constitute the upper-bound in our experimental setting.

###  5.1 Machine Translation

We tested DeepL222https://www.deepl.com/, MarianMT Junczys-Dowmunt et al.
(2018); Tiedemann and Thottingal (2020), M2M100 (1.2B) Fan et al. (2020) and
mBART (mbart-large-50) Tang et al. (2020). A qualitative analysis performed
during the projection of the OTE labels established that DeepL produced the
more fluent translations. Thus, we decided to use DeepL (web version during
the second half of 2021) to perform the machine translation for our data-based
cross-lingual transfer experiments. The exception was Turkish, which is not
supported by DeepL. In this case we use M2M100.

###  5.2 Word Alignments

For word alignments, we use the AWESoME Dou and Neubig (2021) system. AWESoME
leverages multilingual pretrained language models and fine-tune them on
parallel text. Unsupervised training objectives over the parallel corpus
improve the alignment quality of the models. AWESoME authors claim that the
model works best with mBERT Devlin et al. (2019) as backbone, so we follow
their advice. Although we also experimented with GIZA++ Och and Ney (2003),
FastAlign Dyer et al. (2013) and SimAlign Dou and Neubig (2021), systems based
on alignments from AWESoME produced the highest F1 scores when comparing the
model projections and manually annotated projections (see Section 7).

To train the alignment models we use the English gold-labelled dataset
together with the respective MT system translations as parallel corpora. We
augment the training data with 50,000 random parallel sentences from ParaCrawl
v8 Esplà et al. (2019) for all the language pairs except Turkish, for which we
use 50,000 random parallel sentences from the raw CCAligned v1 corpus El-
Kishky et al. (2020). CCAligned has received some criticism Kreutzer et al.
(2022), but the available English-Turkish parallel data is very limited. In
Section 7 we analyze the performance of the alignment systems, and we show
that CCAligned does not hurt the performance of the aligners.

###  5.3 Sequence Labelling Models

We use three state-of-the-art multilingual pre-trained language models for
sequence labelling: multilingual BERT (mBERT) Devlin et al. (2019) and XLM-
RoBERTa (XML-R) base and large Conneau et al. (2020). For both models, we add
a token classification layer (linear layer) on top of each token
representation. We use the sequence labelling implementation of the
Huggingface open-source (Apache-2.0 License) library Wolf et al. (2019). F1
scores and standard deviation scores are reported by averaging the results of
5 runs with different random seeds. Details on models sizes, hyper-parameters
and datasets are provided in the Appendix (A, B and C).

##  6 Experiments

###  6.1 Opinion Target Extraction

Opinion Target Extraction (OTE) results are reported in Table 1. The zero-shot
model transfer using mBERT obtains better results for Spanish and French.
However, for Dutch, Russian and Turkish the best results are obtained by the
data-transfer approaches. The overall picture changes when using XLM-RoBERTa
(XLM-R) base. First, the zero-shot baseline is much closer to the gold upper
bound than that of mBERT. This shows that XLM-R has better multilingual
transfer learning capabilities for this task. In fact, the zero-shot transfer
outperforms the translate-train and translate-test approaches for all
languages except Turkish. Second, the XLM-R base results on gold-labelled data
are substantially better than those of mBERT. Finally, XLM-R large offers the
best cross-lingual transfer capabilities, as the zero-shot transfer is clearly
superior for every language, including Turkish.

A common trait for all three models in the OTE benchmark is that the
translate-train approach is superior to the translate-test approach in the
large majority of the cases. As expected, all the approaches achieve a
performance significantly lower than the gold upper bound.

###  6.2 Named Entity Recognition

If we compare the OTE results with those obtained for NER (Table 2), we see a
number of different patterns. First, the zero-shot approach using mBERT
outperforms the data-based cross-lingual transfer methods (translate-train and
translate-test) for the majority of languages . Second, unlike in OTE, the
translate-test is systematically better than translate-train. Third, the mBERT
performance on CoNLL data is similar to that of XLM-R base. Finally, fine-
tuning XLM-R base on translated and projected data obtains better results for
German and Italian than the zero-shot method. However, XLM-R large provides
obtains the same results as for OTE, obtaining the best results for every
language in the zero-shot setting. This validates the findings of the OTE
results, namely, that the performance of the zero-shot method heavily depends
on the characteristics of the multilingual language model used.

mBERT Gold Zero-shot Trans-Train Trans-Test EN 76.2 $\pm$ 0.9 - - - ES 75.2
$\pm$ 0.5 68.4 $\pm$ 0.6 67.9 $\pm$ 0.8 62.2 $\pm$ 1.2 FR 74.0 $\pm$ 1.1 62.7
$\pm$ 1.2 59.7 $\pm$ 1.2 57.6 $\pm$ 1.1 NL 69.7 $\pm$ 0.9 61.7 $\pm$ 0.8 64.3
$\pm$ 1.5 67.0 $\pm$ 0.8 RU 72.5 $\pm$ 0.5 53.8 $\pm$ 2.2 62.9 $\pm$ 0.6 59.7
$\pm$ 0.4 TR 62.0 $\pm$ 1.2 45.3 $\pm$ 4.0 45.7 $\pm$ 2.3 35.5 $\pm$ 2.4 XLM-R
base Gold Zero-shot Trans-Train Trans-Test EN 84.4 $\pm$ 0.9 - - - ES 81.1
$\pm$ 0.7 78.2 $\pm$ 0.4 72.5 $\pm$ 0.7 62.9 $\pm$ 0.9 FR 80.2 $\pm$ 0.6 72.7
$\pm$ 0.3 64.7 $\pm$ 0.8 60.0 $\pm$ 0.6 NL 80.8 $\pm$ 1.7 75.5 $\pm$ 0.8 70.0
$\pm$ 1.6 71.0 $\pm$ 1.5 RU 81.5 $\pm$ 0.3 74.9 $\pm$ 0.9 69.5 $\pm$ 0.3 62.2
$\pm$ 1.6 TR 69.0 $\pm$ 1.1 58.1 $\pm$ 3.5 58.9 $\pm$ 1.8 36.4 $\pm$ 1.8 XLM-R
large Gold Zero-shot Trans-Train Trans-Test EN 86.4 $\pm$ 1.1 - - - ES 83.6
$\pm$ 0.1 79.3 $\pm$ 0.8 73.7 $\pm$ 1.1 64.0 $\pm$ 1.4 FR 82.2 $\pm$ 0.6 74.6
$\pm$ 1.7 66.1 $\pm$ 0.6 60.7 $\pm$ 0.6 NL 80.4 $\pm$ 2.1 77.7 $\pm$ 1.9 74.0
$\pm$ 1.3 72.9 $\pm$ 1.8 RU 82.8 $\pm$ 0.4 76.8 $\pm$ 1.3 69.3 $\pm$ 2.3 62.2
$\pm$ 1.3 TR 72.3 $\pm$ 2.4 62.4 $\pm$ 1.0 57.8 $\pm$ 2.4 33.7 $\pm$ 0.9

Table 1: OTE F1 score with models of different capacity.

mBERT Gold Zero-shot Trans-Train Trans-Test EN 90.7 $\pm$ 0.3 - - - ES 87.4
$\pm$ 0.4 74.6 $\pm$ 0.4 69.5 $\pm$ 0.4 70.8 $\pm$ 0.6 DE 82.0 $\pm$ 0.4 71.0
$\pm$ 0.9 70.1 $\pm$ 0.3 70.6 $\pm$ 0.5 NL 90.8 $\pm$ 0.4 78.5 $\pm$ 0.5 74.4
$\pm$ 0.6 75.4 $\pm$ 0.8 IT 84.7 $\pm$ 0.3 68.2 $\pm$ 0.5 68.7 $\pm$ 0.5 70.7
$\pm$ 0.3 XLM-R base Gold Zero-shot Trans-Train Trans-Test EN 90.4 $\pm$ 0.2 -
- - ES 87.7 $\pm$ 0.2 75.0 $\pm$ 0.4 70.1 $\pm$ 0.6 72.5 $\pm$ 0.2 DE 83.1
$\pm$ 0.3 67.9 $\pm$ 0.5 70.5 $\pm$ 0.5 70.1 $\pm$ 0.8 NL 89.8 $\pm$ 0.2 78.1
$\pm$ 0.6 73.3 $\pm$ 0.9 74.7 $\pm$ 0.4 IT 84.3 $\pm$ 0.3 71.2 $\pm$ 0.5 71.1
$\pm$ 0.4 71.7 $\pm$ 0.3 XLM-R large Gold Zero-shot Trans-Train Trans-Test EN
92.4 $\pm$ 0.1 - - - ES 88.9 $\pm$ 0.2 79.5 $\pm$ 1.0 70.9 $\pm$ 0.6 74.0
$\pm$ 0.5 DE 85.1 $\pm$ 0.6 74.5 $\pm$ 0.7 73.7 $\pm$ 0.5 72.9 $\pm$ 0.3 NL
92.9 $\pm$ 0.7 82.3 $\pm$ 0.6 77.5 $\pm$ 0.9 77.2 $\pm$ 0.6 IT 87.5 $\pm$ 0.2
76.0 $\pm$ 0.5 73.7 $\pm$ 0.4 73.5 $\pm$ 0.6

Table 2: NER F1 score with models of different capacity.

Previous research has demonstrated that cross-lingual transfer with mBERT
works best for topologically similar languages Pires et al. (2019); Wu and
Dredze (2020), which is somewhat coherent with the results obtained for
Spanish and French, where the zero-shot transfer is superior to the Translate-
train and Translate-test approaches, while it is worse for Russian and
Turkish. Additionally, it is worth noting that mBERT has been trained using
only Wikipedia text for 104 languages.

In contrast, XLM-R (both base and large) have been trained using CommonCrawl
Wenzek et al. (2019), a much larger multilingual corpus with a variety of
texts extracted from the Web, perhaps also including texts of similar domain
to those in the OTE datasets. This may also account for the large differences
in OTE performance between XLM-R base and mBERT. In this sense, the similar
performance between mBERT and XLM-R base for NER might be partially due to the
fact that the CoNLL and Evalita datasets consist of news stories in which most
of the labelled entities may appear in the Wikipedia, the texts used to pre-
train mBERT.

The performance of the XLM-R large shows that pretrained models with larger
capacity help to obtain strong performance across languages, also for zero-
shot cross-lingual methods. Still, data-based cross-lingual transfer
(Translate-Train and Translate-Test) approaches remain useful if access to the
required hardware for working with such larger language models is not
available.

Finally, Table 3 lists the results of previous methods that leverage parallel
data and/or annotation projections to perform cross-lingual transfer on the
NER CoNLL 2002-2003 data. By comparing previous work with our zero-shot
baselines using mBERT, XLM-R base and XLM-R large, we can see that the XLM-R
large in the zero-shot setting still outperforms most previous approaches. The
only exception being the results obtained by Li et al. (2021) for German.

Models ES DE NL mBERT Dou and Neubig (2021) 64.3 - - BiLSTM + CRF Jain et al.
(2019) 73.5 61.5 69.9 BiLSTM + CRF Guo and Roth (2021) 77.9 71.4 80.6 XLM-R
large Li et al. (2021) 78.9 76.9 79.7 mBERT (Ours - zero-shot) 74.6 71.0 78.5
XLM-R base (Ours - zero-shot) 75.0 67.9 78.1 XLM-R large (Ours - zero-shot)
79.5 74.5 82.3 XLM-R base (Ours - Translate train) 70.1 70.5 73.3 XLM-R base
(Ours - Translate test) 72.5 70.1 74.7 XLM-R large (Ours - Translate train)
70.9 73.7 77.5 XLM-R large (Ours - Translate test) 74.0 72.9 77.2

Table 3: Comparison between the previous research methods that leverage
projections, the zero-shot baselines and our annotation projections in the
2002-2003 NER CoNLL datasets. F1 score reported

##  7 Error Analysis

The experiments described in Section 6 showed that translate-train and
translate-test perform worse than the zero-shot approach when using XLM-R
large. In this section we will assess the performance of the machine
translation and word alignment models. Furthermore, we will undertake an error
analysis to better understand the shortcomings of translate-train and
translate-test with respect to the zero-shot cross-lingual transfer.

###  7.1 Evaluating the Projection Method

We start our experiments by analyzing the quality of our automatically
projected annotations. In order to do that, human annotators manually
projected the labels from the English OTE gold-labelled data to the automatic
translations to Spanish, French and Russian using DeepL and M2M100 for
Turkish. The annotators are NLP PhD candidates with either native and/or
proficient skills in both English and the target language. See Section E for
more details.

We compare the projections of the annotations automatically generated by the
different word alignment methods with those provided by the human annotators.
Table 4 shows that the language model-based methods (SimAlign and AWESoME)
outperform the statistically based methods (GIZA++ and FastAlign) by a wide
margin in all languages. Furthermore, AWESoME consistently outperforms
SimAlign for every language. The performance of the AWESoME system confirms
that it is possible to generate high quality annotations close to those
generated by human experts. The results also show that for Turkish performance
is lower than for the other languages. This is the case for the methods that
require parallel data (GIZA++, FastAlign and AWESoME) as well as SimALign that
does not require parallel data. So we can attribute the lower performance to
the difficulty of projecting annotations for the English-Turkish pair and not
the usage of the CCAligned corpus.

GIZA++ FastAlign SimAlign AWESoME ES 77.0 75.0 86.7 91.5 FR 73.3 72.9 86.3
91.3 RU 72.4 76.9 87.7 93.4 TR 64.0 68.4 81.9 88.5

Table 4: OTE F1 score between the human annotation projections vs the
automatic projections generated using different alignment models.

While Table 4 shows that we generate high quality annotation projections, the
best model, AWESoME, still makes some mistakes. To explore the effect of these
mistakes we fine-tune XLM-R large models on the manually projected train
datasets and compare their performance on the gold-labelled test sets with the
models trained on the AWESoME automatically projected data. Table 5 shows that
the models obtained using the manually projected data are sightly better,
except for Turkish, which once again acts as outlier. In any case, as the
results obtained by fine-tuning on the manually projected data are still worse
than the zero-shot method, this experiment proves that the projection of
annotations is not responsible for the data-based cross-lingual transfer
methods to be inferior to the zero-shot baseline.

XLMR | Trans-Train | Trans-Train (Manual)
---|---|---
ES | 73.7 $\pm$ 1.1 |  75.1 $\pm$ 1.2
FR | 66.1 $\pm$ 0.6 |  67.9 $\pm$ 1.0
RU | 69.3 $\pm$ 1.3 |  69.4 $\pm$ 2.1
TR |  57.8 $\pm$ 2.4 | 50.6 $\pm$ 1.4
Table 5: XLM-R large OTE F1 score when training with automatically and
manually projected datasets

###  7.2 Downstream Evaluation of Machine Translation Models

In order to evaluate the influence of the machine translation system used, we
translate the English gold-labelled data using four different translation
systems. We fixed AWESoME as the word aligner for annotation projection. We
fine-tune XLM-R large with each of the generated training data and evaluate it
against the gold-labelled test data from OTE. As Table 6 shows, there are no
big differences in the final F1 scores when using different translation
systems (Turkish is again being the exception), we decided to carry on using
DeepL based on the manual assessment mentioned in Section 3.

MarianMT Mbart M2M100 DeepL ES 75.6 $\pm$ 0.8 75.3 $\pm$ 0.7 74.2 $\pm$ 0.8
73.7 $\pm$ 1.1 FR 64.5 $\pm$ 1.6 66.4 $\pm$ 1.1 64.9 $\pm$ 1.3 66.1 $\pm$ 0.6
NL 70.0 $\pm$ 2.0 68.8 $\pm$ 4.0 70.1 $\pm$ 3.1 74.0 $\pm$ 1.3 RU 66.6 $\pm$
4.4 69.7 $\pm$ 1.4 69.7 $\pm$ 0.7 69.3 $\pm$ 2.3 TR 49.5 $\pm$ 2.9 56.1 $\pm$
5.2 57.8 $\pm$ 2.4 -

Table 6: OTE F1 score of different XLM-R large models trained using data
generated with different translation systems.

###  7.3 Where do the models fail?

To better understand what is happening we identify the most common false
negatives and positives for both OTE and NER tasks. Table 7 shows the most
frequent false negatives and positives where there is a big mismatch between
methods.

As it has been previously noticed Agerri and Rigau (2019), in the ABSA data
the words “comida” (food) and “restaurante” (restaurant) are highly ambiguous,
so we could expect the models to fail with these words. In addition, we have
found out 4 main sources of errors, which are analyzed below.

#### Many-to-one translations:

This is stereotypical of targets such as “trato” and “atención” in Spanish,
which, in addition to “servicio”, are used to refer to “service” in English.
There are 160 sentences in the English gold-labelled data containing the word
“service”; in 153 of them “service” is labelled as target. DeepL
systematically translates it as “servicio”. However, as shown by Table 8, in
the Spanish gold-labelled data “service” is also commonly referred as “trato”
or “atención”, instead of “servicio”.

This would result in a training set without any occurrences of “trato” and
“atención” which often occur in the gold-labelled test data. Both the zero-
shot and the data-based cross-lingual transfer approaches fail to correctly
label these words, which shows a problem of using automatically translated
data. Interestingly, the zero-shot approach using XLM-R large correctly
classifies “trato” (only fails to label 1 of the 19 occurrences). As shown by
our experimental results, XLM-R large is more robust than mBERT and XLM-R
base.

Something similar happens with the word “place”, which in Spanish can be most
frequently translated as “lugar” or “sitio”. However, DeepL almost always
translates it as “lugar” which results in “sitio” being absent in the
automatically generated training data while being more frequent than “lugar”
in the gold-labelled data. Note that this is not a problem for the “translate-
test”, given that the translation direction is Spanish to English.

#### Errors induced by incorrect or missing alignments:

For NER we found errors of different nature. Articles and prepositions (i.e.
“de”, “la”) are among the words with higher false positive rate for the
translate-test and translate-train approaches. We can attribute it to word
alignment errors. Large multi-word named entities such as “Consejo General de
la Arquitectura Técnica de España” (General Council of Technical Architecture
of Spain) are labelled as entities. Word aligners struggle to correctly align
articles in these complex expressions specially when a one-to-many or many-to-
one alignment is required. In fact, in this example, the word aligners we
tested failed to correctly align “of” with “de la”.

#### Errors induced by dataset inconsistencies:

Another issue is the differences across languages in the original gold-
labelled annotations. Thus, “Gobierno” (Government) and “Estado” (State) are
labelled as organizations in the Spanish gold-labelled data, but they are not
considered to be entities in the English gold-labelled data. The opposite
occurs with demonym words. They are labelled as miscellaneous entities in the
English data but in Spanish they are not annotated. Cross-lingual models are
likely to fail labelling these cases.

#### Lost in Translation:

Finally, there is another group of words related to Spanish Government names
which are not commonly used in English for the same contexts (i.e. “Economía”
to refer to the“Ministry of Economy” or “Ministerio de Economía” in Spanish,
“Junta” for “local government”, or “Plan” for “government projects”). While
these words appear frequently in the Spanish data as part of commonly used
named entities, that is not the case in the English data, where it is
customary to use “Treasury Department” (or variations thereof) which are
correctly translated into Spanish by DeepL as “Departamento del Tesoro”. This
means that, during fine-tuning on the translated data, the model is not
receiving any signal to learn that “Economy” may be part of an organization
entity. This may explain why the zero-shot method performs better for cases
such as “Economía”, “Hacienda”, “Plan” and “Junta”, listed in Table 7.

GOLD Zero-shot Tr-Train Tr-Test Total B Xb Xl B Xb Xl B Xb Xl B Xb Xl OTE
False Negatives comida 3 3 2 6 2 1 4 1 1 1 9 5 98 restaurante 7 5 7 9 5 6 7 6
6 7 12 10 43 servicio 2 2 2 2 1 1 2 0 1 1 1 2 85 trato 1 1 0 5 6 1 14 10 5 6 8
6 19 atención 2 3 3 8 2 3 7 1 3 7 7 7 13 lugar 0 0 0 2 0 0 1 0 0 0 1 0 12
sitio 1 0 0 5 1 1 3 3 3 2 1 1 14 NER False Negatives de 32 29 33 45 51 90 233
252 264 148 146 167 450 la 4 5 3 10 12 16 63 62 62 45 44 45 174 Gobierno 0 0 0
17 53 64 72 70 75 30 45 67 80 Estado 0 0 0 4 4 8 9 8 9 6 6 8 10 Administación
0 0 0 4 8 11 10 11 11 5 5 7 11 Economía 0 0 0 2 6 2 7 8 8 5 6 8 8 Plan 0 0 0 1
2 2 3 5 5 1 4 7 8 Junta 0 0 0 0 0 0 4 10 8 2 3 5 24 Hacienda 0 0 0 1 3 0 4 4 4
4 3 4 5 NER False Positives español 0 0 0 16 16 2 16 16 12 13 14 15 0 catalán
0 0 0 8 8 5 7 7 8 8 8 8 0

Table 7: Most common false negatives and positives were there is a big
mismatch between methods and the total number of labelled apperances of the
word in the test data. B is the acronym for mBERT, Xb for XLM-R base and Xl
for XLM-R large.

En.Word Es.Word En Gold Es Gold Es Translate Service Servicio 153 229 133
Treatment Trato 0 54 0 Attention Atención 2 35 0 Place Sitio 120 41 2 Place
Lugar 120 19 91

Table 8: Number of times words appear as target words in the train datasets

Summarizing, we see that machine translation data often generates a signal
which is, due to inherent differences in language use, different to the signal
received when using gold-labelled data in the target language. This
disagreement seems to be the most common reason for the larger number of false
positive and negatives of the data-based cross-lingual transfer method with
respect to the zero-shot technique.

##  8 Concluding Remarks

In this paper we described an in-depth and comprehensive evaluation of model-
based and data-based zero-resource cross-lingual sequence labelling on two
different tasks.

Contrary to what previous research suggests, zero-shot transfer approach is
the best performing method when using high capacity multilingual language
models such as XLM-R large. However, data-based cross-lingual transfer
approaches are still useful when having a model with poor downstream cross-
lingual performance. For example, when using a pretrained language model not
trained for a specific domain, or when the required hardware for working with
such larger language models is not readily available.

A detailed error analysis demonstrates that data-based cross-lingual transfer
is hindered by machine translations which, although linguistically sound, do
not align with the cultural behaviour of the target language use. Moreover,
the results also show that the different word alignments methods (for
annotation projection) are of high quality, obtaining comparable results with
respect to manually generated alignments.

In any case, our results establish that there is still room for improving the
cross-lingual performance of zero-resource sequence labelling.

## Acknowledgments

We are grateful to Nayla Escribano, Suna Şeyma Uçar and Olia Toporkov for
their help in manually projecting the labels from the English gold-labelled
data into the automatically translated dataset. We are also thankful to the
anonymous reviewers for their insightful comments. This work has been
supported by the following projects: (i) DeepKnowledge (PID2021-127777OB-C21)
funded by MCIN/AEI/10.13039/501100011033 and FEDER Una manera de hacer Europa;
(ii) Disargue (TED2021-130810B-C21), MCIN/AEI/10.13039/501100011033 and
European Union NextGenerationEU/PRTR; (iii) DeepR3 (TED2021-130295B-C31)
funded by MCIN/AEI/10.13039/501100011033 and EU NextGeneration programme
EU/PRTR; (iv) Antidote (PCI2020-120717-2) funded by
MCIN/AEI/10.13039/501100011033 and by European Union NextGenerationEU/PRTR.
Iker García-Ferrero is supported by a doctoral grant from the Basque
Government (PRE_2021_2_0219). Rodrigo Agerri’s work is also supported by the
RYC-2017-23647 fellowship (MCIN/AEI/10.13039/501100011033 y por El FSE
invierte en tu futuro).

## Limitations

We compare baseline cross-lingual zero-shot model transfer with machine
translation and annotation projection. We do not explore alternative cross-
lingual data-based methods, such as the usage of available parallel corpora
instead of a machine translated corpus. We also skip evaluating methods to
improve model-transfer approaches such as the ones described in Section 2.2.
We may also consider that our annotation projection approach and zero-shot
model transfer approach work for Indo-European languages, while their
performance for other language families remains unknown. Finally, the error
analysis was performed for the EN-ES language pair only.

In any case, we believe that our main claim still holds. Even though MT
quality has substantially improved over the last few years, our results
indicate the current optimal solution to perform cross-lingual transfer is by
using large multilingual language models such as XLM-RoBERTa-large. Thus, our
error analysis suggests that this might be due to important differences in
language use. More specifically, MT often generates a textual signal which is
different to what the models are exposed to when using gold standard data,
which affects both the fine-tuning and evaluation processes. This is confirmed
by our error analysis which shows that mistranslations are not the main source
of errors in the data-transfer method.

## References

  * Agerri et al. (2018) Rodrigo Agerri, Yiling Chung, Itziar Aldabe, Nora Aranberri, Gorka Labaka, and German Rigau. 2018.  Building named entity recognition taggers via parallel corpora.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_ , Miyazaki, Japan. European Language Resources Association (ELRA).
  * Agerri and Rigau (2016) Rodrigo Agerri and German Rigau. 2016.  Robust multilingual named entity recognition with shallow semi-supervised features.  _Artificial Intelligence_ , 238:63–82.
  * Agerri and Rigau (2019) Rodrigo Agerri and German Rigau. 2019.  Language independent sequence labelling for opinion target extraction.  _Artificial Intelligence_ , 268:85–95.
  * Akbik et al. (2018) Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018.  Contextual string embeddings for sequence labeling.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 1638–1649, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Bari et al. (2021) M Saiful Bari, Tasnim Mohiuddin, and Shafiq Joty. 2021.  Uxla: A robust unsupervised data augmentation framework for zero-resource cross-lingual nlp.
  * Conneau et al. (2020) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020.  Unsupervised cross-lingual representation learning at scale.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8440–8451, Online. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_ , pages 4171–4186. Association for Computational Linguistics.
  * Dou and Neubig (2021) Zi-Yi Dou and Graham Neubig. 2021.  Word alignment by fine-tuning embeddings on parallel corpora.  _CoRR_ , abs/2101.08231.
  * Dyer et al. (2013) Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013.  A simple, fast, and effective reparameterization of IBM model 2.  In _Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 9-14, 2013, Westin Peachtree Plaza Hotel, Atlanta, Georgia, USA_ , pages 644–648. The Association for Computational Linguistics.
  * Ehrmann et al. (2011) Maud Ehrmann, Marco Turchi, and Ralf Steinberger. 2011.  Building a multilingual named entity-annotated corpus using annotation projection.  In _Proceedings of the International Conference Recent Advances in Natural Language Processing 2011_ , pages 118–124, Hissar, Bulgaria. Association for Computational Linguistics.
  * El-Kishky et al. (2020) Ahmed El-Kishky, Vishrav Chaudhary, Francisco Guzmán, and Philipp Koehn. 2020.  CCAligned: A massive collection of cross-lingual web-document pairs.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020)_ , pages 5960–5969, Online. Association for Computational Linguistics.
  * Esplà et al. (2019) Miquel Esplà, Mikel Forcada, Gema Ramírez-Sánchez, and Hieu Hoang. 2019\.  ParaCrawl: Web-scale parallel corpora for the languages of the EU.  In _Proceedings of Machine Translation Summit XVII: Translator, Project and User Tracks_ , pages 118–119, Dublin, Ireland. European Association for Machine Translation.
  * Fan et al. (2020) Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, and Armand Joulin. 2020.  Beyond english-centric multilingual machine translation.  _CoRR_ , abs/2010.11125.
  * Fei et al. (2020) Hao Fei, Meishan Zhang, and Donghong Ji. 2020.  Cross-lingual semantic role labeling with high-quality translated training corpus.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7014–7026, Online. Association for Computational Linguistics.
  * Guo and Roth (2021) Ruohao Guo and Dan Roth. 2021.  Constrained labeled data generation for low-resource named entity recognition.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4519–4533, Online. Association for Computational Linguistics.
  * Hu et al. (2021) Zechuan Hu, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, and Kewei Tu. 2021.  Risk minimization for zero-shot sequence labeling.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4909–4920, Online. Association for Computational Linguistics.
  * Jain et al. (2019) Alankar Jain, Bhargavi Paranjape, and Zachary C. Lipton. 2019.  Entity projection via machine translation for cross-lingual NER.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 1083–1092, Hong Kong, China. Association for Computational Linguistics.
  * Joshi et al. (2020) Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020.  The state and fate of linguistic diversity and inclusion in the NLP world.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6282–6293, Online. Association for Computational Linguistics.
  * Junczys-Dowmunt et al. (2018) Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, André F. T. Martins, and Alexandra Birch. 2018.  Marian: Fast neural machine translation in C++.  In _Proceedings of ACL 2018, System Demonstrations_ , pages 116–121, Melbourne, Australia. Association for Computational Linguistics.
  * Kreutzer et al. (2022) Julia Kreutzer, Isaac Caswell, Lisa Wang, Ahsan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allahsera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoît Sagot, Clara Rivera, Annette Rios, Isabel Papadimitriou, Salomey Osei, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Andre Niyongabo Rubungo, Toan Q. Nguyen, Mathias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni, Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F. P. Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Ballı, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefego Ahia, Sweta Agrawal, and Mofetoluwa Adeyemi. 2022.  Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets.  _Transactions of the Association for Computational Linguistics_ , 10:50–72.
  * Lample et al. (2016) Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016.  Neural architectures for named entity recognition.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 260–270, San Diego, California. Association for Computational Linguistics.
  * Li et al. (2021) Bing Li, Yujie He, and Wenjin Xu. 2021.  Cross-lingual named entity recognition using parallel corpus: A new approach using xlm-roberta alignment.  _CoRR_ , abs/2101.11112.
  * Liu et al. (2021) Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto, and Pascale Fung. 2021.  Crossner: Evaluating cross-domain named entity recognition.  In _Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021_ , pages 13452–13460. AAAI Press.
  * Ni et al. (2017) Jian Ni, Georgiana Dinu, and Radu Florian. 2017.  Weakly supervised cross-lingual named entity recognition via effective annotation and representation projection.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1470–1480, Vancouver, Canada. Association for Computational Linguistics.
  * Och and Ney (2003) Franz Josef Och and Hermann Ney. 2003.  A systematic comparison of various statistical alignment models.  _Computational Linguistics_ , 29(1):19–51.
  * Ouyang et al. (2021) Xuan Ouyang, Shuohuan Wang, Chao Pang, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2021.  Ernie-m: Enhanced multilingual representation by aligning cross-lingual semantics with monolingual corpora.
  * Pires et al. (2019) Telmo Pires, Eva Schlinger, and Dan Garrette. 2019.  How multilingual is multilingual BERT? In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4996–5001, Florence, Italy. Association for Computational Linguistics.
  * Pontiki et al. (2016) Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, Mohammad AL-Smadi, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clercq, Véronique Hoste, Marianna Apidianaki, Xavier Tannier, Natalia Loukachevitch, Evgeniy Kotelnikov, Nuria Bel, Salud María Jiménez-Zafra, and Gülşen Eryiğit. 2016.  SemEval-2016 task 5: Aspect based sentiment analysis.  In _Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)_ , pages 19–30, San Diego, California. Association for Computational Linguistics.
  * Rahimi et al. (2019) Afshin Rahimi, Yuan Li, and Trevor Cohn. 2019.  Massively multilingual transfer for NER.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 151–164, Florence, Italy. Association for Computational Linguistics.
  * Speranza (2009) Manuela Speranza. 2009.  The named entity recognition task at evalita 2009.  In _EVALITA 2009_.
  * Tang et al. (2020) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020.  Multilingual translation with extensible multilingual pretraining and finetuning.  _CoRR_ , abs/2008.00401.
  * Tiedemann and Thottingal (2020) Jörg Tiedemann and Santhosh Thottingal. 2020.  OPUS-MT – building open translation services for the world.  In _Proceedings of the 22nd Annual Conference of the European Association for Machine Translation_ , pages 479–480, Lisboa, Portugal. European Association for Machine Translation.
  * Tjong Kim Sang (2002) Erik F. Tjong Kim Sang. 2002.  Introduction to the CoNLL-2002 shared task: Language-independent named entity recognition.  In _COLING-02: The 6th Conference on Natural Language Learning 2002 (CoNLL-2002)_.
  * Tjong Kim Sang and De Meulder (2003) Erik F. Tjong Kim Sang and Fien De Meulder. 2003.  Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition.  In _Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003_ , pages 142–147.
  * Wang and Manning (2014) Mengqiu Wang and Christopher D. Manning. 2014.  Cross-lingual projected expectation regularization for weakly supervised learning.  _Transactions of the Association for Computational Linguistics_ , 2:55–66.
  * Wang et al. (2019) Zirui Wang, Jiateng Xie, Ruochen Xu, Yiming Yang, Graham Neubig, and Jaime Carbonell. 2019.  Cross-lingual alignment vs joint training: A comparative study and a simple unified framework.  _arXiv preprint arXiv:1910.04708_.
  * Wenzek et al. (2019) Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, and Edouard Grave. 2019.  Ccnet: Extracting high quality monolingual datasets from web crawl data.  _CoRR_ , abs/1911.00359.
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2019.  Huggingface’s transformers: State-of-the-art natural language processing.  _CoRR_ , abs/1910.03771.
  * Wu et al. (2020) Qianhui Wu, Zijia Lin, Börje Karlsson, Jian-Guang Lou, and Biqing Huang. 2020\.  Single-/multi-source cross-lingual NER via teacher-student learning on unlabeled data in target language.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6505–6514, Online. Association for Computational Linguistics.
  * Wu and Dredze (2020) Shijie Wu and Mark Dredze. 2020.  Are all languages created equal in multilingual BERT? In _Proceedings of the 5th Workshop on Representation Learning for NLP_ , pages 120–130, Online. Association for Computational Linguistics.
  * Xie et al. (2018) Jiateng Xie, Zhilin Yang, Graham Neubig, Noah A. Smith, and Jaime Carbonell. 2018\.  Neural cross-lingual named entity recognition with minimal resources.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 369–379, Brussels, Belgium. Association for Computational Linguistics.

##  Appendix A Model size

We experiment with multilingual BERT (mBERT) Devlin et al. (2019) and XLM-
RoBERTa (XML-R) base and large Conneau et al. (2020). We list the number of
parameters of each model in Table 9

Model | #params
---|---
multilingual BERT | 110M
XLM-RoBERTa-base | 250M
XLM-RoBERTa-large | 560M
Table 9: Number of parameters for the language models that we use in our
experiments

##  Appendix B Hyper parameters

###  B.1 Word alignment models

We train AWESoME with $8$ batch size and $2e-5$ learning rate for $40,000$
steps, with all the unsupervised training objectives (mlm,tlm,tlm_full,so,psi)
and softmax extraction method. We use mBERT as backbone. For SimAlign we run
inference with 0.0 distortion rate, 1.0 null align rate and the "itermax"
matching method. We use bpe tokens and mBERT backbone. We use the MGIZA
multicore implementation 333https://github.com/moses-smt/mgiza of GIZA++ with
the recommended configuration file 444https://pastebin.com/b1ksHtUy. We use
FastAlign with the default hyper-parameters. For both, GIZA++ and FastAlign we
combine the forward and backward directions of the alignments using the grow-
diag-final-and algorithm.

###  B.2 Sequence Labelling models

For OTE we use a batch size of $32$, $5e-5$ learning rate, we train the model
for $10$ epochs and $128$ maximum sequence length. Since only a train and test
splits are available for the OTE task, we use the train set as both, train and
development data. For NER we use a batch size of $32$, $2e-5$ learning rate,
we train the model for $4$ epochs and 256 maximum sequence length. We use the
default values (sequence labelling implementation of the Huggingface library
555https://github.com/huggingface/transformers/tree/main/examples/pytorch/token-
classification) for the remaining hyperparameters. For both tasks we use the
BILOU encoding scheme.

##  Appendix C Datasets Size

We list the dataset size (number of sentences) of the datasets we use.

For OTE we use the SemEval-2016 Task 5 Aspect Based Sentiment Analysis (ABSA)
datasets Pontiki et al. (2016). We list the size of the datasets in Table 10.

Lang | Train | Test
---|---|---
EN | 2000 | 676
ES | 2070 | 881
FR | 1664 | 668
NL | 1722 | 575
RU | 3655 | 1209
TR | 1232 | 144
Table 10: Number of sentences in the OTE datasets

For NER we use the Spanish and Dutch data from the CoNLL-2002 datasets Tjong
Kim Sang (2002). For English and German we use the CoNLL-2003 datasets Tjong
Kim Sang and De Meulder (2003) and for Italian we use Evalita 2009 data
Speranza (2009). We list the size of these datasets in Table 11.

| Train | Dev | Test
---|---|---|---
EN | 14987 | 3466 | 3684
ES | 6871 | 1914 | 1516
DE | 12705 | 3068 | 3160
NL | 15806 | 2895 | 5195
IT | 11227 | 0 | 4136
Table 11: Number of sentences in the NER datasets

##  Appendix D Computer infrastructure

We perform all our experiments using a single NVIDIA A30 GPU with 24GB memory.
The machine used has two Xeon Gold 6226R CPUs and 256GB RAM.

##  Appendix E Manual Projection of the datasets

Human annotators manually projected the labels from the English OTE gold data
to the automatic translations to Spanish, French and Russian using DeepL and
m2m10 for Turkish The annotators are NLP PhD candidates with either native
and/or proficient skills in both English and the target language. We describe
the experiment in Section 7.1. For the purpose of this experiment, we
developed an application to assist during the annotation process. The
annotator sees the sentence in English, where there is a highlighted target
and must select the same target in a translated target sentence. Figure 4
shows two screenshots from the application. The full guidelines and the code
of the application provided to the annotators are available at
https://github.com/ikergarcia1996/Annotation-Projection-App.

Figure 4: Application used to manually annotate the projections


# Generic properties of purely $\alpha$-Hausdorff continuous operators

M. Aloisio, S. L. Carvalho, C. R. de Oliveira and G. Santana Corresponding
author.

(May 2021)

###### Abstract

It is shown that the time-average quantum return probability of (Baire)
generic states of systems with purely $\alpha$-Hausdorff continuous spectrum
has an oscillating behaviour between a (maximum) fast power-law decay and a
(minimum) slow power-law decay. As an application, a new result on the quantum
dynamics of Schrödinger operators with limit-periodic potentials is presented.

##  1 Introduction

Let $T$ be a self-adjoint operator in a separable complex Hilbert space
$\mathcal{H}$. The (time-average) quantum return probability, which gives the
(time-average) probability of finding the system at time $t>0$ in its initial
state $\psi\in\mathcal{H}$, is given by

$W_{\psi}^{T}(t):=\frac{1}{t}\int_{0}^{t}|\langle\psi,e^{-isT}\psi\rangle|^{2}\,{\mathrm{d}}s.$

The study of the large time behaviour of this dynamic quantity plays an
important role in spectral theory and quantum dynamics; for instance, “good”
continuity properties of the spectral measure $\mu_{\psi}^{T}$ of the pair
$(T,\psi)$ are associated with “good” transport properties, in the sense of
the following theorem. Recall that $\mu_{\psi}^{T}$ is uniformly
$\alpha$-Hölder continuous (denoted U$\alpha$H) if there exists a constant
$C>0$ such that, for every interval $I\subset\mathbb{R}$ with $|I|<1$, one has
$\mu_{\psi}^{T}(I)<C|I|^{\alpha}$, where $|\cdot|$ denotes the Lebesgue
measure.

######  Theorem 1.1 (Theorem 3.1 in [20]).

Let $\alpha\in[0,1]$.

  1. (i)

If $\mu_{\psi}^{T}$ is U$\alpha$H, then there exists a positive constant
$C_{\psi}$ such that, for every $t>0$,

$W_{\psi}^{T}(t)<C_{\psi}t^{-\alpha}.$

  2. (ii)

If there exists a positive constant $C_{\psi}$ such that, for every $t>0$,

$W_{\psi}^{T}(t)<C_{\psi}t^{-\alpha},$

then $\mu_{\psi}^{T}$ is U$\frac{\alpha}{2}$H.

Part (i) of Theorem 1.1 is, indeed, a particular case of a result by
Strichartz [21].

This notion of $\alpha$-Hölder continuity is, in some sense, related to the
notion of $\alpha$-Hausdorff continuity. Given $\alpha\in[0,1]$, recall that a
positive Borel measure $\mu$ on $\mathbb{R}$ is $\alpha$-Hausdorff continuous
(denoted $\alpha\mathrm{c}$) if $\mu(\Lambda)=0$ for any Borel set
$\Lambda\subset\mathbb{R}$ with $h^{\alpha}(\Lambda)=0$, and that $\mu$ is
$\alpha$-Hausdorff singular (denoted $\alpha\mathrm{s}$) if there exists a
Borel set $\Lambda$ with $h^{\alpha}(\Lambda)=0$ such that
$\mu({\mathbb{R}}\backslash\Lambda)=0$, where $h^{\alpha}$ denoted the
$\alpha$-dimensional Hausdorff (exterior) measure [18].

######  Definition 1.1.

Let $T$ be a self-adjoint operator. We say that $T$ has purely
$\alpha$-Hausdorff continuous spectrum if, for each $\psi\in\mathcal{H}$, the
spectral measure $\mu_{\psi}^{T}$ of the pair $(T,\psi)$ is $\alpha$-Hausdorff
continuous.

For $\alpha\in[0,1]$ and a self-adjoint operator $T$, the $\alpha$-Hausdorff
continuous and $\alpha$-Hausdorff singular subspaces induced by $T$ are
defined, respectively, by

${\mathcal{H}}_{\alpha\mathrm{c}}^{T}:=\\{\psi\in{\mathcal{H}}\mid\mu_{\psi}^{T}\
{\rm is}\ \alpha\mathrm{c}\\}\quad{\rm\ and\
}\quad{\mathcal{H}}^{T}_{\alpha\mathrm{s}}:=\\{\psi\in{\mathcal{H}}\mid\mu_{\psi}^{T}\
{\rm is}\ \alpha\mathrm{s}\\}.$

Such subspaces are closed (in norm), mutually orthogonal, invariant under $T$
and
${\mathcal{H}}={\mathcal{H}}_{\alpha\textrm{c}}^{T}\oplus{\mathcal{H}}^{T}_{\alpha\text{s}}$
(see [20] for details). Set

${\mathcal{H}}^{T}_{\text{uH}}(\alpha):=\\{\psi\in{\mathcal{H}}\mid\mu_{\psi}^{T}\
{\rm is}\ {\rm U}\alpha{\rm H}\\}.$

The next result, due to Last, establishes the aforementioned connection
between U$\alpha$H and $\alpha$c measures.

######  Theorem 1.2.

Let $\alpha\in[0,1]$ and let $T$ be a bounded self-adjoint operator. Then,
${\mathcal{H}}_{\mathrm{uH}}^{T}(\alpha)$ is a subspace of $\mathcal{H}$ and

$\overline{{\mathcal{H}}_{\mathrm{uH}}^{T}(\alpha)}={\mathcal{H}}_{\alpha\mathrm{c}}^{T},$

where $\overline{\ \ \cdot\ \ }$ denotes the norm-closure in $\mathcal{H}$.

Suppose that $T$ has purely $\alpha$-Hausdorff continuous spectrum and let
$\psi\in\mathcal{H}$. Taking into account Theorems 1.1 and 1.2, one may ask
the following questions about the (Baire) typical behaviour of the spectral
measure $\mu_{\psi}^{T}$:

  1. (i)

What can be said about the modulus of continuity of the distribution of
$\mu_{\psi}^{T}$?

  2. (ii)

What can be said about the asymptotic behaviour of $W_{\psi}^{T}(t)$ as
$t\to\infty$?

In this work, we show that for a (Baire) typical $\psi\in\mathcal{H}$,
$\mu_{\psi}^{T}$ does not have any kind of Hölder regularity and that the
(time-average) quantum return probability $W^{T}_{\psi}(t)$ has an oscillating
behaviour between a (maximum) fast power-law decay and a (minimum) slow power-
law decay. More precisely, we prove the following result.

######  Theorem 1.3.

Let $\alpha\in[0,1]$ and $T$ be a bounded self-adjoint operator with purely
$\alpha$-Hausdorff continuous spectrum. Then, the set of $\psi\in\mathcal{H}$
such that, for all $k\in\mathbb{N}$,

  1. (i)

$\mu_{\psi}^{T}$ is not U$(1/k)$H,

  2. (ii)

$\liminf_{t\to\infty}t^{\alpha-1/k}W_{\psi}^{T}(t)=0\quad
and\quad\limsup_{t\to\infty}t^{1/k}W_{\psi}^{T}(t)=\infty,$

is generic in $\mathcal{H}$ (that is, such set contains a $G_{\delta}$ dense
subset of $\mathcal{H}$).

######  Remark 1.1.

We note that due to Theorems 1.1 and 1.2, the states in the $\alpha$-Hausdorff
continuous subspace ${\mathcal{H}}_{\alpha{\mathrm{c}}}$ may be well
approximated by states for which $W_{\psi}^{T}(t)$ has a fast power-law decay,
that is,

${\mathcal{H}}_{\alpha{\mathrm{c}}}\subset\overline{\\{\psi\mid\sup_{t>0}t^{\alpha}W_{\psi}^{T}(t)<\infty\\}},$

where $\overline{\ \ \cdot\ \ }$ denotes the norm-closure in $\mathcal{H}$.
Nevertheless, Theorem 1.3 shows that if
$\eta\in\big{\\{}\psi\mid\sup_{t>0}t^{\alpha}W_{\psi}^{T}(t)<\infty\big{\\}}$
and $\xi\in{\mathcal{H}}_{\alpha{\mathrm{c}}}$, then the asymptotic behaviours
of $W_{\eta}^{T}(t)$ and $W_{\xi}^{T}(t)$ can be quite different; in
particular, it is not possible to remove the closure in the inclusion above.

Before we proceed, some words about notation are required: $\mathcal{H}$ will
always denote an infinite dimensional and separable complex Hilbert space, and
if $T$ is a self-adjoint operator in $\mathcal{H}$, then $\mathcal{D}(T)$,
${\rm rng}\,T$ and $\sigma(T)$ denote, respectively, its domain, range and
spectrum. For each Borel set $\Lambda\subset\mathbb{R}$, $P^{T}(\Lambda)$
represents the spectral resolution of $T$ over $\Lambda$. $\mu$ always
indicates a finite positive Borel measure on $\mathbb{R}$. For each
$x\in\mathbb{R}$ and each $\epsilon>0$, $B(x;\epsilon)$ denotes the open
interval $(x-\epsilon,x+\epsilon)$.

In order to study the asymptotic behaviour of $W_{\psi}^{T}(t)$, we recall the
notions of lower and upper $q$-generalized fractal dimensions of $\mu$, $q>0$
and $q\neq 1$, defined respectively as

$D_{\mu}^{-}(q):=\liminf_{\epsilon\downarrow
0}\frac{\ln\biggr{[}\int\mu(B(x;\epsilon))^{q-1}{\mathrm{d}}\mu(x)\biggr{]}}{(q-1)\ln\epsilon}\quad{\rm
and}\quad D_{\mu}^{+}(q):=\limsup_{\epsilon\downarrow
0}\frac{\ln\biggr{[}\int\mu(B(x;\epsilon))^{q-1}{\mathrm{d}}\mu(x)\biggr{]}}{(q-1)\ln\epsilon},$

with the integrals taken over the support of $\mu$.

The next identities, extracted from [6, 19], show that the correlation
dimensions (i.e., generalized fractal dimensions for $q=2$) rule the time-
average power-law decay rates of $W_{\psi}^{T}(t)$, that is,

$\liminf_{t\to\infty}\frac{\ln W_{\psi}^{T}(t)}{\ln
t}=-D^{+}_{\mu^{T}_{\psi}}(2),$ (1)

$\limsup_{t\to\infty}\frac{\ln W_{\psi}^{T}(t)}{\ln
t}=-D^{-}_{\mu^{T}_{\psi}}(2).$ (2)

As a direct consequence of Theorem 1.3 and the above identities, one has the
following result.

######  Corollary 1.1.

Let $\alpha\in[0,1]$ and $T$ be a bounded self-adjoint operator with purely
$\alpha$-Hausdorff continuous spectrum. Then,

$\big{\\{}\psi\in\mathcal{H}\mid
D^{+}_{\mu^{T}_{\psi}}(2)\geq\alpha\quad\textrm{and}\quad
D^{-}_{\mu^{T}_{\psi}}(2)=0\big{\\}}$

is generic in $\mathcal{H}$.

######  Remark 1.2.

For $\alpha=1$ (that is, when $T$ has purely absolutely continuous spectrum),
Theorem 1.3 states that the typical dynamical situation is characterized by
extreme values of the power-law decay rates of the (time-average) quantum
return probabilities. Here, for a typical vector $\psi$ one has
$D^{-}_{\mu^{T}_{\psi}}(2)=0$, and so the return probability decays (power-
law) as slow as possible for a sequence of time going to infinity, though the
spectrum is purely absolutely continuous. Note that Corollary 1.1 for purely
absolutely continuous spectrum can be seen as the counterpart of the following
situation detailed in [2]: for operators with pure point spectrum dense in an
interval, there exists a generic set of states $\psi$ whose spectral measures
have maximal upper generalized dimension $D^{+}_{\mu^{T}_{\psi}}(q)=1$
$(0<q<1)$; such states are, therefore, (weakly) delocalized (see [2, 4, 15,
17]).

There are in the literature (see, for instance, [9, 10, 12]), numerous
examples for which our general results apply. As an illustration, we present
examples of Schrödinger operators for which the hypotheses of Theorem 1.3 and
Corollary 1.1 are satisfied.

######  Example 1.1 (Free Hamiltonian).

Let $H_{0}:\mathcal{H}^{2}(\mathbb{R})\subset{\rm
L}^{2}(\mathbb{R})\rightarrow{\rm L}^{2}(\mathbb{R})$ be given by the law
$(H_{0}\psi)(x)=-\psi^{\prime\prime}(x)$, and set $H:=H_{0}P^{H_{0}}([0,1])$
(a bounded self-adjoint operator). $H_{0}$ has purely absolutely continuous
spectrum; see [14] for details.

######  Example 1.2 (Sturmian operators).

Consider the class of Schrödinger operators with sturmian potentials, that is,
discrete one-dimensional Schrödinger operators, denoted by
$H_{\lambda,\theta,\beta}$, on $\ell^{2}(\mathbb{Z})$ whose actions are given
by

$(H_{\lambda,\theta,\beta}u)_{n}=u_{n+1}+u_{n-1}+\lambda
v_{n}^{\theta,\beta}u_{n},$

where

$\lambda v_{n}^{\theta,\beta}=\chi_{[1-\theta,1)}(n\theta+\beta{\rm mod}1),$

with coupling constant $\lambda\in\mathbb{R}$, rotation number
$\theta\in\mathbb{T}$ and phase $\beta\in\mathbb{T}$.

Let $\theta\in\mathbb{T}$ be a (irrational) number of bounded density. Then
for every $\lambda\neq 0$, there exists $0<\alpha(\lambda,\theta)<1$ such that
for every $\beta\in\mathbb{T}$, $H_{\lambda,\theta,\beta}$ has purely
$\alpha$-Hausdorff continuous spectrum; see [12] for details.

Application to limit-periodic operators. Let the class of Schrödinger
operators with limit-periodic potentials, that is, discrete one-dimensional
ergodic Schrödinger operator defined on $\ell^{2}(\mathbb{Z})$ by the law

$(H^{w}_{g,\tau}u)_{n}=u_{n+1}+u_{n-1}+v_{n}(w)u_{n},$ (3)

where

$v_{n}(w)=g(\tau^{n}(w));$ (4)

here, $w$ belongs to a Cantor group $\Omega$ (an abelian topological group
which is compact, totally disconnected and perfect, i.e., it has no isolated
points), $\tau:\Omega\longrightarrow\Omega$ is a minimal translation on
$\Omega$ and $g:\Omega\rightarrow\mathbb{R}$ is a continuous sampling function
(that is, $g\in C(\Omega,\mathbb{R})$, where $C(\Omega,\mathbb{R})$ is endowed
of the norm of uniform convergence). For more details, see [5, 7, 10]. For
each $w\in\Omega$, let $X_{w}$ be the set of limit-periodic operators
$H^{w}_{g,\tau}$ given by (3) and (4), endowed with the metric

$d(H^{w}_{g,\tau},H^{w}_{g^{\prime},\tau})=\|g-g^{\prime}\|_{\infty}.$

######  Theorem 1.4.

There exists a generic set $\mathcal{M}\subset\ell^{2}(\mathbb{Z})$ so that,
for each $\psi\in\mathcal{M}$, the set of operators $H^{w}_{g,\tau}\in X_{w}$
such that, for each $k\geq 1$,

$\liminf_{t\to\infty}t^{1-1/k}W_{\psi}^{T}(t)=0\quad{and}\quad\limsup_{t\to\infty}t^{1/k}W_{\psi}^{T}(t)=\infty,$

is generic in $X_{w}$.

###### Proof.

Let $C_{\rm ac}:=\\{T\in X_{w}\mid\sigma(T)$ is with purely absolutely
continuous$\\}$. Since $(X_{w},d)$ is a separable space, $C_{{\rm ac}}\subset
X_{w}$ (endowed with the induced topology) is also separable. Let $(T_{j})$ be
a dense sequence in $C_{{\rm ac}}$ (being, therefore, dense in $X_{w}$, since
$C_{{\rm ac}}$ is dense in $X_{w}$ by Theorem 1.1 in [11]). If
$\mu^{j}_{\psi}$ denotes the spectral measure of the pair $(T_{j},\psi)$, it
follows from Corollary 1.1 that

$\mathcal{M}=\displaystyle\bigcap_{j}\big{\\{}\psi\in\ell^{2}(\mathbb{Z})\mid
D^{+}_{\mu^{j}_{\psi}}(2)=1\ \text{and}\ D^{-}_{\mu^{j}_{\psi}}(2)=0\big{\\}}$

is generic in $\ell^{2}(\mathbb{Z})$. Since, by Proposition 2.4 in [4], for
every $\psi\in\mathcal{M}$,

$X_{01}(\psi):=\big{\\{}T\mid D^{+}_{\mu^{T}_{\psi}}(2)=1\ \text{and}\
D^{-}_{\mu^{T}_{\psi}}(2)=0\big{\\}}\supset\cup_{j}\\{T_{j}\\}$

is a set $G_{\delta}$ in $X_{w}$, it follows that $X_{01}(\psi)$ is also a
generic in $X_{w}$.

The proof is now a direct consequence of identities (1) and (2). ∎

######  Remark 1.3.

It is worth underlying that if it is true that there exists a dense subset of
$C(\Omega,\mathbb{R})$ such that $H_{g,\tau}^{w}$ has pure point spectrum for
every $w\in\Omega$, then the result in Theorem 1.4 will be a direct
consequence of Theorem 1.2 in [4]. But, to the best of our knowledge, this is
still an open problem, even for (Haar-)a.e. $w\in\Omega$.

The organization of this paper is as follows. In Section 2, we present a study
of the typical local scale spectral properties of self-adjoint operators with
purely continuous spectrum. More precisely, we show that the local scale
properties of a purely continuous spectral measure, typically in Baire sense,
display a erratic behaviour (see Theorem 2.1 ahead). In Section 3, we use such
result to prove Theorem 1.3.

##  2 Local properties of spectral measures and dynamics

Recall that the pointwise lower and upper scaling exponents of the measure
$\mu$ at $x\in\mathbb{R}$ are defined, respectively, by

$d_{\mu}^{-}(x):=\liminf_{\epsilon\downarrow
0}\frac{\ln\mu(B(x;\epsilon))}{\ln\epsilon}\quad{\rm and}\quad
d_{\mu}^{+}(x):=\limsup_{\epsilon\downarrow
0}\frac{\ln\mu(B(x;\epsilon))}{\ln\epsilon}$

if, for all small enough $\epsilon>0$, $\mu(B(w,\epsilon))>0$; one sets
$d_{\mu}^{\mp}(x):=\infty$, otherwise.

Note that, for each $x\in\mathbb{R}$ and each $\epsilon>0$,

$\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu(y)\geq\int_{B(x;{1/t})}e^{-2t|x-y|}{\mathrm{d}}\mu(y)\geq
e^{-2}\mu(B(x;{1/t})).$

On the other hand, for each $0<\delta<1$ and each $t>0$,

$\displaystyle\\!\\!\\!\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu(y)$
$\displaystyle=$
$\displaystyle\int_{B(x;\frac{1}{t^{1-\delta}})}e^{-2t|x-y|}{\mathrm{d}}\mu(y)+\int_{B(x;\frac{1}{t^{1-\delta}})^{c}}e^{-2t|x-y|}{\mathrm{d}}\mu(y)$
(5) $\displaystyle\leq$
$\displaystyle\mu\big{(}B\big{(}x,{1/t^{1-\delta}}))+e^{-t^{\delta}}\mu(\mathbb{R}).$

Thus, at least when $\mu$ has a certain local regularity (with respect to the
Lebesgue measure), we expect that
$\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu(y)$ and $\mu(B(x;{1/t}))$ are
asymptotically comparable as $t\to\infty$. In this sense, the following
identities are expected:

$\liminf_{t\to\infty}\frac{\ln[\int_{\mathbb{R}}e^{-2t|w-y|}{\mathrm{d}}\mu(y)]}{\ln
t}=-d_{\mu}^{+}(w),$ (6)
$\limsup_{t\to\infty}\frac{\ln[\int_{\mathbb{R}}e^{-2t|w-y|}{\mathrm{d}}\mu(y)]}{\ln
t}=-d_{\mu}^{-}(w).$ (7)

Indeed, these identities were proven in [3] (note that since it is not
possible to compare directly the two terms on the right-hand side of (5), some
caution should be exercised when checking (6) and (7)). We use such identities
in the proof of Theorem 2.1.

######  Theorem 2.1.

Let $T$ be a bounded self-adjoint operator with purely continuous spectrum.
Then, for each $x\in\sigma(T)$, the set

${\mathcal{G}}(x):=\big{\\{}\psi\in\mathcal{H}\mid
d^{-}_{\mu_{\psi}^{T}}(x)=0\ \ \text{and}\ \
d^{+}_{\mu_{\psi}^{T}}(x)=\infty\big{\\}}$

is generic in $\mathcal{H}$.

######  Remark 2.1.

Let $0\neq\psi\in\mathcal{H}$. Since, by dominated convergence, for every
$t>0$ the mapping

$\sigma(T)\ni
x\mapsto\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu_{\psi}^{T}(y)$

is continuous, it follows that

${\mathcal{J}}_{\psi}:=\big{\\{}x\in\sigma(T)\mid
d^{+}_{\mu_{\psi}^{T}}(x)=\infty\,\,\,{\rm
and}\,\,\,d^{-}_{\mu_{\psi}^{T}}(x)=0\big{\\}}\;=\;A_{-}\cap A_{+},$

is a $G_{\delta}$ set in $\sigma(T)$, since

$A_{-}=\bigcap_{l\geq 1}\bigcap_{n\geq 1}\bigcap_{k\geq 1}\bigcup_{t\geq
k}\Big{\\{}x\in\sigma(T)\mid
t^{l}\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu_{\psi}^{T}(y)<\frac{1}{n}\Big{\\}}$

and

$A_{+}=\bigcap_{l\geq 1}\bigcap_{n\geq 1}\bigcap_{k\geq 1}\bigcup_{t\geq
k}\Big{\\{}x\in\sigma(T)\mid
t^{{1}/{l}}\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu_{\psi}^{T}(y)>n\Big{\\}}.$

Now let $(x_{n})_{n\in\mathbb{N}}\subset\sigma(T)$ be a dense sequence in
$\sigma(T)$. So, if

$\psi\in\bigcap_{n}{\mathcal{G}}(x_{n})=\\{\varphi\mid
d^{-}_{\mu_{\varphi}^{T}}(x_{n})=0{\rm\,\,and\,\,}d^{+}_{\mu_{\varphi}^{T}}(x_{n})=\infty,{\rm\,\,for\,\,each\,\,}n\in\mathbb{N}\\},$

it follows that ${\mathcal{J}}_{\psi}$ is generic in $\sigma(T)$. We note that
this remark is particularly interesting when $T$ has purely absolutely
continuous spectrum, since it shows the striking difference between the
typical behaviour of $d^{\pm}_{\mu_{\psi}^{T}}$ from the topological and
measure points of view; namely, if $\mu_{\psi}^{T}$ is purely absolutely
continuous, then
$\mu_{\psi}^{T}$-$\operatorname*{ess.inf}d_{\mu_{\psi}^{T}}^{-}=1$ (see [18]
for details).

######  Example 2.1.

Let $\Delta:\mathcal{H}^{2}(\mathbb{R}^{n})\subset
L^{2}(\mathbb{R}^{n})\rightarrow L^{2}(\mathbb{R}^{n})$, $n\geq 5$, be the
self-adjoint realization of the Laplacian. $\Delta$ has purely absolutely
continuous spectrum [14], and so, the hypotheses of Theorems 1.3 and 2.1 are
satisfied. We note that, in this example, reasonably localized functions, in
general, do not belong to the sets given in the previous results. For
instance, ${\mathcal{G}}(0)\cap\mathcal{S}(\mathbb{R}^{n})=\emptyset$, where
$\mathcal{S}(\mathbb{R}^{n})$ denote the Schwartz space. Namely, consider the
following statement.

Claim: If $n\geq 5$, then $\mathcal{S}(\mathbb{R}^{n})$ is a subspace of
rng$\,\Delta$.

###### Proof.

Let $M_{0}:\mathcal{D}(M_{0})\subset L^{2}(\mathbb{R}^{n})\rightarrow
L^{2}(\mathbb{R}^{n})$, $\mathcal{D}(M_{0})=\\{u\in
L^{2}(\mathbb{R}^{n})\mid|x|^{2}u\in L^{2}(\mathbb{R}^{n})\\}$, be defined,
for each $u\in\mathcal{D}(M_{0})$, by the action $(M_{0}u)(x)=|x|^{2}u$, with
$x\in\mathbb{R}^{n}$.

For each $f\in\mathcal{S}(\mathbb{R}^{n})$, let $u(x)=\frac{f(x)}{|x|^{2}}$,
$u(0)=0$. We assert that $u\in L^{2}(\mathbb{R}^{n})$; indeed,

$\displaystyle\int_{\mathbb{R}^{n}}\frac{|f(x)|^{2}}{|x|^{4}}{\mathrm{d}}x$
$\displaystyle\leq$
$\displaystyle\|f\|^{2}_{\infty}\int_{B(0;1)}\frac{1}{|x|^{4}}{\mathrm{d}}x+\int_{B(0;1)^{c}}\frac{|f(x)|^{2}}{|x|^{4}}{\mathrm{d}}x$
$\displaystyle\leq$
$\displaystyle\|f\|^{2}_{\infty}\,n\,w_{n}\int_{0}^{1}\frac{r^{n-1}}{r^{4}}{\mathrm{d}}r+\|f\|^{2}_{L^{2}(\mathbb{R}^{n})}<\infty,$

since $n\geq 5$. Therefore, $u\in\mathcal{D}(M_{0})$ and $M_{0}u=f$, from
which follows that $\mathcal{S}(\mathbb{R}^{n})\subset\text{rng\,}M_{0}$.
Finally, consider the Fourier transform
$\mathcal{F}:L^{2}(\mathbb{R}^{n})\rightarrow L^{2}(\mathbb{R}^{n})$. Then,
since
$\mathcal{F}^{-1}\left(\mathcal{S}(\mathbb{R}^{n})\right)=\mathcal{S}(\mathbb{R}^{n})$
and $-\Delta=\mathcal{F}^{-1}M_{0}\mathcal{F}$, the result follows. ∎

It has been shown in [1] (Proposition 3.1) that for every negative self-
adjoint operator $T$, each $\eta\in{\rm rng}\,T$ and each $\psi\in
T^{-1}\\{\eta\\}$, one has for $t>0$,

$\|e^{tT}\eta\|_{\mathcal{H}}\leq\ \frac{\|\psi\|_{\mathcal{H}}}{t\,e}\,.$

Thus, by Claim, if $n\geq 5$ then, for each $f\in\mathcal{S}(\mathbb{R}^{n})$
and $t>0$,

$\|e^{t\Delta}f\|_{L^{2}(\mathbb{R}^{n})}\leq\frac{\|u\|_{L^{2}(\mathbb{R}^{n})}}{t\,e}\,,$

and therefore, by the Spectral Theorem and (7), it follows that for each
$f\in\mathcal{S}(\mathbb{R}^{n})$, $d_{\mu_{f}^{\Delta}}^{-}(0)\geq 2$;
consequently, ${\mathcal{G}}(0)\cap\mathcal{S}(\mathbb{R}^{n})=\emptyset$.

######  Remark 2.2.

Taking into account Example 2.1, it is an interesting open problem to obtain
explicit examples of Schrödinger operators for which the sets presented in the
statements of Theorems 1.3, 2.1 and Corollary 1.1 contain reasonably localized
functions. We recall [16] that given a finite Borel measure $\mu$ on
$[a,b]\subset\mathbb{R}$, there exists a continuous half-line Schrödinger
operator for which the spectral measure coincides with $\mu$ on $[a,b]$;
therefore, such operators do exist.

### Proof of Theorem 2.1

The lemma below plays a fundamental role in the proof of Theorem 2.1.

######  Lemma 2.1 (Lemma 2.1 in [3]).

Let $T$ be a negative self-adjoint operator with $0\in\sigma(T)$, and let
$\alpha:{\mathbb{R}}_{+}\longrightarrow(0,\infty)$ be such that

$\lim_{t\to\infty}\alpha(t)=\infty.$

Then, there exist $\eta\in\mathcal{H}$ and a sequence $t_{j}\rightarrow\infty$
such that, for sufficiently large $j$,

$\mu_{\eta}^{T}\big{(}B(0;{1/t_{j}})\big{)}\geq\frac{1}{\alpha(t_{j})}.$

We divide the proof in 4 steps.

Step 1. Let us show that, for each $\rho>0$ and each $x\in\sigma(T)$,

$\big{\\{}\psi\in\mathcal{H}\mid d^{+}_{\mu^{T}_{\psi}}(x)\geq
d^{-}_{\mu^{T}_{\psi}}(x)\geq\rho\big{\\}}$

is dense in $\mathcal{H}$. Namely, let for each $n\in\mathbb{N}$ and each
$y\in\mathbb{R}$,

$f_{n,\rho}(x,y):=\left(1-e^{-n|x-y|^{\rho}}\right)^{1/2},$

and for each $\psi\neq 0$, let $\psi_{n}:=f_{n,\rho}(x,T)\psi$, where
$f_{n,\rho}(x,T):=P^{T}(f_{n,\rho}(x,\cdot))$. Note that since
$\mu^{T}_{\psi}$ is purely continuous, one gets by the Spectral Theorem and
dominated convergence that

$\displaystyle\|\psi_{n}-\psi\|^{2}$ $\displaystyle=$
$\displaystyle\|f_{n,\rho}(x,T)\psi-\psi\|^{2}$ $\displaystyle=$
$\displaystyle\|(f_{n,\rho}(x,T)-1)\psi\|^{2}$ $\displaystyle=$
$\displaystyle\int_{\mathbb{R}}\left|\left(1-e^{-n|x-y|^{\rho}}\right)^{1/2}-1\right|^{2}{\mathrm{d}}\mu^{T}_{\psi}(y)$
$\displaystyle=$
$\displaystyle\mu^{T}_{\psi}(\\{x\\})+\int_{\mathbb{R}\setminus\\{x\\}}\left|\left(1-e^{-n|x-y|^{\rho}}\right)^{1/2}-1\right|^{2}{\mathrm{d}}\mu^{T}_{\psi}(y)$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}\setminus\\{x\\}}\left|\left(1-e^{-n|x-y|^{\rho}}\right)^{1/2}-1\right|^{2}{\mathrm{d}}\mu^{T}_{\psi}(y)\longrightarrow
0$

as $n\to\infty$, that is, $\psi_{n}\rightarrow\psi$ in $\mathcal{H}$.

Now, by Fubini’s Theorem,

$\displaystyle\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu^{T}_{\psi_{n}}(y)$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu^{T}_{f_{n,\rho}(x,T)\psi}(y)$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}}e^{-2t|x-y|}|f_{n,\rho}(x,y)|^{2}{\mathrm{d}}\mu^{T}_{\psi}(y)$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}}e^{-2t|x-y|}(1-e^{-n|x-y|^{\rho}}){\mathrm{d}}\mu^{T}_{\psi}(y)$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}}e^{-2t|x-y|}|x-y|^{\rho}\frac{(1-e^{-n|x-y|^{\rho}})}{|x-y|^{\rho}}{\mathrm{d}}\mu^{T}_{\psi}(y)$
$\displaystyle\leq$
$\displaystyle\frac{\rho}{2^{\rho}t^{\rho}}\int_{\mathbb{R}}\frac{(1-e^{-n|x-y|^{\rho}})}{|x-y|^{\rho}}{\mathrm{d}}\mu^{T}_{\psi}(y)$
$\displaystyle=$
$\displaystyle\frac{\rho}{2^{\rho}t^{\rho}}\int_{\mathbb{R}}\int_{0}^{n}e^{-s|x-y|^{\rho}}{\mathrm{d}}s\,{\mathrm{d}}\mu^{T}_{\psi}(y)$
$\displaystyle\leq$ $\displaystyle\frac{n\rho}{2^{\rho}t^{\rho}}\|\psi\|^{2}.$

Thus, it follows from identity (7) that for each $n\geq 1$,
$d^{-}_{\mu^{T}_{\psi_{n}}}(x)\geq\rho$, and therefore, that

$\big{\\{}\psi\in\mathcal{H}\mid d^{+}_{\mu^{T}_{\psi}}(x)\geq
d^{-}_{\mu^{T}_{\psi}}(x)\geq\rho\big{\\}}$

is dense in $\mathcal{H}$.

Step 2. Let us show that for every $x\in\sigma(T)$, there exists
$\eta\in\mathcal{H}$ such that $d^{-}_{\mu^{T}_{\eta}}(x)=0$. Set
$L_{x}=(-\infty,x]\cap\sigma(T)$, $T_{x}=TP^{T}(L_{x})$ and
$T^{0}_{x}=T_{x}-xI$. So, by Lemma 2.1, there exist $\eta\in\mathcal{H}$ and
$\varepsilon_{j}\rightarrow 0$ such that, for sufficiently large $j$,

$\mu^{T^{0}_{x}}_{\eta}(B(0;\varepsilon_{j}))\geq\frac{1}{-\ln(\varepsilon_{j})}\Rightarrow\mu^{T_{x}}_{\eta}(B(x;\varepsilon_{j}))\geq\frac{1}{-\ln(\varepsilon_{j})}$

$\Rightarrow\ln\left(\mu^{T}_{\eta}(B(x;\varepsilon_{j}))\right)\geq\ln\left(\mu^{T}_{\eta}(B(x;\varepsilon_{j})\cap
L_{x})\right)=\ln\left(\mu^{T_{x}}_{\eta}(B(x;\varepsilon_{j}))\right)\geq\ln\left(\frac{1}{-\ln(\varepsilon_{j})}\right)$

$\Rightarrow\frac{\ln\left(\mu^{T}_{\eta}(B(x;\varepsilon_{j}))\right)}{\ln\varepsilon_{j}}\leq\frac{\ln\left(\frac{1}{-\ln(\varepsilon_{j})}\right)}{\ln\varepsilon_{j}}\Rightarrow
d^{-}_{\mu^{T}_{\eta}}(x)=0.$

Step 3. Let us show that for every $x\in\sigma(T)$,

$\big{\\{}\psi\in\mathcal{H}\mid d^{-}_{\mu^{T}_{\psi}}(x)=0\big{\\}}$

is dense in $\mathcal{H}$. Namely, let $x\in\sigma(T)$ and set, for every
$n\geq 1$,

$S_{n}:=\left(-\infty,x-\frac{1}{n}\right)\cup\\{x\\}\cup\left(x+\frac{1}{n},\infty\right).$

Set also, for each $\psi\in\mathcal{H}$ and each $n\geq 1$,

$\psi_{n}:=P^{T}(S_{n})\psi+\frac{1}{n}\eta,$

where $\eta$ is given by Step 2. One has that $\psi_{n}\rightarrow\psi$ in
$\mathcal{H}$, since $P^{T}(S_{n})\rightarrow\mathbf{1}$ in the strong sense.
Moreover, for each $n\geq 1$ and each $0<\varepsilon<\frac{1}{n}$, one has

$\displaystyle\mu^{T}_{\psi_{n}}(B(x;\varepsilon))$ $\displaystyle=$
$\displaystyle\langle P^{T}(B(x;\varepsilon))\psi_{n},\psi_{n}\rangle$
$\displaystyle=$ $\displaystyle\langle
P^{T}(B(x;\varepsilon))P^{T}(S_{n})\psi,\psi_{n}\rangle+\frac{1}{n}\langle
P^{T}(B(x;\varepsilon))\eta,\psi_{n}\rangle$ $\displaystyle=$
$\displaystyle\langle P^{T}(B(x;\varepsilon)\cap
S_{n})\psi,\psi_{n}\rangle+\frac{1}{n}\langle
P^{T}(B(x;\varepsilon))\eta,\psi_{n}\rangle$ $\displaystyle=$
$\displaystyle\langle P^{T}(\\{x\\})\psi,\psi_{n}\rangle+\frac{1}{n}\langle
P^{T}(B(x;\varepsilon))\eta,P^{T}(S_{n})\psi\rangle+\frac{1}{n^{2}}\langle
P^{T}(B(x;\varepsilon))\eta,\eta\rangle$ $\displaystyle=$
$\displaystyle\langle P^{T}(\\{x\\})\psi,\psi_{n}\rangle+\frac{1}{n}\langle
P^{T}(\\{x\\})\eta,\psi\rangle+\frac{1}{n^{2}}\langle
P^{T}(B(x;\varepsilon))\eta,\eta\rangle$ $\displaystyle=$
$\displaystyle\frac{1}{n^{2}}\langle P^{T}(B(x;\varepsilon))\eta,\eta\rangle$
$\displaystyle=$
$\displaystyle\frac{1}{n^{2}}\,\mu^{T}_{\eta}(B(x;\varepsilon))\,,$

from which follows that

$d^{-}_{\psi^{T}_{\psi_{n}}}(x)=\liminf_{\varepsilon\downarrow
0}\frac{\ln(\mu^{T}_{\psi_{n}}(B(x;\varepsilon)))}{\ln\varepsilon}=\liminf_{\varepsilon\downarrow
0}\frac{\ln(\mu^{T}_{\eta}(B(x;\varepsilon))}{\ln\varepsilon}=d^{-}_{\psi^{T}_{\eta}}(x)=0.$

Thus,

$\big{\\{}\psi\in\mathcal{H}\mid d^{-}_{\mu^{T}_{\psi}}(x)=0\big{\\}}$

is dense in $\mathcal{H}$.

Step 4. Finally, in this step, we finish the proof. Since, for each
$x\in\mathbb{R}$ and each $t>0$, the mapping

$\
\mathcal{H}\ni\psi\mapsto\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu_{\psi}^{T}(y)=\langle
g_{t}(T,x)\psi,\psi\rangle\,,$

with $g_{t}(y,x)=e^{-2t|x-y|}$, is continuous, it follows that for every
$x\in\mathbb{R}$, each one of the sets

$B_{-}(x):=\big{\\{}\psi\in\mathcal{H}\mid
d^{+}_{\mu_{\psi}^{T}}(x)=\infty\big{\\}}=\bigcap_{l\geq 1}\bigcap_{n\geq
1}\bigcap_{k\geq 1}\bigcup_{t\geq k}\Big{\\{}\psi\in\mathcal{H}\mid
t^{l}\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu_{\psi}^{T}(y)<{1}/{n}\Big{\\}}$

and

$B_{+}(x):=\big{\\{}\psi\in\mathcal{H}\mid
d^{-}_{\mu_{\psi}^{T}}(x)=0\big{\\}}=\bigcap_{l\geq 1}\bigcap_{n\geq
1}\bigcap_{k\geq 1}\bigcup_{t\geq k}\Big{\\{}\psi\in\mathcal{H}\mid
t^{\frac{1}{l}}\int_{\mathbb{R}}e^{-2t|x-y|}{\mathrm{d}}\mu_{\psi}^{T}(y)>n\Big{\\}}$

is a $G_{\delta}$ set in $\mathcal{H}$. Thus, it follows from Steps 1. and 3.
that for each $x\in\sigma(T)$, both $B_{-}(x)$ and $B_{+}(x)$ are generic sets
in $\mathcal{H}$, and so

${\mathcal{G}}(x)=\bigcap_{n\geq 1}\big{\\{}\psi\in\mathcal{H}\mid
d^{-}_{\mu_{\psi}^{T}}(x)=0\ \ \text{and}\ \ d^{+}_{\mu_{\psi}^{T}}(x)\geq
n\big{\\}}$

is also generic in $\mathcal{H}$.

##  3 Proof of Theorem 1.3

Let $\alpha\in[0,1]$. We need the following result.

Claim. If $x\in\sigma(T)$, then
${\mathcal{G}}(x)\subset\big{\\{}\psi\in\mathcal{H}\mid\mu_{\psi}^{T}\,\,{\rm
is\,not}\,{\rm U}(1/k){\rm H},\,\forall k\in\mathbb{N}\big{\\}}$.

Namely, it is enough to note that, if $\mu_{\psi}^{T}$ is ${\rm U}\alpha{\rm
H}$, then for each $x\in\sigma(T)$, $d^{\mp}_{\mu_{\psi}^{T}}(x)\geq\alpha$.

(i) This is a direct consequence of Theorem 2.1 and Claim.

(ii) For $x\in\sigma(T)$, if $\psi\in{\mathcal{G}}(x)$, then by Claim, for
each $k\geq 1$, $\mu_{\psi}^{T}$ is not ${\rm U}(1/2k){\rm H}$. Thus, it
follows from Theorem 1.1 that for each $k\geq 1$,

$\limsup_{t\to\infty}t^{1/k}W_{\psi}^{T}(t)=\infty$

from which follows, again from Theorem 2.1, that for each $k\geq 1$, the set

$\big{\\{}\psi\in\mathcal{H}\mid\limsup_{t\to\infty}t^{1/k}W_{\psi}^{T}(t)=\infty\big{\\}}\supset{\mathcal{G}}(x)$

is generic in $\mathcal{H}$.

It remains to prove that for each $k\geq 1$, the set

$A_{k}:=\big{\\{}\psi\in\mathcal{H}\mid\liminf_{t\to\infty}t^{\alpha-1/k}W_{\psi}^{T}(t)=0\big{\\}}$

is generic in $\mathcal{H}$. The proof that for each $k\geq 1$, $A_{k}$ is a
$G_{\delta}$ subset of $\mathcal{H}$ follows closely the arguments presented
in Remark 2.1. On the other hand it follows from Theorem 1.1(i) that for each
$k\geq 1$,

${\mathcal{H}}_{\text{uH}}^{T}(\alpha)\subset A_{k}\,.$

Finally, since, by Theorem 1.2,

$\overline{{\mathcal{H}}_{\mathrm{uH}}^{T}(\alpha)}={\mathcal{H}}_{\alpha\mathrm{c}}^{T}\,,$

it follows that for each $k\geq 1$, $A_{k}$ is a dense $G_{\delta}$ subset of
$\mathcal{H}$ (recall that $T$ has purely $\alpha$-Hausdorff continuous
spectrum, by hypothesis).

Acknowledgments

M.A. and G.S. were supported by CAPES (Brazilian agency). SLC thanks the
partial support by Fapemig (Minas Gerais state agency; Universal Project under
contract 001/17/CEX-APQ-00352-17) and CRdO thanks the partial support by CNPq
(a Brazilian government agency, under contract 303503/2018-1).

## References

  * [1] M. Aloisio, S. L. Carvalho and C. R. de Oliveira, Category Theorems for Schrödinger Semigroups. Z. Anal. Anwend. 39 (2020), 421–431.
  * [2] M. Aloisio, S. L. Carvalho and C. R. de Oliveira, Quantum quasiballistic dynamics and thick point spectrum. Lett. Math. Phys. 109 (2019), 1891–1906.
  * [3] M. Aloisio, S. L. Carvalho and C. R. de Oliveira, Refined scales of decaying rates of operator semigroups on Hilbert spaces: typical behavior. Proc. Amer. Math. Soc. 148 (2020), 2509–2523.
  * [4] M. Aloisio, S. L. Carvalho and C. R. de Oliveira, Some generic fractal properties of bounded self-adjoint operators. Preprint. arXiv:2007.13162 [math.SP], (2020).
  * [5] A. Avila, On the spectrum and Lyapunov exponent of limit-periodic Schrödinger operators. Commun. Math. Phys. 288 (2009), 907–18.
  * [6] J.-M. Barbaroux, J.-M. Combes and R. Montcho, Remarks on the relation between quantum dynamics and fractal spectra. J. Math. Anal. Appl. 213 (1997), 698-722.
  * [7] S.L. Carvalho and C.R. de Oliveira, A characterization of singular packing subspaces with an application to limit-periodic operators. Forum Mathematicum 29 (2017), 31-40.
  * [8] S. L. Carvalho and C. R. de Oliveira, Correlation dimension wonderland theorems. J. Math. Phys. 57 (2016), 063501, 19 pp.
  * [9] R. C. Charão and R. A. Prado, Invariants of the trace map and uniform spectral properties for discrete Sturmian Dirac operators. Osaka J. Math. 56 (2019), 391-416.
  * [10] D. Damanik, Schrödinger operators with dynamically defined potentials. Ergod. Th. & Dynam. Sys. 37 (2017), 1681–1764.
  * [11] D. Damanik and Z. Gan, Spectral properties of limit-periodic Schrödinger operators. Commun. Pure Appl. Anal. 10 (2011), 859–871.
  * [12] D. Damanik, R. Killip and D. Lenz, Uniform Spectral Properties of One-Dimensional Quasicrystals, III. $\alpha$-Continuity. Commun. Math. Phys. 37 (2000), 191-204.
  * [13] D. Damanik, M. Lukic and W. Yessen, Quantum dynamics of periodic and limit-periodic Jacobi and block Jacobi matrices with applications to some quantum many body problems. Commun. Math. Phys. 337 (2015), 1535–1561.
  * [14] C. R. de Oliveira, Intermediate Spectral Theory and Quantum Dynamics. Basel. Birkhäuser, (2009).
  * [15] R. del Rio, S. Jitomirskaya, Y. Last and B. Simon, Operators with singular continuous spectrum, IV. Hausdorff dimensions, rank one perturbations and localization. J. Analyse Math. 69 (1996), 153–200.
  * [16] I. M. Gelfand and B. M. Levitan, On the determination of a differential equation from its spectral function, (Russian) Izv. Akad. Nauk SSR. Ser. Mat. 15 (1951), 309–360; English transl. in Amer. Math. Soc. Transl. Ser. 21 (1955), 253–304.
  * [17] F. Germinet, A. Kiselev and S. Tcheremchantsev, Transfer matrices and transport for Schrödinger operators. Ann. Inst. Fourier 54 (2004), 787–830
  * [18] K. J. Falconer, Fractal Geometry. Wiley, Chichester, (1990).
  * [19] M. Holschneider, Fractal wavelet dimensions and localization. Commun. Math. Phys. 160 (1994), 457–473.
  * [20] Y. Last, Quantum dynamics and decompositions of singular continuous spectra. J. Funct. Anal. 42 (1996), 406–445.
  * [21] R. S. Strichartz, Fourier asymptotics of fractal measures. J. Funct. Anal. 89 (1990), 154–187.

Email<EMAIL_ADDRESS>Departamento de Matemática, UFAM, Manaus, AM,
369067-005 Brazil

Email<EMAIL_ADDRESS>Departamento de Matemática, UFMG, Belo Horizonte,
MG, 30161-970 Brazil

Email<EMAIL_ADDRESS>Departamento de Matemática, UFSCar, São Carlos,
SP, 13560-970 Brazil

Email<EMAIL_ADDRESS>Departamento de Matemática, UFMG, Belo
Horizonte, MG, 30161-970 Brazil