# Token-level Fitting Issues of Seq2seq Models

Guangsheng Bao1,2, Zhiyang Teng3, and Yue Zhang ,2,4,
1 Zhejiang University
2 School of Engineering, Westlake University
3 Nanyang Technological University
4 Institute of Advanced Technology, Westlake Institute for Advanced Study
2 {baoguangsheng<EMAIL_ADDRESS>
3<EMAIL_ADDRESS>* Corresponding author.

###### Abstract

Sequence-to-sequence (seq2seq) models have been widely used for natural
language processing, computer vision, and other deep learning tasks. We find
that seq2seq models trained with early-stopping suffer from issues at the
token level. In particular, while some tokens in the vocabulary demonstrate
overfitting, others underfit when training is stopped. Experiments show that
the phenomena are pervasive in different models, even in fine-tuned large
pretrained-models. We identify three major factors that influence token-level
fitting, which include token frequency, parts-of-speech, and prediction
discrepancy. Further, we find that external factors such as language, model
size, domain, data scale, and pretraining can also influence the fitting of
tokens.

##  1 Introduction

Deep learning models tend to overfit on relatively small datasets because of
their strong capacity and a massive number of parameters (Brownlee, 2018; Li
et al., 2019; Rice et al., 2020; Bejani and Ghatee, 2021). Studies suggest
regularization and early stopping to control the generalization error caused
by overfitting (Hastie et al., 2009; Zhang et al., 2017; Chatterjee and
Zielinski, 2022). Previous studies mainly analyze the generalization issue on
image classification task (Arpit et al., 2017; Zhang et al., 2021), where the
learning target is relatively simple. In contrast, NLP task such as machine
translation (Zhang et al., 2015; Singh et al., 2017) is more complex with
regard to the learning targets, which involve a sequence of tokens.

Natural languages exhibit a long-tailed distribution of tokens (Powers, 1998).
The long-tail phenomena have been associated with performance degradation of
NLP tasks (Gong et al., 2018; Raunak et al., 2020; Yu et al., 2022), where the
rare (low frequency) tokens are ascribed as hard learning targets and popular
(high frequency) tokens as easy learning targets. These criteria of easiness
of learning targets are intuitive but coarse-grained, which are not associated
with the training dynamics. In this paper, we study the easiness of tokens as
learning targets from the perspective of overfitting and underfitting.
Intuitively, the learning on hard tokens will be slower than that on easy
tokens, which may result in underfitting on hard tokens and overfitting on
easy tokens, as illustrated by Figure 2. We propose two measures to quantify
fitting – _fitting-offset_ and _potential-gain_. Fitting-offset measures the
offset of the best fit from the early-stopping point, which reflects the
degree of overfitting or underfitting. Potential-gain measures the accuracy
gap between the early-stopping checkpoint and the best fit, which also
estimates the accuracy decrease caused by overfitting or underfitting.

(a) Idealized training and validation loss curves, where the model is selected
by early stopping.

(b) Easy tokens overfit at the early-stopping point, while hard tokens
underfit.

Figure 1: Seq2seq models trained with early stopping may suffer from
overfitting or underfitting.

Figure 2: Fitting-offset measures the relative distance of the best fit from
the early-stopping point, which is negative for the left side and positive for
the right side.

We use machine translation as our test bed, training models on English-German
benchmark datasets, including News and Europarl domains. Our extensive
experiments reveal novel findings: 1) Both overfitting and underfitting occur
in a trained seq2seq model. 2) High-frequency tokens are expected to overfit,
but some are found underfitted, and low-frequency tokens are expected to
underfit, but some are found overfitted. 3) Large pretrained models reduce
underfitting effectively during fine-tuning but are less effective on
overfitting. Besides, we propose a direct indicator of easiness – prediction
discrepancy, using the probability difference of predictions made by full
context and local context as a criterion to group tokens.

In addition to tokens, sentences have also been considered as learning
targets, where curriculum learning methods distinguish sentences as easy or
hard (Kocmi and Bojar, 2017; Platanios et al., 2019; Xu et al., 2020). For
example, the length of a sentence is used as a criterion to identify easy
sentences from hard ones, where the easy (short) sentences are first learned
and then the hard (long) sentences. Using our metrics, we find that the length
of a sentence is not a good indicator of easy or hard sentences from the
perspective of overfitting and underfitting.

##  2 Related Work

Previous studies compare the fitting of real data and noise data,
demonstrating that real data is easier to learn and has a faster convergence
speed than noise data Zhang et al. (2017); Arpit et al. (2017); Chatterjee and
Zielinski (2022). The different convergence speeds also provide an explanation
of how early stopping prevents the memorization of noise data. However, these
works do not compare the fitting among different learning targets inside the
real data, neither between different samples nor between different parts of
each sample. In this paper, we conduct experiments on a more complex seq2seq
task instead of simple classification. We study the fitting of token-level
learning targets, demonstrating that both overfitting and underfitting occur
when training seq2seq models.

There are few works studying overfitting and underfitting in NLP. Sun et al.
(2017) report that complex structure leads to overfitting in structured
prediction. Wolfe and Caliskan (2021) demonstrate that low-frequency names
exhibit bias and overfitting in the language model. Varis and Bojar (2021)
illustrate that machine translation models generalize poorly on the test set
with unseen sentence length. These works discuss overfitting issues on
specific conditions, such as complex structure, frequent names, and unseen
length. In comparison, we conduct a systematic analysis of the general
phenomena of overfitting and underfitting in language. Specifically, we
propose quantitative measures, identify major factors, and conduct statistical
hypothesis testing on the phenomena.

##  3 Experimental Settings

###  3.1 Datasets

We use the News corpus as a major dataset for our experiments and analysis,
and we further use the Europarl corpus for the comparison of different domains
and data scales.

News We use News Commentary v11 for training, newstest2015 for validation, and
newstest2016 for testing. The English-German machine translation dataset
contains $236,287$ sentence pairs for training, $2,169$ pairs for validation,
and $2,999$ pairs for testing.

Europarl We use English-German Europarl v7, following Maruf et al. (2019) to
split the train, validation, and test sets. The dataset contains $1,666,904$
sentence pairs for training, $3,587$ pairs for validation, and $5,134$ pairs
for testing.

We tokenize the sentences using MOSES Koehn et al. (2007), followed by
truecase and a BPE Sennrich et al. (2015) with $30,000$ merging operations. We
use separate embedding tables for source and target languages in the model.

###  3.2 Model Configurations

We study the overfitting and underfitting issues on three model
configurations. We use the base model in section 4 and 5, the pre-trained
model in section 6, and the big model for the model-size evaluation in section
7.

Base Model We use the standard Transformer base model Vaswani et al. (2017),
which has 6 layers, 8 heads, 512 output dimensions, and 2048 hidden
dimensions. We train the model with a learning rate of
$5\text{\times}{10}^{-4}$, a dropout of $0.3$, a label smoothing of $0.1$, and
an Adam optimizer Kingma and Ba (2014).

Big Model Following the standard Transformer big model Vaswani et al. (2017),
we use 6 layers, 16 heads, 1024 output dimensions, and 4096 hidden dimensions.
We train the model with a learning rate of $3\text{\times}{10}^{-4}$, a
dropout of $0.3$, a label smoothing of $0.1$, and an Adam optimizer.

Pretrained Large Model We use mBART25 Liu et al. (2020), which has the similar
setting as BART large model Lewis et al. (2020), using 12 layers, 16 heads,
1024 output dimensions, and 4096 hidden dimensions. We fine-tune the model
with a learning rate of $3\text{\times}{10}^{-5}$, a dropout of $0.3$, an
attention-dropout of $0.1$, a label smoothing of $0.2$, and an Adam optimizer.

For each experiment, we train $40$ models using random seeds from $1$ to $40$,
obtaining $40$ samples for the significance test. During the training of the
base or big model, we keep the last $20$ checkpoints for analysis, where the
checkpoint of early-stopping is the $10$-th of the $20$ checkpoints. For
mBART25, we keep the last $10$ checkpoints, and the early-stopping checkpoint
is at the $5$-th of the checkpoints.

###  3.3 Evaluation Metric

Measures We propose two measures: fitting-offset and potential-gain.

Fitting-offset represents how far (i.e., number of epochs) the best fit of a
group of tokens diverges from the point of early stopping. In this paper, we
use epoch as its unit because we evaluate the model using the validation set
at the end of each training epoch. As Figure 2 shows, for the easy tokens, the
fitting-offset is negative, denoting overfitting, where the best fit is before
the early-stopping epoch. For the hard tokens, the fitting-offset is positive,
denoting an underfitting, where the best fit is after the early-stopping
epoch. Using fitting-offset, we can quantify the degree of overfitting and
underfitting.

Potential-gain represents the potential accuracy increase if we move the best
fit to the early-stopping epoch. We calculate the measure by subtracting the
accuracy of the early-stopping checkpoint from the accuracy of the best fit.
Using this measure, we can quantitatively estimate the potential benefits by
fixing the overfitting or underfitting issue.

Significance Test Since the distribution of fitting-offset is unknown, we use
a non-parametric sign-test Dixon and Mood (1946); Hodges (1955) to test our
hypothesis. We train the model $N$ times to obtain $N$ observations on the
fitting-offset. The hypothesis about the overfitting and underfitting can be
expressed by

$\left\\{\begin{array}[]{ll}H_{0}:\text{fitting-offset}=0,&\text{no fitting
issue,}\\\ H_{1}:\text{fitting-offset}\neq 0,&\text{has fitting
issue.}\end{array}\right.$ (1)

If $H_{0}$ is true, the $N$ observations are expected to be half positive and
half negative. The total number of positive observations $N_{+}$ follows a
binomial distribution, through which we decide the rejection region according
to a significance level $\alpha$.

Grouping Ideally, we can calculate the two measures on each token to tell
which tokens are overfitted and which tokens are underfitted at the early-
stopping epoch. However, direct observation of each token is noisy and does
not show obvious patterns. We group the tokens and average the valid losses to
reduce the noise, through which the pattern emerges, and we obtain stable
measures.

Figure 3: Fitting-offset of tokens grouped by token _frequency_.

##  4 Token-level Results

###  4.1 Fitting of Rare Tokens in Seq2seq Model Training

Previous studies suggest that long-tail token distribution affects the
performance of NLP tasks Gong et al. (2018); Raunak et al. (2020); Yu et al.
(2022). We hypothesize that the low-frequency tokens underfit during training
and conduct verification experiments as follows.

Settings We experiment on the News dataset, using a Transformer base model
Vaswani et al. (2017). We categorize the target tokens into high/medium/low-
frequency according to their distribution in the training set, with balanced
probability mass on the three buckets.

Group | Parts-of-speech (POS)
---|---
Noun | NOUN, PRON, PROPN
Verb | VERB, AUX
Adj | ADJ, ADV
Num | NUM
Func | ADP, CONJ, CCONJ, DET, PART, SCONJ
Symb | PUNCT, SYM

Table 1: Groups aggregating parts of speech.

Figure 4: Fitting-offset of tokens grouped by _parts-of-speech_.

Hypothesis Testing For each group of high/medium/low-frequency tokens, we
measure the fitting-offset using the checkpoints of each model, obtaining $40$
samples for each group. We test our hypothesis on each group using a sign-test
(Eq. 1). As a result, we obtain a p-value of $1.9\text{\times}{10}^{-5}$ for
high-frequency and $7.5\text{\times}{10}^{-11}$ for low-frequency, which
strongly supports the hypothesis that the high/low-frequency tokens either
overfit or underfit.

Further as Figure 3 shows, the average fitting-offset for high-frequency
tokens is $-3.7$ with a standard deviation of $3.4$. The negative value of the
fitting-offset indicates that the high-frequency tokens overfit, where the
best fit happens at an average of $3.7$ epochs, before the early-stopping
point. The average fitting-offset for low-frequency tokens is $5.8$ with a
standard deviation of $3.3$. The positive value of the fitting-offset
indicates underfitting, where the best fit happens at $5.8$ epochs, after the
early-stopping point on average. Based on this evidence, we conclude that Both
overfitting and underfitting at the token level occur when training seq2seq
models.

Analysis The significant divergence of fitting-offset between the high/low-
frequency tokens suggests that the frequency of tokens has a significant
influence on their fitting. We quantify the influence using the potential-
gain. In particular, take the low-frequency tokens as an example. The
potential-gain is $0.73$, which means that the average accuracy is expected to
be increased from $45.61$ to $46.34$ if we move the best fit to the early-
stopping epoch. The potential-gain of the high-frequency tokens is $0.05$, and
that of the medium-frequency tokens is $0.23$, which is relatively smaller
than that of the low-frequency tokens, suggesting underfitting of the low-
frequency tokens is the major issue.

Figure 5: Fitting-offset of tokens grouped by _frequency_ and _parts-of-speech_. Frequency | Function | Symbol | Number | Verb | Adj/Adv | Noun
---|---|---|---|---|---|---
High | 56.82 +0.15 | 84.77 +0.59 | nan +nan | 59.13 +1.09 | 71.02 +1.67 | 60.87 +0.89
Med | 49.59 +0.16 | 69.63 +3.41 | 73.47 +1.50 | 47.64 +0.12 | 44.56 +0.24 | 59.21 +0.76
Low | 43.76 +1.38 | 72.24 +3.55 | 74.84 +1.20 | 36.25 +0.51 | 41.18 +0.43 | 48.03 +0.88
Table 2: Potential-gain for each category grouped by _frequency_ and _parts-
of-speech_. The column is in a format of “averaged-accuracy potential-gain”,
where the “+” in the potential-gain indicates an increase in the accuracy and
the “-” indicates a decrease in the accuracy. We mark potential-gains bigger
than $0.5$ with the bold font to indicate their significance.

###  4.2 Linguistic Factors to Token-level Fitting

In section 4.1, we find that the high-frequency tokens tend to overfit and the
low-frequency tokens tend to underfit in the seq2seq model as a group. In
order to further understand a fine-grained correlation between the frequency
and the fitting of a token, we further split the high/low-frequency tokens
into smaller groups and conduct experiments on the specific categories.
Linguistic factors are considered in the detailed experiments.

Parts-of-speech (POS) We speculate that parts-of-speech, as an important
linguistic feature, may provide a different perspective to study the
overfitting and underfitting issues. We group tokens according to their parts-
of-speech as listed in Table 1. Specifically, we first obtain POS tagging on
each word using spaCy 111https://spacy.io/. Then we map the POS of words to
tokens by labeling all the tokens of a word with the same POS. Last, we group
these tokens according to their POS. Take the group Noun as an example. We
group tokens with the POS of NOUN, PRON, and PROPN into one category, naming
Noun. We aggregate the major parts of speech into six groups according to
their functional similarity, as shown in the Table.

As Figure 4 shows, parts-of-speech has a significant influence on the fitting
of tokens. The function words are most likely to overfit, which is likely
because they are close-set and easier to learn from the linguistic
perspective. On the contrary, nouns are most likely to underfit, which can be
due to the openness of the set and the challenging context dependencies.

The potential-gain of nouns is $0.69$, increasing the accuracy from $52.38$ to
$53.07$. The potential-gains of numbers, symbols, verbs, and adj/adv words are
$1.09$, $0.58$, $0.26$, and $0.22$, respectively. Surprisingly, the potential-
gain of function words is negligible, even though they obviously overfit. We
attribute it to the overall high frequency of function words because
sufficient training samples reduce the negative impact of overfitting. It is
confirmed by the detailed potential-gains shown in Table 2, where the function
words with low frequency have a much higher potential-gain of $1.38$.

It is worth noting that we use sub-word tokens in the experiments, where we
assign the POS of a word to all its sub-words. We also tried words as tokens,
which give a similar distribution of fitting offset, and the conclusions hold.

Frequency and Parts-of-speech We combine frequency and POS to make a detailed
analysis of the high/low-frequency tokens. As Figure 5 shows, frequency and
POS work independently. Among the high-frequency tokens, the function words
tend to overfit, while adjectives and nouns tend to underfit. Among the low-
frequency tokens, the symbols tend to overfit, while the adjectives and nouns
tend to underfit. Based on this evidence, we arrive at a counter-intuitive
conclusion that in a seq2seq model, the high-frequency tokens (popular tokens)
mostly overfit but can also underfit, and the low-frequency tokens (rare
tokens) mostly underfit but can also overfit.

When we look into the potential-gains, as shown in Table 2, we see higher
potential-gains than in the previous section. The potential-gain of low-
frequency function words, symbols, and numbers are $1.38$, $3.55$, and $1.20$,
respectively. The potential-gains on med-frequency symbols and numbers are
$3.41$ and $1.50$, respectively. Overall the high-frequency tokens have low
potential-gains, and the verbs and adjectives have potential-gains of $1.09$
and $1.67$, respectively. These results demonstrate that combining the
frequency and linguistic factors reveals stronger overfitting and
underfitting, forecasting higher potential-gains in specific categories.

Figure 6: Fitting-offset of tokens grouped by prediction _discrepancy_.

Figure 7: Fitting-offset of tokens grouped by _discrepancy_ and _frequency_.

Discrepancy | Frequency
---|---
High | Med | Low
Big | 63.91 +0.09 | 26.88 -0.22 | 10.57 +0.11
Med | 66.96 +0.09 | 47.77 +0.31 | 20.22 +0.86
Small | 72.49 +0.09 | 80.95 +0.62 | 73.19 +0.85

Table 3: Potential-gain for each category grouped by _discrepancy_ and
_frequency_.

###  4.3 Prediction Discrepancy as A Factor to Token-level Fitting

Given that neither frequency nor parts of speech are decisive factors, we
propose a new factor – prediction discrepancy, which measures the degree of
dependence on long context for predicting each token in a text sequence. The
intuition behind this factor is that some tokens may require more information
from the surrounding context to be correctly predicted, while others may be
more independent and rely more on their local context. Specifically, consider
a sentence, “I was quoted about 12 months .” the word “months” could most
likely be told according to its local context “12”. In comparison, in a German
sentence “es war schwer , ihn zu kennen .”, where the word-by-word translation
is “it was hard , him to know .” the word “kennen” requires much longer
context to predict.

The prediction discrepancy is calculated as

$D_{j}=|P(Y_{j}|Y_{<j},X)-P(Y_{j}|Y_{j-1},X)|,$ (2)

where $X$ is the source sequence, and $Y$ is the target. For each token
$Y_{j}$, we predict it using its full context $Y_{<j}$ and its local context
$Y_{j}$. We use the discrepancy between these two predictions to indicate its
dependence on long context. We train an altered Transformer model to do the
predictions using two decoders, where one decoder uses the full context of a
target while another decoder uses the local context. The two decoders share
the same token embedding table and encoder. According to the value of
discrepancy, we categorize the tokens into three groups, with big, medium, and
small discrepancy, respectively.

Results As Figure 6 shows, the big-discrepancy tokens have an average fitting-
offset of $-2.7$ with a standard deviation of $3.0$. The medium-discrepancy
tokens have an average fitting-offset of $1.0$ with a standard deviation of
$2.1$. The small-discrepancy tokens have an average fitting-offset of $8.2$
with a standard deviation of $1.8$, showing a trend to exceed the boundary of
$10$. In comparison with frequency, the bigger range of the average fitting-
offsets and the smaller standard deviations suggest that discrepancy is a
better indicator than frequency. This indicates that the discrepancy is a good
indicator of overfitting and underfitting.

The potential-gain of the small-discrepancy tokens is $0.63$, increasing the
average accuracy of the tokens from $75.85$ to $76.48$. In comparison with the
potential-gain of $0.75$ for the low-frequency tokens, which increases the
average accuracy from $45.61$ to $46.34$, the baseline accuracy of small-
discrepancy is much higher, suggesting the effectiveness of discrepancy in
discovering fitting issues among high accuracy predictions.

Discrepancy and Frequency Intuitively, discrepancy and frequency are two
independent factors, given that discrepancy relies on context and frequency
relies on the token itself. As Figure 7 shows, the most significant difference
between high-frequency and med/low-frequency tokens is that med/small
discrepancy tokens with high frequencies tend to overfit, while the med/small
discrepancy tokens with med/low frequencies tend to underfit. In addition, as
shown in Table 3, when frequency and discrepancy are combined to predict the
overfitting and underfitting, the biggest potential-gain of low-frequency
tokens increases from $0.73$ to $0.86$, suggesting that frequency and
discrepancy are two independent factors.

Figure 8: Fitting-offset of tokens grouped by _discrepancy_ and _parts-of-speech_. Discrepancy | Function | Symbol | Number | Verb | Adj/Adv | Noun
---|---|---|---|---|---|---
Big | 45.43 -0.07 | 83.38 +0.51 | 31.58 +2.72 | 20.31 +0.12 | 13.38 -0.10 | 20.58 -0.06
Med | 58.33 +0.08 | 78.79 +0.96 | 36.49 +2.02 | 39.69 +0.25 | 29.51 +0.14 | 33.30 +0.93
Small | 77.06 +0.44 | 77.98 +1.63 | 89.20 +1.15 | 69.90 +0.84 | 71.93 +0.75 | 77.08 +0.85
Table 4: Potential-gain for each category grouped by _discrepancy_ and _parts-
of-speech_.

Discrepancy and Parts-of-speech (POS) As Figure 8 shows, discrepancy and POS
also work orthogonally. Overall, tokens with a smaller discrepancy have a
larger fitting-offset, which consistently appears on numbers, verbs,
adjectives, and nouns. Function words and symbols show a different pattern
that the med-discrepancy tokens tend to have smaller fitting-offset than high-
discrepancy.

As shown in Table 4, small-discrepancy tokens have potential gains of $1.63$
and $1.15$ on symbols and numbers, respectively. The potential gain on numbers
with big/med-discrepancy are $2.72$ and $2.02$, respectively, suggesting the
effectiveness of combining the two factors.

Summary We have identified three independent factors that affect token-level
fitting in seq2seq model training, including frequency, parts-of-speech, and
discrepancy. While the former two are internal to the token, the third is
external and context-dependent. These indicate that the fitting of tokens
results from interestingly complex factors.

Figure 9: Fitting-offset of tokens grouped by _length-of-sentence_.

##  5 Easy Sentences vs Hard Sentences

Curriculum learning starts with easy sentences and then with hard sentences
Kocmi and Bojar (2017); Zhang et al. (2018); Platanios et al. (2019); Xu et
al. (2020); Zhou et al. (2020), whereas different criteria are used to measure
the difficulty of sentences. Among these criteria, the length-of-sentence is
the simplest and most popular one, which hypothesizes that short sentences
will be easy to learn and long sentences will be hard to learn. We test the
hypothesis by evaluating whether short sentences overfit and long sentences
underfit in a trained seq2seq model.

Length-of-sentence We categorize sentences into short/medium/long sentences
according to the sentence length that each bucket is allocated with almost the
same number of sentences. On the News dataset, the length of short sentences
is between $1$ and $18$ tokens, the length of medium sentences between $19$
and $31$, and the length of long sentences between $32$ and $792$.

Hypothesis Testing. We test our hypothesis using sign-test on News English-
German dataset, obtaining a p-value of $3.6\text{\times}{10}^{-5}$ for short-
sentence, $2.1\text{\times}{10}^{-5}$ for medium-sentence, and
$2.6\text{\times}{10}^{-3}$ for long-sentence, which indicates overfitting or
underfitting. The fitting-offset has an average of $1.95$, $2.0$, and $1.38$
for short/medium/long-sentences, respectively. The positive fitting-offsets
suggest that they overfit in the trained models. However, as Figure 9 shows,
the degree of overfitting and underfitting is less than that of frequency
(Figure 3) and discrepancy (Figure 6).

Summary The above experiments suggest that although the length-of-sentence can
differentiate easy sentences from hard sentences, its effectiveness may not be
as significant as other factors such as frequency, discrepancy, and parts-of-
speech. More surprisingly, short sentences are more likely to underfit than
long sentences, which is also confirmed by experiments on pretraining settings
in section 6, suggesting that we could not simply judge the short-sentences as
easy and long-sentences as hard.

(a) Fitting-offset of tokens grouped by _frequency_.

(b) Fitting-offset of tokens grouped by _length-of-sentence_.

(c) Fitting-offset of tokens grouped by _parts-of-speech_.

Figure 11: The distribution of fitting-offset on _pretraining_ setting.

##  6 Fine-tuning of Pretrained Language Models

Fine-tuning on a large pretrained model has become the dominant setting for
NLP tasks Kenton and Toutanova (2019); Lewis et al. (2020); Brown et al.
(2020); Liu et al. (2020). We investigate the overfitting and underfitting
issues, particularly in the pretraining setting.

Hypothesis Testing We first test whether the overfitting and underfitting
issues exist under the pretraining setting. We experiment by fine-tuning
mBART25 on the News English-German dataset.

First, we evaluate frequency as an indicator of overfitting and underfitting.
As Figure 11(a) shows, we obtain a p-value of $1.2\text{\times}{10}^{-3}$ and
an average fitting-offset of $-1.48$ on high-frequency tokens, suggesting the
tendency of overfitting on high-frequency tokens. Results on medium/low-
frequency tokens do not show significance, although the average fitting-
offsets of $0.08$ on medium-frequency tokens suggest slight underfitting.

Next, we consider length-of-sentence. As Figure 11(b) shows, the fine-tuning
tends to overfit for long-sentence and we obtain a p-value of
$1.1\text{\times}{10}^{-2}$ and an average fitting-offset of $-1.2$ on long-
sentences. It suggests obvious overfitting of long sentences, which is
counter-intuitive because it is widely assumed that long-sentence is harder to
learn than short-sentence.

Last, we examine parts of speech. As Figure 11(c) shows, fine-tuning on
function words tends to overfit and that on nouns underfit, which is
consistent with the result on non-pretraining setting (Figure 4). A difference
is that the fine-tuning of verbs shows underfitting, which is not
significantly observed in the non-pretraining settings. We obtain a p-value of
$5.8\text{\times}{10}^{-2}$ and an average fitting-offset of $0.8$ on verbs,
suggesting an observable underfitting on verbs. The issue on function words is
more significant than on verbs, on which we obtain a p-value of
$4.3\text{\times}{10}^{-6}$ and an average fitting-offset of $-2.3$,
suggesting significant overfitting on function words.

The experiments above confirm that the overfitting and underfitting issues
exist in the pretraining setting, although it is not as significant as that in
the non-pretraining settings. In addition, it shows that overfitting is the
major issue in comparison with underfitting. We attribute it to the
effectiveness of large pretraining to prevent underfitting.

It is worth noting that the vocabulary used by the pre-trained model is
different from other non-pretraining settings, which may affect the
statistical figures but the conclusions hold.

##  7 Additional Factors to the Fitting Issue

Most of the figures and tables for this section are placed in Appendix A.

The Language As a comparison of languages, we study the issues on the News
dataset but with a reversed direction of languages, translating German to
English instead of English to German. As Figure 12 shows, our observations on
target English tokens are consistent with previous observations on target
German tokens. First, the high-frequency tokens tend to overfit, while the
low-frequency tokens tend to underfit. Second, the big-discrepancy tokens tend
to overfit, while the small-discrepancy tokens tend to underfit. Third, the
function words tend to overfit, while the nouns tend to underfit. Last, we
also obtain bigger potential-gains by combining the factors and the most
significant potential-gains happen on the consistent categories, such as low-
frequency symbols, high-frequency adj/advs, and big/med-discrepancy numbers,
as the bold numbers in Table 5, 6, and 7 show.

The Model Size To evaluate the influence of model size, we conduct experiments
with a big model on the News English-German dataset. As Figure 16 shows, the
distribution of fitting-offset on each category is very close to that of the
base model but with a smaller range. We attribute it to the faster convergence
of bigger model Li et al. (2020). One significant difference between the big
model and the base model is that the fitting-offset of symbols moves toward
the negative region, suggesting overfitting for symbols in the big model. We
attribute it to the stronger memorization ability of the bigger model.

The Domain Previous experiments are done on the News dataset. To justify that
the phenomena are not domain specific, we conduct the same experiments on the
Europarl English-German dataset, which is from the Europarl domain. We
randomly sample $250,000$ sentence pairs from the Europarl training set for a
fair comparison with the News dataset, which contains $236,287$ samples.

As Figure 17 shows, the observations described in “The Language” block hold
but with slight differences in the distribution of fitting-offset of tokens
grouped by parts-of-speech, in comparison with Figure 4 evaluated on News
dataset. The major difference happens in verbs, adjectives, and nouns,
reflecting a different distribution of topics of Europarl in comparison with
News.

The Data Scale Intuitively, overfitting and underfitting should be more severe
on small datasets. For comparison, we test on a bigger dataset from Europarl.
We sample $500,000$ sentence pairs from the Europarl training set in
comparison with the model trained using $250,000$ samples.

As Figure 18 shows, the observations described in “The Language” block still
hold but the range of the distribution increases. Looking into the potential-
gains, we see that they decrease by about $1/4$ compared to that of the
experiments with $250,000$ samples. The results suggest that the fitting-
offset is more challenging to measure, and the potential-gain decreases when
the model is trained on a larger dataset, which is expected due to the larger
dataset reducing overfitting and underfitting.

##  8 Discussion on Possible Solutions

Our observation and insights are based on _general language phenomena_ : long-
tail distribution, uneven context dependencies, and different parts of speech,
which are general among different NLP tasks, and may deserve further study on
possible solutions.

Intuitively, the model training or fine-tuning process could be improved by
reducing learning on overfitting tokens and increasing learning on
underfitting tokens. One example is the previous work on abstractive
summarization Shi et al. (2018), which uses focal loss to give high-freq
tokens smaller weights and low-freq tokens larger weights, achieved 1.5 points
improvement on ROUGE-2 on the abstractive summarization task. Such _dynamic
token-specific weights_ can partially address the underfitting and overfitting
from the perspective of the frequency factor. However, the issue is far from
being solved, given that a combination of the frequency and discrepancy
factors (Figure 7) reveals more severe fitting issues than the frequency only
(Figure 3). We could expect more gains by designing more appropriate methods
to consider the combined factors.

##  9 Conclusion

We study overfitting and underfitting issues of learning targets in the
context of neural machine translation. Our experiments demonstrate that
overall rare tokens tend to underfit and frequent tokens overfit. We explored
detailed factors related to the overfitting and underfitting issues and
identified three major influencing factors, which include frequency, parts-of-
speech, and discrepancy. This shows that fitting is the result of a complex
interaction between multiple factors. Further experiments demonstrate that the
issues exist as a general problem for both non-pretraining and pretraining
settings. Future work includes the investigation of strategies to alleviate
the overfitting and underfitting issues.

## Limitations

This paper hypothesizes that seq2seq models have token-level overfitting and
underfitting issues, and provides direct evidence to support the hypothesis in
various settings, raising a valuable problem for NLP modeling. However, this
paper does not provide a solution to the problem due to the theoretical and
practical challenges of measuring the convergence speed of each token. We
leave the exploration of this topic to future work.

## Acknowledgements

We would like to thank the anonymous reviewers for their valuable feedback.
This work is funded by the China Strategic Scientific and Technological
Innovation Cooperation Project (grant No. SQ2022YFE020038) and the National
Natural Science Foundation of China (grant NSFC No. 62161160339). Zhiyang Teng
is partially supported by CAAI-Huawei MindSpore Open Fund
(CAAIXSJLJJ-2021-046A).

## References

  * Arpit et al. (2017) Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien. 2017.  A closer look at memorization in deep networks.  In _Proceedings of the 34th International Conference on Machine Learning_ , volume 70 of _Proceedings of Machine Learning Research_ , pages 233–242. PMLR.
  * Bejani and Ghatee (2021) Mohammad Mahdi Bejani and Mehdi Ghatee. 2021.  A systematic review on overfitting control in shallow and deep neural networks.  _Artificial Intelligence Review_ , 54(8):6391–6438.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Brownlee (2018) Jason Brownlee. 2018.  _Better deep learning: train faster, reduce overfitting, and make better predictions_.  Machine Learning Mastery.
  * Chatterjee and Zielinski (2022) Satrajit Chatterjee and Piotr Zielinski. 2022.  On the generalization mystery in deep learning.  _arXiv preprint arXiv:2203.10036_.
  * Dixon and Mood (1946) Wilfrid J Dixon and Alexander M Mood. 1946.  The statistical sign test.  _Journal of the American Statistical Association_ , 41(236):557–566.
  * Gong et al. (2018) Chengyue Gong, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tie-Yan Liu. 2018.  Frage: Frequency-agnostic word representation.  _Advances in neural information processing systems_ , 31.
  * Hastie et al. (2009) Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. 2009\.  _The elements of statistical learning: data mining, inference, and prediction_ , volume 2.  Springer.
  * Hodges (1955) Joseph L Hodges. 1955.  A bivariate sign test.  _The Annals of Mathematical Statistics_ , 26(3):523–527.
  * Kenton and Toutanova (2019) Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of NAACL-HLT_ , pages 4171–4186.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_.
  * Kocmi and Bojar (2017) Tom Kocmi and Ondřej Bojar. 2017.  Curriculum learning and minibatch bucketing in neural machine translation.  In _Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017_ , pages 379–386.
  * Koehn et al. (2007) Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007.  Moses: Open source toolkit for statistical machine translation.  In _Proceedings of the 45th annual meeting of the association for computational linguistics companion volume proceedings of the demo and poster sessions_ , pages 177–180.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880.
  * Li et al. (2019) Haidong Li, Jiongcheng Li, Xiaoming Guan, Binghao Liang, Yuting Lai, and Xinglong Luo. 2019.  Research on overfitting of deep learning.  In _2019 15th International Conference on Computational Intelligence and Security (CIS)_ , pages 78–81. IEEE.
  * Li et al. (2020) Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, and Joey Gonzalez. 2020.  Train big, then compress: Rethinking model size for efficient training and inference of transformers.  In _International Conference on Machine Learning_ , pages 5958–5968. PMLR.
  * Liu et al. (2020) Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020.  Multilingual denoising pre-training for neural machine translation.  _Transactions of the Association for Computational Linguistics_ , 8:726–742.
  * Maruf et al. (2019) Sameen Maruf, André FT Martins, and Gholamreza Haffari. 2019.  Selective attention for context-aware neural machine translation.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 3092–3102.
  * Platanios et al. (2019) Emmanouil Antonios Platanios, Otilia Stretcu, Graham Neubig, Barnabas Poczos, and Tom M Mitchell. 2019.  Competence-based curriculum learning for neural machine translation.  In _Proceedings of NAACL-HLT_ , pages 1162–1172.
  * Powers (1998) David MW Powers. 1998.  Applications and explanations of zipf’s law.  In _New methods in language processing and computational natural language learning_.
  * Raunak et al. (2020) Vikas Raunak, Siddharth Dalmia, Vivek Gupta, and Florian Metze. 2020.  On long-tailed phenomena in neural machine translation.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 3088–3095.
  * Rice et al. (2020) Leslie Rice, Eric Wong, and Zico Kolter. 2020.  Overfitting in adversarially robust deep learning.  In _International Conference on Machine Learning_ , pages 8093–8104. PMLR.
  * Sennrich et al. (2015) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015.  Neural machine translation of rare words with subword units.  _arXiv preprint arXiv:1508.07909_.
  * Shi et al. (2018) Yunsheng Shi, Jun Meng, Jian Wang, Hongfei Lin, and Yumeng Li. 2018.  A normalized encoder-decoder model for abstractive summarization using focal loss.  In _Natural Language Processing and Chinese Computing: 7th CCF International Conference, NLPCC 2018, Hohhot, China, August 26–30, 2018, Proceedings, Part II 7_ , pages 383–392. Springer.
  * Singh et al. (2017) Shashi Pal Singh, Ajai Kumar, Hemant Darbari, Lenali Singh, Anshika Rastogi, and Shikha Jain. 2017.  Machine translation using deep learning: An overview.  In _2017 international conference on computer, communications and electronics (comptelix)_ , pages 162–167. IEEE.
  * Sun et al. (2017) Xu Sun, Weiwei Sun, Shuming Ma, Xuancheng Ren, Yi Zhang, Wenjie Li, and Houfeng Wang. 2017.  Complex structure leads to overfitting: A structure regularization decoding method for natural language processing.  _arXiv preprint arXiv:1711.10331_.
  * Varis and Bojar (2021) Dusan Varis and Ondřej Bojar. 2021.  Sequence length is a domain: Length-based overfitting in transformer models.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 8246–8257.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ , 30.
  * Wolfe and Caliskan (2021) Robert Wolfe and Aylin Caliskan. 2021.  Low frequency names exhibit bias and overfitting in contextualizing language models.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 518–532.
  * Xu et al. (2020) Chen Xu, Bojie Hu, Yufan Jiang, Kai Feng, Zeyang Wang, Shen Huang, Qi Ju, Tong Xiao, and Jingbo Zhu. 2020.  Dynamic curriculum learning for low-resource neural machine translation.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 3977–3989.
  * Yu et al. (2022) Sangwon Yu, Jongyoon Song, Heeseung Kim, Seongmin Lee, Woo-Jong Ryu, and Sungroh Yoon. 2022.  Rare tokens degenerate all tokens: Improving neural text generation via adaptive gradient gating for rare token embeddings.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 29–45.
  * Zhang et al. (2017) Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2017\.  Understanding deep learning requires rethinking generalization.  In _International Conference on Learning Representations_.
  * Zhang et al. (2021) Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2021\.  Understanding deep learning (still) requires rethinking generalization.  _Communications of the ACM_ , 64(3):107–115.
  * Zhang et al. (2015) Jiajun Zhang, Chengqing Zong, et al. 2015.  Deep neural networks in machine translation: An overview.  _IEEE Intell. Syst._ , 30(5):16–25.
  * Zhang et al. (2018) Xuan Zhang, Gaurav Kumar, Huda Khayrallah, Kenton Murray, Jeremy Gwinnup, Marianna J Martindale, Paul McNamee, Kevin Duh, and Marine Carpuat. 2018.  An empirical exploration of curriculum learning for neural machine translation.  _arXiv preprint arXiv:1811.00739_.
  * Zhou et al. (2020) Yikai Zhou, Baosong Yang, Derek F Wong, Yu Wan, and Lidia S Chao. 2020.  Uncertainty-aware curriculum learning for neural machine translation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 6934–6944.

##  Appendix A Appendix

(a) Fitting-offset of tokens grouped by _frequency_.

(b) Fitting-offset of tokens grouped by _discrepancy_.

(c) Fitting-offset of tokens grouped by _parts-of-speech_.

Figure 12: The Language: Fitting-offset of English tokens evaluated on News
_German-English_ dataset.

Figure 13: The Language: Fitting-offset of tokens grouped by _discrepancy_ and
_frequency_.

Discrepancy | Frequency
---|---
High | Med | Low
Big | 67.14 +0.21 | 27.14 -0.27 | 9.90 +0.20
Med | 69.91 +0.07 | 47.08 -0.08 | 14.67 +0.78
Small | 62.99 +0.33 | 81.85 +0.33 | 66.38 +1.35

Table 5: The Language: Potential-gain for each category grouped by _frequency_
and _discrepancy_.

Figure 14: The Language: Fitting-offset of tokens grouped by _frequency_ and _POS_. Frequency | Function | Symbol | Number | Verb | Adj/Adv | Noun
---|---|---|---|---|---|---
High | 64.8 +0.15 | 80.08 +0.86 | nan +nan | 64.51 +1.20 | 38.24 +1.93 | 49.15 +4.01
Med | 49.82 +0.13 | 66.23 +1.67 | 69.43 +1.29 | 47.03 +0.38 | 55.78 +0.34 | 61.54 +0.21
Low | 11.12 +1.75 | 36.09 +5.98 | 72.28 +1.29 | 30.92 +0.57 | 37.76 +0.78 | 42.13 +1.20
Table 6: The Language: Potential-gain for tokens grouped by _frequency_ and
_POS_.

We present the experimental results of the additional factors here, prefixing
the name of each factor on the caption.

Figure 15: The Language: Fitting-offset of tokens grouped by _discrepancy_ and _POS_. Discrepancy | Function | Symbol | Number | Verb | Adj/Adv | Noun
---|---|---|---|---|---|---
Big | 55.19 +0.10 | 77.67 +0.58 | 28.34 +1.78 | 20.76 +0.04 | 15.50 -0.20 | 16.61 +0.19
Med | 63.84 +0.03 | 75.72 +1.05 | 38.57 +2.92 | 32.44 -0.10 | 29.21 +0.35 | 26.49 +0.92
Small | 71.99 +0.20 | 75.22 +1.32 | 86.89 +1.08 | 65.93 +1.14 | 72.88 +0.81 | 72.12 +1.20
Table 7: The Language: Potential-gain for tokens grouped by _discrepancy_ and
_POS_.

(a) Fitting-offset of tokens grouped by _frequency_.

(b) Fitting-offset of tokens grouped by discrepancy.

(c) Fitting-offset of tokens grouped by _parts-of-speech_.

Figure 16: The Model Size: Fitting-offset of German tokens evaluated on News
English-German with _big_ model.

(a) Fitting-offset of tokens grouped by _frequency_.

(b) Fitting-offset of tokens grouped by _discrepancy_.

(c) Fitting-offset of tokens grouped by _parts-of-speech_.

Figure 17: The Domain: Fitting-offset of tokens evaluated on _Europarl_
English-German ($250,000$ samples).

(a) Fitting-offset of tokens grouped by _frequency_.

(b) Fitting-offset of tokens grouped by _discrepancy_.

(c) Fitting-offset of tokens grouped by _parts-of-speech_.

Figure 18: The Data Scale: Fitting-offset of tokens evaluated on Europarl
English-German (_$500,000$ samples_)..


# Three-electron bunches in occupation of a 5-site Coulomb cluster

R. E. Putnam, Jr. and M. E. Raikh  Department of Physics and Astronomy,
University of Utah, Salt Lake City, UT 84112

###### Abstract

Attraction of like charges in a localized system implies that, upon increasing
the Fermi energy, the occupation of the system changes as,
$n\rightarrow(n+2)$, while the occupation, $(n+1)$, is skipped. In this way,
the attraction translates into the bunching of electrons. For a localized
system of $N=4$ sites, attraction of electrons manifests itself in skipping of
$n=2$ occupation. The origin of the attraction is rearrangement of the
occupations of the surrounding sites which plays the role of a polaronic
effect. We consider an $N=5$-site cluster and demonstrate that, with screened
Coulomb repulsion, three-electron bunching becomes possible, i.e. the change
of occupation $n=1\rightarrow n=4$ with $n=2$ and $n=3$ occupations skipped.

##  I Introduction

A question whether two electrons can attract each other without lattice
dynamics involved was previously posed in many papers. Another formulation of
this question is whether the formation of negative-$U$ centers,Anderson1975
which are believed to be due to phonons, is possible as a result of electron-
electron repulsion. An appeal of having purely electronic negative-$U$ centers
is that such centers can be viewed as precursors of purely electronic
superconducting state.

Figure 1: A criterion, $E_{N}^{n}+E_{N}^{n+2}<2E_{N}^{n+1}$, of the attraction
of two electrons implies a concavity of the curve $E_{N}^{n}$. As a result,
the population evolves as $n\rightarrow(n+2)$.  Figure 2: a) Illustration of
the 4-site model. The energy of the central site, $\varepsilon_{0}$, is
smaller than the energies of the corner sites, $\varepsilon_{1}$. For a purely
Coulomb system, the repulsions, $W$ and $V$, are related as $3^{1/2}$. For a
gate-screened interaction they are related as $3^{3/2}$; b) Illustration of
the 5-site model. The repulsions $W$, $U$, and $V$ are related as
$2:2^{1/2}:1$ for a purely Coulomb interaction and as $2^{3}:2^{3/2}:1$ for
the gate-screened interaction.

On the theory side, electron attraction in the repulsive systems was invoked
to (i) explain the valence-skipping phenomenon, see e.g. Refs. Varma, ; Dzero,
; Harrison, ; skipping, ; skipping1, , when the valence of certain elements
does not occur in the compounds which they form; (ii) derive pairing within
certain limits of the Hubbard model,Raghu  which, essentially, amounts to
replacing phonons by magnons.Batista  Also, as demonstrated in Ref. LiangFu, ,
formation of charge-2e excitations, “trimers”, is favored in doped transition-
metal dichalcogenide bilayers near the half-filling. The origin of pairing in
Ref. LiangFu,  with two type of sites having different energy is reduction of
electrostatic energy which dominates over the kinetic energy. Minimal model of
a trimer requires four sites. Note that such a 4-site model of pairing was
previously studied in Ref. Zhukov,  in different relation.

On the experimental side, attention to the issue of possible attraction of
localized electrons was drawn by the early experiments.Ashoori1992 ;
Ashoori1993 ; Ashoori1997  In these experiments, one-by-one magneto-tunneling
events of electrons from an electrode into a big semiconductor island. Certain
portion of events revealed bunching of electrons into pairs. Accounting this
bunching by attraction of electrons within a 4-site model encounters a problem
that incoherent tunneling of two electrons takes too much time. Later
experimentsHeiblum ; Ashoori  suggested that two-electron events take place at
the edge and are related to the formation of the edge states in magnetic
field.

Demonstration of pairing due to repulsion on a truly microscopic level was
reported in Ref. Hamo, . Experimental setup in Ref. Hamo,  was very similar to
the 4-site cluster (2-site polarizer and double-well quantum dot). In a
certain domain of gate voltages, one dot of a double-well system was either
empty or accommodated a pair of electrons as a result of reoccupation of the
dots constituting the polarizer.

Due to flexibility of their nanotube-based technique, the authors of Ref.
Hamo,  suggested several variants for scaling-up their setup. In particular,
their approach can tackle the question whether more complex many-particle
processes can be realized experimentally. In this paper we demonstrate that,
adding one extra site to the 4-site model, opens a possibility of three-
electron bunches. To establish a criterion for such three-electron bunches, we
introduce $E_{N}^{n}$, which is a minimal energy of $n$ electrons in a cluster
of $N$ sites. Next electron enters the cluster when the Fermi level position
in surrounding system is equal to

$\mu_{1}=E_{N}^{n+1}-E_{N}^{n}.$ (1)

Two electrons enter the cluster at

$\mu_{2}=\frac{E_{N}^{n+2}-E_{N}^{n}}{2}.$ (2)

Formation of a $2e$ pair takes place when $\mu_{2}<\mu_{1}$, i.e. when the
usual condition $\big{(}E_{N}^{n+2}+E_{N}^{n}\big{)}<2E_{N}^{n+1}$,
illustrated in Fig. 1, is met. Continuing the reasoning, three electrons enter
the cluster at

$\mu_{3}=\frac{E_{N}^{n+3}-E_{N}^{n}}{3}.$ (3)

Then a $3e$-bunch is favored over single-electron and $2e$-bunch under the
conditions $\mu_{3}<\mu_{1}$ and $\mu_{3}<\mu_{2}$.

##  II Two-electron bunching in a 4-site cluster

For didactic reasons, in this section we review the steps unveiling the two-
electron bunching in the 4-site model. Extension to the 5-site model involves
all the similar steps. On the other hand, the flow of logics offers an insight
about possible extensions.

Step I. A 4-site cluster is illustrated in Fig. 2a. We examine the evolution
of the minimal energy of configurations with varying $n$. The 4-site cluster
is completely filled when n=4, i.e. only one configuration is possible. It is
also obvious that, for $n=1$, the minimal energy corresponds to filling the
central site with energy $\varepsilon_{0}<\varepsilon_{1}$. Thus
$E_{4}^{1}=\varepsilon_{0}$. When $n=2$, the electron in the center can either
stay there or move to the vertex in order to reduce the potential energy.
Thus, the candidates for $E_{4}^{2}$ are

${\tilde{E}}_{4}^{2}=\varepsilon_{0}+\varepsilon_{1}+W~{}\text{or}~{}{\tilde{\tilde{E}}}_{4}^{2}=2\varepsilon_{1}+V.$
(4)

Equally, there are two candidates to minimize the energy when the population
of the cluster is $n=3$, namely

${\tilde{E}}_{4}^{3}=\varepsilon_{0}+2\varepsilon_{1}+2W+V~{}\text{or}~{}{\tilde{\tilde{E}}}_{4}^{3}=3\varepsilon_{1}+3V.$
(5)

Step II. At this step, we make two crucial assumptions:
${\tilde{E}}_{4}^{2}<{\tilde{\tilde{E}}}_{4}^{2}$ and
${\tilde{E}}_{4}^{3}>{\tilde{\tilde{E}}}_{4}^{3}$, which translate into the
following inequalities

$\displaystyle 2\varepsilon_{1}+V>\varepsilon_{0}+\varepsilon_{1}+W,$
$\displaystyle 3\varepsilon_{1}+3V<\varepsilon_{0}+2\varepsilon_{1}+V+2W.$ (6)

These inequalities imply that it is energetically unfavorable for two
electrons to vacate the center, but it is favorable for three electrons to
vacate the center. Note that Eq. (II) can be presented in a concise form

$1-\frac{V}{W}<\frac{\varepsilon_{1}-\varepsilon_{0}}{W}<2\bigg{(}1-\frac{V}{W}\bigg{)}.$
(7)

When the conditions Eq. (II) are met, we have

$E_{4}^{2}=\tilde{E_{4}^{2}},~{}E_{4}^{3}={\tilde{\tilde{E}}}_{4}^{3}.$ (8)

Step III. At this step we require that the dependence $E_{4}^{n}$ is
“concave”, as illustrated in Fig. 1. Using Eq. (8), the pairing condition
$E_{4}^{1}+E_{4}^{3}<2E_{4}^{2}$ takes the form

$\varepsilon_{0}+3\big{(}\varepsilon_{1}+V\big{)}<2\big{(}\varepsilon_{0}+\varepsilon_{1}+W\big{)}.$
(9)

Note that the above condition restricts the energy difference between the
center and the vertex sites:
$\frac{\varepsilon_{1}-\varepsilon_{0}}{W}<2-\frac{3V}{W}$.

Step IV. We now go back to the assumptions made above, Eq. (7), and test
whether they are consistent with concavity. This test is illustrated in Fig.
4. We see that the second inequality is satisfied automatically. To satisfy
the fist inequality, one needs $V<\frac{W}{2}$. As seen from Fig. 2a, for
purely Coulomb interaction, the relation between $V$ and $W$ is
$V=\frac{W}{3^{1/2}}$. Thus, the first condition is not satisfied. It can be,
however, satisfied if the gate is present at a distance, $d$, above the plane
of the cluster. Then the Coulomb interaction, $V(\rho)=\frac{e^{2}}{\rho}$,
gets modified to
$V(\rho)={e^{2}}\Big{[}\frac{1}{\rho}-\frac{1}{\left(\rho^{2}+4d^{2}\right)^{1/2}}\Big{]}$.
Then, for $\rho\gg d$, we have $V=\frac{W}{3^{3/2}}<\frac{W}{2}$, so that in
the domain

$W-V<\varepsilon_{1}-\varepsilon_{0}<2W-3V,$ (10)

shown in Fig. 4a with red, 2e-pairing is possible.

Figure 3: Illustration of three-electron bunching in the cluster of $N$ sites.
The bunch is possible under the conditions
$\frac{1}{3}\left(E_{N}^{n+3}-E_{N}^{n}\right)<\frac{1}{2}\left(E_{N}^{n+2}-E_{N}^{n}\right)<E_{N}^{n+1}-E_{N}^{n}$.
Figure 4: Different domains of the dimensionless imbalance,
$\frac{\varepsilon_{1}-\varepsilon_{0}}{W}$, of the site energies: a) 4-site
model. In the domain of imbalances,
$\Big{\\{}1-\frac{V}{W},2\big{(}1-\frac{V}{W}\big{)}\Big{\\}}$ the energy of
two-electron configuration is minimal when one electron occupies the central
site, while for three-electron configuration the energy is minimal when all
three electrons reside in the vertices. In the grey region the condition of
concavity, illustrated in Fig. 1, is satisfied, so that 2e-pairing occurs in
the domain marked with red. b) 5-site model. In the red domain the occupation
of the cluster with chemical potential evolves as $n=1\rightarrow n=4$.

##  III Three-electron bunching in a 5-site cluster

We now generalize the reasoning from the previous section to the 5-site model
illustrated in Fig. 2b. The central site is surrounded by four vertexes. Three
repulsion energies, shown in the figure, are related as $U<V<W$ in accordance
with distances between the corresponding sites. Similarly to the 4-site model,
we assume that the energy, $\varepsilon_{0}$, of the central site is lower
than $\varepsilon_{1}$-the energy of the vertex sites. Obviously,
$E_{5}^{1}=\varepsilon_{0}$. Our strategy in a search for three-electron
bunching is based on the argument that, by placing electrons in the vertexes,
the configuration loses in its on-site energy, but gains due to the reduction
in potential energy.

Step I. There are three candidates for the double occupation of the cluster.
They are

${\tilde{E}}_{5}^{2}=\varepsilon_{0}+\varepsilon_{1}+W~{}\text{or}~{}{\tilde{\tilde{E}}}_{5}^{2}=2\varepsilon_{1}+V~{}\text{or}~{}\Big{(}{\tilde{\tilde{\tilde{E}}}}\Big{)}_{5}^{2}=2\varepsilon_{1}+U.$
(11)

Since $V>U$, the energy ${\tilde{E}}_{5}^{2}$ is smaller than
${\tilde{\tilde{E}}}_{5}^{2}$ leaving us with only two candidates. One can
also see that there are three candidates for the triple occupation, namely

$\displaystyle{\tilde{E}}_{5}^{3}=3\varepsilon_{1}+2V+U~{}\text{or}~{}{\tilde{\tilde{E}}}_{5}^{3}=2\varepsilon_{1}+\varepsilon_{0}+2W+U,~{}\text{or}~{}$
$\displaystyle\Big{(}{\tilde{\tilde{\tilde{E}}}}\Big{)}_{5}^{3}=2\varepsilon_{1}+\varepsilon_{0}+2W+V.$
(12)

We can now compare ${\tilde{\tilde{E}}}_{5}^{3}$ to
$\Big{(}{\tilde{\tilde{\tilde{E}}}}\Big{)}_{5}^{3}$ and realize that the
latter has higher energy since $V>U$. This, again, leaves us with two
candidates for ${\tilde{E}}_{5}^{3}$.

Finally, the occupation of the cluster with four electrons is possible in two
configuations having different energies. These energies are

${\tilde{E}}_{5}^{4}=\varepsilon_{0}+3\varepsilon_{1}+3W+2V+U,~{}\text{or}~{}{\tilde{\tilde{E}}}_{5}^{4}=4\varepsilon_{1}+4V+2U.$
(13)

Obviously, the state with energy, ${\tilde{E}}_{5}^{4}$ is four-fold
degenerate depending on which vertex is empty.

Step II. At this point, we make three crucial assumptions:

  1. 1.

${\tilde{\tilde{E}}}_{5}^{2}>{\tilde{E}}_{5}^{2},$

  2. 2.

${\tilde{\tilde{E}}}_{5}^{3}>{\tilde{E}}_{5}^{3},$

  3. 3.

${\tilde{\tilde{E}}}_{5}^{4}<{\tilde{E}}_{5}^{4}.$

The assumptions are made in order to ensure that the ground states with $n=2$
and $n=3$ include the center site, while the ground state with $n=4$ includes
the sites in the vertexes. Using Eqs. (11), (III), (III), the above
assumptions can be rewritten as

$\displaystyle\varepsilon_{0}+\varepsilon_{1}+W<2\varepsilon_{1}+U\Rightarrow
1-\frac{U}{W}<\frac{\varepsilon_{1}-\varepsilon_{0}}{W},$ (14)
$\displaystyle\varepsilon_{0}+2\varepsilon_{1}+2W+U>3\varepsilon_{1}+2V+U$
$\displaystyle\Rightarrow
2\Big{(}1-\frac{V}{W}\Big{)}>\frac{\varepsilon_{1}-\varepsilon_{0}}{W},$ (15)
$\displaystyle\varepsilon_{0}+3\varepsilon_{1}+3W+2V+U>4\varepsilon_{1}+4V+2U$
$\displaystyle\Rightarrow
3-\frac{2V}{W}-\frac{U}{W}>\frac{\varepsilon_{1}-\varepsilon_{0}}{W}.$ (16)

Now it is convenient to restructure the above inequalities into two separate
ranges as

$\displaystyle
1-\frac{U}{W}<\frac{\varepsilon_{1}-\varepsilon_{0}}{W}<2\bigg{(}1-\frac{V}{W}\bigg{)},$
$\displaystyle
1-\frac{U}{W}<\frac{\varepsilon_{1}-\varepsilon_{0}}{W}<3-2\frac{V}{W}-\frac{U}{W}.$
(17)

Whether the three-electron bunches are allowed or not depends on whether or
not the domains Eq. (III) overlap, as illustrated in Fig. 4b. After making the
above assumptions we can specify the ground-state configuration for each $n$,
namely

$\displaystyle
E_{5}^{2}={\tilde{E}}_{5}^{2},~{}E_{5}^{3}={\tilde{E}}_{5}^{3},~{}E_{5}^{4}={\tilde{\tilde{E}}}_{5}^{4}$
(18)

Step III. We now require that the ground-state energies $E_{5}^{n}$ are
arranged as shown in Fig. 3, or, in other words, we require that these
energies satisfy the conditions

$\frac{1}{3}\left(E_{5}^{4}-E_{5}^{1}\right)<\frac{1}{2}\left(E_{5}^{3}-E_{5}^{1}\right)<E_{5}^{2}-E_{5}^{1}.$
(19)

The first and the second conditions can be cast into a traditional form

$\displaystyle\frac{\varepsilon_{1}-\varepsilon_{0}}{W}>2\frac{V}{W}+\frac{U}{W},$
$\displaystyle\frac{\varepsilon_{1}-\varepsilon_{0}}{W}<3-4\frac{V}{W}-2\frac{U}{W}.$
(20)

We see that, in the same way as Eq. (III), the necessary requirements for 3-e
bunches restrict the asymmetry in single-electron energies both from below and
from the above.

IV. Now a nontrivial question arises: is there a domain in which the
conditions Eq. (III) and Eq. (III) are consistent with each other? It is
apparent that for purely Coulomb repulsion the requirements Eq. (III) cannot
be met. Indeed, with purely Coulomb interaction, one has
$\frac{U}{W}=\frac{1}{2}$, while $\frac{V}{W}=\frac{1}{2^{1/2}}$. Then the
right-hand side in the second inequality Eq. (III) is negative. Turning to
screened Coulomb repulsion, we have $\frac{U}{W}=\frac{1}{8}$, while
$\frac{V}{W}=\frac{1}{2^{3/2}}$. Then Eq. (III) places the asymmetry,
$\frac{\varepsilon_{1}-\varepsilon_{0}}{W}$, into the interval
$\big{(}0.875,1.29\big{)}$. At the same time, Eq. (III) restricts the
asymmetry to the interval $\big{(}0.83,0.95\big{)}$. We see that two
restricting intervals overlap proving that 3e-bunches are allowed.

##  IV Conclusion

Certainly the conditions for the formation of 3e-bound state in the repulsive
system are more restrictive than the conditions for the formation of the
2e-boumd state. We do not know whether the bunches containing more than three
electrons are possible, but our finding motivates to search for them.

## References

  * (1) P. W. Anderson, “Model for the Electronic Structure of Amorphous Semiconductors,” Phys. Rev. Lett. 34, 953 (1975).
  * (2) C. M. Varma, “Missing Valence States, Diamagnetic Insulators, and Superconductors,” Phys. Rev. Lett. 61, 2713 (1988).
  * (3) W. A. Harrison, “Valence-skipping compounds as
positive-$U$ electronic systems, Phys. Rev. B 74, 245128 (2006).

  * (4) I. Hase and T. Yanagisawa, “Madelung energy of the valence-skipping compound BaBiO3,” Phys. Rev. B 76, 174103 (2007).
  * (5) M. Dzero and J. Shmalian, “Superconductivity in Charge Kondo Systems,” Phys. Rev. Lett. 94, 157003, (2005).
  * (6) H. U. R. Strand, “Valence-skipping and negative-U in the d-band from repulsive local Coulomb interaction,” Phys. Rev. B 90, 155108 (2014).
  * (7) S. Raghu, S. A. Kivelson, and D. J. Scalapino, “Superconductivity in the repulsive Hubbard model: An asymptotically exact weak-coupling solution,” Phys. Rev. B 81, 224505 (2010).
  * (8) S.-S. Zhang, W. Zhu, and C. D. Batista, “Pairing from strong repulsion in triangular lattice Hubbard model,” Phys. Rev. B 97, 140507(R) (2018) .
  * (9) K. Slagle and L. Fu, “Charge transfer excitations, pair density waves, and superconductivity in moiré materials, ” Phys. Rev. B 102, 235423 (2020).
  * (10) M. E. Raikh, L. I. Glazman, and L. E. Zhukov, “Two-Electron State in a Disordered 2D Island: Pairing Caused by the Coulomb Repulsion,” Phys. Rev. Lett. 77, 1354 (1996).
  * (11) R. C. Ashoori, H. L. Stormer, J. C. Weiner, L. N. Pfeiffer, S. J. Pearton, K. W. Baldwin, and K. W. West, “Single-Electron Capacitance Spectroscopy of Discrete Quantum Levels,” Phys. Rev. Lett. 68, 3088 (1992).
  * (12) R. C. Ashoori, H. L. Stormer, J. C. Weiner, L. N. Pfeiffer, K. W. Baldwin, and K. W. West, “N-Electron Ground State Energies of a Quantum Dot in Magnetic Field,” Phys. Rev. Lett. 71, 613 (1993).
  * (13) N. B. Zhitenev, R. C. Ashoori, L. N. Pfeiffer, and K. W. West, “Periodic and Aperiodic Bunching in the Addition Spectra of Quantum Dots,” Phys. Rev. Lett. 79 2308 (1997).
  * (14) A. Demir, N. Staley, S. Aronson, S. Tomarken, K. West, K. Baldwin, L. Pfeiffer, and R. Ashoori, “ Correlated double-electron additions at the edge of a two-dimensional electronic system,” Phys. Rev. Lett. 126, 256802 (2021).
  * (15) H. K. Choi, I. Sivan, A. Rosenblatt, M. Heiblum, V. Umansky, and D. Mahalu, “Robust Electron Pairing in the Integer Quantum Hall Effect Regime,” Nat. Commun. 6, 7435 (2015).
  * (16) A. Hamo, A. Benyamini, I. Shapir, I. Khivrich, J. Waissman, K. Kaasbjerg, Y. Oreg, F. von Oppen, and S. Ilani, “Electron Attraction Mediated by Coulomb Repulsion,” Nature 535, 395 (2016).