# Divergence-Based Domain Transferability for Zero-Shot Classification

Alexander Pugantsov  Richard McCreadie
School of Computing Science, University of Glasgow, UK

###### Abstract

Transferring learned patterns from pretrained neural language models has been
shown to significantly improve effectiveness across a variety of language-
based tasks, meanwhile further tuning on intermediate tasks has been
demonstrated to provide additional performance benefits, provided the
intermediate task is sufficiently related to the target task. However, how to
identify related tasks is an open problem, and brute-force searching effective
task combinations is prohibitively expensive. Hence, the question arises, _are
we able to improve the effectiveness and efficiency of tasks with no training
examples through selective fine-tuning?_ In this paper, we explore statistical
measures that approximate the divergence between domain representations as a
means to estimate whether tuning using one task pair will exhibit performance
benefits over tuning another. This estimation can then be used to reduce the
number of task pairs that need to be tested by eliminating pairs that are
unlikely to provide benefits. Through experimentation over 58 tasks and over
6,600 task pair combinations, we demonstrate that statistical measures can
distinguish effective task pairs, and the resulting estimates can reduce end-
to-end runtime by up to 40%.

##  1 Introduction

As the accuracy of neural models continues to increase, so does the
computational cost of training and storing them. One approach of mitigating
such cost is through using pretrained models to enhance performance on a
downstream task, a paradigm commonly referred to as transfer learning.
However, when and why transfer learning works is not concretely understood.
Traditionally, selecting the best settings, i.e. tasks and hyperparameters,
for transfer often involves an extensive trial-and-error process over many
combinations and can quickly make the prospect of applying transfer learning
undesirable. As such, it would be valuable to estimate whether a task pair
combination will be effective pre-training, i.e. estimate the transferability
of a source task to a target task.

The most optimal transferability metric would be resource-efficient, such that
it is capable of accurately predicting the final performance of the model
whilst minimising the amount of processing required to compute it. To this
end, several works (Van Asch and Daelemans, 2010; Ruder and Plank, 2017;
Ramesh Kashyap et al., 2021) have focused on estimating transferability prior
to fine-tuning, using statistical measures of divergence between the
underlying feature spaces of model pairs. Domain divergence measures are used
to produce a notion of distance between pairs of domains by comparing their
representations and have seen significant usage in works which investigate the
correlation between their estimations and performance change (Van Asch and
Daelemans, 2010; Ramesh Kashyap et al., 2021).

Subsequent transfer learning works have also demonstrated that competitive
model performance can be achieved on some target tasks even if no training
samples for that task are available, an approach known as _zero-data/shot
learning_ (Larochelle et al., 2008). In this work, we investigate the
effectiveness of domain divergence measures in estimating the performance of
zero-shot classification models, wherein models further tuned on one source
task are used to directly predict on the test set of a target task without any
target training samples. Specifically, we leverage the information captured by
these measures as features to an auxiliary learner, whose outputs are used to
rank the most effective source model for transfer to a given target task.
Through the analysis of 58 sentiment classification domains, we:  (1) perform
a correlation analysis between each independent measure and each source-
target, macro-averaged $F_{1}$-score performance output;  (2) and, for each
target task, we train a series of auxiliary regression models to predict their
projected performance;  (3) we then convert these into rankings of source-
target pairs and evaluate the capability of our learners to find the best
source model for each given target domain.

##  2 Experiment Setup

|
---|---
(a) Average NDCG@K for $N_{S}=1000$. | (b) Average NDCG@K for $N_{S}=25000$.
Figure 1: NDCG@K averaged across tasks for different feature sets. Higher is
better. $DIV$, $H$, $\sigma$ denote divergence-, entropy-, and moments-based
measures, respectively.

Measures: Ramesh Kashyap et al. (2021) provide categories of divergence
measures; two of which we use in our work: Geometric measures which calculate
distances between continuous representations such as word embeddings and
Information-theoretic measures which capture the distance between
representations such as frequency-based distributions over co-occurring
n-grams. We do not report higher-order measures as in the aforementioned work,
but instead report moments-based features, which better describe the
characteristics of our individual term distributions—namely the mean,
variance, skewness, and kurtosis of our distributions—as features to our
learner. Following prior work (Tsvetkov et al., 2016; Ruder and Plank, 2017),
we further complement the above measures by making use of several metrics that
capture diversity and prototypicality such as entropy-based features; in our
work, these measures are used with probability distributions, and are, as
such, categorised here as information-theoretic. Specifically, we use the
following metrics:

  * •

Geometric: Cosine distance, $l_{1}$\- (or Manhattan dist.) and $l_{2}$-norm
(or Euclidean dist.).

  * •

Information-theoretic: Rényi and Jensen-Shannon divergences (Wong and You,
1985; Rényi et al., 1961), Bhattacharyya Coeff. (Bhattacharyya, 1943),
Wasserstein distance (Kantorovich, 1960), Entropy and Rényi Entropy (Shannon,
1948; Rényi et al., 1961), Simpson’s Index (Simpson, 1949).

  * •

Moments-based: Mean, variance, skewness, and kurtosis ($\sigma^{n}$ where
$n\in[1..4]$).

Representations: To compute the above metrics, we use two different
representations from prior work by Ruder and Plank (2017), specifically 1)
discrete probabilities of the most common terms across domains, using a fixed-
size vocabulary $V$, where $|V|=10,000$; and 2) a summation over probability-
weighted term embeddings in each document, averaged to produce a single
vector:

  1. (1)

Term Distributions (TD) (Plank and van Noord, 2011): $t\in\mathbb{R}^{|V|}$
where $t_{i}$ is the probability of the $i$-th word in the vocabulary $V$.

  2. (2)

BERT Embeddings (BE) (Devlin et al., 2018):
$\frac{1}{n}\sum_{i}v_{w_{i}}\sqrt{\frac{a}{p(w_{i})}}$ where $n$ is the
number of words with embeddings in the document, $v_{w_{i}}$ is the pretrained
embedding of the $i$-th term, $p(w_{i})$ its probability, and $a$ is a
smoothing factor used to discount frequent probabilities. Following guidelines
by Ruder and Plank (2017), we use this representation with geometric-based
measures only, as embedding vectors can be negative.

Generally, since we are using these representations in a zero-shot setting, we
compute divergences between the source-task training set ($D_{S}$) and the
target-task test set ($D_{T}$). Entropy and moments-based measures are not
used to estimate divergence between domains but used only to compute within-
domain characteristics, i.e. on individual term distributions.

Datasets and Domains: We make use of two ratings prediction datasets with
classes in the range 1-5 and, similarly to Zhang et al. (2015), reformulate
the task as a binary sentiment classification task by merging the provided
labels; 1-2: negative and 3-4: positive. We focus on similar, within-task
(i.e. sentiment classification) datasets to  (1) remove task variation as a
variable,  (2) and to highlight the effectiveness of using statistical
measures to compute divergence between similar domains which may have very
minute differences in semantics and other linguistic phenomena.  The first is
the Amazon Product Reviews dataset, using the review title and review content
fields as features and divide the dataset by the product category labels. As a
supplementary contribution to our work, we create the Multi-Domain Yelp
Business Reviews dataset by extending the original Reviews and Business
datasets provided by the Yelp Dataset Challenge, mapping top-level111We
determined which categories were top-level based on an article written by Yelp
categories of businesses to their respective reviews. After filtering out low-
sample ($\leq 30,000$) domains, we have 42 and 16 domains for the Amazon and
Yelp datasets, respectively.

Implementation Details: We use $\text{BERT}_{base}$ (Devlin et al., 2018) as
our base model in the experiments. With both runtime- and storage-efficiency
in mind, we make use of adapter modules (Pfeiffer et al., 2020) and train each
of the domains as a source task adapter, leaving the rest of BERT’s parameters
frozen. More implementation and hyperparameter details can be found in
Appendix A.

We divide our experiments into two separate settings by source-task sample
size, $N_{S}\in[1000,25000]$. We train 116 source-task adapters (58 $D_{S}$
$\times$ 2 $N_{S}$ settings), and evaluate a total of 6,612 source-target
combinations for analysis. For our auxiliary learner, we use an XGBoost (Chen
and Guestrin, 2016) regression model. We split our training and test sets by
the target task and train 2,900 regression models (for each of the 58 target
domains, 2 sample sizes settings, 5 feature sets, and over 5 random seeds).

##  3 Experiments and Results

Category | Measure | Term Distributions | BERT Embeddings
---|---|---|---
|  | 1K | 25K | 1K | 25K
Geometric | Cosine Dist. | -0.3683* | -0.4801* | -0.3078* | -0.5792*
$L_{1}$ Dist. | -0.3699* | -0.6243* | -0.0792* | -0.4045*
$L_{2}$ Dist. | -0.3345* | -0.3551* | -0.0923* | -0.4228*
| Rényi Div. | -0.4766* | -0.4273* |
| Jensen-Shannon Div. | -0.3726* | -0.5914* |
| Wasserstein Dist. | -0.2225* | -0.3266* |
Info. | Bhattacharyya Coeff. | 0.3700* | 0.5743* |
Theoretic | Entropy ($D_{S}$) | 0.1838* | 0.2275* |
| Entropy ($D_{T}$) | -0.1603* | 0.0486* |
| Rényi Entropy ($D_{S}$) | -0.1836* | -0.2284* |
| Rényi Entropy ($D_{T}$) | 0.1618* | -0.0503* |
| Simpson’s Index ($D_{S}$) | 0.0842* | 0.1359* |
| Simpson’s Index ($D_{T}$) | -0.3127* | -0.1442* |
|  $\sigma^{1}$ ($D_{S}$) | -0.1321* | -0.1792* |
|  $\sigma^{1}$ ($D_{T}$) | -0.1245* | -0.2227* |
|  $\sigma^{2}$ ($D_{S}$) | -0.1289* | -0.1523* |
Moments |  $\sigma^{2}$ ($D_{T}$) | -0.1749* | -0.2549* |
Based |  $\sigma^{3}$ ($D_{S}$) | 0.0106 | 0.0287 |
|  $\sigma^{3}$ ($D_{T}$) | -0.3823* | -0.2643* |
|  $\sigma^{4}$ ($D_{S}$) | 0.0006 | 0.0234 |
|  $\sigma^{4}$ ($D_{T}$) | -0.3491* | -0.2473* |

Table 1: Spearmans $\rho$ correlations between each measure and source-target macro-averaged $F_{1}$-score performance. Asterisk denotes measure was statistically significant (_P_ $\leq 0.05$). |
---|---
(a) Average F1@K vs. Runtime@K for $N_{S}=1000$. | (b) Average F1@K vs. Runtime@K for $N_{S}=25000$.
Figure 2: F1@K averaged across tasks vs. Total Runtime@K of source-task
adapters. Higher is better. Runtime is reported in hours.

To evaluate whether the aforementioned statistical measures are predictive of
task pair transferability, we perform a correlation analysis between the
source-target pairs within each domain, where we contrast the statistical
measure (which provides information about $D_{S}$, $D_{T}$, or the differences
between them) and the resultant performance (measured using macro-averaged
$F_{1}$) when using $D_{S}$ to tune a model for application on task $T$. Table
1 reports Spearman’s Rho ($\rho$) across all sample size settings for each
statistical measure. Higher correlations (distance from 0) indicate increasing
predictiveness of the statistical measure of transferability.

Using the interpretation of Spearman’s Rho ($\rho$) correlation coefficients
by Dancey and Reidy (2007), we make the following observations:  (1) Geometric
measures exhibited a moderate-to-strong correlation for Term Distributions
across both sample size settings, and strong correlations at $N_{S}=25000$ for
BERT Embeddings;  (2) Between-domain Information-theoretic measures also
showed moderate-to-strong performance correlations;  (3) All entropy-based
measures (aside from Simpson’s Index for $D_{T}$) had a weak or negligible
correlation with performance;  (4) Out of all of the higher-order moments of
Term Distributions, only the skewness and kurtosis of $D_{T}$ ($\sigma^{3}$
and $\sigma^{4}$) seemed to have a moderate relationship at $N_{S}=1000$, and,
generally, the moments of $D_{T}$ seemed to be more correlated than that of
$D_{S}$.

Overall, divergence measures with both representations seemed to be more
predictive of source-target performances than with entropy or moments-based
metrics. However, since it is unlikely that each measure was independently
capable of predicting performance, we trained a series of regression models
for each target task, combining these measures. Specifically, we train an
XGBoost (Chen and Guestrin, 2016) regression model (XGBRegressor) with each of
the feature sets as our inputs, over five random seeds, for each of the 58
target domains and 2 sample size settings, producing 2,900 models for
evaluation.

Figure 1 shows the Average NDCG@K values for each of these feature sets. We
average the NDCG@K values across each of the 58 domains, and again over each
of the 5 seeds. For both models, we achieve the best quality ranking using all
of the features ($ALL$). Moreover, using divergence measures with both sets of
representations ($DIV_{TD,BE}$) achieved a better ranking than using them in
isolation ($DIV_{TD}$ or $DIV_{BE}$) for both settings. It is also interesting
to note that the feature set containing only the entropy and moments-based
($H+\sigma$) values achieve better performance than that of those estimated
via divergence measures when the source sample size is significantly limited,
coinciding with patterns found in our correlation analysis (Table 1); it may
be the case that these features are more discriminative in cases where
divergence measures are not as expressive.

Finally, we evaluate the practical, downstream application of our regression
models by considering how they may be used to reduce the search time in
finding appropriate source models for transfer. For this experiment, we assume
the user has a particular training budget $K$ to train task pairs for
transfer. The more task combinations that are tried, the more likely the user
is to find a better-performing model for a particular task. We use our
regression models to determine the order of task pairs to be tried, using the
best feature set from our prior experiments (See Fig. 1). We compare with a
random ordering of source-task models, which we average over five random seeds
to reduce variance. Figure 2 shows the results of our experiments. For
$N_{S}=1000$, the best macro-averaged $F_{1}$ performance score over all tasks
is 0.8482 which, with a grid search over all task combinations, would require
4.7 hours of training. With our approach, we can achieve a 44% reduction in
training time from 4.7 to 2.6 hours to achieve the same performance. For
$N_{S}=25000$, we can achieve the maximum score of 0.8899 through a grid
search of all source-target combinations at a cost of 42.4 hours of training
time. With our approach, we can achieve the same score with only 24.9 hours of
training or a 41% reduction in training time.

In determining the overall runtime of our approach, we factor in the
computational cost associated with generating the features required to train
our regression models. Our feature generation process consists of three
stages:  (1) the generation of term distributions and embedding
representations,  (2) the computation of statistical measures in Table 1,  (3)
and the execution of regression experiments using the $ALL$ feature set.  A
total of 232 term distributions and an equivalent number of embedding
representations (58 target domains each with separate training and test sets,
in two different sample size settings) were generated. The generation of both
sets of representations takes 5.7 minutes at $N_{S}=1000$ and 45.9 minutes at
$N_{S}=25000$. The time taken to compute all statistical measures across both
representations is 3 minutes at $N_{S}=1000$ and 6.6 minutes at $N_{S}=25000$.
Finally, the time taken to run the regression experiments was 5.4 minutes in
total. Despite the added computational cost, our approach has resulted in a
substantial reduction in end-to-end runtime, boasting a 40% reduction at
$N_{S}=1000$ and a 39% reduction at $N_{S}=25000$, demonstrating the
efficiency of our approach and the value-add of predicting which task pairs
are transferable beforehand.

##  4 Conclusions and Future Work

In this paper, we have shown that domain divergence measures and other
statistical quantities are predictive of zero-shot transferability between
tasks, and that this can be used to markedly reduce time when developing
effective zero-shot models. Indeed, by predicting which source-target task
pairs were likely transferable pre-tuning, we were able to reduce the end-to-
end time taken to find the best source-target task pairs (trained on 1,000
source-task samples) by 40%. On the other hand, while we have demonstrated the
value of using these metrics in performance estimation, there are a number of
further directions worth investigating, namely:  (1) examine the
transferability across a wider range of domain and task types;  (2)
investigate more complex, higher-order measures such as those outlined by
Ramesh Kashyap et al. (2021);  (3) and to experiment with few-shot and other
limited data settings.

## Limitations

The most pronounced limitation in our work is the small variance in
performance scores. As can be seen in Figure 2, the difference between the
lower and maximum performances is small. The difference between the minimum
and maximum average performance is 0.0305 and 0.0320 for $N_{S}=1000$ and
$N_{S}=25000$, respectively. Even at the individual, source-target model
level, the standard deviation of performance scores at each source-task sample
size setting is 0.0363 and 0.0311. As such, the benefits of zero-shot transfer
are not as apparent between these domains as they would be where the domains
are more textually distinct. Nevertheless, we believe it is notable that
statistical measures of domain divergence and the other metrics were
sufficiently capable of discerning between more effective source-task pairs,
even when the domains were similar, illustrating the promise of this approach.

## References

  * Bhattacharyya (1943) Anil Bhattacharyya. 1943.  On a measure of divergence between two statistical populations defined by their probability distributions.  _Bull. Calcutta Math. Soc._ , 35:99–109.
  * Chen and Guestrin (2016) Tianqi Chen and Carlos Guestrin. 2016.  Xgboost: A scalable tree boosting system.  _CoRR_ , abs/1603.02754.
  * Dancey and Reidy (2007) Christine P Dancey and John Reidy. 2007.  _Statistics without maths for psychology_.  Pearson education.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: pre-training of deep bidirectional transformers for language understanding.  _CoRR_ , abs/1810.04805.
  * He et al. (2021) Ruidan He, Linlin Liu, Hai Ye, Qingyu Tan, Bosheng Ding, Liying Cheng, Jiawei Low, Lidong Bing, and Luo Si. 2021.  On the effectiveness of adapter-based tuning for pretrained language model adaptation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 2208–2222, Online. Association for Computational Linguistics.
  * Kantorovich (1960) Leonid V Kantorovich. 1960.  Mathematical methods of organizing and planning production.  _Management science_ , 6(4):366–422.
  * Larochelle et al. (2008) Hugo Larochelle, Dumitru Erhan, and Yoshua Bengio. 2008.  Zero-data learning of new tasks.  In _AAAI_ , 2, page 3.
  * Pfeiffer et al. (2021) Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021.  AdapterFusion: Non-destructive task composition for transfer learning.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 487–503, Online. Association for Computational Linguistics.
  * Pfeiffer et al. (2020) Jonas Pfeiffer, Andreas Rücklé, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020.  Adapterhub: A framework for adapting transformers.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020): Systems Demonstrations_ , pages 46–54, Online. Association for Computational Linguistics.
  * Plank and van Noord (2011) Barbara Plank and Gertjan van Noord. 2011.  Effective measures of domain similarity for parsing.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 1566–1576, Portland, Oregon, USA. Association for Computational Linguistics.
  * Ramesh Kashyap et al. (2021) Abhinav Ramesh Kashyap, Devamanyu Hazarika, Min-Yen Kan, and Roger Zimmermann. 2021\.  Domain divergences: A survey and empirical analysis.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1830–1849, Online. Association for Computational Linguistics.
  * Rényi et al. (1961) Alfréd Rényi et al. 1961.  On measures of entropy and information.  In _Proceedings of the fourth Berkeley symposium on mathematical statistics and probability_ , 547-561. Berkeley, California, USA.
  * Ruder and Plank (2017) Sebastian Ruder and Barbara Plank. 2017.  Learning to select data for transfer learning with Bayesian optimization.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 372–382, Copenhagen, Denmark. Association for Computational Linguistics.
  * Shannon (1948) Claude Elwood Shannon. 1948.  A mathematical theory of communication.  _The Bell system technical journal_ , 27(3):379–423.
  * Simpson (1949) Edward H Simpson. 1949.  Measurement of diversity.  _nature_ , 163(4148):688–688.
  * Tsvetkov et al. (2016) Yulia Tsvetkov, Manaal Faruqui, Wang Ling, Brian MacWhinney, and Chris Dyer. 2016\.  Learning the curriculum with Bayesian optimization for task-specific word representation learning.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 130–139, Berlin, Germany. Association for Computational Linguistics.
  * Van Asch and Daelemans (2010) Vincent Van Asch and Walter Daelemans. 2010.  Using domain similarity for performance estimation.  In _Proceedings of the 2010 Workshop on Domain Adaptation for Natural Language Processing_ , pages 31–36, Uppsala, Sweden. Association for Computational Linguistics.
  * Wong and You (1985) Andrew K. C. Wong and Manlai You. 1985.  Entropy and distance of random graphs with application to structural pattern recognition.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , PAMI-7(5):599–609.
  * Zhang et al. (2015) Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  _CoRR_ , abs/1509.01626.

##  Appendix A Implementation Details

Data Preparation. For each task, we sample a holdout validation set for early
stopping and a test set, both of size 2,500 (10% of the maximum sample size),
which remains fixed across both sets of experiments. After filtering by sample
size, i.e. dropping domains with less than 30,000 samples, we had a total of
58 domains for comparison. Prior to sampling, the total number of domains are
43 and 22 for Amazon and Yelp datasets, respectively. We use the use the same
five seeds for both data and model training.

Hyperparameters. We largely follow the recommended learning rate setting of
1e-4 (Pfeiffer et al., 2020; He et al., 2021) for adapter training. We set the
max number of epochs to 50 and an early stopping patience of 5 non-decreasing
epochs. We set the maximum input length to 256 and use a batch size of 32. For
the adapter configuration, we make use of the PfeifferConfig (Pfeiffer et al.,
2021) with default settings.

Computing Infrastructure. We run our experiments on 3$\times$ NVIDIA® TITAN™
RTX GPUs, with 130 Tensor TFLOPs of performance, 576 tensor cores, and 24 GB
of GDDR6 memory. For the CPU, We use 4.5 cores of Intel® Xeon® Gold 5222
Processor (16.5MB Cache, 3.80 GHz) and 96GB of RAM, split across three docker
containers.


* Ng et al. (2021) Davy Tsz Kit Ng, Jac Ka Lok Leung, Samuel Kai Wah Chu, and Maggie Shen Qiao. 2021.  Conceptualizing AI Literacy: An Exploratory Review.  _Computers and Education: Artificial Intelligence_ 2 (2021), 100041\.
  * OpenAI (2021) OpenAI. 2021.  _DALL·E: Creating Images from Text_.  https://openai.com/blog/dall-e/
  * OpenAI (2022a) OpenAI. 2022a.  _DALL·E 2_.  https://openai.com/dall-e-2/
  * OpenAI (2022b) OpenAI. 2022b.  Dall·e 2: Extending Creativity.  https://openai.com/blog/dall-e-2-extending-creativity/
  * OpenAI (2022c) OpenAI. 2022c.  _Dall·E Now Available Without Waitlist_.  https://openai.com/blog/dall-e-now-available-without-waitlist/
  * Oppenlaender (2022) Jonas Oppenlaender. 2022\.  The Creativity of Text-to-Image Generation. In _Proc. International Academic Mindtrek Conference_. 192–202.
  * Organization (2022) National Information Standards Organization. 2022.  _Contributor Roles Taxonomy (CRediT)_.  http://credit.niso.org/
  * Palinkas et al. (2015) Lawrence A Palinkas, Sarah M Horwitz, Carla A Green, Jennifer P Wisdom, Naihua Duan, and Kimberly Hoagwood. 2015.  Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research.  _Administration and Policy in Mental Health and Mental Health Services Research_ 42 (2015), 533–544.
  * Qiao et al. (2022) Han Qiao, Vivian Liu, and Lydia Chilton. 2022.  Initial Images: Using Image Prompts to Improve Subject Representation in Multimodal AI Generated Art. In _Creativity and Cognition_. 15–28.
  * Rajnerowicz (2022) Kazimierz Rajnerowicz. 2022\.  Will AI text-to-image generators turn us all into artists?  https://www.tidio.com/blog/generative-ai-dall-e-examples/
  * Ramesh et al. (2022) Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022.  Hierarchical Text-Conditional Image Generation With Clip Latents.  _arXiv preprint arXiv:2204.06125_ (2022).
  * Ramesh et al. (2021) Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021\.  Zero-Shot Text-To-Image Generation. In _International Conference on Machine Learning_. PMLR, 8821–8831.
  * Rombach et al. (2022) Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022\.  High-Resolution Image Synthesis With Latent Diffusion Models. In _Proc. Conference on Computer Vision and Pattern Recognition_. IEEE, 10684–10695.
  * Savov (2022) Vlad Savov. 2022\.  Dall-E and Midjourney and the future of AI Art Generation.  https://www.bloomberg.com/news/newsletters/2022-08-16/dall-e-and-midjourney-and-the-future-of-ai-art-generation
  * Tschang (2007) F Ted Tschang. 2007\.  Balancing the Tensions Between Rationalization and Creativity in the Video Games Industry.  _Organization science_ 18, 6 (2007), 989–1005.
  * Uusitalo et al. (2022) Severi Uusitalo, Anna Kantosalo, Antti Salovaara, Tapio Takala, and Christian Guckelsberger. 2022\.  Co-creative Product Design with Interactive Evolutionary Algorithms: A Practice-Based Reflection. In _Proc. International Conference on Artificial Intelligence in Music, Sound, Art and Design_. Springer, 292–307.
  * Van Nes et al. (2010) Fenna Van Nes, Tineke Abma, Hans Jonsson, and Dorly Deeg. 2010\.  Language Differences in Qualitative Research: Is Meaning Lost in Translation?  _European Journal of Ageing_ 7 (2010), 313–316.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention Is All You Need.  _Advances in Neural Information Processing Systems_ 30 (2017).
  * Venkatesh et al. (2003) Viswanath Venkatesh, Michael G Morris, Gordon B Davis, and Fred D Davis. 2003.  User Acceptance of Information Technology: Toward a Unified View.  _MIS quarterly_ (2003), 425–478.
  * vox.com (2022) vox.com. 2022.  Bonus video: What ai art means for human artists.  https://www.youtube.com/watch?v=sFBfrZ-N3G4

  *[AI]: Artificial Intelligence
  *[TTIG]: Text-to-Image Generation
  *[LLMs]: Large Language Model


# Coulomb impurities in graphene driven by ultrashort electromagnetic pulses:
Excitation, ionization, and pair creation

Saparboy Rakhmanov  Chirchik State Pedagogical University, 104 Amur Temur
Str., 111700 Chirchik, Uzbekistan  Reinhold Egger  Institut für Theoretische
Physik, Heinrich-Heine-Universität, 40225 Düsseldorf, Germany  Davron
Matrasulov  Turin Polytechnic University in Tashkent, 17 Niyazov Str., 100095
Tashkent, Uzbekistan

###### Abstract

We provide a theory for electronic transitions induced by ultrashort
electromagnetic pulses in two-dimensional artificial relativistic atoms which
are created by a charged impurity in a gapped graphene monolayer. Using a non-
perturbative sudden-perturbation approximation, we derive and discuss
analytical expressions for the probabilities for excitation, ionization and
electron-hole pair creation in this system.

##  I Introduction

Graphene monolayers represent a unique two-dimensional (2D) material with a
broad spectrum of unusual and practically important properties [1]. Apart from
its application potential, a remarkable feature of graphene is that it
provides a table-top laboratory for testing relativistic quantum physics in
two spatial dimensions under strong-coupling conditions. This feature is
related to the fact that the planar quantum electrodynamics (QED) in graphene
is characterized by a coupling constant (the effective fine structure
constant) that can be much larger than in conventional three-dimensional QED
[2, 3, 4]. To elucidate the resulting strongly coupled 2D relativistic quantum
physics, one may create and study artificial relativistic atoms by doping
charged impurities into monolayer graphene [5, 6, 7, 3, 8, 4, 9].

The electron dynamics in 2D graphene monolayers effectively operates under
relativistic conditions similar to the regime realized by high-$Z$ atoms in
ordinary QED. This correspondence allows one to observe different
supercritical effects [2], such as vacuum polarization, pair creation, or
Klein tunneling, which occur when the atomic electron’s energy level reaches
the boundary of the Dirac sea. Earlier, such effects were extensively studied
in the context of conventional 3D QED [10, 11]. However, their experimental
realization is still considered a complicated and expensive task. On the other
hand, recent experiments on Coulomb impurities in graphene have already shown
manifestations of related phenomena [5, 6, 7].

In this paper, we study electronic transitions for relativistic 2D atoms in
gapped graphene monolayers induced by the interaction of the atom with an
ultrashort electromagnetic pulse. The external pulse can be created either by
picosecond (or shorter) lasers, or by a fast and highly charged ion passing
near the surface of the graphene sheet. Our motivation for this study comes
from the fact that the experimental realization of high-$Z$ atoms or ions
interacting with strong electromagnetic fields represents a highly challenging
task which requires facilities with high-power lasers or accelerators for
highly charged ions [10, 11, 12, 13, 14, 15, 16, 17, 18, 19]. Using graphene
for such studies may provide an alternative low-cost table-top realization.
This concerns especially supercritical phenomena such as particle-antiparticle
pair creation and vacuum polarization [2, 20, 21, 22, 23, 24, 25, 26, 27, 28,
29], whose experimental study otherwise requires large-scale facilities for
stripping and accelerating high-$Z$ atoms and highly charged ions, e.g., LHC
(CERN), FAIR (GSI), or RHIC (BNL). Since electronic transitions of the atom
due to its interaction with an ultrashort electromagnetic pulses cannot be
calculated within perturbation theory (e.g., using the Born approximation),
one needs to apply non-perturbative methods. One such approach has been
developed by Matveev in Ref. [30], see also Refs. [31, 32]. Here we use
Matveev’s approach to calculate probabilities and differential cross sections
for the excitation, ionization and particle-antiparticle pair creation in a
planar artificial atom graphene. We note that particle-antiparticle pair
creation may be viewed as “ionization” of the Dirac sea, where an electron
from the filled Dirac sea is excited to the upper continuum. Unlike in 3D QED
with high-$Z$ atoms, such transitions are easier to realize in graphene with a
Coulomb impurity and one does not need to employ high-intensity laser fields.
Electron-hole pair creation via the Schwinger mechanism in graphene with
external time-dependent electromagnetic fields has previously been studied in
Refs. [22, 26, 27, 29].

The remainder of this paper is organized as follows. In Sec. II, mainly
following Ref. [20], we briefly summarize the electronic states of planar
relativistic atoms in gapped graphene monolayers, which represents a
relativistic 2D Coulomb problem. Section III presents a detailed description
of the sudden-perturbation approximation for this artificial relativistic atom
interacting with an ultrashort electromagnetic pulse. In Sec. IV, we present
results for the probabilities and cross sections of different electronic
transitions, including the case of particle-antiparticle pair creation.
Finally, Sec. V provides concluding remarks. Throughout, we use units with
$\hbar=e=1$.

##  II Relativistic atoms in graphene

We here briefly summarize the electronic structure of 2D artificial
relativistic atoms in graphene, mainly following Ref. [20]. Doping a charged
Coulomb impurity into a gapped graphene monolayer causes a capture of free
electrons which form a 2D artificial atom. The electron dynamics of such an
atom is described in terms of the 2D relativistic Dirac equation [20, 33].
Basic mechanisms for vacuum polarization effects, including quasiparticle pair
creation in graphene, induced by time-dependent electromagnetic fields have
been studied in Refs. [2, 20, 33, 21, 22, 23, 24, 25, 26, 27, 28, 29]. In
particular, the critical charge for supercriticality, at which the lowest
bound-state energy level dives into the filled Dirac sea, turns out to be much
lower than for 3D atoms [20, 33].

In the presence of a homogeneous quasiparticle energy gap $M$, which may
arise, e.g., due to strain effects or due to a substrate superlattice
potential [1], the single-electron dynamics in a graphene monolayer with a
Coulomb impurity at the origin is described by the 2D Dirac Hamiltonian [1,
20, 2],

$H_{0}=v_{F}(\sigma_{x}p_{x}+\sigma_{y}p_{y})+M\sigma_{z}-\frac{v_{F}\alpha}{r}\sigma_{0}.$
(1)

Here $v_{F}$ is the Fermi velocity, and the dimensionless number $\alpha$
parametrizes the charge of the Coulomb impurity (multiplied by the fine
structure constant and the inverse dielectric constant). We use ${\bf
r}=(x,y)$, $r=\sqrt{x^{2}+y^{2}}$, $p_{x,y}=-i\partial_{x,y}$, and the
standard Pauli matrices $\sigma_{x,y,z}$ with the identity $\sigma_{0}$. These
matrices act in the sublattice space of graphene’s honeycomb lattice [1],
where we focus on a single K valley in the band structure. We thus assume that
the electromagnetic field of the pulse varies smoothly in space on the scale
of the lattice constant ($a_{0}\sim 2.46$ Å) [1]. In addition, we assume that
no spin-dependent physics is of importance such that the electron spin can be
kept implicit.

In the absence of external fields, the quasiparticle spinor wave function
obeys the stationary Dirac equation, $H_{0}|\psi\rangle=E|\psi\rangle$.
Following Refs. [20, 33], we separate the angular ($\phi$) and radial ($r$)
variables as

$\psi_{n,j}({\bf r})=\left(\begin{array}[]{c}F_{n,j}(r)\,\Phi_{j-1/2}(\phi)\\\
iG_{n,j}(r)\,\Phi_{j+1/2}(\phi)\end{array}\right),$ (2)

where the integers $n$ and half-integers $j$ are principal and angular quantum
numbers, respectively. With integer $m$, the angular eigenfunctions are given
by $\Phi_{m}(\phi)=\frac{1}{\sqrt{2\pi}}e^{im\phi}$. The radial eigenfunctions
$F_{n,j}(r)$ and $G_{n,j}(r)$ are specified in closed form in Ref. [20]. For
the discrete spectrum with energies $|E|<M$, and assuming $0\leq\alpha<1/2$
throughout, the eigenenergies are given by

$E_{n,j}=\frac{M}{\sqrt{1+\frac{\alpha^{2}}{(n+\gamma_{j})^{2}}}},\quad\gamma_{j}=\sqrt{j^{2}-\alpha^{2}},$
(3)

where $n\geq 0$ for $j>0$, and $n>0$ for $j<0$.

Let us next provide explicit expressions for planar spinor waves of a free
Dirac particle and for scattering states in the Coulomb field [20]. These wave
functions are employed below for a calculation of the pair creation and
ionization probabilities caused by ultrashort electromagnetic pulses. Planar
wave states with momentum ${\bf p}$ and energy dispersion
$E=\pm\sqrt{v_{F}^{2}p^{2}+M^{2}}$ (with $p=|{\bf p}|$) can be written as

$\psi_{\pm|E|;\textbf{p}}=\frac{e^{i\textbf{p}\cdot{\bf
r}}}{\sqrt{2|E|}}\left(\begin{matrix}\sqrt{|E+M|}\\\
\pm\sqrt{|E-M|}e^{i\phi_{\textbf{p}}}\\\ \end{matrix}\right),$ (4)

where $\phi_{\bf p}=\arg{(p_{x}+ip_{y})}$. Similarly, spinor spherical waves
are given by

$\psi_{p,j}(\textbf{r})=\frac{1}{\sqrt{2|E|}}\left(\begin{matrix}\sqrt{|E+M|}\,R_{p,j-1/2}(r)\,\Phi_{j-1/2}(\phi)\\\
\pm i\sqrt{|E-M|}\,R_{p,j+1/2}(r)\,\Phi_{j+1/2}(\phi)\\\ \end{matrix}\right),$
(5)

where $R_{p,m}(r)=\sqrt{2\pi p}\,J_{m}(pr)$ with Bessel functions $J_{m}$.

Finally, we turn to scattering states in the Coulomb field, with energies in
the continuous part of the spectrum $(|E|>M$). In the asymptotic regime
$pr\to\infty$, they are given by Eq. (2) with

$\displaystyle F_{p,j}$ $\displaystyle\simeq$
$\displaystyle\sqrt{\frac{2|E+M|}{|E|r}}\cos(pr-j\pi/2+\alpha_{E}\ln
2pr+\delta_{j}),$ (6) $\displaystyle G_{p,j}$ $\displaystyle\simeq$
$\displaystyle\pm\sqrt{\frac{2|E-M|}{|E|r}}\sin(pr-j\pi/2+\alpha_{E}\ln
2pr+\delta_{j}),$

where we use

$\displaystyle e^{-2i\xi_{j}}$ $\displaystyle=$
$\displaystyle\frac{\gamma_{j}-i\alpha_{E}}{j+iM\alpha/v_{F}p},\quad\alpha_{E}=\frac{\alpha
E}{v_{F}p},$ (7) $\displaystyle\delta_{j}$ $\displaystyle=$
$\displaystyle\xi_{j}+\frac{\pi}{2}(j-\gamma_{j})-\arg\Gamma\bigg{(}1+\gamma_{j}+\frac{i\alpha
E}{v_{F}p}\bigg{)},$

with the Gamma function $\Gamma$. Below, the spinor wave function in Eqs. (2)
and (6) is used as final state $|\psi_{f}\rangle$ in our calculations of the
ionization probabilities and cross sections.

##  III Sudden-perturbation approximation

The sudden-perturbation approximation is a powerful tool for describing atoms
in external time-dependent fields if the duration of the external pulse is
much shorter than the characteristic period of the electron bound to the first
Bohr orbit of the atom. Here we will adopt the sudden-perturbation approach of
Ref. [30] to an artificial atom in a 2D graphene layer, see Sec. II, in the
presence of an ultrashort electromagnetic pulse.

In the presence of a spatially inhomogeneous ultrashort pulse $U({\bf r},t)$,
the spinor wave function dynamics is governed by the 2D time-dependent Dirac
equation,

$i\frac{\partial\Psi({\bf r},t)}{\partial t}=[H_{0}+U({\bf r},t)]\Psi({\bf
r},t),$ (8)

with $H_{0}$ in Eq. (1). Describing the electromagnetic pulse in terms of a
vector potential ${\bf A}({\bf r},t)$ and a scalar potential $\varphi({\bf
r},t)$, we have

$U({\bf r},t)=\frac{v_{F}}{c}{\bm{\sigma}}\cdot{\bf A}-\varphi\sigma_{0},$ (9)

where ${\bm{\sigma}}=(\sigma_{x},\sigma_{y})$. We thus obtain

$i\dot{\Psi}=\left[v_{F}{\bm{\sigma}}\cdot\left({\bf p}+\frac{1}{c}{\bf
A}\right)+M\sigma_{z}-\left(\varphi+\frac{v_{F}\alpha}{r}\right)\sigma_{0}\right]\Psi.$
(10)

We now assume a gauge where the scalar potential vanishes, $\varphi=0$, and
the vector potential ${\bf A}({\bf r},t)={\bf A}(\eta({\bf r},t))$ only
depends on $({\bf r},t)$ through the variable

$\eta({\bf r},t)=\omega_{0}t-{\bf k}_{0}\cdot{\bf r},\quad\omega_{0}=c|{\bf
k}_{0}|.$ (11)

The pulse maximum is then described by the wave vector ${\bf k}_{0}$ with
frequency $\omega_{0}$. Next we use the gauge transformation [30]

${\bf A}^{\prime}={\bf A}+\nabla
f,\quad\varphi^{\prime}=\varphi-\frac{1}{c}\frac{\partial f}{\partial t},\quad
f=-{\bf A}\cdot{\bf r}.$ (12)

We thereby obtain

$\displaystyle{\bf A}^{\prime}$ $\displaystyle=$ $\displaystyle\left({\bf
r}\cdot\frac{d{\bf A}}{d\eta}\right){\bf k}_{0},$ (13)
$\displaystyle\varphi^{\prime}$ $\displaystyle=$ $\displaystyle\left({\bf
r}\cdot\frac{d{\bf A}}{d\eta}\right)|{\bf k}_{0}|=-{\bf E}\cdot{\bf r},$

with the electric field ${\bf E}=-|{\bf k}_{0}|\frac{d{\bf A}}{d\eta}.$

To proceed, let us assume that the wave vector ${\bf k}_{0}$ is directed along
the $x$-axis, see Fig. 1. In the new gauge, the external electromagnetic field
can then be written as

$\displaystyle U({\bf r},t)$ $\displaystyle=$
$\displaystyle\frac{v_{F}}{c}{\bm{\sigma}}\cdot{\bf
A}^{\prime}-\varphi^{\prime}\sigma_{0}=-\left(\sigma_{0}-\frac{v_{F}}{c}\frac{{\bm{\sigma}}\cdot{\bf
k}_{0}}{|{\bf k}_{0}|}\right)\varphi^{\prime}=$ (14) $\displaystyle=$
$\displaystyle-\left(\sigma_{0}-\frac{v_{F}}{c}\sigma_{x}\right)\varphi^{\prime}.$

The evolution of the spinor wave function from time $t_{0}$ to time $t$ is
then governed by

$|\Psi(t)\rangle={\cal
T}\exp\left(-i\int_{t_{0}}^{t}dt^{\prime}U(t^{\prime})\right)|\Psi(t_{0})\rangle,$
(15)

where ${\cal T}$ is the time ordering operator. For ultrashort pulses, one
finds $[U(t),U(t^{\prime})]=0$ for all time pairs, see also Ref. [30], and
hence the time ordering operator can effectively be omitted in Eq. (15). We
may thus calculate the transition amplitude $a_{f,i}$ from an initial state
$|\psi_{i}\rangle$ to a final state $|\psi_{f}\rangle$ by using the relation

$a_{f,i}=\langle\psi_{f}|\exp\left(-i\int_{-\infty}^{+\infty}dtU(t)\right)|\psi_{i}\rangle.$
(16)

According to Ref. [30], the transition amplitude in a driven system described
by Eq. (8) is given by Eq. (16) if the perturbation has a $\delta$-function
like time dependence,

$\tilde{U}(t)=U_{0}\delta(ct-x),\quad
U_{0}=c\int_{-\infty}^{+\infty}dt\,U(ct-x).$ (17)

In this limit, the sudden-perturbation approach becomes exact. However, even
if the $\delta$-function limit is only realized approximately, the obtained
relations can be used for accurately solving the time-dependent Dirac equation
(8).

In what follows, we work in the new gauge given by Eq. (12), where we omit the
primes henceforth. Substituting the potential (17) into Eq. (8), we find

$i\dot{\Psi}=\left[v_{F}{\bm{\sigma}}\cdot{\bf
p}+M\sigma_{z}-\frac{v_{F}\alpha}{r}\sigma_{0}-\left(\sigma_{0}-\frac{v_{F}}{c}\sigma_{x}\right)\tilde{\varphi}\right]\Psi,$
(18)

where the ultrashort pulse is described by

$\tilde{\varphi}=\varphi_{0}\delta(ct-x),\quad\varphi_{0}=c\int_{-\infty}^{+\infty}dt\,\varphi(ct-x).$
(19)

The solution of Eq. (18) can be expanded in the complete set of eigenfunctions
of the unperturbed system, see Sec. II. With the 2D spinor wave functions
$\psi_{l}({\bf r})$ in Eq. (2) and the corresponding eigenenergies $E_{l}$,
where $l=(n,j)$ combines principal ($n$) and angular momentum ($j$) quantum
numbers, we write

$\Psi({\bf r},t)=\sum_{l}a_{l}(t)e^{-iE_{l}t}\psi_{l}({\bf r}),$ (20)

with complex-valued and time-dependent expansion coefficients $a_{l}(t)$.
Substituting Eq. (20) into Eq. (18), the time dependence of the coefficient
$a_{f}(t)$ for the final state $|\psi_{f}\rangle$ is governed by

$\frac{da_{f}(t)}{dt}=-ie^{iE_{f}t}\langle\psi_{f}|\tilde{U}(t)|\Psi_{i}\rangle,$
(21)

where we assume that prior to the interaction, the atomic electron was in the
state $|\psi_{i}\rangle$:

$\Psi_{i}({\bf r},t)|_{t\to-\infty}=e^{-iE_{i}t}\psi_{i}({\bf r}),$ (22)

with the initial condition

$a_{f}(t)|_{t\to-\infty}=\delta_{f,i}.$ (23)

For $\tilde{U}(t)=U_{0}\delta(ct-x)$, in order to solve Eq. (21), it is
sufficient to determine $\Psi({\bf r},t)$ only near $x=ct$ [30]. This can be
implemented by transforming to light-cone coordinates,

$x^{-}=ct-x,\quad x^{+}=ct+x,$ (24)

where time and space derivatives of the wave function can be written as

$\displaystyle\dot{\Psi}$ $\displaystyle=$
$\displaystyle\frac{\partial\Psi}{\partial x^{-}}\frac{\partial
x^{-}}{\partial t}=c\frac{\partial\Psi}{\partial x^{-}},$ (25)
$\displaystyle\frac{\partial\Psi}{\partial x}$ $\displaystyle=$
$\displaystyle\frac{\partial\Psi}{\partial x^{-}}\frac{\partial
x^{-}}{\partial x}=-\frac{\partial\Psi}{\partial x^{-}}.$

In the immediate vicinity of $x^{-}=0$, Eq. (18) can be written as [30]

$ic\left(\sigma_{0}-\frac{v_{F}}{c}\sigma_{x}\right)\frac{\partial\Psi}{\partial
x^{-}}=-\left(\sigma_{0}-\frac{v_{F}}{c}\sigma_{x}\right)\tilde{\varphi}\,\Psi,$
(26)

with $\tilde{\varphi}=\varphi_{0}\delta(x^{-})$. With the Heaviside step
function $\theta(x)$, where we use
$\frac{d}{dx}\exp\theta(x)=\delta(x)\exp\theta(x)$, the solution of Eq. (26)
can be written as

$\Psi(x^{-}+\varepsilon)=\exp\left[i\frac{\varphi_{0}}{c}\theta(x^{-})\right]\Psi(x^{-}-\varepsilon),$
(27)

where $\varepsilon$ is an infinitesimal positive quantity. In terms of normal
time and space coordinates with $t=x/c+\varepsilon$, Eq. (27) takes the form

$\Psi({\bf
r},t)=\exp\left[i\frac{\varphi_{0}}{c}\theta(ct-x)\right]\,e^{-iE_{i}t}\,\psi_{i}({\bf
r}).$ (28)

Next we substitute Eq. (28) into Eq. (21) and integrate over time. Taking into
account the initial condition (23), we obtain the transition amplitude
$a_{f,i}$ in Eq. (16) as

$\displaystyle
a_{f,i}=a_{f}(t\to\infty)=\delta_{f,i}+i\int_{-\infty}^{+\infty}dt\,e^{i(E_{f}-E_{i})t}\times$
(29)
$\displaystyle\times\langle\psi_{f}|\left(\sigma_{0}-\frac{v_{F}}{c}\sigma_{x}\right)\varphi_{0}\exp\left[i\theta(ct-x)\frac{\varphi_{0}}{c}\right]\delta(ct-x)|\psi_{i}\rangle.$

Performing the integral, we obtain

$\displaystyle a_{f,i}$ $\displaystyle=$
$\displaystyle\delta_{fi}+\langle\psi_{f}|\left(\sigma_{0}-\frac{v_{F}}{c}\sigma_{x}\right)\times$
(30) $\displaystyle\times$ $\displaystyle
e^{i(E_{f}-E_{i})x/c}\left(e^{i\varphi_{0}/c}-1\right)|\psi_{i}\rangle.$

Using the relation

$\langle\psi_{f}|\left(\sigma_{0}-\frac{v_{F}}{c}\sigma_{x}\right)|\psi_{i}\rangle=\delta_{f,i},$
(31)

we finally arrive at the transition amplitude

$a_{f,i}=\langle\psi_{f}|\left(\sigma_{0}-\frac{v_{F}}{c}\sigma_{x}\right)\,e^{i(E_{f}-E_{i})x/c}\,e^{i\varphi_{0}/c}|\psi_{i}\rangle.$
(32)

Using Eq. (32), we can now calculate transition amplitudes between electronic
states of the artifical atom corresponding to excitation, ionization, or
electron-hole pair creation processes.

We choose the external electromagnetic field as a Gaussian pulse of short time
duration $\lambda$,

${\bf E}({\bf r},t)={\bf
E}_{0}\exp\left[-\frac{1}{\lambda^{2}}\left(t-\frac{{\bf k}_{0}\cdot{\bf
r}}{\omega_{0}}\right)^{2}\right]\cos(\omega_{0}t-{\bf k}_{0}\cdot{\bf r}).$
(33)

Approximating this pulse shape by a $\delta$-potential as in Eq. (19), we find
$\tilde{\varphi}=-{\bf r}\cdot{\bf E}({\bf r},t)$ and therefore

$\varphi_{0}=c\int_{-\infty}^{+\infty}dt\,\tilde{\varphi}(ct-x)=c{\bf
q}\cdot{\bf r},$ (34)

where we define

${\bf q}=-\int_{-\infty}^{+\infty}dt\,{\bf E}({\bf
r},t)=-\frac{\sqrt{\pi}}{\lambda}\,e^{-\omega_{0}^{2}/(2\lambda)^{2}}\,{\bf
E}_{0}.$ (35)

The quantity ${\bf q}$ describes the momentum transfer due to the interaction
of external electromagnetic pulse with the atom. The dependence of transition
probabilities and cross sections on ${\bf q}$ can be probed experimentally.

Figure 1: Rotation of the coordinate system.

In order to simplify the following calculations, we perform a rotation of the
coordinate system as illustrated in Fig. 1. Taking into account that
$\sigma_{x}={\bm{\sigma}}\cdot{\bf k}_{0}/k_{0}$, and introducing the vector
${\bf Q}=(Q_{x},Q_{y})=(\Omega_{f,i}/c,q)$, with the transition frequency
$\Omega_{f,i}=E_{f}-E_{i}$ and $q=|{\bf q}|$, the transition amplitude in Eq.
(32) can be rewritten as

$a_{f,i}=\langle\psi_{f}|\left(\sigma_{0}-\frac{v_{F}}{c}\frac{{\bm{\sigma}}\cdot{\bf
k}_{0}}{k_{0}}\right)e^{i{\bf Q}\cdot{\bf r}}|\psi_{i}\rangle.$ (36)

In the rotated system, the $x^{\prime}$-axis is aligned with ${\bf Q}$, see
Fig. 1. In what follows, we omit the primes again and define the angle
$\theta$ through

$\frac{{\bm{\sigma}}\cdot{\bf
k}_{0}}{k_{0}}=\sigma_{x}\cos\theta-\sigma_{y}\sin\theta.$ (37)

We note that in the rotated system, we have the relation

${\bf Q}\cdot{\bf r}=Qx.$ (38)

Using Eqs. (37) and (38), the transition amplitude (36) follows in the form

$\displaystyle a_{f,i}$ $\displaystyle=$
$\displaystyle\langle\psi_{f}|\left(\sigma_{0}-\frac{v_{F}}{c}\sigma_{x}\cos\theta\right)e^{iQx}|\psi_{i}\rangle+$
(39) $\displaystyle+$
$\displaystyle\frac{v_{F}}{c}\langle\psi_{f}|\sigma_{y}\sin\theta\,e^{iQx}|\psi_{i}\rangle,$

where we can read off the relations

$Q=\sqrt{q^{2}+\Omega_{f,i}^{2}/c^{2}},\quad\cos\theta=\frac{\Omega_{f,i}}{cQ},\quad\sin\theta=\frac{q}{Q}.$
(40)

For the transition from the initial state $|\psi_{i}\rangle$ with energy
$E_{i}$ to the final state $|\psi_{f}\rangle$ with energy $E_{f}$, the
transition probability can thus be written as

$|a_{f,i}|^{2}=\frac{q^{2}}{Q^{2}}\left(\frac{q^{2}}{Q^{2}}|F^{f,i}|^{2}+\frac{v_{F}^{2}}{c^{2}}|G_{y}^{f,i}|^{2}\right),$
(41)

with the matrix elements

$\displaystyle F^{f,i}$ $\displaystyle=$
$\displaystyle\langle\psi_{f}|e^{iQx}|\psi_{i}\rangle-\frac{Qv_{F}}{\Omega_{f,i}}\langle\psi_{f}|\sigma_{x}\,e^{iQx}|\psi_{i}\rangle,$
$\displaystyle G_{y}^{f,i}$ $\displaystyle=$
$\displaystyle\langle\psi_{f}|\sigma_{y}e^{iQx}|\psi_{i}\rangle.$ (42)

Within the sudden-perturbation approximation, which approximates the narrow
Gaussian pulse in Eq. (33) by a $\delta$-function shape, Eqs. (39) and (41)
represent exact expressions for transition amplitudes and probabilities,
respectively. Given the initial and final state wave functions specified in
Sec. II and in Ref. [20], one can thus compute probabilities and cross
sections for different transitions, such as excitation, ionization, or pair
creation. In addition, one may study the angular distribution of the electron
emission cross section,

$\sigma(\vartheta)=\int_{0}^{\infty}dp\,|a_{f,i}|^{2},$ (43)

where $\vartheta$ is the emission angle of the electron (determined by the
final state) and $p$ its momentum.

Finally, let us note that our assumption of a single K valley in graphene’s
band structure, see Eq. (1), implies that the above approach can only be
applied if the momentum transfer $q$ due to the interaction with the incident
photons is small compared to the momentum distance $k$ between K and K’
valleys, given by $k=\frac{4\pi}{\sqrt{3}a_{0}}$ with graphene’s lattice
constant $a_{0}$ [1]. For $q\gtrsim k$, one has to explicitly take into
account both K and K’ valleys, since scattering processes connecting them
become important. For the results shown in Sec. IV, we made sure that $q$ is
always small against $k$. In that case, K valley and spin degeneracies cause
only trivial overall degeneracy factors in our results.

Figure 2: Excitation probability vs momentum transfer $q$ from the ground
state of the artificial atom to the first excited state (a), second excited
state (b, solid lines) and third excited state (b, dashed lines) due to an
ultrashort pulse, for several values of the Coulomb impurity charge parameter
$\alpha$ in Eq. (1).

##  IV Excitation, ionization, and pair creation processes

We next present results of the formalism in Sec. III to study excitation,
ionization, and particle-antiparticle pair creation processes induced by an
external ultrashort electromagnetic pulse in an artificial atom in graphene.
Here, “excitation” is defined as the transition of an electron from a given
initial bound state $|\psi_{i}\rangle$ to another bound state
$|\psi_{f}\rangle$ in the discrete spectrum (see Sec. II and Ref. [20]), while
“ionization” refers to transitions from bound to continuum states described by
the scattering state in Eq. (6). In the following, we use Eq. (41) to
calculate the corresponding transition probabilities. A third process,
possible only with relativistic atoms, is the particle-antiparticle pair
creation, where a state $|\psi_{i}\rangle$ from the filled Dirac sea is
excited to a state $|\psi_{f}\rangle$ in the upper continuum. The final state
is then characterized by the creation of an electron-hole pair. The
corresponding electron and hole wave functions have also been specified in
Sec. II. A remarkable feature of the transition amplitude in Eq. (39) is the
fact that it does not explicitly contain the parameters determining the
electromagnetic pulse, such as the field amplitude $E_{0}$, the frequency
$\omega_{0}$, or the pulse duration $\lambda$. All these parameters appear
only via the momentum transfer parameter $q$ in Eq. (35). In what follows,
transition probabilities and cross sections are therefore shown as a function
of $q$.

###  IV.1 Excitation probabilities

Let us start with the excitation probabilities involving transitions
$|\psi_{i}\rangle\to|\psi_{f}\rangle$ between bound states. We assume that
$|\psi_{i}\rangle$ is given by the lowest-energy state (i.e., the ground
state) of the artificial atom. In Fig. 2(a), excitation probabilities to the
first excited state are shown for several Coulomb charge parameters $\alpha$
as a function of the momentum transfer parameter $q$. With increasing $q$, the
excitation probability first increases, reaches a maximum, and then decreases
again. The position $q=q_{\rm max}$ of the maximum depends on $\alpha$, with
larger values for $q_{\rm max}$ for larger $\alpha$. However, qualitatively,
the curves are similar for different $\alpha$. Next we turn to transitions to
higher excited states of $s$-wave orbital type. Figure 2(b) shows the
corresponding excitation probabilities from the ground state to the second and
third excited state, respectively. In comparison to the first excited state,
the transition probabilities are smaller by more than one order of magnitude,
and require larger momentum transfer $q$ for sizeable values. In particular,
the probabilities for the third excited state become very small.

Figure 3: Ionization probability from the ground state vs momentum transfer
$q$, for different Coulomb charge parameters $\alpha$ with fixed momentum
$p=1$ of the emitted electron. The inset shows results $\alpha=0.3$ and
different $p$.

###  IV.2 Ionization probabilities

We now turn to a discussion of the ionization probability. Figure 3 shows the
corresponding probability for ionization out of the ground state of the
artificial atom as a function of the momentum transfer parameter $q$, again
for different values of Coulomb charge parameter $\alpha$ and for a fixed
value of the final (scattering) state momentum $p$. Interestingly, in Fig. 3,
we observe a finite ionization probability already at small momentum transfer
$q$, in contrast to the excitation probabilities in Fig. 2. Assuming a small
and fixed value of $q$, the ionization probability becomes smaller with
increasing nuclear charge parameter $\alpha$. This observation can be
rationalized by noting that the ground state will have a larger binding energy
for larger $\alpha$. In the inset of Fig. 3, for a fixed value of $\alpha$, we
illustrate the dependence of the ionization probability on the scattering
momentum $p$ of the final state. We observe that for small $q$, the ionization
probability becomes smaller with increasing final-state momentum $p$.

Figure 4: Probability for electron-hole pair production from the ground state
due to an ultrashort pulse vs momentum transfer $q$. Results are shown for
different values of $\alpha$, choosing $p=1$ for the momentum of the emitted
electron. The inset shows results for $\alpha=0.3$ and different $p$.

###  IV.3 Pair creation probabilities

Next, in Fig. 4, we present results for the pair production probability as a
function of the momentum transfer $q$, again for several values of $\alpha$.
For pair production induced by photon-atom (or atom-photon) collisions, one
may consider “free-free” and “bound-free” processes [15]. In bound-free pair
production, an electron is transferred from the filled Dirac sea to an atomic
bound state. We here consider the case of free-free pair production, where an
electron is transferred from the Dirac sea to a continuum state in the upper
continuum. Unlike what we found for the excitation and ionization probability,
for small $q$, the pair production probability now increases with increasing
nuclear charge parameter $\alpha$. Furthermore, the curves are qualitatively
different and less sharp. In Fig. 4, we also illustrate the corresponding
dependence on the final momentum $p$ of the hole state. The above results
illustrate the dependence of transition probabilities and cross sections
mainly as function of the momentum transfer parameter $q$. It is
straightforward to explore the dependence of these quantities on detailed
pulse parameters, such as the strength, frequency and duration of the pulse,
by using Eq. (35).

##  V Conclusions

In this work, we have studied electronic transitions in 2D artificial atoms in
gapped graphene monolayers. Here an atom is created by the presence of a
Coulomb impurity, and the transitions are induced by an ultrashort
electromagnetic pulse. For this scenario, we have developed and applied a
nonperturbative theoretical approach, based on a modified version of Matveev’s
sudden-perturbation approach [30], for the calculation of electronic
excitation, ionization, and electron-hole pair creation probabilities and
cross sections. This approach becomes exact in the limit of $\delta$-type
pulses. The essential assumption behind our theory is therefore that the pulse
is very short.

Analytical expressions for the electronic transition amplitude have been
obtained from the time-dependent Dirac equation, where we find that the
dependence on external field parameters enters solely via the effective
momentum transfer parameter $q$ in Eq. (35). Using this theoretical approach,
the probabilities for excitation, ionization, and pair creation can then be
computed as a function of the momentum transfer $q$.

The proposed model for electronic transitions in artificial relativistic atoms
in graphene could be directly realized using available table-top experimental
setups [5, 6, 7]. In that sense, we are confident that our results can pave
the way for an experimental realization of the corresponding ultrarelativistic
atomic quantum processes in a 2D version, while their 3D counterpart still
seems out of experimental reach. Future work may also consider setups with
several Coulomb impurities, corresponding to artificial molecules or dipoles
[34], in order to study the respective electronic transition probabilities.

###### Acknowledgements.

We acknowledge funding by the Grant REP-05032022/235 (“Ultrafast phenomena and
vacuum effects in relativistic artificial atoms created in graphene”), funded
under the MUNIS Project, supported by the World Bank and the Government of the
Republic of Uzbekistan.

## References

  * Castro Neto  _et al._ [2009] A. H. Castro Neto, F. Guinea, N. M. R. Peres, K. S. Novoselov, and A. K. Geim, The electronic properties of graphene, Rev. Mod. Phys. 81, 109 (2009).
  * Kotov  _et al._ [2012] V. N. Kotov, B. Uchoa, V. M. Pereira, F. Guinea, and A. H. Castro Neto, Electron-Electron Interactions in Graphene: Current Status and Perspectives, Rev. Mod. Phys. 84, 1067 (2012).
  * Pereira  _et al._ [2007] V. M. Pereira, J. Nilsson, and A. H. Castro Neto, Coulomb Impurity Problem in Graphene, Phys. Rev. Lett. 99, 166802 (2007).
  * De Martino  _et al._ [2014a] A. De Martino, D. Klöpfer, D. Matrasulov, and R. Egger, Electric-Dipole-Induced Universality for Dirac Fermions in Graphene, Phys. Rev. Lett. 112, 186603 (2014a).
  * Luican-Mayer  _et al._ [2014] A. Luican-Mayer, M. Kharitonov, G. Li, C.-P. Lu, I. Skachko, A.-M. B. Gonçalves, K. Watanabe, T. Taniguchi, and E. Y. Andrei, Screening Charged Impurities and Lifting the Orbital Degeneracy in Graphene by Populating Landau Levels, Phys. Rev. Lett. 112, 036804 (2014).
  * Wang  _et al._ [2013] Y. Wang, D. Wong, A. V. Shytov, V. W. Brar, S. Choi, Q. Wu, H.-Z. Tsai, W. Regan, A. Zettl, R. K. Kawakami, S. G. Louie, L. S. Levitov, and M. F. Crommie, Observing Atomic Collapse Resonances in Artificial Nuclei on Graphene, Science 340, 734 (2013).
  * Wang  _et al._ [2012] Y. Wang, V. W. Brar, A. V. Shytov, Q. Wu, W. Regan, H.-Z. Tsai, A. Zettl, L. S. Levitov, and M. F. Crommie, Mapping Dirac quasiparticles near a single Coulomb impurity on graphene , Nature Physics 8, 653 (2012).
  * Gamayun  _et al._ [2009] O. V. Gamayun, E. V. Gorbar, and V. P. Gusynin, Supercritical Coulomb center and excitonic instability in graphene, Phys. Rev. B 80, 165429 (2009).
  * Klöpfer  _et al._ [2013] D. Klöpfer, A. De Martino, and R. Egger, Bound States and Supercriticality in Graphene-Based Topological Insulators, Crystals 3, 14 (2013).
  * Greiner  _et al._ [1985] W. Greiner, B. Müller, and J. Rafelski,  _Quantum Electrodynamics of Strong Fields_ (Springer Berlin, Heidelberg, 1985).
  * Popov [2001] V. S. Popov, Critical charge in quantum electrodynamics, Phys. Atom. Nuclei 64, 367 (2001).
  * Baltz [2003] A. J. Baltz, Coulomb corrections in the calculation of ultrarelativistic heavy ion production of continuum ${e}^{+}{e}^{-}$ pairs, Phys. Rev. C 68, 034906 (2003).
  * Baltz and McLerran [1998] A. J. Baltz and L. McLerran, Two center light cone calculation of pair production induced by ultrarelativistic heavy ions, Phys. Rev. C 58, 1679 (1998).
  * Baltz  _et al._ [1993] A. J. Baltz, M. J. Rhoades-Brown, and J. Weneser, Bound-electron–positron pair production in relativistic heavy-ion collisions, Phys. Rev. A 47, 3444 (1993).
  * Eichler [1990] J. Eichler, Theory of relativistic ion-atom collisions, Physics Reports 193, 165 (1990).
  * Stöhlker  _et al._ [2015] T. Stöhlker, Y. A. Litvinov, and for the SPARC Collaboration, Atomic physics experiments at the high energy storage ring, Physica Scripta 2015, 014025 (2015).
  * Hagmann  _et al._ [2013] S. Hagmann, T. Stöhlker, Y. Litvinov, C. Kozhuharov, P.-M. Hillenbrand, U. Spillmann, V. Shabaev, K. Stiebing, M. Lestinsky, A. Surzhykov, A. Voitkiv, B. Franzke, D. Fischer, D. Schneider, D. Jakubassa, A. Artiomov, E. DeFilippo, X. Ma, R. Dörner, and H. Rothard, Few-body quantum dynamics of high-Z ions studied at the future relativistic high-energy storage ring, Physica Scripta 2013, 014086 (2013).
  * Hillenbrand  _et al._ [2013] P. M. Hillenbrand, S. Hagmann, T. Stöhlker, Y. Litvinov, C. Kozhuharov, U. Spillmann, V. Shabaev, K. Stiebing, M. Lestinsky, A. Surzhykov, A. Voitkiv, B. Franzke, D. Fischer, C. Brandau, S. Schippers, A. Mueller, D. Schneider, D. Jakubassa, A. Artiomov, E. DeFilippo, X. Ma, R. Dörner, and H. Rothard, Future experiments using forward electron spectroscopy to study the quantum dynamics of high-Z ions at the ESR/CRYRING storage rings, Physica Scripta 2013, 014087 (2013).
  * Kovalenko  _et al._ [2017] O. Kovalenko, Y. A. Litvinov, and T. Stöhlker, SPARC experiments with highly charged ions at the HESR of FAIR, Journal of Physics: Conference Series 875, 092013 (2017).
  * Novikov [2007] D. S. Novikov, Elastic scattering theory and transport in graphene, Phys. Rev. B 76, 245435 (2007).
  * Allor  _et al._ [2008] D. Allor, T. D. Cohen, and D. A. McGady, Schwinger mechanism and graphene, Phys. Rev. D 78, 096009 (2008).
  * Lewkowicz and Rosenstein [2009] M. Lewkowicz and B. Rosenstein, Dynamics of Particle-Hole Pair Creation in Graphene, Phys. Rev. Lett. 102, 106802 (2009).
  * Lewkowicz  _et al._ [2011] M. Lewkowicz, H. C. Kao, and B. Rosenstein, Signature of the Schwinger pair creation rate via radiation generated in graphene by a strong electric current, Phys. Rev. B 84, 035414 (2011).
  * Klimchitskaya and Mostepanenko [2013] G. L. Klimchitskaya and V. M. Mostepanenko, Creation of quasiparticles in graphene by a time-dependent electric field, Phys. Rev. D 87, 125011 (2013).
  * Fillion-Gourdeau and MacLean [2015] F. Fillion-Gourdeau and S. MacLean, Time-dependent pair creation and the Schwinger mechanism in graphene, Phys. Rev. B 92, 035401 (2015).
  * Akal  _et al._ [2019] I. Akal, R. Egger, C. Müller, and S. Villalba-Chávez, Simulating dynamically assisted production of Dirac pairs in gapped graphene monolayers, Phys. Rev. D 99, 016025 (2019).
  * Akal  _et al._ [2016] I. Akal, R. Egger, C. Müller, and S. Villalba-Chávez, Low-dimensional approach to pair production in an oscillating electric field: Application to bandgap graphene layers, Phys. Rev. D 93, 116006 (2016).
  * Golub  _et al._ [2020] A. Golub, R. Egger, C. Müller, and S. Villalba-Chávez, Dimensionality-driven photoproduction of massive dirac pairs near threshold in gapped graphene monolayers, Phys. Rev. Lett. 124, 110403 (2020).
  * Villalba-Chávez  _et al._ [2022] S. Villalba-Chávez, O. Mathiak, R. Egger, and C. Müller, Light-amplified Landau-Zener conductivity in gapped graphene monolayers: a simulacrum of photo-catalyzed vacuum instability (2022), arXiv:2211.04206 [cond-mat.mes-hall] .
  * Matveev [2005] V. I. Matveev, Sudden-perturbation approximation for the dirac equation, Theor. Math. Phys. 142, 48 (2005).
  * Matveev  _et al._ [2009] V. I. Matveev, S. V. Ryabchenko, D. U. Matrasulov, K. Y. Rakhimov, S. Fritzsche, and T. Stöhlker, Electron loss of fast heavy projectiles in collision with neutral targets, Phys. Rev. A 79, 042710 (2009).
  * Makarov and Matveev [2016] D. N. Makarov and V. I. Matveev, Analytical solution of the Schrödinger equation in the sudden perturbation approximation for an atom by attosecond and shorter electromagnetic pulses, JETP Lett. 103, 415 (2016).
  * Khalilov and Ho [1998] V. R. Khalilov and C.-L. Ho, Mod. Phys. Lett. A 13, 615 (1998).
  * De Martino  _et al._ [2014b] A. De Martino, D. Klöpfer, D. Matrasulov, and R. Egger, Electric-dipole-induced universality for dirac fermions in graphene, Phys. Rev. Lett. 112, 186603 (2014b).