Article information

Article title
A Multilingual Dataset of COVID-19 Vaccination Attitudes on Twitter
Authors
Ninghan Chen1, Xihui Chen*1, Jun Pang1, 2
Affiliations
1 Faculty of Sciences, Technology and Medicine, University of Luxembourg,
L-4364 Esch-sur-Alzette, Luxembourg

2 Interdisciplinary Centre for Security, Reliability and Trust, University of
Luxembourg, L-4364 Esch-sur-Alzette, Luxembourg

Corresponding author’s email address and Twitter handle
{ninghan.chen, xihui.chen<EMAIL_ADDRESS>
Keywords
COVID, vaccination, attitude, Twitter, dataset, health
Abstract
Vaccine hesitancy is considered as one main cause of the stagnant uptake ratio
of COVID-19 vaccines in Europe and the US where vaccines are sufficiently
supplied. Fast and accurate grasp of public attitudes toward vaccination is
critical to address vaccine hesitancy, and social media platforms have proved
to be an effective source of public opinions. In this paper, we describe the
collection and release of a dataset of tweets related to COVID-19 vaccines.
This dataset consists of the IDs of 2,198,090 tweets collected from Western
Europe, 17,934 of which are annotated with the originators’ vaccination
stances. Our annotation will facilitate using and developing data-driven
models to extract vaccination attitudes from social media posts and thus
further confirm the power of social media in public health surveillance. To
lay the groundwork for future research, we not only perform statistical
analysis and visualisation of our dataset, but also evaluate and compare the
performance of established text-based benchmarks in vaccination stance
extraction. We demonstrate one potential use of our data in practice in
tracking the temporal changes of public COVID-19 vaccination attitudes.

Specifications table

Subject |  Social science, Data Science, Computer Science
---|---
Specific subject area |  Vaccination attitudes, Sentient analysis, Social Media
Type of data |  Text
How the data were acquired |  The data are collected through the official Twitter APIs based on effective keywords commonly used in related datasets. The annotation is conducted by a group of 10 multilingual volunteers with three rounds.
Data format |  Raw (Primary)
Labelled (Secondary)
Description of data collection |  The data collection contains the IDs of 2,198,090 tweets related to COVID-19 vaccination which are posted by active Twitter users residing in four geographically adjacent countries in Western Europe. The collected tweets are involved in online discourses related to COVID vaccines or vaccination covering a period of about 14 months after the outbreak of the pandemic. The tweets are filtered according to a list of keywords that are commonly adopted in previous data collections related to vaccination (not limited to COVID-19) and prove to ensure good coverage ratios. To enable the use of data-driven methods based on machine learning and deep learning, 17,934 tweets are annotated with affective vaccination stances (i.e., positive, negative and neutral). The annotation is conducted in three rounds with 10 multilingual students among which a high-level agreement is achieved.
Data source location |  $\bullet$ Country: France, Germany, Belgium, Luxembourg
Data accessibility |  Repository name: Zenodo
Data identification number: 10.5281/zenodo.5851407
Direct URL to data: https://doi.org/10.5281/zenodo.5851407

Value of the Data

  * $\bullet$

Our dataset contains the IDs of 2,198,090 tweets relevant to the discourses on
COVID-19 vaccination for about 14 months after the onset of the pandemic. Its
large scale and long-time span allow researchers to study the vaccination
attitude evolution in Western Europe before and after the first COVID-19
vaccine was approved.

  * $\bullet$

The over 17,000 tweets annotated with vaccination attitudes facilitate
developing and validating new data-driven methods, e.g., in Natural Language
Processing (NLP), to extract vaccination attitudes and other subjective
opinions from social media posts.

  * $\bullet$

The validated performance of existing text-based NLP methods for opinion
extraction demonstrates the power of social media in tracking fine-grained
temporal changes of vaccination attitudes on a daily or weekly basis. The
tracking can subsequently lead to timely and proactive interventions in this
fast-developing pandemic or future public health events of similar types.

  * $\bullet$

The multilingualism of our annotated tweets provides a reliable data source to
evaluate existing and new language transformers in dealing with
multilingualism.

Data Description

We released the IDs of 2,198,090 tweets related to COVID-19 vaccines from
54,381 active Twitter users between January 20, 2020 and March 15, 2021,
spanning about 14 months. The tweets are originated from users located in four
adjacent Western European countries which are among the first group of regions
that received and administered COVID-19 vaccines and are being hit badly by
the pandemic: Belgium, Germany, France and Luxembourg. We manually annotated
17,934 tweets with affective vaccination stances (i.e., positive, negative and
neutral). Due to IRB review and the Twitter Terms of Service, only tweet IDs
are published. All tweets are publicly accessible and researchers are
recommended to download them with official Twitter APIs.

Although social media posts have been used to study vaccination attitudes
since the outbreak of the pandemic [12], only a few datasets are publicly
available. Pierri et al. [15] published a dataset collected from Twitter and
Facebook recording Italian users’ discussions about vaccination. DeVerna et
al. [4] released the CoVaxxy dataset composed of English-language tweets about
the COVID-19 vaccination generated from the US. Chen et al. [3] published the
MMCoVaR dataset which contains only 24,184 tweets related to COVID-19
vaccines, spanning less than one month. Our dataset differs from the above
datasets from three aspects. First, our released tweets covers a sufficient
amount of time before and after the administration of the first COVID-19
vaccine. This enables studies on the evolution of public vaccination
attitudes. Second, the tweets are originated from four countries which can
well portray the first group of COVID-19 vaccine receivers and other European
countries hit badly by the pandemic. Last not least, our annotated tweets
allow for utilising established data-driven methods to learn vaccination
attitudes from tweets, and facilitate the development of new methods.

Our release consists of two files: all_tweet and annotated_tweet. The former
lists the IDs of the 2,198,090 collected tweets while the latter is composed
of a table of 17,934 rows with two columns: _tweet_id_ and _label_. Each row
corresponds to a tweet identified with the _tweet_id_ and the _label_ field
gives its affective attitude towards vaccination. Specifically, we have five
labels which are described as follows:

  * •

Positive (PO): The originator expresses his/her support for the vaccines or
vaccination in the sense that vaccine or vaccination can effectively protect
the public, and will be or has been vaccinated.

  * •

Negative (NG): The originator expresses doubts or disbelief about the
effectiveness of the vaccines or vaccination in combating the pandemic, or
hesitates or refuses to be vaccinated.

  * •

Neutral (NE): No explicit attitude or intention are expressed.

  * •

Positive but dissatisfaction (PD): The originator expresses dissatisfaction or
complaints about the current policies or measures against COVID-19, but still
holds a positive attitude towards vaccination.

  * •

Off-topic (OT): The content is irrelevant to COVID-19 vaccines or vaccination.

We give some examples for each label in Table 2. Special attention should be
paid to the label PD. We notice that there exist a large portion of tweets
expressing the originators’ negative feelings or disagreement about the way
the governments handled the pandemic such as complaints about lock-downs.
However, the originators still showed their belief in vaccines as an effective
and ultimate measure to beat the virus. Such tweets use terms which are
negative inherently and if not explicitly separated, they will confuse
classification methods with the ones that should be labelled as NG.

Table 2: Tweet examples.

Label | Example (Translated to English)
---|---
PO | We have a new weapon against the virus: the vaccine. Hold together, again.
NG |  | My daughter, a nurse at the AP-HP, on the vaccine ”Ah ah ah! They don’t even
---
dream about it, they start with the old ones so that we can attribute the side
effects to age”.
PD | It’s bad enough for individuals to refuse #COVID19 #vaccines for themselves.
But forcing a mass vax site to shutdown, knowing it means vaccines may go to
waste, is criminal.
Call it pandemicide.
NE | Have any diabetics been vaccinated? I need some information
OT | a 10% discount on pet vaccinations next week.

In Figure 2, we display the number of collected tweets on a daily basis.
Sudden surges of daily posts can help understand the changes of public
attention and investigate possible events that cause the changes. We can see
that COVID-19 vaccines or vaccination were rarely discussed before November 9,
2020 when a sudden surge occurred. Since then, the discussion around them
remains popular. To understand the events that promoted the increase, we
extracted the tweets that were generated in the 3 days after November 9, 2020
and created a word cloud to identify the frequently used words (see Figure 2).
With the highlighted words, we infer that the event is probable due to the
first release of the efficacy of the Prizer/Biotech vaccine. With a check on
the news, we confirmed our inference. Pfizer announced the 90% efficacy of its
vaccine co-developed with Biotech.111https://www.pfizer.com/news/press-
release/press-release-detail/pfizer-and-biontech-announce-vaccine-candidate-
against With the similar approach, we checked the peak on December 27, 2021
and the sudden increase on March 15, 2021. We discovered that the former tweet
peak is attributed to the start of the EU Mass Vaccination Campaign while the
later increase is likely to be caused by the news that France and Germany
joined other European countries to temporarily halt the use of the Oxford-
AstraZeneca vaccine.

Figure 1: The temporal distribution of tweets on a daily basis.

Figure 2: Word cloud of the tweets posted between 11/9/2020 and 11/11/2020.

A character of our tweet dataset is the multilingualism which is inherent in
Europe. In Figure 4, we show the percentages of tweets in the top 5 most used
languages. As the official language of France, Belgium and Luxembourg, French
is the dominant language which is used in more than 60% of the collected
tweets. Multilingualism is considered as a challenge in NLP to extract
subjective opinions from texts. Researchers will benefit from our tweet
dataset and our annotation in developing and validating new NLP methods to
address this challenge. We depict the distribution of annotated tweets over
the vaccination attitude labels in Figure 4. Adding up those labelled both PO
and PD, we can see that more than 60% of the tweets express a positive
attitude toward vaccination while about 20% are associated with negative
attitudes.

Figure 3: Distribution of tweet over languages.

Figure 4: Annotated attitude distribution.

Experimental design, materials and methods

##  1\. Data Collection

Our data collection process comprises two steps. In the first step, we
constructed the set of active users from our targeted regions, i.e., Belgium,
Germany, France and Luxembourg. We say a user is _active_ if he/she frequently
participates in vaccine-related discourse and interacts with others. We
focused on these users because active users are more likely to express their
true opinions through tweets. In the second step, we streamed tweets from the
identified active users using manually chosen keywords.

###  1.1 Active user identification

According to the Twitter policy, the only way to get Twitter users is from the
meta-information associated with tweets. Instead of direct searching tweets,
we refer to a publicly available tweet dataset [2]. This continuously updated
dataset contains IDs of tweets related to COVID-19 and originated from users
all over the world. Our data collection spans from January 22, 2020 to March
15, 2021, and covers a period from the early stage of the pandemic to the
beginning of the vaccination campaign. We downloaded the tweet IDs from the
dataset which fall in this period, and then crawled the corresponding tweets
with the Twitter API. In total, we downloaded 51,966,639 tweets from which
15,551,266 Twitter users are obtained. To retain users that are active and
located in the targeted countries, we conducted two sequential filtering
operations: _location-based filtering_ and _activity-based filtering_.

Location-based filtering. We made use of the geographic information in the
metadata of tweets to learn users’ originating countries. If a user has
multiple tweets generated from different countries, although this rarely
occurs among our collected users, we use the country reported in the earliest
tweet. The metadata of a tweet has two fields to store the originator’s
locations: _Geo_ and _Place_. The _Geo_ field records the location generated
by the user’s device while the _Place_ field stores the geographic information
provided by users. The _Geo_ information is accurate and in a unified format
that can be automatically parsed. However, only about 2% of tweets come with
_Geo_ values. As a result, we use the _Place_ field which is usually ambiguous
when the _Geo_ field is not available. To regularise the location format and
remove the ambiguity, we leveraged the the ArcGis Geocoding, which is widely
used in previous research for the same purpose [7]. For example, the _Place_
field value _Moselle_ , input by a user, is converted into a machine-
resolvable location including city, state and country: _Mosselle, Lorraine,
France_. Among the selected users, more than 70% have at least one
geographical field filled. We removed the users located outside the four
countries and finally obtained 767,583 users.

Activity-based filtering. We built a _user interaction graph_ to screen out
inactive users. A user interaction graph is un-directed and weighted. An edge
is created between two users, e.g., $u$ and $u^{\prime}$, when user $u$
retweets or mentions a tweet generated from user $u^{\prime}$ or the other way
around. The weight of the edge is the number of retweets or mentions between
two end users. We first eliminated all edges with weights less than two to
exclude the occasional interactions. Then we removed the vertices with degrees
smaller than 2 to ensure active users possess frequent interactions with
multiple users. In the end, we retained in total 54,381 active users.

###  1.2 Vaccine-related tweet streaming

With the set of active users in the targeted countries, in this step, we
crawled the tweets originated or retweeted by these users that are related to
COVID-19 vaccines or vaccination. Same as previous studies [20, 4], we use
keywords related to COVID-19 vaccines to filter tweets. Two methods have been
adopted to select vaccine-related keywords in the literature [4]. One is
called _snowball sampling_ which iteratively enriches the initial set of
keywords according to the newly downloaded messages. The other method directly
constructs the set of keywords based on expert knowledge and contexts. As many
keyword lists are publicly available and produce rather good coverage [9, 19],
we decided to refer to them and only selected the ones with the best coverage
to keep the list short. As the tweets originated from our targeted countries
are written in multiple languages, which are different from those studied in
the previous works, we translated the selected keywords when necessary. After
multiple rounds of manual validation, we used all the keywords _containing_
the following words as substrings: ‘ _vax_ ’, ‘ _vaccin_ ’, ‘ _covidvic_ ’, ‘
_impfstoff_ ’, ‘ _vacin_ ’, ‘ _vacuna_ ’, ‘ _impfung_ ’. ‘ _sputnikv_ ’, ‘
_astrazeneca_ ’, ‘ _sinovac_ ’, ‘ _pfizer_ ’, ‘ _moderna_ ’, ‘ _janssen_ ’, ‘
_johnson_ ’ and ‘ _biontech_ ’.

We used the Twitter Academic Research API to search relevant tweets based on
the active users’ IDs and the keywords. The API allows for downloading at most
500 tweets for each downloading request. In order to ensure a good coverage,
we constructed a request for each user every month. This enables us to obtain
an acceptable coverage due to the small likelihood that a user posts more than
500 tweets related to COVID-19 vaccines in one month. In total, we downloaded
2,198,090 tweets whose IDs are released.

##  2\. Data Annotation

###  2.1 Annotator training and consolidation

Since the number of downloaded tweets exceeds our capacity to annotate, we
selected a number of tweets that can well represent the linguistic features of
COVID-10 vaccination related tweets. Specifically, we first sorted the
downloaded tweets in the descending order by their numbers of times being
retweeted. We then removed the most frequently retweeted tweet from the
ordered list and added it to the list of tweets to annotate iteratively until
every active user has at least one posted or retweeted message to annotate. In
total, we selected 17,934 tweets.

We hired 10 bachelor students to manually annotate the sampled tweets. All
annotators are proficient in at least two of the four official languages of
the targeted countries, and, in the meantime, were active on Twitter. One
author of this paper acted as the coordinator in charge of annotator training
and annotation consolidation. Each annotator received a tutorial from the
coordinator explaining the semantics of all labels with examples. We also
distributed a guideline illustrating the workflow on the Doccano
platform222github.com/doccano/doccano we built to collect annotators’ input.
To ensure that all annotators held the correct understanding, we conducted a
pilot annotation process in which all annotators were first asked to annotate
100 tweets. The coordinator verified their annotations and provided additional
explanations if necessary. We repeated the process with another 100 tweets.
After two rounds of training, annotators succeeded in understanding the labels
and also became familiar with the Doccano platform.

We first selected one annotator to annotate all the tweets and this full
annotation took approximately 60 hours. We then randomly assigned to each of
the rest 9 annotators around 2,000 tweets and asked them to validate the
labels. Meanwhile, the coordinator went through all annotated tweets. When
disagreeing with the labels given by the first annotator, they added new
labels. This validation took the coordinator about 60 hours and each of the
other annotators 4 hours. In this way, our annotation strategy ensured each
tweet be labelled three times. To solve the conflicts, the coordinator
consolidated all annotations. The label agreed by at least two annotators is
set as the final annotation. For those with three different labels, the
coordinator communicated with the other two annotators and picked the most
appropriate one.

###  2.2 Annotator agreement

To ensure the quality of our annotations, we leverage three widely accepted
measurements to quantitatively evaluate the inter-annotator reliability for
each label: Average Observed Agreement (AOA) [6], Fleiss’ kappa [6], and
Krippendorff’s Alpha [11]. AOA is the average observed agreement between any
pair of annotators. The term “observed agreement” in AOA refers to the
proportion of labels two annotators agree with. Both Fleiss’ kappa and
Krippendorff’s Alpha are applicable to measure the agreement between a fixed
number of annotators, with the difference that Krippendorff’s Alpha can handle
missing labels. The values of all the three measurements range from 0 to 1,
where 0 indicates complete disagreement and 1 indicates absolute agreement.
For Fleiss’ Kappa, 0.41-0.60, 0.61-0.80, and 0.81-1.0 are considered as
moderate agreement, substantial agreement, and excellent agreement,
respectively [6]. Krippendorff’s Alpha is more rigorous than normal standards
[11]. Values between 0.667 and 0.800 are deemed acceptable, while values
greater than or equal to 0.8 are considered highly reliable [11].

Table 3: Inter-annotator agreement (PO: Positive, NG: Negative, NE: Neutral, PD: Positive but dissatisfaction, OT: Off-topic). Label | AOA | Feliss’ kappa | Krippen-dorf’s Alpha
---|---|---|---
PO | 0.72 | 0.73 | 0.73
NG | 0.82 | 0.88 | 0.88
NE | 0.74 | 0.78 | 0.77
PD | 0.61 | 0.63 | 0.62
OT | 0.83 | 0.87 | 0.86

Table 3 summarises the inter-annotator agreement for each annotation label. We
can see that for all labels, AOA scores range from 0.61 to 0.83. This implies
that most of the annotations have at least two annotators in agreement. The
values of the other two measurements are close. The annotators achieved the
highest rank of agreement according to Fleiss’ kappa and Krippendorff’s Alpha
for both the labels NE and OT and the second highest rank on labels PO and NE.
The annotators’ agreement on PD falls drastically compared to other labels,
but still remains moderate according to the ranking criteria of the Fleiss’
Kappa measurement. This can be explained by our difficulties during annotation
in dealing with the special linguist features of PD tweets, i.e., frequently
used negative terms or sarcastic expression. A closer look will lead to
another observation that the extent of agreement on the label PO is slightly
lower. A careful manual investigation reveals that a large proportion of
disagreed annotations also attribute to the sarcasm and irony made to express
their opinions about anit-vaccination. This identified challenge to handle
sarcasm is consistent with the previous finding that people frequently are
confused by sarcasm, which makes comprehension difficult [17]. We take the
following tweet as an example: _I am very disappointed! 16 days after my first
injection of the vaccine against #covid19 I still don’t get the 5g._ This
tweet uses ironic expression joking about anti-vaccination comments but in
fact delivers definite supporting attitude for vaccination. Such tweets
produced misunderstandings among annotators, which are solved in our
consolidation phase.

##  3\. Vaccination attitude calculation with NLP

The application of deep learning and machine learning has revolutionised NLP,
especially in extracting opinions or sentiments from textual contents.
Compared to machine learning models which rely on manually constructed
features, deep learning models can learn effective features automatically with
little manual intervention. Extensive empirical evidence has proved the
overwhelming performance of deep learning models in NLP studies [1]. We
trained and ran several well established NLP models based on machine learning
and deep learning with our annotated tweets. A good performance of the trained
models in classifying tweets will attest the utility and trustworthiness of
our annotation.

Experiment setup. We select Random Forest (RF) and Support Vector Machines
(SVM) as the representative machine learning models due to their wide use.
Regarding deep learning models, we use BERT [5], RoBERTa [13], and DistilBERT
[18]. Such deep learning models are pretrained and produce a low-dimensional
representation for any given piece of text, which can be used as input for
downstream classification methods.

We preprocess the tweets by removing mentions of other users with ‘@’, quoted
hyperlinks and ‘RT’ which stands for “retweet”. We remove tweets with the
label ‘off-topic’ due to their small proportion. To test the models’
capability in dealing with multilingualism, we construct three datasets: the
original annotated tweets, the French-language annotated tweets and the
German-language annotated tweets. We divide each of these three datasets into
training, testing and validation set with the ratio 80%, 10% and 10%,
respectively. To train the selected machine learning models, we use
TfidfVectorizer [14] to convert the preprocessed tweets into the bag of
$n$-gram vectors. We use grid search as the optimiser for SVM and RF. For deep
learning methods, we adopt their publicly available implementation for text
embedding and keep their default settings. The text embeddings are then sent
to a fully-connected ReLU layer with dropout. A linear layer is added on the
top of the final outputs for regression with softmax as the activation
function. We use CrossEntropyLoss as the loss function and Adam as the
optimiser [10]. All models are trained for 30 epochs for optimisation with the
learning rate of $0.00001$, and batch size of 32. We set the maximum sequence
length as 128, which defines the maximum numbers of tokens in a tweet that can
be processed.

Result analysis. Table 4 shows the classification performance evaluated with
conventional measurements. First, we can see that the deep learning methods
outperform the machine learning methods in classifying both multilingual
tweets and those in single languages, and their performances are close. This
confirms the findings in [8]. Second, the results show that multilingualism
affects the classification performance of deep learning models, although we
use a pre-trained model specifically for classifying tweets in multiple
languages.

By comparing with the models’ performance on other classifying tasks in the
literature [1, 8], we observe that the models can achieve the same-level
performances. This implies that our annotation is trustworthy and useful for
future research on vaccination attitude learning. The results we showed in
Table 4 can thus be referred to as benchmarks for comparison.

Table 4: Classification results for different benchmarks.

| Mulitilingual | French | German
---|---|---|---
Model | Precision | Recall | F1 | Accuracy | Precision | Recall | F1 | Accuracy | Precision | Recall | F1 | Accuracy
RF | 0.4317 | 0.3219 | 0.4471 | 0.4749 | 0.5676 | 0.4829 | 0.4754 | 0.5510 | 0.51481 | 0.4978 | 0.4611 | 0.4503
SVM | 0.4001 | 0.3816 | 0.4380 | 0.4263 | 0.5004 | 0.4256 | 0.4141 | 0.4998 | 0.4954 | 0.4037 | 0.4093 | 0.4719
mBERT | 0.6622 | 0.5769 | 0.6132 | 0.6466 | 0.7016 | 0.6933 | 0.7004 | 0.7184 | 0.6999 | 0.6875 | 0.6919 | 0.7038
XLM-RoBERTa | 0.6801 | 0.5848 | 0.6271 | 0.6618 | 0.7023 | 0.7018 | 0.7145 | 0.7086 | 0.7102 | 0.6971 | 0.7081 | 0.7079
Distil-mBERT | 0.6768 | 0.5834 | 0.6287 | 0.6601 | 0.6978 | 0.6916 | 0.7084 | 0.7065 | 0.7094 | 0.7004 | 0.7068 | 0.7071

##  4\. Temporal analysis of vaccination attitudes

We present a description of an application of our tweet dataset and annotation
to illustrate one potential use of our release. Specifically, we visualise the
temporal evolution of the various vaccination attitudes and analyse the
possible causes to some changes that require more attention.

We apply XLM-Roberta model trained in the previous experiment to calculate the
vaccination attitudes of the tweets in the tweet dataset. Figure 5 shows the
distribution of the predicted vaccination attitudes. We can see the
distribution is similar to that shown in Figure 4 with slightly more tweets
with negative attitudes.

Figure 5: Attitude distribution of all tweets.

We draw the temporal evolution of the numbers of tweets that are classified as
NE, PO and PD in Figure 6. Note we ignore the tweets with label OT due to
their small amount. Recall that significant discussion on COVID-19 vaccines
started from November 8, 2020 as shown in Figure 2. Therefore, we focus on the
attitude changes from that date. We can see that the number of tweets
containing different attitudes toward vaccination changes over time. Compared
to the temporal distribution of daily tweets shown in Figure 6, we observe
that the growth in the total number of tweets is not accompanied by a
proportional change of vaccination attitudes. Specifically, the number of
neutral tweets varies less from day to day and remains stable at the same
level compared to tweets with other attitude labels. Based on previous
research reporting that the content of tweets is highly correlated with real-
world situations [16], we make a hypothesis that real-world events may
contribute to the fluctuating numbers of tweets with different attitudes.

Figure 6: Temporal distribution of tweets with different vaccination attitude
labels.

In monitoring vaccine hesitancy, more attention should be paid to the
fluctuations of the negative attitudes. In the following, we take four peaks
of the curve of negative tweets as examples and discuss the potential causes.
We follow the same approach used in the temporal analysis of tweets volumes.
We first plot word clouds to identify the most frequently used keywords and
then search these keywords on the Internet to understand the events that may
contribute to the changes. The first peak happened around December 9, 2020 due
to negative news and misinformation about the vaccine efficacy. For instance,
on December 9, 2020, two healthcare workers were reported to have experienced
symptoms after receiving their first shots, and on December 11, Sanofi and GSK
delayed COVID-19 vaccine. A wide spread piece of misinformation during this
period is also identified, saying that 6 people had died after vaccinated and
another 4 developed Bells Palsy. Another two peaks occurred around January 15,
2021 and February 10, 2021, respectively. We notice that both of these two
peaks attribute to the propagation of a large volume of misinformation. Take
two pieces of misinformation identified for the peak in January, 2021 as
examples. One said that on January 14, the Norwegian Medicines Agency reported
that a total of 29 people had suffered side effects, 13 of which were fatal.
The other was about the death of an Indian healthcare worker after receiving
Covid-19 vaccines. The last peak in March, 2021 is due to the negative news
that multiple countries decided to suspend the use of the AstraZeneca vaccine
due to the reported negative effects.

From the above discussion, we can see that with NLP deep learning models,
public vaccination attitudes can be extracted. When social network data are
available, we can track almost in real time the changes of vaccination
attitudes and understand the potential causes. This may finally help the
governments make corresponding effective intervention methods proactively.

Ethics statements
We strictly adhered to the Twitter Developer Agreement and
Policies333https://developer.twitter.com/en/developer-terms/agreement-and-
policy in the collection and distribution of data. Our release is also
compliant with the EU General Data Protection Regulation (GDPR). The released
data do not contain any personally identification information. Only tweet IDs
and annotation are published.

CRediT author statement

Ninghan Chen: Data curation, Visualization, Investigation. Xihui Chen:
Conceptualization, Methodology, Writing- Original Draft. Jun Pang:
Supervision, Validation, Writing- Reviewing and Editing.

Acknowledgments

Funding: This work is supported by Luxembourg’s Fonds National de la Recherche
via grants COVID-19/2020-1/14700602 (PandemicGR), PRIDE17/12252781/DRIVEN and
PRIDE15/10621687/SPsquared.

Declaration of Competing Interest

  * $\text{\hbox to0.0pt{$\checkmark$\hss}}\square$

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

  * $\square$

The authors declare the following financial interests/personal relationships
which may be considered as potential competing interests:

References

## References

  * [1] Bharathi Raja Chakravarthi, Navya Jose, Shardul Suryawanshi, Elizabeth Sherly, and John Philip McCrae.  A sentiment analysis dataset for code-mixed malayalam-english.  In Dorothee Beermann, Laurent Besacier, Sakriani Sakti, and Claudia Soria, editors, Proc. 2020 Joint Workshop on Spoken Language Technologies for Under-resourced languages and Collaboration and Computing for Under-Resourced Languages, (SLTU), pages 177–184. European Language Resources association, 2020.
  * [2] Emily Chen, Kristina Lerman, and Emilio Ferrara.  Tracking social media discourse about the COVID-19 pandemic: Development of a public coronavirus Twitter data set.  JMIR Public Health and Surveillance, 6(2):e19273, 2020.
  * [3] Mingxuan Chen, Xinqiao Chu, and K. P. Subbalakshmi.  Mmcovar: Multimodal COVID-19 vaccine focused data repository for fake news detection and a baseline architecture for classification.  CoRR, abs/2109.06416, 2021.
  * [4] Matthew R. DeVerna, Francesco Pierri, Bao Tran Truong, John Bollenbacher, David Axelrod, Niklas Loynes, Christopher Torres-Lugo, Kai-Cheng Yang, Filippo Menczer, and John Bryden.  Covaxxy: A collection of english-language twitter posts about COVID-19 vaccines.  In Ceren Budak, Meeyoung Cha, Daniele Quercia, and Lexing Xie, editors, Proc. 2021 International AAAI Conference on Web and Social Media, (ICWSM), pages 992–999. AAAI Press, 2021.
  * [5] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  BERT: pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 4171–4186. ACL, 2019.
  * [6] Joseph L Fleiss, Bruce Levin, and Myunghee Cho Paik.  Statistical methods for rates and proportions.  John Wiley & Sons, 2013.
  * [7] Brent J. Hecht, Lichan Hong, Bongwon Suh, and Ed H. Chi.  Tweets from justin bieber’s heart: the dynamics of the location field in user profiles.  In Proc. 2011 International Conference on Human Factors in Computing Systems CHI, pages 237–246. ACM, 2011.
  * [8] Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson.  XTREME: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation.  In Proc. 2020 International Conference on Machine , (ICML), volume 119, pages 4411–4421. PMLR, 2020.
  * [9] Amir Hussain, Ahsen Tahir, Zain Hussain, Zakariya Sheikh, Mandar Gogate, Kia Dashtipour, Azhar Ali, and Aziz Sheikh.  Artificial intelligence–enabled analysis of public attitudes on facebook and twitter toward covid-19 vaccines in the united kingdom and the united states: Observational study.  Journal of Medical Internet research, 23(4):e26627, 2021.
  * [10] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In Proc. 2015 International Conference on Learning Representations (ICLR). ICLR, 2015.
  * [11] Klaus Krippendorff.  Estimating the reliability, systematic error and random error of interval data.  Educational and Psychological Measurement, 30(1):61–70, 1970.
  * [12] Siru Liu and Jialin Liu.  Public attitudes toward COVID-19 vaccines on English-language twitter: A sentiment analysis.  Vaccine, 39(39):5499–5505, 2021.
  * [13] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized BERT pretraining approach.  CoRR, abs/1907.11692, 2019.
  * [14] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.  Scikit-learn: Machine learning in Python.  Journal of Machine Learning Research, 12:2825–2830, 2011.
  * [15] Francesco Pierri, Silvio Pavanetto, Marco Brambilla, and Stefano Ceri.  Vaccinitaly: monitoring italian conversations around vaccines on twitter.  CoRR, abs/2101.03757, 2021.
  * [16] Neha Puri, Eric A Coomes, Hourmazd Haghbayan, and Keith Gunaratne.  Social media and vaccine hesitancy: new updates for the era of COVID-19 and globalized infectious diseases.  Human Vaccines & Immunotherapeutics, 16(11):2586–2593, 2020.
  * [17] Antonio Reyes, Paolo Rosso, and Tony Veale.  A multidimensional approach for detecting irony in Twitter.  Language Resources and Evaluation, 47(1):239–268, 2013.
  * [18] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf.  Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter.  CoRR, abs/1910.01108, 2019.
  * [19] Ana Lucía Schmidt, Fabiana Zollo, Antonio Scala, Cornelia Betsch, and Walter Quattrociocchi.  Polarization of the vaccination debate on facebook.  Vaccine, 36(25):3606–3612, 2018.
  * [20] Samira Yousefinaghani, Rozita Dara, Samira Mubareka, Andrew Papadopoulos, and Shayan Sharif.  An analysis of COVID-19 vaccine sentiments and opinions on Twitter.  International Journal of Infectious Diseases, 2021.


# Robust and fast post-processing of single-shot spin qubit detection events
with a neural network

Tom Struck  JARA-FIT Institute for Quantum Information, Forschungszentrum
Jülich GmbH and RWTH Aachen University, Aachen, Germany  Javed Lindner  JARA-
FIT Institute for Quantum Information, Forschungszentrum Jülich GmbH and RWTH
Aachen University, Aachen, Germany  Arne Hollmann  JARA-FIT Institute for
Quantum Information, Forschungszentrum Jülich GmbH and RWTH Aachen University,
Aachen, Germany  Floyd Schauer  Andreas Schmidbauer  Institut für
Experimentelle und Angewandte Physik, Universität Regensburg, Regensburg,
Germany  Dominique Bougeard  Institut für Experimentelle und Angewandte
Physik, Universität Regensburg, Regensburg, Germany  Lars R. Schreiber  JARA-
FIT Institute for Quantum Information, Forschungszentrum Jülich GmbH and RWTH
Aachen University, Aachen, Germany<EMAIL_ADDRESS>
(August 29, 2024)

###### Abstract

Establishing low-error and fast detection methods for qubit readout is crucial
for efficient quantum error correction. Here, we test neural networks to
classify a collection of single-shot spin detection events, which are the
readout signal of our qubit measurements. This readout signal contains a
stochastic peak, for which a Bayesian inference filter including Gaussian
noise is theoretically optimal. Hence, we benchmark our neural networks
trained by various strategies versus this latter algorithm. Training of the
network with 106 experimentally recorded single-shot readout traces does not
improve the post-processing performance. A network trained by synthetically
generated measurement traces performs similar in terms of the detection error
and the post-processing speed compared to the Bayesian inference filter. This
neural network turns out to be more robust to fluctuations in the signal
offset, length and delay as well as in the signal-to-noise ratio. Notably, we
find an increase of 7 % in the visibility of the Rabi-oscillation when we
employ a network trained by synthetic readout traces combined with measured
signal noise of our setup. Our contribution thus represents an example of the
beneficial role which software and hardware implementation of neural networks
may play in scalable spin qubit processor architectures.

##  1 Introduction

Fast and high-fidelity single-shot readout of qubits is vital for the
realisation of quantum information processing. Since quantum error correction
schemes require frequent qubit readout [6], the qubit measurement time should
not be much longer than the qubit manipulation time to avoid speed
limitations. The readout scheme depends upon the specific qubit realisation
and can be discriminated into two categories [2]: The measurement signal
starts either immediately after the trigger of the detection process [7], or
it is delayed randomly by a turn-on time [8]. While spin-to-charge conversion
of a singlet-triplet spin readout by Pauli-spin blockade falls into the first
category [9, 10], single-spin detection by energy-dependent tunneling to a
weakly tunnel-coupled reservoir falls into the second [8, 11]. For the latter,
the analog measurement signal is often post-processed by peak-signal filters
to assign a binary qubit readout. Examples for peak-signal filter are wavelet
edge detection [12], signal threshold [8, 13] and slope threshold after
filtering the signal with total variation denoising [14, 4].

If only one single spin detection cycle is considered, a Bayesian inference
filter capturing the tunneling constants and typical noise is optimal [2]. The
readout speed with the Bayesian filter can be improved by adaptive decisions
which allow to balance measurement time versus read-out fidelity [15]. As the
signal-to-noise ratio (SNR) of the detection signal is lowered for qubits in a
dense array [21] or for charge detectors operating at elevated temperature,
post-processing robust to low SNR is essential for future quantum computing
architectures and hot electron spin qubits [16], motivating the testing of
alternatives to the theoretically optimal Bayesian method.

Figure 1: Single-shot measurement traces and architecture of the neural
network. a) Sketch of the Zeeman energy-resolved tunneling from the QD to the
reservoir with a tunnel rate $\Gamma_{i}$. Two typical SET current traces
showing the cycle of emptying and loading stage of the QD followed by the
qubit readout stage are also shown. Only when the qubit state is
$\ket{\uparrow}$-spin, the electron tunnels out of the QD at time $t_{i}$ and
a $\ket{\downarrow}$-spin tunnels on the QD at time $t_{f}$ resulting in a
peak in the SET current (blue trace). b) Zoom-in of two measured SET current
traces spanning the readout stage. The blue and red traces are classified to
be $\ket{\uparrow}$ and $\ket{\downarrow}$, respectively. c) Current signal
spanning the readout stage averaged for many $\ket{\downarrow}$ and
$\ket{\uparrow}$ traces (blue), fitted by equation 1 (dashed red line). d)
Architecture of the network used to classify peak traces. One trace spanning
the readout time is used as input (top) and classification is output by two
neurons (bottom), giving the probability of $\ket{\downarrow}$ and
$\ket{\uparrow}$ trace.

##  2 Results

Here, we report on the performance of neural networks, which have been
previously used to tune the electrostatics of devices[23, 24, 25, 26], to
post-process single spin readout by spin-to-charge conversion. We compare
their robustness and post-processing time to a Bayesian inference filter. We
find that a neural network can perform similarly to the Bayesian inference
filter on synthetic data and slightly outperforms it on real data, if it is
made more robust by variations in the training data. Considerably better
performance is achieved on the classification of measured data, when the
neural network is trained with synthetic traces combined with measured noise.

Our qubit system is a single electron spin qubit trapped in an
electrostatically defined Si/SiGe quantum dot (QD) [14]. The qubit is encoded
by the electron spin states $\ket{\uparrow}$ and $\ket{\downarrow}$, which are
energetically split by an in-plane magnetic field of 668 mT. Our detection
signal consists of the single-shot readout of spin orientations via spin-to-
charge conversion: Setting the chemical potential of the two-dimensional
electron reservoir (electron temperature 114 mK) as plotted in Fig. 1a at time
$t=0$ s, energetically only an electron in the $\ket{\uparrow}$ state can
tunnel into the reservoir after a time $t_{i}$ following Poisson statistics
[8]. At a time $t_{f}$, the empty QD is reinitialized by an electron in a
$\ket{\downarrow}$-state from the reservoir. The two tunnel events are
detected by the current $I_{SET}$ of a capacitively coupled single-electron
transistor (SET). Signal traces for $\ket{\uparrow}$ and $\ket{\downarrow}$
events are shown exemplarily in Fig. 1b. Averaging the $I_{SET}$ traces of
$3\cdot 10^{5}$ $\ket{\uparrow}$-events (Fig. 1c ), we fit the distribution by

$\langle I_{SET}(t)\rangle\propto e^{-t\Gamma_{f}}(1-e^{-t\Gamma_{i}})$ (1)

where $I_{SET}(t)$ is the current signal, meaning that $\langle
I_{SET}(t)\rangle$ is proportional to the probability that the QD is
unoccupied (Fig. 1c). $\Gamma_{i}$ and $\Gamma_{f}$ are the tunneling rates
out and into the QD, respectively. For the plotted example, we find
$\Gamma_{i}^{-1}=0.20\pm 0.07\,$ms and $\Gamma_{f}^{-1}=2.20\pm 0.004\,$ms.

We consider a network implemented in tensorflow [5] with keras and investigate
its post-processing performance to classify the $I_{SET}$ traces into
$\ket{\uparrow}$\- and $\ket{\downarrow}$-events. The input layer is connected
to four 1D convolutional layers with kernel sizes (101,51,25,10), a filter
depth of (32,16,16,8) and ReLU [19] activation (Fig. 1d). Inbetween each
convolutional layer, a maxpooling layer with size 3 is inserted. The
convolutional layers feed into a dense network with three layers of size
(64,32,2). The first two layers use ReLU activation and the last layer uses
softmax, since the last two neurons categorize into 1 and 0 for a
$\ket{\uparrow}$\- and $\ket{\downarrow}$-event, respectively. This neural
network architecture was selected after testing variations of the network,
both by changing it by hand and with Bayesian optimisation of some of the
network parameters: We optimised the kernel sizes, filter depths of the
convolutional layers, as well as the number and size of the dense layers and
the dropout rate after each dense layer using the Bayesian optimisation. We
find that sufficiently large networks have a similar error rate. If the
network is too small, e.g. three small dense layers and no convolutional
layers, the achieved accuracy decreases by approx. 4-5%. A network without
convolutional layers is also able to reach the same accuracy, but converges
slower. Note, that too large networks become inefficient as far as training
and evaluation time is concerned. The size of the architecture chosen here,
represents the best compromise we found.

We train this network architecture by three qualitatively different sets of
traces and will call theses trained networks $B$, $C$ and $D$. In all cases,
we employ the neural network architecture explained above with the adam
optimiser [20] and categorical cross-entropy as the loss function. For the
training of network $B$, we synthesize $\ket{\uparrow}$\- and
$\ket{\downarrow}$-traces with Gaussian noise and
$\Gamma=\Gamma_{f}/\Gamma_{i}=1$. We express the SNR by $r$, where $r$ is the
power signal to-noise-ratio integrated over the average high signal time
$\langle t_{f}-t_{i}\rangle$ [2]

$r^{-1}=\int_{0}^{\langle t_{f}-t_{i}\rangle}dt\int_{0}^{\langle
t_{f}-t_{i}\rangle}dt^{\prime}\langle\delta\Psi(t)\delta\Psi(t^{\prime})\rangle$
(2)

where $\Psi$ is the signal trace scaled between $-1$ and 1,
$\delta\Psi(t)=\Psi(t)-\langle\Psi(t)\rangle$, the deviation of the signal
from the noiseless signal. Examples of traces with three different $r$ are
plotted as inset in Fig. 2a. The synthetic traces are generated such that the
position of the lower level in the noiseless signal is at -1 and the high
level is at 1. Then we add Gaussian noise to this trace. The network $B$ is
trained by synthetic traces with various $r$. It is trained on $5\cdot 10^{5}$
traces with an equal distribution of traces synthesized with $r$ ranging from
1 to 400. Since we synthesize the training data, the correct labeling of the
event traces is ensured. This is in contrast to the network $C$, which we
train with 106 measured traces collected over two months of continuously
running experiments with one device. During the measurements we tried to keep
$\Gamma_{i}$ and $\Gamma_{f}$ in the range specified in Fig. 1c. The measured
data has to be classified, since we need labels for the training. We used a
Bayesian inference filter for this classification [2]. The network $D$ is
trained by traces, which are synthesized similar to the traces for the network
$B$, but instead of Gaussian noise we generate noise from the measured power
spectrum of the experimental setup, which encompasses the qubit device and all
the setup electronics representing our common noise sources. Since we
synthesize the peak of these training traces as we do for the training data of
the network $B$, labeling is $100\,\%$ correct, while the labeling of the
training data of network $C$ is defective due to the error of the Bayesian
inference filter.

Figure 2: a) Classification error of the neural network $B$, trained with
synthetic $\ket{\uparrow}$\- and $\ket{\downarrow}$-traces, compared to
classification by the Bayesian inference filter, considering Gaussian noise,
$\Gamma=1$ and various $r$. The network $C$ is trained on meausured data and
network $D$ is trained with synthetic traces superimposed with typical noise
from the experimental setup. The green inset traces are example input traces
for $\ket{\uparrow}$-events at a given $r$.b)-d) Robustness of the Bayesian
inference filter and the neural networks $B$, $C$ and $D$ to three parameters
of synthesized traces superimposed by Gaussian noise. b) Robustness to the SNR
$r$ variations. c) Robustness to an offset of the low-current level of
$I_{SET}$. The x-axis here indicates the deviation of the lower level from the
value of -1. d) Robustness to variations in the tunnel rates $\Gamma$.

Next, we compare the classification error of the neural networks $B$, $C$ and
$D$ to the Bayesian inference filter from Ref. [2] (see equation C4 in the
appendix therein) as a function of the SNR. In order to determine the
classification error, defined by the sum of the $\ket{\downarrow}$-states
labeled $\ket{\uparrow}$, we synthesized 100.000 $\ket{\uparrow}$\- and
100.000 $\ket{\downarrow}$-traces with Gaussian noise, $\Gamma=1$ and various
$r$ values. The classification error of the network $B$ is nearly the same,
compared to the Bayesian inference algorithm, within numerical uncertainties
(Fig. 2a). This result is not surprising, since neural networks having a large
enough size can emulate any function [3]. Remarkably, however, the network $B$
classifies synthetic traces of various $r$, as it has been trained by various
$r$ values, meaning that the network $B$ is made robust against $r$
fluctuations. The neural networks $C$ and $D$ show a significant larger
classification error than the network $B$ and the Bayesian estimate. This is
due to the fact that these networks are trained with at least partially
experimental contribution, which are not captured by the synthetic traces used
to determine the classification error in Fig. 2a. Specifically, the typical
experimental noise is more involved than just Gaussian noise and the networks
$C$ and $D$ were only trained by the measured SNR range. Note that the network
$C$ also contains the classification error of the Bayesian inference filter
used for labeling the training data set. The network $D$ is trained on
synthetic traces superimposed with experimental noise. If we determine the
classification error by such synthetic traces superimposed with the
experimentally measured noise spectrum, the $D$ network achieves already a
lower classification error (2.7 %) compared to the Bayesian inference filter
(5.7 %).

In experiments typically a SNR of $r\approx 400$ is achievable [18, 17].
According to Fig. 2a, we can achieve a low qubit detection error rate of less
than $1\%$ with the network $B$ as well as with the Bayesian inference filter.
However, this result only refers to the ideal synthetic traces. Real signal
traces contain noise with complicated noise spectral densities originating
from e.g. interference noise, charge noise from an ensemble two-level
fluctuators, and Johnson noise. Thermal excitation, spin relaxation or co-
tunneling faster than the time-scale of $t_{i}$ and $t_{f}$ lead to defective
signal traces as well. These latter effects can be suppressed by proper tuning
of the ratio of the tunnel rates to measurement bandwidth and the ratio of the
Zeeman energy to the electron temperature. Challenging are slow variations of
the SNR $r$, the current offset of the SET and variations of the ratio between
tunnel-in and -out rate $\Gamma$, due to low-frequency charge-noise or
uncompensated cross-capacitive couplings to other gates. Therefore, we
investigate the robustness of the post-processing filters with respect to
these parameters in the following paragraph.

Fig. 2b shows the robustness of the different approaches to variations in the
$r$-parameter. Since the networks do not get $r$ as an input parameter, the
dependencies here are the same as in Fig. 2 a. In contrast to Fig. 2a , now
the Bayesian filter has a fixed $r$-parameter set to 200. Comparing the
Bayesian inference filter to the network $B$, remarkably, the Bayesian
inference filter performs generally worse than the neural network $B$, e.g. it
shows up to 2% additional error at $r=100$, and it deviates sharply from the
optimum as the SNR decreases. Here the Bayesian algorithm seems to readily
interpret single spikes in the noisy signal trace as peaks and labels them as
$\ket{\uparrow}$ by error. Note that the error rate of the Bayesian inference
filter saturates at an error above 1 % for an increasing SNR. This at first
sight surprising observation is caused by $\ket{\uparrow}$-traces classified
as $\ket{\downarrow}$, since the peak is mistaken to be Gaussian noise as the
amplitude of Gaussian noise is much lower than the Bayesian filter expected by
the implemented $r=200$. Very important is the position of the overall
$I_{SET}$ level: The Bayesian estimate quickly fails at predicting the correct
result (Fig. 2c ), while the neural network $B$ shows no increase of error in
a wide offset range of -0.75 to 0.75. Here, the offset is to be understood as
the error in assigning -1 to the lower signal level, while the total amplitude
is kept at 2. Finally, we investigate the robustness of the different methods
with respect to the $\Gamma$ parameter (Fig. 2d). We observe that all post-
processing approaches except for network $C$ remain accurate for $\Gamma>1$.
If $\Gamma\ll 1$ and thus $\Gamma_{f}\ll\Gamma_{i}$, the classification error
rises for every method, since the peak beginning starts to exceed the
measurement window, i.e. the predefined length of the signal trace.

Figure 3: Detection of a Rabi-driven single electron spin analysed by
different post-processing methods. a) Spin $\ket{\uparrow}$-probability
$P(\ket{\uparrow})$ as a function of the driving time $t$. The same
measurement traces are classified by the Bayesian inference filter, and the
neural networks $B$, $C$ and $D$. Solid lines are fits to Eq. 3. b)
$P(\ket{\uparrow})$ from panel a) classified by the neural networks $B$, $C$
and $D$ each subtracted by $P(\ket{\uparrow})$ classified by the Bayesian
inference filter (colors equal to panel a).

We now apply the neural network approach and the Bayesian inference filter to
classify experimentally recorded datasets. In contrast to synthetic traces,
here, we face the fundamental problem that we cannot know a priori whether
traces correspond to a $\ket{\uparrow}$\- or $\ket{\downarrow}$-state. To
benchmark our post-processing methods, we use the detection of Rabi-driven
qubits following the formula

$P(\ket{\uparrow})=\frac{V}{2}\exp(-\frac{t}{T_{R}})\cos(2\pi\nu t)+P_{0},$
(3)

where $t$ is the Rabi driving time, $\nu$ the Rabi-frequency, $T_{R}$ a Rabi-
specific spin decay. $V$ and $P_{0}$ are the visibility and the offset of the
Rabi oscillations, respectively. Thus, we expect a continuous variation of
probabilities to find the $\ket{\uparrow}$ state ($P(\ket{\uparrow})$).
Although $V$ and $P_{0}$ are reduced due to initialisation errors of the qubit
and manipulation errors during Rabi driving (e.g off-resonant driving), it is
reasonable to assume that the classification error of post-processing the
readout traces reduces $V$ as well. Hence, we presume that a larger $V$
corresponds to a lower classification error if the same set of readout data is
post-processed. Before the data is analysed with the methods described above,
it is rescaled and the lower level offset is removed. For rescaling we use the
known height of the peak of $200\,$pA and the lower level is estimated with
the median of the trace. Each data point in Fig. 3a is the average of 250
traces classified to be either in $\ket{\downarrow}$ or $\ket{\uparrow}$
state. After fitting the data by Eq. 3 as shown in Fig. 3a, we find a
visibility of $V=0.664\pm 0.004$ for the analysis using the Bayesian inference
filter. The network $C$, which is trained on real data, has a lower visibility
of $V=0.644\pm 0.004$ and thus does not perform better than the Bayesian
inference filter. The neural network $B$, for which we find $V=0.695\pm
0.004$, slightly outperforms the Bayesian estimation mainly due to the
superior robustness of the neural network to variations in the $I_{SET}$
offset (Fig. 2c). The error in the lower level estimation can occur if a large
portion of the signal is on the higher level, since in this case the median
will estimate a wrong lower level current. The network $D$ reveals a
significant larger visibility of $V=0.760\pm 0.004$ compared to the Bayesian
inference filter and all the other neural networks, thus its classification
error is the lowest. Mainly the classification error of $\ket{\uparrow}$
traces is reduced (Fig. 3b). In contrast to network $B$ and the Bayesian
inference filter, network $D$ is trained on the realistic noise spectrum, but
does not suffer from the labeling problem of the real training data used for
network $C$. Hence, this hybrid training approach outperforms all other
training methods as well as the Bayesian inference filter based on a
reasonably simple noise model.

Apart from the classification error and robustness to variations in a real
experiment, the time $T$ required for the post-processing per trace is an
important performance parameter. It adds up to the measurement time and can
become critical for real-time feedback e.g. required during quantum error
correction. In order to compare $T$ of the different classification traces, we
let the whole post-processing run as efficiently as possible on the same
computer equipped with an Intel i9 9900K processor. The differential equations
for the Bayesian algorithm are solved using a Runge-Kutta method, in a Python
script using numba just-in-time compilation. The neural networks runs with the
tensorflow package. We find that $T$ for all neural networks is $\approx
50\,\mu$s and $\approx 200\,\mu$s for the Bayesian inference filter.
Importantly, $T$ is of the same order of magnitude as the fastest reported
experimental measurement times [22], hence representing a relevant
contribution to classification processes if it runs on a computer. Note that a
peak finder algorithm used in Refs. [14, 4] required approximately 100 times
longer. $T$ of the Bayesian inference filter might be boosted by hardware
encoding of the algorithm e.g. in field programmable gate arrays. This is also
possible with the neural network in dedicated neural network hardware chips.
Hence, both methods present advantages for low-temperature and low-power
control electronics in the future.

##  3 Discussion

In summary, we have shown that the neural network approach is a competitive
alternative to post-processing of single-shot spin detection events by a
Bayesian inference filter. The processing speeds are similar, with a slight
advantage for the neural network. We have benchmarked the performance of the
neural network versus a Bayesian inference filter on synthetic and
experimental data, using different training methods for the network. Since the
Bayesian filter is required to classify experimental traces, training the
network with 106 experimental traces is of no advantage. On the synthetic
data, we find a network trained with synthetic traces to yield a similar error
rate than the Bayesian filter, while it slightly outperforms the latter in
terms of robustness versus variations in experimental parameters such as SNR,
the signal current offset and the tunnel couplings. This advantage is even
more pronounced for the classification of real measurement data: our neural
network trained with a combination of synthetic data and measured noise
outperforms the Bayesian benchmark by 7 %, as seen from the visibility of Rabi
oscillations of the spin qubit. Here, the combination of an absence of
labelling errors in the training data and the setup-specific noise proves to
be particularly advantageous.

Given that that our results should be representative for qubit types with
stochastic readout schemes and that the realtime performance of the neural
network can be further optimized by running it on dedicated hardware, neural
networks can represent an important building block for cryoelectronics
yielding high-fidelity readout in scalable qubit architectures.

##  4 Acknowledgements

We thank Uwe Klemradt and Hendrik Bluhm for valuable discussion. This work has
been funded by the German Research Foundation (DFG) within the projects BO
3140/4-1, 289786932 and the cluster of excellence "Matter and light for
quantum computing" (ML4Q) as well as by the Federal Ministry of Education and
Research under Contract No. FKZ: 13N14778. Project Si-QuBus received funding
from the QuantERA ERA-NET Cofund in Quantum Technologies implemented within
the European Union’s Horizon 2020 Programme.

##  5 Author contributions

T.S. and J.L. developed the network architecture. T.S. did the computation and
analysis of the data assisted by F.S. and A.S. and supported by L.R.S. T.S.
and A.H. measured noise spectra and experimental signal traces on a sample
fabricated by F.S., A.S. and D.B. L.R.S. conceived and supervised the study
and all authors discussed the results. T.S., A.S., D.B. and L.R.S. wrote the
manuscript, which all other authors reviewed.

##  6 Competing interests

The authors declare no competing interests.

##  7 Data availability

The datasets generated during and/or analysed during the current study are
available from the corresponding author on reasonable request.

## References

  * [1]
  * [2] D’Anjou, B. and Coish, W. A.  Optimal post-processing for a generic single-shot qubit readout.  _Phys. Rev. A_ 89, 012313 (2014).
  * [3] Leshno, Moshe _et al._ Multilayer feedforward networks with a nonpolynomial activation function can approximate any function.  _Neural Networks_ 6, 861–867 (1993).
  * [4] Struck, Tom _et al._ Low-frequency spin qubit energy splitting noise in highly.  _npj Quantum Inf._ 6, 40 (2020).
  * [5] Martín Abadi _et al._ TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems.  __ , (2015).
  * [6] Fowler, Austin G _et al._ Surface codes: Towards practical large-scale quantum computation.  _Phys. Rev. A_ 86, 032324 (2012).
  * [7] Robledo, Lucio _et al._ High-fidelity projective read-out of a solid-state spin quantum register.  _Nature_ 477, 574–578 (2011).
  * [8] Elzerman, J. M._et al._ Single-shot read-out of an individual electron spin in a quantum dot.  _Nature_ 430, 431–435 (2004).
  * [9] Petta, J R _et al._ Coherent Manipulation of Coupled Electron Spins in Semiconductor Quantum Dots.  _Science_ 309, 2180–2184 (2005).
  * [10] Maune, B. M._et al._ Coherent singlet-triplet oscillations in a silicon-based double quantum dot.  _Nature_ 481, 344–347 (2012).
  * [11] Morello, Andrea _et al._ Single-shot readout of an electron spin in silicon.  _Nature_ 467, 687–691 (2010).
  * [12] Prance, J. R._et al._ Identifying single electron charge sensor events using wavelet edge detection.  _Nanotechnology_ 26, 215201 (2015).
  * [13] Nowack, K. C._et al._ Single-shot read-out of an individual electron spin in a quantum dot.  _Science_ 333, 1269–1273 (2011).
  * [14] Hollmann, Arne _et al._ Large, Tunable Valley Splitting and Single-Spin Relaxation Mechanisms in a Si/Six Ge1-x Quantum Dot.  _Phys. Rev. Appl._ 430, 034068 (2020).
  * [15] D’Anjou, B._et al._ Maximal adaptive-decision speedups in quantum-state readout.  _Phys. Rev. X_ 6, 011017 (2016).
  * [16] Vandersypen, L. M. K._et al._ Interfacing spin qubits in quantum dots and donors—hot, dense, and coherent.  _Phys. Rev. Appl._ 3, 34 (2017).
  * [17] Yoneda, Jun _et al._ A quantum-dot spin qubit with coherence limited by charge noise and fidelity higher than 99.9%.  _Nature Nanotech._ 13, 102–106 (2018).
  * [18] Simmons, C. B._et al._ Tunable spin loading and T1 of a silicon spin qubit measured by single-shot readout.  _Phys. Rev. Lett._ 106, 156804 (2011).
  * [19] Nair, Vinod _et al._ Rectified Linear Units Improve Restricted Boltzmann Machines Vinod Nair.  _Proceedings of ICML_ 27, 807-814 (2010).
  * [20] Diederik P. Kingma _et al._ Adam: A Method for Stochastic Optimization.  _arXiv_ , 1412.6980 (2017).
  * [21] Li, Ruoyu _et al._ A crossbar network for silicon quantum dot qubits.  _Science Advances_ 4, eaar3960 (2018).
  * [22] Vink, I. T._et al._ Cryogenic amplifier for fast real-time detection of single-electron tunneling.  _Appl. Phys. Lett._ 91, (2007).
  * [23] Lennon, D. T._et al._ Efficiently measuring a quantum device using machine learning.  _npj Quantum Inf_ 5, 79 (2020).
  * [24] Kalantre, S.S._et al._ Machine learning techniques for state recognition and auto-tuning in quantum dots.  _npj Quantum Inf_ 5, 6 (2019).
  * [25] Nguyen, V._et al._ Deep reinforcement learning for efficient measurement of quantum devices.  _arXiv_ , 2009.14825 (2020).
  * [26] Zwolak, Justyna P._et al._ Autotuning of Double-Dot Devices In Situ with Machine Learning.  _Phys. Rev. Applied_ 13, 034075 (2020).


* The key rule which leads to inefficiencies is Var Non-Local.  It requires a chain of function lookups in general, each one going out one lexical scoping level.  All of those lookups had to already happen, so a cache could keep them.  But as mentioned above the hashing on the whole stack will not be efficient.

* So, the proposed solution is to flip the data structure around, and build the cached values into the stack.  Concretely, replace each stack frame $\lab$ with pair $\langle \lab, \lab' / \stk \rangle$ where $\lab' / \stk$ is the value of the calling function.

* Putting these values in the stack frame de facto caches exactly the lookup result we need, and it is always where we need it.

* Using the above data structure, there is only one inefficiency left and that is how non-local variable lookup requires a search up through all the enclosing lexical levels one by one.  There is nothing that can be done about this cost, but programs in practice never get over around 10 or so lexical levels deep however big the code base (after optimizing non-needed Currying away) and so we can view this as a constant.  Note also that a more eager version is possible which pushes down links.  The lazy version now is the “access links” approach in compilers and the eager version is the “display” approach.

* For the above to work the implementation needs to use an immutable data structure for the stack (i.e. an OCaml list).  This is because a mutable stack will write over some frames which may be part of some function value, and to the path to the variable will be lost.  Fortunately immutable stacks are themselves as efficient as mutable ones, it is only GC cost that they pay.

* Can start in middle with no environment; cite the DDSE paper.

One thing Shiwei pointed out however is he did run into the inefficiencies of hashing the lists in DBMC and realized that hash consing could improve the performance significantly.  So, while the above optimization would also speed up DBMC, it is not so clear it will be a big improvement over hash consing.

The primary overhead in practice of this implementation vs a standard functional language with closures is the need for heap allocation of stack frames, plus the added overhead on the garbage collector.  In an optimal implementation there probably would be something done to amortize this cost, e.g. allocate 10 frames at a time in recursive programs.

A version of the rules with the modified stack grammar is found in Figure <ref>.

The modified stack grammar looks non-well-founded at first perhaps, but the stacks referenced in frames are always shorter and so it eventually bottoms out.  Also all of these stacks are shared in a functional implementation so there is no additional time overhead beyond a constant.

[stack frames]
⟨, / ⟩
[call stacks]
[, …]

^/ /
Var Local
() =

_2 / _v / _v
^/ (⟨(_1 _2)^', ⟩) _v / _v

Var Non-Local
() =  _1

^_1 / _1 _v / _v
^/ (⟨(_1 _2)^_2,_1 / _1 ⟩) _v / _v

_1 / ( )^_1/ _1

/ (⟨, _1 / _1 ⟩) _v / _v
(_1 _2)^/ _v / _v

Optimized Operational Semantics Grammar and Rules