#  TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual
Content in Finance

Fengbin Zhu1,2, Wenqiang Lei1, Youcheng Huang 3, Chao Wang2, Shuo Zhang4,
Jiancheng Lv3, Fuli Feng1, Tat-Seng Chua1
1National University of Singapore, 26Estates Pte Ltd, 3Sichuan University,
4Bloomberg
{zhfengbin<EMAIL_ADDRESS><EMAIL_ADDRESS>∗Corresponding
author

###### Abstract

Hybrid data combining both tabular and textual content (e.g., financial
reports) are quite pervasive in the real world. However, Question Answering
(QA) over such hybrid data is largely neglected in existing research. In this
work, we extract samples from real financial reports to build a new large-
scale QA dataset containing both Tabular And Textual data, named TAT-QA, where
numerical reasoning is usually required to infer the answer, such as addition,
subtraction, multiplication, division, counting, comparison/sorting, and their
compositions. We further propose a novel QA model termed TagOp, which is
capable of reasoning over both tables and text. It adopts sequence tagging to
extract relevant cells from the table along with relevant spans from the text
to infer their semantics, and then applies symbolic reasoning over them with a
set of aggregation operators to arrive at the final answer. TagOp achieves
$58.0$% in F1, which is an $11.1$% absolute increase over the previous best
baseline model, according to our experiments on TAT-QA. But this result still
lags far behind the performance of human expert, i.e. $90.8$% in F1. It
demonstrates that our TAT-QA is very challenging and can serve as a benchmark
for training and testing powerful QA models that address hybrid data. Our
dataset is publicly available for non-commercial use at
https://nextplusplus.github.io/TAT-QA/.

##  1 Introduction

Existing QA systems largely focus on only unstructured text Hermann et al.
(2015); Rajpurkar et al. (2016); Dua et al. (2019); Yang et al. (2018); Li et
al. (2020); Nie et al. (2020), structured knowledge base (KB) (Berant et al.,
2013; Yih et al., 2015; Talmor and Berant, 2018), or semi-structured tables
(Pasupat and Liang, 2015; Zhong et al., 2017; Yu et al., 2018; Zhang and
Balog, 2019; Zhang et al., 2020). Though receiving growing interests (Das et
al., 2017; Sun et al., 2019; Chen et al., 2020b, 2021), works on hybrid data
comprising of unstructured text and structured or semi-structured KB/tables
are rare. Recently, Chen et al. (2020b) attempt to simulate a type of hybrid
data through manually linking table cells to Wiki pages via hyperlinks.
However, such connection between table and text is relatively loose.

Figure 1:  An example of TAT-QA. The left dashed line box shows a hybrid
context. The rows with blue background are row header while the column with
grey is column header. The right solid line box shows corresponding question,
answer with its scale, and derivation to arrive at the answer.

In the real world, a more common hybrid data form is, the table (that usually
contains numbers) is more comprehensively linked to text, e.g., semantically
related or complementary. Such hybrid data are very pervasive in various
scenarios like scientific research papers, medical reports, financial reports,
etc. The left box of Figure 1 shows a real example from some financial report,
where there is a table containing row/column header and numbers inside, and
also some paragraphs describing it. We call the hybrid data like this example
hybrid context in QA problems, as it contains both tabular and textual
content, and call the paragraphs associated paragraphs to the table. To
comprehend and answer a question from such hybrid context relies on the close
relation between table and paragraphs, and usually requires numerical
reasoning. For example, one needs to identify “revenue from the external
customers” in the describing text so as to understand the content of the
table. As for _“How much does the commercial cloud revenue account for the
total revenue in 2019?”_ , one needs to get the total revenue in 2019, i.e.
“$125,843$ million” from the table and commercial cloud revenue, i.e. “$38.1$
billion”, from the text to infer the answer.

To stimulate progress of QA research over such hybrid data, we propose a new
dataset, named TAT-QA (Tabular And Textual dataset for Question Answering).
The hybrid contexts in TAT-QA are extracted from real-world financial reports,
each composed of a table with row/col header and numbers, as well as at least
two paragraphs that describe, analyse or complement the content of this table.
Given hybrid contexts, we invite annotators with financial knowledge to
generate questions that are useful in real-world financial analyses and
provide answers accordingly. It is worth mentioning that a large portion of
questions in TAT-QA demand numerical reasoning, for which derivation of the
answer is also labeled to facilitate developing explainable models. In total,
TAT-QA contains $16,552$ questions associated with $2,757$ hybrid contexts
from $182$ reports.

We further propose a novel TagOp model based on TAT-QA. Taking as input the
given question, table and associated paragraphs, TagOp applies sequence
tagging to extract relevant cells from the table and relevant spans from text
as the evidences. Then it applies symbolic reasoning over them with a set of
aggregation operators to arrive at the final answer. Predicting the magnitude
of a number is an important aspect when tackling hybrid data in TAT-QA,
including thousand, million, billion, etc. that are often omitted or shown
only in headers or associated paragraphs of the table for brevity. We term
such magnitude of a number as its scale. Take Question 6 in Figure 1 as an
example: _“How much of the total revenue in 2018 did not come from devices?”_
The numerical value in the answer is obtained by subtraction: “$110,360$ \-
$5,134$”, while the scale “million” is identified from the first-row header of
the table. In TagOp, we incorporate a multi-class classifier for scale
prediction.

We test three types of QA models on TAT-QA, specially addressing tabular,
textual, and hybrid data. Our TagOp achieves $58.0$% in terms of F1, which is
a $11.1$% absolute increase over the best baseline model, according to our
experiments on TAT-QA. It is worth noting that the results still lag far
behind performance of human experts, i.e. $90.8$% in F1. We can see that to
tackle the QA task over the hybrid data as in TAT-QA is challenging and more
effort is demanded. We expect our TAT-QA dataset and TagOp model to serve as a
benchmark and baseline respectively to contribute to the development of QA
models for hybrid data, especially those requiring numerical reasoning.

##  2 Dataset Construction and Analysis

We here explain how we construct TAT-QA and analyze its statistics to better
reveal its proprieties.

###  2.1 Data Collection and Preprocessing

In TAT-QA there are two forms of data: tables and their relevant text, which
are extracted from real-world financial reports.

In particular, we first download about $500$ financial reports released in the
past two years from an online website111https://www.annualreports.com/. We
adopt the table detection model in (Li et al., 2019) to detect tables in these
reports, and apply Apache PDFBox222https://pdfbox.apache.org/ library to
extract the table contents to be processed with our annotation tool. We only
keep those tables with $3$ $\sim$ $30$ rows and $3$ $\sim$ $6$ columns.
Finally, about $20,000$ candidate tables are retained, which have no standard
schema and lots of numbers inside. The corresponding reports with selected
tables are also kept. Note that these candidate tables may still contain
errors, such as containing too few or many rows/cols, mis-detected numbers,
which will be manually picked out and deleted or fixed during the annotation
process.

###  2.2 Dataset Annotation

The annotation is done with our self-developed tool. All the annotators are
with financial background knowledge.

Adding Relevant Paragraphs to Tables We build valid hybrid contexts based on
the original reports kept in the previous step. A valid hybrid context in TAT-
QA consists of a table and at least two associated paragraphs surrounding it,
as shown in the left box in Figure 1. To associate enough relevant paragraphs
to a candidate table, the annotators first check whether there are $\geq 2$
paragraphs around this table, and then check whether they are relevant,
meaning the paragraphs should be describing, analysing or complementing the
content in the table. If yes, then all the surrounding paragraphs will be
associated to this table. Otherwise, the table will be skipped
(discarded).333About two thirds of candidate tables were discarded.

Question-Answer Pair Creation Based on the valid hybrid contexts, the
annotators are then asked to create question-answer pairs, where the questions
need to be useful in real-world financial analyses. In addition, we encourage
them to create questions that can be answered by people without much finance
knowledge and use common words instead of the same words appeared in the
hybrid context (Rajpurkar et al., 2016). Given one hybrid context, at least
$6$ questions are generated, including extracted and calculated questions. For
extracted questions, the answers can be a single span or multiple spans from
either the table or the associated paragraphs. For calculated questions,
numerical reasoning is required to produce the answers, including addition,
subtraction, multiplication, division, counting, comparison/sorting and their
compositions. Furthermore, we particularly ask the annotators to annotate the
right scale for the numerical answer when necessary.

Answer Type and Derivation Annotation The answers in TAT-QA have three types:
a single span or multiple spans extracted from the table or text, as well as a
generated answer (usually obtained through numerical reasoning). The
annotators will also need to label its type after they generate an answer. For
generated answers, the corresponding derivations are provided to facilitate
the development of explainable QA models, including two types: 1) an
arithmetic expression, like _( $11,386$ \- $10,353$)/$10,353$)_ for Question 8
in Figure 1, which can be executed to arrive at the final answer; and 2) a set
of items separated with _“##”_ , like _“device ## enterprise services”_ for
Question 4 in Figure 1 where the count of items equals the answer. We further
divide questions in TAT-QA into four kinds: Span, Spans, Arithmetic and
Counting, where the latter two kinds correspond to the above two types of
deviations, to help us better investigate the numerical reasoning capability
of a QA model.

Answer Source Annotation For each answer, annotators are required to specify
the source(s) it is derived from, including Table, Text, and Table-text
(both). This is to force the model to learn to aggregate information from
hybrid sources to infer the answer, thus lift its generalizability. For
example, to answer Question 7 in Figure 1: _“How much does the commercial
cloud revenue account for the total revenue in 2019?”_ , we can observe from
the derivation that “$125,843$ million” comes from the table while “$38.1$
billion” from text.

###  2.3 Quality Control

To ensure the quality of annotation in TAT-QA, we apply strict quality control
procedures.

Competent Annotators To build TAT-QA, financial domain knowledge is necessary.
Hence, we employ about $30$ university students majored in finance or similar
disciplines as annotators. We give all candidate annotators a minor test and
only those with $95\%$ correct rate are hired. Before starting the annotation
work, we give a training session to the annotators to help them fully
understand our annotation requirements and also learn the usage of our
annotation system.

Two-round Validation For each annotation, we ask two different verifiers to
perform a two-round validation after it is submitted, including checking and
approval, to ensure its quality. We have five verifiers in total, including
two annotators who have good performance on this project and three graduate
students with financial background. In the checking phase, a verifier checks
the submitted annotation and asks the annotator to fix it if any mistake or
problem is found. In the approval phase, a different verifier inspects the
annotation again that has been confirmed by the first verifier, and then
approves it if no problem is found.

###  2.4 Dataset Analysis

Averagely, an annotator can label two hybrid contexts per hour; the whole
annotation work lasts about three months. Finally, we attain a total of
$2,757$ hybrid contexts and $16,552$ corresponding question-answer pairs from
$182$ financial reports. The hybrid contexts are randomly split into training
set ($80\%$), development set ($10\%$) and test set ($10\%$); hence all
questions about a particular hybrid context belong to only one of the splits.
We show the basic statistics of each split in Table 1, and the question
distribution regarding answer source and answer type in Table 2. In Figure 1,
we give an example from TAT-QA, demonstrating the various reasoning types and
percentage of each reasoning type over the whole dataset.

Statistic | Train | Dev | Test
---|---|---|---
# of hybrid contexts | 2,201 | 278 | 278
# of questions | 13,215 | 1,668 | 1,669
Avg. rows / table | 9.4 | 9.7 | 9.3
Avg. cols / table | 4.0 | 3.9 | 4.0
Avg. paragraphs / table | 4.8 | 4.9 | 4.6
Avg. paragraph len [words] | 43.6 | 44.8 | 42.6
Avg. question len [words] | 12.5 | 12.4 | 12.4
Avg. answer len [words] | 4.1 | 4.1 | 4.3
Table 1: Basic statistics of each split in TAT-QA |  Table |  Text |  Table-text |  Total
---|---|---|---|---
Span |  1,801 |  3,496 |  1,842 |  7,139
Spans |  777 |  258 |  1,037 |  2,072
Counting |  106 |  5 |  266 |  377
Arithmetic |  4,747 |  143 |  2,074 |  6,964
Total |  7,431 |  3,902 |  5,219 |  16,552
Table 2: Question distribution regarding different answer types and sources in
TAT-QA

##  3 TagOp Model

We introduce a novel QA model, named TagOp, which first applies sequence
TAGging to extract relevant cells from the table and text spans from the
paragraphs inspired by (Li et al., 2016; Sun et al., 2016; Segal et al.,
2020). This step is analogy to slot filling or schema linking, whose
effectiveness has been demonstrated in dialogue systems Lei et al. (2018); Jin
et al. (2018) and semantic parsing Lei et al. (2020). And then TagOp performs
symbolic reasoning over them with a set of aggregation OPerators to arrive at
the final answer. The overall architecture is illustrated in Figure 2.

Figure 2:  Illustration of the architecture of proposed TagOp model. Given
Question 6 in Figure 1 where the hybrid context is also shown, TagOp supports
$10$ operators, which are described in Section 3.2.

###  3.1 Sequence Tagging

Given a question, TagOp first extracts supporting evidences from its hybrid
context (i.e. the table and associated paragraphs) via sequence tagging with
the Inside–Outside tagging (IO) approach (Ramshaw and Marcus, 1995). In
particular, it assigns each token either I or O label and takes those tagged
with I as the supporting evidences for producing the answer. The given
question, flattened table by row (Herzig et al., 2020) and associated
paragraphs are input sequentially to a transformer-based encoder like RoBERTa
(Liu et al., 2019), as shown in the bottom part of Figure 2, to obtain
corresponding representations. Each sub-token is tagged independently, and the
corresponding cell in the table or word in the paragraph would be regarded as
positive if any of its sub-tokens is tagged with I. For the paragraphs, the
continuous words that are predicted as positive are combined as a span. During
testing, all positive cells and spans are taken as the supporting evidences.
Formally, for each sub-token $t$ in the paragraph, the probability of the tag
is computed as

$\displaystyle\mathbf{p}^{\textrm{tag}}_{\textrm{t}}$
$\displaystyle=\textrm{softmax}(\textrm{FFN}({h_{t}}))$ (1)

where FFN is a two-layer feed-forward network with GELU (Hendrycks and Gimpel,
2016) activation and $h_{t}$ is the representation of sub-token $t$.

###  3.2 Aggregation Operator

Next, we perform symbolic reasoning over obtained evidences to infer the final
answer, for which we apply an aggregation operator. In our TagOp, there are
ten types of aggregation operators. For each input question, an operator
classifier is applied to decide which operator the evidences would go through;
for some operators sensitive to the order of input numbers, an auxiliary
number order classifier is used. The aggregation operators are explained as
below, covering most reasoning types as listed in Figure 1.

  * •

Span-in-text: To select the span with the highest probability from predicted
candidate spans. The probability of a span is the highest probability of all
its sub-tokens tagged I.

  * •

Cell-in-table: To select the cell with the highest probability from predicted
candidate cells. The probability of a cell is the highest probability of all
its sub-tokens tagged I.

  * •

Spans: To select all the predicted cell and span candidates;

  * •

Sum: To sum all predicted cells and spans purely consisting of numbers;

  * •

Count: To count all predicted cells and spans;

  * •

Average: To average over all the predicted cells and spans purely consisting
of numbers;

  * •

Multiplication: To multiply all predicted cells and spans purely consisting of
numbers;

  * •

Division: To first rank all the predicted cells and spans purely consisting of
numbers based on their probabilities, and then apply division calculation to
top-two;

  * •

Difference: To first rank all predicted numerical cells and spans based on
their probabilities, and then apply subtraction calculation to top-two.

  * •

Change ratio: For the top-two values after ranking all predicted numerical
cells and spans based on their probabilities, compute the change ratio of the
first value compared to the second one.

Operator Classifier To predict the right aggregation operator, a multi-class
classifier is developed. In particular, we take the vector of [CLS] as input
to compute the probability:

$\displaystyle\mathbf{p}^{\textrm{op}}$
$\displaystyle=\textrm{softmax}(\textrm{FFN}(\texttt{[CLS]})$ (2)

where FFN denotes a two-layer feed-forward network with the GELU activation.

Number Order Classifier For operators of Difference, Division and Change
ratio, the order of the input two numbers matters in the final result. Hence
we additionally append a number order classifier after them, formulated as

$\displaystyle\mathbf{p}^{\textrm{order}}$
$\displaystyle=\textrm{softmax}(\textrm{FFN}(\textrm{avg}(\texttt{$h_{t1}$},\texttt{$h_{t2}$}))$
(3)

where FFN denotes a two-layer feed-forward network with the GELU activation,
$h_{t1}$, $h_{t2}$ are representations of the top two tokens according to
probability, and “avg” means average. For a token, its probability is the
highest probability of all its sub-tokens tagged I, and its representation is
the average over those of its sub-tokens.

###  3.3 Scale Prediction

Till now we have attained the string or numerical value to be contained in the
final answer. However, a right prediction of a numerical answer should not
only include the right number but also the correct scale. This is a unique
challenge over TAT-QA and very pervasive in the context of finance. We develop
a multi-class classifier to predict the scale. Generally, the scale in TAT-QA
may be None, Thousand, Million, Billion, and Percent. Taking as input the
concatenated representation of [CLS], the table and paragraphs sequentially,
the multi-class classifier computes the probability of the scale as

$\displaystyle\mathbf{p}^{\textrm{scale}}$
$\displaystyle=\textrm{softmax}(\textrm{FFN}([\texttt{[CLS]};h_{tab};h_{p}])$
(4)

where $h_{tab}$ and $h_{p}$ are the representations of the table and the
paragraphs respectively, which are obtained by applying an average pooling
over the representations of their corresponding tokens,“;” denotes
concatenation, and FFN denotes a two-layer feed-forward network with the GELU
activation.

After obtaining the scale, the numerical or string prediction is multiplied or
concatenated with the corresponding scale as the final prediction to compare
with the ground-truth answer respectively.

###  3.4 Training

To optimize TagOp, the overall loss is the sum of the loss of the above four
classification tasks:

$\begin{split}\textrm{$\mathcal{L}$}=~{}&\textrm{NLL}(\mathbf{log(P}^{\textrm{tag}}),\mathbf{G}^{\textrm{tag}})~{}\textrm{+}\\\
&\textrm{NLL}(\mathbf{log(P}^{\textrm{op}}),\mathbf{G}^{\textrm{op}})~{}\textrm{+}\\\
&\textrm{NLL}(\mathbf{log(P}^{\textrm{scale}}),\mathbf{G}^{\textrm{scale}})~{}\textrm{+}\\\
&\textrm{NLL}(\mathbf{log(P}^{\textrm{order}}),\mathbf{G}^{\textrm{order}})\\\
\end{split}$ (5)

where NLL(·) is the negative log-likelihood loss, $\mathbf{G}^{\textrm{tag}}$
and $\mathbf{G}^{\textrm{op}}$ come from the supporting evidences which are
extracted from the annotated answer and derivation. We locate the evidence in
the table first if it is among the answer sources, and otherwise in its
associated paragraphs. Note we only keep the first found if an evidence
appears multiple times in the hybrid context. $\mathbf{G}^{\textrm{scale}}$
uses the annotated scale of the answer; $\mathbf{G}^{\textrm{order}}$ is
needed when the ground-truth operator is one of Difference, Division and
Change ratio, which is obtained by mapping the two operands extracted from
their corresponding ground-truth deviation in the input sequence. If their
order is the same as that in the input sequence,
$\mathbf{G}^{\textrm{order}}=0$; otherwise it is $1$.

##  4 Experiments and Results

###  4.1 Baselines

Textual QA Models We adopt two reading comprehension (RC) models as baselines
over textual data: BERT-RC (Devlin et al., 2018), which is a SQuAD-style RC
model; and NumNet+ V2 444https://github.com/llamazing/numnet_plus (Ran et al.,
2019), which achieves promising performance on DROP that requires numerical
reasoning over textual data. We adapt them to our TAT-QA as follows. We
convert the table to a sequence by row, also as input to the models, followed
by tokens from the paragraphs. Besides, we add a multi-class classifier,
exactly as in our TagOp, to enable the two models to predict the scale based
on Eq. (4).

Tabular QA Model We employ TaPas for WikiTableQuestion (WTQ) (Herzig et al.,
2020) as a baseline over tabular data. TaPas is pretrained over large-scale
tables and associated text from Wikipedia jointly for table parsing. To train
it, we heuristically locate the evidence in the table with the annotated
answer or derivation, which is the first matched one if a same value appears
multiple times. In addition, we remove the “numerical rank id” feature in its
embedding layer, which ranks all values per numerical column in the table but
does not make sense in TAT-QA. Similar to above textual QA setting, we add an
additional multi-class classifier to predict the scale as in Eq. (4).

Hybrid QA Model We adopt HyBrider (Chen et al., 2020b) as our baseline over
hybrid data, which tackles tabular and textual data from Wikipedia. We use the
code released in the original paper555https://github.com/wenhuchen/HybridQA,
but adapt it to TAT-QA. Concretely, each cell in the table of TAT-QA is
regarded as “linked” with associated paragraphs of this table, like hyperlinks
in the original paper, and we only use its cell matching mechanism to link the
question with the table cells in its linking stage. The selected cells and
paragraphs are fed into the RC model in the last stage to infer the answer.
For ease of training on TAT-QA, we also omit the prediction of the scale, i.e.
we regard the predicted scale by this model as always correct.

###  4.2 Evaluation Metrics

We adopt the popular Exact Match (EM) and numeracy-focused F1 score (Dua et
al., 2019) to measure model performance on TAT-QA. However, the original
implementation of both metrics is insensitive to whether a value is positive
or negative in the answer as the minus is omitted in evaluation. Since this
issue is crucial for correctly interpreting numerical values, especially in
the finance domain, we keep the plus-minus of a value when calculating them.
In addition, the numeracy-focused F1 score is set to $0$ unless the predicted
number multiplied by predicted scale equals exactly the ground truth.

###  4.3 Results and Analysis

In the following, we report our experimental results on dev and test sets of
TAT-QA.

Comparison with Baselines We first compare our TagOp with three types of
previous QA models as described in Section 4.1. The results are summarized in
Table 3. It can be seen that our model is always superior to other baselines
in terms of both metrics, with very large margins over the second best, namely
50.1/58.0 vs. 37.0/46.9 in EM/F1 on test set of TAT-QA respectively. This well
reveals the effectiveness of our method that reasons over both tabular and
textual data involving lots of numerical contents. For two textual QA
baselines, NumNet+ V2 performs better than BERT-RC, which is possibly
attributed to the stronger capability of numerical reasoning of the latter,
but it is still worse than our method. The tabular QA baseline Tapas for WTQ
is trained with only tabular data in TAT-QA, showing very limited capability
to process hybrid data, as can be seen from its performance. The HyBrider is
the worst among all baseline models, because it is designed for HybridQA (Chen
et al., 2020b) which does not focus on the comprehensive interdependence of
table and paragraphs, nor numerical reasoning.

However, all the models perform significantly worse than human
performance666The human performance is evaluated by asking annotators to
answer 50 randomly sampled hybrid contexts (containing 301 questions) from our
test set. Note the human performance is still not 100% correct because our
questions require relatively heavy cognitive load like tedious numerical
calculations. Comparing human performance of F1 in SQUAD (Rajpurkar et al.,
2016) ($86.8$%) and DROP (Dua et al., 2019)) ($96.4$%), the score ($90.8$%) in
our dataset already indicates a good quality and annotation consistency in our
dataset., indicating TAT-QA is challenging to current QA models and more
efforts on hybrid QA are demanded.

Answer Type and Source Analysis Furthermore, we analyze detailed performance
of TagOp w.r.t answer type and source in Table 4. It can be seen that TagOp
performs better on the questions whose answers rely on the tables compared to
those from the text. This is probably because table cells have clearer
boundaries than text spans to the model, thus it is relatively easy for the
model to extract supporting evidences from the tables leveraging sequence
tagging techniques. In addition, TagOp performs relatively worse on arithmetic
questions compared with other types. This may be because the calculations for
arithmetic questions are diverse and harder than other types, indicating the
challenge of TAT-QA, especially for the requirement of numerical reasoning.

Method | Dev | Test
---|---|---
EM | F1 | EM | F1
Human | - | - | 84.1 | 90.8
Textual QA
BERT-RC | 9.5 | 17.9 | 9.1 | 18.7
NumNet+ V2 | 38.1 | 48.3 | 37.0 | 46.9
Tabular QA
TaPas for WTQ | 18.9 | 26.5 | 16.6 | 22.8
Hybrid QA
HyBrider | 6.6 | 8.3 | 6.3 | 7.5
TagOp | 55.2 | 62.7 | 50.1 | 58.0
Table 3: Performance of different models on dev and test set of TAT-QA. Best results are marked in bold. |  Table |  Text |  Table-text |  Total
---|---|---|---|---
|  EM/F1 |  EM/F1 |  EM/F1 |  EM/F1
Span |  56.5/57.8 |  45.2/70.6 |  68.2/71.7 |  54.1/67.9
Spans |  66.3/77.0 |  19.0/59.1 |  63.2/76.9 |  60.0/75.1
Counting |  63.6/63.6 |  -/- |  62.1/62.1 |  62.5/62.5
Arithmetic |  41.1/41.1 |  27.3/27.3 |  46.5/46.5 |  42.5/42.5
Total |  47.8/49.3 |  43.3/68.7 |  58.3/62.2 |  50.1/58.0
Table 4: Detailed experimental results of TagOp w.r.t. answer types and
sources on test set.

Results of TagOp with Different Operators We here investigate the
contributions of the ten aggregation operators to the final performance of
TagOp. As shown in Table 5, we devise nine variants of the full model of
TagOp; based on the variant of TagOp with only one operator (e.g. Span-in-
text), for each of other variants, we add one more operator back. As can be
seen from the table, all added operators can benefit the model performance.
Furthermore, we find that some operators like _Span-in-text_ , _Cell-in-table_
, _Difference_ and _Average_ make more contributions than others. In
comparison, _Sum_ and _Multiplication_ bring little gain or even decline.
After analysis, we find this is because the instances of _Sum_ or
_Multiplication_ are minor in our test set, which are easily influenced by
randomness.

Model | Dev | Test
---|---|---
EM | F1 | EM | F1
\+ Span-in-text | 13.4 | 20.5 | 14.1 | 21.8
\+ Cell-in-table | 25.4 | 36.0 | 24.1 | 35.3
\+ Spans | 33.6 | 41.3 | 31.3 | 39.4
\+ Sum | 33.8 | 41.3 | 31.2 | 39.1
\+ Count | 35.9 | 43.5 | 32.7 | 40.6
\+ Average | 43.3 | 50.6 | 38.2 | 45.9
\+ Multiplication | 44.2 | 51.4 | 37.9 | 46.0
\+ Division | 45.0 | 52.5 | 39.2 | 47.5
\+ Difference | 51.4 | 58.7 | 45.1 | 53.3
\+ Change ratio (Full) | 55.2 | 62.7 | 50.1 | 58.0
Table 5: Performance with different aggregation operators of TagOp model.

Error Analysis We further investigate our TagOp by analysing error cases. We
randomly sample $100$ error instances from the test set, and classify them
into five categories as shown in Table 6, each with an example: (1) _Wrong
Evidence_ ($55$%), meaning the model obtained wrong supporting evidence from
the hybrid context; (2) _Missing Evidence_ ($29$%), meaning the model failed
to extract the supporting evidence for the answer; (3) _Wrong Calculation_
($9$%), meaning the model failed to compute the answer with the correct
supporting evidence; (4) _Unsupported Calculation_ ($4$%), meaning the ten
operators defined cannot support this calculation; (5) _Scale Error_ ($3$%),
meaning the model failed to predict the scale of the numerical value in an
answer.

We can then observe about $84$% error is caused by the failure to extract the
supporting evidence from the table and paragraphs given a question. This
demonstrates more efforts are needed to strengthen the model’s capability of
precisely aggregating information from hybrid contexts.

After instance-level analysis, we find another interesting error resource is
the dependence on domain knowledge. While we encourage annotators to create
questions answerable by humans without much finance knowledge, we still find
domain knowledge is required for some questions. For example, given the
question _“What is the gross profit margin of the company in 2015?”_ , the
model needs to extract the gross profit and revenue from the hybrid context
and compute the answer according to the finance formula _(“gross profit margin
= gross profit / revenue”)_. How to integrate such finance knowledge into QA
models to answer questions in TAT-QA still needs further exploration.

Wrong Evidence (55%) |  Q: How much did the level 2 OFA change by from 2018 year end to 2019 year end?
---|---
|  G: 375 \- 2,032
|  P: 1,941 \- 2,032
Missing Evidence (29%) |  Q: How many years did adjusted EBITDA exceed $4,000 million?
|  G: _count_(2017, 2018, 2019)
|  P: _count_(2017, 2018)
Wrong Calculation (9%) |  Q: What is the change in the % of pre-tax loss from 2018 to 2019?
|  G: 39% - 20%
|  P: 20% - 39%
Unsupported Calculation (4%) |  Q: What is the proportion of investor relations and consultants over the total operating expense in 2019?
|  G: (105,639 + 245,386) /19,133,139
|  P: 245,386 / 19,133,139
Scale Error (3%) |  Q: What is the closing price in March, 2020?
|  G: 0.22
|  P: 0.22 million
Table 6: Examples of error and corresponding percentage. Q, G, P denote
question, ground truth, prediction.

##  5 Related Work

QA Datasets Currently, there are many datasets for QA tasks, focusing on text,
or KB/table. Textual ones include CNN/Daily Mail Hermann et al. (2015), SQuAD
Rajpurkar et al. (2016), etc. Recently deep reasoning over textual data has
gained increasing attention Zhu et al. (2021), e.g. multi-hop reasoning (Yang
et al., 2018; Welbl et al., 2018). DROP (Dua et al., 2019) is built to develop
numerical reasoning capability of QA models, which in this sense is similar to
TAT-QA, but only focuses on textual data. KB/Tabular QA aims to automatically
answer questions via well-structured KB (Berant et al., 2013; Talmor and
Berant, 2018; Yih et al., 2015) or semi-structured tables (Pasupat and Liang,
2015; Zhong et al., 2017; Yu et al., 2018). Comparably, QA over hybrid data
receives limited efforts, focusing on mixture of KB/tables and text. HybridQA
(Chen et al., 2020b) is one existing hybrid dataset for QA tasks, where the
context is a table connected with Wiki pages via hyperlinks.

Numerical Reasoning Numerical reasoning is key to many NLP tasks like question
answering (Dua et al., 2019; Ran et al., 2019; Andor et al., 2019; Chen et
al., 2020a; Pasupat and Liang, 2015; Herzig et al., 2020; Yin et al., 2020;
Zhang and Balog, 2020) and arithmetic word problems (Kushman et al., 2014;
Mitra and Baral, 2016; Huang et al., 2017; Ling et al., 2017). To our best
knowledge, no prior work attempts to develop models able to perform numerical
reasoning over hybrid contexts.

##  6 Conclusion

We propose a new challenging QA dataset TAT-QA, comprising real-word hybrid
contexts where the table contains numbers and has comprehensive dependencies
on text in finance domain. To answer questions in TAT-QA, the close relation
between table and paragraphs and numerical reasoning are required. We also
propose a baseline model TagOp based on TAT-QA, aggregating information from
hybrid context and performing numerical reasoning over it with pre-defined
operators to compute the final answer. Experiments show TAT-QA dataset is very
challenging and more effort is demanded for tackling QA tasks over hybrid
data. We expect our TAT-QA dataset and TagOp model would serve as a benchmark
and baseline respectively to help build more advanced QA models, facilitating
the development of QA technologies to address more complex and realistic
hybrid data, especially those requiring numerical reasoning.

## Acknowledgments

The authors gratefully acknowledge Zhuyun Dai for giving valuable suggestions
on this study, Xinnan Zhang for developing the data annotation tool, and Tong
Ye and Ming Wei Chan for their work on checking the annotation quality. Our
thanks also go to all the anonymous reviewers for their positive feedback.
This research is supported by the NExT Research Centre, Singapore.

## References

  * Andor et al. (2019) Daniel Andor, Luheng He, Kenton Lee, and Emily Pitler. 2019.  Giving BERT a calculator: Finding operations and arguments with reading comprehension.  In _EMNLP-IJCNLP_ , pages 5947–5952. ACL.
  * Berant et al. (2013) Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013.  Semantic parsing on Freebase from question-answer pairs.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pages 1533–1544, Seattle, Washington, USA. ACL.
  * Chen et al. (2020a) Kunlong Chen, Weidi Xu, Xingyi Cheng, Zou Xiaochuan, Yuyu Zhang, Le Song, Taifeng Wang, Yuan Qi, and Wei Chu. 2020a.  Question directed graph attention network for numerical reasoning over text.  In _EMNLP-IJCNLP_ , pages 6759–6768. ACL.
  * Chen et al. (2021) Wenhu Chen, Ming-Wei Chang, Eva Schlinger, William Yang Wang, and William W. Cohen. 2021.  Open question answering over tables and text.  In _ICLR_.
  * Chen et al. (2020b) Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Yang Wang. 2020b.  HybridQA: A dataset of multi-hop question answering over tabular and textual data.  In _Findings of the Association for Computational Linguistics: EMNLP_ , pages 1026–1036. ACL.
  * Das et al. (2017) Rajarshi Das, Manzil Zaheer, Siva Reddy, and Andrew McCallum. 2017.  Question answering on knowledge bases and text using universal schema and memory networks.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics_ , pages 358–365. ACL.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: pre-training of deep bidirectional transformers for language understanding.  _CoRR_ , abs/1810.04805.
  * Dua et al. (2019) Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019.  DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs.  In _Proc. of NAACL_.
  * Hendrycks and Gimpel (2016) Dan Hendrycks and Kevin Gimpel. 2016.  Bridging nonlinearities and stochastic regularizers with gaussian error linear units.  _CoRR_ , abs/1606.08415.
  * Hermann et al. (2015) Karl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015.  Teaching machines to read and comprehend.  In _Proceedings of the 28th International Conference on Neural Information Processing Systems_ , pages 1693–1701. MIT Press.
  * Herzig et al. (2020) Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Müller, Francesco Piccinno, and Julian Eisenschlos. 2020.  TaPas: Weakly supervised table parsing via pre-training.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4320–4333. ACL.
  * Huang et al. (2017) Danqing Huang, Shuming Shi, Chin-Yew Lin, and Jian Yin. 2017.  Learning fine-grained expressions to solve math word problems.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 805–814. ACL.
  * Jin et al. (2018) Xisen Jin, Wenqiang Lei, Zhaochun Ren, Hongshen Chen, Shangsong Liang, Yihong Zhao, and Dawei Yin. 2018.  Explicit state tracking with semi-supervisionfor neural dialogue generation.  In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_ , pages 1403–1412.
  * Kushman et al. (2014) Nate Kushman, Yoav Artzi, Luke Zettlemoyer, and Regina Barzilay. 2014.  Learning to automatically solve algebra word problems.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics_ , pages 271–281. ACL.
  * Lei et al. (2018) Wenqiang Lei, Xisen Jin, Min-Yen Kan, Zhaochun Ren, Xiangnan He, and Dawei Yin. 2018\.  Sequicity: Simplifying task-oriented dialogue systems with single sequence-to-sequence architectures.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1437–1447.
  * Lei et al. (2020) Wenqiang Lei, Weixin Wang, Zhixin Ma, Tian Gan, Wei Lu, Min-Yen Kan, and Tat-Seng Chua. 2020.  Re-examining the role of schema linking in text-to-sql.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6943–6954.
  * Li et al. (2020) Jiaqi Li, Ming Liu, Min-Yen Kan, Zihao Zheng, Zekun Wang, Wenqiang Lei, Ting Liu, and Bing Qin. 2020.  Molweni: A challenge multiparty dialogues-based machine reading comprehension dataset with discourse structure.  _CoRR_ , abs/2004.05080.
  * Li et al. (2019) Minghao Li, Lei Cui, Shaohan Huang, Furu Wei, Ming Zhou, and Zhoujun Li. 2019.  Tablebank: A benchmark dataset for table detection and recognition.
  * Li et al. (2016) Peng Li, Wei Li, Zhengyan He, Xuguang Wang, Ying Cao, Jie Zhou, and Wei Xu. 2016\.  Dataset and neural recurrent sequence labeling model for open-domain factoid question answering.
  * Ling et al. (2017) Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. 2017.  Program induction by rationale generation: Learning to solve and explain algebraic word problems.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics_ , pages 158–167. ACL.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized BERT pretraining approach.  _CoRR_ , abs/1907.11692.
  * Mitra and Baral (2016) Arindam Mitra and Chitta Baral. 2016.  Learning to use formulas to solve simple arithmetic problems.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics_ , pages 2144–2153. ACL.
  * Nie et al. (2020) Liqiang Nie, Yongqi Li, Fuli Feng, Xuemeng Song, Meng Wang, and Yinglong Wang. 2020\.  Large-scale question tagging via joint question-topic embedding learning.  _ACM Transactions on Information Systems (TOIS)_ , 38(2):1–23.
  * Pasupat and Liang (2015) Panupong Pasupat and Percy Liang. 2015.  Compositional semantic parsing on semi-structured tables.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing_ , pages 1470–1480. ACL.
  * Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016.  SQuAD: 100,000+ questions for machine comprehension of text.  In _EMNLP-IJCNLP_ , pages 2383–2392. ACL.
  * Ramshaw and Marcus (1995) Lance Ramshaw and Mitch Marcus. 1995.  Text chunking using transformation-based learning.  In _Third Workshop on Very Large Corpora_.
  * Ran et al. (2019) Qiu Ran, Yankai Lin, Peng Li, Jie Zhou, and Zhiyuan Liu. 2019.  NumNet: Machine reading comprehension with numerical reasoning.  In _EMNLP-IJCNLP_ , pages 2474–2484.
  * Segal et al. (2020) Elad Segal, Avia Efrat, Mor Shoham, Amir Globerson, and Jonathan Berant. 2020.  A simple and effective model for answering multi-span questions.  In _EMNLP-IJCNLP_ , pages 3074–3080. ACL.
  * Sun et al. (2019) Haitian Sun, Tania Bedrax-Weiss, and William Cohen. 2019.  PullNet: Open domain question answering with iterative retrieval on knowledge bases and text.  In _EMNLP-IJCNLP_ , pages 2380–2390. ACL.
  * Sun et al. (2016) Huan Sun, Hao Ma, Xiaodong He, Wen-tau Yih, Yu Su, and Xifeng Yan. 2016.  Table cell search for question answering.  In _Proceedings of the 25th International Conference on World Wide Web_ , WWW ’16, page 771–782. International World Wide Web Conferences Steering Committee.
  * Talmor and Berant (2018) Alon Talmor and Jonathan Berant. 2018.  The web as a knowledge-base for answering complex questions.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 641–651.
  * Welbl et al. (2018) Johannes Welbl, Pontus Stenetorp, and Sebastian Riedel. 2018.  Constructing datasets for multi-hop reading comprehension across documents.  _Transactions of the Association for Computational Linguistics_ , pages 287–302.
  * Yang et al. (2018) Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018.  HotpotQA: A dataset for diverse, explainable multi-hop question answering.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_. ACL.
  * Yih et al. (2015) Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. 2015.  Semantic parsing via staged query graph generation: Question answering with knowledge base.  In _Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing_ , pages 1321–1331.
  * Yin et al. (2020) Pengcheng Yin, Graham Neubig, Wen-tau Yih, and Sebastian Riedel. 2020.  TaBERT: Pretraining for joint understanding of textual and tabular data.  In _ACL_ , pages 8413–8426. ACL.
  * Yu et al. (2018) Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. 2018.  Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3911–3921.
  * Zhang and Balog (2019) Shuo Zhang and Krisztian Balog. 2019.  Auto-completion for data cells in relational tables.  In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_ , CIKM ’19, pages 761–770.
  * Zhang and Balog (2020) Shuo Zhang and Krisztian Balog. 2020.  Web table extraction, retrieval, and augmentation: A survey.  _ACM Trans. Intell. Syst. Technol._ , 11(2):13:1–13:35.
  * Zhang et al. (2020) Shuo Zhang, Zhuyun Dai, Krisztian Balog, and Jamie Callan. 2020.  Summarizing and exploring tabular data in conversational search.  SIGIR ’20, pages 1537–1540.
  * Zhong et al. (2017) Victor Zhong, Caiming Xiong, and Richard Socher. 2017.  Seq2sql: Generating structured queries from natural language using reinforcement learning.  _CoRR_ , abs/1709.00103.
  * Zhu et al. (2021) Fengbin Zhu, Wenqiang Lei, Chao Wang, Jianming Zheng, Soujanya Poria, and Tat-Seng Chua. 2021.  Retrieving and reading: A comprehensive survey on open-domain question answering.  _CoRR_ , abs/2101.00774.

##  Appendix A Appendix

###  A.1 Table Analysis

To maintain the semi-structured nature of financial tables, we almost keep the
same table structure in TAT-QA as that in the original financial reports. We
sample $100$ hybrid contexts from the training set to conduct a manual
evaluation to assess the complexity of the table structures. Specifically, we
analyze the distribution w.r.t. the number of row headers, as shown in Table
7. It can be seen that around $79\%$ of the tables have two or more row-
headers, indicating large difficulty in interpreting financial tables. In
addition, we have also found that all sampled tables all have one column
header.

# of Row Header | Proportion (%)
---|---
1 | 21
2 | 68
3 | 9
more than 3 | 2
Table 7: Distribution of no. of row-headers in TAT-QA.

###  A.2 Operator Classifier

We present the proportion of questions that should go through each aggregation
operator (ground truth), as well as the performance of our operator classifier
on dev and test set in Table 8.

Operator | Dev | Test
---|---|---
| % | Acc | % | Acc
Span-in-text | 20.9 | 92.3 | 21.3 | 91.6
Cell-in-table | 21.1 | 91.2 | 21.6 | 86.7
Spans | 13.0 | 96.8 | 12.6 | 93.8
Sum | 3.4 | 86.0 | 2.5 | 76.2
Count | 1.9 | 93.8 | 2.4 | 100.0
Average | 8.5 | 100.0 | 5.9 | 100.0
Multiplication | 0.2 | 33.3 | 0.1 | 0.0
Division | 1.0 | 76.5 | 1.0 | 87.5
Difference | 14.1 | 96.6 | 15.9 | 96.6
Change ratio | 9.3 | 96.1 | 10.2 | 95.3
Other | 6.6 | 0.0 | 6.6 | 0.0
Table 8: Ground truth proportion of questions that should be fed to different
operators and prediction accuracy by operator classifier of TagOp on dev and
test set of TAT-QA.

###  A.3 Scale Prediction

We report the proportion of the ground truth scale in an answer and also the
performance of our scale predictor on dev and test set in Table 9.

Scale | Dev | Test
---|---|---
| % | Acc | % | Acc
None | 47.6 | 92.4 | 50.3 | 90.1
Thousand | 20.7 | 96.8 | 19.2 | 95.3
Million | 15.2 | 92.1 | 12.9 | 90.2
Billion | 0.4 | 28.6 | - | -
Percent | 16.1 | 95.9 | 17.7 | 95.9
Table 9: The proportion of ground truth scale on dev and test set of TAT-QA
with prediction accuracy by scale predictor of TagOp.


# Investigation of Genomic Effect of Zirconium Oxide Nanoparticles in
Escherichia coli Bacteria

Simin Rashidia,b, and Bahram Golestani Eimania aDepartment of Biology, Faculty
of Science, Islamic Azad University, Urmia, Iran.  b(Current Affiliation)
School of Medicine and Dentistry, Gold Coast Campus, Griffith University, Gold
Coast, QLD 2222, Australia.

###### Abstract

Due to the concerns of the society about the increase of antibiotic resistant
infections, many studies and research have been done on nanoparticles and
applications of nano-biotechnology. Zirconium Oxide ($\text{ZrO}_{2}$) in
which called zirconia, is a white oxide of zirconium metal that its diameter
is 20 nm. The colloidal size of these particles is often smaller than
bacterial and eukaryotic cells. The main intention of this paper is to
investigate the effect of different doses of $\text{ZrO}_{2}$ NPs on the
sequence’s changes for the Escherichia coli (E. coli) genome. At the first
step, E. coli was cultured in eosin methylene blue agar and brain heart broth
(BHB) mediums, respectively. Then, bacteria were treated with $\text{ZrO}_{2}$
NPs at concentrations of 100, 250, and 350 $\mu$g/ml. After treatment, the
growth of bacteria was evaluated by utilizing spectrophotometry at 600 nm
after incubation times including 2, 4, 6, 8, and 24 hours at 37 ∘C. At the
second step, the extraction of DNA was performed by using control and treated
samples. Then, the changes in the sequence of bacterial genome were
investigated using RAPD markers. Finally, NTSYS-PC platform was employed in
order to analyze of the results extracted by electrophoresis of products on
agarose gel. In this paper, it was observed that $\text{ZrO}_{2}$ NPs can
inhibit the growth of bacteria at concentrations of 250 and 350 $\mu$g/ml
after 8 hours of treatment. It was also found that the $\text{ZrO}_{2}$ NPs at
different concentrations have not changed the genome sequence of E. coli.
Furthermore, it was concluded that the $\text{ZrO}_{2}$ NPs with the
concentration of 350 $\mu$g/ml had the highest inhibitory properties without
significant changing in the genomic sequence of E. coli.

###### keywords:

Zirconium Oxide nanoparticles, Escherichia coli, RAPD marker, Genomic changes

††journal: arXiv

##  1 Introduction

E. coli is a gram-negative bacillus, oxidase and catalase-negative,
facultative anaerobic, and non-spore forming. This bacteria decarboxylates the
amino acid lysine and uses citrate as the only source of carbon. These
bacteria are commonly found in the intestines of warm-blooded organisms and
prevent the establishment of pathogenic bacteria in the intestine. E. coli is
an opportunistic organism that causes eye infections, urinary tract infection,
pneumonia, meningitis, and septicemia for the human. Different strains of E.
coli show diverse and complex pathogenic mechanisms in the occurrence of
diarrhea. One of these pathogenic mechanisms is the production of various
enterotoxins in which some of these enterotoxins cause pathogenesis in humans
as well as some of them cause pathogenesis in animals [1, 2]. It is difficult
to control the spread of pathogenic bacteria because antibiotic resistance is
a global problem between bacteria, leading to the spread of resistance among
other bacteria and the emergence of more resistant strains. The use of NPs is
a promising strategy against antibiotic resistance. In general, NPs are mainly
categorized in two groups consisting of metallic and non-metallic from where
metal NPs have significant antimicrobial properties on gram-positive and gram-
negative bacteria [3, 4]. NPs have some special properties that have been
considered by scientists in recent years [5]. NPs depict unique properties
because of their size and surface to volume ratio in which the smaller size of
the nanoparticle leads the greater the antibacterial activity as well as the
high surface to volume ratio causes the NPs to interact with the microbial
membrane [6, 7, 8, 9]. NPs are widely used in different applications such as
food processing and packaging, disinfection of textiles and equipment, and in
the biosensors for rapid detecting of contaminants. In addition, NPs are
utilized for therapeutic processes in biological and pharmaceutical research
activities due to their higher potential as well as many studies have shown
that NPs can have negative effects on the growth of microorganisms [10, 11].

There are many antibacterial mechanisms for metal NPs that are disruption of
energy transduction, generating reactive oxygen species (ROS), ion release,
biomolecular damage, and electrostatic reaction between NPs and bacterial
membranes. The interaction of NPs with macromolecules are based on the
difference between the negative charge of microorganisms and the positive
charge of NPs and this difference leads to junction of NPs with cell surface
and consequently cause cell death. Another antimicrobial mechanisms of metal
NPs is that they perforate the bacterial cell wall and accordingly penetrate
into the cell [12, 13]. $\text{ZrO}_{2}$ NPs have been considered due to their
better electrical and mechanical characteristics, high dielectric constant and
wide band gap as well as they are extensively used in dental implants and
building bone and hair tissue [14]. Moreover, $\text{ZrO}_{2}$ is used to
remove phosphate from contaminated water [15]. Additionally, the powders of
$\text{ZrO}_{2}$ are employed in chromatography because $\text{ZrO}_{2}$ is
used as a support surface for the separation of proteins and enzymes [16].
These NPs affect the bacterium Streptococcus mutans by using their hydrophobic
surface and attach to the bacterium instead of forming a biofilm. It is
noticeable that increasing the hydrophobic surface of $\text{ZrO}_{2}$ NPs
increases the adhesion forces of these NPs to attach bacteria [17].
$\text{ZrO}_{2}$ NPs are also used in biological and medicinal applications
and have high biomedical, antimicrobial, and anti-fungal activities as well as
significantly inhibit the growth of Rhizoctonia solani [18]. Furthermore,
$\text{ZrO}_{2}$ NPs are utilized in agriculture such as germination of plants
[19] as well as they have anti-fungal and antibacterial properties against
Candida albicans and E. coli, respectively [20].

Metal NPs may cause mutations in bacteria and antimicrobial activity by
altering the base pair [21]. By considering the mentioned challenges, in order
to use of $\text{ZrO}_{2}$ as an antibacterial agent on microorganisms, it is
needed to be examined effective antimicrobial concentrations and their
genotoxic effect as a suitable agent instead of antibiotics. The contributions
of this paper are listed in the following:

$\bullet$ The effect of $\text{ZrO}_{2}$ NPs on the bacterial genome has not
been investigated by now. This paper investigates the effect of different
doses of $\text{ZrO}_{2}$ NPs on the genome of E. coli bacteria.

$\bullet$ The genotoxic changes of DNA sequences are also explored using RAPD-
PCR method. It is important to note that the RAPD-PCR primers are randomly
attached to different sequences on DNA and randomly amplify fragments. This
method is used to determine the genome differences between two groups of
organisms in terms of genomic DNA sequence.

The outlines of this paper are as follows: Section 2 explains the material and
methods. The results of the experiments are provided in Section 3. In Section
4, appropriate comparisons between the presented work with other literature
are explained through discussion. Finally, Section 5 remarks the conclusion
including future directions.

##  2 Material and Methods

This section presents the details of employed materials and methods. The
Bacterial culture and the required conditions, treatment of bacteria with
$\text{ZrO}_{2}$ NPs, and DNA extraction and RAPD-PCR along with proper
evaluation of RAPD-PCR results as well as analyzes of electrophoresis data are
investigated in this section.

###  2.1 The Bacterial culture and the required conditions

In this stage, E. coli O157:H7 was characterized on eosin methylene blue agar.
The bacteria were cultured in 7 ml of BHB medium and placed in a shaking
incubator at 37 ∘C with 200 rpm for 24 hours before treatment. The bacterial
growth rate was measured by measuring the absorbance density (OD) of the
culture medium at 600 nm wavelength [15].

###  2.2 Treatment of bacteria with $\text{ZrO}_{2}$ NPs

$\text{ZrO}_{2}$ NPs with 20 nm diameter were prepared and the characteristics
of these NPs were analyzed by transmission electron microscopy (TEM). Then, to
investigate the effect of $\text{ZrO}_{2}$ NPs on bacteria, phosphate-buffered
saline with pH:7.4 were used as a solvent on the concentrations of 100, 250,
and 350 $\mu$g/ml in order to make a solution. In the next step, after
treatment with the mentioned three concentrations (i.e. 100, 250, and 350
$\mu$g/ml) and incubating at 37 ∘C with 200 rpm, the OD of each test tubes
were read at 600 nm wavelength.

###  2.3 DNA extraction and RAPD-PCR

DNA of the control and treated bacteria was extracted using DNA extraction
kits (Genet Bio Cat: No.K-3000) according to the manufactures’ instructions.
The quantity and quality of the extracted DNA were analyzed by
spectrophotometry and electrophoresis on 1% agarose gel, respectively.
Furthermore, RAPD-PCR was used to investigate of $\text{ZrO}_{2}$ NPs’ effect
on the bacterial genome. To this end, 10 random primers were used. It is
worthy to note that the sequence and characteristics of these 10 primers are
given in Table 1. The 25 $\mu$l PCR reaction including 10.5 $\mu$l of
deionized water, 1 $\mu$l of primer, 1 $\mu$l of extracted DNA, and 12.5
$\mu$l of the master mix. In addition, the temperature program of PCR is
listed in Table 2.

Table 1: Nucleotide sequence of RAPD-PCR primers Primer name | Primer nucleotide sequence
---|---
OPR12 | ACAGGTGCGT
OPS11 | AGTCGGTGG
OPA11 | CAATCGCCGT
OPS13 | GTCGTTCCTG
OPQ14 | GGACGCTTCA
OPB08 | GGTGACGCAG
OPS03 | CAGAGGTCCC
OPT17 | TCTGGTGAGG
OPR11 | GTAGCCGTCT
OPS14 | AAAGGGGTCC
Table 2: The temperature program of PCR Categories
---
Cycle | Stage name |  Temperature (∘C) | Time (sec) | Cycles’ number
First | Initial denaturation | 95 | 300 | 1
Second | Denaturation | 95 | 35 | 40
Annealing | 30 | 45 | 40
Extension | 72 | 45 | 40
Third | Final extension | 72 | 420 | 1

###  2.4 Evaluation of RAPD-PCR results

After PCR reaction, for detecting the bands, 10 $\mu$l of PCR products were
electrophoresed on 2% agarose gel (26×14 cm) containing red safe in TBE buffer
(1x) for 4 hours at 120 V. Furthermore, the DNA ladder marker was used at 100
to 1500 bp in order to determine the size of the product.

Figure 1: TEM image of $\text{ZrO}_{2}$ NPs Figure 2: XRD analysis results for
$\text{ZrO}_{2}$ NPs

###  2.5 Analyzes of electrophoresis data

The bands on the gel were based on the existence and absence with 1 and 0,
respectively. Then the data were molecularly analyzed in NTSYS-PC Dic platform
and the dendrogram was plotted by (Un weighted Pair Group Mean Average) UPGMA
method.

##  3 Results

This section explains the results of transmission electron microscopy (TEM)
and X-ray diffraction (XRD) of $\text{ZrO}_{2}$ NPs as well as the
antimicrobial properties of these NPs are evaluated. Furthermore, RAPD-PCR
product analyzes and the data analysis based on RAPD-PCR are investigated.

###  3.1 Results for the outputs of TEM and XRD

To verify the accuracy of $\text{ZrO}_{2}$ NPs, TEM is used. The extracted
results by TEM from $\text{ZrO}_{2}$ NPs and XRD analysis are depicted in
Figs. 1 and 2, respectively. In these figures, $\text{ZrO}_{2}$ NPs are seen
in black spots and are stabilized in silica pores. In addition, zirconium NPs
are less than 20 nm in size as shown in Fig. 1. In Fig. 2, XRD analysis
confirms the presence of $\text{ZrO}_{2}$ NPs as well as 2-Theta consisting of
36 and 38 are related to $\text{ZrO}_{2}$ NPs.

###  3.2 Evaluation of antimicrobial properties of $\text{ZrO}_{2}$ NPs

Table 3 summarizes the results of the antimicrobial activity of
$\text{ZrO}_{2}$ NPs on E. coli bacteria. As indicated in Table 3,
$\text{ZrO}_{2}$ NPs at the concentrations of 250 and 350 $\mu$g/ml inhibit
the growth of E. coli after 8 hours. Also, Fig. 3 shows the rate of change of
OD in proportion to the changes in the concentration of $\text{ZrO}_{2}$ NPs
at different times after treatment. Fig. 3 plots the changes in OD at
different time intervals in proportion to changes in the concentrations of
$\text{ZrO}_{2}$ NPs.

Table 3: Antimicrobial activity of $\text{ZrO}_{2}$ NPs Categories
---
Sample |  Tube number |  Before treatment |  2 hours after treatment |  4 hours after treatment |  6 hours after treatment |  8 hours after treatment |  24 hours after treatment
Control | 1 | 0.3 | 1.2 | 1.3 | 1.5 | 1.6 | 1.7
100 $\mu$g/ml | 2 | 0.3 | 1.2 | 1.5 | 1.8 | 1.7 | 1.6
2 | 0.3 | 1.1 | 1.5 | 1.7 | 1.6 | 1.6
250 $\mu$g/ml | 3 | 0.3 | 1.2 | 1.5 | 1.6 | 1.2 | 1.1
3 | 0.3 | 1.1 | 1.4 | 1.7 | 1.3 | 1.0
350 $\mu$g/ml | 4 | 0.3 | 1.0 | 1.3 | 1.1 | 0.8 | 0.7
4 | 0.3 | 1.0 | 1.2 | 1.1 | 0.9 | 0.7
Figure 3: Evaluation of OD changes in proportion to changes in
$\text{ZrO}_{2}$ nanoparticle concentrations at different times after
treatment

###  3.3 RAPD-PCR product analyzes

The results of electrophoresis on RAPD-PCR products with 10 primers on 2%
(26×14 cm) agarose gel are illustrated in Figs. 4 and 5. The results for the
bands were respectively scored based on their existence or absence in the
samples with 1 or 0 in which are shown in Tables 4 and 5. Table 4 indicates
the results for the bands extracted from Fig. 4 as well as Table 5 shows the
results for the bands extracted from Fig. 5. By considering Tables 4 and 5,
the conclusion is based on the differences in the bands that each primer forms
with the control and treated samples. In addition, the results for the primers
are expressing in detail as the following terms:

$\bullet$ Primer OPQ14: There are 4 bands on the gel. It is worthy to note
that all these 4 bands have difference between the control and treated groups.

$\bullet$ Primer OPS13: There are 3 bands on the gel. For this primer, there
is no difference between the control and treated groups.

$\bullet$ Primers OPS11, OPS03, OPT17, and OPS14: For these primers, there is
only 1 band on the gel. Furthermore, there are no difference between the
control and treated groups.

$\bullet$ Primer OPA11: There are 4 bands on the gel. It is important to note
that 2 of these 4 bands have difference between the control and treated
groups.

$\bullet$ Primer OPB08: There are 2 bands on the gel. Also, all these bands
have difference between the control and treated groups.

$\bullet$ Primer OPR12: There are 5 bands on the gel. Moreover, there is no
difference between the control and treated groups.

$\bullet$ Primer OPS13: There are 3 bands on the gel. Again, there is no
difference between the control and treated groups.

Figure 4: The electrophoresis results of RAPD-PCR product on agarose gel;
Columns 1 to 4: Primer OPQ14, Column 5: Marker, Columns 6 to 9: Primer OPS13,
Columns 10 to 13: Primer OPS14, Column 14: Marker, Columns 15 to 18: Primer
OPS11 Figure 5: The electrophoresis results of RAPD-PCR product on agarose
gel;
Columns 1 to 4: Primer OPS03, Columns 5 to 8: Primer OPA11, Column 9: Marker,
Columns 10 to 13: Primer OPB08, Columns 14 to 17: Primer OPA11, Column 18:
Marker, Columns 19 to 22: Primer OPR12, Columns 23 to 26: Primer OPT17, Column
27: Marker Table 4: Banding results related to the primers of Fig. 4
Categories
---
Primer | Band (bp) | C | $\text{T}_{1}$ | $\text{T}_{2}$ | $\text{T}_{3}$
OPQ14 | 1500 | 1 | 1 | 0 | 1
1400 | 1 | 1 | 0 | 1
1300 | 1 | 1 | 0 | 1
500 | 0 | 0 | 1 | 0
OPS13 | 1700 | 1 | 1 | 1 | 1
1400 | 1 | 1 | 1 | 1
300 | 1 | 1 | 0 | 1
OPS14 | 850 | 1 | 1 | 1 | 1
OPS11 | 550 | 1 | 1 | 1 | 1
Table 5: Banding results related to the primers of Fig. 5 Categories
---
Primer | Band (bp) | C | $\text{T}_{1}$ | $\text{T}_{2}$ | $\text{T}_{3}$
OPS03 | 1300 | 1 | 1 | 1 | 1
OPA11 | 1500 | 1 | 1 | 1 | 1
1300 | 1 | 1 | 1 | 1
950 | 1 | 0 | 1 | 1
600 | 1 | 0 | 1 | 1
OPB08 | 1000 | 1 | 1 | 1 | 0
700 | 1 | 0 | 1 | 0
OPR11 | 1600 | 0 | 0 | 1 | 0
1300 | 1 | 1 | 1 | 1
OPR12 | 1400 | 1 | 1 | 1 | 1
1100 | 1 | 1 | 1 | 1
1000 | 1 | 1 | 1 | 1
900 | 1 | 1 | 1 | 1
800 | 1 | 1 | 1 | 0
OPT17 | 500 | 1 | 1 | 1 | 1

###  3.4 Analyzis based on RAPD data

To analyzis the results based on RAPD data, the NTSYS-PC platform is employed.
For this aim, the similarity matrix is calculated for the control and treated
samples based on Dic method. The results of similiraty matrix are shown in
Table 6 and indicate the genetic distance between the samples are vary from
0.82 to 1.000. It is noticale that the closer numbers to 1 means the greater
genetic similarity among the samples. Reffering to the results from Table 6,
the genome of the treated bacteria is very similar to the control bacteria.

Table 6: Similarity matrix for the control and treated samples Dice
(Czekanowski or Sorenson) Measure
---
|  | 1:ctr | 2:t100 | 3:t250 | 4:t350
Case | 1:ctr | 1.0000000 |  |  |
2:t100 | 0.9900000 | 1.0000000 |  |
3:t250 | 0.9268293 | 0.9368930 | 1.0000000 |
4:t350 | 0.9090909 | 0.9091337 | 0.8292683 | 1.0000000
This is a similarity matrix |  |  |  |  |

Fig. 6 indicates the extracted dendrogram from NTSYS-PC platform based on RAPD
test analyzes using UPGMA method. The dendrogram shown in Fig. 6 has drawn to
evaluate the genetic diversity of treated samples with $\text{ZrO}_{2}$ NPs.
According to the Fig. 6, it is observed that $\text{ZrO}_{2}$ NPs in treatment
3 with the concentration of 350 $\mu$g/ml has the highest inhibitory effect
without significant changes in the bacterial genome.

##  4 Discussion

Excessive use of antibiotics has made bacteria resistant to antibiotics. To
resolve this problem, nanotechnology has been considered to combat microbial
resistance. Most research has suggested that the use of NPs is a way to remove
antibiotic-resistant bacteria [22, 23]. The effect of NPs on the genomes of
bacteria and other environmental organisms has great importance. In this
study, specific concentrations of $\text{ZrO}_{2}$ NPs have been used to treat
bacteria in order to evaluate their antimicrobial and genomic effect.

The results of the present study show that $\text{ZrO}_{2}$ NPs have good
antimicrobial effect at doses of 250 and 350 $\mu$g/ml and can almost inhibit
the growth of E. coli. Jangra et al. Showed in their study that zirconia only
inhibits E. coli, while Zr(IV) crystals have inhibitory activity against gram-
positive and gram-negative bacteria and fungal strains. These researchers also
showed that the antimicrobial activity of zirconia and Zr(IV) crystals occurs
through surface atoms [24]. Gowri et al. found that $\text{ZrO}_{2}$ NPs had
antimicrobial activity against E. coli and Staphylococcus aureus. In addition,
these NPs have anti-fungal properties against Candida albicans and Aspergillus
niger [20]. However, the studies in [20, 24] have not investigated the
genotoxic effect of $\text{ZrO}_{2}$ NPs on the bacterial genome. Atalay et
al. in [25] examined the effect of $\text{ZrO}_{2}$ NPs on eukaryotic cells
such as mouse fibroblasts and they have stated that $\text{ZrO}_{2}$ NPs cause
damage and induce apoptosis in mouse fibroblast cells [25]. In 2020, Khan et
al. studied surface changes in $\text{ZrO}_{2}$ NPs and bonded glutamic acid
to $\text{ZrO}_{2}$ NPs during their experiments [26]. In [26], it is stated
that when glutamic acid is bonded to $\text{ZrO}_{2}$ NPs, antimicrobial
property is effectively enhanced compared to pristine $\text{ZrO}_{2}$ NPs.
Aziz et al. in [27] tested the antibacterial property of $\text{ZrO}_{2}$ NPs
combined with ceftazidime on Klebsiella pneumoniae. In [27], it is concluded
that the antimicrobial property of ceftazidime combined with $\text{ZrO}_{2}$
NPs is higher than ceftazidime and $\text{ZrO}_{2}$ NPs. Furthermore, the
combination of $\text{ZrO}_{2}$ NPs with ceftazidime is a useful method for
the treatment of infections caused by Klebsiella pneumoniae [27]. Derbalah et
al. examined and tested the role of $\text{ZrO}_{2}$ NPs on plants [18]. In
[18], it is concluded that $\text{ZrO}_{2}$ NPs have higher antifungal
activity and significantly inhibit the growth of Rhizoctonia solani. In
addition, these $\text{ZrO}_{2}$ NPs can control cucumber root rot disease,
effectively [18]. Jalil et al. have also investigated the antifungal property
of $\text{ZrO}_{2}$ NPs in [28] and stated that $\text{ZrO}_{2}$ NPs inhibit
the growth of Fusarium graminearum at a concentration of 200 mg/ml. In [29],
Zhang et al. stated that the concentration of NPs plays a pivotal role
compared to their size in view point of antimicrobial effect. It is concluded
in [29] that with increasing the concentration of NPs, the antimicrobial
activity increases which is consistent with the results of the this paper. In
2015, Gawri et al. studied the antimicrobial properties of $\text{ZrO}_{2}$
NPs versus E. coli and Staphylococcus aureus [30]. The results extracted from
[30] show that $\text{ZrO}_{2}$ NPs are effectively inhibit E. coli than
Staphylococcus aureus with the same concentration of $\text{ZrO}_{2}$ NPs. It
is also important to note that the antimicrobial activity of metal oxide NPs
such as silver, copper and zinc has been proven in many studies [31].
Regarding to the mentioned issues, this paper proves that $\text{ZrO}_{2}$ NPs
have antimicrobial activity at higher concentrations and after about 8 hours.
Furthermore, it is concluded that $\text{ZrO}_{2}$ NPs require more time and
concentration for their operation. It is worthy to note that these NPs cannot
be a good choice for the cases in which time and concentration are important.

Figure 6: The extracted dendrogram from analyzes based on RAPD test using the
UPGMA method

Another important issue which has been considered in this paper is the effect
of NPs on the bacterial genome. There are different methods for the
interaction between NPs and bacteria [32]. One of these methods is that the
NPs attach to the bacterial membrane and penetrate into the bacterial cell. In
this method, bacterial membranes consist of sulfur-containing proteins in
which NPs react not only with these proteins, but also with phosphorus-
containing compounds including DNA. Another method is that the NPs attack to
the respiratory chain which lead to cell death [33, 34]. As another mechanism
for the effect of NPs on the bacteria is that the metal oxide NPs make
disturbance in transcription and translation [35]. Moreover, these metal oxide
NPs cause fragmentation of single-stranded DNA and affect gene expression as
well as can also affect the ability of RNA polymerase to open the helix and
make transcription. In general, any factor that damages DNA can cause that
cell to die [36]. In this paper, in order to study the genotoxic effect of
$\text{ZrO}_{2}$ NPs, E. coli O157:H7 has been used as a model for gram-
negative bacteria. By considering this issue, 10 primers have been used in the
RAPD-PCR reaction. By this reaction, the existence or absence of the bands
happens which means changing in the DNA sequence by $\text{ZrO}_{2}$ NPs. It
is noticeable that some of these primers do not identify the target sequences
and subsequently the relevant fragment is not amplified. The difference
between the bands observed in the treated and control samples expresses that
the primer target sequences have undergone changes in the treated bacteria
that indicating a direct or indirect mutation by NPs [37, 38]. Lee et al. have
reported that by disrupting the mechanism of DNA polymerase, the replication
will also be disrupted [39]. Disrupting of replication causes changes in DNA
sequences that lead to differences in target sequences by RAPD primers [40].
It can be concluded from [39, 40] that NPs can disrupt genes which control the
mechanisms of transcription and replication and also can affect RNA
polymerase. However, this paper demonstrates that $\text{ZrO}_{2}$ NPs do not
cause significant mutations in the genome sequences of E. coli bacteria.

##  5 Conclusion

In this paper, the effect of $\text{ZrO}_{2}$ NPs studied on E. coli. The
results show that the growth of E. coli bacteria is inhibited after treatment
with $\text{ZrO}_{2}$ NPs at concentrations of 250 and 350 $\mu$g/ml after 8
hours. Therefore, $\text{ZrO}_{2}$ NPs are able to enter the bacterial cell
and induce inhibiting effect. Furthermore, since $\text{ZrO}_{2}$ NPs have not
altered the genome of E. coli at different concentrations, it might be
possible to utilize these NPs for therapeutic goals in order to deliver drugs
to human cells because these NPs do not cause significant mutations in healthy
cells. In conclusion, $\text{ZrO}_{2}$ NPs with a concentration of 350
$\mu$g/ml have the highest inhibitory property without significant altering in
the genomic DNA sequence of E. coli bacteria. The synergistic effect of
$\text{ZrO}_{2}$ NPs and nisin on E. coli genome will be focused and
investigated as a future work.

## References

  * Nataro and Kaper [1998] Nataro, J. P.; Kaper, J. B. Diarrheagenic escherichia coli. _Clinical microbiology reviews_ 1998, _11_ , 142–201.
  * Law [2000] Law, D. Virulence factors of Escherichia coli O157 and other Shiga toxin-producing E. coli. _Journal of Applied Microbiology_ 2000, _88_ , 729–745.
  * Hosseini et al. [2018] Hosseini, S. S.; Ghaemi, E.; Koohsar, F. Influence of ZnO nanoparticles on Candida albicans isolates biofilm formed on the urinary catheter. _Iranian journal of microbiology_ 2018, _10_ , 424.
  * Annu et al. [2020] Annu, A.; Sivasankari, C.; Krupasankar, U. Synthesis and characerization of $\text{ZrO}_{2}$ nanoparticle by leaf extract bioreduction process for its biological studies. _Materials Today: Proceedings_ 2020,
  * Hasan [2015] Hasan, S. A review on nanoparticles: their synthesis and types. _Res. J. Recent Sci_ 2015, _2277_ , 2502.
  * Murthy et al. [2020] Murthy, S.; Effiong, P.; Fei, C. C. _Metal Oxide Powder Technologies_ ; Elsevier, 2020; pp 233–251.
  * Jones et al. [2008] Jones, N.; Ray, B.; Ranjit, K. T.; Manna, A. C. Antibacterial activity of ZnO nanoparticle suspensions on a broad spectrum of microorganisms. _FEMS microbiology letters_ 2008, _279_ , 71–76.
  * Vahedi et al. [2017] Vahedi, M.; Hosseini-Jazani, N.; Yousefi, S.; Ghahremani, M. Evaluation of anti-bacterial effects of nickel nanoparticles on biofilm production by Staphylococcus epidermidis. _Iranian Journal of Microbiology_ 2017, _9_ , 160.
  * Amelia et al. [2012] Amelia, M.; Lincheneau, C.; Silvi, S.; Credi, A. Electrochemical properties of CdSe and CdTe quantum dots. _Chemical Society Reviews_ 2012, _41_ , 5728–5743.
  * Christian et al. [2008] Christian, P.; Von der Kammer, F.; Baalousha, M.; Hofmann, T. Nanoparticles: structure, properties, preparation and behaviour in environmental media. _Ecotoxicology_ 2008, _17_ , 326–343.
  * King et al. [2018] King, T.; Osmond-McLeod, M. J.; Duffy, L. L. Nanotechnology in the food sector and potential applications for the poultry industry. _Trends in Food Science & Technology_ 2018, _72_ , 62–73.
  * Bayroodi and Jalal [2016] Bayroodi, E.; Jalal, R. Modulation of antibiotic resistance in Pseudomonas aeruginosa by ZnO nanoparticles. _Iranian journal of microbiology_ 2016, _8_ , 85.
  * Fariba et al. [2016] Fariba, A.; Bahram, G. E.; Farrokh, K.; Tabrizy, S.; Pooneh, S. S. An investigation of the effect of copper oxide and silver nanoparticles on E. coli genome by RAPD molecular markers. _J Adv Biotechnol Microbiol_ 2016, _1_ , 1–6.
  * Paun et al. [2013] Paun, D.; Petris, R.; Carsote, M.; Ferechide, D.; Poiana, C. Particular aspects in autoimmune thyroid diseases. _Practica Medicala_ 2013, _3_ , 173–177.
  * Zhang et al. [2017] Zhang, C.; Li, Y.; Wang, F.; Yu, Z.; Wei, J.; Yang, Z.; Ma, C.; Li, Z.; Xu, Z.; Zeng, G. Performance of magnetic zirconium-iron oxide nanoparticle in the removal of phosphate from aqueous solution. _Applied Surface Science_ 2017, _396_ , 1783–1792.
  * Banerjee et al. [2016] Banerjee, K.; Prithviraj, M.; Augustine, N.; Pradeep, S. P.; Thiagarajan, P. Analytical characterization and antimicrobial activity of nano zirconia particles. _J. Chem. Pharm. Sci_ 2016, _9_.
  * Yu et al. [2016] Yu, P.; Wang, C.; Zhou, J.; Jiang, L.; Xue, J.; Li, W. Influence of surface properties on adhesion forces and attachment of Streptococcus mutans to zirconia in vitro. _BioMed research international_ 2016, _2016_.
  * Derbalah et al. [2019] Derbalah, A.; Elsharkawy, M. M.; Hamza, A.; El-Shaer, A. Resistance induction in cucumber and direct antifungal activity of zirconium oxide nanoparticles against Rhizoctonia solani. _Pesticide biochemistry and physiology_ 2019, _157_ , 230–236.
  * Yousif et al. [2019] Yousif, A. M.; Ahmed, S. H.; Nuaman, R. S.; Al-Karkhi, I. H. T. Effect of Zirconium Oxide and Yttrium oxide Nanoparticles on Plant Germination of Cucumis Sativus. _Indian Journal of Public Health Research & Development_ 2019, _10_.
  * Gowri et al. [2014] Gowri, S.; Gandhi, R. R.; Sundrarajan, M. Structural, optical, antibacterial and antifungal properties of zirconia nanoparticles by biobased protocol. _Journal of Materials Science & Technology_ 2014, _30_ , 782–790.
  * Brown [2016] Brown, T. A. _Gene cloning and DNA analysis: an introduction_ ; John Wiley & Sons, 2016.
  * Imran et al. [2015] Imran, M.; Revol-Junelles, A.-M.; Paris, C.; Guedon, E.; Linder, M.; Desobry, S. Liposomal nanodelivery systems using soy and marine lecithin to encapsulate food biopreservative nisin. _LWT-Food Science and Technology_ 2015, _62_ , 341–349.
  * Allahverdiyev et al. [2011] Allahverdiyev, A. M.; Abamor, E. S.; Bagirova, M.; Rafailovich, M. Antimicrobial effects of $\text{TiO}_{2}$ and Ag2O nanoparticles against drug-resistant bacteria and leishmania parasites. _Future microbiology_ 2011, _6_ , 933–940.
  * Jangra et al. [2012] Jangra, S. L.; Stalin, K.; Dilbaghi, N.; Kumar, S.; Tawale, J.; Singh, S. P.; Pasricha, R. Antimicrobial activity of zirconia ($\text{ZrO}_{2}$) nanoparticles and zirconium complexes. _Journal of nanoscience and nanotechnology_ 2012, _12_ , 7105–7112.
  * Atalay et al. [2018] Atalay, H.; Celik, A.; Ayaz, F. Investigation of genotoxic and apoptotic effects of zirconium oxide nanoparticles (20 nm) on L929 mouse fibroblast cell line. _Chemico-biological interactions_ 2018, _296_ , 98–104.
  * Khan et al. [2020] Khan, M.; Shaik, M. R.; Khan, S. T.; Adil, S. F.; Kuniyil, M.; Khan, M.; Al-Warthan, A. A.; Siddiqui, M. R. H.; Nawaz Tahir, M. Enhanced Antimicrobial Activity of Biofunctionalized Zirconia Nanoparticles. _ACS omega_ 2020, _5_ , 1987–1996.
  * Aziz et al. [2019] Aziz, S. N.; Al-Sallami, K. J.; Abd, S. Y.; Al-Musawi, A. M. A.; Mohammed, M. A.; Abed, A.-M. H.; Mohammed, S. Q. IMPROVING THE ANTIBACTERIAL ACTIVITY BY THE COMBINATION OF ZIRCONIUM OXIDE NANOPARTICLES ($\text{ZrO}_{2}$) AND CEFTAZIDIME AGAINST KLEBSIELLA PNEUMONIAE. _Global Journal of Public Health Medicine_ 2019, _1_ , 16–20.
  * Jalill and Numan [2016] Jalill, R. D. A.; Numan, R. S. Silver nitrate and zirconium oxide nanoparticles as management of wheat damping-off caused by Fusarium graminearum. _Journal of Genetic and Environmental Resources Conservation_ 2016, _4_ , 85–93.
  * Zhang et al. [2007] Zhang, Z.; Yang, M.; Huang, M.; Hu, Y.; Xie, J.; others Study on germicidal efficacy and toxicity of compound disinfectant gel of nanometer silver and chlorhexidine acelate. _Chinese Journal of Health Laboratory Technology_ 2007, _17_ , 1403–1406.
  * Gowri et al. [2015] Gowri, S.; Gandhi, R. R.; Senthil, S.; Sundrarajan, M. Effect of Calcination Temperature on Nyctanthes Plant Mediated Zirconia Nanoparticles; Optical and Antibacterial Activity for Optimized Zirconia. _Journal of Bionanoscience_ 2015, _9_ , 181–189.
  * Hsueh et al. [2017] Hsueh, Y.-H.; Tsai, P.-H.; Lin, K.-S. Ph-dependent antimicrobial properties of copper oxide nanoparticles in staphylococcus aureus. _International journal of molecular sciences_ 2017, _18_ , 793.
  * Selvarani and Prema [2013] Selvarani, M.; Prema, P. Evaluation of antibacterial efficacy of chemically synthesized copper and zerovalent iron nanoparticles. _Asian J. Pharm. Clin. Res_ 2013, _6_ , 222–227.
  * Damm et al. [2008] Damm, C.; Münstedt, H.; Rösch, A. The antimicrobial efficacy of polyamide 6/silver-nano-and microcomposites. _Materials Chemistry and Physics_ 2008, _108_ , 61–66.
  * Panacek et al. [2006] Panacek, A.; Kvitek, L.; Prucek, R.; Kolar, M.; Vecerova, R.; Pizurova, N.; Sharma, V. K.; Neverna, T. j.; Zboril, R. Silver colloid nanoparticles: synthesis, characterization, and their antibacterial activity. _The Journal of Physical Chemistry B_ 2006, _110_ , 16248–16253.
  * Martel [2006] Martel, S. Method and system for controlling micro-objects or micro-particles. 2006; US Patent App. 11/145,007.
  * Huh and Kwon [2011] Huh, A. J.; Kwon, Y. J. “Nanoantibiotics”: a new paradigm for treating infectious diseases using nanomaterials in the antibiotics resistant era. _Journal of controlled release_ 2011, _156_ , 128–145.
  * Chang et al. [2012] Chang, Y.-N.; Zhang, M.; Xia, L.; Zhang, J.; Xing, G. The toxic effects and mechanisms of CuO and ZnO nanoparticles. _Materials_ 2012, _5_ , 2850–2871.
  * King et al. [2006] King, M. D.; Humphrey, B. J.; Wang, Y. F.; Kourbatova, E. V.; Ray, S. M.; Blumberg, H. M. Emergence of community-acquired methicillin-resistant Staphylococcus aureus USA 300 clone as the predominant cause of skin and soft-tissue infections. _Annals of internal medicine_ 2006, _144_ , 309–317.
  * Li et al. [2006] Li, Y.; Leung, P.; Yao, L.; Song, Q.; Newton, E. Antimicrobial effect of surgical masks coated with nanoparticles. _Journal of Hospital Infection_ 2006, _62_ , 58–63.
  * Pal et al. [2007] Pal, S.; Tak, Y. K.; Song, J. M. Does the antibacterial activity of silver nanoparticles depend on the shape of the nanoparticle? A study of the gram-negative bacterium Escherichia coli. _Applied and environmental microbiology_ 2007, _73_ , 1712–1720.