# Fusing Context Into Knowledge Graph for Commonsense Question Answering

Yichong Xu , Chenguang Zhu∗, Ruochen Xu, Yang Liu, Michael Zeng, Xuedong Huang
Microsoft Cognitive Services Research Group
<EMAIL_ADDRESS>
Equal contribution

###### Abstract

Commonsense question answering (QA) requires a model to grasp commonsense and
factual knowledge to answer questions about world events. Many prior methods
couple language modeling with knowledge graphs (KG). However, although a KG
contains rich structural information, it lacks the context to provide a more
precise understanding of the concepts. This creates a gap when fusing
knowledge graphs into language modeling, especially when there is insufficient
labeled data. Thus, we propose to employ external entity descriptions to
provide contextual information for knowledge understanding. We retrieve
descriptions of related concepts from Wiktionary and feed them as additional
input to pre-trained language models. The resulting model achieves state-of-
the-art result in the CommonsenseQA dataset and the best result among non-
generative models in OpenBookQA. Our code is available at
https://github.com/microsoft/DEKCOR-CommonsenseQA.

##  1 Introduction

One critical aspect of human intelligence is the ability to reason over
everyday matters based on observation and knowledge. This capability is
usually shared by most people as a foundation for communication and
interaction with the world. Therefore, commonsense reasoning has emerged as an
important task in natural language understanding, with various datasets and
models proposed in this area (Ma et al., 2019; Talmor et al., 2018; Wang et
al., 2020; Lv et al., 2020).

While massive pre-trained models (Devlin et al., 2018; Liu et al., 2019) are
effective in language understanding, they lack modules to explicitly handle
knowledge and commonsense. Also, structured data like knowledge graph is much
more efficient in representing commonsense compared with unstructured text.
Therefore, there have been multiple methods coupling language models with
various forms of knowledge graphs (KG) for commonsense reasoning, including
knowledge bases (Sap et al., 2019; Yu et al., 2020b), relational paths (Lin et
al., 2019), graph relation network (Feng et al., 2020) and heterogeneous graph
(Lv et al., 2020). These methods combine the merits of language modeling and
structural knowledge information and improve the performance of commonsense
reasoning and question answering.

However, there is still a non-negligible gap between the performance of these
models and humans. One reason is that, although a KG can encode topological
information between the concepts, it lacks rich context information. For
instance, for a graph node for the entity “Mona Lisa”, the graph depicts its
relations to multiple other entities. But given this neighborhood information,
it is still hard to infer that it is a painting. On the other hand, we can
retrieve the precise definition of “Mona Lisa” from external sources, e.g. the
definition of Mona Lisa in Wiktionary is “A painting by Leonardo da Vinci,
widely considered as the most famous painting in history”. To represent
structured data that can be seamlessly integrated into language models, we
need to provide a panoramic view of each concept in the knowledge graph,
including its neighboring concepts, relations to them, and a definitive
description of it.

Thus, we propose the DEKCOR model, i.e. DEscriptive Knowledge for COmmonsense
question answeRing, to tackle multiple choice commonsense questions. Given a
question and a choice, we first extract the contained concepts. Then, we
extract the edge between the question concept and the choice concept in
ConceptNet (Speer et al., 2017). If such an edge does not exist, we compute a
relevance score for each knowledge triple (subject, relation, object)
containing the choice concept, and select the one with the highest score.
Next, we retrieve the definition of these concepts from Wiktionary via
multiple criteria of text matching. Finally, we feed the question, choice,
selected triple and definitions into the language model ALBERT (Lan et al.,
2019) to produce a score indicating how likely this choice is the correct
answer.

We evaluate our model on CommonsenseQA (Talmor et al., 2018) and OpenBookQA
(Mihaylov et al., 2018). On CommonsenseQA, it outperforms the previous state-
of-the-art result by 1.2% (single model) and 3.8% (ensemble model) on the test
set. On OpenBookQA, our model outperforms all baselines other than two large-
scale models based on T5 (Raffel et al., 2019). We further conduct ablation
studies to demonstrate the effectiveness of fusing context into the knowledge
graph.

##  2 Related work

Several different approaches have been investigated for leveraging external
knowledge sources to answer commonsense questions. Min et al. (2019) addresses
open-domain QA by retrieving from a passage graph, where vertices are passages
and edges represent relationships derived from external knowledge bases and
co-occurrence. Sap et al. (2019) introduces the ATOMIC graph with 877k textual
descriptions of inferential knowledge (e.g. if-then relation) to answer causal
questions. Lin et al. (2019) projects questions and choices to the knowledge-
based symbolic space as a schema graph. It then utilizes path-based LSTM to
give scores. Feng et al. (2020) adopts the multi-hop graph relation network
(MHGRN) to perform reasoning unifying path-based methods and graph neural
networks. Lv et al. (2020) proposes to extract evidence from both structured
knowledge base such as ConceptNet and Wikipedia text and conduct graph-based
representation and inference for commonsense reasoning. Wang et al. (2020)
employs GPT-2 to generate paths between concepts in a knowledge graph, which
can dynamically provide multi-hop relations between any pair of concepts.

Several studies have utilized knowledge descriptions for different tasks. Yu
et al. (2020a) uses description text from Wikipedia for knowledge-text co-
pretraining. Xie et al. (2016) encodes the semantics of entity descriptions in
knowledge graphs to improve the performance on knowledge graph completion and
entity classification. Chen et al. (2018) co-trains the knowledge graph
embeddings and entity description representation for cross-lingual entity
alignment. Concurrent with our work, Chen et al. (2020) also insert knowledge
descriptions into commonsense question answering. Compared with our work, the
proposed method in Chen et al. (2020) is much more complex, e.g. involving
training additional rankers on retrieved text, while our result outperforms
Chen et al. on CommonsenseQA.

##  3 Method

###  3.1 Knowledge Retrieval

Problem formulation. In this paper, we focus on the following QA task: given a
commonsense question $q$, select the correct answer from several choices
$c_{1},...,c_{n}$. In most cases, the question does not contain any mentions
of the answer. Therefore, external knowledge sources can be used to provide
additional information. We adopt ConceptNet (Speer et al., 2017) as our
knowledge graph $G=(V,E)$, which contains over 8 million entities as nodes and
over 21 million relations as edges. In the following, we use triple to refer
to two neighboring nodes and the edge connecting them, i.e. $(u\in
V,p=(u,v)\in E,v\in V)$, with $u$ being the subject, $p$ the relation, and $v$
the object.

Suppose the question mentions an entity $e_{q}\in V$ and the choice contains
an entity $e_{c}\in V$111CommonsenseQA provides the question/choice entity.
For OpenBookQA, we choose from the extracted entities that are most frequent
in retrieved facts. See Appendix for details.. We then employ the KCR method
(Lin, 2020) to select relation triples. If there is a direct edge $r$ from
$e_{q}$ to $e_{c}$ in $G$, we choose this triple ($e_{q}$, $r$, $e_{c}$).
Otherwise, we retrieve all the $N$ triples containing $e_{c}$. Each triple $j$
is assigned a score $s_{j}$ which is the product of its triple weight $w_{j}$
provided by ConceptNet and relation type weight $t_{r_{j}}$:

$\displaystyle s_{j}$ $\displaystyle=w_{j}\cdot
t_{r_{j}}=w_{j}\cdot\frac{N}{N_{r_{j}}}$ (1)

Here, $r_{j}$ is the relation type of the triple $j$, and $N_{r_{j}}$ is the
number of triples among the retrieved triples that have the relation type
$r_{j}$. In other words, this process favors rarer relation types. Finally,
the triple with the highest weight is chosen.

Figure 1: In our model, the input to ALBERT includes the question, choice,
entity names, description text and triple. An attention-based weighted sum and
a softmax layer process the output from ALBERT to produce the prediction.

###  3.2 Contextual information

The retrieved entities and relations from the knowledge graph are described by
their surface form. Without additional context, it is hard for the language
model to understand its exact meaning, especially for proper nouns.

Therefore, we leverage large-scale online dictionaries to provide definitions
as context. We use a dump of Wiktionary222https://www.wiktionary.org/ which
includes definitions of 999,614 concepts. For every concept, we choose its
first definition entry in Wiktionary as the description. For every
question/choice concept, we find its closest match in Wiktionary by using the
following forms in order: i) original form; ii) lemma form by Spacy (Honnibal
and Montani, 2017); iii) base word (last word). For example, the concept
“taking notes” does not appear in its original form in Wiktionary, but its
lemma form “take notes” is in Wiktionary and we get its description text: “To
make a record of what one hears or observes for future reference”. In this
way, we find descriptions of all entities in our experiments. The descriptions
of the question and choice concept are denoted by $d_{q}$ and $d_{c}$,
respectively.

Finally, we feed the question, choice, descriptions and triple (from Section
3.1) into the ALBERT model (Lan et al., 2019) in the following format: [CLS]
$q$ $c$ [SEP] $e_{q}$: $d_{q}$ [SEP] $e_{c}$: $d_{c}$ [SEP] triple.

###  3.3 Reasoning

On top of ALBERT, we leverage an attention-based weighted sum and a softmax
layer to generate the relevance score for the question-choice pair. In detail,
suppose the output representations of ALBERT is
$({\bm{x}}_{0},...,{\bm{x}}_{m})$, where ${\bm{x}}_{i}\in R^{d}$. We compute a
weighted sum of these embeddings based on attention:

$\displaystyle q_{i}$ $\displaystyle={\bm{u}}^{T}{\bm{x}}_{i}$ (2)
$\displaystyle\alpha_{i}$ $\displaystyle=\mbox{softmax}(q_{i})$ (3)
$\displaystyle{\bm{v}}$
$\displaystyle=\sum_{i=0}^{m}{\alpha_{i}{\bm{x}}_{i}},$ (4)

where ${\bm{u}}$ is a parameter vector. The relevance score between the
question and the choice is then $s=\mbox{softmax}({\bm{v}}^{T}{\bm{b}})$,
where ${\bm{b}}\in R^{d}$ is a parameter vector and the softmax is computed
over all choices for the cross-entropy loss function.

The architecture of our model DEKCOR and the construction of input is shown in
Fig. 1.

Table 1: Statistics of CommonsenseQA (CSQA) and OpenBookQA (OBQA). Dataset | Train | Dev | Test | Choices
---|---|---|---|---
CSQA | 9,741 | 1,221 | 1,140 | 5
OBQA | 4,957 | 500 | 500 | 4
|  |  |  |
Table 2: Accuracy on the test set of CommonsenseQA. Methods | Single | Ensemble
---|---|---
BERT+OMCS | 62.5 | -
RoBERTa | 72.1 | 72.5
RoBERTa+HyKAS | 73.2 | -
XLNet+DREAM | - | 73.3
RoBERTa+KE | 73.3 | -
RoBERTa+KEDGN | - | 74.4
XLNet+GraphReason | 75.3 | -
ALBERT | - | 76.5
RoBERTa+MHGRN | 75.4 | 76.5
ALBERT+PG-Full | 75.6 | 78.2
T5 | 78.1 | -
ALBERT+KRD | 78.4 | -
UnifiedQA | 79.1 | -
ALBERT+KCR | 79.5 | -
DEKCOR (ours) | 80.7 | 83.3
|  |
Table 3: Accuracy on the test set of OpenBookQA. Methods | Accuracy
---|---
BERT + Careful Selection | 72.0
AristoRoBERTa | 77.8
ALBERT + KB | 81.0
ALBERT + PG-Full | 81.8
TTTTT (T5-3B) | 83.2
UnifiedQA (T5-11B) | 87.2
DEKCOR (ours) | 82.4
|
Table 4: Ablation results on the dev sets of CommonsenseQA and OpenBookQA. Methods | CSQA | OBQA
---|---|---
DEKCOR | 84.7 | 82.2
Triple Only | 82.0 | 80.0
Description Only | 80.3 | 81.8
No Context | 78.9 | 80.0
|  |

##  4 Experiments

###  4.1 Datasets and baselines

We evaluate our model on two benchmark datasets of multiple-choice questions
for commonsense question answering: CommonsenseQA (Talmor et al., 2018) and
OpenBookQA (Mihaylov et al., 2018). CommonsenseQA creates questions from
ConceptNet entities and relations; OpenBookQA probes elementary science
knowledge from a book of 1,326 facts. The statistics of the datasets is
provided in Table 1. For OpenBookQA, we follow prior approaches (Wang et al.,
2020) to append top 5 retrieved facts provided by Aristo team (Clark et al.,
2019) to the input. We also pre-train our OpenBookQA model on CommonsenseQA’s
training set as we find it helps to boost the performance.

We compare our models with state-of-the-art baselines, which all employ pre-
trained models including RoBERTa Liu et al. (2019), XLNet Yang et al. (2019),
ALBERT Lan et al. (2019) and T5 Raffel et al. (2019) and some adopt additional
modules to process knowledge information. A detailed description of the
baselines is in the Appendix.

###  4.2 Results

CommonsenseQA. Table 2 shows the accuracy on the test set of CommonsenseQA.
For a fair comparison, we categorize the results into single models and
ensemble models. Our ensemble model consists of 7 single models with different
initialization random seeds, and its output is the majority of choices
selected by these single models. More implementation details are shown in the
Appendix.

Our proposed DEKCOR outperforms the previous state-of-the-art result by 1.2%
(single model) and 3.8% (ensemble model). This demonstrates the effectiveness
of the usage of knowledge description to provide context.

Furthermore, we notice two trends based on the results. First, the underlying
pre-trained language model is important in commonsense QA quality. In general,
we observe this order of accuracy among these language models:
BERT$<$RoBERTa$<$XLNet$<$ALBERT$<$T5. Second, the additional knowledge module
is critical to provide external information for reasoning. For example,
RoBERTa+KEDGN outperforms the vanilla RoBERTa by 1.9%, and our model
outperforms the vanilla ALBERT model by 6.8% in accuracy.

OpenBookQA. Table 3 shows the test set accuracy on OpenBookQA. All results are
from single models. Note that the two best-performing models, i.e. UnifiedQA
(Khashabi et al., 2020) and TTTTT (Raffel et al., 2019), are based on the T5
generation model, with 11B and 3B parameters respectively. Thus, they are
computationally very expensive. Except these T5-based systems, DEKCOR achieves
the best accuracy among all baselines.

Ablation study. Table 4 shows that the usage of concept descriptions from
Wiktionary and triple from ConceptNet can help improve the accuracy of DEKCOR
on the dev set of CommonsenseQA by 2.7% and 4.4% respectively. We observe
similar results on OpenBookQA. This demonstrates that additional context
information can help with fusing knowledge graph into language modeling for
commonsense question answering.

CommonsenseQA Question:
---
Bats have many quirks, with the exception of  .
Question entity description:
bat: Any of the flying mammals of the order Chiroptera, usually small and
nocturnal, insectivorous or frugivorous.
Model w/o description chooses: eating bugs
Model w/ description chooses: laying eggs
OBQA Question:
Alligators  .
Question entity description:
alligator: Either of two species of large amphibious reptile, …, which have
sharp teeth and very strong jaws…
Model w/o description chooses: eat gar
Model w/ description chooses: are warm-blooded
Table 5: Examples from CommonsenseQA and OBQA dataset showing the
effectiveness of entity descriptions.

Case Study. Table 5 shows two examples from CommonsenseQA and OBQA
respectively. In the first example, without additional description the model
would not know relevant information about bats, like they are insectivorous,
leading to the wrong answer “eating bugs”. With the description, the model
knows that bats eat bugs, so it chooses “laying eggs” as the answer.
Similarly, for the second question, the “sharp teeth and very strong jaws” in
the description hint that alligators are likely carnivorous, and reptiles are
likely cold-blooded. The entity description leads to the correct answer of
“eat gar”.

##  5 Conclusions

In this paper, we propose to fuse context information into knowledge graphs
for commonsense question answering. As a knowledge graph often lacks
descriptions for the contained entities and relations, we leverage Wiktionary
to provide definitive text for each entity as additional input to the pre-
trained language model ALBERT. The resulting DEKCOR model achieves state-of-
the-art results on the benchmark datasets CommonsenseQA and OpenBookQA.
Ablation studies demonstrate the effectiveness of the proposed usage of
knowledge description and knowledge triple information in commonsense question
answering.

## Acknowledgements

We thank the anonymous reviewers for their valuable comments.

## References

  * Banerjee et al. (2019) Pratyay Banerjee, Kuntal Kumar Pal, A. Mitra, and Chitta Baral. 2019.  Careful selection of knowledge to solve open book question answering.  In _ACL_.
  * Chen et al. (2018) Muhao Chen, Yingtao Tian, Kai-Wei Chang, Steven Skiena, and Carlo Zaniolo. 2018\.  Co-training embeddings of knowledge graphs and entity descriptions for cross-lingual entity alignment.  _arXiv preprint arXiv:1806.06478_.
  * Chen et al. (2020) Qianglong Chen, Feng Ji, Haiqing Chen, and Yin Zhang. 2020.  Improving commonsense question answering by graph-based iterative retrieval over multiple knowledge sources.  In _Proceedings of the 28th International Conference on Computational Linguistics_ , pages 2583–2594, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  * Clark et al. (2019) Peter Clark, Oren Etzioni, Daniel Khashabi, Tushar Khot, Bhavana Dalvi Mishra, Kyle Richardson, Ashish Sabharwal, Carissa Schoenick, Oyvind Tafjord, Niket Tandon, et al. 2019.  From’f’to’a’on the ny regents science exams: An overview of the aristo project.  _arXiv preprint arXiv:1909.01958_.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Feng et al. (2020) Yanlin Feng, Xinyue Chen, Bill Yuchen Lin, Peifeng Wang, Jun Yan, and Xiang Ren. 2020.  Scalable multi-hop relational reasoning for knowledge-aware question answering.  _arXiv preprint arXiv:2005.00646_.
  * Honnibal and Montani (2017) Matthew Honnibal and Ines Montani. 2017.  spacy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing.  _To appear_ , 7(1).
  * Khashabi et al. (2020) Daniel Khashabi, Tushar Khot, Ashish Sabharwal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. 2020.  Unifiedqa: Crossing format boundaries with a single qa system.  _arXiv preprint arXiv:2005.00700_.
  * Lan et al. (2019) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019.  Albert: A lite bert for self-supervised learning of language representations.  _arXiv preprint arXiv:1909.11942_.
  * Lin et al. (2019) Bill Yuchen Lin, Xinyue Chen, Jamin Chen, and Xiang Ren. 2019.  Kagnet: Knowledge-aware graph networks for commonsense reasoning.  _arXiv preprint arXiv:1909.02151_.
  * Lin (2020) Jession Lin. 2020.  Knowledge chosen by relations.  https://github.com/jessionlin/csqa/blob/master/Model_details.md.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter. 2017.  Decoupled weight decay regularization.  _arXiv preprint arXiv:1711.05101_.
  * Lv et al. (2020) Shangwen Lv, Daya Guo, Jingjing Xu, Duyu Tang, Nan Duan, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, and Songlin Hu. 2020.  Graph-based reasoning over heterogeneous external knowledge for commonsense question answering.
  * Ma et al. (2019) Kaixin Ma, Jonathan Francis, Quanyang Lu, Eric Nyberg, and Alessandro Oltramari. 2019.  Towards generalizable neuro-symbolic systems for commonsense question answering.  _arXiv preprint arXiv:1910.14087_.
  * Mihaylov et al. (2018) Todor Mihaylov, Peter Clark, Tushar Khot, and A. Sabharwal. 2018.  Can a suit of armor conduct electricity? a new dataset for open book question answering.  In _EMNLP_.
  * Min et al. (2019) Sewon Min, Danqi Chen, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2019.  Knowledge guided text retrieval and reading for open domain question answering.  _arXiv preprint arXiv:1911.03868_.
  * Raffel et al. (2019) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _arXiv preprint arXiv:1910.10683_.
  * Sap et al. (2019) Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. 2019.  Atomic: An atlas of machine commonsense for if-then reasoning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 3027–3035.
  * Speer et al. (2017) Robyn Speer, Joshua Chin, and Catherine Havasi. 2017.  Conceptnet 5.5: An open multilingual graph of general knowledge.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 31.
  * Talmor et al. (2018) Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2018.  Commonsenseqa: A question answering challenge targeting commonsense knowledge.  _arXiv preprint arXiv:1811.00937_.
  * Wang et al. (2020) Peifeng Wang, Nanyun Peng, Pedro A. Szekely, and X. Ren. 2020.  Connecting the dots: A knowledgeable path generator for commonsense question answering.  In _EMNLP_.
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019.  Huggingface’s transformers: State-of-the-art natural language processing.  _ArXiv_ , pages arXiv–1910.
  * Xie et al. (2016) Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. 2016.  Representation learning of knowledge graphs with entity descriptions.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  pages 5753–5763.
  * Yu et al. (2020a) Donghan Yu, Chenguang Zhu, Yiming Yang, and Michael Zeng. 2020a.  Jaket: Joint pre-training of knowledge graph and language understanding.  _arXiv preprint arXiv:2010.00796_.
  * Yu et al. (2020b) Wenhao Yu, Chenguang Zhu, Zaitang Li, Zhiting Hu, Qingyun Wang, Heng Ji, and Meng Jiang. 2020b.  A survey of knowledge-enhanced text generation.  _arXiv preprint arXiv:2010.04389_.

##  Appendix A Implementation Details

Identification of $e_{q}$ and $e_{c}$. CommonsenseQA specifies the question
entity in each question and each answer choice is also an entity in
ConceptNet. We use them as $e_{q}$ and $e_{c}$. For OpenBookQA, we identify
all ConceptNet entities in the question and answer text and count their number
of occurrences in the retrieved text. For a triple $(e_{q},r,e_{c})$, we
define its weight as $n_{e_{q}}+n_{e_{c}}$, where $n_{e}$ is the number of
occurrences in retrieved text. The edge with the largest weight is picked. If
no edge is found between question and answer entities, we use the answer
entity with the most occurrences to find triples. For Wiktionary descriptions,
we find descriptions for $e_{q}$ and $e_{c}$ with the most occurrences as
well.

Using ConceptNet. Since ConceptNet contains a lot of weak relations, we only
use the following relations for our triples: CausesDesire, HasProperty,
CapableOf, PartOf, AtLocation, Desires, HasPrerequisite, HasSubevent, Antonym,
Causes.

Optimization. We use the AdamW (Loshchilov and Hutter, 2017) optimizer with a
learning rate of 2e-5. The batch size is 8. We limit the maximum length of the
input sequence to 192 tokens. The model is trained for 10 epochs. We use the
Huggingface (Wolf et al., 2019) implementation for the ALBERT model.

##  Appendix B Baseline Methods

GraphReason (Lv et al., 2020) retrieves knowledge from both structured
knowledge base and plain text.
PG-FULL (Wang et al., 2020) fine-tunes GPT-2 on ConceptNet to generate
knowledgeable paths between knowledge graph concepts.
UnifiedQA (Khashabi et al., 2020) pre-trains T5 on a variety of QA datasets
for general QA tasks.
MHGRN (Feng et al., 2020) adopts the multi-hop graph relation network to
perform reasoning.
HyKAS (Ma et al., 2019) employs an option comparison network to consume
ConceptNet triples.
ALBERT+KRD retrieves commonsense knowledge from Open Mind Common Sense and
then uses a self-attention module to compute a weighted sum of these triple
representations.
BERT + Selection Banerjee et al. (2019) improves the result on OpenBookQA via
abductive information retrieval , information gain based re-ranking, passage
selection and weighted scoring.
ALBERT+KB also improves retrieval results on OpenBookQA by token-based and
embedding-based retrieval. TTTTT (Raffel et al., 2019) fine-tunes the T5
language generation model on OpenBookQA.


Protecting Intellectual Property of Generative Adversarial Networks
from Ambiguity Attacks

Ding Sheng Ong^1       Chee Seng Chan^1Corresponding author, e-mail<EMAIL_ADDRESS>,       Kam Woh Ng^2
Lixin Fan^2
Qiang Yang^3

^1 University of Malaya

^2 WeBank

^3 Hong Kong University of Science and Technology

§ APPENDIX I - OVERVIEW OF THE VERIFICATION PROCESS IN MODEL PROTECTION

Generally, the verification process as shown in Fig. <ref>, a suspicious online model will be first remotely queried through API calls using a specific input keys (trigger set) that were initially selected to trigger the watermark information. As such, this is a black-box verification where a final model prediction (for CNN model, the image classification results) is obtained. This initial step is usually perform to collect evidence from everywhere so that an owner can sue a suspected party who used (infringed) his/her models illegally. Once the owner has sufficient evidence, a second verification process which is to extract watermark from the suspected model and compare if the watermark is from the owner. This process is a white-box verification, which means the owner needs to have to access the model physically, and usually this second step is go through law enforcement.

An overview of general watermarking scheme 2-steps verification process. In Step 1, it can be noticed that the protected model is trained to deliberately output specific (incorrect) labels for a particular set of inputs $T$ that is known as the "trigger set". In Step 2, the watermark is extracted from the model to proof ownership.

§ APPENDIX II - EXTENSION TO OTHER GENERATIVE MODELS - VARIATIONAL AUTOENCODER

As mentioned in the main paper, Section 3, line 246, our proposed framework can be easily extended to other deep generative models with trivial modification. Here we show an example with Variational Autoencoder (VAE). In general, VAE consists of 2 parts which are the probabilistic encoder, $q_{\psi}(\boldsymbol{z}|\boldsymbol{x})$ and the generative model, $p_{\theta}(\boldsymbol{x},\boldsymbol{z})$. The Encoder approximate the posterior in the form of multivariate Gaussian. The loss of a VAE is given below:

\begin{equation}
\mathcal{L}_{\text{VAE}} = \frac{1}{2}\left(1 + \log(\boldsymbol{\sigma^2})-\boldsymbol{\mu}^2-\boldsymbol{\sigma}^2\right) + \log(p_{\theta}(\boldsymbol{x}|\boldsymbol{z}))
\end{equation}

where $\boldsymbol{z}=\boldsymbol{\mu}+\boldsymbol{\sigma}\odot\boldsymbol{\epsilon}$ and $\boldsymbol{\epsilon}\sim \mathcal{N}(0, \boldsymbol{I})$. The input to the generative model is the posterior, $\boldsymbol{z}$ which is either from the probabilistic encoder or sampled from a multivariate Gaussian distribution, $\mathcal{N}(\boldsymbol{z};\boldsymbol{\mu},\boldsymbol{\sigma}\boldsymbol{I})$.

Since $\boldsymbol{z}$ is a vector of $n$ dimension and the output is an image (similar to DCGAN), we can use the exact same transformation functions defined in Section 3.1.1 (DCGAN) in the main paper, to create our trigger input using Eq. 1 and the target using Eq. 2, respectively. Thus, the overall new objective is shown as:

\begin{equation}
\mathcal{L}_{{\text{VAE}}_w}=\mathcal{L}_{\text{VAE}} + \lambda \mathcal{L}_{w}
\end{equation}

§.§ Experimental results

For the VAE experiment, we set $\lambda = 0.1$ unlike other GAN experiments in the main paper as the KLD loss and the reconstruction loss are in very small scale. We perform the experiment on the public dataset - CIFAR10, and the results are shown in Table <ref>.

FID     Q$_{wm}$

VAE     229.6874 $\pm$ 3.80     -

VAE$_w$     226.8893 $\pm$ 0.86     0.9973 $\pm$ 0.014

VAE$_{ws}$     231.1154 $\pm$ 0.63     0.9964 $\pm$ 0.014

VAE results on CIFAR10.

§ APPENDIX III - NETWORK ARCHITECTURE

§.§ DCGAN

Table <ref> - <ref> show the standard CNN models for CIFAR-10 and CUB-200 used in our experiments on image
generation ( DCGAN). The slopes of all LeakyReLU functions in the networks are set to 0.1.

$\mathit{z}$ $\in$ $\mathbb{R}^{128}$   $\mathcal{N}$(0,1)

dense $\to$ $M_g$ $\times$ $M_g$ $\times$ 512

4 $\times$ 4, stride=2 deconv. BN 256 ReLU

4 $\times$ 4, stride=2 deconv. BN 128 ReLU

4 $\times$ 4, stride=2 deconv. BN 64 ReLU

3 $\times$ 3, stride=2 conv. 3 Tanh

Generator, $M_g$ = 4 for CIFAR10 and $M_g$ = 8 for CUB-200.

RGB image $\mathit{x}$ $\in$ $\mathbb{R}^{M \times M \times 3}$

3 $\times$ 3, stride=1 conv. 64 LeakyReLU

4 $\times$ 4, stride=2 conv. 64 LeakyReLU

3 $\times$ 3, stride=1 conv. 128 LeakyReLU

4 $\times$ 4, stride=2 conv. 128 LeakyReLU

3 $\times$ 3, stride=1 conv. 256 LeakyReLU

4 $\times$ 4, stride=2 conv. 256 LeakyReLU

3 $\times$ 3, stride=1 conv. 512 LeakyReLU

dense $\to$ 1

Discriminator, $M$ = 32 for CIFAR10 and $M$ = 64 for CUB-200.

§.§ CycleGAN

Table <ref> - <ref> show the architecture for CycleGAN.

4cRGB Image     $128\times 128\times 3$

Conv.IN.ReLU     $7\times 7$     stride=1     padding=3     $128\times 128\times 64$

Conv.IN.ReLU     $3\times 3$     stride=2     padding=1     $64\times 64\times 128$

Conv.IN.ReLU     $3\times 3$     stride=2     padding=1     $32\times 32\times 256$

ResidualBlock     -     -     -     $32\times 32\times 256$

ResidualBlock     -     -     -     $32\times 32\times 256$

ResidualBlock     -     -     -     $32\times 32\times 256$

ResidualBlock     -     -     -     $32\times 32\times 256$

ResidualBlock     -     -     -     $32\times 32\times 256$

ResidualBlock     -     -     -     $32\times 32\times 256$

Deconv.In.ReLU     $3\times 3$     stride=2     padding=1     $64\times 64\times 128$

Deconv.In.ReLU     $3\times 3$     stride=2     padding=1     $128\times 128\times 64$

Conv.Tanh     $7\times 7$     stride=1     padding=3     $128\times 128\times 3$

ResNet Generator architecture of CycleGAN. Reflection Padding was used and all normalization layers are Instance Normalization according to the author's work.

4cRGB Image     $128\times 128\times 3$

Conv.lReLU     $4\times 4$     stride=2     padding=1     $64\times 64\times 64$

Conv.IN.lReLU     $4\times 4$     stride=2     padding=1     $32\times 32\times 128$

Conv.IN.lReLU     $4\times 4$     stride=2     padding=1     $16\times 16\times 256$

Conv.IN.lReLU     $4\times 4$     stride=2     padding=1     $8\times 8\times 512$

Conv     $4\times 4$     stride=2     padding=1     $4\times 4\times 1$

$70\times 70$ PatchGAN <cit.> was used as Discriminator of CycleGAN. Leaky ReLU with slope of 0.2 was used except the last layer.

§ APPENDIX IV - NETWORK COMPLEXITY

Table <ref> shows the computational complexity as a result of the additional of our regularization terms on GANs model. We observe that adding a new regularization term to embed watermark and signature has no effect to the inference time. As for training time, it is expected that it has an impact on training time but the effect is very minor. We believe that it is the computational cost at the inference stage that is required to be minimized, since network inference is going to be performed frequently by the end users. While extra costs at the training stage, on the other hand, are not prohibitive since they are performed by the network owners, with the motivation to protect the model ownerships.

Relative Time

DCGAN     1.00

DCGAN$_{w}$     1.25

DCGAN$_{ws}$     1.26

SRGAN     1.00

SRGAN$_{w}$     1.19

SRGAN$_{ws}$     1.23

CycleGAN     1.00

CycleGAN$_{w}$     1.15

CycleGAN$_{ws}$     1.17

The impact of the framework to the training time. The values in the table are relative to the baseline model.

CIFAR10: First row is the sample watermark logo. Second row is the images generated by DCGAN $G(z)$ ( original task) and the last row shows the watermarked images if a trigger input is provided to the protected DCGAN model where each of them is a different protected model trained on different response output set.

CUB-200: First row is the images generated by DCGAN $G(z)$ ( original task) and the second row shows the watermarked images if a trigger input is provided to the protected DCGAN model where each of them is a different protected model trained on different response output set.

The first column shows the trigger input $\vect{x}_\omega$ to SRGAN. Next three columns are the response output $G(\vect{x}_\omega)$ when the trigger input is provided to the protected generators .

Image pairs from CycleGAN models trained on Cityscapes datasets, respectively

Fine-tuning: For every pair, from left to right - $G(x_\omega)$ before and after fine-tuning. It shows that our proposed method is robust against removal attack (fine-tuning) as it is clearly noticed that the embedded watermark (top left corner) is still remained intact for DCGAN, SRGAN and CycleGAN.

3|c||E     3c||X     3c||A     3c||M     3c||P     3c||L     3c|E

$\gamma$     +/-     bit     $\gamma$     +/-     bit      $\gamma$     +/-     bit      $\gamma$     +/-     bit      $\gamma$     +/-     bit      $\gamma$     +/-     bit      $\gamma$     +/-     bit
-0.50     -     0     -0.22     -     0     -0.49     -     0     -0.24     -     0     -0.17     -     0     -0.44     -     0     -0.23     -     0
0.46     +     1      0.40     +     1      0.39     +     1      0.39     +     1      0.56     +     1      0.52     +     1      0.52     +     1
-0.42     -     0     -0.26     -     0     -0.44     -     0     -0.19     -     0     -0.17     -     0     -0.48     -     0     -0.28     -     0
-0.64     -     0      0.54     +     1     -0.17     -     0     -0.36     -     0      0.65     +     1     -0.62     -     0     -0.43     -     0
-0.25     -     0      0.43     +     1     -0.15     -     0      0.58     +     1     -0.53     -     0      0.37     +     1     -0.51     -     0
0.25     +     1     -0.14     -     0     -0.52     -     0      0.24     +     1     -0.56     -     0      0.49     +     1      0.22     +     1
-0.61     -     0     -0.45     -     0     -0.44     -     0     -0.18     -     0     -0.20     -     0     -0.47     -     0     -0.26     -     0
0.57     +     1     -0.34     -     0      0.35     +     1      0.55     +     1     -0.40     -     0     -0.55     -     0      0.32     +     1

Example of the trained batch normalization weight $\vect{\gamma}$ of DCGAN$_{ws}$ using the word "EXAMPLE" as an unique key. We use 8-bits to represent each character.

§ APPENDIX V - EXTENDED RESULTS

§.§ DCGAN

Fig. <ref> is the extended results of the original task of DCGAN ( image synthesis), as well as three different types of watermark logos with CIFAR10 dataset when a trigger input is provided, while Fig. <ref> illustrates the results with CUB-200 dataset.

§.§ SRGAN

For SRGAN, the extended results are shown in Fig. <ref>.

§.§ CycleGAN

For CycleGAN, the extended results are shown in Fig. <ref>.

Ambiguity attack - DCGAN: It can be seen that the quality of the image drop significantly when the sign of the $\gamma^{BN}$ of DCGAN$_{ws}$ is modified. Left to right: The amount (from 0% to 100%) of the sign is being modified.

Ambiguity attack - SRGAN: It can be seen that the quality of the images drop significantly when the sign of SRGAN$_{ws}$ is being modified. Left to right: The amount (from 10% to 100%) of the sign is being modified.

$\lambda = 0.1$

$\lambda = 0.5$

$\lambda = 1.0$

$\lambda = 5.0$

$\lambda = 10.0$

Effects of different $\lambda$ to original model performance (top) and quality of generated watermark (bottom).

§.§ Appendix VI - Robustness against removal attack

Fine-tuning. Fig. <ref> shows the qualitative results for Section 4.5, line 698 that our proposed method is robust against fine-tuning attack as highlighted in Table 7. We can clearly see that the watermark (on the top left corner) remains intact after fine-tuning.

§.§ Resilience against ambiguity attack

This section shows the full results of applying sign loss (Eq. 11 in the main paper) to embed a signature into BN-scale, $\gamma^{BN}$. The implementation details is given in Section 3.2 of the main paper. Herein, we show the example of how the unique key  "EXAMPLE" is embedded into our DCGAN's batch normalization weight. Table <ref> shows how to decode the trained scale, $\gamma^{BN}$ to retrieve the signature embedded. Also, please note that even that there are 2 "E", their $\gamma^{BN}$ are different from each other.

Fig. <ref> - <ref> are the complete results to complement Figure 9 - 10 in Section 4.6. It can be clearly visualize from Fig. <ref> that the quality of the generated SR-images is very poor where obvious artefact can be observed even the signature signs are modified at only 10%. With this, we can deduce that the scale signs (Eq. 11) enforced in this way remain rather persistent against ambiguity attacks.

§ APPENDIX VII - ABLATION STUDY

§.§ Coefficient $\lambda$.

In this section, qualitatively, we illustrate in Fig. <ref> the effects of different $\lambda$ settings ( from 0.1 $\to$ 10) on the original GAN model performance against the quality of the generated watermark (this is similar to Table 10) with CIFAR10 dataset. From visual inspection on Fig. <ref>, it is hard to deduce that which $\lambda$ is an ideal choice. In this paper, based on the quantitative results (FID vs. SSIM) in Table 10, we set $\lambda = 1.0$.

§.§ $n$ vs. $c$.

In this section, qualitatively, we illustrate in Fig. <ref> - <ref> different $n$ and $c$ settings to understand the tradeoffs between the original GAN model performance against the quality of the generated watermark (this is similar to Table 11) with CIFAR10 dataset. From here, it can be noticed that it is, however, very hard to distinguish from a naked eye point of view which setting is having the best tradeoffs with the exception that it is very clear that setting $c = 0$ is not ideal. This is because from Fig. <ref>a-c, we can notice that the generated images are all almost black ( for $n = 5, 10, 15$). This phenomenon happens because the training input of DCGAN has a normal distribution of $\mu = 0$, therefore it is conflicting with the trigger input which is also set as $c = 0$.

As a summary, trigger input set must have a very different distribution from the training data. In this paper, based on the quantitative results (FID vs. SSIM) reported in Table 7, we conclude that setting $n = 5$ and $c = -10$ is the most ideal.

$n = 5$ ; $c = -10$

$n = 10$ ; $c = -10$

$n = 15$ ; $c = -10$

$n = 5$ ; $c = -5$

$n = 10$ ; $c = -5$

$n = 15$ ; $c = -5$

Effects of different $n$ and $c$ to original model performance (top) and quality of generated watermark (bottom) (cont. in Fig. <ref>).

$n = 5$ ; $c = 0$

$n = 10$ ; $c = 0$

$n = 15$ ; $c = 0$

$n = 5$ ; $c = 5$

$n = 10$ ; $c = 5$

$n = 15$ ; $c = 5$

$n = 5$ ; $c = 10$

$n = 10$ ; $c = 10$

$n = 15$ ; $c = 10$

(cont.) Effects of different $n$ and $c$ to original model performance (top) and quality of generated watermark (bottom).