\in \mathbb{R} ^{|s|, d}$.

Similarly, $\textbf{C}^{p}$ and $\textbf{C}^{o}$ are obtained from the same transformer encoder for the predicate and object, respectively.

Fact Encoder {#subsec:fact_enc}
------------

In contrast to general Sequence-to-Sequence (Seq2Seq) model [@sutskever2014sequence], the input fact is not a word sequence but instead a structured triplet $F=(s,p,o)$. We employ a fact encoder to transform each atom in the fact into a fixed embedding, and the embedding is obtained from a KB embedding matrix. For example, the subject embedding $\textbf{e}^{s} \in \mathbb{R} ^{d}$ is looked up from the KB embedding matrix $\textbf{E}_{f} \in \mathbb{R}^{k,d}$, where $k$ represents the size of KB vocabulary, and the size of KB embedding is equal to the number of hidden units ($d$) in Equation \[eq:head\]. Similarly, the predicate embedding $\textbf{e}^{p}$ and the object embedding $\textbf{e}^{o}$ are mapped from the KB embedding matrix $\textbf{E}_{f}$, where $\textbf{E}_{f}$ is pre-trained using *TransE* [@NIPS2013_5071] to capture much more fact information in previous work [@N18-1020]. In our model, $\textbf{E}_{f}$ can be pre-trained or randomly initiated (Details in Sec. \[subsec:Pre-trained KB\]).

Context-Augmented Fact Encoder {#subsec:ctx_fact_enc}
------------------------------

In order to combine both the context encoder information and the fact encoder information, we propose a context-augmented fact encoder which applies the gated fusion unit 