for that of $\textbf{x}^{p}$ (predicate context) or $\textbf{x}^{o}$ (object context). Based on the query matrix, transformer encoder works as follows: $$\begin{gathered}
\footnotesize
    \textbf{Q}_{j}=\textbf{QW}_{j}^{Q}, \textbf{K}_{j}=\textbf{KW}_{j}^{K}, \textbf{V}_{j}=\textbf{VW}_{j}^{V} \label{eq:subspace} \\
        head_{j} = {\rm softmax} (\textbf{Q}_{j}\textbf{K}_{j}^{T} / \sqrt{d/h})\textbf{V}_{j} \label{eq:head} \\
    \textbf{H}^{s}={\rm Concat}(head_1,head_2,...,head_{h})\textbf{W}_{0} \label{eq:concat} \\
    \textbf{N}^{s} = {\rm LayerNorm}(\textbf{Q} + \textbf{H}^{s}) \label{eq:layernorm} \\
    \textbf{C}^{s} = {\rm max}(0, \textbf{N}^{s} \textbf{W}_{1} + b_{1}) \textbf{W}_{2} + b_{2} \label{eq:FFN}\end{gathered}$$ where $\textbf{K}$ and $\textbf{V}$ are the key matrix and value matrix, respectively. It is called self-attention because $\textbf{K}$ and $\textbf{V}$ are equal to the query matrix $\textbf{Q} \in \mathbb{R} ^{|s|, d}$ in the encoding stage, where $d$ represents the number of hidden units. And $h$ denotes the number of the heads in multi-head attention mechanism of the transformer encoder. It first projects the input matrixes ($\textbf{Q}$, $\textbf{K}$, $\textbf{V}$) into subspaces $h$ times mapped by different linear projections $\textbf{W}_{j}^{Q}$, $\textbf{W}_{j}^{K}$, $\textbf{W}_{j}^{V} \in \mathbb{R} ^{|s|, d/h}$ ($j=1,2,...,h$) in Equation \[eq:subspace\]. And then $h$ projections perform the scaled dot-product attention to obtain the representation of each head in parallel (Equation \[eq:head\]). Representations for all parallel heads are concatenated together in Equation \[eq:concat\]. After residual connection, layer normalization (Equation \[eq:layernorm\]) and feed forward operation (Equation \[eq:FFN\]), we can obtain the subject context matrix $\textbf{C}^{s}=\{\textbf{c}_{1}^{s},\textbf{c}_{2}^{s},...,\textbf{c}_{|s|}^{s}\} 