copy mechanism (KB copy and context copy), which allows copying from KBs and textual contexts. Specifically, we first map the input of the decoder into an embedding representation by looking up word embedding matrix, then we use position embedding [@NIPS2017_7181] to enhance sequential information. Compared with the transformer encoder in Sec. \[subsec:ctx\_enc\], transformer decoder has an extra sub-layer: a fact multi-head attention layer, which is similar to Equation \[eq:subspace\]-\[eq:FFN\], where the query matrix is initiated with previous decoder sub-layer while both the key matrix and the value matrix are the augmented fact representation $\textbf{H}_{f}$. After feedforward and multiple transformer layers, we obtain the decoder state $\textbf{s}_t$ at time step $t$, and then $\textbf{s}_t$ could be leveraged to generate the target question sequence word by word.

As depicted in Figure \[fig:overall-structure\], we propose multi-level copy mechanism to generate question words. At each time step $t$, given decoder state $\textbf{s}_t$ together with input fact $F$, textual contexts $C$ and vocabulary $V$, the probabilistic function for generating any target question word $y_t$ is calculated as: $$\begin{gathered}
    \begin{split}
        P(y_{t}|\textbf{s}_t, \! y_{t-1}, \! F, \! C) \! = \! p_{genv} P_{genv}(y_{t}|\textbf{s}_t, \! V) +\!\\
        p_{cpkb} P_{cpkb}(y_{t}|\textbf{s}_t, \!F) \! + \! p_{cpctx} P_{cpctx}(y_{t}|\textbf{s}_t,\! C) \label{eq:yt}
    \end{split} 