word embedding for the token (green units), $$e_i = \textnormal{concat}(d_i, a_i, x_i)
\vspace{-0.1cm}$$

[**Encoder**]{} As for the encoder itself, we use bidirectional LSTMs to read the input $\mathbf{e} = (e_1,...,e_n)$ in both the forward and backward directions. After encoding, we obtain two sequences of hidden vectors, namely, $\overrightarrow{\mathbf{h}} = (\overrightarrow{h_1},..., \overrightarrow{h_n})$ and $\overleftarrow{\mathbf{h}} = (\overleftarrow{h_1},..., \overleftarrow{h_n})$. The final output state of the encoder is the concatenation of $\overrightarrow{\mathbf{h}}$ and $\overleftarrow{\mathbf{h}}$ where $$h_i = \textnormal{concat}(\overrightarrow{h_i}, \overleftarrow{h_i})
\vspace{-0.1cm}$$

[**Question Decoder with Attention & Copy**]{} On top of the feature-rich encoder, we use LSTMs with attention [@bahdanau2014neural] as the decoder for generating the question $y_1,..., y_m$ one token at a time. To deal with rare/unknown words, the decoder also allows directly copying words from the source sentence via pointing [@vinyals2015pointer].

At each time step $t$, the decoder LSTM reads the previous word embedding $w_{t-1}$ and previous hidden state $s_{t-1}$ to compute the new hidden state, $$s_t = \textnormal{LSTM} (w_{t-1}, s_{t-1})
    \vspace{-0.1cm}$$ Then we calculate the [*attention distribution*]{} $\alpha_t$ as in , $$\begin{gathered}
    e_{t,i} = h_i^{T}\mathbf{W}_c s_{t-1} \\
    \alpha_{t} = \textnormal{softmax}(e_{t})
\end{gathered}$$ where $\mathbf{W}_c$ is a weight matrix and attention distribution $\alpha_t$ is a probability distribution over the source sentence words. With $\alpha_t$, we 