can obtain the context vector $h^{*}_t$, $$h^{*}_t = \sum_{i=1}^{n} \alpha_t^{i}h_i
    \vspace{-0.1cm}$$ Then, using the context vector $h_t^{*}$ and hidden state $s_t$, the probability distribution over the target (question) side vocabulary is calculated as, $$P_{vocab} = \textnormal{softmax}(\mathbf{W}_d \textnormal{concat}(h_t^{*}, s_t))$$ Instead of directly using $P_{vocab}$ for training/generating with the fixed target side vocabulary, we also consider [*copying*]{} from the source sentence. The copy probability is based on the context vector $h_t^{*}$ and hidden state $s_t$, $$\lambda_{t}^{copy} = \sigma \left( \mathbf{W}_e h^{*}_t + \mathbf{W}_f s_t\right)
    \vspace{-0.1cm}$$ and the probability distribution over the source sentence words is the sum of the attention scores of the corresponding words, $$P_{copy}(w) = \sum_{i=1}^{n} \alpha_t^{i} * \mathds{1}\{w == w_i\}
\vspace{-0.2cm}$$ Finally, we obtain the probability distribution over the dynamic vocabulary (i.e., union of original target side and source sentence vocabulary) by summing over $P_{copy}$ and $P_{vocab}$, $$P(w) =  \lambda_t^{copy} P_{copy}(w) + (1 -\lambda_t^{copy}) P_{vocab}(w)
    $$ where $\sigma$ is the sigmoid function, and $\mathbf{W}_d$, $\mathbf{W}_e$, $\mathbf{W}_f$ are weight matrices.

Answer Span Identification
--------------------------

We frame the problem of identifying candidate answer spans from a paragraph as a sequence labeling task and base our model on the BiLSTM-CRF approach for named entity recognition [@huang2015bidirectional]. Given a paragraph of $n$ tokens, 