the negative log-likelihood for the entire correct sequence: $-\log(p_\mathbf{y})$, $$p_\mathbf{y} = \frac{\textnormal{exp}(q(\mathbf{x}^{'},\mathbf{y}))}{\sum_{\mathbf{y}^{'} \in \mathbf{Y}^{'}}\textnormal{exp}(q(\mathbf{x}^{'},\mathbf{y}^{'}))}$$ where $q(\mathbf{x}^{'},\mathbf{y}) = \sum_{t=1}^{n} P_{t,y_t} + \sum_{t=0}^{n-1}A_{y_t,y_{t+1}}$, $P_{t,y_t}$ is the score of assigning tag $y_t$ to the $t^{th}$ token, and $A_{y_t,y_{t+1}}$ is the transition score from tag $y_t$ to $y_{t+1}$, the scoring matrix $A$ is to be learned. $\mathbf{Y}^{'}$ represents all the possible tagging sequences.

Experiments
===========

Dataset {#sec:dataset}
-------

We use the SQuAD dataset [@rajpurkar2016squad] to train our models. It is one of the largest general purpose QA datasets derived from Wikipedia with over 100k questions posed by crowdworkers on a set of Wikipedia articles. The answer to each question is a segment of text from the corresponding Wiki passage. The crowdworkers were users of Amazon’s Mechanical Turk located in the US or Canada. To obtain high-quality articles, the authors sampled 500 articles from the top 10,000 articles obtained by Nayuki’s Wikipedia’s internal PageRanks. The question-answer pairs were generated by annotators from a paragraph; and although the dataset is typically used to evaluate reading comprehension, it has also been used in an open domain QA setting [@chen2017reading; @wang2018r3]. For training/testing answer extraction systems, we pair each paragraph in the dataset with the gold answer spans 