mention-pair score introduced from the external system [@clark2016improving] helps the neural network better encode coreference knowledge.

[l|lll]{} & BLEU-3 & BLEU-4 & METEOR\

  ----------------
  Seq2seq + copy
  (w/ ans.)
  ----------------

  : Evaluation results for question generation on the portion that requires coreference knowledge (36.42% examples of the original test set).[]{data-label="tab:qg-portion"}

& 17.81 & 12.30 & 17.11\
ContextNQG & 18.05 & 12.53 & 17.33\
CorefNQG & **18.46** & **12.96** & **17.58**\

To better understand the effect of coreference resolution, we also evaluate our model and the baseline models on just that portion of the test set that requires pronoun resolution (36.42% of the examples) and show the results in Table \[tab:qg-portion\]. The gaps of performance between our model and the baseline models are still significant. Besides, we see that all three systems’ performance drop on this partial test set, which demonstrates the hardness of generating questions for the cases that require pronoun resolution (passage context).

We also show in Table \[tab:qg\] the results of the QG models trained on the training set augmented with noisy examples with predicted answer spans. There is a consistent but acceptable drop for each model on this new training set, given the inaccuracy of predicted answer 