w/o answer-aware loss& 41.09 & 68.68 & 47.75 w/o diversified contexts & 40.53 & 68.52 & 47.66

In order to validate the effectiveness of model components, we remove some important components in our model, including context copy, KB copy, answer-aware loss and diversified contexts. The results are shown in Table \[tab: Ablation\]. We can see that removing any component brings performance decline on all metrics. It demonstrates that all these components are useful. Specifically, the last line in Table \[tab: Ablation\], replacing diversified contexts with contexts used in , has more obvious performance degradation.

Performances on Naturalness {#subsec:Human Evaluation}
---------------------------

[p[2.8cm]{}&lt;p[2.5cm]{}&lt;]{} Model & Naturalness & 2.96 & 2.23 Our Model$_{\rm ans\_loss}$ & **3.56**

Human evaluation is important for generated questions. Following , we sample 100 questions from each system, and then two annotators measure the naturalness by a score of 0-5. The Kappa coefficient for inter-annotator is 0.629, and p-value for all scores is less than 0.005. As shown in Table \[tab: naturalness\], perform poorly on naturalness, while our model obtains the highest score on naturalness, which demonstrates our model can deliver more natural questions than baselines.

Discussion
----------

### Without Pre-trained KB Embeddings {#subsec:Pre-trained KB}

[p[2.58cm]{}&lt;p[1.0cm]{}&lt;p[1.13cm]{}&lt;p[1.33cm]{}&lt;p[1.36cm]{}&lt;]{} Model& TransE & 