Knowledge-grounded dialogue is a task of generating an informative response based on both discourse context and external knowledge. As we focus on better modeling the knowledge selection in the multi-turn knowledge-grounded dialogue, we propose a sequential latent variable model as the first approach to this matter. The model named sequential knowledge transformer (SKT) can keep track of the prior and posterior distribution over knowledge; as a result, it can not only reduce the ambiguity caused from the diversity in knowledge selection of conversation but also better leverage the response information for proper choice of knowledge. Our experimental results show that the proposed model improves the knowledge selection accuracy and subsequently the performance of utterance generation. We achieve the new state-of-the-art performance on Wizard of Wikipedia (Dinan et al., 2019) as one of the most large-scale and challenging benchmarks. We further validate the effectiveness of our model over existing conversation methods in another knowledge-based dialogue Holl-E dataset (Moghe et al., 2018). Knowledge-grounded dialogue is a task of generating an informative response based on both discourse context and selected external knowledge (Ghazvininejad et al., 2018) . For example, it is more descriptive and engaging to respond "I've always been more of a fan of the American football team from Pittsburgh, the Steelers!" than "Nice, I like football too.". As it has been one of the key milestone tasks in conversational research (Zhang et al., 2018) , a majority of previous works have studied how to effectively combine given knowledge and dialogue context to generate an utterance (Zhang et al., 2018; Li et al., 2019b; Parthasarathi & Pineau, 2018; Madotto et al., 2018; Gopalakrishnan et al., 2019) . Recently, Dinan et al. (2019) proposed to tackle the knowledge-grounded dialogue by decomposing it into two sub-problems: first selecting knowledge from a large pool of candidates and generating a response based on the selected knowledge and context. In this work, we investigate the issue of knowledge selection in the multi-turn knowledge-grounded dialogue, since practically the selection of pertinent topics is critical to better engage humans in conversation, and technically the utterance generation becomes easier with a more powerful and consistent knowledge selector in the system. Especially, we focus on developing a sequential latent variable model for knowledge selection, which has not been discussed in previous research. We believe it brings several advantages for more engaging and accurate knowledge-based chit-chat. First, it can correctly deal with the diversity in knowledge selection of conversation. Since one can choose any knowledge to carry on the conversation, there can be one-to-many relations between dialogue context and knowledge selection. Such multimodality by nature makes the training of a dialogue system much more difficult in a data-driven way. However, if we can sequentially model the history of knowledge selection in previous turns, we can reduce the scope of probable knowledge candidates at current turn. Second, the sequential latent model can better leverage the response information, which makes knowledge selection even more accurate. It is naturally easy to select the knowledge in the pool once the response is known, because the response is generated based on the selected knowledge. Our sequential model can keep track of prior and posterior distribution over knowledge, which are sequentially updated considering the responses in previous turns, and thus we can better predict the knowledge by sampling from the posterior. Third, the latent model works even when the knowledge selection labels for previous dialogue are not available, which is common (Dinan et al., 2019) . Table 1 : Accuracy of knowledge selection with and without knowing the response. We test with GRU (Cho et al., 2014) , Transformer (Vaswani et al., 2017) and BERT (Devlin et al., 2019) as the sentence encoder. For human evaluation, we randomly sample 20 dialogues and ask human annotators to select the most likely knowledge sentence from the pool. Finally, the contributions of this work are as follows. 1. We propose a novel model named sequential knowledge transformer (SKT). To the best of our knowledge, our model is the first attempt to leverage a sequential latent variable model for knowledge selection, which subsequently improves knowledge-grounded chit-chat. 2. Our experimental results show that the proposed model improves not only the knowledge selection accuracy but also the performance of utterance generation. As a result, we achieve the new state-of-the-art performance on Wizard of Wikipedia (Dinan et al., 2019 ) and a knowledge-annotated version of Holl-E (Moghe et al., 2018) dataset. This work investigated the issue of knowledge selection in multi-turn knowledge-grounded dialogue, and proposed a sequential latent variable model, for the first time, named sequential knowledge transformer (SKT). Our method achieved the new state-of-the-art performance on the Wizard of Wikipedia benchmark (Dinan et al., 2019) and a knowledge-annotated version of Holl-E dataset (Moghe et al., 2018) . There are several promising future directions beyond this work. First, we can explore other inference models such as sequential Monte Carlo methods using filtering variational objectives (Maddison et al., 2017a) . Second, we can study the interpretability of knowledge selection such as measuring the uncertainty of attention (Heo et al., 2018) .
Our approach is the first attempt to leverage a sequential latent variable model for knowledge selection in the multi-turn knowledge-grounded dialogue. It achieves the new state-of-the-art performance on Wizard of Wikipedia benchmark. A sequential latent variable model for knowledge selection in dialogue generation that extends the posterior attention model to the latent knowledge selection problem and achieves higher performances than previous state-of-the-art models. A novel architecture for selecting knowledge-grounded multi-turn dialogue that yields state of the art on relevant benchmarks datasets, and scores higher in human evaluations.