In this paper, we extend the persona-based sequence-to-sequence (Seq2Seq) neural network conversation model to a multi-turn dialogue scenario by modifying the state-of-the-art hredGAN architecture to simultaneously capture utterance attributes such as speaker identity, dialogue topic, speaker sentiments and so on. The proposed system, phredGAN has a persona-based HRED generator (PHRED) and a conditional discriminator. We also explore two approaches to accomplish the conditional discriminator: (1) $phredGAN_a$, a system that passes the attribute representation as an additional input into a traditional adversarial discriminator, and (2) $phredGAN_d$, a dual discriminator system which in addition to the adversarial discriminator, collaboratively predicts the attribute(s) that generated the input utterance. To demonstrate the superior performance of phredGAN over the persona SeqSeq model, we experiment with two conversational datasets, the Ubuntu Dialogue Corpus (UDC) and TV series transcripts from the Big Bang Theory and Friends. Performance comparison is made with respect to a variety of quantitative measures as well as crowd-sourced human evaluation. We also explore the trade-offs from using either variant of $phredGAN$ on datasets with many but weak attribute modalities (such as with Big Bang Theory and Friends) and ones with few but strong attribute modalities (customer-agent interactions in Ubuntu dataset). Recent advances in machine learning especially with deep neural networks has lead to tremendous progress in natural language processing and dialogue modeling research BID13 BID14 BID10 . Nevertheless, developing a good conversation model capable of fluent interaction between a human and a machine is still in its infancy stage. Most existing work relies on limited dialogue history to produce response with the assumption that the model parameters will capture all the modalities within a dataset. However, this is not true as dialogue corpora tend to be strongly multi-modal and practical neural network models find it difficult to disambiguate characteristics such as speaker personality, location and sub-topic in the data.Most work in this domain has primarily focused on optimizing dialogue consistency. For example, Serban et al. BID10 BID12 a) and BID15 introduced a Hierarchical Recurrent Encoder-Decoder (HRED) network architecture that combines a series of recurrent neural networks to capture long-term context state within a dialogue. However, the HRED system suffers from lack of diversity and does not have any guarantee on the generator output since the output conditional probability is not calibrated. BID8 tackles these problems by training a modified HRED generator alongside an adversarial discriminator in order to increase diversity and provide a strong and calibrated guarantee to the generator's output. While the hredGAN system improves upon response quality, it does not capture speaker and other attributes modality within a dataset and fails to generate persona specific responses in datasets with multiple modalities.On the other hand, there has been some recent work on introducing persona into dialogue models. For example, BID5 integrates attribute embeddings into a single turn (Seq2Seq) generative dialogue model. In this work, Li et al. consider persona models one with Speaker-only representation and the other with Speaker and Addressee representations (Speaker-Addressee model), both of which capture certain speaker identity and interactions. BID7 continue along the Figure 1 : The PHRED generator with local attention -The attributes C, allows the generator to condition its response on the utterance attributes such as speaker identity, subtopics and so on. same line of thought by considering a Seq2Seq dialogue model with Responder-only representation. In both of these cases, the attribute representation is learned during the system training. BID16 proposed a slightly different approach. Here, the attributes are a set of sentences describing the profile of the speaker. In this case, the attributes representation is not learned. The system however learns how to attend to different parts of the attributes during training. Still, the above persona-based models have limited dialogue history (single turn); suffer from exposure bias worsening the trade-off between personalization and conversation quality and cannot generate multiple responses given a dialogue context. This is evident in the relatively short and generic responses produced by these systems, even though they generally capture the persona of the speaker.In order to overcome these limitations, we propose two variants of an adversarially trained persona conversational generative system, phredGAN , namely phredGAN a and phredGAN d . Both systems aim to maintain the response quality of hredGAN and still capture speaker and other attribute modalities within the conversation. In fact, both systems use the same generator architecture (PHRED generator), i.e., an hredGAN generator BID8 with additional utterance attribute representation at its encoder and decoder inputs as depicted in Figure 1 . Conditioning on external attributes can be seen as another input modality as is the utterance into the underlying system. The attribute representation is an embedding that is learned together with the rest of model parameters similar to BID5 . Injecting attributes into a multi-turn dialogue system allows the model to generate responses conditioned on particular attribute(s) across conversation turns. Since the attributes are discrete, it also allows for exploring different what-if scenarios of model responses. The difference between the two systems is in the discriminator architecture based on how the attribute is treated.We train and sample both variants of phredGAN similar to the procedure for hredGAN BID8 . To demonstrate model capability, we train on a customer service related data such as the Ubuntu Dialogue Corpus (UDC) that is strongly bimodal between question poser and answerer, and transcripts from a multi-modal TV series The Big Bang Theory and Friends with quantitative and qualitative analysis. We examine the trade-offs between using either system in bi-modal or multi-modal datasets, and demonstrate system superiority over state-of-the-art persona conversational models in terms of dialogue response quality and quantitatively with perplexity, BLEU, ROUGE and distinct n-gram scores. In this paper, we improve upon state-of-the-art persona-based response generation models by exploring two persona conversational models: phredGAN a which passes the attribute representation as an additional input into a traditional adversarial discriminator, and phredGAN d a dual discriminator system which in addition to the adversarial discriminator from hredGAN , collaboratively predicts the attribute(s) that are intrinsic to the input utterance. Both systems demonstrate quantitative improvements upon state-of-the-art persona conversational systems such as the work from BID5 with respect to both quantitative automatic and qualitative human measures.Our analysis also demonstrates how both variants of phredGAN perform differently on datasets with weak and strong modality. One of our future direction is to take advantage of phredGAN d 's ability to predict utterance attribute such as speaker identity from just the utterance. We believe its performance can be improved even with weak modality by further conditioning adversarial updates on both the attribute and adversarial discriminator accuracies. Overall, this paper demonstrates clear benefits from adversarial training of persona generative dialogue system and leaves the door open for more interesting work to be accomplished in this domain. (Xi, Ci) with N utterances. Each utterance mini batch i contains Mi word tokens.
This paper develops an adversarial learning framework for neural conversation models with persona This paper proposes an extension to hredGAN to simultaneously learn a set of attribute embeddings that represent the persona of each speaker and generate persona-based responses