LEATHER: A Framework for Learning to Generate Human-like Text in Dialogue

Algorithms for text-generation in <PRE_TAG>dialogue</POST_TAG> can be misguided. For example, in task-oriented settings, reinforcement learning that optimizes only <PRE_TAG>task-success</POST_TAG> can lead to abysmal lexical diversity. We hypothesize this is due to poor theoretical understanding of the objectives in text-generation and their relation to the learning process (i.e., model training). To this end, we propose a new theoretical framework for learning to generate text in <PRE_TAG>dialogue</POST_TAG>. Compared to existing theories of learning, our framework allows for analysis of the multi-faceted goals inherent to text-generation. We use our framework to develop theoretical guarantees for learners that adapt to unseen data. As an example, we apply our theory to study data-shift within a cooperative learning algorithm proposed for the GuessWhat?! visual <PRE_TAG>dialogue</POST_TAG> game. From this insight, we propose a new algorithm, and empirically, we demonstrate our proposal improves both <PRE_TAG>task-success</POST_TAG> and human-likeness of the generated text. Finally, we show statistics from our theory are empirically predictive of multiple qualities of the generated <PRE_TAG>dialogue</POST_TAG>, suggesting our theory is useful for model-selection when human evaluations are not available.