In this paper, we explore \textit{summary-to-article generation}: the task of generating long articles given a short summary, which provides finer-grained content control for the generated text. To prevent sequence-to-sequence (seq2seq) models from degenerating into language models and better controlling the long text to be generated, we propose a hierarchical generation approach which first generates a sketch of intermediate length based on the summary and then completes the article by enriching the generated sketch. To mitigate the discrepancy between the ``oracle'' sketch used during training and the noisy sketch generated during inference, we propose an end-to-end joint training framework based on multi-agent reinforcement learning. For evaluation, we use text summarization corpora by reversing their inputs and outputs, and introduce a novel evaluation method that employs a summarization system to summarize the generated article and test its match with the original input summary. Experiments show that our proposed hierarchical generation approach can generate a coherent and relevant article based on the given summary, yielding significant improvements upon conventional seq2seq models. In contrast to the well-explored text generation tasks like machine translation (Bahdanau et al., 2014) and text summarization (See et al., 2017) , open-ended long text generation is much less explored. The existing studies on long text generation either generate long text unconditionally, such as GPT-2 (Radford et al.), or generate long text conditioning on a single sentence prompt (Fan et al., 2018; Keskar et al., 2019; Zellers et al., 2019) . Although they can generate seemingly fluent text in a general domain/topic, they suffer from a lack of fine-grained control of content to be generated, which may result in generating much undesirable text and make them difficult to use in practice. In this paper, we study long text generation with fine-grained content control. We explore summary-toarticle generation: the task of generating a coherent and relevant long article based on a short summary of 3 to 5 sentences which summarizes the main content of the article to be generated. Compared to the previously studied unconditional or prompt-based long text generation, summary-to-article generation specifies the content to be generated more clearly, leading to finer-grained control of text generation. As prior work (Fan et al., 2018) points out, however, it remains challenging to generate a long, coherent and relevant article based on a summary because complex and underspecified dependencies between the summary and the article are much harder to model than the closer dependencies required for language modeling, which makes standard seq2seq models prone to degenerating into language models, neglecting salient information provided in the summary and resulting in undesirable outputs. To address this challenge, inspired by previous work that attempts to generate text with multiple steps (Dalianis & Hovy, 1993; Reiter & Dale, 2000) , we propose a hierarchical summary-to-article generation approach which decomposes the task into two subtasks: summary-to-sketch generation and sketch-to-article generation. As illustrated in Figure 1 , the sketch is of an intermediate length and serves as a draft of the output article to be generated and outlines its main content, which resembles how people plan to write long articles in their mind. This hierarchical generation approach avoids the need for seq2seq text generation models to extend the length of source text too much, thus alleviating the aforementioned degenerating problem and enhancing the coherence and relevance of the generated text. To bridge the gap between training and inference, which arises from the discrepancy between the extracted "oracle sketch" used during training and the noisy sketch generated during inference, we propose a gated model fusion mechanism Figure 1: Illustration of the proposed hierarchical generation model for summary-to-article generation: (a) The conventional seq2seq model generates an article directly based on the summary; (b) Our proposed hierarchical summary-to-article generation approach first generates a sketch based on the summary and then completes the article based on the generated sketch. which establishes a skip-connection from the input summary to the sketch-to-article generation model, and jointly train the summary-to-article and the sketch-to-article generation model to communicate and cooperate with each other in an end-to-end fashion with multi-agent reinforcement learning. For evaluation, we use the text summarization corpora by reversing their inputs and outputs as our dataset. We also introduce a novel evaluation metric -ROUGE-rec which calculates how much the original summary can be reconstructed from the generated article. Experiments on the CNN/DM and BIGPATENT datasets demonstrate our proposed hierarchical generation approach can generate fluent, coherent and relevant articles based on a given summary, yielding better results than conventional seq2seq models. Our contributions are threefold: • We explore the task of summary-to-article generation which provides finer-grained control of generated long text and propose a hierarchical summary-to-article generation approach. • We propose a gated model fusion mechanism and a multi-agent reinforcement learning with denoising seq2seq pretraining objective to help bridge the gap between training and inference of the hierarchical generation model. • We propose a novel evaluation method for summary-to-article generation. Experimental results demonstrate that the proposed evaluation metric correlates better with human evaluation than the traditional metrics like perplexity for this task. We explore the task of summary-to-article generation and propose a novel hierarchical summary-toarticle generation approach. The approach first drafts a sketch that outlines the article to be generated based on the summary, then generates the article based on information in the summary and the sketch. We propose an end-to-end joint training framework through the multi-agent reinforcement learning to train the hierarchical model and evaluate its performance in multiple datasets. The experimental results show that our approach can generate a coherent and relevant article based on a given summary, outperforming the conventional seq2seq models for summary-to-article generation. However, since the input summary only contains a subset of the information of the article, a summary-to-article generation model will tend to generate fabricated content by filling in the rest of the narrative, which may arise ethical concerns. We will investigate the characteristic of additional information included in the generated articles and seek approaches to control them in our future work. (2019) pretrains a conditional language model to generate fake news based on given one-sentence headline which specifies the topic of the generated news. Keskar et al. (2019) pretrains a conditional langauge model to generate contents in the domain specified by a "domain code". Although they can generate seemingly fluent text in a general domain/topic, they suffer from a lack of fine-grained control of content to be generated, which may result in generating much undesirable text and make them difficult to use in practice. Decomposing text generation Decomposing text generation into several steps has been explored in both statistical template-based text generation approaches (Wahlster et al., 1993; Dalianis & Hovy, 1993) and neural text generation models (Fan et al., 2018; Xu et al., 2018) . Strategies for decomposing long text generation have been explored by transferring sentence compression, text summarization, and keyword extraction models to build an outline for guiding long text generation models. However, these approaches are built for generating a short "pseudo-summary" unconditionally or based on a single sentence. The intermediary training data for these approaches is thus generated and noisy. In addition, previous work on decomposing text generation generally either constructs intermediary output of roughly the same length of the final output (Fan et al., 2019; Xu et al., 2018) , or generates a very short "plan" in a higher level (Yao et al., 2019) . As a result, these approaches do not address the major difficulty of long text generation, which is the large difference of the length of input and output text in the seq2seq model. Indeed,the hierarchical model of Xu et al. (2018) and Yao et al. (2019) is designed for generate sentences and short stories within 50 words, and the hierarchical model of Fan et al. (2018) only generate a single sentence prompt. More recently, Fan et al. (2019) propose to first generate an action plan, then generate a anonymized story and fill in the entities in the last step. Their deconposition method extend the length of sequence in the first step, thus is orthogonal to our proposed method. Our decomposition approach is different from the aforementioned approaches in two perspectives: 1) the sketches used in our work are extracted with the guidance from both the article and the summary, thus of much better quality, and 2) we construct sketches of intermediate length, thus providing more adequate information for generation final output and reducing the difficulty of expanding the length of input by a large ratio in one pass. Denoising pretraining for seq2seq models Pretraining a denoising autoencoder for text generation is explored in recent works (Edunov et al., 2018; Lample et al., 2017; Wang et al., 2019; Zhao et al., 2019) . The motivation of their approaches is to tackle the data sparsity problem while we employ the denoising objective for training the sketch-to-article generation model to be better adapted to generated sketches. As a result, the corruption methods in our work are different and our model is trained to directly output the target sequence instead of reconstructing the original input. Learning to communicate and cooperate between multiple agents The idea of training multiple agents to communicate and cooperate with each other for accomplishing a common goal is well explored in multi-agent reinforcement learning literatures (Lowe et al., 2017; Foerster et al., 2018; Das et al., 2017) . The most similar work to ours is that of Lee et al. (2019) , which pretrains two translation models of Fr-En and En-De respectively, and train them to perform Fr-De translation cooperatively with reinforcement learning. Model fusion Previous work has investigated the integration of language models with seq2seq models and the fusion of two identical seq2seq models. Gulcehre et al. (2015) combined a trained language model with a trained seq2seq model to learn a gating function that joins them. Sriram et al. (2017) propose training the seq2seq model given the fixed language model then learning a gate to filter the information from the language model. Fan et al. (2018) propose to train another identical seq2seq model based on a pretrained seq2seq model. To our knowledge, our work is the first to investigate the fusion of two seq2seq model with different input and the same output to combine different source of information and prevent error-propagation.
we explore the task of summary-to-article generation and propose a hierarchical generation scheme together with a jointly end-to-end reinforcement learning framework to train the hierarchical model. To address the issue of degeneration in summary-to-article generation, this paper proposes a hierarchical generation approach which first generates an intermediate sketch of the article and then the full article.