Q: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
The Copyright of this work is owned by the Association for Computational Linguistics (ACL). However, each of the authors and the employers for whom the work was performed reserve all other rights, specifically including the following: ... (4) The right to make copies of the work for internal distribution within the author’s organization and for external distribution as a preprint, reprint, technical report, or related class of document. The original paper is to appear in: ACL 2014: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics We introduce a novel approach for building language models based on a systematic, recursive exploration of skip n-gram models which are interpolated using modified KneserNey smoothing. Our approach generalizes language models as it contains the classical interpolation with lower order models as a special case. In this paper we motivate, formalize and present our approach. In an extensive empirical experiment over English text corpora we demonstrate that our generalized language models lead to a substantial reduction of perplexity between 3.1% and 12.7% in comparison to traditional language models using modified Kneser-Ney smoothing. Furthermore, we investigate the behaviour over three other languages and a domain specific corpus where we observed consistent improvements. Finally, we also show that the strength of our approach lies in its ability to cope in particular with sparse training data. Using a very small training data set of only 736 KB text we yield improvements of even 25.7% reduction of perplexity.
A:
A Generalized Language Model as the Combination of Skipped n-grams and Modified Kneser-Ney Smoothing