In this paper, we design a generic framework for learning a robust text classification model that achieves accuracy comparable to standard full models under test-time
 budget constraints. We take a different approach from existing methods and learn to dynamically delete a large fraction of unimportant words by a low-complexity selector such that the high-complexity classifier only needs to process a small fraction of important words. In addition, we propose a new data aggregation method to train the classifier, allowing it to make accurate predictions even on fragmented sequence of words. Our end-to-end method achieves state-of-the-art performance while its computational complexity scales linearly with the small fraction of important words in the whole corpus. Besides, a single deep neural network classifier trained by our framework can be dynamically tuned to different budget levels at inference time. Recent advances in deep neural networks (DNN) has improved the performance of natural language processing tasks such as document classification, question answering, and sentiment analysis BID29 BID20 BID22 BID31 . These approaches process the entire text and construct representations of words and phrases in order to perform target tasks. While these models do realize high accuracy, their computational-time scales linearly with the size of the documents, which can be slow for documents containing many sentences. In this context, various approaches based on modifying the existing RNN or LSTM architecture have been proposed BID21 ; BID31 to speed-up processing. However, processing is still fundamentally sequential, which in turn requires loading entire documents to process, limiting compute gains. We proposed a budgeted learning framework for learning a robust classifier under test-time budget constraints. We demonstrated that training classifiers with data aggregation work well with low-complexity selectors based on word-embedding or bag-of-word model and achieve good performance with fragmented input. The future work includes applying the proposed framework to other text reading tasks and improving the data aggregation strategy by applying learning to search approaches BID4 .
Modular framework for document classification and data aggregation technique for making the framework robust to various distortion, and noise and focus only on the important words.  The authors consider training a RNN-based text classification where there is a resource restriction on test-time prediction, and provide an approach using a masking mechanism to reduce words/phrases/sentences used in prediction followed by a classifier to handle those components.