Indian Institute of Technology Bombay, India; University of Texas at Dallas, USA; Media and Data Science Research (MDSR) Lab, Adobe Inc., India
Efficient training;Language Models;Data selection
INGENIOUS: Using Informative Data Subsets for Efficient Pre-Training of Language Models