Google Research India; Google Deepmind; Mila – Quebec AI Institute
pre-training;multilingual pretraining;self-influence;language models;data reweighting
Self-Influence Guided Data Reweighting for Language Model Pre-training