Universitat Pompeu Fabra, Barcelona, Spain; Universitat de Barcelona, Barcelona, Spain
Language Models;word frequency;pre-training corpus;low-resource languages
Frequency Balanced Datasets Lead to Better Language Models