Inria, Paris, France; Inria, Paris, France; Sorbonne Université, Paris, France
Nathan Godey, Éric Clergerie, Benoît Sagot
representation learning;NLP;language modeling;pretraining;contrastive
Headless Language Models: Learning without Predicting with Contrastive Weight Tying