Sorbonne Université, CNRS, ISIR, Paris, France; Sorbonne Université, CNRS, ISIR, Paris, France; Meta AI; Sorbonne Université, CNRS, ISIR, Paris, France; Valeo.ai
Alexandre Rame, Guillaume Couairon, Corentin Dancette, Jean-Baptiste Gaya, Mustafa Shukor, Laure Soulier, Matthieu Cord
Deep learning;Foundation models;Fine-tuning;Reward optimization;Linear mode connectivity;Weight averaging;Model soups;Robustness;Generalization;Alignment;Multi objective learning.
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards