The results have a few issues which make evaluating the contribution difficult:
1. The paper lacks a comparison with some existing works, particularly methods involve iterative PPO/DPO method that train a reward model simultaneously and reward ensembles [1].
[1] Coste T, Anwar U, Kirk R, et al. Reward model ensembles help mitigate overoptimization.
2. The alignment of relabeled reward data with human annotator judgments remains insufficiently validated.
2. The alignment of relabeled reward data with human annotator judgments remains insufficiently validated.