Robust Preference Optimization through Reward Model Distillation

29 May 2024

Papers citing "Robust Preference Optimization through Reward Model Distillation"

6 / 6 papers shown

Title
CREAM: Consistency Regularized Self-Rewarding Language Models Z. Wang Weilei He Zhiyuan Liang Xuchao Zhang Chetan Bansal Ying Wei Weitong Zhang Huaxiu Yao ALM 50 7 0 16 Oct 2024
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization Noam Razin Sadhika Malladi Adithya Bhaskar Danqi Chen Sanjeev Arora Boris Hanin 53 10 0 11 Oct 2024
Direct Preference Optimization with an Offset Afra Amini Tim Vieira Ryan Cotterell 42 22 0 16 Feb 2024
WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 72 43 0 22 Jan 2024
Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles Yuanzhao Zhai Han Zhang Yu Lei Yue Yu Kele Xu Dawei Feng Bo Ding Huaimin Wang AI4CE 37 18 0 30 Dec 2023
COMBO: Conservative Offline Model-Based Policy Optimization Tianhe Yu Aviral Kumar Rafael Rafailov Aravind Rajeswaran Sergey Levine Chelsea Finn OffRL 167 338 0 16 Feb 2021