ODIN: Disentangled Reward Mitigates Hacking in RLHF

ODIN: Disentangled Reward Mitigates Hacking in RLHF

11 February 2024

Heng-Chiao Huang

Bryan Catanzaro

Papers citing "ODIN: Disentangled Reward Mitigates Hacking in RLHF"

11 / 11 papers shown

Title
Energy-Based Reward Models for Robust Language Model Alignment Anamika Lochab Ruqi Zhang 36 0 0 17 Apr 2025
Adversarial Training of Reward Models Alexander Bukharin Haifeng Qian Shengyang Sun Adithya Renduchintala Soumye Singhal Z. Wang Oleksii Kuchaiev Olivier Delalleau T. Zhao AAML 27 0 0 08 Apr 2025
Probabilistic Uncertain Reward Model Wangtao Sun Xiang Cheng Xing Yu Haotian Xu Zhao Yang Shizhu He Jun Zhao Kang Liu 56 0 0 28 Mar 2025
Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown Xingzhou Lou Dong Yan Wei Shen Yuzi Yan Jian Xie Junge Zhang 41 21 0 01 Oct 2024
RRM: Robust Reward Model Training Mitigates Reward Hacking Tianqi Liu Wei Xiong Jie Jessie Ren Lichang Chen Junru Wu ... Yuan Liu Bilal Piot Abe Ittycheriah Aviral Kumar Mohammad Saleh AAML 47 11 0 20 Sep 2024
On the Transformations across Reward Model, Parameter Update, and In-Context Prompt Deng Cai Huayang Li Tingchen Fu Siheng Li Weiwen Xu ... Leyang Cui Yan Wang Lemao Liu Taro Watanabe Shuming Shi KELM 21 2 0 24 Jun 2024
Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators Yann Dubois Balázs Galambosi Percy Liang Tatsunori Hashimoto ALM 25 314 0 06 Apr 2024
WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 92 92 0 22 Jan 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Unsolved Problems in ML Safety Dan Hendrycks Nicholas Carlini John Schulman Jacob Steinhardt 156 268 0 28 Sep 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 273 1,561 0 18 Sep 2019