InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic
Reward Modeling

InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling

14 February 2024

Liang Ding

Papers citing "InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modeling"

14 / 14 papers shown

Title
Reasoning without Regret Tarun Chitra OffRL LRM 23 0 0 14 Apr 2025
Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanations Pedro Ferreira Wilker Aziz Ivan Titov LRM 26 0 0 07 Apr 2025
The Mind in the Machine: A Survey of Incorporating Psychological Theories in LLMs Zizhou Liu Ziwei Gong Lin Ai Zheng Hui Run Chen Colin Wayne Leach Michelle R. Greene Julia Hirschberg LLMAG 55 0 0 28 Mar 2025
Reward Shaping to Mitigate Reward Hacking in RLHF Jiayi Fu Xuandong Zhao Chengyuan Yao H. Wang Qi Han Yanghua Xiao 80 6 0 26 Feb 2025
Learning to Substitute Words with Model-based Score Ranking Hongye Liu Ricardo Henao 36 0 0 09 Feb 2025
The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hacking Yuchun Miao Sen Zhang Liang Ding Yuqi Zhang L. Zhang Dacheng Tao 81 3 0 31 Jan 2025
Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases Ziyi Zhang Sen Zhang Yibing Zhan Yong Luo Yonggang Wen Dacheng Tao EGVM 22 8 0 13 Feb 2024
ODIN: Disentangled Reward Mitigates Hacking in RLHF Lichang Chen Chen Zhu Davit Soselia Jiuhai Chen Tianyi Zhou Tom Goldstein Heng-Chiao Huang M. Shoeybi Bryan Catanzaro AAML 42 51 0 11 Feb 2024
WARM: On the Benefits of Weight Averaged Reward Models Alexandre Ramé Nino Vieillard Léonard Hussenot Robert Dadashi Geoffrey Cideron Olivier Bachem Johan Ferret 92 92 0 22 Jan 2024
Intention Analysis Makes LLMs A Good Jailbreak Defender Yuqi Zhang Liang Ding Lefei Zhang Dacheng Tao LLMSV 14 15 0 12 Jan 2024
Uncertainty-Penalized Reinforcement Learning from Human Feedback with Diverse Reward LoRA Ensembles Yuanzhao Zhai Han Zhang Yu Lei Yue Yu Kele Xu Dawei Feng Bo Ding Huaimin Wang AI4CE 55 31 0 30 Dec 2023
Improving Generalization of Alignment with Human Preferences through Group Invariant Learning Rui Zheng Wei Shen Yuan Hua Wenbin Lai Shihan Dou ... Xiao Wang Haoran Huang Tao Gui Qi Zhang Xuanjing Huang 52 14 0 18 Oct 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 273 1,561 0 18 Sep 2019