Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning
from Human Feedback

Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback

8 October 2023

Wei Shen

Xuanjing Huang

Papers citing "Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback"

14 / 14 papers shown

Title
Beyond Reward Hacking: Causal Rewards for Large Language Model Alignment Chaoqi Wang Zhuokai Zhao Yibo Jiang Zhaorun Chen Chen Zhu ... Jiayi Liu Lizhu Zhang Xiangjun Fan Hao Ma Sinong Wang 70 3 0 17 Jan 2025
An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems Hashmath Shaik Alex Doboli OffRL ELM 98 0 0 31 Dec 2024
Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknown Xingzhou Lou Dong Yan Wei Shen Yuzi Yan Jian Xie Junge Zhang 45 21 0 01 Oct 2024
RRM: Robust Reward Model Training Mitigates Reward Hacking Tianqi Liu Wei Xiong Jie Jessie Ren Lichang Chen Junru Wu ... Yuan Liu Bilal Piot Abe Ittycheriah Aviral Kumar Mohammad Saleh AAML 52 12 0 20 Sep 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 50 13 0 24 Jun 2024
Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators Yann Dubois Balázs Galambosi Percy Liang Tatsunori Hashimoto ALM 53 315 0 06 Apr 2024
ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedback Zhenyu Hou Yiin Niu Zhengxiao Du Xiaohan Zhang Xiao Liu ... Qinkai Zheng Minlie Huang Hongning Wang Jie Tang Yuxiao Dong ALM 22 17 0 01 Apr 2024
A Survey on Causal Reinforcement Learning Yan Zeng Ruichu Cai Fuchun Sun Libo Huang Z. Hao CML 26 27 0 10 Feb 2023
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 225 500 0 28 Sep 2022
Defining and Characterizing Reward Hacking Joar Skalse Nikolaus H. R. Howe Dmitrii Krasheninnikov David M. Krueger 57 53 0 27 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 275 1,583 0 18 Sep 2019
Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedback Carolin (Haas) Lawrence Stefan Riezler OffRL 171 56 0 03 May 2018