RRHF: Rank Responses to Align Language Models with Human Feedback without tears

11 April 2023

Zheng Yuan

Hongyi Yuan

Chuanqi Tan

Papers citing "RRHF: Rank Responses to Align Language Models with Human Feedback without tears"

32 / 282 papers shown

Title
Making Large Language Models Better Reasoners with Alignment Peiyi Wang Lei Li Liang Chen Feifan Song Binghuai Lin Yunbo Cao Tianyu Liu Zhifang Sui ALM LRM 28 64 0 05 Sep 2023
Efficient RLHF: Reducing the Memory Usage of PPO Michael Santacroce Yadong Lu Han Yu Yuan-Fang Li Yelong Shen 27 27 0 01 Sep 2023
Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models Hritik Bansal John Dang Aditya Grover ALM 27 20 0 30 Aug 2023
Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities Maximilian Mozes Xuanli He Bennett Kleinberg Lewis D. Griffin 31 76 0 24 Aug 2023
From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Models Jing Yao Xiaoyuan Yi Xiting Wang Jindong Wang Xing Xie ALM 14 42 0 23 Aug 2023
#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models K. Lu Hongyi Yuan Zheng Yuan Runji Lin Junyang Lin Chuanqi Tan Chang Zhou Jingren Zhou ALM LRM 27 63 0 14 Aug 2023
A Preliminary Study of the Intrinsic Relationship between Complexity and Alignment Ying Zhao Yu Bowen Binyuan Hui Haiyang Yu Fei Huang Yongbin Li N. Zhang 39 22 0 10 Aug 2023
Wider and Deeper LLM Networks are Fairer LLM Evaluators Xinghua Zhang Yu Bowen Haiyang Yu Yangyu Lv Tingwen Liu Fei Huang Hongbo Xu Yongbin Li ALM 40 83 0 03 Aug 2023
Scaling Relationship on Learning Mathematical Reasoning with Large Language Models Zheng Yuan Hongyi Yuan Cheng Li Guanting Dong Keming Lu Chuanqi Tan Chang Zhou Jingren Zhou LRM ALM 25 159 0 03 Aug 2023
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback Stephen Casper Xander Davies Claudia Shi T. Gilbert Jérémy Scheurer ... Erdem Biyik Anca Dragan David M. Krueger Dorsa Sadigh Dylan Hadfield-Menell ALM OffRL 39 470 0 27 Jul 2023
PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback Bo Shen Jiaxin Zhang Taihong Chen Daoguang Zan Bing Geng ... Ailun Yu Jichuan Ji Jingyang Zhao Yuenan Guo Qianxiang Wang ALM ELM 25 73 0 27 Jul 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Saeed Mian OffRL 46 523 0 12 Jul 2023
Hierarchical Pretraining for Biomedical Term Embeddings Bryan Cai Sihang Zeng Yucong Lin Zheng Yuan D. Zhou Lu Tian 22 0 0 01 Jul 2023
Preference Ranking Optimization for Human Alignment Feifan Song Yu Bowen Minghao Li Haiyang Yu Fei Huang Yongbin Li Houfeng Wang ALM 21 235 0 30 Jun 2023
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards Alexandre Ramé Guillaume Couairon Mustafa Shukor Corentin Dancette Jean-Baptiste Gaya Laure Soulier Matthieu Cord MoMe 35 135 0 07 Jun 2023
Fine-Tuning Language Models with Advantage-Induced Policy Alignment Banghua Zhu Hiteshi Sharma Felipe Vieira Frujeri Shi Dong Chenguang Zhu Michael I. Jordan Jiantao Jiao OSLM 23 39 0 04 Jun 2023
Fine-Grained Human Feedback Gives Better Rewards for Language Model Training Zeqiu Wu Yushi Hu Weijia Shi Nouha Dziri Alane Suhr Prithviraj Ammanabrolu Noah A. Smith Mari Ostendorf Hannaneh Hajishirzi ALM 25 303 0 02 Jun 2023
Training Socially Aligned Language Models on Simulated Social Interactions Ruibo Liu Ruixin Yang Chenyan Jia Ge Zhang Denny Zhou Andrew M. Dai Diyi Yang Soroush Vosoughi ALM 18 44 0 26 May 2023
On Learning to Summarize with Large Language Models as References Yixin Liu Kejian Shi Katherine S He Longtian Ye Alexander R. Fabbri Pengfei Liu Dragomir R. Radev Arman Cohan ELM 18 68 0 23 May 2023
Aligning Large Language Models through Synthetic Feedback Sungdong Kim Sanghwan Bae Jamin Shin Soyoung Kang Donghyun Kwak Kang Min Yoo Minjoon Seo ALM SyDa 73 67 0 23 May 2023
Making Language Models Better Tool Learners with Execution Feedback Shuofei Qiao Honghao Gui Chengfei Lv Qianghuai Jia Huajun Chen Ningyu Zhang LLMAG 34 46 0 22 May 2023
Evaluation of medium-large Language Models at zero-shot closed book generative question answering René Peinl Johannes Wirth ELM 18 7 0 19 May 2023
AutoTrial: Prompting Language Models for Clinical Trial Design Zifeng Wang Cao Xiao Jimeng Sun LM&MA LRM 6 15 0 19 May 2023
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation Patrick Fernandes Aman Madaan Emmy Liu António Farinhas Pedro Henrique Martins ... José G. C. de Souza Shuyan Zhou Tongshuang Wu Graham Neubig André F. T. Martins ALM 113 56 0 01 May 2023
WizardLM: Empowering Large Language Models to Follow Complex Instructions Can Xu Qingfeng Sun Kai Zheng Xiubo Geng Pu Zhao Jiazhan Feng Chongyang Tao Daxin Jiang ALM 29 901 0 24 Apr 2023
RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment Hanze Dong Wei Xiong Deepanshu Goyal Yihan Zhang Winnie Chow Rui Pan Shizhe Diao Jipeng Zhang Kashun Shum Tong Zhang ALM 11 399 0 13 Apr 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 298 4,077 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 306 11,909 0 04 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 211 1,656 0 15 Oct 2021
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning Runxin Xu Fuli Luo Zhiyuan Zhang Chuanqi Tan Baobao Chang Songfang Huang Fei Huang LRM 136 178 0 13 Sep 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,453 0 23 Jan 2020
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 275 1,587 0 18 Sep 2019