RLCD: Reinforcement Learning from Contrastive Distillation for Language Model Alignment

24 July 2023

Papers citing "RLCD: Reinforcement Learning from Contrastive Distillation for Language Model Alignment"

29 / 29 papers shown

Title
OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses Angela Lopez-Cardona Sebastian Idesis Miguel Barreda-Ángeles Sergi Abadal Ioannis Arapakis 46 0 0 13 Mar 2025
Understanding Synthetic Context Extension via Retrieval Heads Xinyu Zhao Fangcong Yin Greg Durrett 36 0 0 31 Dec 2024
Jailbreak Instruction-Tuned LLMs via end-of-sentence MLP Re-weighting Yifan Luo Zhennan Zhou Meitan Wang Bin Dong 14 0 0 14 Oct 2024
MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization Yougang Lyu Lingyong Yan Zihan Wang Dawei Yin Pengjie Ren Maarten de Rijke Z. Z. Ren 55 6 0 10 Oct 2024
Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models Angela Lopez-Cardona Carlos Segura Alexandros Karatzoglou Sergi Abadal Ioannis Arapakis ALM 48 2 0 02 Oct 2024
Boosting Reward Model with Preference-Conditional Multi-Aspect Synthetic Data Generation Jiaming Shen Ran Xu Yennie Jun Zhen Qin Tianqi Liu Carl Yang Yi Liang Simon Baumgartner Michael Bendersky SyDa 53 4 0 22 Jul 2024
Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning Jifan Zhang Lalit P. Jain Yang Guo Jiayi Chen Kuan Lok Zhou ... Scott Sievert Timothy Rogers Kevin Jamieson Robert Mankoff Robert Nowak 29 5 0 15 Jun 2024
Language Models Resist Alignment Jiaming Ji Kaile Wang Tianyi Qiu Boyuan Chen Jiayi Zhou Changye Li Hantao Lou Yaodong Yang 34 1 0 10 Jun 2024
Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study Shusheng Xu Wei Fu Jiaxuan Gao Wenjie Ye Weiling Liu Zhiyu Mei Guangju Wang Chao Yu Yi Wu 33 131 0 16 Apr 2024
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences Corby Rosset Ching-An Cheng Arindam Mitra Michael Santacroce Ahmed Hassan Awadallah Tengyang Xie 144 113 0 04 Apr 2024
Reward Generalization in RLHF: A Topological Perspective Tianyi Qiu Fanzhi Zeng Jiaming Ji Dong Yan Kaile Wang Jiayi Zhou Yang Han Josef Dai Xuehai Pan Yaodong Yang AI4CE 17 2 0 15 Feb 2024
V-STaR: Training Verifiers for Self-Taught Reasoners Arian Hosseini Xingdi Yuan Nikolay Malkin Aaron C. Courville Alessandro Sordoni Rishabh Agarwal ReLM LRM 35 99 0 09 Feb 2024
Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulation Xianghe Pang Shuo Tang Rui Ye Yuxin Xiong Bolun Zhang Yanfeng Wang Siheng Chen 114 27 0 08 Feb 2024
Weaver: Foundation Models for Creative Writing Tiannan Wang Jiamin Chen Qingrui Jia Shuai Wang Ruoyu Fang ... Xiaohua Xu Ningyu Zhang Huajun Chen Yuchen Eleanor Jiang Wangchunshu Zhou 17 18 0 30 Jan 2024
Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Loss Jing Xu Andrew Lee Sainbayar Sukhbaatar Jason Weston 8 86 0 27 Dec 2023
Alleviating Hallucinations of Large Language Models through Induced Hallucinations Yue Zhang Leyang Cui Wei Bi Shuming Shi HILM 34 49 0 25 Dec 2023
Reasons to Reject? Aligning Language Models with Judgments Weiwen Xu Deng Cai Zhisong Zhang Wai Lam Shuming Shi ALM 16 13 0 22 Dec 2023
Safe RLHF: Safe Reinforcement Learning from Human Feedback Josef Dai Xuehai Pan Ruiyang Sun Jiaming Ji Xinbo Xu Mickel Liu Yizhou Wang Yaodong Yang 11 279 0 19 Oct 2023
Gaining Wisdom from Setbacks: Aligning Large Language Models via Mistake Analysis Kai Chen Chunwei Wang Kuo Yang Jianhua Han Lanqing Hong ... Zhenguo Li Dit-Yan Yeung Lifeng Shang Xin Jiang Qun Liu 35 24 0 16 Oct 2023
SALMON: Self-Alignment with Instructable Reward Models Zhiqing Sun Yikang Shen Hongxin Zhang Qinhong Zhou Zhenfang Chen David D. Cox Yiming Yang Chuang Gan ALM SyDa 11 35 0 09 Oct 2023
A Long Way to Go: Investigating Length Correlations in RLHF Prasann Singhal Tanya Goyal Jiacheng Xu Greg Durrett 34 139 0 05 Oct 2023
Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models Hritik Bansal John Dang Aditya Grover ALM 14 20 0 30 Aug 2023
ZYN: Zero-Shot Reward Models with Yes-No Questions for RLAIF Víctor Gallego SyDa 35 4 0 11 Aug 2023
A Comprehensive Overview of Large Language Models Humza Naveed Asad Ullah Khan Shi Qiu Muhammad Saqib Saeed Anwar Muhammad Usman Naveed Akhtar Nick Barnes Ajmal Saeed Mian OffRL 46 499 0 12 Jul 2023
Learning by Distilling Context Charles Burton Snell Dan Klein Ruiqi Zhong ReLM LRM 151 44 0 30 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
CIC: Contrastive Intrinsic Control for Unsupervised Skill Discovery Michael Laskin Hao Liu Xue Bin Peng Denis Yarats Aravind Rajeswaran Pieter Abbeel SSL 74 65 0 01 Feb 2022
Understanding Deep Contrastive Learning via Coordinate-wise Optimization Yuandong Tian 52 34 0 29 Jan 2022
Challenges in Detoxifying Language Models Johannes Welbl Amelia Glaese J. Uesato Sumanth Dathathri John F. J. Mellor Lisa Anne Hendricks Kirsty Anderson Pushmeet Kohli Ben Coppin Po-Sen Huang LM&MA 242 191 0 15 Sep 2021