Self-Augmented Preference Optimization: Off-Policy Paradigms for
Language Model Alignment

Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment

31 May 2024

Mingyuan Zhou

Papers citing "Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment"

9 / 9 papers shown

Title
VideoSAVi: Self-Aligned Video Language Models without Human Supervision Yogesh Kulkarni Pooyan Fazli VLM 90 2 0 01 Dec 2024
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences Corby Rosset Ching-An Cheng Arindam Mitra Michael Santacroce Ahmed Hassan Awadallah Tengyang Xie 144 113 0 04 Apr 2024
LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement Nicholas Lee Thanakul Wattanawong Sehoon Kim K. Mangalam Sheng Shen Gopala Anumanchipalli Michael W. Mahoney Kurt Keutzer A. Gholami 58 12 0 22 Mar 2024
Step-On-Feet Tuning: Scaling Self-Alignment of LLMs via Bootstrapping Haoyu Wang Guozheng Ma Ziqiao Meng Zeyu Qin Li Shen ... Liu Liu Yatao Bian Tingyang Xu Xueqian Wang Peilin Zhao 55 12 0 12 Feb 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 159 437 0 02 Feb 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 224 291 0 18 Jan 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 275 1,561 0 18 Sep 2019
The Woman Worked as a Babysitter: On Biases in Language Generation Emily Sheng Kai-Wei Chang Premkumar Natarajan Nanyun Peng 204 607 0 03 Sep 2019