Right Now, Wrong Then: Non-Stationary Direct Preference Optimization
under Preference Drift

Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift

26 July 2024

Sayak Ray Chowdhury

Ilija Bogunovic

Papers citing "Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift"

12 / 12 papers shown

Title
Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts Haoxiang Wang Wei Xiong Tengyang Xie Han Zhao Tong Zhang 44 13 0 18 Jun 2024
Group Robust Preference Optimization in Reward-free RLHF Shyam Sundhar Ramesh Yifan Hu Iason Chaimalas Viraj Mehta Pier Giuseppe Sessa Haitham Bou-Ammar Ilija Bogunovic 14 13 0 30 May 2024
Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferences Corby Rosset Ching-An Cheng Arindam Mitra Michael Santacroce Ahmed Hassan Awadallah Tengyang Xie 144 113 0 04 Apr 2024
RewardBench: Evaluating Reward Models for Language Modeling Nathan Lambert Valentina Pyatkin Jacob Morrison Lester James Validad Miranda Bill Yuchen Lin ... Sachin Kumar Tom Zick Yejin Choi Noah A. Smith Hanna Hajishirzi ALM 62 210 0 20 Mar 2024
Direct Preference Optimization with an Offset Afra Amini Tim Vieira Ryan Cotterell 68 54 0 16 Feb 2024
KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh Winnie Xu Niklas Muennighoff Dan Jurafsky Douwe Kiela 153 437 0 02 Feb 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 215 291 0 18 Jan 2024
Sample Efficient Preference Alignment in LLMs via Active Exploration Viraj Mehta Vikramjeet Das Ojash Neopane Yijia Dai Ilija Bogunovic Ilija Bogunovic W. Neiswanger Stefano Ermon Jeff Schneider Willie Neiswanger OffRL 25 12 0 01 Dec 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Weighted Linear Bandits for Non-Stationary Environments Yoan Russac Claire Vernade Olivier Cappé 75 100 0 19 Sep 2019
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 273 1,561 0 18 Sep 2019
Time-Varying Gaussian Process Bandit Optimization Ilija Bogunovic Jonathan Scarlett V. Cevher 47 82 0 25 Jan 2016