Title
Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approach Jiancong Xiao Bojian Hou Zhanliang Wang Ruochen Jin Q. Long Weijie Su Li Shen 32 0 0 04 May 2025
Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning Kai Ye Hongyi Zhou Jin Zhu Francesco Quinzan C. Shi 32 1 0 03 Apr 2025
Can RLHF be More Efficient with Imperfect Reward Models? A Policy Coverage Perspective Jiawei Huang Bingcong Li Christoph Dann Niao He OffRL 85 0 0 26 Feb 2025
What is the Alignment Objective of GRPO? Milan Vojnovic Se-Young Yun 70 2 0 25 Feb 2025
Stackelberg Game Preference Optimization for Data-Efficient Alignment of Language Models Xu Chu Zhixin Zhang Tianyu Jia Yujie Jin 77 0 0 25 Feb 2025
Improving LLM General Preference Alignment via Optimistic Online Mirror Descent Yuheng Zhang Dian Yu Tao Ge Linfeng Song Zhichen Zeng Haitao Mi Nan Jiang Dong Yu 60 1 0 24 Feb 2025
Faster WIND: Accelerating Iterative Best-of- $N$ Distillation for LLM Alignment Tong Yang Jincheng Mei H. Dai Zixin Wen Shicong Cen Dale Schuurmans Yuejie Chi Bo Dai 45 4 0 20 Feb 2025
Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF Shicong Cen Jincheng Mei Katayoon Goshvadi Hanjun Dai Tong Yang Sherry Yang Dale Schuurmans Yuejie Chi Bo Dai OffRL 65 23 0 20 Feb 2025
Multi-Step Alignment as Markov Games: An Optimistic Online Gradient Descent Approach with Convergence Guarantees Yongtao Wu Luca Viano Yihang Chen Zhenyu Zhu Kimon Antonakopoulos Quanquan Gu V. Cevher 54 0 0 18 Feb 2025
Preference learning made easy: Everything should be understood through win rate Lily H. Zhang Rajesh Ranganath 85 0 0 14 Feb 2025
Incentivize without Bonus: Provably Efficient Model-based Online Multi-agent RL for Markov Games Tong Yang Bo Dai Lin Xiao Yuejie Chi OffRL 64 2 0 13 Feb 2025
Nearly Optimal Sample Complexity of Offline KL-Regularized Contextual Bandits under Single-Policy Concentrability Qingyue Zhao Kaixuan Ji Heyang Zhao Tong Zhang Q. Gu OffRL 45 0 0 09 Feb 2025
Design Considerations in Offline Preference-based RL Alekh Agarwal Christoph Dann T. V. Marinov OffRL 56 0 0 08 Feb 2025
The Battling Influencers Game: Nash Equilibria Structure of a Potential Game and Implications to Value Alignment Young Wu Yancheng Zhu Jin-Yi Cai Xiaojin Zhu 96 0 0 03 Feb 2025
Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning Hao Sun M. Schaar 94 14 0 28 Jan 2025
An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems Hashmath Shaik Alex Doboli OffRL ELM 149 0 0 31 Dec 2024
Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching A. Jain Harley Wiltzer Jesse Farebrother Irina Rish Glen Berseth Sanjiban Choudhury 52 1 0 11 Nov 2024
Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models Michael Noukhovitch Shengyi Huang Sophie Xhonneux Arian Hosseini Rishabh Agarwal Aaron C. Courville OffRL 79 5 0 23 Oct 2024
Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment Mingzhi Wang Chengdong Ma Qizhi Chen Linjian Meng Yang Han Jiancong Xiao Zhaowei Zhang Jing Huo Weijie Su Yaodong Yang 32 4 0 22 Oct 2024
Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning H. Fernando Han Shen Parikshit Ram Yi Zhou Horst Samulowitz Nathalie Baracaldo Tianyi Chen CLL 56 2 0 20 Oct 2024
Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both Abhijnan Nath Changsoo Jung Ethan Seefried Nikhil Krishnaswamy 134 1 0 11 Oct 2024
Enhancing Multi-Step Reasoning Abilities of Language Models through Direct Q-Function Optimization Guanlin Liu Kaixuan Ji Ning Dai Zheng Wu Chen Dun Q. Gu Lin Yan Quanquan Gu Lin Yan OffRL LRM 48 9 0 11 Oct 2024
RainbowPO: A Unified Framework for Combining Improvements in Preference Optimization Hanyang Zhao Genta Indra Winata Anirban Das Shi-Xiong Zhang D. Yao Wenpin Tang Sambit Sahu 54 5 0 05 Oct 2024
The Crucial Role of Samplers in Online Direct Preference Optimization Ruizhe Shi Runlong Zhou Simon S. Du 58 8 0 29 Sep 2024
Alignment of Diffusion Models: Fundamentals, Challenges, and Future Buhua Liu Shitong Shao Bao Li Lichen Bai Zhiqiang Xu Haoyi Xiong James Kwok Sumi Helal Zeke Xie 45 12 0 11 Sep 2024
Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learning Yuheng Zhang Dian Yu Baolin Peng Linfeng Song Ye Tian Mingyue Huo Nan Jiang Haitao Mi Dong Yu 35 15 0 30 Jun 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 59 14 0 24 Jun 2024
Fast Last-Iterate Convergence of Learning in Games Requires Forgetful Algorithms Yang Cai Gabriele Farina Julien Grand-Clément Christian Kroer Chung-Wei Lee Haipeng Luo Weiqiang Zheng 50 6 0 15 Jun 2024
Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning Jifan Zhang Lalit P. Jain Yang Guo Jiayi Chen Kuan Lok Zhou ... Scott Sievert Timothy Rogers Kevin Jamieson Robert Mankoff Robert Nowak 39 5 0 15 Jun 2024
Fusion-PSRO: Nash Policy Fusion for Policy Space Response Oracles Jiesong Lian Yucong Huang Chengdong Ma Mingzhi Wang Ying Wen Long Hu Yixue Hao 57 0 0 31 May 2024
Offline Regularised Reinforcement Learning for Large Language Models Alignment Pierre Harvey Richemond Yunhao Tang Daniel Guo Daniele Calandriello M. G. Azar ... Gil Shamir Rishabh Joshi Tianqi Liu Rémi Munos Bilal Piot OffRL 46 22 0 29 May 2024
Bayesian WeakS-to-Strong from Text Classification to Generation Ziyun Cui Ziyang Zhang Wen Wu Wen Wu Chao Zhang 33 1 0 24 May 2024
Aligning Transformers with Continuous Feedback via Energy Rank Alignment Shriram Chennakesavalu Frank Hu Sebastian Ibarraran Grant M. Rotskoff 35 3 0 21 May 2024
Understanding the performance gap between online and offline alignment algorithms Yunhao Tang Daniel Guo Zeyu Zheng Daniele Calandriello Yuan Cao ... Rémi Munos Bernardo Avila-Pires Michal Valko Yong Cheng Will Dabney OffRL OnRL 27 61 0 14 May 2024
Learn Your Reference Model for Real Good Alignment Alexey Gorbatovski Boris Shaposhnikov Alexey Malakhov Nikita Surnachev Yaroslav Aksenov Ian Maksimov Nikita Balagansky Daniil Gavrilov OffRL 54 26 0 15 Apr 2024
Navigating the Landscape of Large Language Models: A Comprehensive Review and Analysis of Paradigms and Fine-Tuning Strategies Benjue Weng LM&MA 46 7 0 13 Apr 2024
Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewards Haoxiang Wang Yong Lin Wei Xiong Rui Yang Shizhe Diao Shuang Qiu Han Zhao Tong Zhang 40 71 0 28 Feb 2024
Blending Data-Driven Priors in Dynamic Games Justin Lidard Haimin Hu Asher Hancock Zixu Zhang Albert Gimó Contreras ... Deepak Gopinath Guy Rosman Naomi Ehrich Leonard María Santos J. F. Fisac OffRL 43 5 0 21 Feb 2024
Reinforcement Learning from Human Feedback with Active Queries Kaixuan Ji Jiafan He Quanquan Gu 24 17 0 14 Feb 2024
Generalized Preference Optimization: A Unified Approach to Offline Alignment Yunhao Tang Z. Guo Zeyu Zheng Daniele Calandriello Rémi Munos Mark Rowland Pierre Harvey Richemond Michal Valko Bernardo Avila-Pires Bilal Piot 32 88 0 08 Feb 2024
The Consensus Game: Language Model Generation via Equilibrium Search Athul Paul Jacob Yikang Shen Gabriele Farina Jacob Andreas 39 19 0 13 Oct 2023
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 227 502 0 28 Sep 2022
Human-in-the-loop: Provably Efficient Preference-based Reinforcement Learning with General Function Approximation Xiaoyu Chen Han Zhong Zhuoran Yang Zhaoran Wang Liwei Wang 123 60 0 23 May 2022
Teaching language models to support answers with verified quotes Jacob Menick Maja Trebacz Vladimir Mikulik John Aslanides Francis Song ... Mia Glaese Susannah Young Lucy Campbell-Gillingham G. Irving Nat McAleese ELM RALM 243 259 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 313 11,953 0 04 Mar 2022
Frank-Wolfe Algorithms for Saddle Point Problems Gauthier Gidel Tony Jebara Simon Lacoste-Julien 42 70 0 25 Oct 2016