Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspective

6 April 2024

Papers citing "Towards Analyzing and Understanding the Limitations of DPO: A Theoretical Perspective"

8 / 8 papers shown

Title
Teaching Large Language Models to Reason through Learning and Forgetting Tianwei Ni Allen Nie Sapana Chaudhary Yao Liu Huzefa Rangwala Rasool Fakoor ReLM CLL LRM 41 0 0 15 Apr 2025
Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization Noam Razin Sadhika Malladi Adithya Bhaskar Danqi Chen Sanjeev Arora Boris Hanin 89 12 0 11 Oct 2024
Latent Feature Mining for Predictive Model Enhancement with Large Language Models Bingxuan Li Pengyi Shi Amy Ward 43 9 0 06 Oct 2024
3D-Properties: Identifying Challenges in DPO and Charting a Path Forward Yuzi Yan Yibo Miao J. Li Yipin Zhang Jian Xie Zhijie Deng Dong Yan 36 11 0 11 Jun 2024
Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Model Qi Gou Cam-Tu Nguyen 19 8 0 28 Mar 2024
Direct Preference Optimization with an Offset Afra Amini Tim Vieira Ryan Cotterell 68 54 0 16 Feb 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
$Understanding Dataset Difficulty with $\mathcal{V}$-Usable Information$ Understanding Dataset Difficulty with $\mathcal{V}$ -Usable Information Kawin Ethayarajh Yejin Choi Swabha Swayamdipta 154 157 0 16 Oct 2021