RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation

30 April 2024

Chanwoo Park

Mingyang Liu

Dingwen Kong

Kaiqing Zhang

Asuman Ozdaglar

ArXiv PDF HTML

Papers citing "RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation"

26 / 26 papers shown

Title
Inducing Robustness in a 2 Dimensional Direct Preference Optimization Paradigm Sarvesh Shashidhar Ritik Nachiketa Patil Suraj Racha Ganesh Ramakrishnan 22 0 0 03 May 2025
Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment Xiaotian Zhang Ruizhe Chen Yang Feng Zuozhu Liu 40 0 0 17 Apr 2025
A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models Zhouhang Xie Junda Wu Yiran Shen Yu Xia Xintong Li ... Sachin Kumar Bodhisattwa Prasad Majumder Jingbo Shang Prithviraj Ammanabrolu Julian McAuley 33 0 0 09 Apr 2025
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications Jian-Yu Guan J. Wu J. Li Chuanqi Cheng Wei Yu Wu LM&MA 69 0 0 21 Mar 2025
From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment J. Li Jian-Yu Guan Songhao Wu Wei Yu Wu Rui Yan 62 1 0 19 Mar 2025
Strategyproof Reinforcement Learning from Human Feedback Thomas Kleine Buening Jiarui Gan Debmalya Mandal Marta Z. Kwiatkowska 47 0 0 13 Mar 2025
When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference Learning Yijiang River Dong Tiancheng Hu Yinhong Liu Ahmet Üstün Nigel Collier 78 1 0 26 Feb 2025
MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment Tianze Wang Dongnan Gui Yifan Hu Shuhang Lin Linjun Zhang 36 0 0 25 Feb 2025
Game Theory Meets Large Language Models: A Systematic Survey Haoran Sun Yusen Wu Yukun Cheng Xu Chu LM&MA OffRL AI4CE 55 1 0 13 Feb 2025
The Battling Influencers Game: Nash Equilibria Structure of a Potential Game and Implications to Value Alignment Young Wu Yancheng Zhu Jin-Yi Cai Xiaojin Zhu 94 0 0 03 Feb 2025
Clone-Robust AI Alignment Ariel D. Procaccia Benjamin G. Schiffer Shirley Zhang 33 1 0 17 Jan 2025
Online Learning from Strategic Human Feedback in LLM Fine-Tuning Shugang Hao Lingjie Duan 84 3 0 22 Dec 2024
FedRLHF: A Convergence-Guaranteed Federated Framework for Privacy-Preserving and Personalized RLHF Flint Xiaofeng Fan Cheston Tan Yew-Soon Ong Roger Wattenhofer Wei Tsang Ooi 80 1 0 20 Dec 2024
Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Models Mohammad Beigi Sijia Wang Ying Shen Zihao Lin Adithya Kulkarni ... Ming Jin Jin-Hee Cho Dawei Zhou Chang-Tien Lu Lifu Huang 21 1 0 26 Oct 2024
Multi-objective Reinforcement Learning: A Tool for Pluralistic Alignment Peter Vamplew Conor F. Hayes Cameron Foale Richard Dazeley Hadassah Harland 37 0 0 15 Oct 2024
BAPO: Base-Anchored Preference Optimization for Personalized Alignment in Large Language Models Gihun Lee Minchan Jeong Yujin Kim Hojung Jung Jaehoon Oh Sangmook Kim Se-Young Yun 24 1 0 30 Jun 2024
Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Efforts Zhaoxuan Tan Zheyuan Liu Meng-Long Jiang 34 20 0 15 Jun 2024
Direct Preference Optimization With Unobserved Preference Heterogeneity Keertana Chidambaram Karthik Vinay Seetharaman Vasilis Syrgkanis 39 7 0 23 May 2024
Axioms for AI Alignment from Human Feedback Luise Ge Daniel Halpern Evi Micha Ariel D. Procaccia Itai Shapira Yevgeniy Vorobeychik Junlin Wu 35 15 0 23 May 2024
A Unified Linear Programming Framework for Offline Reward Learning from Human Demonstrations and Feedback Kihyun Kim Jiawei Zhang Asuman Ozdaglar P. Parrilo OffRL 33 1 0 20 May 2024
Offline Multi-task Transfer RL with Representational Penalization Avinandan Bose S. S. Du Maryam Fazel OffRL 49 12 0 19 Feb 2024
Personalized Language Modeling from Personalized Human Feedback Xinyu Li Zachary C. Lipton Liu Leqi ALM 63 47 0 06 Feb 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 306 11,909 0 04 Mar 2022
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 275 1,587 0 18 Sep 2019
New Analysis and Algorithm for Learning with Drifting Distributions M. Mohri Andrés Munoz Medina 83 123 0 19 May 2012
Domain Adaptation: Learning Bounds and Algorithms Yishay Mansour M. Mohri Afshin Rostamizadeh 179 788 0 19 Feb 2009