Personalized Soups: Personalized Large Language Model Alignment via
Post-hoc Parameter Merging

Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging

17 October 2023

Bill Yuchen Lin

Luke Zettlemoyer

Hannaneh Hajishirzi

Yejin Choi

Prithviraj Ammanabrolu

Papers citing "Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging"

17 / 117 papers shown

Title
ICDPO: Effectively Borrowing Alignment Capability of Others via In-context Direct Preference Optimization Feifan Song Yuxuan Fan Xin Zhang Peiyi Wang Houfeng Wang 25 8 0 14 Feb 2024
MaxMin-RLHF: Towards Equitable Alignment of Large Language Models with Diverse Human Preferences Souradip Chakraborty Jiahao Qiu Hui Yuan Alec Koppel Furong Huang Dinesh Manocha Amrit Singh Bedi Mengdi Wang ALM 17 46 0 14 Feb 2024
Personalized Language Modeling from Personalized Human Feedback Xinyu Li Zachary C. Lipton Liu Leqi ALM 63 46 0 06 Feb 2024
Decoding-time Realignment of Language Models Tianlin Liu Shangmin Guo Leonardo Bianco Daniele Calandriello Quentin Berthet Felipe Llinares-López Jessica Hoffmann Lucas Dixon Michal Valko Mathieu Blondel AI4CE 51 35 0 05 Feb 2024
Panacea: Pareto Alignment via Preference Adaptation for LLMs Yifan Zhong Chengdong Ma Xiaoyuan Zhang Ziran Yang Haojun Chen Qingfu Zhang Siyuan Qi Yaodong Yang 49 30 0 03 Feb 2024
Rethinking the Role of Proxy Rewards in Language Model Alignment Sungdong Kim Minjoon Seo SyDa ALM 23 0 0 02 Feb 2024
Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives Chen Gao Xiaochong Lan Nian Li Yuan Yuan Jingtao Ding Zhilun Zhou Fengli Xu Yong Li LLMAG AI4CE LM&Ro 29 97 0 19 Dec 2023
Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferences Minyoung Hwang Luca Weihs Chanwoo Park Kimin Lee Aniruddha Kembhavi Kiana Ehsani 22 18 0 14 Dec 2023
On Diversified Preferences of Large Language Model Alignment Dun Zeng Yong Dai Pengyu Cheng Longyue Wang Tianhao Hu Wanshun Chen Nan Du Zenglin Xu ALM 22 16 0 12 Dec 2023
User Modeling in the Era of Large Language Models: Current Research and Future Directions Zhaoxuan Tan Meng-Long Jiang 16 8 0 11 Dec 2023
Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization Zhanhui Zhou Jie Liu Chao Yang Jing Shao Yu Liu Xiangyu Yue Wanli Ouyang Yu Qiao 24 47 0 05 Oct 2023
$Decolonial AI Alignment: Openness, Viśe\d{s}a-Dharma, and Including Excluded Knowledges$ Decolonial AI Alignment: Openness, Viśe\d{s}a-Dharma, and Including Excluded Knowledges Kush R. Varshney 15 2 0 10 Sep 2023
Aligning Large Language Models through Synthetic Feedback Sungdong Kim Sanghwan Bae Jamin Shin Soyoung Kang Donghyun Kwak Kang Min Yoo Minjoon Seo ALM SyDa 73 67 0 23 May 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 157 576 0 06 Apr 2023
Teaching language models to support answers with verified quotes Jacob Menick Maja Trebacz Vladimir Mikulik John Aslanides Francis Song ... Mia Glaese Susannah Young Lucy Campbell-Gillingham G. Irving Nat McAleese ELM RALM 229 255 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,730 0 04 Mar 2022
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 275 1,561 0 18 Sep 2019