Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging

17 October 2023

Luke Zettlemoyer

Yejin Choi

Prithviraj Ammanabrolu

MoMe

ArXiv PDF HTML

Papers citing "Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging"

50 / 117 papers shown

Title
Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes Zhuocheng Gong Jian-Yu Guan Wei Yu Wu Huishuai Zhang Dongyan Zhao 59 1 0 08 May 2025
Steerable Chatbots: Personalizing LLMs with Preference-Based Activation Steering Jessica Y. Bo Tianyu Xu Ishan Chatterjee Katrina Passarella-Ward Achin Kulshrestha D Shin LLMSV 66 0 0 07 May 2025
PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model Baijiong Lin Weisen Jiang Yuancheng Xu Hao Chen Ying Chen 21 0 0 06 May 2025
YoChameleon: Personalized Vision and Language Generation Thao Nguyen Krishna Kumar Singh Jing Shi Trung H. Bui Yong Jae Lee Yuheng Li MLLM 82 0 0 29 Apr 2025
LoRe: Personalizing LLMs via Low-Rank Reward Modeling Avinandan Bose Zhihan Xiong Yuejie Chi Simon S. Du Lin Xiao Maryam Fazel 26 0 0 20 Apr 2025
Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment Xiaotian Zhang Ruizhe Chen Yang Feng Zuozhu Liu 40 0 0 17 Apr 2025
Never Start from Scratch: Expediting On-Device LLM Personalization via Explainable Model Selection Haoming Wang Boyuan Yang Xiangyu Yin Wei Gao 28 0 0 15 Apr 2025
A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models Zhouhang Xie Junda Wu Yiran Shen Yu Xia Xintong Li ... Sachin Kumar Bodhisattwa Prasad Majumder Jingbo Shang Prithviraj Ammanabrolu Julian McAuley 31 0 0 09 Apr 2025
Retrieval Augmented Generation with Collaborative Filtering for Personalized Text Generation Teng Shi Jun Xu Xiao Zhang Xiaoxue Zang Kai Zheng Yang Song Han Li RALM 3DV 38 0 0 08 Apr 2025
Collab: Controlled Decoding using Mixture of Agents for LLM Alignment Souradip Chakraborty Sujay Bhatt Udari Madhushani Sehwag Soumya Suvra Ghosal Jiahao Qiu Mengdi Wang Dinesh Manocha Furong Huang Alec Koppel Sumitra Ganesh 44 2 0 27 Mar 2025
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications Jian-Yu Guan J. Wu J. Li Chuanqi Cheng Wei Yu Wu LM&MA 69 0 0 21 Mar 2025
Capturing Individual Human Preferences with Reward Features André Barreto Vincent Dumoulin Yiran Mao Nicolas Perez-Nieves Bobak Shahriari Yann Dauphin Doina Precup Hugo Larochelle ALM 57 1 0 21 Mar 2025
From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment J. Li Jian-Yu Guan Songhao Wu Wei Yu Wu Rui Yan 59 1 0 19 Mar 2025
Enhanced Soups for Graph Neural Networks Joseph Zuber Aishwarya Sarkar Joseph Jennings Ali Jannesari 40 0 0 14 Mar 2025
Robust Multi-Objective Controlled Decoding of Large Language Models Seongho Son William Bankes Sangwoong Yoon Shyam Sundhar Ramesh Xiaohang Tang Ilija Bogunovic 39 0 0 11 Mar 2025
Language Model Personalization via Reward Factorization Idan Shenfeld Felix Faltings Pulkit Agrawal Aldo Pacchiano 43 1 0 08 Mar 2025
Personalized Text Generation with Contrastive Activation Steering Jinghao Zhang Y. Liu W. Wang Qiang Liu Shu Wu Liang Wang Tat-Seng Chua LLMSV 41 0 0 07 Mar 2025
CoPL: Collaborative Preference Learning for Personalizing LLMs Youngbin Choi Seunghyuk Cho M. Lee Moonjeong Park Yesong Ko Jungseul Ok Dongwoo Kim 58 0 0 03 Mar 2025
PEO: Improving Bi-Factorial Preference Alignment with Post-Training Policy Extrapolation Yuxuan Liu 35 0 0 03 Mar 2025
Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity Yupu Hao Pengfei Cao Zhuoran Jin Huanxuan Liao Yubo Chen Kang Liu Jun Zhao LLMAG 69 1 0 02 Mar 2025
Distributionally Robust Reinforcement Learning with Human Feedback Debmalya Mandal Paulius Sasnauskas Goran Radanović 39 1 0 01 Mar 2025
Robust Multi-Objective Preference Alignment with Online DPO Raghav Gupta Ryan Sullivan Yunxuan Li Samrat Phatale Abhinav Rastogi 32 0 0 01 Mar 2025
LIVS: A Pluralistic Alignment Dataset for Inclusive Public Spaces Rashid Mushkani Shravan Nayak Hugo Berard Allison Cohen Shin Koseki Hadrien Bertrand 54 2 0 27 Feb 2025
When Personalization Meets Reality: A Multi-Faceted Analysis of Personalized Preference Learning Yijiang River Dong Tiancheng Hu Yinhong Liu Ahmet Üstün Nigel Collier 78 1 0 26 Feb 2025
FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users Anikait Singh Sheryl Hsu Kyle Hsu E. Mitchell Stefano Ermon Tatsunori Hashimoto Archit Sharma Chelsea Finn SyDa OffRL 57 1 0 26 Feb 2025
MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment Tianze Wang Dongnan Gui Yifan Hu Shuhang Lin Linjun Zhang 31 0 0 25 Feb 2025
Pastiche Novel Generation Creating: Fan Fiction You Love in Your Favorite Author's Style Xueran Han Yuhan Liu Mingzhe Li W. Liu Sen Hu Rui Yan Zhiqiang Xu Xiuying Chen 59 0 0 24 Feb 2025
Drift: Decoding-time Personalized Alignments with Implicit User Preferences Minbeom Kim Kang-il Lee Seongho Joo Hwaran Lee Thibaut Thonet Kyomin Jung AI4TS 105 1 0 20 Feb 2025
A Survey of Personalized Large Language Models: Progress and Future Directions Jiahong Liu Zexuan Qiu Zhongyang Li Quanyu Dai Jieming Zhu Minda Hu Menglin Yang Irwin King LM&MA 46 2 0 17 Feb 2025
Learning to Summarize from LLM-generated Feedback Hwanjun Song Taewon Yun Yuho Lee Jihwan Oh Gihun Lee Jason (Jinglun) Cai Hang Su 73 1 0 28 Jan 2025
Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond Weiyu Chen Xiaoyuan Zhang Baijiong Lin Xi Victoria Lin Han Zhao Qingfu Zhang James T. Kwok 73 1 0 19 Jan 2025
Reasoning-Enhanced Self-Training for Long-Form Personalized Text Generation Alireza Salemi Cheng-rong Li Mingyang Zhang Qiaozhu Mei Weize Kong Tao Chen Zhuowan Li Michael Bendersky Hamed Zamani LRM RALM ReLM 52 6 0 07 Jan 2025
Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment Jianfei Zhang Jun Bai B. Li Yanmeng Wang Rumei Li Chenghua Lin Wenge Rong 39 0 0 31 Dec 2024
From General to Specific: Tailoring Large Language Models for Personalized Healthcare Ruize Shi Hong Huang Wei Zhou Kehan Yin Kai Zhao Yun Zhao LM&MA AI4MH 69 0 0 20 Dec 2024
Test-Time Alignment via Hypothesis Reweighting Yoonho Lee Jonathan Williams Henrik Marklund Archit Sharma E. Mitchell Anikait Singh Chelsea Finn 91 3 0 11 Dec 2024
From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents Xinyi Mou Xuanwen Ding Qi He Liang Wang Jingcong Liang ... L. Sun Jiayu Lin Jie Zhou Xuanjing Huang Zhongyu Wei LLMAG LM&Ro AI4CE 77 11 0 04 Dec 2024
SPICA: Retrieving Scenarios for Pluralistic In-Context Alignment Quan Ze Chen K. J. Kevin Feng Chan Young Park Amy X. Zhang 26 0 0 16 Nov 2024
Contextualized Evaluations: Taking the Guesswork Out of Language Model Evaluations Chaitanya Malaviya Joseph Chee Chang Dan Roth Mohit Iyyer Mark Yatskar Kyle Lo ELM 40 4 0 11 Nov 2024
Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings Miguel Moura Ramos Tomás Almeida Daniel Vareta Filipe Azevedo Sweta Agrawal Patrick Fernandes André F. T. Martins 31 1 0 08 Nov 2024
PMoL: Parameter Efficient MoE for Preference Mixing of LLM Alignment Dongxu Liu Bing Xu Yinzhuo Chen Bufan Xu Wenpeng Lu Muyun Yang T. Zhao MoE 26 0 0 02 Nov 2024
Active Preference-based Learning for Multi-dimensional Personalization Minhyeon Oh Seungjoon Lee Jungseul Ok 26 1 0 01 Nov 2024
CURATe: Benchmarking Personalised Alignment of Conversational AI Assistants Lize Alberts Benjamin Ellis Andrei Lupu Jakob Foerster ELM 34 0 0 28 Oct 2024
ComPO: Community Preferences for Language Model Personalization Sachin Kumar Chan Young Park Yulia Tsvetkov Noah A. Smith Hannaneh Hajishirzi 16 5 0 21 Oct 2024
How to Find the Exact Pareto Front for Multi-Objective MDPs? Yining Li Peizhong Ju Ness B. Shroff 40 0 0 21 Oct 2024
SudoLM: Learning Access Control of Parametric Knowledge with Authorization Alignment Qin Liu Fei Wang Chaowei Xiao Muhao Chen 42 0 0 18 Oct 2024
Personalized Adaptation via In-Context Preference Learning Allison Lau Younwoo Choi Vahid Balazadeh Keertana Chidambaram Vasilis Syrgkanis Rahul G. Krishnan VLM OffRL 13 2 0 17 Oct 2024
Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs Sumanth Doddapaneni Mohammed Safi Ur Rahman Khan Dilip Venkatesh Raj Dabre Anoop Kunchukuttan Mitesh M. Khapra ELM 35 1 0 17 Oct 2024
Exploring Model Kinship for Merging Large Language Models Yedi Hu Yunzhi Yao N. Zhang Shumin Deng H. Chen MoMe 34 1 0 16 Oct 2024
$\llinstruct: An Instruction-tuned model for English Language Proficiency Assessments$ \llinstruct: An Instruction-tuned model for English Language Proficiency Assessments Debanjan Ghosh Sophia Chan ALM 16 0 0 12 Oct 2024
Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements Jingyu Zhang Ahmed Elgohary Ahmed Magooda Daniel Khashabi Benjamin Van Durme 50 2 0 11 Oct 2024