ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2310.03708
  4. Cited By
Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct
  Preference Optimization
v1v2v3 (latest)

Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization

Annual Meeting of the Association for Computational Linguistics (ACL), 2023
5 October 2023
Zhanhui Zhou
Jie Liu
Chao Yang
Jing Shao
Yu Liu
Xiangyu Yue
Wanli Ouyang
Yu Qiao
ArXiv (abs)PDFHTML

Papers citing "Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization"

50 / 60 papers shown
Precise Attribute Intensity Control in Large Language Models via Targeted Representation Editing
Precise Attribute Intensity Control in Large Language Models via Targeted Representation Editing
Rongzhi Zhang
Meghaj Tarte
Yuzhao Heng
Xiang Chen
Tong Yu
Lingkai Kong
Sudheer Chava
Chao Zhang
115
0
0
14 Oct 2025
OrthAlign: Orthogonal Subspace Decomposition for Non-Interfering Multi-Objective Alignment
OrthAlign: Orthogonal Subspace Decomposition for Non-Interfering Multi-Objective Alignment
Guanbin Li
Zhihao Xu
Junhao Dong
Jian Zhao
Yuchen Yuan
...
Zhengtao Yao
Huahui Yi
Dongrui Liu
Xinfeng Li
Kun Wang
244
1
0
29 Sep 2025
MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problems
MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problems
Yuki Ichihara
Yuu Jinnai
Tetsuro Morimura
Mitsuki Sakamoto
Ryota Mitsuhashi
Eiji Uchibe
173
4
0
26 Sep 2025
Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
Yining Lu
Zilong Wang
Shiyang Li
Xin Liu
Changlong Yu
Qingyu Yin
Zhan Shi
Zixuan Zhang
Meng Jiang
123
4
0
14 Sep 2025
FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation
FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation
Mengchao Wang
Qiang Wang
Fan Jiang
Mu Xu
EGVMVGen
129
3
0
15 Aug 2025
Beyond Single: A Data Selection Principle for LLM Alignment via Fine-Grained Preference Signals
Beyond Single: A Data Selection Principle for LLM Alignment via Fine-Grained Preference Signals
Jia Zhang
Y. Liu
Chen-Xi Zhang
Yi Liu
Yi-Xuan Jin
Lan-Zhe Guo
Yu-Feng Li
127
0
0
11 Aug 2025
Aligning LLMs on a Budget: Inference-Time Alignment with Heuristic Reward Models
Aligning LLMs on a Budget: Inference-Time Alignment with Heuristic Reward Models
Mason Nakamura
Saaduddin Mahmud
K. H. Wray
Hamed Zamani
S. Zilberstein
110
0
0
07 Aug 2025
Sotopia-RL: Reward Design for Social Intelligence
Sotopia-RL: Reward Design for Social Intelligence
Haofei Yu
Zhengyang Qi
Yining Zhao
Kolby Nottingham
Keyang Xuan
Bodhisattwa Prasad Majumder
Hao Zhu
Paul Pu Liang
Jiaxuan You
OffRL
219
6
0
05 Aug 2025
PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization
PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization
Han Jiang
Dongyao Zhu
Zhihua Wei
Xiaoyuan Yi
Ziang Xiao
Xing Xie
200
1
0
22 Jul 2025
CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steering
CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steering
Hang Lv
Sheng Liang
Hao Wang
Hongchao Gu
Yaxiong Wu
Wei Guo
Defu Lian
Yong Liu
Tong Xu
188
4
0
07 Jul 2025
A Framework for Controllable Multi-objective Learning with Annealed Stein Variational Hypernetworks
A Framework for Controllable Multi-objective Learning with Annealed Stein Variational Hypernetworks
Minh-Duc Nguyen
Dung D. Le
300
0
0
07 Jun 2025
HSCR: Hierarchical Self-Contrastive Rewarding for Aligning Medical Vision Language Models
HSCR: Hierarchical Self-Contrastive Rewarding for Aligning Medical Vision Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2025
Songtao Jiang
Yan Zhang
Yeying Jin
Hongwei Wang
Y. Wu
Yang Feng
Jian Wu
Zuozhu Liu
221
3
0
01 Jun 2025
Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models
Conformal Arbitrage: Risk-Controlled Balancing of Competing Objectives in Language Models
William Overman
Mohsen Bayati
228
2
0
01 Jun 2025
Learning Safety Constraints for Large Language Models
Learning Safety Constraints for Large Language Models
Xin Chen
Yarden As
Andreas Krause
184
9
0
30 May 2025
Differential Information Distribution: A Bayesian Perspective on Direct Preference Optimization
Differential Information Distribution: A Bayesian Perspective on Direct Preference Optimization
Yunjae Won
Hyunji Lee
Hyeonbin Hwang
Minjoon Seo
309
0
0
29 May 2025
Multi-objective Large Language Model Alignment with Hierarchical Experts
Multi-objective Large Language Model Alignment with Hierarchical Experts
Zhuo Li
Guodong DU
Weiyang Guo
Yigeng Zhou
Xiucheng Li
...
Fangming Liu
Yequan Wang
Deheng Ye
Min Zhang
Jing Li
ALMMoE
335
2
0
27 May 2025
Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
Understanding the Performance Gap in Preference Learning: A Dichotomy of RLHF and DPO
Ruizhe Shi
Minhak Song
Runlong Zhou
Zihan Zhang
Maryam Fazel
S. S. Du
316
6
0
26 May 2025
MOSLIM:Align with diverse preferences in prompts through reward classification
MOSLIM:Align with diverse preferences in prompts through reward classification
Yu Zhang
Wanli Jiang
Zhengyu Yang
197
2
0
24 May 2025
Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundary
Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundary
Licheng Pan
Yongqi Tong
Xin Zhang
Xiaolu Zhang
Jun Zhou
Zhixuan Chu
325
2
0
23 May 2025
Online Iterative Self-Alignment for Radiology Report Generation
Online Iterative Self-Alignment for Radiology Report GenerationAnnual Meeting of the Association for Computational Linguistics (ACL), 2025
Ting Xiao
Lei Shi
Yang Zhang
HaoFeng Yang
Zhe Wang
Chenjia Bai
352
0
0
17 May 2025
Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
Zhuocheng Gong
Jian Guan
Wei Wu
Huishuai Zhang
Dongyan Zhao
342
4
0
08 May 2025
PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model
PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model
Xiaoyuan Zhang
Weisen Jiang
Yuancheng Xu
Hao Chen
Ying-Cong Chen
335
7
0
06 May 2025
A Survey on Progress in LLM Alignment from the Perspective of Reward Design
A Survey on Progress in LLM Alignment from the Perspective of Reward Design
Miaomiao Ji
Yanqiu Wu
Zhibin Wu
Shoujin Wang
Jian Yang
Mark Dras
Usman Naseem
379
9
0
05 May 2025
Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors
Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors
Ren-Wei Liang
Chin-Ting Hsu
Chan-Hung Yu
Saransh Agrawal
Shih-Cheng Huang
Shang-Tse Chen
Kuan-Hao Huang
Shao-Hua Sun
Shao-Hua Sun
336
2
0
27 Apr 2025
ParetoHqD: Fast Offline Multiobjective Alignment of Large Language Models using Pareto High-quality Data
ParetoHqD: Fast Offline Multiobjective Alignment of Large Language Models using Pareto High-quality Data
Haoran Gu
Handing Wang
Yi Mei
Mengjie Zhang
Yaochu Jin
342
3
0
23 Apr 2025
Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment
Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgmentAnnual Meeting of the Association for Computational Linguistics (ACL), 2025
Xiaotian Zhang
Ruizhe Chen
Yang Feng
Zuozhu Liu
385
5
0
17 Apr 2025
REWARD CONSISTENCY: Improving Multi-Objective Alignment from a Data-Centric Perspective
REWARD CONSISTENCY: Improving Multi-Objective Alignment from a Data-Centric Perspective
Zhihao Xu
Yongqi Tong
Xin Zhang
Jun Zhou
Xiting Wang
230
2
0
15 Apr 2025
Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretability
Towards Understanding and Improving Refusal in Compressed Models via Mechanistic Interpretability
Vishnu Kabir Chhabra
Mohammad Mahdi Khalili
AI4CE
246
0
0
05 Apr 2025
Natural Language Generation
Natural Language GenerationTheoretical Issues In Natural Language Processing (TINLP), 2018
Emiel van Miltenburg
Chenghua Lin
308
2
0
20 Mar 2025
BalancedDPO: Adaptive Multi-Metric Alignment
BalancedDPO: Adaptive Multi-Metric Alignment
Dipesh Tamboli
Souradip Chakraborty
Aditya Malusare
B. Banerjee
Amrit Singh Bedi
Vaneet Aggarwal
EGVM
250
2
0
16 Mar 2025
UC-MOA: Utility-Conditioned Multi-Objective Alignment for Distributional Pareto-Optimality
UC-MOA: Utility-Conditioned Multi-Objective Alignment for Distributional Pareto-Optimality
Zelei Cheng
Xin-Qiang Cai
Yuting Tang
Pushi Zhang
Boming Yang
Masashi Sugiyama
Xinyu Xing
515
1
0
10 Mar 2025
PEO: Improving Bi-Factorial Preference Alignment with Post-Training Policy Extrapolation
Yuxuan Liu
257
0
0
03 Mar 2025
Robust Multi-Objective Preference Alignment with Online DPOAAAI Conference on Artificial Intelligence (AAAI), 2025
Raghav Gupta
Ryan Sullivan
Yunxuan Li
Samrat Phatale
Abhinav Rastogi
227
8
0
01 Mar 2025
The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue Agents
The Rise of Darkness: Safety-Utility Trade-Offs in Role-Playing Dialogue AgentsAnnual Meeting of the Association for Computational Linguistics (ACL), 2025
Yihong Tang
Kehai Chen
X. Bai
Zhengyu Niu
Binghai Wang
J. Tang
Min Zhang
LLMAG
272
3
0
28 Feb 2025
Societal Alignment Frameworks Can Improve LLM Alignment
Karolina Stañczak
Nicholas Meade
Mehar Bhatia
Hattie Zhou
Konstantin Böttinger
...
Timothy P. Lillicrap
Ana Marasović
Sylvie Delacroix
Gillian K. Hadfield
Siva Reddy
1.0K
3
0
27 Feb 2025
Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systems
Faster, Cheaper, Better: Multi-Objective Hyperparameter Optimization for LLM and RAG Systems
Matthew Barker
Andrew Bell
Evan Thomas
James Carr
Thomas Andrews
Umang Bhatt
437
8
0
25 Feb 2025
MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment
MPO: An Efficient Post-Processing Framework for Mixing Diverse Preference Alignment
Tianze Wang
Dongnan Gui
Yifan Hu
Shuhang Lin
Linjun Zhang
583
4
0
25 Feb 2025
Drift: Decoding-time Personalized Alignments with Implicit User Preferences
Drift: Decoding-time Personalized Alignments with Implicit User Preferences
Minbeom Kim
Kang-il Lee
Seongho Joo
Hwaran Lee
Thibaut Thonet
Kyomin Jung
AI4TS
594
10
0
20 Feb 2025
Rethinking Diverse Human Preference Learning through Principal Component Analysis
Rethinking Diverse Human Preference Learning through Principal Component AnalysisAnnual Meeting of the Association for Computational Linguistics (ACL), 2025
Feng Luo
Rui Yang
Hao Sun
Chunyuan Deng
Jiarui Yao
Jingyan Shen
Huan Zhang
Hanjie Chen
426
6
0
18 Feb 2025
STAIR: Improving Safety Alignment with Introspective Reasoning
STAIR: Improving Safety Alignment with Introspective Reasoning
Yuanhang Zhang
Siyuan Zhang
Yao Huang
Zeyu Xia
Zhengwei Fang
Xiao Yang
Ranjie Duan
Dong Yan
Yinpeng Dong
Jun Zhu
LRMLLMSV
412
42
0
04 Feb 2025
Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond
Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond
Weiyu Chen
Xiaoyuan Zhang
Xiaoyuan Zhang
Xi Lin
Han Zhao
Gang Qu
James T. Kwok
470
21
0
19 Jan 2025
Pareto-Optimal Energy Alignment for Designing Nature-Like Antibodies
Pareto-Optimal Energy Alignment for Designing Nature-Like Antibodies
Yibo Wen
Chenwei Xu
Jerry Yao-Chieh Hu
Han Liu
Han Liu
DiffM
277
6
0
30 Dec 2024
Orbit: A Framework for Designing and Evaluating Multi-objective Rankers
Orbit: A Framework for Designing and Evaluating Multi-objective RankersInternational Conference on Intelligent User Interfaces (IUI), 2024
Chenyang Yang
Tesi Xiao
Michael Shavlovsky
Jane Hsieh
Tongshuang Wu
307
1
0
07 Nov 2024
Comparison-based Active Preference Learning for Multi-dimensional Personalization
Comparison-based Active Preference Learning for Multi-dimensional PersonalizationAnnual Meeting of the Association for Computational Linguistics (ACL), 2024
Minhyeon Oh
Seungjoon Lee
Jungseul Ok
326
1
0
01 Nov 2024
Constraint Back-translation Improves Complex Instruction Following of Large Language Models
Constraint Back-translation Improves Complex Instruction Following of Large Language Models
Yunjia Qi
Hao Peng
Xinyu Wang
Bin Xu
Lei Hou
Juanzi Li
448
6
0
31 Oct 2024
L3Ms -- Lagrange Large Language Models
L3Ms -- Lagrange Large Language ModelsInternational Conference on Learning Representations (ICLR), 2024
Guneet S. Dhillon
Xingjian Shi
Yee Whye Teh
Alex Smola
1.1K
1
0
28 Oct 2024
2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional
  Supervision
2D-DPO: Scaling Direct Preference Optimization with 2-Dimensional Supervision
Shilong Li
Yancheng He
Hui Huang
Xingyuan Bu
Qingbin Liu
Hangyu Guo
Weixun Wang
Jihao Gu
Yuchi Xu
Bo Zheng
227
9
0
25 Oct 2024
Improving Inverse Folding for Peptide Design with Diversity-regularized
  Direct Preference Optimization
Improving Inverse Folding for Peptide Design with Diversity-regularized Direct Preference Optimization
Ryan Park
Darren J. Hsu
C. Brian Roland
Maria Korshunova
Chen Tessler
Shie Mannor
Olivia Viessmann
Bruno Trentini
212
6
0
25 Oct 2024
COS-DPO: Conditioned One-Shot Multi-Objective Fine-Tuning Framework
COS-DPO: Conditioned One-Shot Multi-Objective Fine-Tuning FrameworkConference on Uncertainty in Artificial Intelligence (UAI), 2024
Yinuo Ren
Tesi Xiao
Michael Shavlovsky
Lexing Ying
Holakou Rahmanian
323
0
0
10 Oct 2024
Inference-Time Language Model Alignment via Integrated Value Guidance
Inference-Time Language Model Alignment via Integrated Value GuidanceConference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Zhixuan Liu
Zhanhui Zhou
Yuanfu Wang
Chao Yang
Yu Qiao
170
15
0
26 Sep 2024
12
Next
Page 1 of 2