Group Robust Preference Optimization in Reward-free RLHF

30 May 2024

ArXiv (abs)PDF HTML HuggingFace (1 upvotes)

Papers citing "Group Robust Preference Optimization in Reward-free RLHF"

38 / 38 papers shown

Title
OpenApps: Simulating Environment Variations to Measure UI-Agent Reliability Karen Ullrich Jingtong Su Claudia Shi Arjun Subramonian Amir Bar Ivan Evtimov Nikolaos Tsilivis Randall Balestriero Julia Kempe Mark Ibrahim 72 0 0 25 Nov 2025
Reflective Personalization Optimization: A Post-hoc Rewriting Framework for Black-Box Large Language Models Teqi Hao Xioayu Tan Shaojie Shi Yinghui Xu Xihe Qiu 168 0 0 07 Nov 2025
Offline Clustering of Preference Learning with Active-data Augmentation Jingyuan Liu Fatemeh Ghaffari Xuchuang Wang Xutong Liu Mohammad Hajiesmaili Carlee Joe-Wong OffRL 146 0 0 30 Oct 2025
$α$ -LoRA: Effective Fine-Tuning via Base Model Rescaling Aymane El Firdoussi El Mahdi Chayti Mohamed El Amine Seddik Martin Jaggi 96 0 0 24 Oct 2025
Code-driven Number Sequence Calculation: Enhancing the inductive Reasoning Abilities of Large Language Models Kedi Chen Zhikai Lei Xu Guo Xuecheng Wu Siyuan Zeng ... J. Zhou Liang He Qipeng Guo Kai Chen Wei-na Zhang AIMat AI4TS LRM 231 0 0 16 Oct 2025
GCPO: When Contrast Fails, Go Gold Hao Wu Wei Liu 100 0 0 09 Oct 2025
Murphys Laws of AI Alignment: Why the Gap Always Wins Madhava Gaikwad ALM 173 1 0 04 Sep 2025
SharedRep-RLHF: A Shared Representation Approach to RLHF with Diverse Preferences Arpan Mukherjee Marcello Bullo Deniz Gündüz 85 0 0 03 Sep 2025
Relative Advantage Debiasing for Watch-Time Prediction in Short-Video Recommendation Emily Liu Kuan Han Minfeng Zhan Bocheng Zhao Guanyu Mu Yang Song CML AI4TS 129 0 0 14 Aug 2025
Pref-GUIDE: Continual Policy Learning from Real-Time Human Feedback via Preference-Based Learning Zhengran Ji Boyuan Chen 148 1 0 10 Aug 2025
PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization Han Jiang Dongyao Zhu Zhihua Wei Xiaoyuan Yi Ziang Xiao Xing Xie 143 1 0 22 Jul 2025
Principled Foundations for Preference Optimization Wenxuan Zhou Shujian Zhang Brice Magdalou John Lambert Ehsan Amid Richard Nock Andrew Straiton Hard 238 0 0 10 Jul 2025
From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information? Zhanke Zhou Xiao Feng Zhaocheng Zhu Jiangchao Yao Sanmi Koyejo Bo Han LRM 233 33 0 09 Jun 2025
Corrector Sampling in Language Models Itai Gat Neta Shaul Uriel Singer Y. Lipman KELM AI4TS 125 0 0 06 Jun 2025
Beyond RLHF and NLHF: Population-Proportional Alignment under an Axiomatic Framework Kihyun Kim Jiawei Zhang Asuman Ozdaglar P. Parrilo 220 1 0 05 Jun 2025
Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Models Soumya Suvra Ghosal Souradip Chakraborty Avinash Reddy Yifu Lu Mengdi Wang Dinesh Manocha Furong Huang Mohammad Ghavamzadeh Amrit Singh Bedi ReLM LRM 322 17 0 04 Jun 2025
DDO: Dual-Decision Optimization for LLM-Based Medical Consultation via Multi-Agent Collaboration Zhihao Jia Mingyi Jia Junwen Duan Jianxin Wang 203 1 0 24 May 2025
Stable Reinforcement Learning for Efficient Reasoning Muzhi Dai Shixuan Liu Qingyi Si OffRL LRM 280 12 0 23 May 2025
Group Distributionally Robust Optimization with Flexible Sample Queries Haomin Bai Dingzhi Yu Shuai Li Haipeng Luo Lijun Zhang 169 0 0 21 May 2025
CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs Zheyu Shen Ziyao Wang Bowei Tian Meng Liu Sihan Chen Shwai He Bowei Tian Wanghao Ye Yiting Wang Ang Li LRM 202 3 0 19 May 2025
Fair-PP: A Synthetic Dataset for Aligning LLM with Personalized Preferences of Social Equity Qi Zhou Jie Zhang Dongxia Wang Qiang Liu Tianlin Li Jin Song Dong Wenhai Wang Qing Guo SyDa 236 1 0 17 May 2025
FairPO: Robust Preference Optimization for Fair Multi-Label Learning Soumen Kumar Mondal Akshit Varmora Prateek Chanda Ganesh Ramakrishnan 263 0 0 05 May 2025
Dynamic Early Exit in Reasoning Models Chenxu Yang Qingyi Si Yongjie Duan Zheliang Zhu Chenyu Zhu Zheng Lin Zheng Lin Li Cao Weiping Wang ReLM LRM 463 92 0 22 Apr 2025
Kongzi: A Historical Large Language Model with Fact Enhancement Jiashu Yang Ningning Wang Yian Zhao Chaoran Feng Junjia Du Hao Pang Zhirui Fang Xuxin Cheng HILM ALM LRM 203 2 0 13 Apr 2025
Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuning Kai Ye Hongyi Zhou Jin Zhu Francesco Quinzan C. Shi 359 4 0 03 Apr 2025
The Mind in the Machine: A Survey of Incorporating Psychological Theories in LLMs Zizhou Liu Ziwei Gong Lin Ai Zheng Hui Run Chen Colin Wayne Leach Michelle R. Greene Julia Hirschberg LLMAG 917 5 0 28 Mar 2025
Strategyproof Reinforcement Learning from Human Feedback Thomas Kleine Buening Jiarui Gan Debmalya Mandal Marta Z. Kwiatkowska 227 2 0 12 Mar 2025
Group Preference Alignment: Customized LLM Response Generation from In-Situ Conversations Ishani Mondal Jack W. Stokes S. Jauhar Longqi Yang Mengting Wan Xiaofeng Xu Xia Song Jennifer Neville 215 1 0 11 Mar 2025
Robust Multi-Objective Controlled Decoding of Large Language Models Seongho Son William Bankes Sangwoong Yoon Shyam Sundhar Ramesh Xiaohang Tang Ilija Bogunovic 296 5 0 11 Mar 2025
FedRLHF: A Convergence-Guaranteed Federated Framework for Privacy-Preserving and Personalized RLHFAdaptive Agents and Multi-Agent Systems (AAMAS), 2024 Flint Xiaofeng Fan Cheston Tan Yew-Soon Ong Roger Wattenhofer Wei Tsang Ooi 388 1 0 20 Dec 2024
Strada-LLM: Graph LLM for traffic prediction Seyed Mohamad Moghadas Yangxintong Lyu Alexandre Alahi Alexandre Alahi AI4TS 444 4 0 28 Oct 2024
Diverging Preferences: When do Annotators Disagree and do Models Know? Michael J.Q. Zhang Zhilin Wang Jena D. Hwang Yi Dong Olivier Delalleau Yejin Choi Eunsol Choi Xiang Ren Valentina Pyatkin 226 26 0 18 Oct 2024
Towards a Unified View of Preference Learning for Large Language Models: A Survey Bofei Gao Feifan Song Yibo Miao Zefan Cai Zhiyong Yang ... Houfeng Wang Zhifang Sui Peiyi Wang Baobao Chang Baobao Chang 380 16 0 04 Sep 2024
Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift Seongho Son William Bankes Sayak Ray Chowdhury Brooks Paige Ilija Bogunovic 317 8 0 26 Jul 2024
DPO Meets PPO: Reinforced Token Optimization for RLHF Han Zhong Zikang Shan Guhao Feng Wei Xiong Xinle Cheng Li Zhao Di He Jiang Bian Liwei Wang 569 96 0 29 Apr 2024
Reinforcement Learning from Human Feedback with Active Queries Kaixuan Ji Jiafan He Quanquan Gu 327 31 0 14 Feb 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 822 442 0 18 Jan 2024
Crowd-PrefRL: Preference-Based Reward Learning from Crowds David Chhan Ellen R. Novoseller Vernon J. Lawhern 391 7 0 17 Jan 2024