Title
Learning Guarantee of Reward Modeling Using Deep Neural Networks Yuanhang Luo Yeheng Ge Ruijian Han Guohao Shen 14 0 0 10 May 2025
Improving RL Exploration for LLM Reasoning through Retrospective Replay Shihan Dou Muling Wu Jingwen Xu Rui Zheng Tao Gui Qi Zhang Xuanjing Huang OffRL LRM 22 0 0 19 Apr 2025
Reasoning without Regret Tarun Chitra OffRL LRM 23 0 0 14 Apr 2025
SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models Hardy Chen Haoqin Tu Fali Wang Hui Liu X. Tang Xinya Du Yuyin Zhou Cihang Xie ReLM VLM OffRL LRM 63 6 0 10 Apr 2025
Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback Wei Shen Guanlin Liu Zheng Wu Ruofei Zhu Qingping Yang Chao Xin Yu Yue Lin Yan 74 8 0 28 Mar 2025
When Preferences Diverge: Aligning Diffusion Models with Minority-Aware Adaptive DPO L. Zhang Chen Liu C. Xu Kai Hu Donghao Luo Chengjie Wang Yanwei Fu Yuan Yao 42 0 0 21 Mar 2025
RankPO: Preference Optimization for Job-Talent Matching Y. Zhang M. Wang Yu Wang Xiaohui Wang 41 0 0 13 Mar 2025
Language Model Personalization via Reward Factorization Idan Shenfeld Felix Faltings Pulkit Agrawal Aldo Pacchiano 43 1 0 08 Mar 2025
Visualising Policy-Reward Interplay to Inform Zeroth-Order Preference Optimisation of Large Language Models Alessio Galatolo Zhenbang Dai Katie Winkle Meriem Beloucif 47 0 0 05 Mar 2025
FANS -- Formal Answer Selection for Natural Language Math Reasoning Using Lean4 Jiarui Yao Ruida Wang Tong Zhang LRM 52 0 0 05 Mar 2025
Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference Wenjie Qiu Yi-Chen Li Xuqin Zhang Tianyi Zhang Y. Zhang Zongzhang Zhang Yang Yu ALM 46 0 0 01 Mar 2025
RLTHF: Targeted Human Feedback for LLM Alignment Yifei Xu Tusher Chakraborty Emre Kıcıman Bibek Aryal Eduardo Rodrigues ... Rafael Padilha Leonardo Nunes Shobana Balakrishnan Songwu Lu Ranveer Chandra 96 1 0 24 Feb 2025
Lean and Mean: Decoupled Value Policy Optimization with Global Value Guidance Chenghua Huang Lu Wang Fangkai Yang Pu Zhao Z. Li Qingwei Lin Dongmei Zhang Saravan Rajmohan Qi Zhang OffRL 52 1 0 24 Feb 2025
Refining Positive and Toxic Samples for Dual Safety Self-Alignment of LLMs with Minimal Human Interventions Jingxin Xu Guoshun Nan Sheng Guan Sicong Leng Y. Liu Zixiao Wang Yuyang Ma Zhili Zhou Yanzhao Hou Xiaofeng Tao LM&MA 53 0 0 08 Feb 2025
Can LLMs Rank the Harmfulness of Smaller LLMs? We are Not There Yet Berk Atil Vipul Gupta Sarkar Snigdha Sarathi Das R. Passonneau 83 0 0 07 Feb 2025
The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hacking Yuchun Miao Sen Zhang Liang Ding Yuqi Zhang L. Zhang Dacheng Tao 81 3 0 31 Jan 2025
An Overview and Discussion on Using Large Language Models for Implementation Generation of Solutions to Open-Ended Problems Hashmath Shaik Alex Doboli OffRL ELM 63 0 0 31 Dec 2024
AddrLLM: Address Rewriting via Large Language Model on Nationwide Logistics Data Qinchen Yang Zhiqing Hong Dongjiang Cao Haotian Wang Zejun Xie Tian He Yunhuai Liu Yu Yang Desheng Zhang KELM 62 0 0 17 Nov 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 55 45 1 15 Nov 2024
Cross-lingual Transfer of Reward Models in Multilingual Alignment Jiwoo Hong Noah Lee Rodrigo Martínez-Castaño César Rodríguez James Thorne 44 3 0 23 Oct 2024
MM-Eval: A Multilingual Meta-Evaluation Benchmark for LLM-as-a-Judge and Reward Models Guijin Son Dongkeun Yoon Juyoung Suk Javier Aula-Blasco Mano Aslan Vu Trong Kim Shayekh Bin Islam Jaume Prats-Cristià Lucía Tormo-Bañuelos Seungone Kim ELM LRM 25 0 0 23 Oct 2024
Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment Mingzhi Wang Chengdong Ma Qizhi Chen Linjian Meng Yang Han Jiancong Xiao Zhaowei Zhang Jing Huo Weijie Su Yaodong Yang 30 4 0 22 Oct 2024
Weak-to-Strong Generalization beyond Accuracy: a Pilot Study in Safety, Toxicity, and Legal Reasoning Ruimeng Ye Yang Xiao Bo Hui ALM ELM OffRL 27 2 0 16 Oct 2024
RMB: Comprehensively Benchmarking Reward Models in LLM Alignment Enyu Zhou Guodong Zheng B. Wang Zhiheng Xi Shihan Dou ... Yurong Mou Rui Zheng Tao Gui Qi Zhang Xuanjing Huang ALM 54 13 0 13 Oct 2024
Taming Overconfidence in LLMs: Reward Calibration in RLHF Jixuan Leng Chengsong Huang Banghua Zhu Jiaxin Huang 26 7 0 13 Oct 2024
Reward Learning From Preference With Ties Jinsong Liu Dongdong Ge Ruihao Zhu 19 3 0 05 Oct 2024
Aligning LLMs with Individual Preferences via Interaction Shujin Wu May Fung Cheng Qian Jeonghwan Kim Dilek Z. Hakkani-Tür Heng Ji 26 9 0 04 Oct 2024
Investigating on RLHF methodology Alexey Kutalev Sergei Markoff 24 0 0 02 Oct 2024
LASeR: Learning to Adaptively Select Reward Models with Multi-Armed Bandits Duy Nguyen Archiki Prasad Elias Stengel-Eskin Mohit Bansal 23 2 0 02 Oct 2024
The Perfect Blend: Redefining RLHF with Mixture of Judges Tengyu Xu Eryk Helenowski Karthik Abinav Sankararaman Di Jin Kaiyan Peng ... Gabriel Cohen Yuandong Tian Hao Ma Sinong Wang Han Fang 31 9 0 30 Sep 2024
VickreyFeedback: Cost-efficient Data Construction for Reinforcement Learning from Human Feedback Guoxi Zhang Jiuding Duan 14 1 0 27 Sep 2024
Modulated Intervention Preference Optimization (MIPO): Keep the Easy, Refine the Difficult Cheolhun Jang 20 0 0 26 Sep 2024
Policy Filtration in RLHF to Fine-Tune LLM for Code Generation Wei Shen Chuheng Zhang OffRL 28 6 0 11 Sep 2024
Resultant: Incremental Effectiveness on Likelihood for Unsupervised Out-of-Distribution Detection Yewen Li Chaojie Wang Xiaobo Xia Xu He Ruyi An Dong Li Tongliang Liu Bo An Xinrun Wang OODD 42 0 0 05 Sep 2024
Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates Hui Wei Shenghua He Tian Xia Andy H. Wong Jingyang Lin Mei Han Mei Han ALM ELM 57 22 0 23 Aug 2024
SEAL: Systematic Error Analysis for Value ALignment Manon Revel Matteo Cargnelutti Tyna Eloundou Greg Leppert 40 3 0 16 Aug 2024
LIONs: An Empirically Optimized Approach to Align Language Models Xiao Yu Qingyang Wu Yu Li Zhou Yu ALM 27 3 0 09 Jul 2024
Towards Comprehensive Preference Data Collection for Reward Modeling Yulan Hu Qingyang Li Sheng Ouyang Ge Chen Kaihui Chen Lijun Mei Xucheng Ye Fuzheng Zhang Yong Liu SyDa 32 4 0 24 Jun 2024
Aligning Large Language Models from Self-Reference AI Feedback with one General Principle Rong Bao Rui Zheng Shihan Dou Xiao Wang Enyu Zhou Bo Wang Qi Zhang Liang Ding Dacheng Tao ALM 40 0 0 17 Jun 2024
Toward Optimal LLM Alignments Using Two-Player Games Rui Zheng Hongyi Guo Zhihan Liu Xiaoying Zhang Yuanshun Yao ... Tao Gui Qi Zhang Xuanjing Huang Hang Li Yang Liu 58 5 0 16 Jun 2024
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence Junru Lu Jiazheng Li Siyu An Meng Zhao Yulan He Di Yin Xing Sun 31 13 0 16 Jun 2024
Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs Rui Yang Ruomeng Ding Yong Lin Huan Zhang Tong Zhang 21 42 0 14 Jun 2024
Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms Miaosen Zhang Yixuan Wei Zhen Xing Yifei Ma Zuxuan Wu ... Zheng-Wei Zhang Qi Dai Chong Luo Xin Geng Baining Guo VLM 33 1 0 13 Jun 2024
PAL: Pluralistic Alignment Framework for Learning from Heterogeneous Preferences Daiwei Chen Yi Chen Aniket Rege Ramya Korlakai Vinayak 35 16 0 12 Jun 2024
Legend: Leveraging Representation Engineering to Annotate Safety Margin for Preference Datasets Duanyu Feng Bowen Qin Chen Huang Youcheng Huang Zheng-Wei Zhang Wenqiang Lei 44 2 0 12 Jun 2024
Prototypical Reward Network for Data-Efficient RLHF Jinghan Zhang Xiting Wang Yiqiao Jin Changyu Chen Xinhao Zhang Kunpeng Liu ALM 18 18 0 06 Jun 2024
AgentGym: Evolving Large Language Model-based Agents across Diverse Environments Zhiheng Xi Yiwen Ding Wenxiang Chen Boyang Hong Honglin Guo ... Qi Zhang Xipeng Qiu Xuanjing Huang Zuxuan Wu Yu-Gang Jiang LLMAG LM&Ro 38 28 0 06 Jun 2024
Aligning Large Language Models via Fine-grained Supervision Dehong Xu Liang Qiu Minseok Kim Faisal Ladhak Jaeyoung Do 27 2 0 04 Jun 2024
Dishonesty in Helpful and Harmless Alignment Youcheng Huang Jingkun Tang Duanyu Feng Zheng-Wei Zhang Wenqiang Lei Jiancheng Lv Anthony G. Cohn LLMSV 25 3 0 04 Jun 2024
Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment Yueqin Yin Zhendong Wang Yujia Xie Weizhu Chen Mingyuan Zhou 30 4 0 31 May 2024