RRHF: Rank Responses to Align Language Models with Human Feedback without tears

11 April 2023

Zheng Yuan

Hongyi Yuan

Chuanqi Tan

Papers citing "RRHF: Rank Responses to Align Language Models with Human Feedback without tears"

50 / 282 papers shown

Title
A Survey on Progress in LLM Alignment from the Perspective of Reward Design Miaomiao Ji Yanqiu Wu Zhibin Wu Shoujin Wang Jian Yang Mark Dras Usman Naseem 39 0 0 05 May 2025
Sailing AI by the Stars: A Survey of Learning from Rewards in Post-Training and Test-Time Scaling of Large Language Models Xiaobao Wu LRM 72 1 0 05 May 2025
Aligning Large Language Models with Healthcare Stakeholders: A Pathway to Trustworthy AI Integration Kexin Ding Mu Zhou Akshay Chaudhari Shaoting Zhang Dimitris N. Metaxas LM&MA 43 0 0 02 May 2025
Unlearning Sensitive Information in Multimodal LLMs: Benchmark and Attack-Defense Evaluation Vaidehi Patil Yi-Lin Sung Peter Hase Jie Peng Tianlong Chen Mohit Bansal AAML MU 81 3 0 01 May 2025
Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment Xiaotian Zhang Ruizhe Chen Yang Feng Zuozhu Liu 40 0 0 17 Apr 2025
Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data Shuai Zhao Linchao Zhu Yi Yang 39 1 0 14 Apr 2025
A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future Jialun Zhong Wei Shen Yanzeng Li Songyang Gao Hua Lu Yicheng Chen Yang Zhang Wei Zhou Jinjie Gu Lei Zou LRM 38 2 0 12 Apr 2025
2D-Curri-DPO: Two-Dimensional Curriculum Learning for Direct Preference Optimization Mengyang Li Zhong Zhang 27 0 0 10 Apr 2025
JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration Yunlong Lin Zixu Lin Haoyu Chen Panwang Pan C. Li Sixiang Chen Yeying Jin W. J. Li Xinghao Ding 25 1 0 05 Apr 2025
Instruction-Oriented Preference Alignment for Enhancing Multi-Modal Comprehension Capability of MLLMs Zitian Wang Yue Liao Kang Rong Fengyun Rao Yibo Yang Si Liu 70 0 0 26 Mar 2025
InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment Y. Lu Qichao Wang H. Cao Xierui Wang Xiaoyin Xu Min Zhang 59 0 0 24 Mar 2025
Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization Zefeng Zhang Hengzhu Tang Jiawei Sheng Zhenyu Zhang Yiming Ren Zhenyang Li Dawei Yin Duohe Ma Tingwen Liu 47 0 0 23 Mar 2025
DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning R. Zhao Junliang Ye Z. Wang Guangce Liu Yiwen Chen Yikai Wang J. Zhu AI4CE 45 0 0 19 Mar 2025
Augmented Adversarial Trigger Learning Zhe Wang Yanjun Qi 53 0 0 16 Mar 2025
OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses Angela Lopez-Cardona Sebastian Idesis Miguel Barreda-Ángeles Sergi Abadal Ioannis Arapakis 46 0 0 13 Mar 2025
SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator Trajectories Muzhi Zhu Yuzhuo Tian Hao Chen Chunluan Zhou Qingpei Guo Y. Liu M. Yang Chunhua Shen MLLM VLM 72 0 0 11 Mar 2025
Towards Autonomous Reinforcement Learning for Real-World Robotic Manipulation with Large Language Models Niccolò Turcato Matteo Iovino Aris Synodinos Alberto Dalla Libera R. Carli Pietro Falco LM&Ro 43 0 0 06 Mar 2025
Process-based Self-Rewarding Language Models Shimao Zhang Xiao Liu Xin Zhang Junxiao Liu Zheheng Luo Shujian Huang Yeyun Gong ReLM SyDa LRM 93 2 0 05 Mar 2025
Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable Tiansheng Huang Sihao Hu Fatih Ilhan Selim Furkan Tekin Zachary Yahn Yichang Xu Ling Liu 53 8 0 01 Mar 2025
Robust Multi-Objective Preference Alignment with Online DPO Raghav Gupta Ryan Sullivan Yunxuan Li Samrat Phatale Abhinav Rastogi 34 0 0 01 Mar 2025
Self-rewarding correction for mathematical reasoning Wei Xiong Hanning Zhang Chenlu Ye Lichang Chen Nan Jiang Tong Zhang ReLM KELM LRM 67 9 0 26 Feb 2025
AMPO: Active Multi-Preference Optimization Taneesh Gupta Rahul Madhavan Xuchao Zhang Chetan Bansal Saravan Rajmohan 55 0 0 25 Feb 2025
Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data Siqi Guo Ilgee Hong Vicente Balmaseda Changlong Yu Liang Qiu Xin Liu Haoming Jiang Tuo Zhao Tianbao Yang 43 0 0 25 Feb 2025
Larger or Smaller Reward Margins to Select Preferences for Alignment? Kexin Huang Junkang Wu Ziqian Chen Xue Wang Jinyang Gao Bolin Ding Jiancan Wu Xiangnan He X. Wang 42 0 0 25 Feb 2025
Pastiche Novel Generation Creating: Fan Fiction You Love in Your Favorite Author's Style Xueran Han Yuhan Liu Mingzhe Li W. Liu Sen Hu Rui Yan Zhiqiang Xu Xiuying Chen 62 0 0 24 Feb 2025
Improving LLM General Preference Alignment via Optimistic Online Mirror Descent Yuheng Zhang Dian Yu Tao Ge Linfeng Song Zhichen Zeng Haitao Mi Nan Jiang Dong Yu 56 1 0 24 Feb 2025
SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters Teng Xiao Yige Yuan Z. Chen Mingxiao Li Shangsong Liang Z. Ren V. Honavar 93 5 0 21 Feb 2025
Simplify RLHF as Reward-Weighted SFT: A Variational Method Yuhao Du Z. Li Pengyu Cheng Zhihong Chen Yuejiao Xie Xiang Wan Anningzhe Gao 35 1 0 20 Feb 2025
Faster WIND: Accelerating Iterative Best-of- $N$ Distillation for LLM Alignment Tong Yang Jincheng Mei H. Dai Zixin Wen Shicong Cen Dale Schuurmans Yuejie Chi Bo Dai 43 4 0 20 Feb 2025
DrugImproverGPT: A Large Language Model for Drug Optimization with Fine-Tuning via Structured Policy Optimization Xuefeng Liu Songhao Jiang Siyu Chen Zhuoran Yang Yuxin Chen Ian T. Foster Rick L. Stevens LM&MA OffRL 48 0 0 11 Feb 2025
IPO: Iterative Preference Optimization for Text-to-Video Generation Xiaomeng Yang Zhiyu Tan Xuecheng Nie VGen 101 1 0 04 Feb 2025
Panacea: Mitigating Harmful Fine-tuning for Large Language Models via Post-fine-tuning Perturbation Y. Wang Tiansheng Huang Li Shen H. Yao Haotian Luo Rui Liu Naiqiang Tan Jiaxing Huang Dacheng Tao AAML MoMe CLL 111 2 0 30 Jan 2025
LiPO: Listwise Preference Optimization through Learning-to-Rank Tianqi Liu Zhen Qin Junru Wu Jiaming Shen Misha Khalman ... Mohammad Saleh Simon Baumgartner Jialu Liu Peter J. Liu Xuanhui Wang 133 48 0 28 Jan 2025
Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning Hao Sun M. Schaar 92 14 0 28 Jan 2025
Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model Yueqin Yin Shentao Yang Yujia Xie Ziyi Yang Yuting Sun Hany Awadalla Weizhu Chen Mingyuan Zhou 48 0 0 07 Jan 2025
Improving GenIR Systems Based on User Feedback Qingyao Ai Zhicheng Dou Min Zhang 99 0 0 06 Jan 2025
WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct Haipeng Luo Qingfeng Sun Can Xu Pu Zhao Jian-Guang Lou ... Xiubo Geng Qingwei Lin Shifeng Chen Yansong Tang Dongmei Zhang OSLM LRM 108 408 0 03 Jan 2025
FaGeL: Fabric LLMs Agent empowered Embodied Intelligence Evolution with Autonomous Human-Machine Collaboration Jia Liu Min Chen LM&Ro AI4CE 32 2 0 28 Dec 2024
Understanding the Logic of Direct Preference Alignment through Logic Kyle Richardson Vivek Srikumar Ashish Sabharwal 85 2 0 23 Dec 2024
REFA: Reference Free Alignment for multi-preference optimization Taneesh Gupta Rahul Madhavan Xuchao Zhang Chetan Bansal Saravan Rajmohan 89 1 0 20 Dec 2024
VideoDPO: Omni-Preference Alignment for Video Diffusion Generation Runtao Liu Haoyu Wu Zheng Ziqiang Chen Wei Yingqing He Renjie Pi Qifeng Chen VGen 83 11 0 18 Dec 2024
Energy-Based Preference Model Offers Better Offline Alignment than the Bradley-Terry Preference Model Yuzhong Hong Hanshan Zhang Junwei Bao Hongfei Jiang Yang Song OffRL 77 1 0 18 Dec 2024
Preference-Oriented Supervised Fine-Tuning: Favoring Target Model Over Aligned Large Language Models Yuchen Fan Yuzhong Hong Qiushi Wang Junwei Bao Hongfei Jiang Yang Song 80 1 0 17 Dec 2024
Reinforcement Learning Enhanced LLMs: A Survey Shuhe Wang Shengyu Zhang J. Zhang Runyi Hu Xiaoya Li Tianwei Zhang Jiwei Li Fei Wu G. Wang Eduard H. Hovy OffRL 134 7 0 05 Dec 2024
ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning Yang Wu Huayi Zhang Yizheng Jiao Lin Ma Xiaozhong Liu Jinhong Yu Dongyu Zhang Dezhi Yu Wei Xu 78 1 0 01 Dec 2024
Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering Xinyan Guan Yanjiang Liu Xinyu Lu Boxi Cao Ben He ... Le Sun Jie Lou Bowen Yu Y. Lu Hongyu Lin ALM 79 2 0 18 Nov 2024
SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization Hongrui Jia Chaoya Jiang Haiyang Xu Wei Ye Mengfan Dong Ming Yan Ji Zhang Fei Huang Shikun Zhang MLLM 89 2 0 17 Nov 2024
Structured Dialogue System for Mental Health: An LLM Chatbot Leveraging the PM+ Guidelines Yixiang Chen Xinyu Zhang Jinran Wang Xurong Xie Nan Yan Hui Chen Lan Wang AI4MH 37 2 0 16 Nov 2024
Approximated Variational Bayesian Inverse Reinforcement Learning for Large Language Model Alignment Yuang Cai Yuyu Yuan Jinsheng Shi Qinhong Lin 35 0 0 14 Nov 2024
Adapting While Learning: Grounding LLMs for Scientific Problems with Intelligent Tool Usage Adaptation Bohan Lyu Yadi Cao Duncan Watson-Parris Leon Bergen Taylor Berg-Kirkpatrick Rose Yu 61 3 0 01 Nov 2024