Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge

28 July 2024

Jason Weston

Papers citing "Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge"

50 / 60 papers shown

Title
Improving Model Alignment Through Collective Intelligence of Open-Source LLMS Junlin Wang Roy Xie Shang Zhu Jue Wang Ben Athiwaratkun Bhuwan Dhingra S. Song Ce Zhang James Y. Zou ALM 24 0 0 05 May 2025
Sailing AI by the Stars: A Survey of Learning from Rewards in Post-Training and Test-Time Scaling of Large Language Models Xiaobao Wu LRM 60 0 0 05 May 2025
Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models Bang Zhang Ruotian Ma Qingxuan Jiang Peisong Wang Jiaqi Chen ... Fanghua Ye Jian Li Yifan Yang Zhaopeng Tu Xiaolong Li LLMAG ELM ALM 95 25 1 01 May 2025
DeepCritic: Deliberate Critique with Large Language Models Wenkai Yang Jingwen Chen Yankai Lin Ji-Rong Wen ALM LRM 28 0 0 01 May 2025
ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning Jingyang Yi Jiazheng Wang ReLM OODD LRM 61 0 0 30 Apr 2025
Anyprefer: An Agentic Framework for Preference Data Synthesis Yiyang Zhou Z. Wang Tianle Wang Shangyu Xing Peng Xia ... Chetan Bansal Weitong Zhang Ying Wei Mohit Bansal Huaxiu Yao 54 0 0 27 Apr 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong X. Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Yu Jiang ALM ELM 84 0 0 26 Apr 2025
Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM Judgments Y. Li Jama Hussein Mohamud Chongren Sun Di Wu Benoit Boulet LLMAG ELM 70 0 0 23 Apr 2025
AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset Ivan Moshkov Darragh Hanley Ivan Sorokin Shubham Toshniwal Christof Henkel Benedikt D. Schifferer Wei Du Igor Gitman ReLM LRM 40 1 0 23 Apr 2025
Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Survey Ahsan Bilal Muhammad Ahmed Mohsin Muhammad Umer Muhammad Awais Khan Bangash Muhammad Ali Jamshed LLMAG LRM AI4CE 47 0 0 20 Apr 2025
Efficient MAP Estimation of LLM Judgment Performance with Prior Transfer Huaizhi Qu Inyoung Choi Zhen Tan Song Wang Sukwon Yun Qi Long Faizan Siddiqui Kwonjoon Lee Tianlong Chen 41 0 0 17 Apr 2025
A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future Jialun Zhong Wei Shen Yanzeng Li Songyang Gao Hua Lu Yicheng Chen Yang Zhang Wei Zhou Jinjie Gu Lei Zou LRM 38 1 0 12 Apr 2025
VideoAgent2: Enhancing the LLM-Based Agent System for Long-Form Video Understanding by Uncertainty-Aware CoT Zhuo Zhi Qiangqiang Wu Minghe shen W. J. Li Yinchuan Li Kun Shao Kaiwen Zhou LLMAG 33 0 0 06 Apr 2025
Inference-Time Scaling for Generalist Reward Modeling Zijun Liu P. Wang R. Xu Shirong Ma Chong Ruan Peng Li Yang Janet Liu Y. Wu OffRL LRM 46 9 0 03 Apr 2025
Process Reward Modeling with Entropy-Driven Uncertainty Lang Cao Renhong Chen Yingtian Zou Chao Peng Wu Ning ... Y. Wang Peishuo Su Mofan Peng Zijie Chen Yitong Li 34 0 0 28 Mar 2025
Reasoning Beyond Limits: Advances and Open Problems for LLMs M. Ferrag Norbert Tihanyi Merouane Debbah ELM OffRL LRM AI4CE 59 2 0 26 Mar 2025
InCo-DPO: Balancing Distribution Shift and Data Quality for Enhanced Preference Optimization Yunan Wang Jijie Li Bo Zhang Liangdong Wang Guang Liu 58 0 0 20 Mar 2025
OASST-ETC Dataset: Alignment Signals from Eye-tracking Analysis of LLM Responses Angela Lopez-Cardona Sebastian Idesis Miguel Barreda-Ángeles Sergi Abadal Ioannis Arapakis 43 0 0 13 Mar 2025
Conformal Prediction for Image Segmentation Using Morphological Prediction Sets Luca Mossina Corentin Friedrich MedIm 54 1 0 07 Mar 2025
SEOE: A Scalable and Reliable Semantic Evaluation Framework for Open Domain Event Detection Yi-Fan Lu Xian-Ling Mao Tian Lan Tong Zhang Yu-Shi Zhu Heyan Huang 47 0 0 05 Mar 2025
Process-based Self-Rewarding Language Models Shimao Zhang Xiao Liu Xin Zhang Junxiao Liu Zheheng Luo Shujian Huang Yeyun Gong ReLM SyDa LRM 93 2 0 05 Mar 2025
Towards Effective and Efficient Context-aware Nucleus Detection in Histopathology Whole Slide Images Zhongyi Shui Ruizhe Guo Honglin Li Yuxuan Sun Yunlong Zhang Chenglu Zhu Jiatong Cai Pingyi Chen Yanzhou Su Lin Yang 44 0 0 04 Mar 2025
IPO: Your Language Model is Secretly a Preference Classifier Shivank Garg Ayush Singh Shweta Singh Paras Chopra 47 1 0 22 Feb 2025
M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation Zhaopeng Feng Jiayuan Su Jiamei Zheng Jiahan Ren Yan Zhang Jian Wu Hongwei Wang Zuozhu Liu ELM 198 0 0 21 Feb 2025
Self-Consistency of the Internal Reward Models Improves Self-Rewarding Language Models Xin Zhou Yiwen Guo Ruotian Ma Tao Gui Qi Zhang Xuanjing Huang LRM 81 2 0 13 Feb 2025
ARIES: Stimulating Self-Refinement of Large Language Models by Iterative Preference Optimization Yongcheng Zeng Xinyu Cui Xuanfa Jin Guoqing Liu Zexu Sun ... Dong Li Ning Yang Jianye Hao H. Zhang J. Wang LRM LLMAG 74 1 0 08 Feb 2025
From Drafts to Answers: Unlocking LLM Potential via Aggregation Fine-Tuning Yafu Li Zhilin Wang Tingchen Fu Ganqu Cui Sen Yang Yu Cheng 40 1 0 21 Jan 2025
WarriorCoder: Learning from Expert Battles to Augment Code Large Language Models Huawen Feng Pu Zhao Qingfeng Sun Can Xu Fangkai Yang ... Qianli Ma Qingwei Lin Saravan Rajmohan Dongmei Zhang Qi Zhang AAML ALM 62 0 0 23 Dec 2024
SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models Jiale Cheng Xiao-Chang Liu C. Wang Xiaotao Gu Y. Lu Dan Zhang Yuxiao Dong J. Tang Hongning Wang Minlie Huang LRM 117 3 0 16 Dec 2024
TapeAgents: a Holistic Framework for Agent Development and Optimization Dzmitry Bahdanau Nicolas Angelard-Gontier Gabriel Huang Ehsan Kamalloo Rafael Pardinas ... Jordan Prince Tremblay Karam Ghanem S. Parikh Mitul Tiwari Quaizar Vohra 65 2 0 11 Dec 2024
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge Dawei Li Bohan Jiang Liangjie Huang Alimohammad Beigi Chengshuai Zhao ... Canyu Chen Tianhao Wu Kai Shu Lu Cheng Huan Liu ELM AILaw 106 61 0 25 Nov 2024
Self-Generated Critiques Boost Reward Modeling for Language Models Yue Yu Zhengxing Chen Aston Zhang L Tan Chenguang Zhu ... Suchin Gururangan Chao-Yue Zhang Melanie Kambadur Dhruv Mahajan Rui Hou LRM ALM 84 14 0 25 Nov 2024
Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineering Xinyan Guan Yanjiang Liu Xinyu Lu Boxi Cao Ben He ... Le Sun Jie Lou Bowen Yu Y. Lu Hongyu Lin ALM 79 2 0 18 Nov 2024
Anchored Alignment for Self-Explanations Enhancement Luis Felipe Villa-Arenas Ata Nizamoglu Qianli Wang Sebastian Möller Vera Schmitt 16 0 0 17 Oct 2024
Thinking LLMs: General Instruction Following with Thought Generation Tianhao Wu Janice Lan Weizhe Yuan Jiantao Jiao Jason Weston Sainbayar Sukhbaatar LRM 16 12 0 14 Oct 2024
MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimization Yougang Lyu Lingyong Yan Zihan Wang Dawei Yin Pengjie Ren Maarten de Rijke Z. Z. Ren 55 6 0 10 Oct 2024
Self-Boosting Large Language Models with Synthetic Preference Data Qingxiu Dong Li Dong Xingxing Zhang Zhifang Sui Furu Wei SyDa 34 1 0 09 Oct 2024
Towards Self-Improvement of LLMs via MCTS: Leveraging Stepwise Knowledge with Curriculum Preference Learning Xiyao Wang Linfeng Song Ye Tian Dian Yu Baolin Peng Haitao Mi Furong Huang Dong Yu LRM 47 9 0 09 Oct 2024
MentalArena: Self-play Training of Language Models for Diagnosis and Treatment of Mental Health Disorders Cheng-rong Li May Fung Qingyun Wang Chi Han Manling Li Jindong Wang Heng Ji AI4MH 43 0 0 09 Oct 2024
O1 Replication Journey: A Strategic Progress Report -- Part 1 Yiwei Qin Xuefeng Li Haoyang Zou Yixiu Liu Shijie Xia ... Yixin Ye Weizhe Yuan Hector Liu Y. Li Pengfei Liu VLM 37 67 0 08 Oct 2024
Self-rationalization improves LLM as a fine-grained judge Prapti Trivedi Aditya Gulati Oliver Molenschot Meghana Arakkal Rajeev Rajkumar Ramamurthy Keith Stevens Tanveesh Singh Chaudhery Jahnavi Jambholkar James Y. Zou Nazneen Rajani LRM 25 3 0 07 Oct 2024
Better Instruction-Following Through Minimum Bayes Risk Ian Wu Patrick Fernandes Amanda Bertsch Seungone Kim Sina Pakazad Graham Neubig 48 9 0 03 Oct 2024
Generative Reward Models Dakota Mahan Duy Phung Rafael Rafailov Chase Blagden Nathan Lile Louis Castricato Jan-Philipp Fränken Chelsea Finn Alon Albalak VLM SyDa OffRL 27 24 0 02 Oct 2024
Seeing Eye to AI: Human Alignment via Gaze-Based Response Rewards for Large Language Models Angela Lopez-Cardona Carlos Segura Alexandros Karatzoglou Sergi Abadal Ioannis Arapakis ALM 48 2 0 02 Oct 2024
Beyond Scalar Reward Model: Learning Generative Judge from Preference Data Ziyi Ye Xiangsheng Li Qiuchi Li Qingyao Ai Yujia Zhou Wei Shen Dong Yan Yiqun Liu 36 10 0 01 Oct 2024
Aligning Language Models Using Follow-up Likelihood as Reward Signal Chen Zhang Dading Chong Feng Jiang Chengguang Tang Anningzhe Gao Guohua Tang Haizhou Li ALM 29 2 0 20 Sep 2024
Model-in-the-Loop (MILO): Accelerating Multimodal AI Data Annotation with LLMs Yifan Wang David Stevens Pranay Shah Wenwen Jiang Miao Liu ... Boying Gong Daniel Lee Jiabo Hu Ning Zhang Bob Kamma 30 1 0 16 Sep 2024
AIPO: Improving Training Objective for Iterative Preference Optimization Yaojie Shen Xinyao Wang Yulei Niu Ying Zhou Lexin Tang Libo Zhang Fan Chen Longyin Wen 20 2 0 13 Sep 2024
From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks Andreas Stephan D. Zhu Matthias Aßenmacher Xiaoyu Shen Benjamin Roth ELM 45 4 0 06 Sep 2024
Towards a Unified View of Preference Learning for Large Language Models: A Survey Bofei Gao Feifan Song Yibo Miao Zefan Cai Z. Yang ... Houfeng Wang Zhifang Sui Peiyi Wang Baobao Chang Baobao Chang 41 11 0 04 Sep 2024