Dynamic Evaluation of Large Language Models by Meta Probing Agents

21 February 2024

Xing Xie

Papers citing "Dynamic Evaluation of Large Language Models by Meta Probing Agents"

26 / 26 papers shown

Title
Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs Chun-Hsiao Yeh Chenyu Wang Shengbang Tong Ta-Ying Cheng Rouyu Wang Tianzhe Chu Yuexiang Zhai Yubei Chen Shenghua Gao Yi Ma LRM 61 0 0 21 Apr 2025
MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models Pengfei Zhou Fanrui Zhang Xiaopeng Peng Zhaopan Xu Jiaxin Ai ... Kai Wang Xiaojun Chang Wenqi Shao Yang You K. Zhang ELM LRM 30 0 0 08 Apr 2025
The Emperor's New Clothes in Benchmarking? A Rigorous Examination of Mitigation Strategies for LLM Benchmark Data Contamination Yifan Sun Han Wang Dongbai Li Gang Wang Huan Zhang AAML 53 0 0 20 Mar 2025
RefuteBench 2.0 -- Agentic Benchmark for Dynamic Evaluation of LLM Responses to Refutation Instruction Jianhao Yan Yun Luo Yue Zhang LLMAG 50 1 0 25 Feb 2025
Your Language Model May Think Too Rigidly: Achieving Reasoning Consistency with Symmetry-Enhanced Training Yihang Yao Zhepeng Cen Miao Li William Jongwon Han Yuyou Zhang Emerson Liu Zuxin Liu Chuang Gan Ding Zhao ReLM LRM 70 0 0 25 Feb 2025
Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluation Simin Chen Yiming Chen Zexin Li Yifan Jiang Zhongwei Wan ... Dezhi Ran Tianle Gu H. Li Tao Xie Baishakhi Ray 41 2 0 23 Feb 2025
Unbiased Evaluation of Large Language Models from a Causal Perspective Meilin Chen Jian Tian Liang Ma Di Xie Weijie Chen Jiang Zhu ALM ELM 52 0 0 10 Feb 2025
Breaking Focus: Contextual Distraction Curse in Large Language Models Yue Huang Yanbo Wang Zixiang Xu Chujie Gao Siyuan Wu Jiayi Ye Xiuying Chen Pin-Yu Chen X. Zhang AAML 43 1 0 03 Feb 2025
LLM-Powered Benchmark Factory: Reliable, Generic, and Efficient Peiwen Yuan Shaoxiong Feng Yiwei Li X. U. Wang Y. Zhang Jiayi Shi Chuyi Tan Boyuan Pan Yao Hu Kan Li 66 2 0 02 Feb 2025
Addressing Data Leakage in HumanEval Using Combinatorial Test Design Jeremy S. Bradbury Riddhi More 61 1 0 02 Dec 2024
MMGenBench: Fully Automatically Evaluating LMMs from the Text-to-Image Generation Perspective Hailang Huang Yong Wang Zixuan Huang Huaqiu Li Tongwen Huang Xiangxiang Chu Richong Zhang MLLM LM&MA EGVM 83 1 0 21 Nov 2024
Project MPG: towards a generalized performance benchmark for LLM capabilities Lucas Spangher Tianle Li William Arnold Nick Masiewicki Xerxes Dotiwalla Rama Parusmathi Peter Grabowski Eugene Ie Dan Gruhl 36 0 0 28 Oct 2024
AutoBench-V: Can Large Vision-Language Models Benchmark Themselves? Han Bao Yue Huang Yanbo Wang Jiayi Ye Xiangqi Wang Xiuying Chen Mohamed Elhoseiny X. Zhang Mohamed Elhoseiny Xiangliang Zhang 47 7 0 28 Oct 2024
Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping Yue Yang S. Zhang Wenqi Shao Kaipeng Zhang Yi Bin Yu Wang Ping Luo 28 3 0 11 Oct 2024
Reliable and diverse evaluation of LLM medical knowledge mastery Yuxuan Zhou Xien Liu Chen Ning Xiao Zhang Ji Wu MedIm 29 0 0 22 Sep 2024
Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklist Zihao Zhou Shudong Liu Maizhen Ning Wei Liu Jindong Wang Derek F. Wong Xiaowei Huang Qiufeng Wang Kaizhu Huang ELM LRM 61 23 0 11 Jul 2024
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations Md Tahmid Rahman Laskar Sawsan Alqahtani M Saiful Bari Mizanur Rahman Mohammad Abdullah Matin Khan ... Chee Wei Tan Md. Rizwan Parvez Enamul Hoque Shafiq R. Joty Jimmy Huang ELM ALM 19 25 0 04 Jul 2024
DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph Zhehao Zhang Jiaao Chen Diyi Yang LRM 32 7 0 25 Jun 2024
AgentReview: Exploring Peer Review Dynamics with LLM Agents Yiqiao Jin Qinlin Zhao Yiyang Wang Hao Chen Kaijie Zhu Yijia Xiao Jindong Wang LLMAG 35 13 0 18 Jun 2024
Benchmark Data Contamination of Large Language Models: A Survey Cheng Xu Shuhao Guan Derek Greene Mohand-Tahar Kechadi ELM ALM 34 38 0 06 Jun 2024
Benchmarking Benchmark Leakage in Large Language Models Ruijie Xu Zengzhi Wang Run-Ze Fan Pengfei Liu 53 42 0 29 Apr 2024
The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey Tula Masterman Sandi Besen Mason Sawtell Alex Chao LM&Ro LLMAG 32 42 0 17 Apr 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 230 291 0 18 Jan 2024
Don't Make Your LLM an Evaluation Benchmark Cheater Kun Zhou Yutao Zhu Zhipeng Chen Wentong Chen Wayne Xin Zhao Xu Chen Yankai Lin Ji-Rong Wen Jiawei Han ELM 105 136 0 03 Nov 2023
Revealing the structure of language model capabilities Ryan Burnell Hank Hao Andrew R. A. Conway José Hernández Orallo ELM 37 17 0 14 Jun 2023
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022