Title
TRAIL: Trace Reasoning and Agentic Issue Localization Darshan Deshpande Varun Gangal Hersh Mehta Jitin Krishnan Anand Kannappan Rebecca Qian 20 0 0 13 May 2025
LLAMAPIE: Proactive In-Ear Conversation Assistants Tuochao Chen Nicholas Batchelder Alisa Liu Noah A. Smith Shyamnath Gollakota 60 0 0 07 May 2025
LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning Joy Lim Jia Yin Daniel Zhang-Li Jifan Yu H. Li Shangqing Tu ... Zhiyuan Liu Huiqin Liu Lei Hou Juanzi Li Bin Xu 22 0 0 04 May 2025
VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations for Synthetic Videos Zongxia Li Xiyang Wu Yubin Qin Guangyao Shi Hongyang Du Dinesh Manocha Tianyi Zhou Jordan Boyd-Graber MLLM 41 0 0 02 May 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong X. Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Yu Jiang ALM ELM 84 0 0 26 Apr 2025
Process Reward Models That Think Muhammad Khalifa Rishabh Agarwal Lajanugen Logeswaran Jaekyeom Kim Hao Peng Moontae Lee Honglak Lee Lu Wang OffRL ALM LRM 44 1 0 23 Apr 2025
Trillion 7B Technical Report Sungjun Han Juyoung Suk Suyeong An Hyungguk Kim Kyuseok Kim Wonsuk Yang Seungtaek Choi Jamin Shin 51 0 0 21 Apr 2025
Med-CoDE: Medical Critique based Disagreement Evaluation Framework Mohit Gupta Akiko Aizawa R. Shah LM&MA ELM 30 0 0 21 Apr 2025
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators Yilun Zhou Austin Xu Peifeng Wang Caiming Xiong Shafiq R. Joty ELM ALM LRM 45 2 0 21 Apr 2025
PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom Production Large Language Model Pipelines Reya Vir Shreya Shankar Harrison Chase Will Fu-Hinthorn Aditya G. Parameswaran AI4TS 32 0 0 20 Apr 2025
Pairwise or Pointwise? Evaluating Feedback Protocols for Bias in LLM-Based Evaluation Tuhina Tripathi Manya Wadhwa Greg Durrett S. Niekum 27 0 0 20 Apr 2025
Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCR Y. Zhang Tianyi Liang Xinyue Huang Erfei Cui Xu Guo Pei Chu Chenhui Li Ru Zhang Wenhai Wang Gongshen Liu 61 0 0 15 Apr 2025
EMAFusion: A Self-Optimizing System for Seamless LLM Selection and Integration Soham Shah Kumar Shridhar Surojit Chatterjee Souvik Sen 34 0 0 14 Apr 2025
Large Language Models as Span Annotators Zdeněk Kasner Vilém Zouhar Patrícia Schmidtová Ivan Kartáč Kristýna Onderková Ondřej Plátek Dimitra Gkatzia Saad Mahamood Ondrej Dusek Simone Balloccu ALM 30 0 0 11 Apr 2025
DocAgent: A Multi-Agent System for Automated Code Documentation Generation Dayu Yang Antoine Simoulin Xin Qian Xiaoyi Liu Yuwei Cao Zhaopu Teng Grey Yang LLMAG 54 0 0 11 Apr 2025
AgentAda: Skill-Adaptive Data Analytics for Tailored Insight Discovery Amirhossein Abaskohi A. Ramesh Shailesh Nanisetty Chirag Goel David Vazquez Christopher Pal Spandana Gella Giuseppe Carenini I. Laradji 29 0 0 10 Apr 2025
Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge Riccardo Cantini A. Orsino Massimo Ruggiero Domenico Talia AAML ELM 40 0 0 10 Apr 2025
Toward Holistic Evaluation of Recommender Systems Powered by Generative Models Yashar Deldjoo Nikhil Mehta M. Sathiamoorthy Shuai Zhang Pablo Castells Julian McAuley EGVM ELM 64 1 0 09 Apr 2025
Cultural Learning-Based Culture Adaptation of Language Models Chen Cecilia Liu Anna Korhonen Iryna Gurevych 39 0 0 03 Apr 2025
When To Solve, When To Verify: Compute-Optimal Problem Solving and Generative Verification for LLM Reasoning Nishad Singhi Hritik Bansal Arian Hosseini Aditya Grover Kai-Wei Chang Marcus Rohrbach Anna Rohrbach OffRL LRM 37 0 0 01 Apr 2025
Zero-shot Benchmarking: A Framework for Flexible and Scalable Automatic Evaluation of Language Models José P. Pombal Nuno M. Guerreiro Ricardo Rei André F. T. Martins ALM 66 0 0 01 Apr 2025
Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Compute Jianhao Chen Zishuo Xun Bocheng Zhou Han Qi Qiaosheng Zhang ... Wei Hu Yuzhong Qu W. Ouyang Wanli Ouyang Shuyue Hu 74 0 0 01 Apr 2025
KOFFVQA: An Objectively Evaluated Free-form VQA Benchmark for Large Vision-Language Models in the Korean Language Yoonshik Kim Jaeyoon Jung 35 0 0 31 Mar 2025
XL-Instruct: Synthetic Data for Cross-Lingual Open-Ended Generation Vivek Iyer Ricardo Rei Pinzhen Chen Alexandra Birch SyDa LM&MA 66 0 0 29 Mar 2025
Scaling Evaluation-time Compute with Reasoning Models as Process Evaluators Seungone Kim Ian Wu Jinu Lee Xiang Yue Seongyun Lee ... Kiril Gashteovski Carolin (Haas) Lawrence J. Hockenmaier Graham Neubig Sean Welleck LRM 48 2 0 25 Mar 2025
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications Jian-Yu Guan J. Wu J. Li Chuanqi Cheng Wei Yu Wu LM&MA 69 0 0 21 Mar 2025
Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settings Austin Xu Srijan Bansal Yifei Ming Semih Yavuz Shafiq R. Joty ELM 89 3 0 19 Mar 2025
REPA: Russian Error Types Annotation for Evaluating Text Generation and Judgment Capabilities Alexander Pugachev Alena Fenogenova Vladislav Mikhailov Ekaterina Artemova 32 0 0 17 Mar 2025
OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMs Ivan Kartáč Mateusz Lango Ondrej Dusek ELM 46 1 0 14 Mar 2025
MentalChat16K: A Benchmark Dataset for Conversational Mental Health Assistance Jia Xu Tianyi Wei Bojian Hou Patryk Orzechowski Shu Yang Ruochen Jin Rachael Paulbeck Joost B. Wagenaar George Demiris Li Shen AI4MH 37 0 0 13 Mar 2025
DeFine: A Decomposed and Fine-Grained Annotated Dataset for Long-form Article Generation Ming Wang Fang Wang Minghao Hu Li He Haiyang Wang ... Li Li Zhunchen Luo Wei Luo Xiaoying Bai Guotong Geng 61 0 0 10 Mar 2025
Is Your Video Language Model a Reliable Judge? M. Liu Wensheng Zhang 56 1 0 07 Mar 2025
RocketEval: Efficient Automated LLM Evaluation via Grading Checklist Tianjun Wei Wei Wen Ruizhi Qiao Xing Sun Jianghong Ma ALM ELM 45 1 0 07 Mar 2025
How Do Hackathons Foster Creativity? Towards AI Collaborative Evaluation of Creativity at Scale Jeanette Falk Yiyi Chen Janet Rafner Mike Zhang Johannes Bjerva Alexander Nolte 53 1 0 06 Mar 2025
Targeted Distillation for Sentiment Analysis Yice Zhang Guangyu Xie Jingjie Lin Jianzhu Bao Qianlong Wang Xi Zeng Ruifeng Xu 53 0 0 05 Mar 2025
Teaching Metric Distance to Autoregressive Multimodal Foundational Models Jiwan Chung Saejin Kim Yongrae Jo J. Park Dongjun Min Youngjae Yu 69 0 0 04 Mar 2025
SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction Lu Dai Yijie Xu Jinhui Ye Hao Liu Hui Xiong 3DV RALM 74 2 0 03 Mar 2025
BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge Terry Tong Fei-Yue Wang Zhe Zhao M. Chen AAML ELM 37 1 0 01 Mar 2025
Judge as A Judge: Improving the Evaluation of Retrieval-Augmented Generation through the Judge-Consistency of Large Language Models Shuliang Liu Xinze Li Zhenghao Liu Yukun Yan Cheng Yang Zheni Zeng Zhiyuan Liu Maosong Sun Ge Yu RALM 91 1 0 26 Feb 2025
Learning to Align Multi-Faceted Evaluation: A Unified and Robust Framework Kaishuai Xu Tiezheng YU Wenjun Hou Yi Cheng Liangyou Li Xin Jiang Lifeng Shang Q. Liu Wenjie Li ELM 66 0 0 26 Feb 2025
Independent Mobility GPT (IDM-GPT): A Self-Supervised Multi-Agent Large Language Model Framework for Customized Traffic Mobility Analysis Using Machine Learning Models Fengze Yang Xiaoyue Cathy Liu Lingjiu Lu Bingzhang Wang Chenxi 38 0 0 25 Feb 2025
Optimizing Model Selection for Compound AI Systems Lingjiao Chen Jared Quincy Davis Boris Hanin Peter Bailis Matei A. Zaharia James Y. Zou Ion Stoica 54 1 0 20 Feb 2025
An Empirical Analysis of Uncertainty in Large Language Model Evaluations Qiujie Xie Qingqiu Li Zhuohao Yu Yuejie Zhang Yue Zhang Linyi Yang ELM 58 1 0 15 Feb 2025
Preference Leakage: A Contamination Problem in LLM-as-a-judge Dawei Li Renliang Sun Yue Huang Ming Zhong Bohan Jiang J. Han X. Zhang Wei Wang Huan Liu 65 11 0 03 Feb 2025
CoddLLM: Empowering Large Language Models for Data Analytics Jiani Zhang Hengrui Zhang Rishav Chakravarti Yiqun Hu Patrick K. L. Ng Asterios Katsifodimos Huzefa Rangwala George Karypis Alon Halevy SyDa ELM 101 0 0 01 Feb 2025
Learning to Explore and Select for Coverage-Conditioned Retrieval-Augmented Generation Takyoung Kim Kyungjae Lee Y. Jang Ji Yong Cho Gangwoo Kim Minseok Cho Moontae Lee 95 0 0 28 Jan 2025
Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judge Aparna Elangovan Jongwoo Ko Lei Xu Mahsa Elyasi Ling Liu S. Bodapati Dan Roth 41 5 0 28 Jan 2025
A 2-step Framework for Automated Literary Translation Evaluation: Its Promises and Pitfalls Sheikh Shafayat Dongkeun Yoon Woori Jang Jiwoo Choi Alice H. Oh Seohyon Jung 91 1 0 03 Jan 2025
LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation Eunsu Kim Juyoung Suk Seungone Kim Niklas Muennighoff Dongkwan Kim Alice H. Oh ELM 78 1 0 31 Dec 2024
Disentangling Preference Representation and Text Generation for Efficient Individual Preference Alignment Jianfei Zhang Jun Bai B. Li Yanmeng Wang Rumei Li Chenghua Lin Wenge Rong 41 0 0 31 Dec 2024