Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators

6 April 2024

Papers citing "Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators"

50 / 256 papers shown

Title
Assessing Robustness to Spurious Correlations in Post-Training Language Models Julia Shuieh Prasann Singhal Apaar Shanker John Heyer George Pu Samuel Denton LRM 12 0 0 09 May 2025
Improving Model Alignment Through Collective Intelligence of Open-Source LLMS Junlin Wang Roy Xie Shang Zhu Jue Wang Ben Athiwaratkun Bhuwan Dhingra S. Song Ce Zhang James Y. Zou ALM 24 0 0 05 May 2025
R-Bench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation Meng-Hao Guo Jiajun Xu Yi Zhang Jiaxi Song Haoyang Peng ... Yongming Rao Houwen Peng Han Hu Gordon Wetzstein Shi-Min Hu ELM LRM 49 0 0 04 May 2025
Cer-Eval: Certifiable and Cost-Efficient Evaluation Framework for LLMs G. Wang Z. Chen Bo Li Haifeng Xu 20 0 0 02 May 2025
Ask, Fail, Repeat: Meeseeks, an Iterative Feedback Benchmark for LLMs' Multi-turn Instruction-following Ability Jiaming Wang Yunke Zhao Peng Ding Jun Kuang Zongyu Wang Xuezhi Cao Xunliang Cai 50 0 0 30 Apr 2025
Automatic Legal Writing Evaluation of LLMs Ramon Pires Roseval Malaquias Junior Rodrigo Nogueira AILaw ELM 72 0 0 29 Apr 2025
Anyprefer: An Agentic Framework for Preference Data Synthesis Yiyang Zhou Z. Wang Tianle Wang Shangyu Xing Peng Xia ... Chetan Bansal Weitong Zhang Ying Wei Mohit Bansal Huaxiu Yao 52 0 0 27 Apr 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong X. Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Yu Jiang ALM ELM 84 0 0 26 Apr 2025
Learning Explainable Dense Reward Shapes via Bayesian Optimization Ryan Koo Ian Yang Vipul Raheja Mingyi Hong Kwang-Sung Jun Dongyeop Kang 12 0 0 22 Apr 2025
Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model Junshu Pan Wei Shen Shulin Huang Qiji Zhou Yue Zhang 69 0 0 22 Apr 2025
The Bitter Lesson Learned from 2,000+ Multilingual Benchmarks Minghao Wu Weixuan Wang Sinuo Liu Huifeng Yin Xintong Wang Yu Zhao Chenyang Lyu Longyue Wang Weihua Luo Kaifu Zhang ELM 69 0 0 22 Apr 2025
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators Yilun Zhou Austin Xu Peifeng Wang Caiming Xiong Shafiq R. Joty ELM ALM LRM 41 1 0 21 Apr 2025
DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models Chengyu Wang Junbing Yan Yuanhao Yue Jun Huang 12 0 0 21 Apr 2025
Pairwise or Pointwise? Evaluating Feedback Protocols for Bias in LLM-Based Evaluation Tuhina Tripathi Manya Wadhwa Greg Durrett S. Niekum 22 0 0 20 Apr 2025
ParaPO: Aligning Language Models to Reduce Verbatim Reproduction of Pre-training Data Tong Chen Faeze Brahman Jiacheng Liu Niloofar Mireshghallah Weijia Shi Pang Wei Koh Luke Zettlemoyer Hannaneh Hajishirzi 28 0 0 20 Apr 2025
MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Space Yicheng Chen Yining Li Kai Hu Zerun Ma Haochen Ye Kai Chen 22 0 0 18 Apr 2025
Efficient MAP Estimation of LLM Judgment Performance with Prior Transfer Huaizhi Qu Inyoung Choi Zhen Tan Song Wang Sukwon Yun Qi Long Faizan Siddiqui Kwonjoon Lee Tianlong Chen 39 0 0 17 Apr 2025
ImPart: Importance-Aware Delta-Sparsification for Improved Model Compression and Merging in LLMs Yan Yang Yixia Li Hongru Wang Xuetao Wei Jianqiao Yu Yun-Nung Chen Guanhua Chen MoMe 24 0 0 17 Apr 2025
Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data Shuai Zhao Linchao Zhu Yi Yang 24 1 0 14 Apr 2025
AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender Weixiang Zhao Jiahe Guo Yulin Hu Yang Deng An Zhang ... Xinyang Han Yanyan Zhao Bing Qin Tat-Seng Chua Ting Liu AAML LLMSV 37 0 0 13 Apr 2025
AIR: A Systematic Analysis of Annotations, Instructions, and Response Pairs in Preference Dataset Bingxiang He Wenbin Zhang Jiaxi Song Cheng Qian Z. Fu ... Hui Xue Ganqu Cui Wanxiang Che Zhiyuan Liu Maosong Sun 26 0 0 04 Apr 2025
Brains vs. Bytes: Evaluating LLM Proficiency in Olympiad Mathematics Hamed Mahdavi Alireza Hashemi Majid Daliri Pegah Mohammadipour Alireza Farhadi Samira Malek Yekta Yazdanifard Amir Khasahmadi V. Honavar ELM LRM 40 1 0 01 Apr 2025
JudgeLRM: Large Reasoning Models as a Judge Nuo Chen Zhiyuan Hu Qingyun Zou Jiaying Wu Qian Wang Bryan Hooi Bingsheng He ReLM ELM LRM 38 4 0 31 Mar 2025
Controlling Large Language Model with Latent Actions Chengxing Jia Ziniu Li Pengyuan Wang Yi-Chen Li Zhenyu Hou Yuxiao Dong Y. Yu 46 0 0 27 Mar 2025
Boosting Large Language Models with Mask Fine-Tuning M. Zhang Yue Bai Huan Wang Yizhou Wang Qihua Dong Y. Fu CLL 43 0 0 27 Mar 2025
Scaling Evaluation-time Compute with Reasoning Models as Process Evaluators Seungone Kim Ian Wu Jinu Lee Xiang Yue Seongyun Lee ... Kiril Gashteovski Carolin (Haas) Lawrence J. Hockenmaier Graham Neubig Sean Welleck LRM 42 2 0 25 Mar 2025
LLM-Based Insight Extraction for Contact Center Analytics and Cost-Efficient Deployment Varsha Embar Ritvik Shrivastava Vinay Damodaran Travis Mehlinger Yu-Chung Hsiao Karthik Raghunathan 24 0 0 24 Mar 2025
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications Jian-Yu Guan J. Wu J. Li Chuanqi Cheng Wei Yu Wu LM&MA 69 0 0 21 Mar 2025
Synthetic Clarification and Correction Dialogues about Data-Centric Tasks -- A Teacher-Student Approach Christian Poelitz Nick McKenna 35 1 0 18 Mar 2025
Aligning Multimodal LLM with Human Preference: A Survey Tao Yu Y. Zhang Chaoyou Fu Junkang Wu Jinda Lu ... Qingsong Wen Z. Zhang Yan Huang Liang Wang T. Tan 64 2 0 18 Mar 2025
Take Off the Training Wheels Progressive In-Context Learning for Effective Alignment Zhenyu Liu Dongfang Li Xinshuo Hu X. Zhao Yibin Chen Baotian Hu Min-Ling Zhang 38 1 0 13 Mar 2025
Representation-based Reward Modeling for Efficient Safety Alignment of Large Language Model Qiyuan Deng X. Bai Kehai Chen Yaowei Wang Liqiang Nie Min Zhang OffRL 55 0 0 13 Mar 2025
ASIDE: Architectural Separation of Instructions and Data in Language Models Egor Zverev Evgenii Kortukov Alexander Panfilov Soroush Tabesh Alexandra Volkova Sebastian Lapuschkin Wojciech Samek Christoph H. Lampert AAML 52 1 0 13 Mar 2025
Compositional Subspace Representation Fine-tuning for Adaptive Large Language Models Andy Zhou MoMe 87 0 0 13 Mar 2025
Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation Wenlong Meng Fan Zhang Wendao Yao Zhenyuan Guo Y. Li Chengkun Wei Wenzhi Chen AAML 36 1 0 11 Mar 2025
Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectors Runxi Cheng Feng Xiong Yongxian Wei Wanyun Zhu Chun Yuan MoMe 52 0 0 11 Mar 2025
VLRMBench: A Comprehensive and Challenging Benchmark for Vision-Language Reward Models Jiacheng Ruan Wenzhen Yuan Xian Gao Ye Guo Daoxin Zhang Zhe Xu Yao Hu Ting Liu Yuzhuo Fu LRM VLM 51 4 0 10 Mar 2025
WildIFEval: Instruction Following in the Wild Gili Lior Asaf Yehudai Ariel Gera L. Ein-Dor 62 0 0 09 Mar 2025
No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding Michael Krumdick Charles Lovering Varshini Reddy Seth Ebner Chris Tanner ALM ELM 49 2 0 07 Mar 2025
This Is Your Doge, If It Please You: Exploring Deception and Robustness in Mixture of LLMs Lorenz Wolf Sangwoong Yoon Ilija Bogunovic 39 0 0 07 Mar 2025
DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models Ruizhe Chen Wenhao Chai Zhifei Yang Xiaotian Zhang Joey Tianyi Zhou Tony Q. S. Quek Soujanya Poria Zuozhu Liu 48 0 0 06 Mar 2025
Adding Alignment Control to Language Models Wenhong Zhu Weinan Zhang Rui Wang 50 0 0 06 Mar 2025
Benchmarking Large Language Models on Multiple Tasks in Bioinformatics NLP with Prompting Jiyue Jiang Pengan Chen J. T. Wang Dongchen He Ziqin Wei ... Yimin Fan Xiangyu Shi J. Sun Chuan Wu Y. Li LM&MA 35 0 0 06 Mar 2025
AlignDistil: Token-Level Language Model Alignment as Adaptive Policy Distillation Songming Zhang Xue Zhang Tong Zhang Bojie Hu Yufeng Chen Jinan Xu 40 1 0 04 Mar 2025
Improving LLM-as-a-Judge Inference with the Judgment Distribution Victor Wang Michael J.Q. Zhang Eunsol Choi 49 0 0 04 Mar 2025
CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdom Yisen Li Lingfeng Yang Wenxuan Shen Pan Zhou Yao Wan Weiwei Lin D. Z. Chen 67 0 0 03 Mar 2025
Sentence-level Reward Model can Generalize Better for Aligning LLM from Human Preference Wenjie Qiu Yi-Chen Li Xuqin Zhang Tianyi Zhang Y. Zhang Zongzhang Zhang Yang Yu ALM 41 0 0 01 Mar 2025
BadJudge: Backdoor Vulnerabilities of LLM-as-a-Judge Terry Tong Fei-Yue Wang Zhe Zhao M. Chen AAML ELM 37 1 0 01 Mar 2025
Weaker LLMs' Opinions Also Matter: Mixture of Opinions Enhances LLM's Mathematical Reasoning Yanan Chen Ali Pesaranghader Tanmana Sadhu LRM 52 0 0 26 Feb 2025
FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users Anikait Singh Sheryl Hsu Kyle Hsu E. Mitchell Stefano Ermon Tatsunori Hashimoto Archit Sharma Chelsea Finn SyDa OffRL 43 1 0 26 Feb 2025