Changing Answer Order Can Decrease MMLU Accuracy

Changing Answer Order Can Decrease MMLU Accuracy

27 June 2024

Candace Ross

Papers citing "Changing Answer Order Can Decrease MMLU Accuracy"

8 / 8 papers shown

Title
When2Call: When (not) to Call Tools Hayley Ross Ameya Sunil Mahabaleshwarkar Yoshi Suhara 92 0 0 26 Apr 2025
Improving Model Evaluation using SMART Filtering of Benchmark Datasets Vipul Gupta Candace Ross David Pantoja R. Passonneau Megan Ung Adina Williams 34 1 0 26 Oct 2024
Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machina Yuan Gao Dokyun Lee Gordon Burtch Sina Fazelpour LRM 38 7 0 25 Oct 2024
CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning Joshua Ong Jun Leang Aryo Pradipta Gema Shay B. Cohen ReLM LRM ReCod 31 2 0 14 Oct 2024
Mitigating Selection Bias with Node Pruning and Auxiliary Options Hyeong Kyu Choi Weijie Xu Chi Xue Stephanie Eckman Chandan K. Reddy 21 1 0 27 Sep 2024
Composable Interventions for Language Models Arinbjorn Kolbeinsson Kyle O'Brien Tianjin Huang Shanghua Gao Shiwei Liu ... Anurag J. Vaidya Faisal Mahmood Marinka Zitnik Tianlong Chen Thomas Hartvigsen KELM MU 68 5 0 09 Jul 2024
The Curious Case of Absolute Position Embeddings Koustuv Sinha Amirhossein Kazemnejad Siva Reddy J. Pineau Dieuwke Hupkes Adina Williams 66 15 0 23 Oct 2022
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 238 1,898 0 31 Dec 2020