Large Language Models Are Not Robust Multiple Choice Selectors

7 September 2023

Jie Zhou

Papers citing "Large Language Models Are Not Robust Multiple Choice Selectors"

38 / 38 papers shown

Title
What do Language Model Probabilities Represent? From Distribution Estimation to Response Prediction Eitan Wagner Omri Abend 27 0 0 04 May 2025
Systematic Bias in Large Language Models: Discrepant Response Patterns in Binary vs. Continuous Judgment Tasks Yi-Long Lu C. Zhang Wei Wang 29 0 0 28 Apr 2025
When2Call: When (not) to Call Tools Hayley Ross Ameya Sunil Mahabaleshwarkar Yoshi Suhara 92 0 0 26 Apr 2025
KETCHUP: K-Step Return Estimation for Sequential Knowledge Distillation Jiabin Fan Guoqing Luo Michael Bowling Lili Mou OffRL 63 0 0 26 Apr 2025
Large language models could be rote learners Yuyang Xu Renjun Hu Haochao Ying J. Wu Xing Shi Wei Lin ELM 75 0 0 11 Apr 2025
Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems? Kai Yan Yufei Xu Zhengyin Du Xuesong Yao Z. Wang Xiaowen Guo Jiecao Chen ReLM ELM LRM 90 3 0 01 Apr 2025
A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrieval Yu Zhang Shutong Qiao Jiaqi Zhang Tzu-Heng Lin Chen Gao Y. Li LM&Ro LM&MA 87 1 0 07 Mar 2025
Reversal Blessing: Thinking Backward May Outpace Thinking Forward in Multi-choice Questions Yizhe Zhang Richard He Bai Zijin Gu Ruixiang Zhang Jiatao Gu Emmanuel Abbe Samy Bengio Navdeep Jaitly LRM BDL 53 1 0 25 Feb 2025
IPO: Your Language Model is Secretly a Preference Classifier Shivank Garg Ayush Singh Shweta Singh Paras Chopra 66 1 0 22 Feb 2025
Savaal: Scalable Concept-Driven Question Generation to Enhance Human Learning Kimia Noorbakhsh Joseph Chandler Pantea Karimi M. Alizadeh H. Balakrishnan LRM 44 1 0 18 Feb 2025
Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models Prateek Chhikara 34 1 0 16 Feb 2025
Aligning Black-box Language Models with Human Judgments Gerrit J. J. van den Burg Gen Suzuki Wei Liu Murat Sensoy ALM 71 0 0 07 Feb 2025
The Order Effect: Investigating Prompt Sensitivity to Input Order in LLMs Bryan Guan Tanya Roosta Peyman Passban Mehdi Rezagholizadeh 97 0 0 06 Feb 2025
Option-ID Based Elimination For Multiple Choice Questions Zhenhao Zhu Bulou Liu Qingyao Ai Y. Liu 54 0 0 25 Jan 2025
Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators Yinhong Liu Han Zhou Zhijiang Guo Ehsan Shareghi Ivan Vulić Anna Korhonen Nigel Collier ALM 128 67 0 20 Jan 2025
Are You Doubtful? Oh, It Might Be Difficult Then! Exploring the Use of Model Uncertainty for Question Difficulty Estimation Leonidas Zotos H. Rijn Malvina Nissim 65 0 0 16 Dec 2024
Improving Model Evaluation using SMART Filtering of Benchmark Datasets Vipul Gupta Candace Ross David Pantoja R. Passonneau Megan Ung Adina Williams 49 1 0 26 Oct 2024
Unleashing Multi-Hop Reasoning Potential in Large Language Models through Repetition of Misordered Context Sangwon Yu Ik-hwan Kim Jongyoon Song Saehyung Lee Junsung Park Sungroh Yoon LRM 65 0 0 09 Oct 2024
Position: LLM Unlearning Benchmarks are Weak Measures of Progress Pratiksha Thaker Shengyuan Hu Neil Kale Yash Maurya Zhiwei Steven Wu Virginia Smith MU 45 10 0 03 Oct 2024
Mitigating Selection Bias with Node Pruning and Auxiliary Options Hyeong Kyu Choi Weijie Xu Chi Xue Stephanie Eckman Chandan K. Reddy 29 1 0 27 Sep 2024
How to Make the Most of LLMs' Grammatical Knowledge for Acceptability Judgments Yusuke Ide Yuto Nishida Miyu Oba Miyu Oba Justin Vasselli Hidetaka Kamigaito Taro Watanabe 31 2 0 19 Aug 2024
Forecasting Live Chat Intent from Browsing History Se-eun Yoon A. Rabiah Zaid Alibadi Surya Kallumadi Julian McAuley AI4TS 21 0 0 07 Aug 2024
Eliminating Position Bias of Language Models: A Mechanistic Approach Ziqi Wang Hanlin Zhang Xiner Li Kuan-Hao Huang Chi Han Shuiwang Ji Sham Kakade Hao Peng Heng Ji 49 12 0 01 Jul 2024
Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models Lynn Chua Badih Ghazi Yangsibo Huang Pritish Kamath Ravi Kumar Pasin Manurangsi Amer Sinha Chulin Xie Chiyuan Zhang 51 1 0 23 Jun 2024
Talking Heads: Understanding Inter-layer Communication in Transformer Language Models Jack Merullo Carsten Eickhoff Ellie Pavlick 56 12 0 13 Jun 2024
OLMES: A Standard for Language Model Evaluations Yuling Gu Oyvind Tafjord Bailey Kuehl Dany Haddad Jesse Dodge Hannaneh Hajishirzi ELM 32 13 0 12 Jun 2024
Discovering Bias in Latent Space: An Unsupervised Debiasing Approach Dyah Adila Shuai Zhang Boran Han Yuyang Wang AAML LLMSV 27 6 0 05 Jun 2024
Unveiling Selection Biases: Exploring Order and Token Sensitivity in Large Language Models Sheng-Lun Wei Cheng-Kuang Wu Hen-Hsen Huang Hsin-Hsi Chen 26 10 0 05 Jun 2024
Towards Hierarchical Multi-Agent Workflows for Zero-Shot Prompt Optimization Yuchi Liu Jaskirat Singh Gaowen Liu Ali Payani Liang Zheng LLMAG 74 4 0 30 May 2024
On Fairness of Low-Rank Adaptation of Large Models Zhoujie Ding Ken Ziyu Liu Pura Peetathawatchai Berivan Isik Sanmi Koyejo 38 4 0 27 May 2024
Intuitive Fine-Tuning: Towards Simplifying Alignment into a Single Process Ermo Hua Biqing Qi Kaiyan Zhang Yue Yu Ning Ding Xingtai Lv Kai Tian Bowen Zhou 32 3 0 20 May 2024
Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering Jie Ma Min Hu Pinghui Wang Wangchun Sun Lingyun Song Hongbin Pei Jun Liu Youtian Du 32 4 0 18 Apr 2024
JsonTuning: Towards Generalizable, Robust, and Controllable Instruction Tuning Chang Gao Wenxuan Zhang Guizhen Chen Wai Lam 47 5 0 04 Oct 2023
Leveraging Large Language Models for Multiple Choice Question Answering Joshua Robinson Christopher Rytting David Wingate ELM 138 181 0 22 Oct 2022
On the Relation between Sensitivity and Accuracy in In-context Learning Yanda Chen Chen Zhao Zhou Yu Kathleen McKeown He He 180 77 0 16 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 4,048 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,730 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022