GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models

7 October 2024

Iman Mirzadeh

Mehrdad Farajtabar

Papers citing "GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models"

23 / 23 papers shown

Title
Optimization Problem Solving Can Transition to Evolutionary Agentic Workflows Wenhao Li Bo Jin Mingyi Hong Changhong Lu Xiangfeng Wang 36 0 0 07 May 2025
TutorGym: A Testbed for Evaluating AI Agents as Tutors and Students Daniel Weitekamp M. N. Siddiqui Christopher James Maclellan LLMAG ELM 18 0 0 02 May 2025
FLIP Reasoning Challenge Andreas Plesner Turlan Kuzhagaliyev Roger Wattenhofer AAML VLM LRM 67 0 0 16 Apr 2025
Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems? Kai Yan Yufei Xu Zhengyin Du Xuesong Yao Z. Wang Xiaowen Guo Jiecao Chen ReLM ELM LRM 87 3 0 01 Apr 2025
Brains vs. Bytes: Evaluating LLM Proficiency in Olympiad Mathematics Hamed Mahdavi Alireza Hashemi Majid Daliri Pegah Mohammadipour Alireza Farhadi Samira Malek Yekta Yazdanifard Amir Khasahmadi V. Honavar ELM LRM 38 1 0 01 Apr 2025
TRA: Better Length Generalisation with Threshold Relative Attention Mattia Opper Roland Fernandez P. Smolensky Jianfeng Gao 37 0 0 29 Mar 2025
L0-Reasoning Bench: Evaluating Procedural Correctness in Language Models via Simple Program Execution Simeng Sun Cheng-Ping Hsieh Faisal Ladhak Erik Arakelyan Santiago Akle Serano Boris Ginsburg ReLM ELM LRM 39 0 0 28 Mar 2025
Toward an Evaluation Science for Generative AI Systems Laura Weidinger Deb Raji Hanna M. Wallach Margaret Mitchell Angelina Wang Olawale Salaudeen Rishi Bommasani Sayash Kapoor Deep Ganguli Sanmi Koyejo EGVM ELM 60 3 0 07 Mar 2025
BIG-Bench Extra Hard Mehran Kazemi Bahare Fatemi Hritik Bansal John Palowitch Chrysovalantis Anastasiou ... Kate Olszewska Yi Tay Vinh Q. Tran Quoc V. Le Orhan Firat ELM LRM 115 4 0 26 Feb 2025
The Relationship Between Reasoning and Performance in Large Language Models -- o3 (mini) Thinks Harder, Not Longer Marthe Ballon Andres Algaba Vincent Ginis LRM ReLM 36 4 0 24 Feb 2025
InductionBench: LLMs Fail in the Simplest Complexity Class Wenyue Hua Tyler Wong Sun Fei Liangming Pan Adam Jardine William Yang Wang LRM 48 2 0 20 Feb 2025
MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs Andreas Opedal Haruki Shirakami Bernhard Schölkopf Abulhair Saparov Mrinmaya Sachan LRM 50 1 0 17 Feb 2025
Do Large Language Models Reason Causally Like Us? Even Better? Hanna M. Dettki Brenden M. Lake Charley M. Wu Bob Rehder ReLM ELM LRM 90 0 0 14 Feb 2025
MATH-Perturb: Benchmarking LLMs' Math Reasoning Abilities against Hard Perturbations Kaixuan Huang Jiacheng Guo Zihao Li X. Ji Jiawei Ge ... Yangsibo Huang Chi Jin Xinyun Chen Chiyuan Zhang Mengdi Wang AAML LRM 66 7 0 10 Feb 2025
Large Language Models for Multi-Robot Systems: A Survey Peihan Li Zijian An Shams Abrar Lifeng Zhou LM&Ro LRM 36 4 0 06 Feb 2025
Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignment Cheryl Li Tianyuan Xu Yiwen Guo LRM 62 2 0 05 Feb 2025
PhD Knowledge Not Required: A Reasoning Challenge for Large Language Models C. Anderson Joydeep Biswas Aleksander Boruch-Gruszecki Federico Cassano Molly Q. Feldman Joydeep Biswas Francesca Lucchetti Zixuan Wu Arjun Guha ReLM ELM LRM 37 3 0 03 Feb 2025
Spend Wisely: Maximizing Post-Training Gains in Iterative Synthetic Data Boostrapping Pu Yang Yunzhen Feng Ziyuan Chen Yuhang Wu Zhuoyuan Li DiffM 89 0 0 31 Jan 2025
Out-of-distribution generalization via composition: a lens through induction heads in Transformers Jiajun Song Zhuoyan Xu Yiqiao Zhong 60 4 0 31 Dec 2024
MCP-Solver: Integrating Language Models with Constraint Programming Systems Stefan Szeider 25 0 0 31 Dec 2024
On Large Language Models in Mission-Critical IT Governance: Are We Ready Yet? Matteo Esposito Francesco Palagiano Valentina Lenarduzzi Davide Taibi ELM 62 2 0 16 Dec 2024
One-Layer Transformer Provably Learns One-Nearest Neighbor In Context Zihao Li Yuan Cao Cheng Gao Yihan He Han Liu Jason M. Klusowski Jianqing Fan Mengdi Wang MLT 39 1 0 16 Nov 2024
Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimization Zeyuan Ma Hongshu Guo Yue-jiao Gong Jun Zhang Kay Chen Tan 90 2 0 01 Nov 2024