TinyGSM: achieving >80% on GSM8k with small language models

14 December 2023

Papers citing "TinyGSM: achieving >80% on GSM8k with small language models"

40 / 40 papers shown

Title
Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining Rosie Zhao Alexandru Meterez Sham Kakade C. Pehlevan Samy Jelassi Eran Malach ReLM LRM 33 2 0 10 Apr 2025
Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models Junyu Chen Han Cai Junsong Chen E. Xie Shang Yang Haotian Tang Muyang Li Y. Lu Song Han DiffM 54 35 0 20 Jan 2025
System-2 Mathematical Reasoning via Enriched Instruction Tuning Huanqia Cai Yijun Yang Zhifeng Li LRM 62 0 0 22 Dec 2024
Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generation Sukai Huang Trevor Cohn N. Lipovetzky LRM 68 0 0 14 Dec 2024
Proposing and solving olympiad geometry with guided tree search Chi Zhang Jiajun Song Siyu Li Yitao Liang Yuxi Ma Wei Wang Yixin Zhu Song-Chun Zhu AIMat LRM 69 3 0 14 Dec 2024
Evolutionary Pre-Prompt Optimization for Mathematical Reasoning Mathurin Videau Alessandro Leite Marc Schoenauer O. Teytaud ReLM LRM 66 0 0 05 Dec 2024
Inference Scaling fLaws: The Limits of LLM Resampling with Imperfect Verifiers Benedikt Stroebl Sayash Kapoor Arvind Narayanan LRM 80 6 0 26 Nov 2024
Mixture of Parrots: Experts improve memorization more than reasoning Samy Jelassi Clara Mohri David Brandfonbrener Alex Gu Nikhil Vyas Nikhil Anand David Alvarez-Melis Yuanzhi Li Sham Kakade Eran Malach MoE 16 3 0 24 Oct 2024
A Survey on Data Synthesis and Augmentation for Large Language Models Ke Wang Jiahui Zhu Minjie Ren Z. Liu Shiwei Li ... Chenkai Zhang Xiaoyu Wu Qiqi Zhan Qingjie Liu Yunhong Wang SyDa 36 13 0 16 Oct 2024
TinyAgent: Function Calling at the Edge Lutfi Eren Erdogan Nicholas Lee Siddharth Jha Sehoon Kim Ryan Tabrizi Suhong Moon Coleman Hooper Gopala Anumanchipalli Kurt Keutzer Amir Gholami LLMAG 31 10 0 01 Sep 2024
SIaM: Self-Improving Code-Assisted Mathematical Reasoning of Large Language Models Dian Yu Baolin Peng Ye Tian Linfeng Song Haitao Mi Dong Yu ALM LRM 28 0 0 28 Aug 2024
Evaluating Language Model Math Reasoning via Grounding in Educational Curricula L. Lucy Tal August Rose E. Wang Luca Soldaini Courtney Allison Kyle Lo ReLM LRM 19 1 0 08 Aug 2024
Prover-Verifier Games improve legibility of LLM outputs Jan Hendrik Kirchner Yining Chen Harri Edwards Jan Leike Nat McAleese Yuri Burda LRM AAML 20 24 0 18 Jul 2024
LEMoN: Label Error Detection using Multimodal Neighbors Haoran Zhang Aparna Balagopalan Nassim Oufattole Hyewon Jeong Yan Wu Jiacheng Zhu Marzyeh Ghassemi 29 0 0 10 Jul 2024
Merging Improves Self-Critique Against Jailbreak Attacks Victor Gallego AAML MoMe 21 3 0 11 Jun 2024
MoPS: Modular Story Premise Synthesis for Open-Ended Automatic Story Generation Yan Ma Yu Qiao Pengfei Liu 19 5 0 09 Jun 2024
Automatic Instruction Evolving for Large Language Models Weihao Zeng Can Xu Yingxiu Zhao Jianguang Lou Weizhu Chen SyDa 35 9 0 02 Jun 2024
Small Language Models for Application Interactions: A Case Study Beibin Li Yi Zhang Sébastien Bubeck Jeevan Pathuri Ishai Menache 21 3 0 23 May 2024
LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play Li-Chun Lu Shou-Jen Chen Tsung-Min Pai Chan-Hung Yu Hung-yi Lee Shao-Hua Sun LLMAG 35 10 0 10 May 2024
Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems Qihuang Zhong Kang Wang Ziyang Xu Juhua Liu Liang Ding Bo Du LRM AIMat 44 3 0 23 Apr 2024
Describe-then-Reason: Improving Multimodal Mathematical Reasoning through Visual Comprehension Training Mengzhao Jia Zhihan Zhang W. Yu Fangkai Jiao Meng-Long Jiang VLM ReLM LRM 35 7 0 22 Apr 2024
Self-Explore to Avoid the Pit: Improving the Reasoning Capabilities of Language Models with Fine-grained Rewards Hyeonbin Hwang Doyoung Kim Seungone Kim Seonghyeon Ye Minjoon Seo LRM ReLM 23 7 0 16 Apr 2024
LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement Nicholas Lee Thanakul Wattanawong Sehoon Kim K. Mangalam Sheng Shen Gopala Anumanchipalli Michael W. Mahoney Kurt Keutzer A. Gholami 56 12 0 22 Mar 2024
Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision Zhiqing Sun Longhui Yu Yikang Shen Weiyang Liu Yiming Yang Sean Welleck Chuang Gan 23 30 0 14 Mar 2024
SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models Yu Yang Siddhartha Mishra Jeffrey N Chiang Baharan Mirzasoleiman 26 17 0 12 Mar 2024
Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language Models Changyu Chen Xiting Wang Ting-En Lin Ang Lv Yuchuan Wu Xin Gao Ji-Rong Wen Rui Yan Yongbin Li ReLM LRM 21 8 0 04 Mar 2024
Orca-Math: Unlocking the potential of SLMs in Grade School Math Arindam Mitra Hamed Khanpour Corby Rosset Ahmed Hassan Awadallah ALM MoE LRM 22 62 0 16 Feb 2024
OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset Shubham Toshniwal Ivan Moshkov Sean Narenthiran Daria Gitman Fei Jia Igor Gitman 17 75 0 15 Feb 2024
Refined Direct Preference Optimization with Synthetic Data for Behavioral Alignment of LLMs Víctor Gallego SyDa 19 6 0 12 Feb 2024
CultureLLM: Incorporating Cultural Differences into Large Language Models Cheng-rong Li Mengzhou Chen Jindong Wang Sunayana Sitaram Xing Xie VLM 36 17 0 09 Feb 2024
Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervision Zihan Wang Yunxuan Li Yuexin Wu Liangchen Luo Le Hou Hongkun Yu Jingbo Shang LRM 23 18 0 05 Feb 2024
MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models Justin Chih-Yao Chen Swarnadeep Saha Elias Stengel-Eskin Mohit Bansal LRM LLMAG 22 1 0 02 Feb 2024
Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling Pratyush Maini Skyler Seto Richard He Bai David Grangier Yizhe Zhang Navdeep Jaitly SyDa 15 54 0 29 Jan 2024
Augmenting Math Word Problems via Iterative Question Composing Haoxiong Liu Yifan Zhang Yifan Luo Andrew Chi-Chih Yao SyDa LRM 14 34 0 17 Jan 2024
ReFT: Reasoning with Reinforced Fine-Tuning Trung Quoc Luong Xinbo Zhang Zhanming Jie Peng Sun Xiaoran Jin Hang Li OffRL LRM ReLM 29 79 0 17 Jan 2024
LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model Yichen Zhu Minjie Zhu Ning Liu Zhicai Ou Xiaofeng Mou Jian Tang 63 89 0 04 Jan 2024
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models Zixiang Chen Yihe Deng Huizhuo Yuan Kaixuan Ji Quanquan Gu SyDa 19 269 0 02 Jan 2024
Ziya2: Data-centric Learning is All LLMs Need Ruyi Gan Ziwei Wu Renliang Sun Junyu Lu Xiaojun Wu ... Ping Yang Qi Yang Hao Wang Jiaxing Zhang Yan Song VLM ALM 11 16 0 06 Nov 2023
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022