Teaching Small Language Models to Reason

16 December 2022

Lucie Charlotte Magister

Papers citing "Teaching Small Language Models to Reason"

44 / 44 papers shown

Title
Recall with Reasoning: Chain-of-Thought Distillation for Mamba's Long-Context Memory and Extrapolation Junyu Ma Tianqing Fang Z. Zhang Hongming Zhang Haitao Mi Dong Yu ReLM RALM LRM 110 0 0 06 May 2025
KETCHUP: K-Step Return Estimation for Sequential Knowledge Distillation Jiabin Fan Guoqing Luo Michael Bowling Lili Mou OffRL 63 0 0 26 Apr 2025
ConTextual: Improving Clinical Text Summarization in LLMs with Context-preserving Token Filtering and Knowledge Graphs Fahmida Liza Piya Rahmatollah Beheshti 95 0 0 23 Apr 2025
Efficient Reasoning Models: A Survey Sicheng Feng Gongfan Fang Xinyin Ma Xinchao Wang ReLM LRM 116 0 0 15 Apr 2025
VITED: Video Temporal Evidence Distillation Yujie Lu Yale Song William Yang Wang Lorenzo Torresani Tushar Nagarajan 121 0 0 17 Mar 2025
Scientific Reasoning: Assessment of Multimodal Generative LLMs Florian Dreyer Ekaterina Kolos Daria Matiash ReLM LRM 62 0 0 03 Mar 2025
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning Xinghao Chen Zhijing Sun Wenjin Guo Miaoran Zhang Yanjun Chen ... Hui Su Yijie Pan Dietrich Klakow Wenjie Li Xiaoyu Shen LRM 51 4 0 25 Feb 2025
Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment Zhili Liu Yunhao Gou Kai Chen Lanqing Hong Jiahui Gao ... Yu Zhang Zhenguo Li Xin Jiang Q. Liu James T. Kwok MoE 96 9 0 20 Feb 2025
Towards Reasoning Ability of Small Language Models Gaurav Srivastava Shuxiang Cao Xuan Wang ReLM LRM 49 4 0 17 Feb 2025
Rationalization Models for Text-to-SQL Gaetano Rossiello Nhan Pham Michael R. Glass Junkyu Lee Shankar Subramanian ReLM LRM 50 0 0 10 Feb 2025
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents Weiwei Sun Lingyong Yan Xinyu Ma Shuaiqiang Wang Pengjie Ren Zhumin Chen Dawei Yin Z. Ren RALM ALM ELM LRM LM&MA 74 285 0 31 Dec 2024
Towards Efficient and Explainable Hate Speech Detection via Model Distillation Paloma Piot Javier Parapar 75 173 0 18 Dec 2024
Mentor-KD: Making Small Language Models Better Multi-step Reasoners Hojae Lee Junho Kim SangKeun Lee LRM 26 1 0 11 Oct 2024
DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback Zaid Khan Elias Stengel-Eskin Jaemin Cho Mohit Bansal VGen 38 1 0 08 Oct 2024
Small Language Models: Survey, Measurements, and Insights Zhenyan Lu Xiang Li Dongqi Cai Rongjie Yi Fangming Liu Xiwen Zhang Nicholas D. Lane Mengwei Xu ObjD LRM 51 36 0 24 Sep 2024
Neural-Symbolic Collaborative Distillation: Advancing Small Language Models for Complex Reasoning Tasks Huanxuan Liao Shizhu He Yao Xu Yuanzhe Zhang Kang Liu Jun Zhao LRM 53 3 0 20 Sep 2024
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images Zhecan Wang Junzhang Liu Chia-Wei Tang Hani Alomari Anushka Sivakumar ... Haoxuan You A. Ishmam Kai-Wei Chang Shih-Fu Chang Chris Thomas CoGe VLM 59 2 0 19 Sep 2024
Efficient Knowledge Distillation: Empowering Small Language Models with Teacher Model Insights Mohamad Ballout U. Krumnack Gunther Heidemann Kai-Uwe Kühnberger 31 2 0 19 Sep 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 58 23 0 10 Sep 2024
Path-Consistency: Prefix Enhancement for Efficient Inference in LLM Jiace Zhu Yingtao Shen Jie Zhao An Zou LLMAG LRM 27 4 0 25 Aug 2024
Reflective Instruction Tuning: Mitigating Hallucinations in Large Vision-Language Models Jinrui Zhang Teng Wang Haigang Zhang Ping Lu Feng Zheng MLLM LRM VLM 25 3 0 16 Jul 2024
Retrieved In-Context Principles from Previous Mistakes Hao-Lun Sun Yong-jia Jiang Bo Wang Yingyan Hou Yan Zhang Pengjun Xie Fei Huang 50 1 0 08 Jul 2024
Improve Student's Reasoning Generalizability through Cascading Decomposed CoTs Distillation Chengwei Dai Kun Li Wei Zhou Song Hu LRM 44 3 0 30 May 2024
Beyond Imitation: Learning Key Reasoning Steps from Dual Chain-of-Thoughts in Reasoning Distillation Chengwei Dai Kun Li Wei Zhou Song Hu LRM 36 5 0 30 May 2024
Distilling Reasoning Ability from Large Language Models with Adaptive Thinking Xiao Chen Sihang Zhou K. Liang Xinwang Liu ReLM LRM 29 2 0 14 Apr 2024
SAAS: Solving Ability Amplification Strategy for Enhanced Mathematical Reasoning in Large Language Models Hyeonwoo Kim Gyoungjin Gim Yungi Kim Jihoo Kim Byungju Kim Wonseok Lee Chanjun Park ReLM LRM 34 1 0 05 Apr 2024
Enhancing Reasoning Capacity of SLM using Cognitive Enhancement Jonathan Pan Swee Liang Wong Xin Wei Chia Yidi Yuan LRM 35 0 0 01 Apr 2024
Can Small Language Models be Good Reasoners for Sequential Recommendation? Yuling Wang Changxin Tian Binbin Hu Yanhua Yu Ziqi Liu Zhiqiang Zhang Jun Zhou Liang Pang Xiao Wang LRM 34 25 0 07 Mar 2024
Learning to Maximize Mutual Information for Chain-of-Thought Distillation Xin Chen Hanxian Huang Yanjun Gao Yi Wang Jishen Zhao Ke Ding 35 11 0 05 Mar 2024
Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Models Jiang Zhang Qiong Wu Yiming Xu Cheng Cao Zheng Du Konstantinos Psounis 28 14 0 13 Dec 2023
Merging Generated and Retrieved Knowledge for Open-Domain QA Yunxiang Zhang Muhammad Khalifa Lajanugen Logeswaran Moontae Lee Honglak Lee Lu Wang RALM 26 37 0 22 Oct 2023
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning Jiasheng Ye Zaixiang Zheng Yu Bao Lihua Qian Quanquan Gu DiffM 52 14 0 23 Aug 2023
Sci-CoT: Leveraging Large Language Models for Enhanced Knowledge Distillation in Small Models for Scientific QA Yuhan Ma Haiqi Jiang Chenyou Fan LRM 26 14 0 09 Aug 2023
Learning Evaluation Models from Large Language Models for Sequence Generation Chenglong Wang Hang Zhou Kai-Chun Chang Tongran Liu Chunliang Zhang Quan Du Tong Xiao Yue Zhang Jingbo Zhu ELM 34 3 0 08 Aug 2023
Let GPT be a Math Tutor: Teaching Math Word Problem Solvers with Customized Exercise Generation Zhenwen Liang W. Yu Tanmay Rajpurohit Peter Clark Xiangliang Zhang Ashwin Kaylan 25 37 0 22 May 2023
MoT: Memory-of-Thought Enables ChatGPT to Self-Improve Xiaonan Li Xipeng Qiu ReLM KELM LRM AI4MH 21 32 0 09 May 2023
Natural Language Reasoning, A Survey Fei Yu Hongbo Zhang Prayag Tiwari Benyou Wang ReLM LRM 28 49 0 26 Mar 2023
Towards Reasoning in Large Language Models: A Survey Jie Huang Kevin Chen-Chuan Chang LM&MA ELM LRM 22 579 0 20 Dec 2022
Reasoning with Language Model Prompting: A Survey Shuofei Qiao Yixin Ou Ningyu Zhang Xiang Chen Yunzhi Yao Shumin Deng Chuanqi Tan Fei Huang Huajun Chen ReLM ELM LRM 51 310 0 19 Dec 2022
Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model Jacob Eisenstein D. Andor Bernd Bohnet Michael Collins David M. Mimno LRM 187 24 0 05 Oct 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,236 0 21 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 317 8,448 0 28 Jan 2022
Understanding Factuality in Abstractive Summarization with FRANK: A Benchmark for Factuality Metrics Artidoro Pagnoni Vidhisha Balachandran Yulia Tsvetkov HILM 215 305 0 27 Apr 2021
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 245 671 0 06 Jan 2021