Distilling Reasoning Capabilities into Smaller Language Models

1 December 2022

Papers citing "Distilling Reasoning Capabilities into Smaller Language Models"

50 / 113 papers shown

Title
Distilling Reasoning Ability from Large Language Models with Adaptive Thinking Xiao Chen Sihang Zhou K. Liang Xinwang Liu ReLM LRM 27 2 0 14 Apr 2024
SAAS: Solving Ability Amplification Strategy for Enhanced Mathematical Reasoning in Large Language Models Hyeonwoo Kim Gyoungjin Gim Yungi Kim Jihoo Kim Byungju Kim Wonseok Lee Chanjun Park ReLM LRM 34 1 0 05 Apr 2024
Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought Jooyoung Lee Fan Yang Thanh Tran Qian Hu Emre Barut Kai-Wei Chang Chengwei Su ReLM LLMAG LRM 14 10 0 04 Apr 2024
Okay, Let's Do This! Modeling Event Coreference with Generated Rationales and Knowledge Distillation Abhijnan Nath Shadi Manafi Avyakta Chelle Nikhil Krishnaswamy 38 1 0 04 Apr 2024
TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale Pengcheng Jiang Cao Xiao Zifeng Wang Parminder Bhatia Jimeng Sun Jiawei Han LRM 21 10 0 15 Mar 2024
Self-Consistency Boosts Calibration for Math Reasoning Ante Wang Linfeng Song Ye Tian Baolin Peng Lifeng Jin Haitao Mi Jinsong Su Dong Yu LRM 16 5 0 14 Mar 2024
The pitfalls of next-token prediction Gregor Bachmann Vaishnavh Nagarajan 35 58 0 11 Mar 2024
Socratic Reasoning Improves Positive Text Rewriting Anmol Goel Nico Daheim Iryna Gurevych Iryna Gurevych LRM 36 4 0 05 Mar 2024
AS-ES Learning: Towards Efficient CoT Learning in Small Models Nuwa Xi Yuhan Chen Sendong Zhao Hao Wang Bing Qin Ting Liu LRM 41 1 0 04 Mar 2024
Distilling Text Style Transfer With Self-Explanation From LLMs Chiyu Zhang Honglong Cai Yuezhang Li Li Yuexin Wu Le Hou Muhammad Abdul-Mageed 33 10 0 02 Mar 2024
Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding Benjamin Bergner Andrii Skliar Amelie Royer Tijmen Blankevoort Yuki Markus Asano B. Bejnordi 58 5 0 26 Feb 2024
DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers Xirui Li Ruochen Wang Minhao Cheng Tianyi Zhou Cho-Jui Hsieh AAML 39 37 0 25 Feb 2024
HiGPT: Heterogeneous Graph Language Model Jiabin Tang Yuhao Yang Wei Wei Lei Shi Long Xia Dawei Yin Chao Huang 39 20 0 25 Feb 2024
$C^3$ : Confidence Calibration Model Cascade for Inference-Efficient Cross-Lingual Natural Language Understanding Taixi Lu Haoyu Wang Huajie Shao Jing Gao Huaxiu Yao 33 0 0 25 Feb 2024
Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning Debjit Paul Robert West Antoine Bosselut Boi Faltings ReLM LRM 33 20 0 21 Feb 2024
Large Language Models for Data Annotation: A Survey Zhen Tan Dawei Li Song Wang Alimohammad Beigi Bohan Jiang Amrita Bhattacharjee Mansooreh Karami Jundong Li Lu Cheng Huan Liu SyDa 42 49 0 21 Feb 2024
ELAD: Explanation-Guided Large Language Models Active Distillation Yifei Zhang Bo Pan Chen Ling Yuntong Hu Liang Zhao 41 5 0 20 Feb 2024
Can LLMs Compute with Reasons? Harshit Sandilya Peehu Raj J. Bafna Srija Mukhopadhyay Shivansh Sharma Ellwil Sharma Arastu Sharma Neeta Trivedi Manish Shrivastava Rajesh Kumar LRM 22 0 0 19 Feb 2024
Distilling Large Language Models for Text-Attributed Graph Learning Bo Pan Zhengwu Zhang Yifei Zhang Yuntong Hu Liang Zhao 30 13 0 19 Feb 2024
On the Roles of LLMs in Planning: Embedding LLMs into Planning Graphs H. Zhuo Xin Chen Rong Pan 26 2 0 18 Feb 2024
AutoPRM: Automating Procedural Supervision for Multi-Step Reasoning via Controllable Question Decomposition Zhaorun Chen Zhuokai Zhao Zhihong Zhu Ruiqi Zhang Xiang Li Bhiksha Raj Huaxiu Yao LRM 25 25 0 18 Feb 2024
LaCo: Large Language Model Pruning via Layer Collapse Yifei Yang Zouying Cao Hai Zhao 19 52 0 17 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 36 46 0 15 Feb 2024
A Survey on Transformer Compression Yehui Tang Yunhe Wang Jianyuan Guo Zhijun Tu Kai Han Hailin Hu Dacheng Tao 29 27 0 05 Feb 2024
MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models Justin Chih-Yao Chen Swarnadeep Saha Elias Stengel-Eskin Mohit Bansal LRM LLMAG 30 13 0 02 Feb 2024
Distilling LLMs' Decomposition Abilities into Compact Language Models Denis Tarasov Kumar Shridhar SyDa OffRL LRM 40 2 0 02 Feb 2024
Contextualization Distillation from Large Language Model for Knowledge Graph Completion Dawei Li Zhen Tan Tianlong Chen Huan Liu KELM 25 12 0 28 Jan 2024
Distilling Mathematical Reasoning Capabilities into Small Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang LRM 32 9 0 22 Jan 2024
Deciphering Textual Authenticity: A Generalized Strategy through the Lens of Large Language Semantics for Detecting Human vs. Machine-Generated Text Mazal Bethany Brandon Wherry Emet Bethany Nishant Vishwamitra Anthony Rios Peyman Najafirad DeLMO 28 3 0 17 Jan 2024
PizzaCommonSense: Learning to Model Commonsense Reasoning about Intermediate Steps in Cooking Recipes Aissatou Diallo Antonis Bikakis Luke Dickens Anthony Hunter Rob Miller LRM 24 2 0 12 Jan 2024
Mixed Distillation Helps Smaller Language Model Better Reasoning Chenglin Li Qianglong Chen Liangyue Li Wang Caiyu Yicheng Li Zhang Yin Yin Zhang LRM 26 11 0 17 Dec 2023
Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks Lingfeng Sun Devesh K. Jha Chiori Hori Siddarth Jain Radu Corcodel Xinghao Zhu Masayoshi Tomizuka Diego Romeres LM&Ro LLMAG 22 20 0 11 Dec 2023
Large Multimodal Model Compression via Efficient Pruning and Distillation at AntGroup Maolin Wang Yao-Min Zhao Jiajia Liu Jingdong Chen Chenyi Zhuang Jinjie Gu Ruocheng Guo Xiangyu Zhao 18 6 0 10 Dec 2023
Physical Reasoning and Object Planning for Household Embodied Agents Ayush Agrawal Raghav Prabhakar Anirudh Goyal Dianbo Liu LM&Ro LRM 11 0 0 22 Nov 2023
From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models Zachary Englhardt Chengqian Ma Margaret E. Morris X. Xu Chun-Cheng Chang Lianhui Qin Daniel J. McDuff Xin Liu Shwetak N. Patel Vikram Iyer AI4MH 37 11 0 21 Nov 2023
Efficient End-to-End Visual Document Understanding with Rationale Distillation Wang Zhu Alekh Agarwal Mandar Joshi Robin Jia Jesse Thomason Kristina Toutanova 18 2 0 16 Nov 2023
Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs Michael J.Q. Zhang Eunsol Choi 32 26 0 16 Nov 2023
Mind's Mirror: Distilling Self-Evaluation Capability and Comprehensive Thinking from Large Language Models Weize Liu Guocong Li Kai Zhang Bang Du Qiyuan Chen Xuming Hu Hongxia Xu Jintai Chen Jian Wu LRM 18 6 0 15 Nov 2023
The ART of LLM Refinement: Ask, Refine, and Trust Kumar Shridhar Koustuv Sinha Andrew Cohen Tianlu Wang Ping Yu Ramakanth Pasunuru Mrinmaya Sachan Jason Weston Asli Celikyilmaz LLMAG ReLM LRM 22 24 0 14 Nov 2023
First-Step Advantage: Importance of Starting Right in Multi-Step Math Reasoning Kushal Kumar Jain Moritz Miller Niket Tandon Kumar Shridhar ReLM LRM 35 2 0 14 Nov 2023
VerityMath: Advancing Mathematical Reasoning by Self-Verification Through Unit Consistency Vernon Toh Ratish Puduppully Nancy F. Chen LRM 25 5 0 13 Nov 2023
Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models Ruida Wang Wangchunshu Zhou Mrinmaya Sachan 19 32 0 20 Oct 2023
Democratizing Reasoning Ability: Tailored Learning from Large Language Model Zhaoyang Wang Shaohan Huang Yuxuan Liu Jiahai Wang Minghui Song ... Haizhen Huang Furu Wei Weiwei Deng Feng Sun Qi Zhang LRM 27 11 0 20 Oct 2023
GraphGPT: Graph Instruction Tuning for Large Language Models Jiabin Tang Yuhao Yang Wei Wei Lei Shi Lixin Su Suqi Cheng Dawei Yin Chao Huang 29 121 0 19 Oct 2023
Recurrent Neural Language Models as Probabilistic Finite-state Automata Anej Svete Ryan Cotterell 30 2 0 08 Oct 2023
The Role of Federated Learning in a Wireless World with Foundation Models Zihan Chen Howard H. Yang Y. C. Tay Kai Fong Ernest Chong Tony Q. S. Quek AI4CE 27 6 0 06 Oct 2023
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models L. Yu Weisen Jiang Han Shi Jincheng Yu Zhengying Liu Yu Zhang James T. Kwok Zheng Li Adrian Weller Weiyang Liu OSLM LRM 39 325 0 21 Sep 2023
SCREWS: A Modular Framework for Reasoning with Revisions K. Shridhar Harsh Jhamtani Hao Fang Benjamin Van Durme Jason Eisner Patrick Xia KELM LRM 25 14 0 20 Sep 2023
Boosting Logical Reasoning in Large Language Models through a New Framework: The Graph of Thought Bin Lei Pei-Hung Lin C. Liao Caiwen Ding ReLM ELM LRM AI4CE 19 38 0 16 Aug 2023
A Survey on Model Compression for Large Language Models Xunyu Zhu Jian Li Yong Liu Can Ma Weiping Wang 24 190 0 15 Aug 2023