Mixed Distillation Helps Smaller Language Model Better Reasoning

Mixed Distillation Helps Smaller Language Model Better Reasoning

17 December 2023

Papers citing "Mixed Distillation Helps Smaller Language Model Better Reasoning"

16 / 16 papers shown

Title
Efficient Reasoning Models: A Survey Sicheng Feng Gongfan Fang Xinyin Ma Xinchao Wang ReLM LRM 50 0 0 15 Apr 2025
Training Small Reasoning LLMs with Cognitive Preference Alignment Wenrui Cai Chengyu Wang Junbing Yan Jun Huang Xiangzhong Fang LRM 26 0 0 14 Apr 2025
LexPam: Legal Procedure Awareness-Guided Mathematical Reasoning Kepu Zhang Guofu Xie Weijie Yu Mingyue Xu Xu Tang Yaxin Li Jun Xu AILaw ELM LRM 59 0 0 03 Apr 2025
SIKeD: Self-guided Iterative Knowledge Distillation for mathematical reasoning Shivam Adarsh Kumar Shridhar Caglar Gulcehre Nicholas Monath Mrinmaya Sachan LRM 19 2 0 24 Oct 2024
PatentGPT: A Large Language Model for Intellectual Property Zilong Bai Ruiji Zhang Linqing Chen Qijun Cai Yuan Zhong ... Fu Bian Xiaolong Gu Lisha Zhang Weilei Wang Changyang Tu 28 0 0 28 Apr 2024
A Survey on Knowledge Distillation of Large Language Models Xiaohan Xu Ming Li Chongyang Tao Tao Shen Reynold Cheng Jinyang Li Can Xu Dacheng Tao Tianyi Zhou KELM VLM 23 94 0 20 Feb 2024
MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models Justin Chih-Yao Chen Swarnadeep Saha Elias Stengel-Eskin Mohit Bansal LRM LLMAG 22 1 0 02 Feb 2024
Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future Zheng Chu Jingchang Chen Qianglong Chen Weijiang Yu Tao He Haotian Wang Weihua Peng Ming-Yu Liu Bing Qin Ting Liu LRM AI4CE 8 148 0 27 Sep 2023
Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning Tianhua Zhang Jiaxin Ge Hongyin Luo Yung-Sung Chuang Mingye Gao Yuan Gong Xixin Wu Yoon Kim Helen M. Meng James R. Glass LRM ReLM 68 10 0 19 Sep 2023
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 198 283 0 03 May 2023
Ask Me Anything: A simple strategy for prompting language models Simran Arora A. Narayan Mayee F. Chen Laurel J. Orr Neel Guha Kush S. Bhatia Ines Chami Frederic Sala Christopher Ré ReLM LRM 190 160 0 05 Oct 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 2,712 0 24 May 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 297 3,163 0 21 Mar 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 242 460 0 06 Jan 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020