Sci-CoT: Leveraging Large Language Models for Enhanced Knowledge
Distillation in Small Models for Scientific QA

Sci-CoT: Leveraging Large Language Models for Enhanced Knowledge Distillation in Small Models for Scientific QA

9 August 2023

Papers citing "Sci-CoT: Leveraging Large Language Models for Enhanced Knowledge Distillation in Small Models for Scientific QA"

17 / 17 papers shown

Title
Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability Daniel Hendriks Philipp Spitzer Niklas Kühl G. Satzger 27 0 0 22 Apr 2025
VideoCoT: A Video Chain-of-Thought Dataset with Active Annotation Tool Yan Wang Yawen Zeng Jingsheng Zheng Xiaofen Xing Jin Xu Xiangmin Xu MLLM LRM VGen 35 11 0 07 Jul 2024
Can We Trust LLMs? Mitigate Overconfidence Bias in LLMs through Knowledge Transfer Haoyan Yang Yixuan Wang Xingyin Xu Hanyuan Zhang Yirong Bian 30 6 0 27 May 2024
Can only LLMs do Reasoning?: Potential of Small Language Models in Task Planning Gawon Choi Hyemin Ahn LM&Ro LRM 24 1 0 05 Apr 2024
Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought Jooyoung Lee Fan Yang Thanh Tran Qian Hu Emre Barut Kai-Wei Chang Chengwei Su ReLM LLMAG LRM 14 10 0 04 Apr 2024
From Large to Tiny: Distilling and Refining Mathematical Expertise for Math Word Problems with Weakly Supervision Qingwen Lin Boyan Xu Zhengting Huang Ruichu Cai 21 2 0 21 Mar 2024
Learning to Maximize Mutual Information for Chain-of-Thought Distillation Xin Chen Hanxian Huang Yanjun Gao Yi Wang Jishen Zhao Ke Ding 35 11 0 05 Mar 2024
AS-ES Learning: Towards Efficient CoT Learning in Small Models Nuwa Xi Yuhan Chen Sendong Zhao Hao Wang Bing Qin Ting Liu LRM 36 1 0 04 Mar 2024
Model Compression and Efficient Inference for Large Language Models: A Survey Wenxiao Wang Wei Chen Yicong Luo Yongliu Long Zhengkai Lin Liye Zhang Binbin Lin Deng Cai Xiaofei He MQ 36 46 0 15 Feb 2024
Grounding Foundation Models through Federated Transfer Learning: A General Framework Yan Kang Tao Fan Hanlin Gu Xiaojin Zhang Lixin Fan Qiang Yang AI4CE 61 19 0 29 Nov 2023
Two Timin': Repairing Smart Contracts With A Two-Layered Approach Abhinav Jain Ehan Masud Michelle Han Rohan Dhillon Sumukh Rao Arya Joshi Salar Cheema Saurav Kumar 17 0 0 14 Sep 2023
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 204 498 0 03 May 2023
Federated Prompting and Chain-of-Thought Reasoning for Improving LLMs Answering Xiangyang Liu Tianqi Pang Chenyou Fan FedML LRM 40 23 0 27 Apr 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 207 1,089 0 20 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 4,048 0 24 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022