Multi-Granularity Semantic Revision for Large Language Model Distillation

14 July 2024

Papers citing "Multi-Granularity Semantic Revision for Large Language Model Distillation"

7 / 7 papers shown

Title
DistiLLM: Towards Streamlined Distillation for Large Language Models Jongwoo Ko Sungnyun Kim Tianyi Chen SeYoung Yun 61 25 0 06 Feb 2024
Knowledge Distillation for Closed-Source Language Models Hongzhan Chen Xiaojun Quan Hehong Chen Ming Yan Ji Zhang BDL 33 2 0 13 Jan 2024
CBQ: Cross-Block Quantization for Large Language Models Xin Ding Xiaoyu Liu Zhijun Tu Yun-feng Zhang Wei Li ... Hanting Chen Yehui Tang Zhiwei Xiong Baoqun Yin Yunhe Wang MQ 22 11 0 13 Dec 2023
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 204 498 0 03 May 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 157 576 0 06 Apr 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,730 0 04 Mar 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020