Revisiting Knowledge Distillation for Autoregressive Language Models

Revisiting Knowledge Distillation for Autoregressive Language Models

19 February 2024

Liang Ding

Li Shen

Bo Du

Papers citing "Revisiting Knowledge Distillation for Autoregressive Language Models"

5 / 5 papers shown

Title
CORAL: Learning Consistent Representations across Multi-step Training with Lighter Speculative Drafter Yepeng Weng Dianwen Mei Huishi Qiu Xujie Chen Li Liu Jiang Tian Zhongchao Shi 42 0 0 24 Feb 2025
ARB-LLM: Alternating Refined Binarizations for Large Language Models Zhiteng Li X. Yan Tianao Zhang Haotong Qin Dong Xie Jiang Tian Zhongchao Shi Linghe Kong Yulun Zhang Xiaokang Yang MQ 26 2 0 04 Oct 2024
Language Model Cascades: Token-level uncertainty and beyond Neha Gupta Harikrishna Narasimhan Wittawat Jitkrittum A. S. Rawat A. Menon Sanjiv Kumar UQLM 33 41 0 15 Apr 2024
Towards Making the Most of ChatGPT for Machine Translation Keqin Peng Liang Ding Qihuang Zhong Li Shen Xuebo Liu Min Zhang Y. Ouyang Dacheng Tao LRM 83 203 0 24 Mar 2023
Understanding and Improving Lexical Choice in Non-Autoregressive Translation Liang Ding Longyue Wang Xuebo Liu Derek F. Wong Dacheng Tao Zhaopeng Tu 91 76 0 29 Dec 2020