Sentence-Level or Token-Level? A Comprehensive Study on Knowledge
Distillation

Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation

23 April 2024

Jingxuan Wei

Linzhuang Sun

Xu Tan

Ruifeng Guo

Papers citing "Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation"

6 / 6 papers shown

Title
Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models Junjie Yang Junhao Song Xudong Han Ziqian Bi Tianyang Wang ... Y. Zhang Qian Niu Benji Peng Keyu Chen Ming Liu VLM 40 0 0 18 Apr 2025
Just KIDDIN: Knowledge Infusion and Distillation for Detection of INdecent Memes Rahul Garg Trilok Padhi Hemang Jain Ugur Kursuncu Ponnurangam Kumaraguru 72 2 0 19 Nov 2024
Cascade-Aware Training of Language Models Congchao Wang Sean Augenstein Keith Rush Wittawat Jitkrittum Harikrishna Narasimhan A. S. Rawat A. Menon Alec Go 28 4 0 29 May 2024
Hard Gate Knowledge Distillation -- Leverage Calibration for Robust and Reliable Language Model Dongkyu Lee Zhiliang Tian Ying Zhao Ka Chun Cheung N. Zhang 28 3 0 22 Oct 2022
BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation Haoran Xu Benjamin Van Durme Kenton W. Murray 42 57 0 09 Sep 2021
Understanding and Improving Lexical Choice in Non-Autoregressive Translation Liang Ding Longyue Wang Xuebo Liu Derek F. Wong Dacheng Tao Zhaopeng Tu 91 76 0 29 Dec 2020