Knowledge Distillation of Transformer-based Language Models Revisited

Knowledge Distillation of Transformer-based Language Models Revisited

29 June 2022

Jianwei Zhang

Yunfei Chu

Jingren Zhou

Hongxia Yang

Papers citing "Knowledge Distillation of Transformer-based Language Models Revisited"

5 / 5 papers shown

Title
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs Akhiad Bercovich Tomer Ronen Talor Abramovich Nir Ailon Nave Assaf ... Ido Shahaf Oren Tropp Omer Ullman Argov Ran Zilberstein Ran El-Yaniv 77 1 0 28 Nov 2024
Compact Language Models via Pruning and Knowledge Distillation Saurav Muralidharan Sharath Turuvekere Sreenivas Raviraj Joshi Marcin Chochowski M. Patwary M. Shoeybi Bryan Catanzaro Jan Kautz Pavlo Molchanov SyDa MQ 30 37 0 19 Jul 2024
Latent Feature-Guided Diffusion Models for Shadow Removal Kangfu Mei Luis Figueroa Zhe-nan Lin Zhihong Ding Scott D. Cohen Vishal M. Patel DiffM 24 18 0 04 Dec 2023
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,460 0 23 Jan 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,950 0 20 Apr 2018