Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in
Sequence-Level Knowledge Distillation

Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation

19 June 2024

Xiyao Wang

Furong Huang

Papers citing "Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation"

13 / 13 papers shown

Title
Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models Junjie Yang Junhao Song Xudong Han Ziqian Bi Tianyang Wang ... Y. Zhang Qian Niu Benji Peng Keyu Chen Ming Liu VLM 40 0 0 18 Apr 2025
Memorization Inheritance in Sequence-Level Knowledge Distillation for Neural Machine Translation Verna Dankers Vikas Raunak VLM 56 0 0 03 Feb 2025
MergeME: Model Merging Techniques for Homogeneous and Heterogeneous MoEs Yuhang Zhou Giannis Karamanolakis Victor Soto Anna Rumshisky Mayank Kulkarni Furong Huang Wei Ai Jianhua Lu MoMe 101 0 0 03 Feb 2025
CSRec: Rethinking Sequential Recommendation from A Causal Perspective Xiaoyu Liu Jiaxin Yuan Yuhang Zhou Jingling Li Furong Huang Wei Ai CML HAI 19 0 0 23 Aug 2024
Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation Hamidreza Rouzegar Masoud Makrehchi 25 9 0 17 Jun 2024
Large Language Models and Causal Inference in Collaboration: A Survey Xiaoyu Liu Paiheng Xu Junda Wu Jiaxin Yuan Yifan Yang ... Haoliang Wang Tong Yu Julian McAuley Wei Ai Furong Huang ELM LRM 72 5 0 14 Mar 2024
Improving the TENOR of Labeling: Re-evaluating Topic Models for Content Analysis Zongxia Li Andrew Mao Daniel Stephens Pranav Goel Emily Walpole A. Dima Juan Fung Jordan L. Boyd-Graber 53 10 0 29 Jan 2024
Explore Spurious Correlations at the Concept Level in Language Models for Text Classification Yuhang Zhou Paiheng Xu Xiaoyu Liu Bang An Wei Ai Furong Huang LRM 71 20 0 15 Nov 2023
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes Lokesh Nagalapatti Chun-Liang Li Chih-Kuan Yeh Hootan Nakhost Yasuhisa Fujii Alexander Ratner Ranjay Krishna Chen-Yu Lee Tomas Pfister ALM 204 498 0 03 May 2023
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 229 2,413 0 06 Oct 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 4,048 0 24 May 2022
Balancing Methods for Multi-label Text Classification with Long-Tailed Class Distribution Yi Huang Buse Giledereli Abdullatif Köksal Arzucan Özgür Elif Özkirimli VLM 25 76 0 10 Sep 2021
Cold-start Active Learning through Self-supervised Language Modeling Michelle Yuan Hsuan-Tien Lin Jordan L. Boyd-Graber 104 180 0 19 Oct 2020