Knowledge Distillation as Semiparametric Inference

20 April 2021

Papers citing "Knowledge Distillation as Semiparametric Inference"

27 / 27 papers shown

Title
Learning Critically: Selective Self Distillation in Federated Learning on Non-IID Data Yuting He Yiqiang Chen Xiaodong Yang H. Yu Yi-Hua Huang Yang Gu FedML 55 20 0 20 Apr 2025
Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models Junjie Yang Junhao Song Xudong Han Ziqian Bi Tianyang Wang ... Y. Zhang Qian Niu Benji Peng Keyu Chen Ming Liu VLM 47 0 0 18 Apr 2025
Modifying Final Splits of Classification Tree for Fine-tuning Subpopulation Target in Policy Making Lei Bill Wang Zhenbang Jiao Fangyi Wang 55 0 0 24 Feb 2025
A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs A. S. Rawat Veeranjaneyulu Sadhanala Afshin Rostamizadeh Ayan Chakrabarti Wittawat Jitkrittum ... Rakesh Shivanna Sashank J. Reddi A. Menon Rohan Anil Sanjiv Kumar 28 2 0 24 Oct 2024
Distilling interpretable causal trees from causal forests Patrick Rehill CML 28 0 0 02 Aug 2024
How to Train the Teacher Model for Effective Knowledge Distillation Shayan Mohajer Hamidi Xizhen Deng Renhao Tan Linfeng Ye Ahmed H. Salamah 32 2 0 25 Jul 2024
Exploring Dark Knowledge under Various Teacher Capacities and Addressing Capacity Mismatch Xin-Chun Li Wen-Shu Fan Bowen Tao Le Gan De-Chuan Zhan 22 2 0 21 May 2024
Automated Efficient Estimation using Monte Carlo Efficient Influence Functions Raj Agrawal Sam Witty Andy Zane Eli Bingham 24 2 0 29 Feb 2024
Data-Driven Estimation of the False Positive Rate of the Bayes Binary Classifier via Soft Labels Minoh Jeong Martina Cardone Alex Dytso 29 0 0 27 Jan 2024
Bayes Conditional Distribution Estimation for Knowledge Distillation Based on Conditional Mutual Information Linfeng Ye Shayan Mohajer Hamidi Renhao Tan En-Hui Yang VLM 37 12 0 16 Jan 2024
Transparency challenges in policy evaluation with causal machine learning -- improving usability and accountability Patrick Rehill Nicholas Biddle CML ELM 24 3 0 20 Oct 2023
Towards the Fundamental Limits of Knowledge Transfer over Finite Domains Qingyue Zhao Banghua Zhu 25 4 0 11 Oct 2023
Confidence Ranking for CTR Prediction Jian Zhu Congcong Liu Pei Wang Xiwei Zhao Zhangang Lin Jingping Shao 17 2 0 28 Jun 2023
Parallel Neurosymbolic Integration with Concordia Jonathan Feldstein Modestas Jurcius Efthymia Tsamoura NAI 25 1 0 01 Jun 2023
Knowledge Distillation Performs Partial Variance Reduction M. Safaryan Alexandra Peste Dan Alistarh 6 6 0 27 May 2023
LightTS: Lightweight Time Series Classification with Adaptive Ensemble Distillation -- Extended Version David Campos Miao Zhang B. Yang Tung Kieu Chenjuan Guo Christian S. Jensen AI4TS 40 47 0 24 Feb 2023
Learning From Biased Soft Labels Hua Yuan Ning Xu Yuge Shi Xin Geng Yong Rui FedML 24 6 0 16 Feb 2023
Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications Chengyu Dong OOD UQCV BDL AI4CE 21 0 0 02 Feb 2023
On student-teacher deviations in distillation: does it pay to disobey? Vaishnavh Nagarajan A. Menon Srinadh Bhojanapalli H. Mobahi Surinder Kumar 41 9 0 30 Jan 2023
Supervision Complexity and its Role in Knowledge Distillation Hrayr Harutyunyan A. S. Rawat A. Menon Seungyeon Kim Surinder Kumar 22 12 0 28 Jan 2023
Asymmetric Temperature Scaling Makes Larger Networks Teach Well Again Xin-Chun Li Wenxuan Fan Shaoming Song Yinchuan Li Bingshuai Li Yunfeng Shao De-Chuan Zhan 34 30 0 10 Oct 2022
Scalable Regularization of Scene Graph Generation Models using Symbolic Theories Davide Buffelli Efthymia Tsamoura 8 2 0 06 Sep 2022
Informed Learning by Wide Neural Networks: Convergence, Generalization and Sampling Complexity Jianyi Yang Shaolei Ren 21 3 0 02 Jul 2022
Toward Student-Oriented Teacher Network Training For Knowledge Distillation Chengyu Dong Liyuan Liu Jingbo Shang 25 6 0 14 Jun 2022
Selective Cross-Task Distillation Su Lu Han-Jia Ye De-Chuan Zhan 23 0 0 25 Apr 2022
Teacher's pet: understanding and mitigating biases in distillation Michal Lukasik Srinadh Bhojanapalli A. Menon Sanjiv Kumar 11 25 0 19 Jun 2021
Conditional Image Synthesis With Auxiliary Classifier GANs Augustus Odena C. Olah Jonathon Shlens GAN 224 3,190 0 30 Oct 2016