Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning

17 December 2020

Papers citing "Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning"

15 / 215 papers shown

Title
Assessing Generalization of SGD via Disagreement Yiding Jiang Vaishnavh Nagarajan Christina Baek J. Zico Kolter 53 107 0 25 Jun 2021
Improving Multi-Modal Learning with Uni-Modal Teachers Chenzhuang Du Tingle Li Yichen Liu Zixin Wen Tianyu Hua Yue Wang Hang Zhao 18 45 0 21 Jun 2021
Teacher's pet: understanding and mitigating biases in distillation Michal Lukasik Srinadh Bhojanapalli A. Menon Sanjiv Kumar 11 25 0 19 Jun 2021
S-LIME: Stabilized-LIME for Model Explanation Zhengze Zhou Giles Hooker Fei Wang FAtt 17 86 0 15 Jun 2021
Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning Zixin Wen Yuanzhi Li SSL MLT 24 131 0 31 May 2021
OCHADAI-KYOTO at SemEval-2021 Task 1: Enhancing Model Generalization and Robustness for Lexical Complexity Prediction Y. Taya L. Pereira Fei Cheng Ichiro Kobayashi 20 0 0 12 May 2021
Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updates Yuqing Xie Yi-An Lai Yuanjun Xiong Yi Zhang Stefano Soatto UQCV 9 16 0 07 May 2021
Structured Ensembles: an Approach to Reduce the Memory Footprint of Ensemble Methods Jary Pomponi Simone Scardapane A. Uncini UQCV 41 7 0 06 May 2021
Track, Check, Repeat: An EM Approach to Unsupervised Tracking Adam W. Harley Yiming Zuo Jing Wen Ayush Mangal Shubhankar Potdar Ritwick Chaudhry Katerina Fragkiadaki 25 9 0 07 Apr 2021
Students are the Best Teacher: Exit-Ensemble Distillation with Multi-Exits Hojung Lee Jong-Seok Lee 17 7 0 01 Apr 2021
Photoacoustic image synthesis with generative adversarial networks Melanie Schellenberg J. Gröhl Kris K. Dreher Jan-Hinrich Nolke Niklas Holzwarth M. Tizabi A. Seitel Lena Maier-Hein GAN MedIm 22 17 0 29 Mar 2021
Randomization-based Machine Learning in Renewable Energy Prediction Problems: Critical Literature Review, New Results and Perspectives Javier Del Ser D. Casillas-Pérez L. Cornejo-Bueno Luis Prieto-Godino J. Sanz-Justo C. Casanova-Mateo S. Salcedo-Sanz AI4CE 29 42 0 26 Mar 2021
Knowledge Distillation in Deep Learning and its Applications Abdolmaged Alkhulaifi Fahad Alsahli Irfan Ahmad FedML 12 76 0 17 Jul 2020
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 192 473 0 12 Jun 2018
Comparison and anti-concentration bounds for maxima of Gaussian random vectors Victor Chernozhukov Denis Chetverikov Kengo Kato 57 219 0 21 Jan 2013