Do Deep Nets Really Need to be Deep?

21 December 2013

Lei Jimmy Ba

Papers citing "Do Deep Nets Really Need to be Deep?"

50 / 337 papers shown

Title
Secure Your Ride: Real-time Matching Success Rate Prediction for Passenger-Driver Pairs Yuandong Wang Hongzhi Yin Lian Wu Tong Chen Chunyang Liu 18 7 0 14 Sep 2021
On the Efficiency of Subclass Knowledge Distillation in Classification Tasks A. Sajedi Konstantinos N. Plataniotis 16 4 0 12 Sep 2021
FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Models Lan Zhang Dapeng Wu Xiaoyong Yuan FedML 38 47 0 08 Sep 2021
Compact representations of convolutional neural networks via weight pruning and quantization Giosuè Cataldo Marinò A. Petrini D. Malchiodi Marco Frasca MQ 21 4 0 28 Aug 2021
Learning Energy-Based Approximate Inference Networks for Structured Applications in NLP Lifu Tu BDL 35 0 0 27 Aug 2021
Efficient training of lightweight neural networks using Online Self-Acquired Knowledge Distillation Maria Tzelepi Anastasios Tefas 11 6 0 26 Aug 2021
Supervised Compression for Resource-Constrained Edge Computing Systems Yoshitomo Matsubara Ruihan Yang Marco Levorato Stephan Mandt 19 57 0 21 Aug 2021
DarkGAN: Exploiting Knowledge Distillation for Comprehensible Audio Synthesis with GANs J. Nistal Stefan Lattner G. Richard 21 8 0 03 Aug 2021
Developing efficient transfer learning strategies for robust scene recognition in mobile robotics using pre-trained convolutional neural networks H. Baumgartl Ricardo Buettner 3DPC 54 3 0 23 Jul 2021
SAGE: A Split-Architecture Methodology for Efficient End-to-End Autonomous Vehicle Control Arnav V. Malawade Mohanad Odema Sebastien Lajeunesse-DeGroot M. A. Al Faruque 26 20 0 22 Jul 2021
Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies F. Xie Han Yuan Yilin Ning M. Ong Mengling Feng W. Hsu B. Chakraborty Nan Liu 27 83 0 21 Jul 2021
Mitigating severe over-parameterization in deep convolutional neural networks through forced feature abstraction and compression with an entropy-based heuristic Nidhi Gowdra R. Sinha Stephen G. MacDonell W. Yan 21 9 0 27 Jun 2021
Simple Distillation Baselines for Improving Small Self-supervised Models Jindong Gu Wei Liu Yonglong Tian 19 8 0 21 Jun 2021
Knowledge Distillation via Instance-level Sequence Learning Haoran Zhao Xin Sun Junyu Dong Zihe Dong Qiong Li 31 23 0 21 Jun 2021
We Can Always Catch You: Detecting Adversarial Patched Objects WITH or WITHOUT Signature Binxiu Liang Jiachun Li Jianjun Huang AAML 33 12 0 09 Jun 2021
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression Weiyue Su Xuyi Chen Shi Feng Jiaxiang Liu Weixin Liu Yu Sun Hao Tian Hua Wu Haifeng Wang 34 13 0 04 Jun 2021
Data-Free Knowledge Distillation for Heterogeneous Federated Learning Zhuangdi Zhu Junyuan Hong Jiayu Zhou FedML 27 630 0 20 May 2021
What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory Rahul Parhi Robert D. Nowak MLT 38 70 0 07 May 2021
Performance Evaluation of Deep Convolutional Maxout Neural Network in Speech Recognition Arash Dehghani Seyyed Ali Seyyedsalehi 20 2 0 04 May 2021
ImageNet-21K Pretraining for the Masses T. Ridnik Emanuel Ben-Baruch Asaf Noy Lihi Zelnik-Manor SSeg VLM CLIP 184 689 0 22 Apr 2021
Balanced Knowledge Distillation for Long-tailed Learning Shaoyu Zhang Chen Chen Xiyuan Hu Silong Peng 48 57 0 21 Apr 2021
Distill on the Go: Online knowledge distillation in self-supervised learning Prashant Shivaram Bhat Elahe Arani Bahram Zonooz SSL 22 28 0 20 Apr 2021
Knowledge Distillation as Semiparametric Inference Tri Dao G. Kamath Vasilis Syrgkanis Lester W. Mackey 40 31 0 20 Apr 2021
Efficient Transformers in Reinforcement Learning using Actor-Learner Distillation Emilio Parisotto Ruslan Salakhutdinov 42 44 0 04 Apr 2021
Student Network Learning via Evolutionary Knowledge Distillation Kangkai Zhang Chunhui Zhang Shikun Li Dan Zeng Shiming Ge 22 83 0 23 Mar 2021
Compacting Deep Neural Networks for Internet of Things: Methods and Applications Ke Zhang Hanbo Ying Hongning Dai Lin Li Yuangyuang Peng Keyi Guo Hongfang Yu 18 38 0 20 Mar 2021
Membership Inference Attacks on Machine Learning: A Survey Hongsheng Hu Z. Salcic Lichao Sun Gillian Dobbie Philip S. Yu Xuyun Zhang MIACV 35 412 0 14 Mar 2021
Contrastive Semi-supervised Learning for ASR Alex Xiao Christian Fuegen Abdel-rahman Mohamed 26 20 0 09 Mar 2021
Deep Model Intellectual Property Protection via Deep Watermarking Jie Zhang Dongdong Chen Jing Liao Weiming Zhang Huamin Feng G. Hua Nenghai Yu 21 106 0 08 Mar 2021
Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges Yoshitomo Matsubara Marco Levorato Francesco Restuccia 33 199 0 08 Mar 2021
Adaptive Multi-Teacher Multi-level Knowledge Distillation Yuang Liu Wei Zhang Jun Wang 28 157 0 06 Mar 2021
Deep ReLU Networks Preserve Expected Length Boris Hanin Ryan Jeong David Rolnick 29 14 0 21 Feb 2021
Resilient Machine Learning for Networked Cyber Physical Systems: A Survey for Machine Learning Security to Securing Machine Learning for CPS Felix O. Olowononi D. Rawat Chunmei Liu 34 132 0 14 Feb 2021
Copycat CNN: Are Random Non-Labeled Data Enough to Steal Knowledge from Black-box Models? Jacson Rodrigues Correia-Silva Rodrigo Berriel C. Badue Alberto F. de Souza Thiago Oliveira-Santos MLAU 13 14 0 21 Jan 2021
Learning to Augment for Data-Scarce Domain BERT Knowledge Distillation Lingyun Feng Minghui Qiu Yaliang Li Haitao Zheng Ying Shen 43 10 0 20 Jan 2021
Resolution-Based Distillation for Efficient Histology Image Classification Joseph DiPalma A. Suriawinata L. Tafe Lorenzo Torresani Saeed Hassanpour 35 35 0 11 Jan 2021
Learning from Weakly-labeled Web Videos via Exploring Sub-Concepts Kunpeng Li Zizhao Zhang Guanhang Wu Xuehan Xiong Chen-Yu Lee Zhichao Lu Y. Fu Tomas Pfister 29 5 0 11 Jan 2021
Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead Maurizio Capra Beatrice Bussolino Alberto Marchisio Guido Masera Maurizio Martina Muhammad Shafique BDL 59 140 0 21 Dec 2020
ISD: Self-Supervised Learning by Iterative Similarity Distillation Ajinkya Tejankar Soroush Abbasi Koohpayegani Vipin Pillai Paolo Favaro Hamed Pirsiavash SSL 27 44 0 16 Dec 2020
Movie Summarization via Sparse Graph Construction Pinelopi Papalampidi Frank Keller Mirella Lapata 24 32 0 14 Dec 2020
Amodal Segmentation Based on Visible Region Segmentation and Shape Prior Yuting Xiao Yanyu Xu Z. Zhong Weixin Luo Jiawei Li Shenghua Gao 23 51 0 10 Dec 2020
Cross-Layer Distillation with Semantic Calibration Defang Chen Jian-Ping Mei Yuan Zhang Can Wang Yan Feng Chun-Yen Chen FedML 45 286 0 06 Dec 2020
Data-Free Model Extraction Jean-Baptiste Truong Pratyush Maini R. Walls Nicolas Papernot MIACV 15 181 0 30 Nov 2020
Dynamic Hard Pruning of Neural Networks at the Edge of the Internet Lorenzo Valerio F. M. Nardini A. Passarella R. Perego 20 12 0 17 Nov 2020
Robustness and Diversity Seeking Data-Free Knowledge Distillation Pengchao Han Jihong Park Shiqiang Wang Yejun Liu 15 12 0 07 Nov 2020
Are wider nets better given the same number of parameters? A. Golubeva Behnam Neyshabur Guy Gur-Ari 27 44 0 27 Oct 2020
Black-Box Ripper: Copying black-box models using generative evolutionary algorithms Antonio Bărbălău Adrian Cosma Radu Tudor Ionescu Marius Popescu MIACV MLAU 30 43 0 21 Oct 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 242 611 0 13 Oct 2020
Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor Xinyu Wang Yong-jia Jiang Zhaohui Yan Zixia Jia Nguyen Bach Tao Wang Zhongqiang Huang Fei Huang Kewei Tu 26 10 0 10 Oct 2020
Cross-Lingual Text Classification with Minimal Resources by Transferring a Sparse Teacher Giannis Karamanolakis Daniel J. Hsu Luis Gravano VLM 36 63 0 06 Oct 2020