Blockwisely Supervised Neural Architecture Search with Knowledge Distillation

29 November 2019

Xiaodan Liang

Xiaojun Chang

Papers citing "Blockwisely Supervised Neural Architecture Search with Knowledge Distillation"

27 / 27 papers shown

Title
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs Akhiad Bercovich Tomer Ronen Talor Abramovich Nir Ailon Nave Assaf ... Ido Shahaf Oren Tropp Omer Ullman Argov Ran Zilberstein Ran El-Yaniv 77 1 0 28 Nov 2024
DONNAv2 -- Lightweight Neural Architecture Search for Vision tasks Sweta Priyadarshi Tianyu Jiang Hsin-Pai Cheng S. Rama Krishna Viswanath Ganapathy C. Patel 31 0 0 26 Sep 2023
Knowledge Distillation from Single to Multi Labels: an Empirical Study Youcai Zhang Yuzhuo Qin Heng-Ye Liu Yanhao Zhang Yaqian Li X. Gu VLM 51 2 0 15 Mar 2023
Pareto-aware Neural Architecture Generation for Diverse Computational Budgets Yong Guo Yaofo Chen Yin Zheng Qi Chen P. Zhao Jian Chen Junzhou Huang Mingkui Tan 21 5 0 14 Oct 2022
BLOX: Macro Neural Architecture Search Benchmark and Algorithms Thomas C. P. Chau L. Dudziak Hongkai Wen Nicholas D. Lane Mohamed S. Abdelfattah 14 13 0 13 Oct 2022
Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey Dalin Zhang Kaixuan Chen Yan Zhao B. Yang Li-Ping Yao Christian S. Jensen 38 3 0 22 Aug 2022
Revisiting Label Smoothing and Knowledge Distillation Compatibility: What was Missing? Keshigeyan Chandrasegaran Ngoc-Trung Tran Yunqing Zhao Ngai-man Cheung 78 41 0 29 Jun 2022
Distributed Training for Deep Learning Models On An Edge Computing Network Using ShieldedReinforcement Learning Tanmoy Sen Haiying Shen OffRL 11 5 0 01 Jun 2022
Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search Minbin Huang Zhijian Huang Changlin Li Xin Chen Hangyang Xu Zhenguo Li Xiaodan Liang 28 18 0 12 Apr 2022
When NAS Meets Trees: An Efficient Algorithm for Neural Architecture Search Guocheng Qian Xuanyang Zhang G. Li Chen Zhao Yukang Chen X. Zhang Bernard Ghanem Jian-jun Sun 37 4 0 11 Apr 2022
Automated Progressive Learning for Efficient Training of Vision Transformers Changlin Li Bohan Zhuang Guangrun Wang Xiaodan Liang Xiaojun Chang Yi Yang 16 46 0 28 Mar 2022
Hot-Refresh Model Upgrades with Regression-Alleviating Compatible Training in Image Retrieval Binjie Zhang Yixiao Ge Yantao Shen Yu Li Chun Yuan Xuyuan Xu Yexin Wang Ying Shan VLM 25 7 0 24 Jan 2022
PP-LCNet: A Lightweight CPU Convolutional Neural Network Cheng Cui Tingquan Gao Shengyun Wei Yuning Du Ruoyu Guo ... X. Lv Qiwen Liu Xiaoguang Hu Dianhai Yu Yanjun Ma ObjD 39 121 0 17 Sep 2021
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation Chenhe Dong Guangrun Wang Hang Xu Jiefeng Peng Xiaozhe Ren Xiaodan Liang 16 28 0 15 Sep 2021
AutoFormer: Searching Transformers for Visual Recognition Minghao Chen Houwen Peng Jianlong Fu Haibin Ling ViT 36 259 0 01 Jul 2021
iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients Miao Zhang Steven W. Su Shirui Pan Xiaojun Chang Ehsan Abbasnejad Reza Haffari 10 68 0 21 Jun 2021
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Changlin Li Tao Tang Guangrun Wang Jiefeng Peng Bing Wang Xiaodan Liang Xiaojun Chang ViT 35 105 0 23 Mar 2021
Towards Personalized Federated Learning A. Tan Han Yu Li-zhen Cui Qiang Yang FedML AI4CE 183 840 0 01 Mar 2021
AlphaNet: Improved Training of Supernets with Alpha-Divergence Dilin Wang Chengyue Gong Meng Li Qiang Liu Vikas Chandra 147 44 0 16 Feb 2021
Inter-layer Transition in Neural Architecture Search Benteng Ma Jing Zhang Yong-quan Xia Dacheng Tao 21 4 0 30 Nov 2020
DOTS: Decoupling Operation and Topology in Differentiable Architecture Search Yuchao Gu Li-Juan Wang Yun-Hai Liu Yi Yang Yu-Huan Wu Shao-Ping Lu Ming-Ming Cheng 23 48 0 02 Oct 2020
Cyclic Differentiable Architecture Search Hongyuan Yu Houwen Peng Yan Huang Jianlong Fu Hao Du Liang Wang Haibin Ling 3DPC 19 48 0 18 Jun 2020
Multi-fidelity Neural Architecture Search with Knowledge Distillation I. Trofimov Nikita Klyuchnikov Mikhail Salnikov Alexander N. Filippov Evgeny Burnaev 27 15 0 15 Jun 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,832 0 09 Jun 2020
A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions Pengzhen Ren Yun Xiao Xiaojun Chang Po-Yao (Bernie) Huang Zhihui Li Xiaojiang Chen Xin Wang AI4CE 29 653 0 01 Jun 2020
MixPath: A Unified Approach for One-shot Neural Architecture Search Xiangxiang Chu Shun Lu Xudong Li Bo-Wen Zhang 19 21 0 16 Jan 2020
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 264 5,326 0 05 Nov 2016