DisWOT: Student Architecture Search for Distillation WithOut Training

28 March 2023

Papers citing "DisWOT: Student Architecture Search for Distillation WithOut Training"

29 / 29 papers shown

Title
Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models Junjie Yang Junhao Song Xudong Han Ziqian Bi Tianyang Wang ... Y. Zhang Qian Niu Benji Peng Keyu Chen Ming Liu VLM 40 0 0 18 Apr 2025
CustomKD: Customizing Large Vision Foundation for Edge Model Improvement via Knowledge Distillation Jungsoo Lee Debasmit Das Munawar Hayat Sungha Choi Kyuwoong Hwang Fatih Porikli VLM 63 0 0 23 Mar 2025
Asymmetric Decision-Making in Online Knowledge Distillation:Unifying Consensus and Divergence Zhaowei Chen Borui Zhao Yuchen Ge Yuhao Chen Renjie Song Jiajun Liang 42 0 0 09 Mar 2025
Multi-Level Feature Distillation of Joint Teachers Trained on Distinct Image Datasets Adrian Iordache B. Alexe Radu Tudor Ionescu 29 1 0 29 Oct 2024
LPZero: Language Model Zero-cost Proxy Search from Zero Peijie Dong Lujun Li Xiang Liu Zhenheng Tang Xuebo Liu Qiang Wang Xiaowen Chu 51 2 0 07 Oct 2024
Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensembling Yuxuan Yao Han Wu Mingyang Liu Sichun Luo Xiongwei Han Jie Liu Zhijiang Guo Linqi Song 56 4 0 03 Oct 2024
Student-Oriented Teacher Knowledge Refinement for Knowledge Distillation Chaomin Shen Yaomin Huang Haokun Zhu Jinsong Fan Guixu Zhang 21 0 0 27 Sep 2024
OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strength Le Yang Ziwei Zheng Yizeng Han Shiji Song Gao Huang Fan Li 21 1 0 22 Sep 2024
Bandwidth-Aware and Overlap-Weighted Compression for Communication-Efficient Federated Learning Zichen Tang Junlin Huang Rudan Yan Yuxin Wang Zhenheng Tang S. Shi Amelie Chi Zhou Xiaowen Chu FedML 44 2 0 27 Aug 2024
NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models Cheng Lin Lujun Li Dezhi Li Jie Zou Wei Xue Yike Guo AI4TS 28 4 0 18 Aug 2024
Computer Vision Model Compression Techniques for Embedded Systems: A Survey Alexandre Lopes Fernando Pereira dos Santos D. Oliveira Mauricio Schiezaro Hélio Pedrini 26 5 0 15 Aug 2024
STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMs Peijie Dong Lujun Li Dayou Du Yuhan Chen Zhenheng Tang ... Wei Xue Wenhan Luo Qi-fei Liu Yi-Ting Guo Xiaowen Chu MQ 43 4 0 03 Aug 2024
Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Models Peijie Dong Lujun Li Zhenheng Tang Xiang Liu Xinglin Pan Qiang-qiang Wang Xiaowen Chu 48 23 0 05 Jun 2024
CKD: Contrastive Knowledge Distillation from A Sample-wise Perspective Wencheng Zhu Xin Zhou Pengfei Zhu Yu Wang Qinghua Hu VLM 56 1 0 22 Apr 2024
Knowledge Distillation Based on Transformed Teacher Matching Kaixiang Zheng En-Hui Yang 19 16 0 17 Feb 2024
ParZC: Parametric Zero-Cost Proxies for Efficient NAS Peijie Dong Lujun Li Xinglin Pan Zimian Wei Xiang Liu Qiang-qiang Wang Xiaowen Chu 43 3 0 03 Feb 2024
Auto-Prox: Training-Free Vision Transformer Architecture Search via Automatic Proxy Discovery Zimian Wei Lujun Li Peijie Dong Zheng Hui Anggeng Li Menglong Lu H. Pan Zhiliang Tian Dongsheng Li ViT 37 16 0 14 Dec 2023
TVT: Training-Free Vision Transformer Search on Tiny Datasets Zimian Wei H. Pan Lujun Li Peijie Dong Zhiliang Tian Xin-Yi Niu Dongsheng Li ViT 28 7 0 24 Nov 2023
Pruning Large Language Models via Accuracy Predictor Yupeng Ji Yibo Cao Jiu-si Liu KELM 27 4 0 18 Sep 2023
Improving Scene Graph Generation with Superpixel-Based Interaction Learning Jingyi Wang Can Zhang Jinfa Huang Bo Ren Zhidong Deng 16 7 0 04 Aug 2023
Guided Distillation for Semi-Supervised Instance Segmentation Tariq Berrada Camille Couprie Alahari Karteek Jakob Verbeek 21 9 0 03 Aug 2023
EMQ: Evolving Training-free Proxies for Automated Mixed Precision Quantization Peijie Dong Lujun Li Zimian Wei Xin-Yi Niu Zhiliang Tian H. Pan MQ 33 28 0 20 Jul 2023
Generalizable Lightweight Proxy for Robust NAS against Diverse Perturbations Hyeonjeong Ha Minseon Kim S. Hwang OOD AAML 12 5 0 08 Jun 2023
NORM: Knowledge Distillation via N-to-One Representation Matching Xiaolong Liu Lujun Li Chao Li Anbang Yao 39 66 0 23 May 2023
A Survey on Recent Teacher-student Learning Studies Min Gao 15 3 0 10 Apr 2023
Gradient-Guided Knowledge Distillation for Object Detectors Qizhen Lan Qingze Tian 19 7 0 07 Mar 2023
A Light-weight Deep Human Activity Recognition Algorithm Using Multi-knowledge Distillation Runze Chen Haiyong Luo Fang Zhao Xuechun Meng Zhiqing Xie Yida Zhu VLM HAI 12 2 0 06 Jul 2021
Connection Sensitivity Matters for Training-free DARTS: From Architecture-Level Scoring to Operation-Level Sensitivity Analysis Miao Zhang Wei Huang Li Wang 13 1 0 22 Jun 2021
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 187 472 0 12 Jun 2018