Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2107.01378
Cited By
v1
v2
v3
v4 (latest)
Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation
3 July 2021
Zhiwei Hao
Jianyuan Guo
Ding Jia
Kai Han
Yehui Tang
Chao Zhang
Dacheng Tao
Yunhe Wang
ViT
Re-assign community
ArXiv (abs)
PDF
HTML
Github
Papers citing
"Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation"
50 / 50 papers shown
Rethinking Vision Transformer Depth via Structural Reparameterization
Chengwei Zhou
Vipin Chaudhary
Gourav Datta
ViT
162
0
0
24 Nov 2025
From Low-Rank Features to Encoding Mismatch: Rethinking Feature Distillation in Vision Transformers
Huiyuan Tian
Bonan Xu
Shijian Li
Xin Jin
165
1
0
19 Nov 2025
Distillation Dynamics: Towards Understanding Feature-Based Distillation in Vision Transformers
Huiyuan Tian
Bonan Xu Shijian Li
Shijian Li
274
1
0
10 Nov 2025
No Alignment Needed for Generation: Learning Linearly Separable Representations in Diffusion Models
Junno Yun
Yasar Utku Alçalar
Mehmet Akçakaya
163
4
0
25 Sep 2025
ResidualViT for Efficient Temporally Dense Video Encoding
Mattia Soldan
Fabian Caba Heilbron
Bernard Ghanem
Josef Sivic
Bryan C. Russell
225
1
0
16 Sep 2025
UNIFORM: Unifying Knowledge from Large-scale and Diverse Pre-trained Models
Yimu Wang
Weiming Zhuang
Chen Chen
Jiabo Huang
Jingtao Li
Lingjuan Lyu
FedML
242
1
0
27 Aug 2025
Cross-Architecture Distillation Made Simple with Redundancy Suppression
Weijia Zhang
Yuehao Liu
Wu Ran
Chao Ma
239
3
0
29 Jul 2025
A Layered Self-Supervised Knowledge Distillation Framework for Efficient Multimodal Learning on the Edge
Tarique Dahri
Zulfiqar Ali Memon
Zhenyu Yu
Mohd Yamani Idna Idris
Sheheryar Khan
Sadiq Ahmad
Maged Shoman
Saddam Aziz
Rizwan Qureshi
261
0
0
08 Jun 2025
MoKD: Multi-Task Optimization for Knowledge Distillation
Zeeshan Hayder
A. Cheraghian
Lars Petersson
Mehrtash Harandi
VLM
459
0
0
13 May 2025
Position: Beyond Euclidean -- Foundation Models Should Embrace Non-Euclidean Geometries
Neil He
Jiahong Liu
Buze Zhang
N. Bui
Ali Maatouk
Menglin Yang
Irwin King
Melanie Weber
Rex Ying
375
9
0
11 Apr 2025
Distilling Knowledge from Heterogeneous Architectures for Semantic Segmentation
AAAI Conference on Artificial Intelligence (AAAI), 2025
Yuanmin Huang
Kai Hu
Yuhui Zhang
Z. Chen
Xieping Gao
349
4
0
10 Apr 2025
Random Conditioning with Distillation for Data-Efficient Diffusion Model Compression
Computer Vision and Pattern Recognition (CVPR), 2025
Dohyun Kim
S. Park
Geonhee Han
Seung Wook Kim
Paul Hongsuck Seo
DiffM
394
1
0
02 Apr 2025
U-REPA: Aligning Diffusion U-Nets to ViTs
Yuchuan Tian
Hanting Chen
Mengyu Zheng
Yuchen Liang
Chao Xu
Yunhe Wang
495
14
0
24 Mar 2025
Jointly Understand Your Command and Intention:Reciprocal Co-Evolution between Scene-Aware 3D Human Motion Synthesis and Analysis
Xuehao Gao
Yang Yang
Shaoyi Du
Guo-Jun Qi
Junwei Han
566
1
0
01 Mar 2025
Janus: Collaborative Vision Transformer Under Dynamic Network Environment
IEEE Conference on Computer Communications (IEEE INFOCOM), 2025
Linyi Jiang
Silvery Fu
Yifei Zhu
Bo Li
ViT
945
5
0
14 Feb 2025
iFormer: Integrating ConvNet and Transformer for Mobile Application
International Conference on Learning Representations (ICLR), 2025
Chuanyang Zheng
ViT
461
12
0
26 Jan 2025
Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deployment
International Conference on Artificial Neural Networks (ICANN), 2025
Xubin Wang
Weijia Jia
Weijia Jia
620
21
0
04 Jan 2025
Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers
Mingliang Xu
Yuyao Zhou
Yuxin Zhang
Shen Li
Shen Li
Jiayi Ji
Zhanpeng Zeng
Rongrong Ji
MQ
947
0
0
21 Dec 2024
On the Surprising Effectiveness of Attention Transfer for Vision Transformers
Neural Information Processing Systems (NeurIPS), 2024
Alexander C. Li
Yuandong Tian
Bin Chen
Deepak Pathak
Xinlei Chen
253
15
0
14 Nov 2024
DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture
Computer Vision and Pattern Recognition (CVPR), 2024
Qianlong Xiang
Miao Zhang
Yuzhang Shang
Yue Yu
Yan Yan
Liqiang Nie
DiffM
425
25
0
05 Sep 2024
UNIC: Universal Classification Models via Multi-teacher Distillation
European Conference on Computer Vision (ECCV), 2024
Mert Bulent Sariyildiz
Philippe Weinzaepfel
Thomas Lucas
Diane Larlus
Yannis Kalantidis
451
20
0
09 Aug 2024
Neural-based Video Compression on Solar Dynamics Observatory Images
Atefeh Khoshkhahtinat
Ali Zafari
P. Mehta
Nasser M. Nasrabadi
Barbara J. Thompson
M. Kirk
D. D. Silva
374
1
0
12 Jul 2024
ViT-1.58b: Mobile Vision Transformers in the 1-bit Era
Zhengqing Yuan
Rong Zhou
Hongyi Wang
Lifang He
Yanfang Ye
Lichao Sun
MQ
235
14
0
26 Jun 2024
Efficient Multimodal Large Language Models: A Survey
Yizhang Jin
Jian Li
Yexin Liu
Tianjun Gu
Kai Wu
...
Xin Tan
Zhenye Gan
Yabiao Wang
Chengjie Wang
Lizhuang Ma
LRM
371
107
0
17 May 2024
Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers
Diana-Nicoleta Grigore
Mariana-Iuliana Georgescu
J. A. Justo
T. Johansen
Andreea-Iuliana Ionescu
Radu Tudor Ionescu
401
4
0
14 Apr 2024
The Need for Speed: Pruning Transformers with One Recipe
Samir Khaki
Konstantinos N. Plataniotis
405
17
0
26 Mar 2024
V
k
D
:
V_kD:
V
k
D
:
Improving Knowledge Distillation using Orthogonal Projections
Computer Vision and Pattern Recognition (CVPR), 2024
Roy Miles
Ismail Elezi
Jiankang Deng
390
29
0
10 Mar 2024
Tiny Reinforcement Learning for Quadruped Locomotion using Decision Transformers
Orhan Eren Akgün
Néstor Cuevas
Matheus Farias
Daniel Garces
273
1
0
20 Feb 2024
A Survey on Transformer Compression
Yehui Tang
Yunhe Wang
Jianyuan Guo
Zhijun Tu
Kai Han
Hailin Hu
Dacheng Tao
584
73
0
05 Feb 2024
A Manifold Representation of the Key in Vision Transformers
Li Meng
Morten Goodwin
Anis Yazidi
P. Engelstad
508
1
0
01 Feb 2024
One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation
Neural Information Processing Systems (NeurIPS), 2023
Zhiwei Hao
Jianyuan Guo
Kai Han
Yehui Tang
Han Hu
Yunhe Wang
Chang Xu
362
152
0
30 Oct 2023
Understanding the Effects of Projectors in Knowledge Distillation
Yudong Chen
Sen Wang
Jiajun Liu
Xuwei Xu
Frank de Hoog
Brano Kusy
Zi Huang
397
4
0
26 Oct 2023
Talking Models: Distill Pre-trained Knowledge to Downstream Models via Interactive Communication
Zhe Zhao
Qingyun Liu
Huan Gui
Bang An
Lichan Hong
Ed H. Chi
207
1
0
04 Oct 2023
Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism
Neural Information Processing Systems (NeurIPS), 2023
Chengcheng Wang
Wei He
Ying Nie
Jianyuan Guo
Chuanjian Liu
Kai Han
Yunhe Wang
ObjD
463
526
0
20 Sep 2023
DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices
IEEE Transactions on Mobile Computing (IEEE TMC), 2023
Guanyu Xu
Zhiwei Hao
Yong Luo
Han Hu
J. An
Shiwen Mao
ViT
296
44
0
10 Sep 2023
A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023
Lorenzo Papa
Paolo Russo
Irene Amerini
Luping Zhou
410
119
0
05 Sep 2023
LGViT: Dynamic Early Exiting for Accelerating Vision Transformer
ACM Multimedia (ACM MM), 2023
Guanyu Xu
Jiawei Hao
Li Shen
Han Hu
Yong Luo
Hui Lin
J. Shen
349
38
0
01 Aug 2023
VanillaKD: Revisit the Power of Vanilla Knowledge Distillation from Small Scale to Large Scale
Zhiwei Hao
Jianyuan Guo
Kai Han
Han Hu
Chang Xu
Yunhe Wang
252
21
0
25 May 2023
Bi-ViT: Pushing the Limit of Vision Transformer Quantization
AAAI Conference on Artificial Intelligence (AAAI), 2023
Yanjing Li
Sheng Xu
Mingbao Lin
Xianbin Cao
Chuanjian Liu
Xiao Sun
Baochang Zhang
ViT
MQ
240
22
0
21 May 2023
Visual Tuning
ACM Computing Surveys (ACM Comput. Surv.), 2023
Bruce X. B. Yu
Jianlong Chang
Haixin Wang
Lin Liu
Shijie Wang
...
Lingxi Xie
Haojie Li
Zhouchen Lin
Qi Tian
Chang Wen Chen
VLM
537
64
0
10 May 2023
RIFormer: Keep Your Vision Backbone Effective While Removing Token Mixer
Computer Vision and Pattern Recognition (CVPR), 2023
Jiahao Wang
Songyang Zhang
Yong Liu
Taiqiang Wu
Yujiu Yang
Xihui Liu
Kai-xiang Chen
Ping Luo
Dahua Lin
274
31
0
12 Apr 2023
Knowledge Distillation in Vision Transformers: A Critical Review
Gousia Habib
Tausifa Jan Saleem
Brejesh Lall
401
25
0
04 Feb 2023
PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers
IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2022
Zhikai Li
Mengjuan Chen
Junrui Xiao
Qingyi Gu
ViT
MQ
318
65
0
13 Sep 2022
I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference
IEEE International Conference on Computer Vision (ICCV), 2022
Zhikai Li
Qingyi Gu
MQ
516
171
0
04 Jul 2022
Chemical transformer compression for accelerating both training and inference of molecular modeling
Yi Yu
K. Börjesson
148
0
0
16 May 2022
Depth Estimation with Simplified Transformer
John Yang
Le An
Anurag Dixit
Jinkyu Koo
Su Inn Park
MDE
244
23
0
28 Apr 2022
Patch Similarity Aware Data-Free Quantization for Vision Transformers
European Conference on Computer Vision (ECCV), 2022
Zhikai Li
Liping Ma
Mengjuan Chen
Junrui Xiao
Qingyi Gu
MQ
ViT
465
77
0
04 Mar 2022
Meta Knowledge Distillation
Jihao Liu
Boxiao Liu
Jiaming Song
Yu Liu
360
33
0
16 Feb 2022
Multi-Dimensional Model Compression of Vision Transformer
IEEE International Conference on Multimedia and Expo (ICME), 2021
Zejiang Hou
S. Kung
ViT
232
24
0
31 Dec 2021
A Survey on Visual Transformer
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020
Kai Han
Yunhe Wang
Hanting Chen
Xinghao Chen
Jianyuan Guo
...
Chunjing Xu
Yixing Xu
Zhaohui Yang
Yiman Zhang
Dacheng Tao
ViT
1.3K
3,405
0
23 Dec 2020
1
Page 1 of 1
