Supervised Masked Knowledge Distillation for Few-Shot Transformers

25 March 2023

Papers citing "Supervised Masked Knowledge Distillation for Few-Shot Transformers"

25 / 25 papers shown

Title
Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification Lanyun Zhu T. Chen Deyi Ji Jieping Ye J. Liu 39 2 0 28 Jan 2025
V"Mean"ba: Visual State Space Models only need 1 hidden dimension Tien-Yu Chi Hung-Yueh Chiang Chi-Chih Chang N. Huang Kai-Chiang Wu 83 0 0 21 Dec 2024
Multi-Level Feature Distillation of Joint Teachers Trained on Distinct Image Datasets Adrian Iordache B. Alexe Radu Tudor Ionescu 29 1 0 29 Oct 2024
Masked Image Modeling: A Survey Vlad Hondru Florinel-Alin Croitoru Shervin Minaee Radu Tudor Ionescu N. Sebe 64 6 0 13 Aug 2024
CKD: Contrastive Knowledge Distillation from A Sample-wise Perspective Wencheng Zhu Xin Zhou Pengfei Zhu Yu Wang Qinghua Hu VLM 56 1 0 22 Apr 2024
Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers Diana-Nicoleta Grigore Mariana-Iuliana Georgescu J. A. Justo T. Johansen Andreea-Iuliana Ionescu Radu Tudor Ionescu 26 0 0 14 Apr 2024
Controllable Relation Disentanglement for Few-Shot Class-Incremental Learning Yuanen Zhou Richang Hong Yanrong Guo Lin Liu Shijie Hao Hanwang Zhang 18 0 0 17 Mar 2024
Task Attribute Distance for Few-Shot Learning: Theoretical Analysis and Applications Minyang Hu Hong Chang Zong Guo Bingpeng Ma Shiguang Shan Xilin Chen VLM 21 1 0 06 Mar 2024
GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creation Ayan Banerjee Sanket Biswas Josep Lladós Umapada Pal 38 1 0 17 Feb 2024
Exploring Hyperspectral Anomaly Detection with Human Vision: A Small Target Aware Detector Jitao Ma Weiying Xie Yunsong Li 28 1 0 02 Jan 2024
Unsupervised Video Domain Adaptation with Masked Pre-Training and Collaborative Self-Training Arun V. Reddy William Paul Corban Rivera Ketul Shah Celso M. de Melo Rama Chellappa 37 4 0 05 Dec 2023
Mitigating the Effect of Incidental Correlations on Part-based Learning Gaurav Bhatt Deepayan Das Leonid Sigal Vineeth N. Balasubramanian 16 4 0 30 Sep 2023
A survey on efficient vision transformers: algorithms, techniques, and performance benchmarking Lorenzo Papa Paolo Russo Irene Amerini Luping Zhou 25 41 0 05 Sep 2023
Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work Qiangchang Wang Yilong Yin 23 0 0 02 Jun 2023
Epistemic Graph: A Plug-And-Play Module For Hybrid Representation Learning Jin Yuan Yang Zhang Yangzhou Du Zhongchao Shi Xin Geng Jianping Fan Yong Rui 14 0 0 30 May 2023
DiGeo: Discriminative Geometry-Aware Learning for Generalized Few-Shot Object Detection Jiawei Ma Yulei Niu Jincheng Xu Shiyuan Huang G. Han Shih-Fu Chang ObjD 19 36 0 16 Mar 2023
Knowledge Distillation in Vision Transformers: A Critical Review Gousia Habib Tausifa Jan Saleem Brejesh Lall 16 15 0 04 Feb 2023
A Unified View of Masked Image Modeling Zhiliang Peng Li Dong Hangbo Bao QiXiang Ye Furu Wei VLM 52 35 0 19 Oct 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,412 0 11 Nov 2021
Rethinking Supervised Pre-training for Better Downstream Transferring Yutong Feng Jianwen Jiang Mingqian Tang R. L. Jin Yue Gao SSL 43 39 0 12 Oct 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 245 558 0 28 Sep 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 303 5,761 0 29 Apr 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,622 0 24 Feb 2021
Free Lunch for Few-shot Learning: Distribution Calibration Shuo Yang Lu Liu Min Xu OODD 208 322 0 16 Jan 2021
CrossTransformers: spatially-aware few-shot transfer Carl Doersch Ankush Gupta Andrew Zisserman ViT 201 330 0 22 Jul 2020