Efficient Self-supervised Vision Transformers for Representation Learning

17 June 2021

Jianwei Yang

Lu Yuan

Papers citing "Efficient Self-supervised Vision Transformers for Representation Learning"

50 / 148 papers shown

Title
GPA-RAM: Grasp-Pretraining Augmented Robotic Attention Mamba for Spatial Task Learning Juyi Sheng Yangjun Liu Sheng Xu Zhixin Yang Mengyuan Liu 51 0 0 28 Apr 2025
Enhancing breast cancer detection on screening mammogram using self-supervised learning and a hybrid deep model of Swin Transformer and Convolutional Neural Network Han Chen Anne L. Martel 43 0 0 28 Apr 2025
Prompt-Guided Attention Head Selection for Focus-Oriented Image Retrieval Yuji Nozawa Yu Lin Kazumoto Nakamura Youyang Ng 38 0 0 02 Apr 2025
Dynamic Accumulated Attention Map for Interpreting Evolution of Decision-Making in Vision Transformer Yi Liao Yongsheng Gao Weichuan Zhang 37 1 0 18 Mar 2025
ADROIT: A Self-Supervised Framework for Learning Robust Representations for Active Learning S. Banerjee Vinay K. Verma SSL 53 0 0 10 Mar 2025
S3PT: Scene Semantics and Structure Guided Clustering to Boost Self-Supervised Pre-Training for Autonomous Driving Maciej K. Wozniak Hariprasath Govindarajan Marvin Klingner Camille Maurice B Ravi Kiran S. Yogamani 3DPC 47 1 0 30 Oct 2024
Accelerating Augmentation Invariance Pretraining Jinhong Lin Cheng-En Wu Yibing Wei Pedro Morgado ViT 23 1 0 27 Oct 2024
On Partial Prototype Collapse in the DINO Family of Self-Supervised Methods Hariprasath Govindarajan Per Sidén Jacob Roll Fredrik Lindsten 16 2 0 17 Oct 2024
Progressive Representation Learning for Real-Time UAV Tracking Changhong Fu Xiang Lei Haobo Zuo L. Yao Guangze Zheng Jia-Yu Pan AI4TS 27 4 0 25 Sep 2024
A Survey of the Self Supervised Learning Mechanisms for Vision Transformers Asifullah Khan A. Sohail M. Fiaz Mehdi Hassan Tariq Habib Afridi ... Muhammad Zaigham Zaheer Kamran Ali Tangina Sultana Ziaurrehman Tanoli Naeem Akhter 41 3 0 30 Aug 2024
A Review of Pseudo-Labeling for Computer Vision Patrick Kage Jay C. Rothenberger Pavlos Andreadis Dimitrios I. Diochnos VLM 29 3 0 13 Aug 2024
POA: Pre-training Once for Models of All Sizes Yingying Zhang Xin Guo Jiangwei Lao Lei Yu Lixiang Ru Jian Wang Guo Ye Huimei He Jingdong Chen Ming Yang 53 1 0 02 Aug 2024
EUDA: An Efficient Unsupervised Domain Adaptation via Self-Supervised Vision Transformer Ali Abedi Q. M. Jonathan Wu Ning Zhang Farhad Pourpanah 26 0 0 31 Jul 2024
Self-Supervised Learning for Text Recognition: A Critical Survey Carlos Peñarrubia J. J. Valero-Mas Jorge Calvo-Zaragoza 69 1 0 29 Jul 2024
Unsqueeze [CLS] Bottleneck to Learn Rich Representations Qing Su Shihao Ji 24 0 0 24 Jul 2024
Query-Efficient Hard-Label Black-Box Attack against Vision Transformers Chao Zhou Xiaowen Shi Yuan-Gen Wang ViT AAML 19 0 0 29 Jun 2024
A Simple Framework for Open-Vocabulary Zero-Shot Segmentation Thomas Stegmüller Tim Lebailly Nikola Dukic Behzad Bozorgtabar Tinne Tuytelaars Jean-Philippe Thiran VLM 31 1 0 23 Jun 2024
SimSAM: Simple Siamese Representations Based Semantic Affinity Matrix for Unsupervised Image Segmentation Chanda Grover Kamra Indra Deep Mastan Nitin Kumar Debayan Gupta 29 1 0 12 Jun 2024
ReduceFormer: Attention with Tensor Reduction by Summation John Yang Le An Su Inn Park 20 0 0 11 Jun 2024
Let Go of Your Labels with Unsupervised Transfer Artyom Gadetsky Yulun Jiang Maria Brbić VLM 27 5 0 11 Jun 2024
DINO as a von Mises-Fisher mixture model Hariprasath Govindarajan Per Sidén Jacob Roll Fredrik Lindsten 21 11 0 17 May 2024
An Experimental Study on Exploring Strong Lightweight Vision Transformers via Masked Image Modeling Pre-Training Jin Gao Shubo Lin Shaoru Wang Yutong Kou Zeming Li Liang Li Congxuan Zhang Xiaoqin Zhang Yizheng Wang Weiming Hu 37 1 0 18 Apr 2024
GLID: Pre-training a Generalist Encoder-Decoder Vision Model Jihao Liu Jinliang Zheng Yu Liu Hongsheng Li VLM 24 3 0 11 Apr 2024
AdaGlimpse: Active Visual Exploration with Arbitrary Glimpse Position and Scale Adam Pardyl Michal Wronka Maciej Wolczyk Kamil Adamczewski Tomasz Trzciñski Bartosz Zieliñski 31 2 0 04 Apr 2024
LoDisc: Learning Global-Local Discriminative Features for Self-Supervised Fine-Grained Visual Recognition Jialu Shi Zhiqiang Wei Jie Nie Lei Huang SSL 24 0 0 06 Mar 2024
On Good Practices for Task-Specific Distillation of Large Pretrained Visual Models Juliette Marrie Michael Arbel Julien Mairal Diane Larlus VLM MQ 30 1 0 17 Feb 2024
Cross-Level Multi-Instance Distillation for Self-Supervised Fine-Grained Visual Categorization Qi Bi Wei Ji Jingjun Yi Haolan Zhan Gui-Song Xia 16 0 0 16 Jan 2024
No More Shortcuts: Realizing the Potential of Temporal Self-Supervision I. Dave Simon Jenni Mubarak Shah 25 7 0 20 Dec 2023
The Counterattack of CNNs in Self-Supervised Learning: Larger Kernel Size might be All You Need Tianjin Huang Tianlong Chen Zhangyang Wang Shiwei Liu 22 1 0 09 Dec 2023
Meta Co-Training: Two Views are Better than One Jay C. Rothenberger Dimitrios I. Diochnos VLM 23 1 0 29 Nov 2023
Do text-free diffusion models learn discriminative visual representations? Soumik Mukhopadhyay M. Gwilliam Yosuke Yamaguchi Vatsal Agarwal Namitha Padmanabhan Archana Swaminathan Tianyi Zhou Abhinav Shrivastava DiffM 22 11 1 29 Nov 2023
SSIN: Self-Supervised Learning for Rainfall Spatial Interpolation Jia Li Yanyan Shen Lei Chen Charles Wang Wai Ng 17 3 0 27 Nov 2023
Event Camera Data Dense Pre-training Yan Yang Liyuan Pan Liu Liu 25 4 0 20 Nov 2023
Patch-Wise Self-Supervised Visual Representation Learning: A Fine-Grained Approach Ali Javidani Mohammad Amin Sadeghi Babak Nadjar Araabi 17 0 0 28 Oct 2023
Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Survey Oriane Siméoni Éloi Zablocki Spyros Gidaris Gilles Puy Patrick Pérez 24 10 0 19 Oct 2023
SegLoc: Visual Self-supervised Learning Scheme for Dense Prediction Tasks of Security Inspection X-ray Images Shervin Halat Mohammad Rahmati Ehsan Nazerfard 25 0 0 12 Oct 2023
CrIBo: Self-Supervised Learning via Cross-Image Object-Level Bootstrapping Tim Lebailly Thomas Stegmüller Behzad Bozorgtabar Jean-Philippe Thiran Tinne Tuytelaars SSL 45 6 0 11 Oct 2023
DiPS: Discriminative Pseudo-Label Sampling with Self-Supervised Transformers for Weakly Supervised Object Localization Shakeeb Murtaza Soufiane Belharbi M. Pedersoli Aydin Sarraf Eric Granger WSOL 35 9 0 09 Oct 2023
Enhancing Representations through Heterogeneous Self-Supervised Learning Zhongyu Li Bo-Wen Yin Yongxiang Liu Li Liu Ming-Ming Cheng SSL 19 2 0 08 Oct 2023
Beyond Grids: Exploring Elastic Input Sampling for Vision Transformers Adam Pardyl Grzegorz Kurzejamski Jan Olszewski Tomasz Trzciñski Bartosz Zieliñski 15 1 0 23 Sep 2023
DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learning Thanh Nguyen T. Pham Chaoning Zhang T. Luu Thang Vu Chang-Dong Yoo 17 9 0 21 Sep 2023
Attention De-sparsification Matters: Inducing Diversity in Digital Pathology Representation Learning S. Kapse Srijan Das Jingwei Zhang Rajarsi R. Gupta Joel H. Saltz Dimitris Samaras Prateek Prasanna 25 9 0 12 Sep 2023
Self-Supervised Transformer with Domain Adaptive Reconstruction for General Face Forgery Video Detection Daichi Zhang Zihao Xiao Jianmin Li Shiming Ge CVBM ViT 22 2 0 09 Sep 2023
Towards Fast and Accurate Image-Text Retrieval with Self-Supervised Fine-Grained Alignment Jiamin Zhuang Jing Yu Yang Ding Xiangyang Qu Yue Hu 13 9 0 27 Aug 2023
A Survey on Self-Supervised Representation Learning Tobias Uelwer Jan Robine Stefan Sylvius Wagner Marc Höftmann Eric Upschulte S. Konietzny Maike Behrendt Stefan Harmeling SSL AI4TS OOD 11 12 0 22 Aug 2023
Stable and Causal Inference for Discriminative Self-supervised Deep Visual Representations Yuewei Yang Hai Helen Li Yiran Chen CML OOD 17 1 0 16 Aug 2023
Revisiting Vision Transformer from the View of Path Ensemble Shuning Chang Pichao Wang Haowen Luo Fan Wang Mike Zheng Shou ViT 21 3 0 12 Aug 2023
MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Features Adrien Bardes Jean Ponce Yann LeCun MDE 26 23 0 24 Jul 2023
Diffusion Models Beat GANs on Image Classification Soumik Mukhopadhyay M. Gwilliam Vatsal Agarwal Namitha Padmanabhan A. Swaminathan Srinidhi Hegde Tianyi Zhou Abhinav Shrivastava DiffM 19 42 1 17 Jul 2023
FLSL: Feature-level Self-supervised Learning Qing Su Anton Netchaev Hai Helen Li Shihao Ji 17 4 0 09 Jun 2023