Efficient Self-supervised Vision Transformers for Representation Learning

17 June 2021

Jianwei Yang

Lu Yuan

Papers citing "Efficient Self-supervised Vision Transformers for Representation Learning"

50 / 148 papers shown

Title
Object Detection with Transformers: A Review Tahira Shehzadi K. Hashmi D. Stricker Muhammad Zeshan Afzal ViT MU 13 26 0 07 Jun 2023
Model Spider: Learning to Rank Pre-Trained Models Efficiently Yi-Kai Zhang Ting Huang Yao-Xiang Ding De-Chuan Zhan Han-Jia Ye 19 22 0 06 Jun 2023
Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training Utku Ozbulak Hyun Jung Lee Beril Boga Esla Timothy Anzaku Ho-min Park Arnout Van Messem W. D. Neve J. Vankerschaver DiffM 16 35 0 23 May 2023
Efficient Large-Scale Visual Representation Learning And Evaluation Eden Dolev A. Awad Denisa Roberts Zahra Ebrahimzadeh Marcin Mejran Vaibhav Malpani Mahir Yavuz 30 0 0 22 May 2023
Visual Tuning Bruce X. B. Yu Jianlong Chang Haixin Wang Lin Liu Shijie Wang ... Lingxi Xie Haojie Li Zhouchen Lin Qi Tian Chang Wen Chen VLM 39 37 0 10 May 2023
Detecting Novelties with Empty Classes Svenja Uhlemeyer Julian Lienen Eyke Hüllermeier Hanno Gottschalk 13 1 0 30 Apr 2023
DIAMANT: Dual Image-Attention Map Encoders For Medical Image Segmentation Yousef Yeganeh Azade Farshad Peter Weinberger Seyed-Ahmad Ahmadi Ehsan Adeli Nassir Navab ViT MedIm 15 0 0 28 Apr 2023
Hint-Aug: Drawing Hints from Foundation Vision Transformers Towards Boosted Few-Shot Parameter-Efficient Tuning Zhongzhi Yu Shang Wu Y. Fu Shunyao Zhang Yingyan Lin 23 6 0 25 Apr 2023
A Cookbook of Self-Supervised Learning Randall Balestriero Mark Ibrahim Vlad Sobal Ari S. Morcos Shashank Shekhar ... Pierre Fernandez Amir Bar Hamed Pirsiavash Yann LeCun Micah Goldblum SyDa FedML SSL 31 270 0 24 Apr 2023
DINOv2: Learning Robust Visual Features without Supervision Maxime Oquab Timothée Darcet Théo Moutakanni Huy Q. Vo Marc Szafraniec ... Hervé Jégou Julien Mairal Patrick Labatut Armand Joulin Piotr Bojanowski VLM CLIP SSL 32 2,983 0 14 Apr 2023
EMP-SSL: Towards Self-Supervised Learning in One Training Epoch Shengbang Tong Yubei Chen Y. Ma Yann LeCun 13 24 0 08 Apr 2023
Rethinking Evaluation Protocols of Visual Representations Learned via Self-supervised Learning Jaehoon Lee Doyoung Yoon Byeongmoon Ji Kyungyul Kim Sangheum Hwang SSL 55 3 0 07 Apr 2023
Adaptive Similarity Bootstrapping for Self-Distillation based Representation Learning Tim Lebailly Thomas Stegmüller Behzad Bozorgtabar Jean-Philippe Thiran Tinne Tuytelaars SSL 14 2 0 23 Mar 2023
CrOC: Cross-View Online Clustering for Dense Visual Representation Learning Thomas Stegmüller Tim Lebailly Behzad Bozorgtabar Tinne Tuytelaars Jean-Philippe Thiran 24 15 0 23 Mar 2023
GeoMIM: Towards Better 3D Knowledge Transfer via Masked Image Modeling for Multi-view 3D Understanding Jihao Liu Tai Wang Boxiao Liu Qihang Zhang Yu Liu Hongsheng Li 25 16 0 20 Mar 2023
Coreset Sampling from Open-Set for Fine-Grained Self-Supervised Learning Sungnyun Kim Sangmin Bae Se-Young Yun 11 9 0 20 Mar 2023
Making Vision Transformers Efficient from A Token Sparsification View Shuning Chang Pichao Wang Ming Lin Fan Wang David Junhao Zhang Rong Jin Mike Zheng Shou ViT 43 23 0 15 Mar 2023
Self-supervised learning of Split Invariant Equivariant representations Q. Garrido Laurent Najman Yann LeCun SSL 16 32 0 14 Feb 2023
A Closer Look at Few-shot Classification Again Xu Luo Hao Wu Ji Zhang Lianli Gao Jing Xu Jingkuan Song 22 48 0 28 Jan 2023
A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends Jie Gui Tuo Chen Jing Zhang Qiong Cao Zhe Sun Haoran Luo Dacheng Tao 24 117 0 13 Jan 2023
Disjoint Masking with Joint Distillation for Efficient Masked Image Modeling Xin Ma Chang-Shu Liu Chunyu Xie Long Ye Yafeng Deng Xiang Ji 18 8 0 31 Dec 2022
DQnet: Cross-Model Detail Querying for Camouflaged Object Detection Wei Sun Chengao Liu Linyan Zhang Yu Li Pengxu Wei Chang-rui Liu J. Zou Jianbin Jiao QiXiang Ye 35 6 0 16 Dec 2022
FlexiViT: One Model for All Patch Sizes Lucas Beyer Pavel Izmailov Alexander Kolesnikov Mathilde Caron Simon Kornblith Xiaohua Zhai Matthias Minderer Michael Tschannen Ibrahim M. Alabdulmohsin Filip Pavetić VLM 28 89 0 15 Dec 2022
Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language Alexei Baevski Arun Babu Wei-Ning Hsu Michael Auli VLM SSL 27 90 0 14 Dec 2022
Benchmarking Self-Supervised Learning on Diverse Pathology Datasets Mingu Kang Heon Song Seonwook Park Donggeun Yoo Sérgio Pereira 11 124 0 09 Dec 2022
A Unified Framework for Contrastive Learning from a Perspective of Affinity Matrix Wenbin Li Meihao Kong Xuesong Yang Lei Wang Jing Huo Yang Gao Jiebo Luo 17 0 0 26 Nov 2022
Could Giant Pretrained Image Models Extract Universal Representations? Yutong Lin Ze Liu Zheng-Wei Zhang Han Hu Nanning Zheng Stephen Lin Yue Cao VLM 38 9 0 03 Nov 2022
Max Pooling with Vision Transformers reconciles class and shape in weakly supervised semantic segmentation Simone Rossetti Damiano Zappia Marta Sanzari M. Schaerf F. Pirri ViT 28 57 0 31 Oct 2022
Foreground Guidance and Multi-Layer Feature Fusion for Unsupervised Object Discovery with Transformers Zhiwei Lin Ze Yang Yongtao Wang ViT 10 2 0 24 Oct 2022
Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets Xiangyu Chen Qinghao Hu Kaidong Li Cuncong Zhong Guanghui Wang ViT 33 11 0 22 Oct 2022
SSiT: Saliency-guided Self-supervised Image Transformer for Diabetic Retinopathy Grading Yijin Huang Junyan Lyu Pujin Cheng Roger Tam Xiaoying Tang ViT MedIm 11 19 0 20 Oct 2022
LAVA: Label-efficient Visual Learning and Adaptation Islam Nassar Munawar Hayat Ehsan Abbasnejad Hamid Rezatofighi Mehrtash Harandi Gholamreza Haffari VLM 8 1 0 19 Oct 2022
Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformers Tao Tang Changlin Li Guangrun Wang Kaicheng Yu Xiaojun Chang Xiaodan Liang ViT 13 1 0 16 Oct 2022
How to Train Vision Transformer on Small-scale Datasets? Hanan Gani Muzammal Naseer Mohammad Yaqub ViT 12 49 0 13 Oct 2022
Exploring Long-Sequence Masked Autoencoders Ronghang Hu Shoubhik Debnath Saining Xie Xinlei Chen 8 17 0 13 Oct 2022
RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank Q. Garrido Randall Balestriero Laurent Najman Yann LeCun SSL 46 71 0 05 Oct 2022
VICRegL: Self-Supervised Learning of Local Visual Features Adrien Bardes Jean Ponce Yann LeCun SSL 17 118 0 04 Oct 2022
Attention Distillation: self-supervised vision transformer students need more guidance Kai Wang Fei Yang Joost van de Weijer ViT 17 14 0 03 Oct 2022
Towards a Unified View on Visual Parameter-Efficient Transfer Learning Bruce X. B. Yu Jianlong Chang Lin Liu Qi Tian Changan Chen VPVLM VLM 68 33 0 03 Oct 2022
Mixed-domain Training Improves Multi-Mission Terrain Segmentation Grace Vincent Alice Yepremyan Jingdao Chen Edwin Y. Goh 14 2 0 27 Sep 2022
Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representations Vadim Tschernezki Iro Laina Diane Larlus Andrea Vedaldi 170 184 0 07 Sep 2022
Global-Local Self-Distillation for Visual Representation Learning Tim Lebailly Tinne Tuytelaars SSL 19 6 0 29 Jul 2022
Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustness Chaoning Zhang Kang Zhang Chenshuang Zhang Axi Niu Jiu Feng Chang-Dong Yoo In So Kweon SSL 28 24 0 22 Jul 2022
Unsupervised Visual Representation Learning by Synchronous Momentum Grouping Bo Pang Yifan Zhang Yaoyi Li Jia Cai Cewu Lu SSL 12 27 0 13 Jul 2022
Vision Transformers: State of the Art and Research Challenges Bo-Kai Ruan Hong-Han Shuai Wen-Huang Cheng ViT 11 17 0 07 Jul 2022
MaiT: Leverage Attention Masks for More Efficient Image Transformers Ling Li Ali Shafiee Ardestani Joseph Hassoun 6 1 0 06 Jul 2022
Guillotine Regularization: Why removing layers is needed to improve generalization in Self-Supervised Learning Florian Bordes Randall Balestriero Q. Garrido Adrien Bardes Pascal Vincent 14 22 0 27 Jun 2022
Self-supervised Learning in Remote Sensing: A Review Yi Wang C. Albrecht Nassim Ait Ali Braham Lichao Mou Xiao Xiang Zhu 12 214 0 27 Jun 2022
Beyond Supervised vs. Unsupervised: Representative Benchmarking and Analysis of Image Representation Learning M. Gwilliam Abhinav Shrivastava SSL 71 19 1 16 Jun 2022
Evaluating Self-Supervised Learning for Molecular Graph Embeddings Hanchen Wang Jean Kaddour Shengchao Liu Jian Tang Joan Lasenby Qi Liu 17 20 0 16 Jun 2022