An Empirical Study of Training Self-Supervised Vision Transformers

5 April 2021

Papers citing "An Empirical Study of Training Self-Supervised Vision Transformers"

50 / 389 papers shown

Title
Exploring Cross-Domain Pretrained Model for Hyperspectral Image Classification Hyungtae Lee Sungmin Eum H. Kwon 19 22 0 07 Apr 2022
MultiMAE: Multi-modal Multi-task Masked Autoencoders Roman Bachmann David Mizrahi Andrei Atanov Amir Zamir 32 265 0 04 Apr 2022
BatchFormerV2: Exploring Sample Relationships for Dense Representation Learning Zhi Hou Baosheng Yu Chaoyue Wang Yibing Zhan Dacheng Tao ViT 15 11 0 04 Apr 2022
Improving Vision Transformers by Revisiting High-frequency Components Jiawang Bai Liuliang Yuan Shutao Xia Shuicheng Yan Zhifeng Li W. Liu ViT 8 90 0 03 Apr 2022
POS-BERT: Point Cloud One-Stage BERT Pre-Training Kexue Fu Peng Gao Shaolei Liu Renrui Zhang Yu Qiao Manning Wang 3DPC 22 18 0 03 Apr 2022
On the Importance of Asymmetry for Siamese Representation Learning Xiao Wang Haoqi Fan Yuandong Tian Daisuke Kihara Xinlei Chen SSL 22 51 0 01 Apr 2022
Self-distillation Augmented Masked Autoencoders for Histopathological Image Classification Yang Luo Zhineng Chen Shengtian Zhou Xieping Gao 25 1 0 31 Mar 2022
mc-BEiT: Multi-choice Discretization for Image BERT Pre-training Xiaotong Li Yixiao Ge Kun Yi Zixuan Hu Ying Shan Ling-yu Duan 34 38 0 29 Mar 2022
Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap Yifei Wang Qi Zhang Yisen Wang Jiansheng Yang Zhouchen Lin 19 98 0 25 Mar 2022
Unsupervised Salient Object Detection with Spectral Cluster Voting Gyungin Shin Samuel Albanie Weidi Xie 13 65 0 23 Mar 2022
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training Zhan Tong Yibing Song Jue Wang Limin Wang ViT 125 1,122 0 23 Mar 2022
Domain Generalization by Mutual-Information Regularization with Pre-trained Models Junbum Cha Kyungjae Lee Sungrae Park Sanghyuk Chun OOD 26 131 0 21 Mar 2022
Object discovery and representation networks Olivier J. Hénaff Skanda Koppula Evan Shelhamer Daniel Zoran Andrew Jaegle Andrew Zisserman João Carreira Relja Arandjelović 38 87 0 16 Mar 2022
P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimation Wenkang Shan Zhenhua Liu Xinfeng Zhang Shanshe Wang Siwei Ma Wen Gao 3DH 23 121 0 15 Mar 2022
RecursiveMix: Mixed Learning with History Lingfeng Yang Xiang Li Borui Zhao Renjie Song Jian Yang VLM 27 18 0 14 Mar 2022
Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs Xiaohan Ding X. Zhang Yi Zhou Jungong Han Guiguang Ding Jian-jun Sun VLM 47 528 0 13 Mar 2022
Backbone is All Your Need: A Simplified Architecture for Visual Object Tracking Boyu Chen Peixia Li Lei Bai Leixian Qiao Qiuhong Shen Bo-wen Li Weihao Gan Wei Wu Wanli Ouyang ViT VOT 20 182 0 10 Mar 2022
MVP: Multimodality-guided Visual Pre-training Longhui Wei Lingxi Xie Wen-gang Zhou Houqiang Li Qi Tian 28 105 0 10 Mar 2022
Multiscale Convolutional Transformer with Center Mask Pretraining for Hyperspectral Image Classification Sen Jia Yifan Wang ViT 35 13 0 09 Mar 2022
Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels Yuchao Wang Haochen Wang Yujun Shen Jingjing Fei Wei Li Guoqiang Jin Liwei Wu Rui Zhao Xinyi Le UQCV 15 331 0 08 Mar 2022
DiT: Self-supervised Pre-training for Document Image Transformer Junlong Li Yiheng Xu Tengchao Lv Lei Cui Chaoxi Zhang Furu Wei ViT VLM 35 159 0 04 Mar 2022
A study on the distribution of social biases in self-supervised learning visual models Kirill Sirotkin Pablo Carballeira Marcos Escudero-Viñolo 22 18 0 03 Mar 2022
Audio Self-supervised Learning: A Survey Shuo Liu Adria Mallol-Ragolta Emilia Parada-Cabeleiro Kun Qian Xingshuo Jing Alexander Kathan Bin Hu Bjoern W. Schuller SSL 29 106 0 02 Mar 2022
Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance Zhuoning Yuan Yuexin Wu Zi-qi Qiu Xianzhi Du Lijun Zhang Denny Zhou Tianbao Yang 32 26 0 24 Feb 2022
Indiscriminate Poisoning Attacks on Unsupervised Contrastive Learning Hao He Kaiwen Zha Dina Katabi AAML 26 32 0 22 Feb 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz X. Wang ViT VLM 189 499 0 22 Feb 2022
Vision-Language Pre-Training with Triple Contrastive Learning Jinyu Yang Jiali Duan Son N. Tran Yi Xu Sampath Chanda Liqun Chen Belinda Zeng Trishul M. Chilimbi Junzhou Huang VLM 29 288 0 21 Feb 2022
Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision Priya Goyal Quentin Duval Isaac Seessel Mathilde Caron Ishan Misra Levent Sagun Armand Joulin Piotr Bojanowski VLM SSL 26 110 0 16 Feb 2022
Meta Knowledge Distillation Jihao Liu Boxiao Liu Hongsheng Li Yu Liu 18 25 0 16 Feb 2022
Distillation with Contrast is All You Need for Self-Supervised Point Cloud Representation Learning Kexue Fu Peng Gao Renrui Zhang Hongsheng Li Yu Qiao Manning Wang SSL 3DPC 20 23 0 09 Feb 2022
Self-supervised Contrastive Learning for Cross-domain Hyperspectral Image Representation Hyungtae Lee H. Kwon SSL 6 17 0 08 Feb 2022
How to Understand Masked Autoencoders Shuhao Cao Peng-Tao Xu David A. Clifton 21 40 0 08 Feb 2022
Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection A. Haliassos Rodrigo Mira Stavros Petridis M. Pantic CVBM 27 125 0 18 Jan 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 28 101 0 15 Jan 2022
Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet? Nenad Tomašev Ioana Bica Brian McWilliams Lars Buesing Razvan Pascanu Charles Blundell Jovana Mitrović SSL 74 80 0 13 Jan 2022
BigDatasetGAN: Synthesizing ImageNet with Pixel-wise Annotations Daiqing Li Huan Ling Seung Wook Kim Karsten Kreis Adela Barriuso Sanja Fidler Antonio Torralba 23 103 0 12 Jan 2022
Robust Contrastive Learning against Noisy Views Ching-Yao Chuang R. Devon Hjelm Xin Eric Wang Vibhav Vineet Neel Joshi Antonio Torralba Stefanie Jegelka Ya-heng Song NoLa 13 68 0 12 Jan 2022
SLIP: Self-supervision meets Language-Image Pre-training Norman Mu Alexander Kirillov David A. Wagner Saining Xie VLM CLIP 42 476 0 23 Dec 2021
Meta-Learning and Self-Supervised Pretraining for Real World Image Translation Ileana Rugina Rumen Dangovski Mark S. Veillette Pooya Khorrami Brian Cheung Olga Simek M. Soljavcić VLM SSL 20 2 0 22 Dec 2021
Are Large-scale Datasets Necessary for Self-Supervised Pre-training? Alaaeldin El-Nouby Gautier Izacard Hugo Touvron Ivan Laptev Hervé Jégou Edouard Grave SSL 15 148 0 20 Dec 2021
Masked Feature Prediction for Self-Supervised Visual Pre-Training Chen Wei Haoqi Fan Saining Xie Chaoxia Wu Alan Yuille Christoph Feichtenhofer ViT 59 655 0 16 Dec 2021
Towards General and Efficient Active Learning Yichen Xie M. Tomizuka Wei Zhan VLM 30 10 0 15 Dec 2021
Self-Supervised Modality-Aware Multiple Granularity Pre-Training for RGB-Infrared Person Re-Identification Lin Wan Qianyan Jing Zongyuan Sun Chuan Zhang Zhihang Li Yehansen Chen SSL 14 5 0 12 Dec 2021
BEVT: BERT Pretraining of Video Transformers Rui Wang Dongdong Chen Zuxuan Wu Yinpeng Chen Xiyang Dai Mengchen Liu Yu-Gang Jiang Luowei Zhou Lu Yuan ViT 30 203 0 02 Dec 2021
Self-supervised Video Transformer Kanchana Ranasinghe Muzammal Naseer Salman Khan F. Khan Michael S. Ryoo ViT 26 84 0 02 Dec 2021
Boosting Discriminative Visual Representation Learning with Scenario-Agnostic Mixup Siyuan Li Zicheng Liu Zedong Wang Di Wu Zihan Liu Stan Z. Li 27 26 0 30 Nov 2021
MC-SSL0.0: Towards Multi-Concept Self-Supervised Learning Sara Atito Muhammad Awais Ammarah Farooq Zhenhua Feng J. Kittler 17 17 0 30 Nov 2021
Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis Yucheng Tang Dong Yang Wenqi Li H. Roth Bennett Landman Daguang Xu V. Nath Ali Hatamizadeh ViT MedIm 22 517 0 29 Nov 2021
SWAT: Spatial Structure Within and Among Tokens Kumara Kahatapitiya Michael S. Ryoo 23 6 0 26 Nov 2021
Contrastive Object-level Pre-training with Spatial Noise Curriculum Learning Chenhongyi Yang Lichao Huang Elliot J. Crowley SSL VLM 21 6 0 26 Nov 2021