An Empirical Study of Training Self-Supervised Vision Transformers

5 April 2021

Papers citing "An Empirical Study of Training Self-Supervised Vision Transformers"

50 / 389 papers shown

Title
Stable Cluster Discrimination for Deep Clustering Qi Qian OOD 19 21 0 24 Nov 2023
Event Camera Data Dense Pre-training Yan Yang Liyuan Pan Liu Liu 27 4 0 20 Nov 2023
What Makes Pre-Trained Visual Representations Successful for Robust Manipulation? Kaylee Burns Zach Witzel Jubayer Ibn Hamid Tianhe Yu Chelsea Finn Karol Hausman OOD SSL 27 22 0 03 Nov 2023
COSTAR: Improved Temporal Counterfactual Estimation with Self-Supervised Learning Chuizheng Meng Yihe Dong Sercan Ö. Arik Yan Liu Tomas Pfister CML AI4TS 21 0 0 01 Nov 2023
Causal Unsupervised Semantic Segmentation Junho Kim Byung-Kwan Lee Yonghyun Ro 33 18 0 11 Oct 2023
Transferable Availability Poisoning Attacks Yiyong Liu Michael Backes Xiao Zhang AAML 19 3 0 08 Oct 2023
Small-scale proxies for large-scale Transformer training instabilities Mitchell Wortsman Peter J. Liu Lechao Xiao Katie Everett A. Alemi ... Jascha Narain Sohl-Dickstein Kelvin Xu Jaehoon Lee Justin Gilmer Simon Kornblith 35 81 0 25 Sep 2023
SCOB: Universal Text Understanding via Character-wise Supervised Contrastive Learning with Online Text Rendering for Bridging Domain Gap Daehee Kim Yoon Kim Donghyun Kim Yumin Lim Geewook Kim Taeho Kil 23 3 0 21 Sep 2023
DimCL: Dimensional Contrastive Learning For Improving Self-Supervised Learning Thanh Nguyen T. Pham Chaoning Zhang T. Luu Thang Vu Chang-Dong Yoo 23 9 0 21 Sep 2023
SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels Henry Hengyuan Zhao Pichao Wang Yuyang Zhao Hao Luo F. Wang Mike Zheng Shou ViT 31 14 0 15 Sep 2023
Representation Synthesis by Probabilistic Many-Valued Logic Operation in Self-Supervised Learning Hiroki Nakamura Masashi Okada T. Taniguchi SSL NAI 23 0 0 08 Sep 2023
Efficient Model Personalization in Federated Learning via Client-Specific Prompt Generation Fu-En Yang Chien-Yi Wang Yu-Chiang Frank Wang VLM FedML 21 59 0 29 Aug 2023
Masked Feature Modelling: Feature Masking for the Unsupervised Pre-training of a Graph Attention Network Block for Bottom-up Video Event Recognition Dimitrios Daskalakis Nikolaos Gkalelis Vasileios Mezaris 34 0 0 24 Aug 2023
Masked Momentum Contrastive Learning for Zero-shot Semantic Understanding Jiantao Wu Shentong Mo Muhammad Awais Sara Atito Zhenhua Feng J. Kittler VLM 27 4 0 22 Aug 2023
Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers N. Frumkin Dibakar Gope Diana Marculescu MQ 33 16 0 21 Aug 2023
CoNe: Contrast Your Neighbours for Supervised Image Classification Mingkai Zheng Shan You Lang Huang Xiu Su Fei Wang Chao Qian Xiaogang Wang Chang Xu VLM 20 0 0 21 Aug 2023
SSTFormer: Bridging Spiking Neural Network and Memory Support Transformer for Frame-Event based Recognition Xiao Wang Zong-Yao Wu Yao Rong Lin Zhu Bowei Jiang Jin Tang Yonghong Tian ViT 69 15 0 08 Aug 2023
Exploring Visual Pre-training for Robot Manipulation: Datasets, Models and Methods Ya Jing Xuelin Zhu Xingbin Liu Qie Sima Taozheng Yang Yunhai Feng Tao Kong LM&Ro 40 16 0 07 Aug 2023
Stochastic positional embeddings improve masked image modeling Amir Bar Florian Bordes Assaf Shocher Mahmoud Assran Pascal Vincent Nicolas Ballas Trevor Darrell Amir Globerson Yann LeCun 14 3 0 31 Jul 2023
MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Features Adrien Bardes Jean Ponce Yann LeCun MDE 33 23 0 24 Jul 2023
PRIOR: Prototype Representation Joint Learning from Medical Images and Reports Pujin Cheng Li Lin Junyan Lyu Yijin Huang Wenhan Luo Xiaoying Tang MedIm 29 45 0 24 Jul 2023
Why Does Little Robustness Help? Understanding and Improving Adversarial Transferability from Surrogate Training Yechao Zhang Shengshan Hu Leo Yu Zhang Junyu Shi Minghui Li Xiaogeng Liu Wei Wan Hai Jin AAML 22 21 0 15 Jul 2023
Inter-Instance Similarity Modeling for Contrastive Learning Chen Shen Dawei Liu H. Tang Zhe Qu Jianxin Wang SSL 26 4 0 21 Jun 2023
Understanding Contrastive Learning Through the Lens of Margins Daniel Rho Taesoo Kim Sooill Park Jaehyun Park Jaehan Park SSL 27 2 0 20 Jun 2023
Learning to Mask and Permute Visual Tokens for Vision Transformer Pre-Training Lorenzo Baraldi Roberto Amoroso Marcella Cornia Lorenzo Baraldi Andrea Pilzer Rita Cucchiara 38 2 0 12 Jun 2023
Joint Channel Estimation and Feedback with Masked Token Transformers in Massive MIMO Systems Mingming Zhao Lin Liu Lifu Liu Mengke Li Qing-min Tian 6 0 0 08 Jun 2023
Context-Aware Self-Supervised Learning of Whole Slide Images Milan Aryal Nasim Yahya Soltani 14 5 0 07 Jun 2023
VIPriors 3: Visual Inductive Priors for Data-Efficient Deep Learning Challenges Robert-Jan Bruintjes A. Lengyel Marcos Baptista-Rios O. Kayhan Davide Zambrano Nergis Tomen J. C. V. Gemert 18 9 0 31 May 2023
Investigating how ReLU-networks encode symmetries Georg Bökman Fredrik Kahl 24 6 0 26 May 2023
Image as First-Order Norm+Linear Autoregression: Unveiling Mathematical Invariance Yinpeng Chen Xiyang Dai Dongdong Chen Mengchen Liu Lu Yuan Zicheng Liu Youzuo Lin 26 2 0 25 May 2023
Medical supervised masked autoencoders: Crafting a better masking strategy and efficient fine-tuning schedule for medical image classification Jia-ju Mao Shu-Hua Guo Yuan Chang Xuesong Yin Binling Nie 26 2 0 10 May 2023
LEMaRT: Label-Efficient Masked Region Transform for Image Harmonization Sheng Liu C. P. Huynh Congmin Chen Maxim Arap Raffay Hamid 25 18 0 25 Apr 2023
Self-Supervised Multi-Object Tracking For Autonomous Driving From Consistency Across Timescales Christopher Lang Alexander Braun Lars Schillingmann Abhinav Valada 15 5 0 25 Apr 2023
Self-supervised Learning by View Synthesis Shaoteng Liu Xiangyu Zhang T. Hu Jiaya Jia 3DV ViT 40 1 0 22 Apr 2023
DINOv2: Learning Robust Visual Features without Supervision Maxime Oquab Timothée Darcet Théo Moutakanni Huy Q. Vo Marc Szafraniec ... Hervé Jégou Julien Mairal Patrick Labatut Armand Joulin Piotr Bojanowski VLM CLIP SSL 101 3,017 0 14 Apr 2023
How Will It Drape Like? Capturing Fabric Mechanics from Depth Images Carlos Rodriguez-Pardo Melania Prieto-Martin Dan Casas Elena Garces 28 12 0 13 Apr 2023
Leveraging Neural Representations for Audio Manipulation Scott H. Hawley C. Steinmetz 25 2 0 10 Apr 2023
Token Boosting for Robust Self-Supervised Visual Transformer Pre-training Tianjiao Li Lin Geng Foo Ping Hu Xindi Shang Hossein Rahmani Zehuan Yuan J. Liu 34 7 0 09 Apr 2023
Efficient OCR for Building a Diverse Digital History Jacob Carlson Tom Bryan Melissa Dell 23 11 0 05 Apr 2023
Revisiting the Evaluation of Image Synthesis with GANs Mengping Yang Ceyuan Yang Yichi Zhang Qingyan Bai Yujun Shen Bo Dai EGVM 27 7 0 04 Apr 2023
Defending Against Patch-based Backdoor Attacks on Self-Supervised Learning Ajinkya Tejankar Maziar Sanjabi Qifan Wang Sinong Wang Hamed Firooz Hamed Pirsiavash L Tan AAML 28 19 0 04 Apr 2023
Masked Autoencoders as Image Processors Huiyu Duan Wei Shen Xiongkuo Min Danyang Tu Long Teng Jia Wang Guangtao Zhai ViT 22 11 0 30 Mar 2023
Soft Neighbors are Positive Supporters in Contrastive Visual Representation Learning Chongjian Ge Jiangliu Wang Zhan Tong Shoufa Chen Yibing Song Ping Luo SSL 22 27 0 30 Mar 2023
Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning Peng Jin Jinfa Huang Pengfei Xiong Shangxuan Tian Chang-rui Liu Xiang Ji Li-ming Yuan Jie Chen 30 49 0 25 Mar 2023
AdPE: Adversarial Positional Embeddings for Pretraining Vision Transformers via MAE+ Xiao Wang Ying Wang Ziwei Xuan Guo-Jun Qi ViT 42 3 0 14 Mar 2023
Align and Attend: Multimodal Summarization with Dual Contrastive Losses Bo He Jun Wang Jielin Qiu Trung Bui Abhinav Shrivastava Zhaowen Wang 22 65 0 13 Mar 2023
Interventional Bag Multi-Instance Learning On Whole-Slide Pathological Images Tiancheng Lin Zhimiao Yu Hongyu Hu Yi Xu Changyi Chen 52 77 0 13 Mar 2023
TQ-Net: Mixed Contrastive Representation Learning For Heterogeneous Test Questions He Zhu Xihua Li Xuemin Zhao Yunbo Cao Shan Yu 15 0 0 09 Mar 2023
Rethinking Visual Prompt Learning as Masked Visual Token Modeling Ning Liao Bowen Shi Xiaopeng Zhang Min Cao Junchi Yan Qi Tian VLM 34 7 0 09 Mar 2023
Learning Discriminative Representations for Skeleton Based Action Recognition Huanyu Zhou Qingjie Liu Yunhong Wang 21 70 0 07 Mar 2023