When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations

3 June 2021

Papers citing "When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations"

50 / 74 papers shown

Title
Sharpness-Aware Minimization with Z-Score Gradient Filtering for Neural Networks Juyoung Yun 38 0 0 05 May 2025
Grams: Gradient Descent with Adaptive Momentum Scaling Yang Cao Xiaoyu Li Zhao-quan Song ODL 85 2 0 22 Dec 2024
Meta Curvature-Aware Minimization for Domain Generalization Z. Chen Yiwen Ye Feilong Tang Yongsheng Pan Yong-quan Xia BDL 191 1 0 16 Dec 2024
Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late in Training Zhanpeng Zhou Mingze Wang Yuchen Mao Bingrui Li Junchi Yan AAML 59 0 0 14 Oct 2024
Analytic Convolutional Layer: A Step to Analytic Neural Network Jingmao Cui Donglai Tao Linmi Tao Ruiyang Liu Yu Cheng 24 0 0 03 Jul 2024
Improving robustness to corruptions with multiplicative weight perturbations Trung Trinh Markus Heinonen Luigi Acerbi Samuel Kaski 41 0 0 24 Jun 2024
Agnostic Sharpness-Aware Minimization Van-Anh Nguyen Quyen Tran Tuan Truong Thanh-Toan Do Dinh Q. Phung Trung Le 38 0 0 11 Jun 2024
LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate A. Fuller Daniel G. Kyrollos Yousef Yassin James R. Green 46 2 0 22 May 2024
A Hybrid Generative and Discriminative PointNet on Unordered Point Sets Yang Ye Shihao Ji PINN 3DPC 33 0 0 19 Apr 2024
Flatness Improves Backbone Generalisation in Few-shot Classification Rui Li Martin Trapp Marcus Klasson Arno Solin 43 0 0 11 Apr 2024
Dissecting Query-Key Interaction in Vision Transformers Xu Pan Aaron Philip Ziqian Xie Odelia Schwartz 32 1 0 04 Apr 2024
Approximate Nullspace Augmented Finetuning for Robust Vision Transformers Haoyang Liu Aditya Singh Yijiang Li Haohan Wang AAML ViT 33 1 0 15 Mar 2024
Text-to-Code Generation with Modality-relative Pre-training Fenia Christopoulou Guchun Zhang Gerasimos Lampouras AI4TS 18 1 0 08 Feb 2024
Momentum-SAM: Sharpness Aware Minimization without Computational Overhead Marlon Becker Frederick Altrock Benjamin Risse 74 5 0 22 Jan 2024
Critical Influence of Overparameterization on Sharpness-aware Minimization Sungbin Shin Dongyeop Lee Maksym Andriushchenko Namhoon Lee AAML 44 1 0 29 Nov 2023
Learning Human Action Recognition Representations Without Real Humans Howard Zhong Samarth Mishra Donghyun Kim SouYoung Jin Rameswar Panda Hildegard Kuehne Leonid Karlinsky Venkatesh Saligrama Aude Oliva Rogerio Feris 24 3 0 10 Nov 2023
Vision Transformers Need Registers Zilong Chen Maxime Oquab Julien Mairal Huaping Liu ViT 37 311 0 28 Sep 2023
Why Does Little Robustness Help? Understanding and Improving Adversarial Transferability from Surrogate Training Yechao Zhang Shengshan Hu Leo Yu Zhang Junyu Shi Minghui Li Xiaogeng Liu Wei Wan Hai Jin AAML 22 21 0 15 Jul 2023
A Novel Site-Agnostic Multimodal Deep Learning Model to Identify Pro-Eating Disorder Content on Social Media J. Feldman 22 0 0 06 Jul 2023
Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima Dongkuk Si Chulhee Yun 28 15 0 16 Jun 2023
DiTTO: A Feature Representation Imitation Approach for Improving Cross-Lingual Transfer Shanu Kumar Abbaraju Soujanya Sandipan Dandapat Sunayana Sitaram Monojit Choudhury VLM 25 1 0 04 Mar 2023
mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization Kayhan Behdin Qingquan Song Aman Gupta S. Keerthi Ayan Acharya Borja Ocejo Gregory Dexter Rajiv Khanna D. Durfee Rahul Mazumder AAML 13 7 0 19 Feb 2023
Mixed Hierarchy Network for Image Restoration Huiyu Gao Depeng Dang 29 14 0 19 Feb 2023
Symbolic Discovery of Optimization Algorithms Xiangning Chen Chen Liang Da Huang Esteban Real Kaiyuan Wang ... Xuanyi Dong Thang Luong Cho-Jui Hsieh Yifeng Lu Quoc V. Le 50 350 0 13 Feb 2023
A Survey on Efficient Training of Transformers Bohan Zhuang Jing Liu Zizheng Pan Haoyu He Yuetian Weng Chunhua Shen 20 47 0 02 Feb 2023
Open-Set Likelihood Maximization for Few-Shot Learning Malik Boudiaf Etienne Bennequin Myriam Tami Antoine Toubhans Pablo Piantanida C´eline Hudelot Ismail Ben Ayed BDL 26 10 0 20 Jan 2023
Stability Analysis of Sharpness-Aware Minimization Hoki Kim Jinseong Park Yujin Choi Jaewook Lee 28 12 0 16 Jan 2023
On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline Nicklas Hansen Zhecheng Yuan Yanjie Ze Tongzhou Mu Aravind Rajeswaran H. Su Huazhe Xu Xiaolong Wang 32 65 0 12 Dec 2022
The Birds Need Attention Too: Analysing usage of Self Attention in identifying bird calls in soundscapes Chandra Kanth Nagesh Abhishek Purushothama 19 2 0 14 Nov 2022
K-SAM: Sharpness-Aware Minimization at the Speed of SGD Renkun Ni Ping Yeh-Chiang Jonas Geiping Micah Goldblum A. Wilson Tom Goldstein 18 8 0 23 Oct 2022
SQuAT: Sharpness- and Quantization-Aware Training for BERT Zheng Wang Juncheng Billy Li Shuhui Qu Florian Metze Emma Strubell MQ 13 7 0 13 Oct 2022
Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach Peng Mi Li Shen Tianhe Ren Yiyi Zhou Xiaoshuai Sun Rongrong Ji Dacheng Tao AAML 27 69 0 11 Oct 2022
SynBench: Task-Agnostic Benchmarking of Pretrained Representations using Synthetic Data Ching-Yun Ko Pin-Yu Chen Jeet Mohapatra Payel Das Lucani E. Daniel 19 3 0 06 Oct 2022
Feature Embedding by Template Matching as a ResNet Block Ada Gorgun Y. Z. Gürbüz Aydin Alatan 20 1 0 03 Oct 2022
Transfer Learning with Pretrained Remote Sensing Transformers A. Fuller K. Millard J.R. Green 27 11 0 28 Sep 2022
Greybox XAI: a Neural-Symbolic learning framework to produce interpretable predictions for image classification Adrien Bennetot Gianni Franchi Javier Del Ser Raja Chatila Natalia Díaz Rodríguez AAML 25 29 0 26 Sep 2022
Colonoscopy Landmark Detection using Vision Transformers Aniruddha Tamhane Tseéla Mida Erez Posner Moshe Bouhnik ViT MedIm 54 5 0 22 Sep 2022
Design of the topology for contrastive visual-textual alignment Zhun Sun 25 1 0 05 Sep 2022
Multiple Instance Neuroimage Transformer Ayush Singla Qingyu Zhao Daniel K. Do Yuyin Zhou K. Pohl Ehsan Adeli ViT MedIm 13 11 0 19 Aug 2022
Deep is a Luxury We Don't Have Ahmed Taha Yen Nhi Truong Vu Brent Mombourquette Thomas P. Matthews Jason Su Sadanand Singh ViT MedIm 18 2 0 11 Aug 2022
Exploring the Design of Adaptation Protocols for Improved Generalization and Machine Learning Safety Puja Trivedi Danai Koutra Jayaraman J. Thiagarajan AAML 20 0 0 26 Jul 2022
PanGu-Coder: Program Synthesis with Function-Level Language Modeling Fenia Christopoulou Gerasimos Lampouras Milan Gritta Guchun Zhang Yinpeng Guo ... Guangtai Liang Jia Wei Xin Jiang Qianxiang Wang Qun Liu ELM SyDa ALM 36 74 0 22 Jul 2022
Assaying Out-Of-Distribution Generalization in Transfer Learning F. Wenzel Andrea Dittadi Peter V. Gehler Carl-Johann Simon-Gabriel Max Horn ... Chris Russell Thomas Brox Bernt Schiele Bernhard Schölkopf Francesco Locatello OOD OODD AAML 51 71 0 19 Jul 2022
Eliminating Gradient Conflict in Reference-based Line-Art Colorization Zekun Li Zhengyang Geng Zhao Kang Wenyu Chen Yibo Yang 18 35 0 13 Jul 2022
A Closer Look at Smoothness in Domain Adversarial Training Harsh Rangwani Sumukh K Aithal Mayank Mishra Arihant Jain R. Venkatesh Babu 25 119 0 16 Jun 2022
INDIGO: Intrinsic Multimodality for Domain Generalization Puneet Mangla Shivam Chandhok Milan Aggarwal V. Balasubramanian Balaji Krishnamurthy VLM 33 2 0 13 Jun 2022
Parotid Gland MRI Segmentation Based on Swin-Unet and Multimodal Images Zián Xu Yin Dai Fayu Liu Siqi Li Shengmei Liu Li-Li Shi Jun Fu ViT MedIm 16 3 0 07 Jun 2022
Which models are innately best at uncertainty estimation? Ido Galil Mohammed Dabbah Ran El-Yaniv UQCV 24 5 0 05 Jun 2022
Temporally Precise Action Spotting in Soccer Videos Using Dense Detection Anchors J. C. V. Soares Avijit Shah Topojoy Biswas 35 32 0 20 May 2022
Multimodal Transformer for Nursing Activity Recognition Momal Ijaz Renato Diaz C. L. P. Chen ViT 22 26 0 09 Apr 2022