Better plain ViT baselines for ImageNet-1k

3 May 2022

Papers citing "Better plain ViT baselines for ImageNet-1k"

50 / 68 papers shown

Title
Benchmarking Feature Upsampling Methods for Vision Foundation Models using Interactive Segmentation Volodymyr Havrylov Haiwen Huang Dan Zhang Andreas Geiger 72 0 0 04 May 2025
A Model Zoo of Vision Transformers Damian Falk Léo Meynent Florence Pfammatter Konstantin Schurholt Damian Borth 32 0 0 14 Apr 2025
EDIT: Enhancing Vision Transformers by Mitigating Attention Sink through an Encoder-Decoder Architecture Wenfeng Feng Guoying Sun 26 0 0 09 Apr 2025
SapiensID: Foundation for Human Recognition Minchul Kim Dingqiang Ye Yiyang Su Feng Liu Xiaoming Liu CVBM VLM 44 0 0 07 Apr 2025
DeBackdoor: A Deductive Framework for Detecting Backdoor Attacks on Deep Models with Limited Data Dorde Popovic Amin Sadeghi Ting Yu Sanjay Chawla Issa M. Khalil AAML 46 0 0 27 Mar 2025
Towards Scalable Modeling of Compressed Videos for Efficient Action Recognition Shristi Das Biswas Efstathia Soufleri Arani Roy Kaushik Roy 54 0 0 17 Mar 2025
Masked Sensory-Temporal Attention for Sensor Generalization in Quadruped Locomotion Dikai Liu Tianwei Zhang Jianxiong Yin Simon See 85 1 0 13 Mar 2025
SASSHA: Sharpness-aware Adaptive Second-order Optimization with Stable Hessian Approximation Dahun Shin Dongyeop Lee Jinseok Chung Namhoon Lee ODL AAML 132 0 0 25 Feb 2025
Myna: Masking-Based Contrastive Learning of Musical Representations Ori Yonay Tracy Hammond Tianbao Yang AAML 51 0 0 20 Feb 2025
Spectral-factorized Positive-definite Curvature Learning for NN Training Wu Lin Felix Dangel Runa Eschenhagen Juhan Bae Richard E. Turner Roger B. Grosse 45 0 0 10 Feb 2025
No More Adam: Learning Rate Scaling at Initialization is All You Need Minghao Xu Lichuan Xiang Xu Cai Hongkai Wen 73 2 0 16 Dec 2024
Enhancing Action Recognition by Leveraging the Hierarchical Structure of Actions and Textual Context Manuel Benavent-Lledo David Mulero-Pérez David Ortiz-Perez José García Rodríguez Antonis Argyros 24 0 0 28 Oct 2024
Understanding Adam Requires Better Rotation Dependent Assumptions Lucas Maes Tianyue H. Zhang Alexia Jolicoeur-Martineau Ioannis Mitliagkas Damien Scieur Simon Lacoste-Julien Charles Guille-Escuret 30 2 0 25 Oct 2024
MatMamba: A Matryoshka State Space Model Abhinav Shukla Sai H. Vemprala Aditya Kusupati Ashish Kapoor Mamba 28 0 0 09 Oct 2024
Snuffy: Efficient Whole Slide Image Classifier Hossein Jafarinia Alireza Alipanah Danial Hamdi Saeed Razavi Nahal Mirzaie M. Rohban 3DH 38 1 0 15 Aug 2024
Adaptive Parametric Activation Konstantinos Panagiotis Alexandridis Jiankang Deng Anh Nguyen Shan Luo 28 2 0 11 Jul 2024
Improving robustness to corruptions with multiplicative weight perturbations Trung Trinh Markus Heinonen Luigi Acerbi Samuel Kaski 34 0 0 24 Jun 2024
Thoracic Surgery Video Analysis for Surgical Phase Recognition S. Mateen Niharika Malvia Syed Abdul Khader Danny Wang Deepti Srinivasan Chi-Fu Jeffrey Yang Lana Schumacher Sandeep Manjanna 16 0 0 13 Jun 2024
What Variables Affect Out-Of-Distribution Generalization in Pretrained Models? Md Yousuf Harun Kyungbok Lee Jhair Gallardo Giri Krishnan Christopher Kanan 26 2 0 23 May 2024
LookHere: Vision Transformers with Directed Attention Generalize and Extrapolate A. Fuller Daniel G. Kyrollos Yousef Yassin James R. Green 34 2 0 22 May 2024
Quantum Vision Transformers for Quark-Gluon Classification Marçal Comajoan Cara Gopal Ramesh Dahale Zhongtian Dong Roy T. Forestano S. Gleyzer ... Kyoungchul Kong Tom Magorsch Konstantin T. Matchev Katia Matcheva Eyup B. Unlu 38 9 0 16 May 2024
Structure-Preserving Network Compression Via Low-Rank Induced Training Through Linear Layers Composition Xitong Zhang Ismail R. Alkhouri Rongrong Wang 31 0 0 06 May 2024
Solving Masked Jigsaw Puzzles with Diffusion Vision Transformers Jinyang Liu Wondmgezahu Teshome S. Ghimire M. Sznaier Octavia Camps DiffM 29 1 0 10 Apr 2024
LocCa: Visual Pretraining with Location-aware Captioners Bo Wan Michael Tschannen Yongqin Xian Filip Pavetić Ibrahim M. Alabdulmohsin Xiao Wang André Susano Pinto Andreas Steiner Lucas Beyer Xiao-Qi Zhai VLM 40 6 0 28 Mar 2024
Clean-image Backdoor Attacks Dazhong Rong Guoyao Yu Shuheng Shen Xinyi Fu Peng Qian Jianhai Chen Qinming He Xing Fu Weiqiang Wang 32 4 0 22 Mar 2024
Retina Vision Transformer (RetinaViT): Introducing Scaled Patches into Vision Transformers Yuyang Shu Michael E. Bain ViT MedIm MDE 29 0 0 20 Mar 2024
Emotion Recognition Using Transformers with Masked Learning Seongjae Min Junseok Yang Sangjun Lim Junyong Lee Sangwon Lee Sejoon Lim 27 8 0 19 Mar 2024
Frozen Feature Augmentation for Few-Shot Image Classification Andreas Bär N. Houlsby Mostafa Dehghani Manoj Kumar VLM 26 4 0 15 Mar 2024
xMLP: Revolutionizing Private Inference with Exclusive Square Activation Jiajie Li Jinjun Xiong 16 0 0 12 Mar 2024
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models Frederik Kunstner Robin Yadav Alan Milligan Mark Schmidt Alberto Bietti 29 26 0 29 Feb 2024
Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling Mahdi Karami Ali Ghodsi VLM 36 6 0 28 Feb 2024
WindDragon: Enhancing wind power forecasting with Automated Deep Learning Julie Keisler E. L. Naour 21 1 0 22 Feb 2024
Convolutional Initialization for Data-Efficient Vision Transformers Jianqiao Zheng Xueqian Li Simon Lucey 30 2 0 23 Jan 2024
Momentum-SAM: Sharpness Aware Minimization without Computational Overhead Marlon Becker Frederick Altrock Benjamin Risse 74 5 0 22 Jan 2024
Always-Sparse Training by Growing Connections with Guided Stochastic Exploration Mike Heddes Narayan Srinivasa T. Givargis Alexandru Nicolau 91 0 0 12 Jan 2024
Universal Pyramid Adversarial Training for Improved ViT Performance Ping Yeh-Chiang Yipin Zhou Omid Poursaeed S. Narayan Shukla Tom Goldstein Ser-Nam Lim AAML ViT 14 0 0 26 Dec 2023
Are Vision Transformers More Data Hungry Than Newborn Visual Systems? Lalit Pandey Samantha M. W. Wood Justin N. Wood 21 11 0 05 Dec 2023
Generalizable Imitation Learning Through Pre-Trained Representations Wei-Di Chang F. Hogan D. Meger Gregory Dudek Gregory Dudek 33 1 0 15 Nov 2023
Fast and Interpretable Face Identification for Out-Of-Distribution Data Using Vision Transformers Hai T. Phan Cindy X. Le Vu Le Yihui He Anh Totti Nguyen 23 3 0 06 Nov 2023
A Quadratic Synchronization Rule for Distributed Deep Learning Xinran Gu Kaifeng Lyu Sanjeev Arora Jingzhao Zhang Longbo Huang 36 1 0 22 Oct 2023
Learning to (Learn at Test Time) Yu Sun Xinhao Li Karan Dalal Chloe Hsu Oluwasanmi Koyejo Carlos Guestrin Xiaolong Wang Tatsunori Hashimoto Xinlei Chen SSL 25 6 0 20 Oct 2023
SILC: Improving Vision Language Pretraining with Self-Distillation Muhammad Ferjad Naeem Yongqin Xian Xiaohua Zhai Lukas Hoyer Luc Van Gool F. Tombari VLM 17 32 0 20 Oct 2023
Replacing softmax with ReLU in Vision Transformers Mitchell Wortsman Jaehoon Lee Justin Gilmer Simon Kornblith ViT 22 31 0 15 Sep 2023
On the Implicit Bias of Adam M. D. Cattaneo Jason M. Klusowski Boris Shigida 23 17 0 31 Aug 2023
RaViTT: Random Vision Transformer Tokens Felipe A. Quezada Carlos F. Navarro Cristian Muñoz Manuel Zamorano Jorge Jara-Wilde Violeta Chang C. Navarro Mauricio Cerda ViT 17 1 0 19 Jun 2023
B-cos Alignment for Inherently Interpretable CNNs and Vision Transformers Moritz D Boehle Navdeeppal Singh Mario Fritz Bernt Schiele 47 26 0 19 Jun 2023
Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Design Ibrahim M. Alabdulmohsin Xiaohua Zhai Alexander Kolesnikov Lucas Beyer VLM 27 56 0 22 May 2023
Joint Moment Retrieval and Highlight Detection Via Natural Language Queries Richard Luo Austin Peng Heidi Yap Koby Beard ViT 16 0 0 08 May 2023
JaxPruner: A concise library for sparsity research Jooyoung Lee Wonpyo Park Nicole Mitchell Jonathan Pilault J. Obando-Ceron ... Hong-Seok Kim Yann N. Dauphin Karolina Dziugaite P. S. Castro Utku Evci 31 14 0 27 Apr 2023
Synthetic Data from Diffusion Models Improves ImageNet Classification Shekoofeh Azizi Simon Kornblith Chitwan Saharia Mohammad Norouzi David J. Fleet VLM DiffM 20 288 0 17 Apr 2023