Mitigating Over-smoothing in Transformers via Regularized Nonlocal
Functionals

Mitigating Over-smoothing in Transformers via Regularized Nonlocal Functionals

1 December 2023

Tan-Minh Nguyen

Richard G. Baraniuk

Papers citing "Mitigating Over-smoothing in Transformers via Regularized Nonlocal Functionals"

7 / 7 papers shown

Title
Transformer Meets Twicing: Harnessing Unattended Residual Information Laziz U. Abdullaev Tan M. Nguyen 41 2 0 02 Mar 2025
Transformer-based Graph Neural Networks for Battery Range Prediction in AIoT Battery-Swap Services Zhao Li Yang Liu Chuan Zhou Xuanwu Liu Xuming Pan Buqing Cao Xindong Wu 65 0 0 17 Feb 2025
Graph Convolutions Enrich the Self-Attention in Transformers! Jeongwhan Choi Hyowon Wi Jayoung Kim Yehjin Shin Kookjin Lee Nathaniel Trask Noseong Park 30 4 0 07 Dec 2023
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 255 4,781 0 24 Feb 2021
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 253 1,828 0 18 Aug 2016
A Decomposable Attention Model for Natural Language Inference Ankur P. Parikh Oscar Täckström Dipanjan Das Jakob Uszkoreit 207 1,367 0 06 Jun 2016
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 296 39,198 0 01 Sep 2014