On the Long Range Abilities of Transformers

On the Long Range Abilities of Transformers

28 November 2023

Itamar Zimerman

Lior Wolf

Papers citing "On the Long Range Abilities of Transformers"

8 / 8 papers shown

Title
EMMA: Empowering Multi-modal Mamba with Structural and Hierarchical Alignment Yifei Xing Xiangyuan Lan Ruiping Wang D. Jiang Wenjun Huang Qingfang Zheng Yaowei Wang Mamba 33 0 0 08 Oct 2024
The Hidden Attention of Mamba Models Ameen Ali Itamar Zimerman Lior Wolf Mamba 29 57 0 03 Mar 2024
Investigating Recurrent Transformers with Dynamic Halt Jishnu Ray Chowdhury Cornelia Caragea 34 1 0 01 Feb 2024
Resurrecting Recurrent Neural Networks for Long Sequences Antonio Orvieto Samuel L. Smith Albert Gu Anushan Fernando Çağlar Gülçehre Razvan Pascanu Soham De 88 258 0 11 Mar 2023
Structured State Space Models for In-Context Reinforcement Learning Chris Xiaoxuan Lu Yannick Schroecker Albert Gu Emilio Parisotto Jakob N. Foerster Satinder Singh Feryal M. P. Behbahani AI4TS 84 80 0 07 Mar 2023
Liquid Structural State-Space Models Ramin Hasani Mathias Lechner Tsun-Hsuan Wang Makram Chahine Alexander Amini Daniela Rus AI4TS 95 93 0 26 Sep 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 237 690 0 27 Aug 2021
U-Net: Convolutional Networks for Biomedical Image Segmentation Olaf Ronneberger Philipp Fischer Thomas Brox SSeg 3DV 229 74,467 0 18 May 2015