Exploring Transformer Extrapolation

Exploring Transformer Extrapolation

19 July 2023

Zhen Qin

Papers citing "Exploring Transformer Extrapolation"

4 / 4 papers shown

Title
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 242 690 0 27 Aug 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Yin Cui Boqing Gong ViT 240 573 0 22 Apr 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,977 0 31 Dec 2020
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 250 13,283 0 25 Aug 2014