On Learning the Transformer Kernel

On Learning the Transformer Kernel

15 October 2021

Sankalan Pal Chowdhury

Kumar Avinava Dubey

Mrinmaya Sachan

Papers citing "On Learning the Transformer Kernel"

8 / 8 papers shown

Title
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers K. Choromanski Shanda Li Valerii Likhosherstov Kumar Avinava Dubey Shengjie Luo Di He Yiming Yang Tamás Sarlós Thomas Weingarten Adrian Weller 14 8 0 03 Feb 2023
Chefs' Random Tables: Non-Trigonometric Random Features Valerii Likhosherstov K. Choromanski Kumar Avinava Dubey Frederick Liu Tamás Sarlós Adrian Weller 13 17 0 30 May 2022
Transformer Quality in Linear Time Weizhe Hua Zihang Dai Hanxiao Liu Quoc V. Le 71 220 0 21 Feb 2022
Redesigning the Transformer Architecture with Insights from Multi-particle Dynamical Systems Subhabrata Dutta Tanya Gautam Soumen Chakrabarti Tanmoy Chakraborty 47 15 0 30 Sep 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 238 578 0 12 Mar 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,724 0 26 Sep 2016