Eigen Analysis of Self-Attention and its Reconstruction from Partial
Computation

Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation

16 June 2021

Srinadh Bhojanapalli

Ayan Chakrabarti

Sanjiv Kumar

Papers citing "Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation"

4 / 4 papers shown

Title
ThinK: Thinner Key Cache by Query-Driven Pruning Yuhui Xu Zhanming Jie Hanze Dong Lei Wang Xudong Lu Aojun Zhou Amrita Saha Caiming Xiong Doyen Sahoo 72 14 0 30 Jul 2024
Attention Mixtures for Time-Aware Sequential Recommendation Viet-Anh Tran Guillaume Salha-Galvan Bruno Sguerra Romain Hennequin 28 21 0 17 Apr 2023
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 246 580 0 12 Mar 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,959 0 20 Apr 2018