SummaryMixing: A Linear-Complexity Alternative to Self-Attention for
Speech Recognition and Understanding

SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understanding

12 July 2023

Titouan Parcollet

Rogier van Dalen

Shucong Zhang

S. Bhattacharya

Papers citing "SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understanding"

8 / 8 papers shown

Title
Lina-Speech: Gated Linear Attention is a Fast and Parameter-Efficient Learner for text-to-speech synthesis Théodor Lemerle Harrison Vanderbyl Vaibhav Srivastav Nicolas Obin Axel Roebel 31 1 0 30 Oct 2024
An Analysis of Linear Complexity Attention Substitutes with BEST-RQ Ryan Whetten Titouan Parcollet Adel Moumen Marco Dinarelli Yannick Esteve 22 0 0 04 Sep 2024
Linear-Complexity Self-Supervised Learning for Speech Processing Shucong Zhang Titouan Parcollet Rogier van Dalen Sourav Bhattacharya 26 1 0 18 Jul 2024
Exploring the Capability of Mamba in Speech Applications Koichi Miyazaki Yoshiki Masuyama Masato Murata Mamba 22 12 0 24 Jun 2024
E-Branchformer: Branchformer with Enhanced merging for speech recognition Kwangyoun Kim Felix Wu Yifan Peng Jing Pan Prashant Sridhar Kyu Jeong Han Shinji Watanabe 36 104 0 30 Sep 2022
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 239 2,554 0 04 May 2021
Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognition Yangyang Shi Yongqiang Wang Chunyang Wu Ching-Feng Yeh Julian Chan Frank Zhang Duc Le M. Seltzer 49 168 0 21 Oct 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020