Short-Long Convolutions Help Hardware-Efficient Linear Attention to
Focus on Long Sequences

Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences

12 June 2024

Zicheng Liu

Stan Z. Li

Papers citing "Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences"

10 / 10 papers shown

Title
Life-Code: Central Dogma Modeling with Multi-Omics Sequence Unification Zicheng Liu Siyuan Li Zhiyuan Chen Lei Xin Fang Wu Chang Yu Qirong Yang Yucheng Guo Y. Yang Stan Z. Li SyDa AI4CE 87 0 0 11 Feb 2025
Wavelet GPT: Wavelet Inspired Large Language Models Prateek Verma AI4TS 15 0 0 04 Sep 2024
Reparameterized Multi-Resolution Convolutions for Long Sequence Modelling Harry Jake Cunningham Giorgio Giannone Mingtian Zhang M. Deisenroth 18 0 0 18 Aug 2024
SUMix: Mixup with Semantic and Uncertain Information Huafeng Qin Xin Jin Hongyu Zhu Hongchao Liao M. El-Yacoubi Xinbo Gao UQCV 23 5 0 10 Jul 2024
VQDNA: Unleashing the Power of Vector Quantization for Multi-Species Genomic Sequence Modeling Siyuan Li Zedong Wang Zicheng Liu Di Wu Cheng Tan Jiangbin Zheng Yufei Huang Stan Z. Li 16 7 0 13 May 2024
Liquid Structural State-Space Models Ramin Hasani Mathias Lechner Tsun-Hsuan Wang Makram Chahine Alexander Amini Daniela Rus AI4TS 95 93 0 26 Sep 2022
Harnessing Hard Mixed Samples with Decoupled Regularizer Zicheng Liu Siyuan Li Ge Wang Cheng Tan Lirong Wu Stan Z. Li 41 17 0 21 Mar 2022
Transformer Quality in Linear Time Weizhe Hua Zihang Dai Hanxiao Liu Quoc V. Le 71 220 0 21 Feb 2022
FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes David W. Romero Robert-Jan Bruintjes Jakub M. Tomczak Erik J. Bekkers Mark Hoogendoorn J. C. V. Gemert 74 81 0 15 Oct 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020