Language Modeling using LMUs: 10x Better Data Efficiency or Improved Scaling Compared to Transformers

5 October 2021

Papers citing "Language Modeling using LMUs: 10x Better Data Efficiency or Improved Scaling Compared to Transformers"

5 / 5 papers shown

Title
Transformers and Cortical Waves: Encoders for Pulling In Context Across Time L. Muller P. Churchland T. Sejnowski 24 6 0 25 Jan 2024
The Spike Gating Flow: A Hierarchical Structure Based Spiking Neural Network for Online Gesture Recognition Zihao Zhao Yanhong Wang Qiaosha Zou Tie Xu Fangbo Tao Jiansong Zhang Xiaoan Wang Chuanjin Richard Shi Junwen Luo Yuan Xie 31 1 0 04 Jun 2022
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,986 0 31 Dec 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 2,009 0 28 Jul 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020