Falcon Mamba: The First Competitive Attention-free 7B Language Model

7 October 2024

Papers citing "Falcon Mamba: The First Competitive Attention-free 7B Language Model"

5 / 5 papers shown

Title
Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism Aviv Bick Eric P. Xing Albert Gu RALM 81 0 0 22 Apr 2025
Multilingual State Space Models for Structured Question Answering in Indic Languages A. Vats Rahul Raja Mrinal Mathur Vinija Jain Aman Chadha 68 1 0 01 Feb 2025
Mamba-Shedder: Post-Transformer Compression for Efficient Selective Structured State Space Models J. P. Muñoz Jinjie Yuan Nilesh Jain Mamba 63 1 0 28 Jan 2025
Low-Rank Interconnected Adaptation Across Layers Yibo Zhong Yao Zhou OffRL MoE 30 1 0 13 Jul 2024
DeciMamba: Exploring the Length Extrapolation Potential of Mamba Assaf Ben-Kish Itamar Zimerman Shady Abu Hussein Nadav Cohen Amir Globerson Lior Wolf Raja Giryes Mamba 53 12 0 20 Jun 2024