Is Mamba Capable of In-Context Learning?

Is Mamba Capable of In-Context Learning?

5 February 2024

Riccardo Grazzi

Julien N. Siems

Thomas Brox

Papers citing "Is Mamba Capable of In-Context Learning?"

9 / 9 papers shown

Title
From Markov to Laplace: How Mamba In-Context Learns Markov Chains Marco Bondaschi Nived Rajaraman Xiuying Wei Kannan Ramchandran Razvan Pascanu Çağlar Gülçehre Michael C. Gastpar Ashok Vardhan Makkuva 58 0 0 17 Feb 2025
Mamba-Based Graph Convolutional Networks: Tackling Over-smoothing with Selective State Space Xin He Y. Wang Wenqi Fan Xu Shen Xin Juan Rui Miao Xin Wang 63 0 0 26 Jan 2025
FACTS: A Factored State-Space Framework For World Modelling Li Nanbo Firas Laakom Yucheng Xu Wenyi Wang Jürgen Schmidhuber AI4TS 45 0 0 28 Oct 2024
State-space models can learn in-context by gradient descent Neeraj Mohan Sushma Yudou Tian Harshvardhan Mestha Nicolo Colombo David Kappel Anand Subramoney 35 3 0 15 Oct 2024
Mitigating Copy Bias in In-Context Learning through Neuron Pruning Ameen Ali Lior Wolf Ivan Titov 27 2 0 02 Oct 2024
U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation Jun Ma Feifei Li Bo Wang Mamba 77 314 0 09 Jan 2024
ForecastPFN: Synthetically-Trained Zero-Shot Forecasting Samuel Dooley Gurnoor Singh Khurana Chirag Mohapatra Siddartha Naidu Colin White AI4TS 79 56 0 03 Nov 2023
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 237 690 0 27 Aug 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 245 1,977 0 31 Dec 2020