Hungry Hungry Hippos: Towards Language Modeling with State Space Models

28 December 2022

Papers citing "Hungry Hungry Hippos: Towards Language Modeling with State Space Models"

50 / 284 papers shown

Title
Towards a theory of learning dynamics in deep state space models Jakub Smékal Jimmy T.H. Smith Michael Kleinman D. Biderman Scott W. Linderman 25 1 0 10 Jul 2024
B'MOJO: Hybrid State Space Realizations of Foundation Models with Eidetic and Fading Memory L. Zancato Arjun Seshadri Yonatan Dukler Aditya Golatkar Yantao Shen Benjamin Bowman Matthew Trager Alessandro Achille Stefano Soatto 29 8 0 08 Jul 2024
Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning Xiaojie Li Yibo Yang Jianlong Wu Bernard Ghanem Liqiang Nie Min Zhang Mamba 36 5 0 08 Jul 2024
On the Power of Convolution Augmented Transformer Mingchen Li Xuechen Zhang Yixiao Huang Samet Oymak 32 0 0 08 Jul 2024
Mamba Hawkes Process Anningzhe Gao Shan Dai Yan Hu Mamba 26 1 0 07 Jul 2024
Learning to (Learn at Test Time): RNNs with Expressive Hidden States Yu Sun Xinhao Li Karan Dalal Jiarui Xu Arjun Vikram ... Xinlei Chen Xiaolong Wang Sanmi Koyejo Tatsunori Hashimoto Carlos Guestrin 50 89 0 05 Jul 2024
VFIMamba: Video Frame Interpolation with State Space Models Guozhen Zhang Chunxu Liu Yutao Cui Xiaotong Zhao Kai Ma Limin Wang 34 8 0 02 Jul 2024
MTMamba: Enhancing Multi-Task Dense Scene Understanding by Mamba-Based Decoders Baijiong Lin Weisen Jiang Pengguang Chen Yu Zhang Shu Liu Ying-Cong Chen Mamba 38 9 0 02 Jul 2024
SE(3)-Hyena Operator for Scalable Equivariant Learning Artem Moskalev Mangal Prakash Rui Liao Tommaso Mansi 27 2 0 01 Jul 2024
From Efficient Multimodal Models to World Models: A Survey Xinji Mai Zeng Tao Junxiong Lin Haoran Wang Yang Chang Yanlan Kang Yan Wang Wenqiang Zhang 29 5 0 27 Jun 2024
Venturing into Uncharted Waters: The Navigation Compass from Transformer to Mamba Yuchen Zou Yineng Chen Zuchao Li Lefei Zhang Hai Zhao 42 1 0 24 Jun 2024
Scaling Laws for Linear Complexity Language Models Xuyang Shen Dong Li Ruitao Leng Zhen Qin Weigao Sun Yiran Zhong LRM 28 6 0 24 Jun 2024
Vision Mamba-based autonomous crack segmentation on concrete, asphalt, and masonry surfaces Zhaohui Chen Elyas Asadi Shamsabadi Sheng Jiang Luming Shen Daniel Dias-da-Costa Mamba 35 3 0 24 Jun 2024
LFMamba: Light Field Image Super-Resolution with State Space Model Wang xia Yao Lu Shunzhou Wang Ziqi Wang Peiqi Xia Tianfei Zhou Mamba 38 4 0 18 Jun 2024
Slot State Space Models Jindong Jiang Fei Deng Gautam Singh Minseung Lee Sungjin Ahn 39 4 0 18 Jun 2024
A Scalable and Effective Alternative to Graph Transformers Kaan Sancak Zhigang Hua Jin Fang Yan Xie Andrey Malevich Bo Long M. F. Balin Ümit V. Çatalyürek 35 1 0 17 Jun 2024
Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection Guowen Zhang Lue Fan Chenhang He Zhen Lei Zhaoxiang Zhang Lei Zhang Mamba 39 20 0 15 Jun 2024
Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences Zicheng Liu Siyuan Li Li Wang Zedong Wang Yunfan Liu Stan Z. Li 22 7 0 12 Jun 2024
MambaLRP: Explaining Selective State Space Sequence Models F. Jafari G. Montavon Klaus-Robert Müller Oliver Eberle Mamba 47 9 0 11 Jun 2024
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling Liliang Ren Yang Liu Yadong Lu Yelong Shen Chen Liang Weizhu Chen Mamba 64 54 0 11 Jun 2024
What Can We Learn from State Space Models for Machine Learning on Graphs? Yinan Huang Siqi Miao Pan Li 39 7 0 09 Jun 2024
Convolution and Attention-Free Mamba-based Cardiac Image Segmentation Abbas Khan Muhammad Asad Martin Benning C. Roney Gregory Slabaugh Mamba 22 2 0 09 Jun 2024
LoCoCo: Dropping In Convolutions for Long Context Compression Ruisi Cai Yuandong Tian Zhangyang Wang Beidi Chen 27 9 0 08 Jun 2024
C-Mamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecasting Chaolv Zeng Zhanyu Liu Guanjie Zheng Linghe Kong Mamba 36 5 0 08 Jun 2024
Efficient 3D Shape Generation via Diffusion Mamba with Bidirectional SSMs Shentong Mo Mamba 14 4 0 07 Jun 2024
RoboMamba: Multimodal State Space Model for Efficient Robot Reasoning and Manipulation Jiaming Liu Mengzhen Liu Zhenyu Wang Lily Lee Kaichen Zhou Pengju An Senqiao Yang Renrui Zhang Yandong Guo Shanghang Zhang LM&Ro LRM Mamba 27 5 0 06 Jun 2024
Audio Mamba: Bidirectional State Space Model for Audio Representation Learning Mehmet Hamza Erol Arda Senocak Jiu Feng Joon Son Chung Mamba 62 18 0 05 Jun 2024
U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation Chenxin Li Xinyu Liu W. J. Li Cheng Wang Hengyu Liu Yifan Liu Zhen Chen Yixuan Yuan MedIm DiffM SSeg 46 72 0 05 Jun 2024
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers Brian K Chen Tianyang Hu Hui Jin Hwee Kuan Lee Kenji Kawaguchi 35 0 0 05 Jun 2024
Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations Sarthak Yadav Z. Tan Mamba 18 10 0 04 Jun 2024
Mamba as Decision Maker: Exploring Multi-scale Sequence Modeling in Offline Reinforcement Learning Jiahang Cao Qiang Zhang Ziqing Wang Jiaxu Wang Hao Cheng Yecheng Shao Wen Zhao Gang Han Yijie Guo Renjing Xu Mamba 43 2 0 04 Jun 2024
Dimba: Transformer-Mamba Diffusion Models Zhengcong Fei Mingyuan Fan Changqian Yu Debang Li Youqiang Zhang Junshi Huang Mamba 54 16 0 03 Jun 2024
State Space Models on Temporal Graphs: A First-Principles Study Jintang Li Ruofan Wu Xinzhou Jin Boqun Ma Liang Chen Zibin Zheng 53 2 0 03 Jun 2024
Pretrained Hybrids with MAD Skills Nicholas Roberts Samuel Guo Zhiqi Gao Satya Sai Srinath Namburi Sonia Cromp Chengjun Wu Chengyu Duan Frederic Sala Mamba 35 0 0 02 Jun 2024
Dual Hyperspectral Mamba for Efficient Spectral Compressive Imaging Jiahua Dong Hui Yin Hongliu Li Wenbo Li Yulun Zhang Salman Khan Fahad Shahbaz Khan Mamba 30 1 0 01 Jun 2024
Decision Mamba: Reinforcement Learning via Hybrid Selective Sequence Modeling Sili Huang Jifeng Hu Zhe Yang Liwei Yang Tao Luo Hechang Chen Lichao Sun Bo Yang Mamba 21 2 0 31 May 2024
Fourier Controller Networks for Real-Time Decision-Making in Embodied Learning Hengkai Tan Songming Liu Kai Ma Chengyang Ying Xingxing Zhang Hang Su Jun Zhu 29 2 0 30 May 2024
ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attention Bencheng Liao Xinggang Wang Lianghui Zhu Qian Zhang Chang Huang 45 3 0 28 May 2024
Coupled Mamba: Enhanced Multi-modal Fusion with Coupled State Space Model Wenbing Li Hang Zhou Junqing Yu Zikai Song Wei Yang Mamba 36 3 0 28 May 2024
The Expressive Capacity of State Space Models: A Formal Language Perspective Yash Sarrof Yana Veitsman Michael Hahn Mamba 30 7 0 27 May 2024
Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention Zhen Qin Weigao Sun Dong Li Xuyang Shen Weixuan Sun Yiran Zhong 36 9 0 27 May 2024
Demystify Mamba in Vision: A Linear Attention Perspective Dongchen Han Ziyi Wang Zhuofan Xia Yizeng Han Yifan Pu Chunjiang Ge Jun Song Shiji Song Bo Zheng Gao Huang Mamba 29 48 0 26 May 2024
MambaTS: Improved Selective State Space Models for Long-term Time Series Forecasting Xiuding Cai Yaoyao Zhu Xueyao Wang Yu Yao Mamba AI4TS 27 7 0 26 May 2024
Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory Nikola Zubić Federico Soldá Aurelio Sulser Davide Scaramuzza LRM BDL 39 5 0 26 May 2024
MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space Jiangwei Weng Zhiqiang Yan Ying Tai J. Qian Jian Yang Jun Li Mamba 22 9 0 25 May 2024
Scaling Diffusion Mamba with Bidirectional SSMs for Efficient Image and Video Generation Shentong Mo Yapeng Tian Mamba 65 16 0 24 May 2024
Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks Jerome Sieber Carmen Amo Alonso A. Didier M. Zeilinger Antonio Orvieto AAML 42 7 0 24 May 2024
PoinTramba: A Hybrid Transformer-Mamba Framework for Point Cloud Analysis Zicheng Wang Zhen Chen Yiming Wu Zhen Zhao Luping Zhou Dong Xu Mamba 43 12 0 24 May 2024
MambaVC: Learned Visual Compression with Selective State Spaces Shiyu Qin Jinpeng Wang Yimin Zhou Bin Chen Tianci Luo Baoyi An Tao Dai Shu-Tao Xia Yaowei Wang Mamba 32 13 0 24 May 2024
PointRWKV: Efficient RWKV-Like Model for Hierarchical Point Cloud Learning Qingdong He Jiangning Zhang Jinlong Peng Haoyang He Yabiao Wang Chengjie Wang 3DPC 35 12 0 24 May 2024