Hungry Hungry Hippos: Towards Language Modeling with State Space Models

28 December 2022

Papers citing "Hungry Hungry Hippos: Towards Language Modeling with State Space Models"

50 / 284 papers shown

Title
DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesis Yao Teng Yue Wu Han Shi Xuefei Ning Guohao Dai Yu-Xiang Wang Zhenguo Li Xihui Liu Mamba 46 32 0 23 May 2024
Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model Yuheng Shi Minjing Dong Chang Xu Mamba 35 31 0 23 May 2024
TENNs-PLEIADES: Building Temporal Kernels with Orthogonal Polynomials Yan Ru Pei Olivier Coenen 24 3 0 20 May 2024
An Investigation of Incorporating Mamba for Speech Enhancement Rong-Yu Chao Wen-Huang Cheng Moreno La Quatra Sabato Marco Siniscalchi Chao-Han Huck Yang Szu-Wei Fu Yu Tsao Mamba 40 25 0 10 May 2024
State-Free Inference of State-Space Models: The Transfer Function Approach Rom N. Parnichkun Stefano Massaroli Alessandro Moro Jimmy T.H. Smith Ramin Hasani ... Hajime Asama Stefano Ermon Taiji Suzuki Atsushi Yamashita Michael Poli 33 4 0 10 May 2024
You Only Cache Once: Decoder-Decoder Architectures for Language Models Yutao Sun Li Dong Yi Zhu Shaohan Huang Wenhui Wang Shuming Ma Quanlu Zhang Jianyong Wang Furu Wei VLM 25 52 0 08 May 2024
HC-Mamba: Vision MAMBA with Hybrid Convolutional Techniques for Medical Image Segmentation Jiashu Xu Mamba 14 9 0 08 May 2024
Matten: Video Generation with Mamba-Attention Yu Gao Jiancheng Huang Xiaopeng Sun Zequn Jie Yujie Zhong Lin Ma 61 11 0 05 May 2024
From Generalization Analysis to Optimization Designs for State Space Models Fusheng Liu Qianxiao Li 19 6 0 04 May 2024
Weight Sparsity Complements Activity Sparsity in Neuromorphic Language Models Rishav Mukherji Mark Schöne Khaleelulla Khan Nazeer Christian Mayr David Kappel Anand Subramoney 32 1 0 01 May 2024
Spectral-Spatial Mamba for Hyperspectral Image Classification Lin Huang Yushi Chen Xin He Mamba 34 27 0 29 Apr 2024
Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges Badri N. Patro Vijay Srinivas Agneeswaran Mamba 30 37 0 24 Apr 2024
A Survey on Visual Mamba Hanwei Zhang Ying Zhu Dan Wang Lijun Zhang Tianxiang Chen Zi Ye Mamba 32 52 0 24 Apr 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu-Xiang Wang 46 78 0 22 Apr 2024
Mechanistic Interpretability for AI Safety -- A Review Leonard Bereska E. Gavves AI4CE 38 111 0 22 Apr 2024
Vim4Path: Self-Supervised Vision Mamba for Histopathology Images Ali Nasiri-Sarvi Vincent Quoc-Huy Trinh Hassan Rivaz Mahdi S. Hosseini 30 6 0 20 Apr 2024
Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motion Xinghan Wang Zixi Kang Yadong Mu Mamba 28 6 0 17 Apr 2024
HSIDMamba: Exploring Bidirectional State-Space Models for Hyperspectral Denoising Yang Liu Jiahua Xiao Yu-Xiao Guo Peilin Jiang Haiwei Yang Fei-Yue Wang Mamba 19 6 0 15 Apr 2024
State Space Model for New-Generation Network Alternative to Transformers: A Survey Xiao Wang Shiao Wang Yuhe Ding Yuehang Li Wentao Wu ... Bowei Jiang Chenglong Li Yaowei Wang Yonghong Tian Jin Tang Mamba 33 48 0 15 Apr 2024
Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length Xuezhe Ma Xiaomeng Yang Wenhan Xiong Beidi Chen Lili Yu Hao Zhang Jonathan May Luke Zettlemoyer Omer Levy Chunting Zhou 40 25 0 12 Apr 2024
MambaDFuse: A Mamba-based Dual-phase Model for Multi-modality Image Fusion Zhe Li Haiwei Pan Kejia Zhang Yuhua Wang Feng Yu Mamba 37 19 0 12 Apr 2024
The Illusion of State in State-Space Models William Merrill Jackson Petty Ashish Sabharwal 46 43 0 12 Apr 2024
FusionMamba: Efficient Image Fusion with State Space Model Siran Peng Xiangyu Zhu Haoyu Deng Zhen Lei Liang-Jian Deng Mamba 36 7 0 11 Apr 2024
HGRN2: Gated Linear RNNs with State Expansion Zhen Qin Songlin Yang Weixuan Sun Xuyang Shen Dong Li Weigao Sun Yiran Zhong LRM 34 45 0 11 Apr 2024
SurvMamba: State Space Model with Multi-grained Multi-modal Interaction for Survival Prediction Ying Chen Jiajing Xie Yuxiang Lin Yuhang Song Wenxian Yang Rongshan Yu Mamba 31 6 0 11 Apr 2024
MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detection Haoyang He Yuhu Bai Jiangning Zhang Qingdong He Hongxu Chen Zhenye Gan Chengjie Wang Xiangtai Li Guanzhong Tian Lei Xie Mamba 55 33 0 09 Apr 2024
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence Bo Peng Daniel Goldstein Quentin G. Anthony Alon Albalak Eric Alcaide ... Bingchen Zhao Qihang Zhao Peng Zhou Jian Zhu Ruijie Zhu 43 73 0 08 Apr 2024
ChangeMamba: Remote Sensing Change Detection with Spatio-Temporal State Space Model Hongruixuan Chen Jian Song Chengxi Han Junshi Xia Naoto Yokoya Mamba 26 73 0 04 Apr 2024
Cross-Architecture Transfer Learning for Linear-Cost Inference Transformers Sehyun Choi 24 3 0 03 Apr 2024
Samba: Semantic Segmentation of Remotely Sensed Images with State Space Model Qinfeng Zhu Yuanzhi Cai Yuan-Sheng Fang Yihan Yang Cheng Chen Lei Fan Anh Nguyen Mamba 43 54 0 02 Apr 2024
NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fields Muhammad Zubair Irshad Sergey Zakahrov Vitor Campagnolo Guizilini Adrien Gaidon Z. Kira Rares Ambrus ViT 32 12 0 01 Apr 2024
HSIMamba: Hyperpsectral Imaging Efficient Feature Learning with Bidirectional State Space for Classification Judy X Yang Jun Zhou Jing Wang Hui Tian Alan Wee-Chung Liew Mamba 28 15 0 30 Mar 2024
MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selection Ali Behrouz Michele Santacatterina Ramin Zabih 37 32 0 29 Mar 2024
Jamba: A Hybrid Transformer-Mamba Language Model Opher Lieber Barak Lenz Hofit Bata Gal Cohen Jhonathan Osin ... Nir Ratner N. Rozen Erez Shwartz Mor Zusman Y. Shoham 21 206 0 28 Mar 2024
Mechanistic Design and Scaling of Hybrid Architectures Michael Poli Armin W. Thomas Eric N. D. Nguyen Pragaash Ponnusamy Bjorn Deiseroth ... Brian Hie Stefano Ermon Christopher Ré Ce Zhang Stefano Massaroli MoE 49 21 0 26 Mar 2024
PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition Chenhongyi Yang Zehui Chen Miguel Espinosa Linus Ericsson Zhenyu Wang Jiaming Liu Elliot J. Crowley Mamba 26 86 0 26 Mar 2024
State Space Models as Foundation Models: A Control Theoretic Overview Carmen Amo Alonso Jerome Sieber M. Zeilinger AI4CE Mamba 33 12 0 25 Mar 2024
VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting Yujin Tang Peijie Dong Zhenheng Tang Xiaowen Chu Junwei Liang Mamba 60 19 0 25 Mar 2024
Understanding Emergent Abilities of Language Models from the Loss Perspective Zhengxiao Du Aohan Zeng Yuxiao Dong Jie Tang UQCV LRM 55 46 0 23 Mar 2024
SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time series Badri N. Patro Vijay Srinivas Agneeswaran Mamba 51 50 0 22 Mar 2024
Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference Han Zhao Min Zhang Wei Zhao Pengxiang Ding Siteng Huang Donglin Wang Mamba 31 64 0 21 Mar 2024
ZigMa: A DiT-style Zigzag Mamba Diffusion Model Vincent Tao Hu S. A. Baumann Ming Gui Olga Grebenkova Pingchuan Ma Johannes S. Fischer Bjorn Ommer 35 42 0 20 Mar 2024
On the low-shot transferability of [V]-Mamba Diganta Misra Jay Gala Antonio Orvieto Mamba 37 1 0 15 Mar 2024
Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference Piotr Nawrot Adrian Lañcucki Marcin Chochowski David Tarjan E. Ponti 28 50 0 14 Mar 2024
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models Zunnan Xu Yukang Lin Haonan Han Sicheng Yang Ronghui Li Yachao Zhang Xiu Li Mamba 46 25 0 14 Mar 2024
SSM Meets Video Diffusion Models: Efficient Video Generation with Structured State Spaces Yuta Oshima Shohei Taniguchi Masahiro Suzuki Yutaka Matsuo 32 7 0 12 Mar 2024
VideoMamba: State Space Model for Efficient Video Understanding Kunchang Li Xinhao Li Yi Wang Yinan He Yali Wang Limin Wang Yu Qiao Mamba 30 174 0 11 Mar 2024
Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling Yair Schiff Chia-Hsiang Kao Aaron Gokaslan Tri Dao Albert Gu Volodymyr Kuleshov Mamba 21 78 0 05 Mar 2024
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models Soham De Samuel L. Smith Anushan Fernando Aleksandar Botev George-Christian Muraru ... David Budden Yee Whye Teh Razvan Pascanu Nando de Freitas Çağlar Gülçehre Mamba 53 116 0 29 Feb 2024
Theoretical Foundations of Deep Selective State-Space Models Nicola Muca Cirone Antonio Orvieto Benjamin Walker C. Salvi Terry Lyons Mamba 45 24 0 29 Feb 2024