Towards Efficient Vision State Space Models via Token Merging

State Space Models (SSMs) have emerged as powerful architectures in computer vision, yet improving their computational efficiency remains crucial for practical and scalablethis http URLtoken reduction serves as an effective approach for model efficiency, applying it to SSMs requires careful consideration of their unique sequential modelingthis http URLthis work, we propose MaMe, a token-merging strategy tailored for SSM-based visionthis http URLaddresses two key challenges: quantifying token importance and preserving sequential properties. Our approach leverages the state transition parameter $\mathbf{\Delta}$ as an informativeness measure and introduces strategic token arrangements to preserve sequential informationthis http URLexperiments demonstrate that MaMe achieves superior efficiency-performance trade-offs for both fine-tuned and off-the-shelf models. Particularly, our approach maintains robustness even under aggressive token reduction where existing methods undergo significant performancethis http URLimage classification, MaMe shows strong generalization capabilities across video and audio domains, establishing an effective approach for enhancing efficiency in diverse SSM applications.