MatFormer: Nested Transformer for Elastic Inference

11 October 2023

Tim Dettmers

Inderjit Dhillon

Papers citing "MatFormer: Nested Transformer for Elastic Inference"

22 / 22 papers shown

Title
ORXE: Orchestrating Experts for Dynamically Configurable Efficiency Qingyuan Wang Guoxin Wang B. Cardiff Deepu John 31 0 0 07 May 2025
Adaptive Audio-Visual Speech Recognition via Matryoshka-Based Multimodal LLMs Umberto Cappellazzo Minsu Kim Stavros Petridis 47 0 0 09 Mar 2025
RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs Zhongzhan Huang Guoming Ling Vincent S. Liang Yupei Lin Yandong Chen Shanshan Zhong Hefeng Wu Liang Lin LRM 54 2 0 08 Mar 2025
Balcony: A Lightweight Approach to Dynamic Inference of Generative Language Models Benyamin Jamialahmadi Parsa Kavehzadeh Mehdi Rezagholizadeh Parsa Farinneya Hossein Rajabzadeh A. Jafari Boxing Chen Marzieh S. Tahaei 42 0 0 06 Mar 2025
Slicing Vision Transformer for Flexible Inference Yitian Zhang Huseyin Coskun Xu Ma Huan Wang Ke Ma Xi Chen Derek Hao Hu Y. Fu ViT 74 0 0 06 Dec 2024
big.LITTLE Vision Transformer for Efficient Visual Recognition He Guo Yulong Wang Zixuan Ye Jifeng Dai Yuwen Xiong ViT 50 0 0 14 Oct 2024
Neural Metamorphosis Xingyi Yang Xinchao Wang 19 1 0 10 Oct 2024
MatMamba: A Matryoshka State Space Model Abhinav Shukla Sai H. Vemprala Aditya Kusupati Ashish Kapoor Mamba 28 0 0 09 Oct 2024
HydraViT: Stacking Heads for a Scalable ViT Janek Haberer A. Hojjat Olaf Landsiedel 19 0 0 26 Sep 2024
Mixture of Nested Experts: Adaptive Processing of Visual Tokens Gagan Jain Nidhi Hegde Aditya Kusupati Arsha Nagrani Shyamal Buch Prateek Jain Anurag Arnab Sujoy Paul MoE 33 7 0 29 Jul 2024
Flextron: Many-in-One Flexible Large Language Model Ruisi Cai Saurav Muralidharan Greg Heinrich Hongxu Yin Zhangyang Wang Jan Kautz Pavlo Molchanov 26 10 0 11 Jun 2024
Matryoshka Query Transformer for Large Vision-Language Models Wenbo Hu Zi-Yi Dou Liunian Harold Li Amita Kamath Nanyun Peng Kai-Wei Chang MLLM 18 8 0 29 May 2024
Superposed Decoding: Multiple Generations from a Single Autoregressive Inference Pass Ethan Shen Alan Fan Sarah M Pratt Jae Sung Park Matthew Wallingford Sham Kakade Ari Holtzman Ranjay Krishna Ali Farhadi Aditya Kusupati 33 2 0 28 May 2024
Efficient Stagewise Pretraining via Progressive Subnetworks Abhishek Panigrahi Nikunj Saunshi Kaifeng Lyu Sobhan Miryoosefi Sashank J. Reddi Satyen Kale Sanjiv Kumar 23 5 0 08 Feb 2024
SHARCS: Efficient Transformers through Routing with Dynamic Width Sub-networks Mohammadreza Salehi Sachin Mehta Aditya Kusupati Ali Farhadi Hannaneh Hajishirzi 12 5 0 18 Oct 2023
AdANNS: A Framework for Adaptive Semantic Search Aniket Rege Aditya Kusupati S. SharanRanjit Alan Fan Qingqing Cao Sham Kakade Prateek Jain Ali Farhadi 18 5 0 30 May 2023
Progressive Ensemble Distillation: Building Ensembles for Efficient Inference D. Dennis Abhishek Shetty A. Sevekari K. Koishida Virginia Smith FedML 11 0 0 20 Feb 2023
SCENIC: A JAX Library for Computer Vision Research and Beyond Mostafa Dehghani A. Gritsenko Anurag Arnab Matthias Minderer Yi Tay 41 67 0 18 Oct 2021
Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference Sneha Kudugunta Yanping Huang Ankur Bapna M. Krikun Dmitry Lepikhin Minh-Thang Luong Orhan Firat MoE 119 104 0 24 Sep 2021
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 83 151 0 17 Sep 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 282 39,170 0 01 Sep 2014