Achieving Sparse Activation in Small Language Models

3 June 2024

Papers citing "Achieving Sparse Activation in Small Language Models"

6 / 6 papers shown

Title
Small Language Models: Survey, Measurements, and Insights Zhenyan Lu Xiang Li Dongqi Cai Rongjie Yi Fangming Liu Xiwen Zhang Nicholas D. Lane Mengwei Xu ObjD LRM 49 31 0 24 Sep 2024
MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT Omkar Thawakar Ashmal Vayani Salman Khan Hisham Cholakal Rao M. Anwer M. Felsberg Timothy Baldwin Eric P. Xing Fahad Shahbaz Khan 46 31 0 26 Feb 2024
PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU Yixin Song Zeyu Mi Haotong Xie Haibo Chen BDL 117 114 0 16 Dec 2023
The Falcon Series of Open Language Models Ebtesam Almazrouei Hamza Alobeidli Abdulaziz Alshamsi Alessandro Cappelli Ruxandra-Aimée Cojocaru ... Quentin Malartic Daniele Mazzotta Badreddine Noune B. Pannier Guilherme Penedo AI4TS ALM 113 389 0 28 Nov 2023
SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance Edouard Yvinec Arnaud Dapogny Matthieu Cord Kévin Bailly 27 7 0 08 Jul 2022
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 131 679 0 31 Jan 2021