MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT

26 February 2024

Omkar Thawakar

Ashmal Vayani

Salman Khan

Hisham Cholakal

Rao M. Anwer

Fahad Shahbaz Khan

Papers citing "MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT"

25 / 25 papers shown

Title
FineScope : Precision Pruning for Domain-Specialized Large Language Models Using SAE-Guided Self-Data Cultivation Chaitali Bhattacharyya Yeseong Kim 43 0 0 01 May 2025
CoSMoEs: Compact Sparse Mixture of Experts Patrick Huber Akshat Shrivastava Ernie Chang Chinnadhurai Sankar Ahmed Aly Adithya Sagar MoE 24 0 0 28 Feb 2025
SB-Bench: Stereotype Bias Benchmark for Large Multimodal Models Vishal Narnaware Ashmal Vayani Rohit Gupta Swetha Sirnam Mubarak Shah 104 3 0 12 Feb 2025
TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models Makoto Shing Kou Misaki Han Bao Sho Yokoi Takuya Akiba VLM 49 1 0 28 Jan 2025
Lillama: Large Language Models Compression via Low-Rank Feature Distillation Yaya Sy Christophe Cerisara Irina Illina MQ 69 0 0 31 Dec 2024
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages Ashmal Vayani Dinura Dissanayake Hasindri Watawana Noor Ahsan Nevasini Sasikumar ... Monojit Choudhury Ivan Laptev Mubarak Shah Salman Khan Fahad A Khan 118 8 0 25 Nov 2024
Personal Intelligence System UniLM: Hybrid On-Device Small Language Model and Server-Based Large Language Model for Malay Nusantara Azree Nazri Olalekan Agbolade Faisal Aziz 15 0 0 09 Oct 2024
Word2Wave: Language Driven Mission Programming for Efficient Subsea Deployments of Marine Robots Ruo Chen David Blow Adnan Abdullah Md Jahidul Islam 38 1 0 27 Sep 2024
InkubaLM: A small language model for low-resource African languages A. Tonja Bonaventure F. P. Dossou Jessica Ojo Jenalea Rajab Fadel Thior ... Anuoluwapo Aremu Pelonomi Moiloa Jade Z. Abbott Vukosi Marivate Benjamin Rosman 25 2 0 30 Aug 2024
TroL: Traversal of Layers for Large Language and Vision Models Byung-Kwan Lee Sangyun Chung Chae Won Kim Beomchan Park Yong Man Ro 24 6 0 18 Jun 2024
Achieving Sparse Activation in Small Language Models Jifeng Song Kai Huang Xiangyu Yin Boyuan Yang Wei Gao 21 4 0 03 Jun 2024
Super Tiny Language Models Dylan Hillier Leon Guertler Cheston Tan Palaash Agrawal Ruirui Chen Bobby Cheng 37 3 0 23 May 2024
OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruning Dan Qiao Yi Su Pinzheng Wang Jing Ye Wen Xie ... Wenliang Chen Guohong Fu Guodong Zhou Qiaoming Zhu Min Zhang MQ 26 0 0 09 May 2024
OpenELM: An Efficient Language Model Family with Open Training and Inference Framework Sachin Mehta Mohammad Hossein Sekhavat Qingqing Cao Maxwell Horton Yanzi Jin ... Iman Mirzadeh Mahyar Najibi Dmitry Belenko Peter Zatloukal Mohammad Rastegari OSLM AIFin 38 49 0 22 Apr 2024
VURF: A General-purpose Reasoning and Self-refinement Framework for Video Understanding Ahmad A Mahmood Ashmal Vayani Muzammal Naseer Salman Khan Fahad Shahbaz Khan LRM 47 7 0 21 Mar 2024
MELTing point: Mobile Evaluation of Language Transformers Stefanos Laskaridis Kleomenis Katevas Lorenzo Minto Hamed Haddadi 19 4 0 19 Mar 2024
EfficientMorph: Parameter-Efficient Transformer-Based Architecture for 3D Image Registration Abu Zahid Bin Aziz Mokshagna Sai Teja Karanam Tushar Kataria Shireen Elhabian ViT MedIm 23 1 0 16 Mar 2024
MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases Zechun Liu Changsheng Zhao Forrest N. Iandola Chen Lai Yuandong Tian ... Ernie Chang Yangyang Shi Raghuraman Krishnamoorthi Liangzhen Lai Vikas Chandra ALM 30 68 0 22 Feb 2024
OLMo: Accelerating the Science of Language Models Dirk Groeneveld Iz Beltagy Pete Walsh Akshita Bhagia Rodney Michael Kinney ... Jesse Dodge Kyle Lo Luca Soldaini Noah A. Smith Hanna Hajishirzi OSLM 124 349 0 01 Feb 2024
SliceGPT: Compress Large Language Models by Deleting Rows and Columns Saleh Ashkboos Maximilian L. Croci Marcelo Gennari do Nascimento Torsten Hoefler James Hensman VLM 122 143 0 26 Jan 2024
GeoChat: Grounded Large Vision-Language Model for Remote Sensing Kartik Kuckreja M. S. Danish Muzammal Naseer Abhijit Das Salman Khan Fahad Shahbaz Khan 14 130 0 24 Nov 2023
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions Minghao Wu Abdul Waheed Chiyu Zhang Muhammad Abdul-Mageed Alham Fikri Aji ALM 118 115 0 27 Apr 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 198 1,089 0 20 Sep 2022
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks Torsten Hoefler Dan Alistarh Tal Ben-Nun Nikoli Dryden Alexandra Peste MQ 128 679 0 31 Jan 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,791 0 17 Sep 2019