Small Language Models: Survey, Measurements, and Insights

24 September 2024

Zhenyan Lu

Xiang Li

Dongqi Cai

Rongjie Yi

Fangming Liu

Xiwen Zhang

Nicholas D. Lane

Mengwei Xu

ObjD

LRM

ArXiv PDF HTML

Papers citing "Small Language Models: Survey, Measurements, and Insights"

23 / 23 papers shown

Title
LSRP: A Leader-Subordinate Retrieval Framework for Privacy-Preserving Cloud-Device Collaboration Y. Zhang Pengyue Jia X. Li Derong Xu Maolin Wang ... Zhaocheng Du Huifeng Guo Y. Liu Ruiming Tang Xiangyu Zhao 30 0 0 08 May 2025
RobotxR1: Enabling Embodied Robotic Intelligence on Large Language Models through Closed-Loop Reinforcement Learning Liam Boyle Nicolas Baumann Paviththiren Sivasothilingam Michele Magno Luca Benini LM&Ro LRM 34 0 0 06 May 2025
Position: Enough of Scaling LLMs! Lets Focus on Downscaling Ayan Sengupta Yash Goel Tanmoy Chakraborty 34 0 0 02 May 2025
The Rise of Small Language Models in Healthcare: A Comprehensive Survey Muskan Garg Shaina Raza Shebuti Rayana Xingyi Liu Sunghwan Sohn LM&MA AILaw 87 0 0 23 Apr 2025
Enhancing Autonomous Driving Systems with On-Board Deployed Large Language Models Nicolas Baumann Cheng Hu Paviththiren Sivasothilingam Haotong Qin Lei Xie Michele Magno Luca Benini 24 1 0 15 Apr 2025
Cat, Rat, Meow: On the Alignment of Language Model and Human Term-Similarity Judgments Lorenz Linhardt Tom Neuhäuser Lenka Tětková Oliver Eberle ALM AI4TS 37 0 0 10 Apr 2025
T1: Tool-integrated Self-verification for Test-time Compute Scaling in Small Language Models Minki Kang Jongwon Jeong Jaewoong Cho ALM LRM 35 2 0 07 Apr 2025
Does Chain-of-Thought Reasoning Help Mobile GUI Agent? An Empirical Study Li Lyna Zhang Longxi Gao Mengwei Xu LRM 32 0 0 21 Mar 2025
PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing Cheng Deng Luoyang Sun Jiwen Jiang Yongcheng Zeng Xinjian Wu ... Haoyang Li Lei Chen Lionel M. Ni H. Zhang Jun Wang 61 0 0 15 Mar 2025
Unveiling the Mathematical Reasoning in DeepSeek Models: A Comparative Study of Large Language Models Afrar Jahin Arif Hassan Zidan Yu Bao Shizhe Liang T. Liu W. Zhang LRM 61 1 0 13 Mar 2025
Cost-Optimal Grouped-Query Attention for Long-Context LLMs Y. Chen Yutong Wu Xu Han Zhiyuan Liu Maosong Sun 56 0 0 12 Mar 2025
Can Small Language Models Reliably Resist Jailbreak Attacks? A Comprehensive Evaluation Wenhui Zhang Huiyu Xu Zhibo Wang Zeqing He Ziqi Zhu Kui Ren AAML PILM 64 0 0 09 Mar 2025
GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devices Xudong Lu Yinghao Chen Renshou Wu Haohao Gao Xi Chen ... Fangyuan Li Yafei Wen Xiaoxin Chen Shuai Ren Hongsheng Li 68 0 0 08 Mar 2025
GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Development Leming Shen Qiang Yang Xinyu Huang Zijing Ma Yuanqing Zheng 22 1 0 02 Mar 2025
From Small to Large Language Models: Revisiting the Federalist Papers So Won Jeong Veronika Rockova 34 0 0 25 Feb 2025
Towards Reasoning Ability of Small Language Models Gaurav Srivastava Shuxiang Cao Xuan Wang ReLM LRM 49 4 0 17 Feb 2025
A Survey of Personalized Large Language Models: Progress and Future Directions Jiahong Liu Zexuan Qiu Zhongyang Li Quanyu Dai Jieming Zhu Minda Hu Menglin Yang Irwin King LM&MA 46 2 0 17 Feb 2025
Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance Borui Xu Yao Chen Zeyi Wen Weiguo Liu Bingsheng He 57 1 0 02 Feb 2025
Language and Planning in Robotic Navigation: A Multilingual Evaluation of State-of-the-Art Models Malak Mansour Ahmed Aly Bahey Tharwat Sarim Hashmi Dong An Ian Reid LM&Ro ELM LRM 51 0 0 07 Jan 2025
Fineweb-Edu-Ar: Machine-translated Corpus to Support Arabic Small Language Models Sultan Alrashed Dmitrii Khizbullin David R. Pugh 28 0 0 10 Nov 2024
MiniPLM: Knowledge Distillation for Pre-Training Language Models Yuxian Gu Hao Zhou Fandong Meng Jie Zhou Minlie Huang 46 5 0 22 Oct 2024
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices Yuxiang Huang Binhang Yuan Xu Han Chaojun Xiao Zhiyuan Liu RALM 64 1 0 02 Oct 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 48 23 0 10 Sep 2024