TinyLlama: An Open-Source Small Language Model

4 January 2024

Wei Lu

Papers citing "TinyLlama: An Open-Source Small Language Model"

50 / 261 papers shown

Title
Intuition-aware Mixture-of-Rank-1-Experts for Parameter Efficient Finetuning Yijiang Liu Rongyu Zhang Huanrui Yang Kurt Keutzer Yuan Du Li Du Shanghang Zhang MoE 36 6 0 13 Apr 2024
NoticIA: A Clickbait Article Summarization Dataset in Spanish Iker García-Ferrero Begoña Altuna 37 2 0 11 Apr 2024
pfl-research: simulation framework for accelerating research in Private Federated Learning Filip Granqvist Congzheng Song Áine Cahill Rogier van Dalen Martin Pelikan Yi Sheng Chan Xiaojun Feng Natarajan Krishnaswami Vojta Jina Mona Chitnis FedML 26 5 0 09 Apr 2024
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies Shengding Hu Yuge Tu Xu Han Chaoqun He Ganqu Cui ... Chaochao Jia Guoyang Zeng Dahai Li Zhiyuan Liu Maosong Sun MoE 38 275 0 09 Apr 2024
Enhancing Clinical Efficiency through LLM: Discharge Note Generation for Cardiac Patients Hyoje Jung Yunha Kim Heejung Choi Hyeram Seo Minkyoung Kim ... Soyoung Ko Byeolhee Kim Suyeon Kim Tae Joon Jun Young-Hak Kim 35 14 0 08 Apr 2024
Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Model Xinrun Du Zhouliang Yu Songyang Gao Ding Pan Yuyang Cheng ... Tianyu Zheng Xinchen Luo Guorui Zhou Wenhu Chen Ge Zhang 46 16 0 05 Apr 2024
Sailor: Open Language Models for South-East Asia Longxu Dou Qian Liu Guangtao Zeng Jia Guo Jiahui Zhou Wei Lu Min-Bin Lin LRM 32 7 0 04 Apr 2024
Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models Taiqiang Wu Chaofan Tao Jiahao Wang Zhe Zhao Ngai Wong ALM 38 14 0 03 Apr 2024
Exploring Backdoor Vulnerabilities of Chat Models Yunzhuo Hao Wenkai Yang Yankai Lin SILM KELM 21 9 0 03 Apr 2024
CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models Xuechen Liang Meiling Tao Yinghui Xia Yiting Xie Jun Wang JingSong Yang LLMAG 21 12 0 02 Apr 2024
Source-Aware Training Enables Knowledge Attribution in Language Models Muhammad Khalifa David Wadden Emma Strubell Honglak Lee Lu Wang Iz Beltagy Hao Peng HILM 34 14 0 01 Apr 2024
Capability-aware Prompt Reformulation Learning for Text-to-Image Generation Jingtao Zhan Qingyao Ai Yiqun Liu Jia Chen Shaoping Ma DiffM 42 4 0 27 Mar 2024
LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning Rui Pan Xiang Liu Shizhe Diao Renjie Pi Jipeng Zhang Chi Han Tong Zhang 33 36 0 26 Mar 2024
MetaAligner: Towards Generalizable Multi-Objective Alignment of Language Models Kailai Yang Zhiwei Liu Qianqian Xie Jimin Huang Tianlin Zhang Sophia Ananiadou 19 14 0 25 Mar 2024
Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inference Han Zhao Min Zhang Wei Zhao Pengxiang Ding Siteng Huang Donglin Wang Mamba 36 65 0 21 Mar 2024
Arcee's MergeKit: A Toolkit for Merging Large Language Models Charles Goddard Shamane Siriwardhana Malikeh Ehghaghi Luke Meyers Vladimir Karpukhin Brian Benedict Mark McQuade Jacob Solawetz MoMe KELM 75 76 0 20 Mar 2024
Semiparametric Token-Sequence Co-Supervision Hyunji Lee Doyoung Kim Jihoon Jun Se June Joo Joel Jang Kyoung-Woon On Minjoon Seo 25 0 0 14 Mar 2024
Mipha: A Comprehensive Overhaul of Multimodal Assistant with Small Language Models Minjie Zhu Yichen Zhu Xin Liu Ning Liu Zhiyuan Xu Chaomin Shen Yaxin Peng Zhicai Ou Feifei Feng Jian Tang VLM 55 20 0 10 Mar 2024
ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment Xiwei Hu Rui Wang Yixiao Fang Bin-Bin Fu Pei Cheng Gang Yu VLM 57 39 0 08 Mar 2024
Embodied Understanding of Driving Scenarios Yunsong Zhou Linyan Huang Qingwen Bu Jia Zeng Tianyu Li Hang Qiu Hongzi Zhu Minyi Guo Yu Qiao Hongyang Li LM&Ro 55 30 0 07 Mar 2024
SaulLM-7B: A pioneering Large Language Model for Law Pierre Colombo T. Pires Malik Boudiaf Dominic Culver Rui Melo ... Andre F. T. Martins Fabrizio Esposito Vera Lúcia Raposo Sofia Morgado Michael Desa ELM AILaw 39 63 0 06 Mar 2024
CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following Kaiyan Zhang Jianyu Wang Ermo Hua Biqing Qi Ning Ding Bowen Zhou SyDa 30 20 0 05 Mar 2024
Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge Heydar Soudani Evangelos Kanoulas Faegheh Hasibi 21 27 0 03 Mar 2024
Accelerating Greedy Coordinate Gradient via Probe Sampling Yiran Zhao Wenyue Zheng Tianle Cai Xuan Long Do Kenji Kawaguchi Anirudh Goyal Michael Shieh 36 11 0 02 Mar 2024
Large Language Models and Games: A Survey and Roadmap Roberto Gallotta Graham Todd Marvin Zammit Sam Earle Antonios Liapis Julian Togelius Georgios N. Yannakakis LLMAG LM&MA AI4CE LRM 45 71 0 28 Feb 2024
MobiLlama: Towards Accurate and Lightweight Fully Transparent GPT Omkar Thawakar Ashmal Vayani Salman Khan Hisham Cholakal Rao M. Anwer M. Felsberg Timothy Baldwin Eric P. Xing Fahad Shahbaz Khan 46 31 0 26 Feb 2024
RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answering Zihan Zhang Meng Fang Ling-Hao Chen RALM 54 11 0 26 Feb 2024
MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases Zechun Liu Changsheng Zhao Forrest N. Iandola Chen Lai Yuandong Tian ... Ernie Chang Yangyang Shi Raghuraman Krishnamoorthi Liangzhen Lai Vikas Chandra ALM 38 68 0 22 Feb 2024
Analysing The Impact of Sequence Composition on Language Model Pre-Training Yu Zhao Yuanbin Qu Konrad Staniszewski Szymon Tworkowski Wei Liu Piotr Milo's Yuxiang Wu Pasquale Minervini 29 13 0 21 Feb 2024
Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding Weilin Zhao Yuxiang Huang Xu Han Wang Xu Chaojun Xiao Xinrong Zhang Yewei Fang Kaihuo Zhang Zhiyuan Liu Maosong Sun 35 10 0 21 Feb 2024
Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs Jiejun Tan Zhicheng Dou Yutao Zhu Peidong Guo Kun Fang Ji-Rong Wen 37 23 0 19 Feb 2024
Self-AMPLIFY: Improving Small Language Models with Self Post Hoc Explanations Milan Bhan Jean-Noël Vittaut N. Chesneau Marie-Jeanne Lesot ReLM LRM 16 3 0 19 Feb 2024
Revisiting Knowledge Distillation for Autoregressive Language Models Qihuang Zhong Liang Ding Li Shen Juhua Liu Bo Du Dacheng Tao KELM 39 15 0 19 Feb 2024
ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao LM&MA 37 22 0 19 Feb 2024
Efficient Multimodal Learning from Data-centric Perspective Muyang He Yexin Liu Boya Wu Jianhao Yuan Yueze Wang Tiejun Huang Bo-Lu Zhao MLLM 30 82 0 18 Feb 2024
OneBit: Towards Extremely Low-bit Large Language Models Yuzhuang Xu Xu Han Zonghan Yang Shuo Wang Qingfu Zhu Zhiyuan Liu Weidong Liu Wanxiang Che MQ 51 36 0 17 Feb 2024
DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows Ajay Patel Colin Raffel Chris Callison-Burch SyDa AI4CE 22 25 0 16 Feb 2024
Into the Unknown: Self-Learning Large Language Models Teddy Ferdinan Jan Kocoñ P. Kazienko 20 2 0 14 Feb 2024
An Embarrassingly Simple Approach for LLM with Strong ASR Capacity Ziyang Ma Guanrou Yang Yifan Yang Zhifu Gao Jiaming Wang ... Fan Yu Qian Chen Siqi Zheng Shiliang Zhang Xie Chen AuLLM 47 37 0 13 Feb 2024
Large Language Models: A Survey Shervin Minaee Tomáš Mikolov Narjes Nikzad M. Asgari-Chenaghlu R. Socher Xavier Amatriain Jianfeng Gao ALM LM&MA ELM 117 353 0 09 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 126 106 0 08 Feb 2024
RevOrder: A Novel Method for Enhanced Arithmetic in Language Models Si Shen Peijun Shen Danhao Zhu MU 22 1 0 06 Feb 2024
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding Yichao Fu Peter Bailis Ion Stoica Hao Zhang 123 134 0 03 Feb 2024
Tiny Titans: Can Smaller Large Language Models Punch Above Their Weight in the Real World for Meeting Summarization? Xue-Yong Fu Md Tahmid Rahman Laskar Elena Khasanova Cheng-Hsiung Chen TN ShashiBhushan ALM 14 20 0 01 Feb 2024
CroissantLLM: A Truly Bilingual French-English Language Model Manuel Faysse Patrick Fernandes Nuno M. Guerreiro António Loison Duarte M. Alves ... François Yvon André F.T. Martins Gautier Viaud C´eline Hudelot Pierre Colombo 41 33 0 01 Feb 2024
H2O-Danube-1.8B Technical Report Philipp Singer Pascal Pfeiffer Yauhen Babakhin Maximilian Jeblick Nischay Dhankhar Gabor Fodor SriSatish Ambati VLM 18 8 0 30 Jan 2024
TeenyTinyLlama: open-source tiny language models trained in Brazilian Portuguese N. Corrêa Sophia Falk Shiza Fatimah Aniket Sen N. D. Oliveira 20 9 0 30 Jan 2024
Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling Pratyush Maini Skyler Seto Richard He Bai David Grangier Yizhe Zhang Navdeep Jaitly SyDa 33 54 0 29 Jan 2024
OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models Fuzhao Xue Zian Zheng Yao Fu Jinjie Ni Zangwei Zheng Wangchunshu Zhou Yang You MoE 15 87 0 29 Jan 2024
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 26 116 0 26 Jan 2024