Title
Deploying Foundation Model Powered Agent Services: A Survey Wenchao Xu Jinyu Chen Peirong Zheng Xiaoquan Yi Tianyi Tian ... Quan Wan Haozhao Wang Yunfeng Fan Qinliang Su Xuemin Shen AI4CE 102 1 0 18 Dec 2024
On the Inductive Bias of Stacking Towards Improving Reasoning Nikunj Saunshi Stefani Karp Shankar Krishnan Sobhan Miryoosefi Sashank J. Reddi Sanjiv Kumar LRM AI4CE 16 2 0 27 Sep 2024
Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization Mohammad Samragh Iman Mirzadeh Keivan Alizadeh Vahid Fartash Faghri Minsik Cho Moin Nabi Devang Naik Mehrdad Farajtabar LRM AI4CE 18 2 0 19 Sep 2024
Sketch: A Toolkit for Streamlining LLM Operations Xin Jiang Xiang Li Wenjia Ma Xuezhi Fang Yiqun Yao ... Xuying Meng Peng Han Jing Li Aixin Sun Yequan Wang 27 0 0 05 Sep 2024
A Mean Field Ansatz for Zero-Shot Weight Transfer Xingyuan Chen Wenwei Kuang Lei Deng Wei Han Bo Bai Goncalo dos Reis 26 1 0 16 Aug 2024
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning Prateek Yadav Colin Raffel Mohammed Muqeeth Lucas Page-Caccia Haokun Liu Tianlong Chen Mohit Bansal Leshem Choshen Alessandro Sordoni MoMe 23 12 0 13 Aug 2024
52B to 1T: Lessons Learned via Tele-FLM Series Xiang Li Yiqun Yao Xin Jiang Xuezhi Fang Chao Wang ... Yequan Wang Zhongjiang He Zhongyuan Wang Xuelong Li Tiejun Huang ALM LRM 26 1 0 03 Jul 2024
Landscape-Aware Growing: The Power of a Little LAG Stefani Karp Nikunj Saunshi Sobhan Miryoosefi Sashank J. Reddi Sanjiv Kumar 35 1 0 04 Jun 2024
Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training Wenyu Du Tongxu Luo Zihan Qiu Zeyu Huang Yikang Shen Reynold Cheng Yike Guo Jie Fu 21 4 0 24 May 2024
Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicity Rheeya Uppaal Apratim De Yiting He Yiquao Zhong Junjie Hu 21 7 0 22 May 2024
Tele-FLM Technical Report Xiang Li Yiqun Yao Xin Jiang Xuezhi Fang Chao Wang ... Yequan Wang Zhongjiang He Zhongyuan Wang Xuelong Li Tiejun Huang 22 3 0 25 Apr 2024
Not All Layers of LLMs Are Necessary During Inference Siqi Fan Xin Jiang Xiang Li Xuying Meng Peng Han Shuo Shang Aixin Sun Yequan Wang Zhongyuan Wang 30 19 0 04 Mar 2024
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models Bin Lin Zhenyu Tang Yang Ye Jiaxi Cui Bin Zhu ... Jinfa Huang Junwu Zhang Yatian Pang Munan Ning Li-ming Yuan VLM MLLM MoE 20 146 0 29 Jan 2024
Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective Ming Zhong Chenxin An Weizhu Chen Jiawei Han Pengcheng He 11 7 0 17 Oct 2023
Compressing LLMs: The Truth is Rarely Pure and Never Simple Ajay Jaiswal Zhe Gan Xianzhi Du Bowen Zhang Zhangyang Wang Yinfei Yang MQ 18 29 0 02 Oct 2023
GrowLength: Accelerating LLMs Pretraining by Progressively Growing Training Length Hongye Jin Xiaotian Han Jingfeng Yang Zhimeng Jiang Chia-Yuan Chang Xia Hu 25 10 0 01 Oct 2023
Deep Fusion: Efficient Network Training via Pre-trained Initializations Hanna Mazzawi X. Gonzalvo Michael Wunder Sammy Jerome Benoit Dherin AI4CE 16 3 0 20 Jun 2023
Masked Structural Growth for 2x Faster Language Model Pre-training Yiqun Yao Zheng-Wei Zhang Jing Li Yequan Wang OffRL AI4CE LRM 32 9 0 04 May 2023
FreeLM: Fine-Tuning-Free Language Model Xiang Li Xin Jiang Xuying Meng Aixin Sun Yequan Wang 31 2 0 02 May 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 197 2,232 0 22 Mar 2023
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing Xiaozhe Ren Pingyi Zhou Xinfan Meng Xinjing Huang Yadao Wang ... Jiansheng Wei Xin Jiang Teng Su Qun Liu Jun Yao ALM MoE 42 59 0 20 Mar 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 234 840 0 05 Oct 2022
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 211 413 0 28 Sep 2022
CofeNet: Context and Former-Label Enhanced Net for Complicated Quotation Extraction Yequan Wang Xiang Li Aixin Sun Xuying Meng Huaming Liao J. Guo 13 1 0 20 Sep 2022
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 209 327 0 23 Aug 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 8,441 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 309 5,177 0 28 Jan 2022
On the Transformer Growth for Progressive BERT Training Xiaotao Gu Liyuan Liu Hongkun Yu Jing Li C. L. P. Chen Jiawei Han VLM 61 49 0 23 Oct 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 217 3,054 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,436 0 17 Sep 2019

FLM-101B: An Open LLM and How to Train It with 100KBudget100K Budget100KBudget

Papers citing "FLM-101B: An Open LLM and How to Train It with $100K Budget"

FLM-101B: An Open LLM and How to Train It with $100K Budget$