Masked Structural Growth for 2x Faster Language Model Pre-training

Masked Structural Growth for 2x Faster Language Model Pre-training

4 May 2023

Zheng-Wei Zhang

Yequan Wang

Papers citing "Masked Structural Growth for 2x Faster Language Model Pre-training"

4 / 4 papers shown

Title
STEP: Staged Parameter-Efficient Pre-training for Large Language Models Kazuki Yano Takumi Ito Jun Suzuki LRM 39 1 0 05 Apr 2025
FLM-101B: An Open LLM and How to Train It with $100K Budget$ Xiang Li Yiqun Yao Xin Jiang Xuezhi Fang Xuying Meng ... LI DU Bowen Qin Zheng-Wei Zhang Aixin Sun Yequan Wang 35 21 0 07 Sep 2023
On the Transformer Growth for Progressive BERT Training Xiaotao Gu Liyuan Liu Hongkun Yu Jing Li C. L. P. Chen Jiawei Han VLM 61 49 0 23 Oct 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,003 0 20 Apr 2018