Reusing Pretrained Models by Multi-linear Operators for Efficient
Training

Reusing Pretrained Models by Multi-linear Operators for Efficient Training

16 October 2023

Lifeng Shang

Xin Jiang

Qun Liu

Papers citing "Reusing Pretrained Models by Multi-linear Operators for Efficient Training"

8 / 8 papers shown

Title
Benchmarking Ultra-Low-Power $μ$ NPUs Josh Millar Yushan Huang Sarab Sethi Hamed Haddadi Anil Madhavapeddy BDL 48 0 0 28 Mar 2025
Tensor Networks Meet Neural Networks: A Survey and Future Perspectives Maolin Wang Y. Pan Zenglin Xu Xiangli Yang Guangxi Li A. Cichocki Andrzej Cichocki 33 19 0 22 Jan 2023
Exploring Low Rank Training of Deep Neural Networks Siddhartha Rao Kamalakara Acyr F. Locatelli Bharat Venkitesh Jimmy Ba Y. Gal Aidan N. Gomez 40 22 0 27 Sep 2022
Firefly Neural Architecture Descent: a General Approach for Growing Neural Networks Lemeng Wu Bo Liu Peter Stone Qiang Liu 49 45 0 17 Feb 2021
On the Transformer Growth for Progressive BERT Training Xiaotao Gu Liyuan Liu Hongkun Yu Jing Li C. L. P. Chen Jiawei Han VLM 61 49 0 23 Oct 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 3,054 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,791 0 17 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018