LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models

4 March 2022

Papers citing "LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models"

7 / 7 papers shown

Title
Variation Matters: from Mitigating to Embracing Zero-Shot NAS Ranking Function Variation P. Rumiantsev Mark Coates 48 0 0 27 Feb 2025
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers Yi Tay Mostafa Dehghani J. Rao W. Fedus Samira Abnar Hyung Won Chung Sharan Narang Dani Yogatama Ashish Vaswani Donald Metzler 188 110 0 22 Sep 2021
Primer: Searching for Efficient Transformers for Language Modeling David R. So Wojciech Mañke Hanxiao Liu Zihang Dai Noam M. Shazeer Quoc V. Le VLM 83 152 0 17 Sep 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 248 1,986 0 31 Dec 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,453 0 23 Jan 2020
Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution T. Elsken J. H. Metzen Frank Hutter 117 498 0 24 Apr 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018