AutoBERT-Zero: Evolving BERT Backbone from Scratch

15 July 2021

Jiahui Gao

Hang Xu

Han Shi

Xiaozhe Ren

Philip L. H. Yu

Xiaodan Liang

Xin Jiang

Zhenguo Li

ArXiv PDF HTML

Papers citing "AutoBERT-Zero: Evolving BERT Backbone from Scratch"

24 / 24 papers shown

Title
LLM-Virus: Evolutionary Jailbreak Attack on Large Language Models Miao Yu Junfeng Fang Yingjie Zhou Xing Fan Kun Wang Shirui Pan Qingsong Wen AAML 61 0 0 03 Jan 2025
An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms Leonardo Lucio Custode Fabio Caraffini Anil Yaman Giovanni Iacca 35 2 0 05 Aug 2024
When Large Language Model Meets Optimization Sen Huang Kaixiang Yang Sheng Qi Rui Wang 37 9 0 16 May 2024
Efficiently Distilling LLMs for Edge Applications Achintya Kundu Fabian Lim Aaron Chew L. Wynter Penny Chong Rhui Dih Lee 37 6 0 01 Apr 2024
Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap Xingyu Wu Sheng-hao Wu Jibin Wu Liang Feng Kay Chen Tan ELM 34 57 0 18 Jan 2024
A Survey of Techniques for Optimizing Transformer Inference Krishna Teja Chitty-Venkata Sparsh Mittal M. Emani V. Vishwanath Arun Somani 31 62 0 16 Jul 2023
AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks Alexander Tornede Difan Deng Theresa Eimer Joseph Giovanelli Aditya Mohan ... Sarah Segel Daphne Theodorakopoulos Tanja Tornede Henning Wachsmuth Marius Lindauer 28 22 0 13 Jun 2023
Training-free Neural Architecture Search for RNNs and Transformers Aaron Serianni Jugal Kalita 20 7 0 01 Jun 2023
DetGPT: Detect What You Need via Reasoning Renjie Pi Jiahui Gao Shizhe Diao Rui Pan Hanze Dong ... Lewei Yao Jianhua Han Hang Xu Lingpeng Kong Tong Zhang Tong Zhang LRM LM&Ro 22 92 0 23 May 2023
EdgeTran: Co-designing Transformers for Efficient Inference on Mobile Edge Platforms Shikhar Tuli N. Jha 34 3 0 24 Mar 2023
Probabilistic Bilevel Coreset Selection Xiao Zhou Renjie Pi Weizhong Zhang Yong Lin Tong Zhang NoLa 15 27 0 24 Jan 2023
Model Agnostic Sample Reweighting for Out-of-Distribution Learning Xiao Zhou Yong Lin Renjie Pi Weizhong Zhang Renzhe Xu Peng Cui Tong Zhang OODD 28 60 0 24 Jan 2023
Shortest Edit Path Crossover: A Theory-driven Solution to the Permutation Problem in Evolutionary Neural Architecture Search Xin Qiu Risto Miikkulainen 18 2 0 25 Oct 2022
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation Ganesh Jawahar Subhabrata Mukherjee Xiaodong Liu Young Jin Kim Muhammad Abdul-Mageed L. Lakshmanan Ahmed Hassan Awadallah Sébastien Bubeck Jianfeng Gao MoE 22 5 0 14 Oct 2022
Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning Jiahui Gao Renjie Pi Yong Lin Hang Xu Jiacheng Ye Zhiyong Wu Weizhong Zhang Xiaodan Liang Zhenguo Li Lingpeng Kong SyDa VLM 57 45 0 25 May 2022
FlexiBERT: Are Current Transformer Architectures too Homogeneous and Rigid? Shikhar Tuli Bhishma Dedhia Shreshth Tuli N. Jha 16 14 0 23 May 2022
LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models Mojan Javaheripi Gustavo de Rosa Subhabrata Mukherjee S. Shah Tomasz Religa C. C. T. Mendes Sébastien Bubeck F. Koushanfar Debadeepta Dey 23 18 0 04 Mar 2022
AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Models Xiaofan Zhang Zongwei Zhou Deming Chen Yu Emma Wang 20 11 0 21 Jan 2022
SuperShaper: Task-Agnostic Super Pre-training of BERT Models with Variable Hidden Dimensions Vinod Ganesan Gowtham Ramesh Pratyush Kumar 23 9 0 10 Oct 2021
Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation Lewei Yao Renjie Pi Hang Xu Wei Zhang Zhenguo Li Tong Zhang 74 38 0 27 May 2021
NAS-Navigator: Visual Steering for Explainable One-Shot Deep Neural Network Synthesis Anjul Tyagi C. Xie Klaus Mueller 8 6 0 28 Sep 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,950 0 20 Apr 2018
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 264 5,326 0 05 Nov 2016
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,740 0 26 Sep 2016