ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT

ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT

30 November 2022

Tong Zhang

Papers citing "ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT"

7 / 7 papers shown

Title
Large Batch Analysis for Adagrad Under Anisotropic Smoothness Yuxing Liu Rui Pan Tong Zhang 26 5 0 21 Jun 2024
Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise Rui Pan Yuxing Liu Xiaoyu Wang Tong Zhang 26 5 0 22 Dec 2023
Data-Efficient French Language Modeling with CamemBERTa Wissam Antoun Benoît Sagot Djamé Seddah 25 7 0 02 Jun 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 30 41 0 07 Apr 2023
Write and Paint: Generative Vision-Language Models are Unified Modal Learners Shizhe Diao Wangchunshu Zhou Xinsong Zhang Jiawei Wang MLLM AI4CE 22 16 0 15 Jun 2022
Black-box Prompt Learning for Pre-trained Language Models Shizhe Diao Zhichao Huang Ruijia Xu Xuechun Li Yong Lin Xiao Zhou Tong Zhang VLM AAML 36 68 0 21 Jan 2022
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 299 6,984 0 20 Apr 2018