Bag of Tricks for Effective Language Model Pretraining and Downstream
Adaptation: A Case Study on GLUE

Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE

18 February 2023

Liang Ding

Bo Du

Li Shen

Yibing Zhan

Papers citing "Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE"

6 / 6 papers shown

Title
Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer Boan Liu Liang Ding Li Shen Keqin Peng Yu Cao Dazhao Cheng Dacheng Tao MoE 34 7 0 15 Oct 2023
Revisiting Token Dropping Strategy in Efficient BERT Pretraining Qihuang Zhong Liang Ding Juhua Liu Xuebo Liu Min Zhang Bo Du Dacheng Tao VLM 27 9 0 24 May 2023
Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao AI4MH 47 231 0 19 Feb 2023
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao 29 26 0 30 May 2022
Interpretable Proof Generation via Iterative Backward Reasoning Hanhao Qu Yu Cao Jun Gao Liang Ding Ruifeng Xu LRM 43 15 0 22 May 2022
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018