Weighted Sampling for Masked Language Modeling

Weighted Sampling for Masked Language Modeling

28 February 2023

Qian Chen

Wen Wang

Papers citing "Weighted Sampling for Masked Language Modeling"

6 / 6 papers shown

Title
TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning Yixuan Su Fangyu Liu Zaiqiao Meng Tian Lan Lei Shu Ehsan Shareghi Nigel Collier 137 57 0 07 Nov 2021
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining Yu Meng Chenyan Xiong Payal Bajaj Saurabh Tiwary Paul N. Bennett Jiawei Han Xia Song 125 202 0 16 Feb 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,918 0 31 Dec 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,588 0 21 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,817 0 17 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018