On the Influence of Masking Policies in Intermediate Pre-training

On the Influence of Masking Policies in Intermediate Pre-training

18 April 2021

Sinong Wang

Hao Ma

Xiang Ren

Madian Khabsa

Papers citing "On the Influence of Masking Policies in Intermediate Pre-training"

5 / 5 papers shown

Title
Salient Span Masking for Temporal Understanding Jeremy R. Cole Aditi Chaudhary Bhuwan Dhingra Partha P. Talukdar 57 11 0 22 Mar 2023
Knowledgeable Salient Span Mask for Enhancing Language Models as Knowledge Base Cunxiang Wang Fuli Luo Yanyang Li Runxin Xu Fei Huang Yue Zhang KELM 39 2 0 17 Apr 2022
Understanding and Improving Sequence-to-Sequence Pretraining for Neural Machine Translation Wenxuan Wang Wenxiang Jiao Yongchang Hao Xing Wang Shuming Shi Zhaopeng Tu Michael Lyu AIMat 39 26 0 16 Mar 2022
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 460 2,592 0 03 Sep 2019
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 523 11,727 0 09 Mar 2017