Word Order Matters when you Increase Masking

8 November 2022

Papers citing "Word Order Matters when you Increase Masking"

6 / 6 papers shown

Title
The Role of Linguistic Priors in Measuring Compositional Generalization of Vision-Language Models Chenwei Wu Erran L. Li Stefano Ermon Patrick Haffner Rong Ge Zaiwei Zhang VLM CoGe 19 0 0 04 Oct 2023
Dynamic Masking Rate Schedules for MLM Pretraining Zachary Ankner Naomi Saphra Davis W. Blalock Jonathan Frankle Matthew L. Leavitt 17 5 0 24 May 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 27 100 0 20 Mar 2023
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 237 690 0 27 Aug 2021
A Simple and Effective Positional Encoding for Transformers Pu-Chin Chen Henry Tsai Srinadh Bhojanapalli Hyung Won Chung Yin-Wen Chang Chun-Sung Ferng 46 61 0 18 Apr 2021
Out of Order: How Important Is The Sequential Order of Words in a Sentence in Natural Language Understanding Tasks? Thang M. Pham Trung Bui Long Mai Anh Totti Nguyen 195 122 0 30 Dec 2020