BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model

11 February 2019

Papers citing "BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model"

44 / 94 papers shown

Title
Step-unrolled Denoising Autoencoders for Text Generation Nikolay Savinov Junyoung Chung Mikolaj Binkowski Erich Elsen Aaron van den Oord DiffM 22 116 0 13 Dec 2021
EdiBERT, a generative model for image editing Thibaut Issenhuth Ugo Tanielian Jérémie Mary David Picard DiffM 35 12 0 30 Nov 2021
Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codes Sam Bond-Taylor P. Hessey Hiroshi Sasaki T. Breckon Chris G. Willcocks DiffM 27 71 0 24 Nov 2021
A Contextual Latent Space Model: Subsequence Modulation in Melodic Sequence Taketo Akama BDL 30 3 0 23 Nov 2021
s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learning Hangbo Bao Li Dong Wenhui Wang Nan Yang Furu Wei 18 11 0 26 Oct 2021
An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models Nicholas Meade Elinor Poole-Dayan Siva Reddy 27 124 0 16 Oct 2021
BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese Nguyen Luong Tran Duong Minh Le Dat Quoc Nguyen 19 52 0 20 Sep 2021
On Language Models for Creoles Heather Lent Emanuele Bugliarello Miryam de Lhoneux Chen Qiu Anders Søgaard 44 20 0 13 Sep 2021
The Impact of Positional Encodings on Multilingual Compression Vinit Ravishankar Anders Søgaard 25 5 0 11 Sep 2021
MvSR-NAT: Multi-view Subset Regularization for Non-Autoregressive Machine Translation Pan Xie Zexian Li Xiaohui Hu 34 11 0 19 Aug 2021
BERT-based distractor generation for Swedish reading comprehension questions using a small-scale dataset Dmytro Kalpakchi Johan Boye 38 20 0 09 Aug 2021
Structured Denoising Diffusion Models in Discrete State-Spaces Jacob Austin Daniel D. Johnson Jonathan Ho Daniel Tarlow Rianne van den Berg DiffM 44 857 0 07 Jul 2021
M6-UFC: Unifying Multi-Modal Controls for Conditional Image Synthesis via Non-Autoregressive Generative Transformers Zhu Zhang Jianxin Ma Chang Zhou Rui Men Zhikang Li Ming Ding Jie Tang Jingren Zhou Hongxia Yang 27 46 0 29 May 2021
Generating abstractive summaries of Lithuanian news articles using a transformer model Lukas Stankevicius M. Lukoševičius 24 2 0 23 Apr 2021
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems E. Razumovskaia Goran Glavaš Olga Majewska Edoardo Ponti Anna Korhonen Ivan Vulić 36 32 0 17 Apr 2021
Does BERT Pretrained on Clinical Notes Reveal Sensitive Data? Eric P. Lehman Sarthak Jain Karl Pichotta Yoav Goldberg Byron C. Wallace OOD MIACV 24 119 0 15 Apr 2021
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models Sam Bond-Taylor Adam Leach Yang Long Chris G. Willcocks VLM TPM 48 485 0 08 Mar 2021
Collaborative Storytelling with Large-scale Neural Language Models Eric Nichols Leo Gao R. Gomez 29 43 0 20 Nov 2020
When Do You Need Billions of Words of Pretraining Data? Yian Zhang Alex Warstadt Haau-Sing Li Samuel R. Bowman 29 136 0 10 Nov 2020
An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution Ryuto Konno Yuichiroh Matsubayashi Shun Kiyono Hiroki Ouchi Ryo Takahashi Kentaro Inui 26 7 0 02 Nov 2020
TweetBERT: A Pretrained Language Representation Model for Twitter Text Analysis Mohiuddin Md Abdul Qudar Vijay K. Mago SSeg 28 35 0 17 Oct 2020
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models Zhengbao Jiang Antonios Anastasopoulos Jun Araki Haibo Ding Graham Neubig HILM KELM 21 139 0 13 Oct 2020
Plan ahead: Self-Supervised Text Planning for Paragraph Completion Task Dongyeop Kang Eduard H. Hovy LRM 42 24 0 11 Oct 2020
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models Nikita Nangia Clara Vania Rasika Bhalerao Samuel R. Bowman 41 645 0 30 Sep 2020
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models Samuel Gehman Suchin Gururangan Maarten Sap Yejin Choi Noah A. Smith 86 1,134 0 24 Sep 2020
X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers Jaemin Cho Jiasen Lu Dustin Schwenk Hannaneh Hajishirzi Aniruddha Kembhavi VLM MLLM 30 102 0 23 Sep 2020
Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related Approaches Juan Cruz-Benito Sanjay Vishwakarma Francisco Martín-Fernández Ismael Faro Ibm Quantum 22 31 0 16 Sep 2020
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 51 956 0 15 Sep 2020
MLMLM: Link Prediction with Mean Likelihood Masked Language Model Louis Clouâtre P. Trempe Amal Zouaq Sarath Chandar 25 43 0 15 Sep 2020
Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation Rasool Fakoor Jonas W. Mueller Nick Erickson Pratik Chaudhari Alex Smola 26 54 0 25 Jun 2020
Enabling Language Models to Fill in the Blanks Chris Donahue Mina Lee Percy Liang 14 195 0 11 May 2020
Commonsense Evidence Generation and Injection in Reading Comprehension Ye Liu Tao Yang Zeyu You Wei Fan Philip S. Yu 33 14 0 11 May 2020
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training Linjie Li Yen-Chun Chen Yu Cheng Zhe Gan Licheng Yu Jingjing Liu MLLM VLM OffRL AI4TS 62 494 0 01 May 2020
Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding Samson Tan Shafiq Joty Lav Varshney Min-Yen Kan 22 34 0 30 Apr 2020
Limits of Detecting Text Generated by Large-Scale Language Models Lav Varshney N. Keskar R. Socher DeLMO 24 18 0 09 Feb 2020
L2RS: A Learning-to-Rescore Mechanism for Automatic Speech Recognition Yuanfeng Song Di Jiang Xuefang Zhao Qian Xu Raymond Chi-Wing Wong Lixin Fan Qiang Yang 29 17 0 25 Oct 2019
XL-Editor: Post-editing Sentences with XLNet Yong-Siang Shih Wei-Cheng Chang Yiming Yang KELM 25 11 0 19 Oct 2019
Summary Level Training of Sentence Rewriting for Abstractive Summarization Sanghwan Bae Taeuk Kim Jihoon Kim Sang-goo Lee 38 68 0 19 Sep 2019
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 85 1,651 0 22 Aug 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 126 3,634 0 06 Aug 2019
Learning Video Representations using Contrastive Bidirectional Transformer Chen Sun Fabien Baradel Kevin Patrick Murphy Cordelia Schmid SSL ViT 27 133 0 13 Jun 2019
A Generalized Framework of Sequence Generation with Application to Undirected Sequence Models Elman Mansimov Alex Jinpeng Wang Sean Welleck Kyunghyun Cho AIMat 28 46 0 29 May 2019
Unified Language Model Pre-training for Natural Language Understanding and Generation Li Dong Nan Yang Wenhui Wang Furu Wei Xiaodong Liu Yu-Chiang Frank Wang Jianfeng Gao M. Zhou H. Hon ELM AI4CE 80 1,551 0 08 May 2019
Mask-Predict: Parallel Decoding of Conditional Masked Language Models Marjan Ghazvininejad Omer Levy Yinhan Liu Luke Zettlemoyer MoE 27 35 0 19 Apr 2019