BERT Rediscovers the Classical NLP Pipeline

15 May 2019

Papers citing "BERT Rediscovers the Classical NLP Pipeline"

50 / 218 papers shown

Title
Layer-wise Analysis of a Self-supervised Speech Representation Model Ankita Pasad Ju-Chieh Chou Karen Livescu SSL 26 287 0 10 Jul 2021
A Novel Deep Reinforcement Learning Based Stock Direction Prediction using Knowledge Graph and Community Aware Sentiments Anil Berk Altuner Zeynep Hilal Kilimci AIFin 9 15 0 02 Jul 2021
The MultiBERTs: BERT Reproductions for Robustness Analysis Thibault Sellam Steve Yadlowsky Jason W. Wei Naomi Saphra Alexander DÁmour ... Iulia Turc Jacob Eisenstein Dipanjan Das Ian Tenney Ellie Pavlick 22 93 0 30 Jun 2021
Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning Colin Wei Sang Michael Xie Tengyu Ma 22 96 0 17 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 24 811 0 14 Jun 2021
LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond Daniel Loureiro A. Jorge Jose Camacho-Collados 27 26 0 26 May 2021
A comparative evaluation and analysis of three generations of Distributional Semantic Models Alessandro Lenci Magnus Sahlgren Patrick Jeuniaux Amaru Cuba Gyllensten Martina Miliani 18 50 0 20 May 2021
Compositional Processing Emerges in Neural Networks Solving Math Problems Jacob Russin Roland Fernandez Hamid Palangi Eric Rosen Nebojsa Jojic P. Smolensky Jianfeng Gao 7 13 0 19 May 2021
How is BERT surprised? Layerwise detection of linguistic anomalies Bai Li Zining Zhu Guillaume Thomas Yang Xu Frank Rudzicz 19 31 0 16 May 2021
Understanding by Understanding Not: Modeling Negation in Language Models Arian Hosseini Siva Reddy Dzmitry Bahdanau R. Devon Hjelm Alessandro Sordoni Aaron C. Courville 11 87 0 07 May 2021
Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality Adithya V Ganesan Matthew Matero Aravind Reddy Ravula Huy-Hien Vu H. A. Schwartz 17 35 0 07 May 2021
Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand? William Merrill Yoav Goldberg Roy Schwartz Noah A. Smith 15 67 0 22 Apr 2021
A multilabel approach to morphosyntactic probing Naomi Tachikawa Shapiro Amandalynne Paullada Shane Steinert-Threlkeld 27 10 0 17 Apr 2021
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little Koustuv Sinha Robin Jia Dieuwke Hupkes J. Pineau Adina Williams Douwe Kiela 34 243 0 14 Apr 2021
What's in your Head? Emergent Behaviour in Multi-Task Transformer Models Mor Geva Uri Katz Aviv Ben-Arie Jonathan Berant LRM 30 11 0 13 Apr 2021
DirectProbe: Studying Representations without Classifiers Yichu Zhou Vivek Srikumar 24 27 0 13 Apr 2021
CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation J. Clark Dan Garrette Iulia Turc John Wieting 25 210 0 11 Mar 2021
The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models Go Inoue Bashar Alhafni Nurpeiis Baimukan Houda Bouamor Nizar Habash 30 223 0 11 Mar 2021
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics Vassilina Nikoulina Maxat Tezekbayev Nuradil Kozhakhmet Madina Babazhanova Matthias Gallé Z. Assylbekov 29 8 0 02 Mar 2021
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 258 346 0 01 Feb 2021
Language Modelling as a Multi-Task Problem Leon Weber Jaap Jumelet Elia Bruni Dieuwke Hupkes 15 13 0 27 Jan 2021
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT Benjamin Muller Yanai Elazar Benoît Sagot Djamé Seddah LRM 21 71 0 26 Jan 2021
Learning to Augment for Data-Scarce Domain BERT Knowledge Distillation Lingyun Feng Minghui Qiu Yaliang Li Haitao Zheng Ying Shen 33 10 0 20 Jan 2021
Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing Minh Nguyen Viet Dac Lai Amir Pouran Ben Veyseh Thien Huu Nguyen 44 132 0 09 Jan 2021
Reservoir Transformers Sheng Shen Alexei Baevski Ari S. Morcos Kurt Keutzer Michael Auli Douwe Kiela 27 17 0 30 Dec 2020
Transformer Feed-Forward Layers Are Key-Value Memories Mor Geva R. Schuster Jonathan Berant Omer Levy KELM 11 740 0 29 Dec 2020
CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade Lei Li Yankai Lin Deli Chen Shuhuai Ren Peng Li Jie Zhou Xu Sun 26 51 0 29 Dec 2020
Identifying Depressive Symptoms from Tweets: Figurative Language Enabled Multitask Learning Framework S. Yadav Jainish Chauhan Joy Prakash Sain K. Thirunarayan A. Sheth Jeremiah A. Schumm 8 42 0 12 Nov 2020
When Do You Need Billions of Words of Pretraining Data? Yian Zhang Alex Warstadt Haau-Sing Li Samuel R. Bowman 21 136 0 10 Nov 2020
Semantic and Relational Spaces in Science of Science: Deep Learning Models for Article Vectorisation Diego Kozlowski Jennifer Dusdal Jun Pang A. Zilian 13 13 0 05 Nov 2020
ABNIRML: Analyzing the Behavior of Neural IR Models Sean MacAvaney Sergey Feldman Nazli Goharian Doug Downey Arman Cohan 15 49 0 02 Nov 2020
Rethinking embedding coupling in pre-trained language models Hyung Won Chung Thibault Févry Henry Tsai Melvin Johnson Sebastian Ruder 93 142 0 24 Oct 2020
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling Wenxuan Zhou Kevin Huang Tengyu Ma Jing Huang 11 273 0 21 Oct 2020
Towards Interpreting BERT for Reading Comprehension Based QA Sahana Ramnath Preksha Nema Deep Sahni Mitesh M. Khapra 34 30 0 18 Oct 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 219 608 0 13 Oct 2020
On the Sub-Layer Functionalities of Transformer Decoder Yilin Yang Longyue Wang Shuming Shi Prasad Tadepalli Stefan Lee Zhaopeng Tu 22 27 0 06 Oct 2020
Beyond The Text: Analysis of Privacy Statements through Syntactic and Semantic Role Labeling Yan Shvartzshnaider Ananth Balashankar Vikas Patidar Thomas Wies L. Subramanian 16 4 0 01 Oct 2020
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data Jonathan Pilault Amine Elhattami C. Pal CLL MoE 19 89 0 19 Sep 2020
Weakly supervised one-stage vision and language disease detection using large scale pneumonia and pneumothorax studies Leo K. Tam Xiaosong Wang E. Turkbey Kevin Lu Yuhong Wen Daguang Xu 15 13 0 31 Jul 2020
Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning Shauharda Khadka Estelle Aflalo Mattias Marder Avrech Ben-David Santiago Miret Shie Mannor Tamir Hazan Hanlin Tang Somdeb Majumdar GNN 19 11 0 14 Jul 2020
Revisiting Few-sample BERT Fine-tuning Tianyi Zhang Felix Wu Arzoo Katiyar Kilian Q. Weinberger Yoav Artzi 30 441 0 10 Jun 2020
Syntactic Structure Distillation Pretraining For Bidirectional Encoders A. Kuncoro Lingpeng Kong Daniel Fried Dani Yogatama Laura Rimell Chris Dyer Phil Blunsom 31 33 0 27 May 2020
On the Robustness of Language Encoders against Grammatical Errors Fan Yin Quanyu Long Tao Meng Kai-Wei Chang 31 33 0 12 May 2020
Finding Universal Grammatical Relations in Multilingual BERT Ethan A. Chi John Hewitt Christopher D. Manning 11 151 0 09 May 2020
Towards Transparent and Explainable Attention Models Akash Kumar Mohankumar Preksha Nema Sharan Narasimhan Mitesh M. Khapra Balaji Vasan Srinivasan Balaraman Ravindran 29 99 0 29 Apr 2020
Experience Grounds Language Yonatan Bisk Ari Holtzman Jesse Thomason Jacob Andreas Yoshua Bengio ... Angeliki Lazaridou Jonathan May Aleksandr Nisnevich Nicolas Pinto Joseph P. Turian 19 350 0 21 Apr 2020
SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings Masoud Jalili Sabet Philipp Dufter François Yvon Hinrich Schütze 23 226 0 18 Apr 2020
Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection Shauli Ravfogel Yanai Elazar Hila Gonen Michael Twiton Yoav Goldberg 8 368 0 16 Apr 2020
Towards Evaluating the Robustness of Chinese BERT Classifiers Boxin Wang Boyuan Pan Xin Li Bo-wen Li AAML 26 8 0 07 Apr 2020
Unsupervised Domain Clusters in Pretrained Language Models Roee Aharoni Yoav Goldberg 18 243 0 05 Apr 2020