What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models

31 July 2019

Papers citing "What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models"

42 / 92 papers shown

Title
Probing Simile Knowledge from Pre-trained Language Models Weijie Chen Yongzhu Chang Rongsheng Zhang Jiashu Pu Guandan Chen Le Zhang Yadong Xi Yijiang Chen Chang Su 16 11 0 27 Apr 2022
minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models Kanishka Misra 19 58 0 24 Mar 2022
How does the pre-training objective affect what large language models learn about linguistic properties? Ahmed Alajrami Nikolaos Aletras 23 20 0 20 Mar 2022
Geographic Adaptation of Pretrained Language Models Valentin Hofmann Goran Glavavs Nikola Ljubevsić J. Pierrehumbert Hinrich Schütze VLM 21 16 0 16 Mar 2022
Neural reality of argument structure constructions Bai Li Zining Zhu Guillaume Thomas Frank Rudzicz Yang Xu 38 26 0 24 Feb 2022
Probing BERT's priors with serial reproduction chains Takateru Yamakoshi Thomas L. Griffiths Robert D. Hawkins 18 12 0 24 Feb 2022
Commonsense Knowledge Reasoning and Generation with Pre-trained Language Models: A Survey Prajjwal Bhargava Vincent Ng ReLM LRM 36 62 0 28 Jan 2022
Few-shot Named Entity Recognition with Cloze Questions V. Gatta V. Moscato Marco Postiglione Giancarlo Sperlí 16 4 0 24 Nov 2021
Using Distributional Principles for the Semantic Study of Contextual Language Models Olivier Ferret 17 1 0 23 Nov 2021
Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey Bonan Min Hayley L Ross Elior Sulem Amir Pouran Ben Veyseh Thien Huu Nguyen Oscar Sainz Eneko Agirre Ilana Heinz Dan Roth LM&MA VLM AI4CE 71 1,029 0 01 Nov 2021
Double Trouble: How to not explain a text classifier's decisions using counterfactuals synthesized by masked language models? Thang M. Pham Trung H. Bui Long Mai Anh Totti Nguyen 21 7 0 22 Oct 2021
ALL Dolphins Are Intelligent and SOME Are Friendly: Probing BERT for Nouns' Semantic Properties and their Prototypicality Marianna Apidianaki Aina Garí Soler 28 18 0 12 Oct 2021
Analysing the Effect of Masking Length Distribution of MLM: An Evaluation Framework and Case Study on Chinese MRC Datasets Changchang Zeng Shaobo Li 16 6 0 29 Sep 2021
AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses Yaman Kumar Singla Swapnil Parekh Somesh Singh J. Li R. Shah Changyou Chen AAML 27 14 0 24 Sep 2021
Awakening Latent Grounding from Pretrained Language Models for Semantic Parsing Qian Liu Dejian Yang Jiahui Zhang Jiaqi Guo Bin Zhou Jian-Guang Lou 48 41 0 22 Sep 2021
Do Prompt-Based Models Really Understand the Meaning of their Prompts? Albert Webson Ellie Pavlick LRM 30 352 0 02 Sep 2021
Differentiable Subset Pruning of Transformer Heads Jiaoda Li Ryan Cotterell Mrinmaya Sachan 37 53 0 10 Aug 2021
Local Structure Matters Most: Perturbation Study in NLU Louis Clouâtre Prasanna Parthasarathi Amal Zouaq Sarath Chandar 22 13 0 29 Jul 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 25 3,828 0 28 Jul 2021
Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude? J. Michaelov Megan D. Bardolph S. Coulson Benjamin Bergen 11 22 0 20 Jul 2021
Automatic Construction of Evaluation Suites for Natural Language Generation Datasets Simon Mille Kaustubh D. Dhole Saad Mahamood Laura Perez-Beltrachini Varun Gangal Mihir Kale Emiel van Miltenburg Sebastian Gehrmann ELM 34 22 0 16 Jun 2021
BERT Embeddings for Automatic Readability Assessment Joseph Marvin Imperial 8 36 0 15 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 37 813 0 14 Jun 2021
How is BERT surprised? Layerwise detection of linguistic anomalies Bai Li Zining Zhu Guillaume Thomas Yang Xu Frank Rudzicz 27 31 0 16 May 2021
Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema Yanai Elazar Hongming Zhang Yoav Goldberg Dan Roth ReLM LRM 37 44 0 16 Apr 2021
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little Koustuv Sinha Robin Jia Dieuwke Hupkes J. Pineau Adina Williams Douwe Kiela 43 243 0 14 Apr 2021
Bertinho: Galician BERT Representations David Vilares Marcos Garcia Carlos Gómez-Rodríguez 57 22 0 25 Mar 2021
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 260 346 0 01 Feb 2021
HateCheck: Functional Tests for Hate Speech Detection Models Paul Röttger B. Vidgen Dong Nguyen Zeerak Talat Helen Z. Margetts J. Pierrehumbert 29 259 0 31 Dec 2020
When Do You Need Billions of Words of Pretraining Data? Yian Zhang Alex Warstadt Haau-Sing Li Samuel R. Bowman 21 136 0 10 Nov 2020
Dynamic Contextualized Word Embeddings Valentin Hofmann J. Pierrehumbert Hinrich Schütze 29 51 0 23 Oct 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 219 608 0 13 Oct 2020
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 22 953 0 15 Sep 2020
Analysis and Evaluation of Language Models for Word Sense Disambiguation Daniel Loureiro Kiamehr Rezaee Mohammad Taher Pilehvar Jose Camacho-Collados 10 13 0 26 Aug 2020
Word meaning in minds and machines Brenden Lake G. Murphy NAI 15 117 0 04 Aug 2020
BERTology Meets Biology: Interpreting Attention in Protein Language Models Jesse Vig Ali Madani L. Varshney Caiming Xiong R. Socher Nazneen Rajani 15 288 0 26 Jun 2020
The Sensitivity of Language Models and Humans to Winograd Schema Perturbations Mostafa Abdou Vinit Ravishankar Maria Barrett Yonatan Belinkov Desmond Elliott Anders Søgaard ReLM LRM 54 34 0 04 May 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 243 1,450 0 18 Mar 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,587 0 21 Jan 2020
oLMpics -- On what Language Model Pre-training Captures Alon Talmor Yanai Elazar Yoav Goldberg Jonathan Berant LRM 17 300 0 31 Dec 2019
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 199 882 0 03 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,950 0 20 Apr 2018