What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models

31 July 2019

Papers citing "What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models"

50 / 92 papers shown

Title
Do Large Language Models know who did what to whom? Joseph M. Denning Xiaohan Bryor Snefjella Idan A. Blank 52 1 0 23 Apr 2025
From No to Know: Taxonomy, Challenges, and Opportunities for Negation Understanding in Multimodal Foundation Models Mayank Vatsa Aparna Bharati S. Mittal Richa Singh 53 0 0 10 Feb 2025
Calling a Spade a Heart: Gaslighting Multimodal Large Language Models via Negation Bin Zhu Hui yan Qi Yinxuan Gui Jingjing Chen Chong-Wah Ngo Ee-Peng Lim 101 1 0 31 Jan 2025
Does ChatGPT Have a Mind? Simon Goldstein B. Levinstein AI4MH LRM 34 5 0 27 Jun 2024
Paraphrasing in Affirmative Terms Improves Negation Understanding MohammadHossein Rezaei Eduardo Blanco 42 1 0 11 Jun 2024
Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence Abhinav Patil Jaap Jumelet Yu Ying Chiu Andy Lapastora Peter Shen Lexie Wang Clevis Willrich Shane Steinert-Threlkeld 27 13 0 24 May 2024
Natural Language Processing RELIES on Linguistics Juri Opitz Shira Wein Nathan Schneider AI4CE 44 7 0 09 May 2024
Influence of Solution Efficiency and Valence of Instruction on Additive and Subtractive Solution Strategies in Humans and GPT-4 Lydia Uhler Verena Jordan Jürgen Buder Markus Huff F. Papenmeier 38 0 0 25 Apr 2024
Language in Vivo vs. in Silico: Size Matters but Larger Language Models Still Do Not Comprehend Language on a Par with Humans Vittoria Dentella Fritz Guenther Evelina Leivada ELM 41 1 0 23 Apr 2024
Should We Fear Large Language Models? A Structural Analysis of the Human Reasoning System for Elucidating LLM Capabilities and Risks Through the Lens of Heidegger's Philosophy Jianqiiu Zhang ELM 35 1 0 05 Mar 2024
Do Large Language Models Mirror Cognitive Language Processing? Yuqi Ren Renren Jin Tongxuan Zhang Deyi Xiong 42 4 0 28 Feb 2024
Uncovering Intermediate Variables in Transformers using Circuit Probing Michael A. Lepori Thomas Serre Ellie Pavlick 72 7 0 07 Nov 2023
Disentangling the Linguistic Competence of Privacy-Preserving BERT Stefan Arnold Nils Kemmerzell Annika Schreiner 25 0 0 17 Oct 2023
Agentività e telicità in GilBERTo: implicazioni cognitive A. Lombardi Alessandro Lenci 18 1 0 06 Jul 2023
Morphosyntactic probing of multilingual BERT models Judit Ács Endre Hamerlik Roy Schwartz Noah A. Smith András Kornai 27 9 0 09 Jun 2023
Prompt to be Consistent is Better than Self-Consistent? Few-Shot and Zero-Shot Fact Verification with Pre-trained Language Models Fengzhu Zeng Wei Gao 17 5 0 05 Jun 2023
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap Q. V. Liao J. Vaughan 36 158 0 02 Jun 2023
Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivity Lindia Tjuatja Emmy Liu Lori S. Levin Graham Neubig 30 2 0 29 May 2023
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models Yuhui Zhang Michihiro Yasunaga Zhengping Zhou Jeff Z. HaoChen James Y. Zou Percy Liang Serena Yeung 39 7 0 27 May 2023
Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarization Dongqi Pu Yifa Wang Vera Demberg 29 21 0 26 May 2023
Not wacky vs. definitely wacky: A study of scalar adverbs in pretrained language models Isabelle Lorge J. Pierrehumbert 36 0 0 25 May 2023
NevIR: Negation in Neural Information Retrieval Orion Weller Dawn J Lawrie Benjamin Van Durme MU 59 18 0 12 May 2023
Rethinking Visual Prompt Learning as Masked Visual Token Modeling Ning Liao Bowen Shi Xiaopeng Zhang Min Cao Junchi Yan Qi Tian VLM 31 7 0 09 Mar 2023
A Discerning Several Thousand Judgments: GPT-3 Rates the Article + Adjective + Numeral + Noun Construction Kyle Mahowald 22 24 0 29 Jan 2023
Dissociating language and thought in large language models Kyle Mahowald Anna A. Ivanova I. Blank Nancy Kanwisher J. Tenenbaum Evelina Fedorenko ELM ReLM 25 209 0 16 Jan 2023
Measure More, Question More: Experimental Studies on Transformer-based Language Models and Complement Coercion Yuling Gu 22 1 0 20 Dec 2022
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation Tianxing He Jingyu Zhang Tianle Wang Sachin Kumar Kyunghyun Cho James R. Glass Yulia Tsvetkov 29 44 0 20 Dec 2022
Implicit causality in GPT-2: a case study H. Huynh T. Lentz Emiel van Miltenburg LRM 22 3 0 08 Dec 2022
Event knowledge in large language models: the gap between the impossible and the unlikely Carina Kauf Anna A. Ivanova Giulia Rambelli Emmanuele Chersoni Jingyuan Selena She Zawad Chowdhury Evelina Fedorenko Alessandro Lenci 35 67 0 02 Dec 2022
Scientific and Creative Analogies in Pretrained Language Models Tamara Czinczoll H. Yannakoudakis Pushkar Mishra Ekaterina Shutova ELM 15 8 0 28 Nov 2022
On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning S. Takagi OffRL 18 7 0 17 Nov 2022
Collateral facilitation in humans and language models J. Michaelov Benjamin Bergen 14 11 0 09 Nov 2022
CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation Abhilasha Ravichander Matt Gardner Ana Marasović 27 34 0 01 Nov 2022
Leveraging Affirmative Interpretations from Negation Improves Natural Language Understanding Md Mosharaf Hossain Eduardo Blanco 27 4 0 26 Oct 2022
Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction Yue Yang Artemis Panagopoulou Marianna Apidianaki Mark Yatskar Chris Callison-Burch 21 2 0 24 Oct 2022
DALL-E 2 Fails to Reliably Capture Common Syntactic Processes Evelina Leivada Elliot Murphy G. Marcus 138 37 0 23 Oct 2022
Probing with Noise: Unpicking the Warp and Weft of Embeddings Filip Klubicka John D. Kelleher 28 4 0 21 Oct 2022
Can Language Representation Models Think in Bets? Zhi–Bin Tang M. Kejriwal 13 6 0 14 Oct 2022
Understanding Prior Bias and Choice Paralysis in Transformer-based Language Representation Models through Four Experimental Probes Ke Shen M. Kejriwal 24 4 0 03 Oct 2022
Representing Affect Information in Word Embeddings Yuhan Zhang Wenqi Chen Ruihan Zhang Xiajie Zhang CVBM 44 3 0 21 Sep 2022
Subject Verb Agreement Error Patterns in Meaningless Sentences: Humans vs. BERT Karim Lasri Olga Seminck Alessandro Lenci Thierry Poibeau 24 4 0 21 Sep 2022
Testing Pre-trained Language Models' Understanding of Distributivity via Causal Mediation Analysis Pangbo Ban Yifan Jiang Tianran Liu Shane Steinert-Threlkeld 48 4 0 11 Sep 2022
Cognitive Modeling of Semantic Fluency Using Transformers Animesh Nighojkar Anna Khlyzova John Licato 16 2 0 20 Aug 2022
Using cognitive psychology to understand GPT-3 Marcel Binz Eric Schulz ELM LLMAG 250 440 0 21 Jun 2022
The Fallacy of AI Functionality Inioluwa Deborah Raji Indra Elizabeth Kumar Aaron Horowitz Andrew D. Selbst 23 179 0 20 Jun 2022
Life after BERT: What do Other Muppets Understand about Language? Vladislav Lialin Kevin Zhao Namrata Shivagunde Anna Rumshisky 39 6 0 21 May 2022
Natural Language Specifications in Proof Assistants Colin S. Gordon Sergey Matskevich 33 1 0 16 May 2022
Improving Contextual Representation with Gloss Regularized Pre-training Yu Lin Zhecheng An Peihao Wu Zejun Ma 19 5 0 13 May 2022
When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to it Sebastian Schuster Tal Linzen 11 25 0 06 May 2022
Developmental Negation Processing in Transformer Language Models Antonio Laverghetta John Licato LRM 25 5 0 29 Apr 2022