A Systematic Assessment of Syntactic Generalization in Neural Language Models

7 May 2020

Papers citing "A Systematic Assessment of Syntactic Generalization in Neural Language Models"

50 / 126 papers shown

Title
MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs Jaap Jumelet Leonie Weissweiler Arianna Bisazza 38 2 0 03 Apr 2025
Construction Identification and Disambiguation Using BERT: A Case Study of NPN Wesley Scivetti Nathan Schneider 44 0 0 24 Mar 2025
Strategic resource allocation in memory encoding: An efficiency principle shaping language processing Weijie Xu Richard Futrell 55 1 0 18 Mar 2025
Are formal and functional linguistic mechanisms dissociated in language models? Michael Hanna Sandro Pezzelle Yonatan Belinkov 47 0 0 14 Mar 2025
Language Models Fail to Introspect About Their Knowledge of Language Siyuan Song Jennifer Hu Kyle Mahowald LRM KELM HILM ELM 79 2 0 10 Mar 2025
Shades of Zero: Distinguishing Impossibility from Inconceivability Jennifer Hu Felix Sosa T. Ullman 43 0 0 27 Feb 2025
XCOMPS: A Multilingual Benchmark of Conceptual Minimal Pairs Linyang He Ercong Nie Sukru Samet Dindar Arsalan Firoozi Adrian Nicolas Florea ... Haotian Ye Jonathan R. Brennan Helmut Schmid Hinrich Schütze Nima Mesgarani 56 1 0 27 Feb 2025
Learning from Impairment: Leveraging Insights from Clinical Linguistics in Language Modelling Research Dominique Brunato 74 0 0 20 Dec 2024
Sneaking Syntax into Transformer Language Models with Tree Regularization Ananjan Nandi Christopher D. Manning Shikhar Murty 74 0 0 28 Nov 2024
Tree Transformers are an Ineffective Model of Syntactic Constituency Michael Ginn 67 0 0 25 Nov 2024
ZhoBLiMP: a Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese Yikang Liu Yeting Shen Hongao Zhu Lilong Xu Zhiheng Qian ... Jialong Tang Pei Zhang Baosong Yang Rui-cang Wang Hai Hu 45 2 0 09 Nov 2024
From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes Zébulon Goriely Richard Diehl Martinez Andrew Caines Lisa Beinborn P. Buttery CLL 44 5 0 30 Oct 2024
Can Language Models Induce Grammatical Knowledge from Indirect Evidence? Miyu Oba Yohei Oseki Akiyo Fukatsu Akari Haga Hiroki Ouchi Taro Watanabe Saku Sugawara 34 1 0 08 Oct 2024
Goldfish: Monolingual Language Models for 350 Languages Tyler A. Chang Catherine Arnett Zhuowen Tu Benjamin Bergen LRM 36 4 0 19 Aug 2024
Design of a Quality Management System based on the EU Artificial Intelligence Act Henryk Mustroph Stefanie Rinderle-Ma 23 1 0 08 Aug 2024
Towards an Analysis of Discourse and Interactional Pragmatic Reasoning Capabilities of Large Language Models A. Robrecht Judith Sieker Clara Lachenmaier Sina Zarieß Stefan Kopp LRM 26 0 0 06 Aug 2024
Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models Yida Zhao Chao Lou Kewei Tu 50 0 0 24 Jul 2024
LLMs' morphological analyses of complex FST-generated Finnish words Anssi Moisio Mathias Creutz M. Kurimo 46 1 0 11 Jul 2024
Functional Faithfulness in the Wild: Circuit Discovery with Differentiable Computation Graph Pruning Lei Yu Jingcheng Niu Zining Zhu Gerald Penn 38 6 0 04 Jul 2024
Black Big Boxes: Do Language Models Hide a Theory of Adjective Order? Jaap Jumelet Lisa Bylinina Willem H. Zuidema Jakub Szymanik 75 4 0 02 Jul 2024
Bayesian Statistical Modeling with Predictors from LLMs Michael Franke Polina Tsvilodub Fausto Carcassi 45 4 0 13 Jun 2024
Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence Abhinav Patil Jaap Jumelet Yu Ying Chiu Andy Lapastora Peter Shen Lexie Wang Clevis Willrich Shane Steinert-Threlkeld 32 13 0 24 May 2024
Enhancing Length Extrapolation in Sequential Models with Pointer-Augmented Neural Memory Hung Le D. Nguyen Kien Do Svetha Venkatesh T. Tran 30 0 0 18 Apr 2024
To Drop or Not to Drop? Predicting Argument Ellipsis Judgments: A Case Study in Japanese Yukiko Ishizuki Tatsuki Kuribayashi Yuichiroh Matsubayashi Ryohei Sasano Kentaro Inui 30 2 0 17 Apr 2024
Investigating grammatical abstraction in language models using few-shot learning of novel noun gender Priyanka Sukumaran Conor Houghton N. Kazanina 38 0 0 15 Mar 2024
Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale Xiang Hu Pengyu Ji Qingyang Zhu Wei Wu Kewei Tu LRM 33 4 0 13 Mar 2024
On the Tip of the Tongue: Analyzing Conceptual Representation in Large Language Models with Reverse-Dictionary Probe Ningyu Xu Qi Zhang Menghan Zhang Peng Qian Xuanjing Huang LRM 59 3 0 22 Feb 2024
Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervision Ryosuke Yoshida Taiga Someya Yohei Oseki 36 0 0 20 Feb 2024
CausalGym: Benchmarking causal interpretability methods on linguistic tasks Aryaman Arora Daniel Jurafsky Christopher Potts 50 21 0 19 Feb 2024
I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses Xuan Ren Biao Wu Lingqiao Liu 31 5 0 17 Feb 2024
Language models align with human judgments on key grammatical constructions Jennifer Hu Kyle Mahowald G. Lupyan Anna A. Ivanova Roger Levy 37 23 0 19 Jan 2024
Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently Kanishka Misra Allyson Ettinger Kyle Mahowald 23 4 0 12 Jan 2024
Mission: Impossible Language Models Julie Kallini Isabel Papadimitriou Richard Futrell Kyle Mahowald Christopher Potts ELM LRM 44 19 0 12 Jan 2024
I am a Strange Dataset: Metalinguistic Tests for Language Models Tristan Thrush Jared Moore Miguel Monares Christopher Potts Douwe Kiela 14 5 0 10 Jan 2024
Can training neural language models on a curriculum with developmentally plausible data improve alignment with human reading behavior? Aryaman Chobey Oliver Smith Anzi Wang Grusha Prasad 35 5 0 30 Nov 2023
Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation Haoyi Wu Kewei Tu 135 3 0 26 Nov 2023
MELA: Multilingual Evaluation of Linguistic Acceptability Ziyin Zhang Yikang Liu Wei Huang Junyu Mao Rui Wang Hai Hu 22 3 0 15 Nov 2023
In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax Aaron Mueller Albert Webson Jackson Petty Tal Linzen ReLM LRM 34 13 0 13 Nov 2023
Large GPT-like Models are Bad Babies: A Closer Look at the Relationship between Linguistic Competence and Psycholinguistic Measures Julius Steuer Marius Mosbach Dietrich Klakow 22 10 0 08 Nov 2023
Can Language Models Be Tricked by Language Illusions? Easier with Syntax, Harder with Semantics Yuhan Zhang Edward Gibson Forrest Davis 33 6 0 02 Nov 2023
Evaluating Neural Language Models as Cognitive Models of Language Acquisition Héctor Javier Vázquez Martínez Annika Lea Heuser Charles D. Yang Jordan Kodner 17 8 0 31 Oct 2023
Pushdown Layers: Encoding Recursive Structure in Transformer Language Models Shikhar Murty Pratyusha Sharma Jacob Andreas Christopher D. Manning AI4CE 46 13 0 29 Oct 2023
CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models Sreyan Ghosh Ashish Seth Sonal Kumar Utkarsh Tyagi Chandra Kiran Reddy Evuru S. Ramaneswaran S. Sakshi Oriol Nieto R. Duraiswami Dinesh Manocha AuLLM VLM CoGe 35 21 0 12 Oct 2023
BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models Wei Qi Leong Jian Gang Ngui Yosephine Susanto Hamsawardhini Rengarajan Kengatharaiyer Sarveswaran William-Chandra Tjhi 23 9 0 12 Sep 2023
Learning the meanings of function words from grounded language using a visual question answering model Eva Portelance Michael C. Frank Dan Jurafsky NAI 17 7 0 16 Aug 2023
Inductive reasoning in humans and large language models Simon J. Han Keith Ransom Andrew Perfors Charles Kemp ELM ReLM LRM 13 37 0 11 Jun 2023
How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases Aaron Mueller Tal Linzen AI4CE 21 20 0 31 May 2023
Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic Structures Jakob Prange Emmanuele Chersoni 32 0 0 30 May 2023
Large Language Models Are Partially Primed in Pronoun Interpretation S. Lam Qingcheng Zeng Kexun Zhang Chenyu You Rob Voigt 23 4 0 26 May 2023
Structural Ambiguity and its Disambiguation in Language Model Based Parsers: the Case of Dutch Clause Relativization G. Wijnholds M. Moortgat 10 3 0 24 May 2023