Information-Theoretic Probing with Minimum Description Length

27 March 2020

Papers citing "Information-Theoretic Probing with Minimum Description Length"

43 / 43 papers shown

Title
Gender Encoding Patterns in Pretrained Language Model Representations Mahdi Zakizadeh Mohammad Taher Pilehvar 43 0 0 09 Mar 2025
Representations as Language: An Information-Theoretic Framework for Interpretability Henry Conklin Kenny Smith MILM 24 1 0 04 Jun 2024
A Modular Approach for Multimodal Summarization of TV Shows Louis Mahon Mirella Lapata 21 9 0 06 Mar 2024
Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models J. Michaelov Catherine Arnett Tyler A. Chang Benjamin Bergen 34 12 0 15 Nov 2023
Uncovering Intermediate Variables in Transformers using Circuit Probing Michael A. Lepori Thomas Serre Ellie Pavlick 70 7 0 07 Nov 2023
Implications of Annotation Artifacts in Edge Probing Test Datasets Sagnik Ray Choudhury Jushaan Kalra 16 0 0 20 Oct 2023
Bridging Information-Theoretic and Geometric Compression in Language Models Emily Cheng Corentin Kervadec Marco Baroni 24 16 0 20 Oct 2023
A Geometric Notion of Causal Probing Clément Guerner Anej Svete Tianyu Liu Alex Warstadt Ryan Cotterell LLMSV 34 12 0 27 Jul 2023
Generalizing Backpropagation for Gradient-Based Interpretability Kevin Du Lucas Torroba Hennigen Niklas Stoehr Alex Warstadt Ryan Cotterell MILM FAtt 19 6 0 06 Jul 2023
Morphosyntactic probing of multilingual BERT models Judit Ács Endre Hamerlik Roy Schwartz Noah A. Smith András Kornai 25 9 0 09 Jun 2023
Gaussian Process Probes (GPP) for Uncertainty-Aware Probing Z. Wang Alexander Ku Jason Baldridge Thomas L. Griffiths Been Kim UQCV 21 11 0 29 May 2023
What is the best recipe for character-level encoder-only modelling? Kris Cao 27 2 0 09 May 2023
Evaluating Representations with Readout Model Switching Yazhe Li J. Bornschein Marcus Hutter 22 0 0 19 Feb 2023
Dissociating language and thought in large language models Kyle Mahowald Anna A. Ivanova I. Blank Nancy Kanwisher J. Tenenbaum Evelina Fedorenko ELM ReLM 23 208 0 16 Jan 2023
Feature-Level Debiased Natural Language Understanding Yougang Lyu Piji Li Yechang Yang Maarten de Rijke Pengjie Ren Yukun Zhao Dawei Yin Z. Ren 23 10 0 11 Dec 2022
Reward Gaming in Conditional Text Generation Richard Yuanzhe Pang Vishakh Padmakumar Thibault Sellam Ankur P. Parikh He He 21 24 0 16 Nov 2022
Enhancing Tabular Reasoning with Pattern Exploiting Training Abhilash Shankarampeta Vivek Gupta Shuo Zhang LMTD RALM ReLM 58 6 0 21 Oct 2022
Probing with Noise: Unpicking the Warp and Weft of Embeddings Filip Klubicka John D. Kelleher 24 4 0 21 Oct 2022
Choose Your Lenses: Flaws in Gender Bias Evaluation Hadas Orgad Yonatan Belinkov 22 35 0 20 Oct 2022
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 30 16 0 18 Oct 2022
Sequential Learning Of Neural Networks for Prequential MDL J. Bornschein Yazhe Li Marcus Hutter AI4TS 20 6 0 14 Oct 2022
Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models Emmy Liu Graham Neubig CoGe 13 10 0 07 Oct 2022
SynBench: Task-Agnostic Benchmarking of Pretrained Representations using Synthetic Data Ching-Yun Ko Pin-Yu Chen Jeet Mohapatra Payel Das Lucani E. Daniel 19 3 0 06 Oct 2022
Sort by Structure: Language Model Ranking as Dependency Probing Max Müller-Eberstein Rob van der Goot Barbara Plank 30 3 0 10 Jun 2022
Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models Esma Balkir S. Kiritchenko I. Nejadgholi Kathleen C. Fraser 13 36 0 08 Jun 2022
Interpretation of Black Box NLP Models: A Survey Shivani Choudhary N. Chatterjee S. K. Saha FAtt 28 10 0 31 Mar 2022
Visualizing the Relationship Between Encoded Linguistic Information and Task Performance Jiannan Xiang Huayang Li Defu Lian Guoping Huang Taro Watanabe Lemao Liu 20 0 0 29 Mar 2022
Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languages Ehsan Aghazadeh Mohsen Fayyaz Yadollah Yaghoobzadeh 31 51 0 26 Mar 2022
Towards Explainable Evaluation Metrics for Natural Language Generation Christoph Leiter Piyawat Lertvittayakumjorn M. Fomicheva Wei-Ye Zhao Yang Gao Steffen Eger AAML ELM 19 20 0 21 Mar 2022
How does the pre-training objective affect what large language models learn about linguistic properties? Ahmed Alajrami Nikolaos Aletras 17 20 0 20 Mar 2022
Sparse Interventions in Language Models with Differentiable Masking Nicola De Cao Leon Schmid Dieuwke Hupkes Ivan Titov 25 27 0 13 Dec 2021
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs Peter Hase Mona T. Diab Asli Celikyilmaz Xian Li Zornitsa Kozareva Veselin Stoyanov Mohit Bansal Srini Iyer KELM LRM 17 79 0 26 Nov 2021
Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP Zhijing Jin Julius von Kügelgen Jingwei Ni Tejas Vaidhya Ayush Kaushal Mrinmaya Sachan Bernhard Schoelkopf CML 25 30 0 07 Oct 2021
Conditional probing: measuring usable information beyond a baseline John Hewitt Kawin Ethayarajh Percy Liang Christopher D. Manning 31 55 0 19 Sep 2021
Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations Mohsen Fayyaz Ehsan Aghazadeh Ali Modarressi Hosein Mohebbi Mohammad Taher Pilehvar 6 21 0 13 Sep 2021
Neuron-level Interpretation of Deep NLP Models: A Survey Hassan Sajjad Nadir Durrani Fahim Dalvi MILM AI4CE 22 79 0 30 Aug 2021
Learning De-identified Representations of Prosody from Raw Audio J. Weston R. Lenain U. Meepegama E. Fristed SSL 24 15 0 17 Jul 2021
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics Vassilina Nikoulina Maxat Tezekbayev Nuradil Kozhakhmet Madina Babazhanova Matthias Gallé Z. Assylbekov 29 8 0 02 Mar 2021
Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers Lisa Anne Hendricks John F. J. Mellor R. Schneider Jean-Baptiste Alayrac Aida Nematzadeh 75 110 0 31 Jan 2021
Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERT Isabel Papadimitriou Ethan A. Chi Richard Futrell Kyle Mahowald 16 44 0 26 Jan 2021
Examining the rhetorical capacities of neural language models Zining Zhu Chuer Pan Mohamed Abdalla Frank Rudzicz 20 10 0 01 Oct 2020
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives Elena Voita Rico Sennrich Ivan Titov 188 181 0 03 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 404 2,576 0 03 Sep 2019