Visualizing and Measuring the Geometry of BERT

6 June 2019

Papers citing "Visualizing and Measuring the Geometry of BERT"

50 / 62 papers shown

Title
Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translation Chiara Manna Afra Alishahi Frédéric Blain Eva Vanmassenhove 24 0 0 13 May 2025
FinchGPT: a Transformer based language model for birdsong analysis Kosei Kobayashi Kosuke Matsuzaki Masaya Taniguchi Keisuke Sakaguchi Kentaro Inui Kentaro Abe 70 0 0 01 Feb 2025
Compress and Compare: Interactively Evaluating Efficiency and Behavior Across ML Model Compression Experiments Angie Boggust Venkatesh Sivaraman Yannick Assogba Donghao Ren Dominik Moritz Fred Hohman VLM 52 3 0 06 Aug 2024
A mathematical framework of intelligence and consciousness based on Riemannian Geometry Meng Lu AI4CE 26 0 0 02 Jul 2024
The Geometry of Categorical and Hierarchical Concepts in Large Language Models Kiho Park Yo Joong Choe Yibo Jiang Victor Veitch 50 26 0 03 Jun 2024
Latent Concept-based Explanation of NLP Models Xuemin Yu Fahim Dalvi Nadir Durrani Marzia Nouri Hassan Sajjad LRM FAtt 24 1 0 18 Apr 2024
(Chat)GPT v BERT: Dawn of Justice for Semantic Change Detection Francesco Periti Haim Dubossarsky Nina Tahmasebi AI4MH 28 13 0 25 Jan 2024
Investigating semantic subspaces of Transformer sentence embeddings through linear structural probing Dmitry Nikolaev Sebastian Padó 46 5 0 18 Oct 2023
Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers Hosein Mohebbi Grzegorz Chrupała Willem H. Zuidema A. Alishahi 30 12 0 15 Oct 2023
Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators? T. Kajitsuka Issei Sato 31 16 0 26 Jul 2023
Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representations Gábor Berend 35 7 0 25 Jul 2023
Morphosyntactic probing of multilingual BERT models Judit Ács Endre Hamerlik Roy Schwartz Noah A. Smith András Kornai 29 9 0 09 Jun 2023
Single Cells Are Spatial Tokens: Transformers for Spatial Transcriptomic Data Imputation Haifang Wen Wenzhuo Tang Wei Jin Jiayuan Ding Renming Liu Xinnan Dai Feng Shi Lulu Shang Jiliang Tang Yuying Xie 27 8 0 06 Feb 2023
Construction Grammar Provides Unique Insight into Neural Language Models Leonie Weissweiler Taiqi He Naoki Otani David R. Mortensen Lori S. Levin Hinrich Schütze 21 13 0 04 Feb 2023
Preserving local densities in low-dimensional embeddings Jonas Fischer R. Burkholz Jilles Vreeken 22 3 0 31 Jan 2023
Byte Pair Encoding for Symbolic Music Nathan Fradet Nicolas Gutowski F. Chhel Jean-Pierre Briot 29 15 0 27 Jan 2023
Deep Learning Models to Study Sentence Comprehension in the Human Brain S. Arana Jacques Pesnot Lerousseau P. Hagoort 21 10 0 16 Jan 2023
SensePOLAR: Word sense aware interpretability for pre-trained contextual word embeddings Jan Engler Sandipan Sikdar Marlene Lutz M. Strohmaier 24 7 0 11 Jan 2023
Analyzing Text Representations under Tight Annotation Budgets: Measuring Structural Alignment César González-Gutiérrez Audi Primadhanty Francesco Cazzaro A. Quattoni 30 0 0 11 Oct 2022
Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models Emmy Liu Graham Neubig CoGe 15 10 0 07 Oct 2022
Lost in Context? On the Sense-wise Variance of Contextualized Word Embeddings Yile Wang Yue Zhang 19 4 0 20 Aug 2022
What does Transformer learn about source code? Kechi Zhang Ge Li Zhi Jin ViT 20 8 0 18 Jul 2022
A Graph Enhanced BERT Model for Event Prediction LI DU Xiao Ding Yue Zhang Kai Xiong Ting Liu Bing Qin 30 10 0 22 May 2022
HyperAid: Denoising in hyperbolic spaces for tree-fitting and hierarchical clustering Eli Chien Puoya Tabaghi O. Milenkovic 21 4 0 19 May 2022
Discovering Latent Concepts Learned in BERT Fahim Dalvi A. Khan Firoj Alam Nadir Durrani Jia Xu Hassan Sajjad SSL 11 56 0 15 May 2022
Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words Kaitlyn Zhou Kawin Ethayarajh Dallas Card Dan Jurafsky 31 66 0 10 May 2022
BERTops: Studying BERT Representations under a Topological Lens Jatin Chauhan Manohar Kaul 16 3 0 02 May 2022
How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness? Xinhsuai Dong Anh Tuan Luu Min-Bin Lin Shuicheng Yan Hanwang Zhang SILM AAML 20 55 0 22 Dec 2021
Using Distributional Principles for the Semantic Study of Contextual Language Models Olivier Ferret 19 1 0 23 Nov 2021
Interpreting Language Models Through Knowledge Graph Extraction Vinitra Swamy Angelika Romanou Martin Jaggi 26 20 0 16 Nov 2021
Interpreting Deep Learning Models in Natural Language Processing: A Review Xiaofei Sun Diyi Yang Xiaoya Li Tianwei Zhang Yuxian Meng Han Qiu Guoyin Wang Eduard H. Hovy Jiwei Li 17 44 0 20 Oct 2021
MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models Qianchu Liu Fangyu Liu Nigel Collier Anna Korhonen Ivan Vulić 131 21 0 19 Sep 2021
Incorporating Residual and Normalization Layers into Analysis of Masked Language Models Goro Kobayashi Tatsuki Kuribayashi Sho Yokoi Kentaro Inui 160 46 0 15 Sep 2021
InceptionXML: A Lightweight Framework with Synchronized Negative Sampling for Short Text Extreme Classification Siddhant Kharbanda Atmadeep Banerjee Devaansh Gupta Akash Palrecha Rohit Babbar 27 9 0 13 Sep 2021
Uniform Manifold Approximation and Projection (UMAP) and its Variants: Tutorial and Survey Benyamin Ghojogh A. Ghodsi Fakhri Karray Mark Crowley 21 22 0 25 Aug 2021
The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models G. dÉon Jason dÉon J. R. Wright Kevin Leyton-Brown 20 74 0 01 Jul 2021
The Limitations of Limited Context for Constituency Parsing Yuchen Li Andrej Risteski 26 4 0 03 Jun 2021
LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyond Daniel Loureiro A. Jorge Jose Camacho-Collados 33 26 0 26 May 2021
Let's Play Mono-Poly: BERT Can Reveal Words' Polysemy Level and Partitionability into Senses Aina Garí Soler Marianna Apidianaki MILM 206 68 0 29 Apr 2021
Pose Recognition with Cascade Transformers Ke Li Shijie Wang Xiang Zhang Yifan Xu Weijian Xu Z. Tu ViT 32 209 0 14 Apr 2021
BBAM: Bounding Box Attribution Map for Weakly Supervised Semantic and Instance Segmentation Jungbeom Lee Jihun Yi Chaehun Shin Sungroh Yoon ISeg 24 172 0 16 Mar 2021
Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do P. Schramowski Cigdem Turan Nico Andersen Constantin Rothkopf Kristian Kersting 25 281 0 08 Mar 2021
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics Vassilina Nikoulina Maxat Tezekbayev Nuradil Kozhakhmet Madina Babazhanova Matthias Gallé Z. Assylbekov 34 8 0 02 Mar 2021
Characterizing English Variation across Social Media Communities with BERT L. Lucy David Bamman 16 35 0 12 Feb 2021
Gender Bias in Multilingual Neural Machine Translation: The Architecture Matters Marta R. Costa-jussá Carlos Escolano Christine Basta Javier Ferrando Roser Batlle-Roca Ksenia Kharitonova 14 18 0 24 Dec 2020
TabTransformer: Tabular Data Modeling Using Contextual Embeddings Xin Huang A. Khetan Milan Cvitkovic Zohar S. Karnin ViT LMTD 157 416 0 11 Dec 2020
Positional Artefacts Propagate Through Masked Language Model Embeddings Ziyang Luo Artur Kulmizev Xiaoxi Mao 24 41 0 09 Nov 2020
Dynamic Contextualized Word Embeddings Valentin Hofmann J. Pierrehumbert Hinrich Schütze 39 51 0 23 Oct 2020
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention Ikuya Yamada Akari Asai Hiroyuki Shindo Hideaki Takeda Yuji Matsumoto 22 662 0 02 Oct 2020
Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models Joseph F DeRose Jiayao Wang M. Berger 17 83 0 03 Sep 2020