BERTnesia: Investigating the capture and forgetting of knowledge in BERT

BERTnesia: Investigating the capture and forgetting of knowledge in BERT

5 June 2021

Papers citing "BERTnesia: Investigating the capture and forgetting of knowledge in BERT"

17 / 17 papers shown

Title
Deep Learning with Pretrained Ínternal World' Layers: A Gemma 3-Based Modular Architecture for Wildfire Prediction Ayoub Jadouli Chaker El Amrani KELM AI4TS 81 0 0 20 Apr 2025
Superscopes: Amplifying Internal Feature Representations for Language Model Interpretation Jonathan Jacobi Gal Niv LRM ReLM 60 0 0 03 Mar 2025
A Mechanistic Interpretation of Syllogistic Reasoning in Auto-Regressive Language Models Geonhee Kim Marco Valentino André Freitas LRM AI4CE 28 7 0 16 Aug 2024
Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models Asma Ghandeharioun Avi Caciularu Adam Pearce Lucas Dixon Mor Geva 34 87 0 11 Jan 2024
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia Giovanni Monea Maxime Peyrard Martin Josifoski Vishrav Chaudhary Jason Eisner Emre Kiciman Hamid Palangi Barun Patra Robert West KELM 51 12 0 04 Dec 2023
Understanding Finetuning for Factual Knowledge Extraction from Language Models Mehran Kazemi Sid Mittal Deepak Ramachandran KELM 34 10 0 26 Jan 2023
Parameter-Efficient Tuning Makes a Good Classification Head Zhuoyi Yang Ming Ding Yanhui Guo Qingsong Lv Jie Tang VLM 40 14 0 30 Oct 2022
Understanding Transformer Memorization Recall Through Idioms Adi Haviv Ido Cohen Jacob Gidron R. Schuster Yoav Goldberg Mor Geva 28 48 0 07 Oct 2022
Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification Muhammad N. ElNokrashy Badr AlKhamissi Mona T. Diab MoMe 17 4 0 30 Sep 2022
Beyond Distributional Hypothesis: Let Language Models Learn Meaning-Text Correspondence Myeongjun Jang Frank Mtumbuka Thomas Lukasiewicz 28 9 0 08 May 2022
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space Mor Geva Avi Caciularu Ke Wang Yoav Goldberg KELM 46 336 0 28 Mar 2022
Finding Structural Knowledge in Multimodal-BERT Victor Milewski Miryam de Lhoneux Marie-Francine Moens 19 9 0 17 Mar 2022
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 263 346 0 01 Feb 2021
K-BERT: Enabling Language Representation with Knowledge Graph Weijie Liu Peng Zhou Zhe Zhao Zhiruo Wang Qi Ju Haotang Deng Ping Wang 231 778 0 17 Sep 2019
Knowledge Enhanced Contextual Word Representations Matthew E. Peters Mark Neumann IV RobertL.Logan Roy Schwartz Vidur Joshi Sameer Singh Noah A. Smith 231 656 0 09 Sep 2019
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 415 2,588 0 03 Sep 2019
Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation Ikuya Yamada Hiroyuki Shindo Hideaki Takeda Yoshiyasu Takefuji 123 317 0 06 Jan 2016