What Happens To BERT Embeddings During Fine-tuning?

29 April 2020

Papers citing "What Happens To BERT Embeddings During Fine-tuning?"

39 / 39 papers shown

Title
Fine-Tuning without Performance Degradation Han Wang Adam White Martha White OnRL 140 0 0 01 May 2025
ReSi: A Comprehensive Benchmark for Representational Similarity Measures Max Klabunde Tassilo Wald Tobias Schumacher Klaus H. Maier-Hein Markus Strohmaier Adriana Iamnitchi AI4TS VLM 70 5 0 13 Mar 2025
Statistical Deficiency for Task Inclusion Estimation Loïc Fosse Frédéric Béchet Benoit Favre Géraldine Damnati Gwénolé Lecorvé Maxime Darrin Philippe Formont Pablo Piantanida 127 0 0 07 Mar 2025
Pixology: Probing the Linguistic and Visual Capabilities of Pixel-based Language Models Kushal Tatariya Vladimir Araujo Thomas Bauwens Miryam de Lhoneux VLM 29 0 0 15 Oct 2024
Reconstructing Human Mobility Pattern: A Semi-Supervised Approach for Cross-Dataset Transfer Learning Xishun Liao Yifan Liu Chenchen Kuai Haoxuan Ma Yueshuai He Shangqing Cao Chris Stanford Jiaqi Ma 33 1 0 03 Oct 2024
An Encoding--Searching Separation Perspective on Bi-Encoder Neural Search Danbinaerin Han Akiko Aizawa Sihun Lee 19 0 0 02 Aug 2024
Fine-tuning Strategies for Domain Specific Question Answering under Low Annotation Budget Constraints Kunpeng Guo Dennis Diefenbach Antoine Gourru Christophe Gravier 13 0 0 17 Jan 2024
Disentangling the Linguistic Competence of Privacy-Preserving BERT Stefan Arnold Nils Kemmerzell Annika Schreiner 25 0 0 17 Oct 2023
Few-Shot Spoken Language Understanding via Joint Speech-Text Models Chung-Ming Chien Mingjiamei Zhang Ju-Chieh Chou Karen Livescu 26 3 0 09 Oct 2023
On Robustness of Finetuned Transformer-based NLP Models Pavan Kalyan Reddy Neerudu S. Oota Mounika Marreddy Venkateswara Rao Kagita Manish Gupta 21 7 0 23 May 2023
Similarity of Neural Network Models: A Survey of Functional and Representational Measures Max Klabunde Tobias Schumacher M. Strohmaier Florian Lemmerich 47 64 0 10 May 2023
PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models Leonardo Ranaldi Elena Sofia Ruzzetti Fabio Massimo Zanzotto 31 6 0 08 May 2023
Towards Efficient Fine-tuning of Pre-trained Code Models: An Experimental Study and Beyond Ensheng Shi Yanlin Wang Hongyu Zhang Lun Du Shi Han Dongmei Zhang Hongbin Sun 28 42 0 11 Apr 2023
Can We Use Probing to Better Understand Fine-tuning and Knowledge Distillation of the BERT NLU? Jakub Ho'scilowicz Marcin Sowanski Piotr Czubowski Artur Janicki 23 2 0 27 Jan 2023
Interpretability in Activation Space Analysis of Transformers: A Focused Survey Soniya Vijayakumar AI4CE 27 3 0 22 Jan 2023
On the Effect of Pre-training for Transformer in Different Modality on Offline Reinforcement Learning S. Takagi OffRL 18 7 0 17 Nov 2022
On the Transformation of Latent Space in Fine-Tuned NLP Models Nadir Durrani Hassan Sajjad Fahim Dalvi Firoj Alam 29 18 0 23 Oct 2022
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 32 16 0 18 Oct 2022
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning Kunbo Ding Weijie Liu Yuejian Fang Weiquan Mao Zhe Zhao Tao Zhu Haoyan Liu Rong Tian Yiren Chen 30 8 0 18 Oct 2022
Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks Zixuan Ke Hu Xu Bing-Quan Liu CLL 235 84 0 06 Dec 2021
CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks Zixuan Ke Bing-Quan Liu Hu Xu Lei Shu CLL 25 55 0 05 Dec 2021
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning Zixuan Ke Bing-Quan Liu Nianzu Ma Hu Xu Lei Shu CLL 181 123 0 05 Dec 2021
Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers Jason Phang Haokun Liu Samuel R. Bowman 22 25 0 17 Sep 2021
Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations Mohsen Fayyaz Ehsan Aghazadeh Ali Modarressi Hosein Mohebbi Mohammad Taher Pilehvar 18 21 0 13 Sep 2021
T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP Raymond Li Wen Xiao Lanjun Wang Hyeju Jang Giuseppe Carenini ViT 15 23 0 31 Aug 2021
Do Vision Transformers See Like Convolutional Neural Networks? M. Raghu Thomas Unterthiner Simon Kornblith Chiyuan Zhang Alexey Dosovitskiy ViT 26 922 0 19 Aug 2021
Theoretical foundations and limits of word embeddings: what types of meaning can they capture? Alina Arseniev-Koehler 28 19 0 22 Jul 2021
A Closer Look at How Fine-tuning Changes BERT Yichu Zhou Vivek Srikumar 24 63 0 27 Jun 2021
How transfer learning impacts linguistic knowledge in deep NLP models? Nadir Durrani Hassan Sajjad Fahim Dalvi 13 48 0 31 May 2021
DirectProbe: Studying Representations without Classifiers Yichu Zhou Vivek Srikumar 27 27 0 13 Apr 2021
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT Benjamin Muller Yanai Elazar Benoît Sagot Djamé Seddah LRM 21 71 0 26 Jan 2021
CascadeBERT: Accelerating Inference of Pre-trained Language Models via Calibrated Complete Models Cascade Lei Li Yankai Lin Deli Chen Shuhuai Ren Peng Li Jie Zhou Xu Sun 26 51 0 29 Dec 2020
Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data Jonathan Pilault Amine Elhattami C. Pal CLL MoE 19 89 0 19 Sep 2020
Revisiting Few-sample BERT Fine-tuning Tianyi Zhang Felix Wu Arzoo Katiyar Kilian Q. Weinberger Yoav Artzi 30 441 0 10 Jun 2020
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives Elena Voita Rico Sennrich Ivan Titov 190 181 0 03 Sep 2019
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 199 882 0 03 May 2018
Hypothesis Only Baselines in Natural Language Inference Adam Poliak Jason Naradowsky Aparajita Haldar Rachel Rudinger Benjamin Van Durme 190 576 0 02 May 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,950 0 20 Apr 2018
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,743 0 26 Sep 2016