Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers

17 September 2021

Papers citing "Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers"

19 / 19 papers shown

Title
Robustly identifying concepts introduced during chat fine-tuning using crosscoders Julian Minder Clement Dumas Caden Juang Bilal Chugtai Neel Nanda 27 0 0 03 Apr 2025
Collective Model Intelligence Requires Compatible Specialization Jyothish Pari Samy Jelassi Pulkit Agrawal MoMe 38 1 0 04 Nov 2024
Tending Towards Stability: Convergence Challenges in Small Language Models Richard Diehl Martinez Pietro Lesci P. Buttery 15 1 0 15 Oct 2024
Cross-layer Attention Sharing for Large Language Models Yongyu Mu Yuzhang Wu Yuchun Fan Chenglong Wang Hengyu Li Qiaozhi He Murun Yang Tong Xiao Jingbo Zhu 36 5 0 04 Aug 2024
CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion Jiayi Yao Hanchen Li Yuhan Liu Siddhant Ray Yihua Cheng Qizheng Zhang Kuntai Du Shan Lu Junchen Jiang 42 14 0 26 May 2024
A Single Linear Layer Yields Task-Adapted Low-Rank Matrices Hwichan Kim S. Sasaki Sho Hoshino Ukyo Honda 23 1 0 22 Mar 2024
Where does In-context Translation Happen in Large Language Models Suzanna Sia David Mueller Kevin Duh LRM 33 0 0 07 Mar 2024
Perturbed examples reveal invariances shared by language models Ruchit Rawal Mariya Toneva AAML 34 0 0 07 Nov 2023
CRaSh: Clustering, Removing, and Sharing Enhance Fine-tuning without Full Large Language Model Kaiyan Zhang Ning Ding Biqing Qi Xuekai Zhu Xinwei Long Bowen Zhou 38 4 0 24 Oct 2023
Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models Phillip Rust Anders Søgaard 25 3 0 17 Aug 2023
What Happens During Finetuning of Vision Transformers: An Invariance Based Investigation Gabriele Merlin Vedant Nanda Ruchit Rawal Mariya Toneva 15 3 0 12 Jul 2023
VendorLink: An NLP approach for Identifying & Linking Vendor Migrants & Potential Aliases on Darknet Markets V. Saxena Nils Rethmeier Gijs Van Dijck Gerasimos Spanakis 24 6 0 04 May 2023
G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks Zhongwei Wan Yichun Yin Wei Zhang Jiaxin Shi Lifeng Shang Guangyong Chen Xin Jiang Qun Liu VLM CLL 26 16 0 07 Dec 2022
Hidden State Variability of Pretrained Language Models Can Guide Computation Reduction for Transfer Learning Shuo Xie Jiahao Qiu Ankita Pasad Li Du Qing Qu Hongyuan Mei 30 16 0 18 Oct 2022
Life after BERT: What do Other Muppets Understand about Language? Vladislav Lialin Kevin Zhao Namrata Shivagunde Anna Rumshisky 34 6 0 21 May 2022
Unified Visual Transformer Compression Shixing Yu Tianlong Chen Jiayi Shen Huan Yuan Jianchao Tan Sen Yang Ji Liu Zhangyang Wang ViT 14 91 0 15 Mar 2022
Similarity Analysis of Contextual Word Representation Models John M. Wu Yonatan Belinkov Hassan Sajjad Nadir Durrani Fahim Dalvi James R. Glass 46 73 0 03 May 2020
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives Elena Voita Rico Sennrich Ivan Titov 188 181 0 03 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018