Paying More Attention to Self-attention: Improving Pre-trained Language Models via Attention Guiding

6 April 2022

Papers citing "Paying More Attention to Self-attention: Improving Pre-trained Language Models via Attention Guiding"

8 / 8 papers shown

Title
Concentrate Attention: Towards Domain-Generalizable Prompt Optimization for Language Models Chengzhengxu Li Xiaoming Liu Zhaohan Zhang Yichen Wang Chen Liu Y. Lan Chao Shen 57 2 0 15 Jun 2024
Evaluating Lexicon Incorporation for Depression Symptom Estimation Kirill Milintsevich Gael Dias Kairit Sirts 23 0 0 30 Apr 2024
Beyond Self-learned Attention: Mitigating Attention Bias in Transformer-based Models Using Attention Guidance Jiri Gesi Iftekhar Ahmed 51 0 0 26 Feb 2024
A Survey of Large Language Models in Medicine: Progress, Application, and Challenge Hongjian Zhou Fenglin Liu Boyang Gu Xinyu Zou Jinfa Huang ... Yefeng Zheng Lei A. Clifton Zheng Li Fenglin Liu David A. Clifton LM&MA 33 107 0 09 Nov 2023
Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorization Yu-Zhu Guo Weihang Peng Y. Chen Liwen Zhang Xiaode Liu Xuhui Huang Zhe Ma 105 35 0 03 May 2023
How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers Michael Hassid Hao Peng Daniel Rotem Jungo Kasai Ivan Montero Noah A. Smith Roy Schwartz 32 24 0 07 Nov 2022
Improving BERT with Syntax-aware Local Attention Zhongli Li Qingyu Zhou Chao Li Ke Xu Yunbo Cao 61 44 0 30 Dec 2020
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 239 611 0 13 Oct 2020