An Exploratory Study on Code Attention in BERT

An Exploratory Study on Code Attention in BERT

5 April 2022

Fatemeh H. Fard

Papers citing "An Exploratory Study on Code Attention in BERT"

11 / 11 papers shown

Title
Beyond Self-learned Attention: Mitigating Attention Bias in Transformer-based Models Using Attention Guidance Jiri Gesi Iftekhar Ahmed 22 0 0 26 Feb 2024
Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit Yao Wan Yang He Zhangqian Bi Jianguo Zhang Hongyu Zhang Yulei Sui Guandong Xu Hai Jin Philip S. Yu 20 20 0 30 Dec 2023
Naturalness of Attention: Revisiting Attention in Code Language Models M. Saad Tushar Sharma 17 2 0 22 Nov 2023
Investigating semantic subspaces of Transformer sentence embeddings through linear structural probing Dmitry Nikolaev Sebastian Padó 37 5 0 18 Oct 2023
Large Language Models for Software Engineering: A Systematic Literature Review Xinying Hou Yanjie Zhao Yue Liu Zhou Yang Kailong Wang Li Li Xiapu Luo David Lo John C. Grundy Haoyu Wang 25 320 0 21 Aug 2023
The EarlyBIRD Catches the Bug: On Exploiting Early Layers of Encoder Models for More Efficient Code Classification Anastasiia Grishina Max Hort Leon Moonen 22 6 0 08 May 2023
APIContext2Com: Code Comment Generation by Incorporating Pre-Defined API Documentation Ramin Shahbazi Fatemeh H. Fard 15 2 0 03 Mar 2023
AST-Probe: Recovering abstract syntax trees from hidden representations of pre-trained language models José Antonio Hernández López M. Weyssow Jesús Sánchez Cuadrado H. Sahraoui 22 22 0 23 Jun 2022
Probing Pretrained Models of Source Code Sergey Troshin Nadezhda Chirkova ELM 25 38 0 16 Feb 2022
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy ... Nan Duan Neel Sundaresan Shao Kun Deng Shengyu Fu Shujie Liu ELM 196 1,103 0 09 Feb 2021
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 404 2,576 0 03 Sep 2019