Relaxed Attention for Transformer Models

Relaxed Attention for Transformer Models

20 September 2022

Tim Fingscheidt

Papers citing "Relaxed Attention for Transformer Models"

11 / 11 papers shown

Title
Sentence-Level or Token-Level? A Comprehensive Study on Knowledge Distillation Jingxuan Wei Linzhuang Sun Yichong Leng Xu Tan Bihui Yu Ruifeng Guo 31 3 0 23 Apr 2024
Self-Supervised Adaptive AV Fusion Module for Pre-Trained ASR Models Christopher Simic Tobias Bocklet 21 5 0 21 Dec 2023
Unraveling Key Factors of Knowledge Distillation Jingxuan Wei Linzhuang Sun Xu Tan Bihui Yu Ruifeng Guo 12 0 0 14 Dec 2023
Multi-Objective Decision Transformers for Offline Reinforcement Learning Abdelghani Ghanem P. Ciblat Mounir Ghogho OffRL 11 1 0 31 Aug 2023
AKVSR: Audio Knowledge Empowered Visual Speech Recognition by Compressing Audio Knowledge of a Pretrained Model Jeong Hun Yeo Minsu Kim J. Choi Dae Hoe Kim Y. Ro 13 17 0 15 Aug 2023
Attend to the Right Context: A Plug-and-Play Module for Content-Controllable Summarization Wen Xiao Lesly Miculicich Yang Liu Pengcheng He Giuseppe Carenini KELM 20 3 0 21 Dec 2022
Analysis of Self-Attention Head Diversity for Conformer-based Automatic Speech Recognition Kartik Audhkhasi Yinghui Huang Bhuvana Ramabhadran Pedro J. Moreno 19 3 0 13 Sep 2022
CMT: Convolutional Neural Networks Meet Vision Transformers Jianyuan Guo Kai Han Han Wu Yehui Tang Chunjing Xu Yunhe Wang Chang Xu ViT 328 500 0 13 Jul 2021
End-to-end Audio-visual Speech Recognition with Conformers Pingchuan Ma Stavros Petridis M. Pantic 79 221 0 12 Feb 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020
VoxCeleb2: Deep Speaker Recognition Joon Son Chung Arsha Nagrani Andrew Zisserman 214 2,224 0 14 Jun 2018