Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient
Classification

v1v2 (latest)

Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification

International Conference on Language Resources and Evaluation (LREC), 2022

30 September 2022

Muhammad N. ElNokrashy

Badr AlKhamissi

ArXiv (abs)PDF HTML

Papers citing "Depth-Wise Attention (DWAtt): A Layer Fusion Method for Data-Efficient Classification"

6 / 6 papers shown

Title
Utilizing Multilingual Encoders to Improve Large Language Models for Low-Resource LanguagesMoratuwa Engineering Research Conference (MERCon), 2025 Imalsha Puranegedara Themira Chathumina Nisal Ranathunga Nisansa de Silva Surangika Ranathunga Mokanarangan Thayaparan 155 0 0 12 Aug 2025
Auto-Compressing Networks Vaggelis Dorovatas Georgios Paraskevopoulos Alexandros Potamianos 304 2 0 11 Jun 2025
MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections Da Xiao Qingye Meng Shengping Li Xingyuan Yuan MoE AI4CE 412 7 0 13 Feb 2025
Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data Badr AlKhamissi Yingtian Tang Abdülkadir Gökce Johannes Mehrer Martin Schrimpf VLM 210 2 0 29 Oct 2024
AdaFisher: Adaptive Second Order Optimization via Fisher Information Damien Martins Gomes Yanlei Zhang Eugene Belilovsky Guy Wolf Mahdi S. Hosseini ODL 487 5 0 26 May 2024
DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging Matteo Pagliardini Amirkeivan Mohtashami François Fleuret Martin Jaggi 226 14 0 04 Feb 2024