Weighted Grouped Query Attention in Transformers

15 July 2024

Papers citing "Weighted Grouped Query Attention in Transformers"

4 / 4 papers shown

Title
Beyond Uniform Query Distribution: Key-Driven Grouped Query Attention Zohaib Khan Muhammad Khaquan Omer Tafveez Burhanuddin Samiwala Agha Ali Raza 22 3 0 15 Aug 2024
OpenELM: An Efficient Language Model Family with Open Training and Inference Framework Sachin Mehta Mohammad Hossein Sekhavat Qingqing Cao Maxwell Horton Yanzi Jin ... Iman Mirzadeh Mahyar Najibi Dmitry Belenko Peter Zatloukal Mohammad Rastegari OSLM AIFin 38 49 0 22 Apr 2024
OLMo: Accelerating the Science of Language Models Dirk Groeneveld Iz Beltagy Pete Walsh Akshita Bhagia Rodney Michael Kinney ... Jesse Dodge Kyle Lo Luca Soldaini Noah A. Smith Hanna Hajishirzi OSLM 127 349 0 01 Feb 2024
BLEU, METEOR, BERTScore: Evaluation of Metrics Performance in Assessing Critical Translation Errors in Sentiment-oriented Text Hadeel Saadany Constantin Orasan 23 35 0 29 Sep 2021