Efficient Training of Language Models with Compact and Consistent Next Token Distributions

3 July 2024

Papers citing "Efficient Training of Language Models with Compact and Consistent Next Token Distributions"

6 / 6 papers shown

Title
OLMo: Accelerating the Science of Language Models Dirk Groeneveld Iz Beltagy Pete Walsh Akshita Bhagia Rodney Michael Kinney ... Jesse Dodge Kyle Lo Luca Soldaini Noah A. Smith Hanna Hajishirzi OSLM 127 349 0 01 Feb 2024
Language Modelling via Learning to Rank A. Frydenlund Gagandeep Singh Frank Rudzicz 32 7 0 13 Oct 2021
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 242 1,977 0 31 Dec 2020
PubMedQA: A Dataset for Biomedical Research Question Answering Qiao Jin Bhuwan Dhingra Zhengping Liu William W. Cohen Xinghua Lu 199 791 0 13 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018
Generalizing and Hybridizing Count-based and Neural Language Models Graham Neubig Chris Dyer 54 31 0 01 Jun 2016