Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

23 May 2023

Papers citing "Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training"

3 / 103 papers shown

Title
Dynamic Recognition of Speakers for Consent Management by Contrastive Embedding Replay Arash Shahmansoori U. Roedig 17 1 0 17 May 2022
The Pile: An 800GB Dataset of Diverse Text for Language Modeling Leo Gao Stella Biderman Sid Black Laurence Golding Travis Hoppe ... Horace He Anish Thite Noa Nabeshima Shawn Presser Connor Leahy AIMat 248 1,986 0 31 Dec 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,460 0 23 Jan 2020