Training Compute-Optimal Protein Language Models

4 November 2024

Papers citing "Training Compute-Optimal Protein Language Models"

7 / 7 papers shown

Title
HELM: Hierarchical Encoding for mRNA Language Modeling Mehdi Yazdani-Jahromi Mangal Prakash Tommaso Mansi Artem Moskalev Rui Liao 78 2 0 13 Mar 2025
Genomic Language Models: Opportunities and Challenges Gonzalo Benegas Chengzhong Ye C. Albors Jianan Canal Li Yun S. Song AI4CE LM&MA ELM 32 18 0 16 Jul 2024
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism DeepSeek-AI Xiao Bi : Xiao Bi Deli Chen Guanting Chen ... Yao Zhao Shangyan Zhou Shunfeng Zhou Qihao Zhu Yuheng Zou LRM ALM 139 298 0 05 Jan 2024
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 240 1,070 0 05 Oct 2022
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers Yi Tay Mostafa Dehghani J. Rao W. Fedus Samira Abnar Hyung Won Chung Sharan Narang Dani Yogatama Ashish Vaswani Donald Metzler 183 89 0 22 Sep 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 4,424 0 23 Jan 2020