Title
NeuralDEM -- Real-time Simulation of Industrial Particulate Flows Benedikt Alkin Tobias Kronlachner Samuele Papa Stefan Pirker Thomas Lichtenegger Johannes Brandstetter PINN AI4CE 38 1 1 14 Nov 2024
HAP: SPMD DNN Training on Heterogeneous GPU Clusters with Automated Program Synthesis Shiwei Zhang Lansong Diao Chuan Wu Zongyan Cao Siyu Wang Wei Lin 35 12 0 11 Jan 2024
UniAP: Unifying Inter- and Intra-Layer Automatic Parallelism by Mixed Integer Quadratic Programming Hao Lin Ke Wu Jie Li Jun Yu Li Wu-Jun Li 26 1 0 31 Jul 2023
PaLM 2 Technical Report Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin ... Ce Zheng Wei Zhou Denny Zhou Slav Petrov Yonghui Wu ReLM LRM 62 1,142 0 17 May 2023
Massively Multilingual Shallow Fusion with Large Language Models Ke Hu Tara N. Sainath Bo-wen Li Nan Du Yanping Huang Andrew M. Dai Yu Zhang Rodrigo Cabrera Z. Chen Trevor Strohman 30 13 0 17 Feb 2023
Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform Shiwei Zhang Lansong Diao Siyu Wang Zongyan Cao Yiliang Gu Chang Si Ziji Shi Zhen Zheng Chuan Wu W. Lin AI4CE 22 4 0 16 Feb 2023
Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models Yuliang Liu Shenggui Li Jiarui Fang Yan Shao Boyuan Yao Yang You OffRL 19 7 0 06 Feb 2023
Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism Xupeng Miao Yujie Wang Youhe Jiang Chunan Shi Xiaonan Nie Hailin Zhang Bin Cui GNN MoE 32 60 0 25 Nov 2022
Efficiently Scaling Transformer Inference Reiner Pope Sholto Douglas Aakanksha Chowdhery Jacob Devlin James Bradbury Anselm Levskaya Jonathan Heek Kefan Xiao Shivani Agrawal J. Dean 32 292 0 09 Nov 2022
Scaling Autoregressive Models for Content-Rich Text-to-Image Generation Jiahui Yu Yuanzhong Xu Jing Yu Koh Thang Luong Gunjan Baid ... Zarana Parekh Xin Li Han Zhang Jason Baldridge Yonghui Wu EGVM 90 1,061 0 22 Jun 2022
Decentralized Training of Foundation Models in Heterogeneous Environments Binhang Yuan Yongjun He Jared Davis Tianyi Zhang Tri Dao Beidi Chen Percy Liang Christopher Ré Ce Zhang 20 90 0 02 Jun 2022
Reducing Activation Recomputation in Large Transformer Models V. Korthikanti Jared Casper Sangkug Lym Lawrence C. McAfee M. Andersch M. Shoeybi Bryan Catanzaro AI4CE 27 252 0 10 May 2022
When does dough become a bagel? Analyzing the remaining mistakes on ImageNet Vijay Vasudevan Benjamin Caine Raphael Gontijo-Lopes Sara Fridovich-Keil Rebecca Roelofs VLM UQCV 33 57 0 09 May 2022
CoCa: Contrastive Captioners are Image-Text Foundation Models Jiahui Yu Zirui Wang Vijay Vasudevan Legg Yeung Mojtaba Seyedhosseini Yonghui Wu VLM CLIP OffRL 57 1,255 0 04 May 2022
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 149 327 0 18 Feb 2022
Unified Scaling Laws for Routed Language Models Aidan Clark Diego de Las Casas Aurelia Guy A. Mensch Michela Paganini ... Oriol Vinyals Jack W. Rae Erich Elsen Koray Kavukcuoglu Karen Simonyan MoE 27 177 0 02 Feb 2022
Skillful Twelve Hour Precipitation Forecasts using Large Context Neural Networks L. Espeholt Shreya Agrawal C. Sønderby M. Kumar Jonathan Heek Carla Bromberg Cenk Gazen Jason Hickey Aaron Bell Nal Kalchbrenner AI4Cl 11 47 0 14 Nov 2021
Large-Scale Differentially Private BERT Rohan Anil Badih Ghazi Vineet Gupta Ravi Kumar Pasin Manurangsi 22 131 0 03 Aug 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,774 0 24 Feb 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,817 0 17 Sep 2019