TensorOpt: Exploring the Tradeoffs in Distributed DNN Training with Auto-Parallelism

16 April 2020

Papers citing "TensorOpt: Exploring the Tradeoffs in Distributed DNN Training with Auto-Parallelism"

5 / 5 papers shown

Title
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training Jared Fernandez Luca Wehrstedt Leonid Shamis Mostafa Elhoushi Kalyan Saladi Yonatan Bisk Emma Strubell Jacob Kahn 200 3 0 20 Nov 2024
UniAP: Unifying Inter- and Intra-Layer Automatic Parallelism by Mixed Integer Quadratic Programming Hao Lin Ke Wu Jie Li Jun Yu Li Wu-Jun Li 33 1 0 31 Jul 2023
TAPS: Topology-Aware Intra-Operator Parallelism Strategy Searching Algorithm for Deep Neural Networks Peng Liang Hao Zheng Teng Su Linbo Qiao Dongsheng Li 14 0 0 11 Jan 2023
Galvatron: Efficient Transformer Training over Multiple GPUs Using Automatic Parallelism Xupeng Miao Yujie Wang Youhe Jiang Chunan Shi Xiaonan Nie Hailin Zhang Bin Cui GNN MoE 37 60 0 25 Nov 2022
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,746 0 26 Sep 2016