Efficient All-to-All Collective Communication Schedules for Direct-Connect Topologies

24 September 2023

Papers citing "Efficient All-to-All Collective Communication Schedules for Direct-Connect Topologies"

4 / 4 papers shown

Title
Hiding Communication Cost in Distributed LLM Training via Micro-batch Co-execution Haiquan Wang Chaoyi Ruan Jia He Jiaqi Ruan Chengjie Tang Xiaosong Ma Cheng-rong Li 73 1 0 24 Nov 2024
Efficient Direct-Connect Topologies for Collective Communications Liangyu Zhao Siddharth Pal Tapan Chugh Weiyang Wang Jason Fantl P. Basu J. Khoury Arvind Krishnamurthy 17 6 0 07 Feb 2022
SpectralFly: Ramanujan Graphs as Flexible and Efficient Interconnection Networks Stephen J. Young Sinan G. Aksoy J. Firoz R. Gioiosa Tobias Hagge Mark Kempton Juan Escobedo Mark Raugas 16 5 0 23 Apr 2021
Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems Maxim Naumov John Kim Dheevatsa Mudigere Srinivas Sridharan Xiaodong Wang ... Krishnakumar Nair Isabel Gao Bor-Yiing Su Jiyan Yang M. Smelyanskiy GNN 38 95 0 20 Mar 2020