Distributed Hybrid CPU and GPU training for Graph Neural Networks on Billion-Scale Graphs

31 December 2021

George Karypis

Papers citing "Distributed Hybrid CPU and GPU training for Graph Neural Networks on Billion-Scale Graphs"

32 / 32 papers shown

Title
MLKV: Efficiently Scaling up Large Embedding Model Training with Disk-based Key-Value Storage Yongjun He R. Waleffe Zhichao Han Johnu George Binhang Yuan ... Yinan Shan Yang Zhao Debojyoti Dutta Theodoros Rekatsinas Ce Zhang AIFin 33 0 0 02 Apr 2025
Deal: Distributed End-to-End GNN Inference for All Nodes Shiyang Chen Xiang Song Vasiloudis Theodore Hang Liu GNN 50 0 0 04 Mar 2025
WaveGAS: Waveform Relaxation for Scaling Graph Neural Networks Jana Vatter Mykhaylo Zayats Marcos Martínez Galindo V. López R. Mayer Hans-Arno Jacobsen Hoang Thanh Lam GNN 43 0 0 27 Feb 2025
Armada: Memory-Efficient Distributed Training of Large-Scale Graph Neural Networks R. Waleffe Devesh Sarda J. Mohoney Emmanouil-Vasileios Vlatakis-Gkaragkounis Theodoros Rekatsinas Shivaram Venkataraman GNN 47 0 0 25 Feb 2025
NeutronTP: Load-Balanced Distributed Full-Graph GNN Training with Tensor Parallelism Xin Ai Hao Yuan Zeyu Ling Qiange Wang Yanfeng Zhang Zhenbo Fu Chaoyi Chen Yu Gu Ge Yu GNN 33 1 0 29 Dec 2024
D3-GNN: Dynamic Distributed Dataflow for Streaming Graph Neural Networks Rustam Guliyev Aparajita Haldar Hakan Ferhatosmanoglu GNN 34 1 0 10 Sep 2024
Heta: Distributed Training of Heterogeneous Graph Neural Networks Yuchen Zhong Junwei Su Chuan Wu Minjie Wang 26 1 0 19 Aug 2024
Graph Neural Network Training Systems: A Performance Comparison of Full-Graph and Mini-Batch Saurabh Bajaj Hui Guan Marco Serafini GNN 33 1 0 01 Jun 2024
CATGNN: Cost-Efficient and Scalable Distributed Training for Graph Neural Networks Xin Huang Weipeng Zhuo Minh Phu Vuong Shiju Li Jongryool Kim Bradley Rees Chul-Ho Lee GNN 25 0 0 02 Apr 2024
ARGO: An Auto-Tuning Runtime System for Scalable GNN Training on Multi-Core Processor Yi-Chien Lin Yuyang Chen Sameh Gobriel Nilesh Jain Gopi Krishna Jha Viktor Prasanna GNN 27 3 0 06 Feb 2024
GLISP: A Scalable GNN Learning System by Exploiting Inherent Structural Properties of Graphs Zhongshu Zhu Bin Jing Xiaopei Wan Zhizhen Liu Lei Liang Jun Zhou 19 2 0 06 Jan 2024
Synergistic Signals: Exploiting Co-Engagement and Semantic Links via Graph Neural Networks Zijie Huang Baolin Li Hafez Asgharzadeh Anne Cocos Lingyi Liu Evan Cox Colby Wise Sudarshan Lamkhede 26 0 0 07 Dec 2023
FastSample: Accelerating Distributed Graph Neural Network Training for Billion-Scale Graphs Hesham Mostafa Adam Grabowski Md Asadullah Turja J. Cerviño Alejandro Ribeiro N. Himayat GNN 21 1 0 29 Nov 2023
GraNNDis: Efficient Unified Distributed Training Framework for Deep GNNs on Large Clusters Jaeyong Song Hongsun Jang Jaewon Jung Youngsok Kim Jinho Lee GNN 14 0 0 12 Nov 2023
Helios: An Efficient Out-of-core GNN Training System on Terabyte-scale Graphs with In-memory Performance Jie Sun Mo Sun Zheng Zhang Jun Xie Zuocheng Shi Zihan Yang Jie M. Zhang Fei Wu Zeke Wang GNN 37 5 0 02 Oct 2023
DistTGL: Distributed Memory-Based Temporal Graph Neural Network Training Hongkuan Zhou Da Zheng Xiang Song George Karypis Viktor Prasanna GNN 21 13 0 14 Jul 2023
A Survey on Graph Neural Network Acceleration: Algorithms, Systems, and Customized Hardware Shichang Zhang Atefeh Sohrabizadeh Cheng Wan Zijie Huang Ziniu Hu Yewen Wang Yingyan Lin Lin Jason Cong Yizhou Sun GNN AI4CE 29 22 0 24 Jun 2023
BatchGNN: Efficient CPU-Based Distributed GNN Training on Very Large Graphs Loc Hoang Rita Brugarolas Brufau Ke Ding Bo Wu GNN 20 2 0 23 Jun 2023
Communication-Efficient Graph Neural Networks with Probabilistic Neighborhood Expansion Analysis and Caching Tim Kaler A. Iliopoulos P. Murzynowski Tao B. Schardl C. E. Leiserson Jie Chen GNN 22 15 0 04 May 2023
HyScale-GNN: A Scalable Hybrid GNN Training System on Single-Node Heterogeneous Architecture Yi-Chien Lin Viktor Prasanna GNN 24 7 0 01 Mar 2023
Scalable Graph Convolutional Network Training on Distributed-Memory Systems G. Demirci Aparajita Haldar Hakan Ferhatosmanoglu GNN 28 9 0 09 Dec 2022
Extreme Acceleration of Graph Neural Network-based Prediction Models for Quantum Chemistry Hatem Helal J. Firoz Jenna A. Bilbrey M. M. Krell Tom Murray Ang Li S. Xantheas Sutanay Choudhury GNN 31 5 0 25 Nov 2022
A Comprehensive Survey on Distributed Training of Graph Neural Networks Haiyang Lin Mingyu Yan Xiaochun Ye Dongrui Fan Shirui Pan Wenguang Chen Yuan Xie GNN 22 23 0 10 Nov 2022
Distributed Graph Neural Network Training: A Survey Yingxia Shao Hongzheng Li Xizhi Gu Hongbo Yin Yawen Li Xupeng Miao Wentao Zhang Bin Cui Lei Chen GNN AI4CE 11 53 0 01 Nov 2022
Layer-Neighbor Sampling -- Defusing Neighborhood Explosion in GNNs M. F. Balin Ümit V. Çatalyürek 19 15 0 24 Oct 2022
Neural Graph Databases Maciej Besta Patrick Iff Florian Scheidl Kazuki Osawa Nikoli Dryden Michal Podstawski Tiancheng Chen Torsten Hoefler AI4CE 45 9 0 20 Sep 2022
Ginex: SSD-enabled Billion-scale Graph Neural Network Training on a Single Machine via Provably Optimal In-memory Caching Yeonhong Park Sunhong Min Jae W. Lee GNN 22 34 0 19 Aug 2022
Characterizing and Understanding Distributed GNN Training on GPUs Haiyang Lin Mingyu Yan Xiaocheng Yang Mo Zou Wenming Li Xiaochun Ye Dongrui Fan GNN 20 13 0 18 Apr 2022
MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networks R. Waleffe J. Mohoney Theodoros Rekatsinas Shivaram Venkataraman GNN 22 25 0 04 Feb 2022
Recursive Multi-Section on the Fly: Shared-Memory Streaming Algorithms for Hierarchical Graph Partitioning and Process Mapping Marcelo Fonseca Faraj Christian Schulz 15 4 0 01 Feb 2022
GraphTheta: A Distributed Graph Neural Network Learning System With Flexible Training Strategy Yongchao Liu Houyi Li Guowei Zhang Xintan Zeng Yongyong Li ... Peng Zhang Zhao Li Kefeng Deng Changhua He Wenguang Chen GNN 42 11 0 21 Apr 2021
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,886 0 15 Sep 2016