Understanding Training Efficiency of Deep Learning Recommendation Models at Scale

11 November 2020

Papers citing "Understanding Training Efficiency of Deep Learning Recommendation Models at Scale"

50 / 51 papers shown

Title
Ember: A Compiler for Efficient Embedding Operations on Decoupled Access-Execute Architectures Marco Siracusa Olivia Hsu Victor Soria-Pardos Joshua Randall Arnaud Grasset ... Doug Joseph Randy Allen Fredrik Kjolstad Miquel Moretó Planas Adrià Armejach 215 0 0 14 Apr 2025
The Sunk Carbon Fallacy: Rethinking Carbon Footprint Metrics for Effective Carbon-Aware SchedulingACM Symposium on Cloud Computing (SoCC), 2024 Noman Bashir Varun Gohil Anagha Belavadi Mohammad Shahrad David Irwin Elsa Olivetti Christina Delimitrou 189 12 0 19 Oct 2024
Data Deletion for Linear Regression with Noisy SGD Zhangjie Xia Chi-Hua Wang Guang Cheng 278 2 0 12 Oct 2024
CTMBIDS: Convolutional Tsetlin Machine Based Intrusion Detection System for DDoS attacks in an SDN environment Rasoul Jafari Gohari Laya Aliahmadipour M. Rafsanjani AAML 94 5 0 05 Sep 2024
HopGNN: Boosting Distributed GNN Training Efficiency via Feature-Centric Model Migration Weijian Chen Shuibing He Haoyang Qu Xuechen Zhang GNN 284 1 0 01 Sep 2024
CADC: Encoding User-Item Interactions for Compressing Recommendation Model Training Data Hossein Entezari Zarch Abdulla Alshabanah Chaoyi Jiang Murali Annavaram 226 1 0 11 Jul 2024
PreSto: An In-Storage Data Preprocessing System for Training Recommendation Models Yunjae Lee Hyeseong Kim Minsoo Rhu 149 7 0 11 Jun 2024
ElasticRec: A Microservice-based Model Serving Architecture Enabling Elastic Resource Scaling for Recommendation Models Yujeong Choi Jiin Kim Minsoo Rhu 193 2 0 11 Jun 2024
Accelerating Recommender Model Training by Dynamically Skipping Stale Embeddings Yassaman Ebrahimzadeh Maboud Muhammad Adnan Divyat Mahajan Shiyang Chen AI4TS 200 0 0 22 Mar 2024
Blox: A Modular Toolkit for Deep Learning Schedulers Saurabh Agarwal Amar Phanishayee Shivaram Venkataraman OffRL 112 11 0 19 Dec 2023
MAD Max Beyond Single-Node: Enabling Large Machine Learning Model Acceleration on Distributed SystemsInternational Symposium on Computer Architecture (ISCA), 2023 Samuel Hsia Alicia Golden Bilge Acun Newsha Ardalani Zach DeVito Gu-Yeon Wei David Brooks Carole-Jean Wu MoE 284 14 0 04 Oct 2023
Saturn: An Optimized Data System for Large Model Deep Learning WorkloadsProceedings of the VLDB Endowment (PVLDB), 2023 Kabir Nagrecha Arun Kumar 296 8 0 03 Sep 2023
Dynamic Embedding Size Search with Minimum Regret for Streaming Recommender SystemInternational Conference on Information and Knowledge Management (CIKM), 2023 Bowei He Xu He Renrui Zhang Yingxue Zhang Ruiming Tang Chen Ma AI4TS 130 13 0 15 Aug 2023
InTune: Reinforcement Learning-based Data Pipeline Optimization for Deep Recommendation ModelsACM Conference on Recommender Systems (RecSys), 2023 Kabir Nagrecha Lingyi Liu P. Delgado Prasanna Padmanabhan OffRL AI4CE 204 8 0 13 Aug 2023
Evaluating and Enhancing Robustness of Deep Recommendation Systems Against Hardware Errors Dongning Ma Xun Jiao Fred Lin Mengshi Zhang Alban Desmaison Thomas Sellinger Daniel Moore Sriram Sankar 111 2 0 17 Jul 2023
Mem-Rec: Memory Efficient Recommendation System using Alternative RepresentationAsian Conference on Machine Learning (ACML), 2023 Gopu Krishna Jha Anthony Thomas Nilesh Jain Sameh Gobriel Tajana Rosing Ravi Iyer 171 4 0 12 May 2023
Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost ModelsConference on Machine Learning and Systems (MLSys), 2023 Daochen Zha Louis Feng Liangchen Luo Bhargav Bhushanam Zirui Liu ... J. McMahon Yuzhen Huang Bryan Clarke A. Kejariwal Helen Zhou 196 10 0 03 May 2023
MP-Rec: Hardware-Software Co-Design to Enable Multi-Path RecommendationInternational Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2023 Samuel Hsia Udit Gupta Bilge Acun Newsha Ardalani Pan Zhong Gu-Yeon Wei David Brooks Carole-Jean Wu 189 20 0 21 Feb 2023
FlexShard: Flexible Sharding for Industry-Scale Sequence Recommendation Models Geet Sethi Pallab Bhattacharya Dhruv Choudhary Carole-Jean Wu Christos Kozyrakis 192 5 0 08 Jan 2023
Systems for Parallel and Distributed Large-Model Deep Learning Training Kabir Nagrecha GNN VLM MoE 139 9 0 06 Jan 2023
A Survey on Federated Recommendation SystemsIEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2022 Zehua Sun Yonghui Xu Wenshu Fan Weiliang He Lanju Kong Fangzhao Wu Yiheng Jiang Li-zhen Cui FedML 230 95 0 27 Dec 2022
Data Leakage via Access Patterns of Sparse Features in Deep Learning-based Recommendation Systems H. Hashemi Wenjie Xiong Liu Ke Kiwan Maeng M. Annavaram G. E. Suh Hsien-Hsin S. Lee 113 6 0 12 Dec 2022
COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training D. Kadiyala Saeed Rashidi Taekyung Heo Abhimanyu Bambhaniya T. Krishna Alexandros Daglis VLM 161 10 0 30 Nov 2022
RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training InfrastructureConference on Machine Learning and Systems (MLSys), 2022 Mark Zhao Dhruv Choudhary Devashish Tyagi A. Somani Max Kaplan ... Jongsoo Park Aarti Basant Niket Agarwal Carole-Jean Wu Christos Kozyrakis VLM 329 10 0 09 Nov 2022
DreamShard: Generalizable Embedding Table Placement for Recommender SystemsNeural Information Processing Systems (NeurIPS), 2022 Daochen Zha Louis Feng Qiaoyu Tan Zirui Liu Kwei-Herng Lai Bhargav Bhushanam Yuandong Tian A. Kejariwal Helen Zhou LMTD OffRL 249 35 0 05 Oct 2022
Understanding Scaling Laws for Recommendation Models Newsha Ardalani Carole-Jean Wu Zeliang Chen Bhargav Bhushanam Adnan Aziz 164 45 0 17 Aug 2022
AutoShard: Automated Embedding Table Sharding for Recommender SystemsKnowledge Discovery and Data Mining (KDD), 2022 Daochen Zha Louis Feng Bhargav Bhushanam Dhruv Choudhary Jade Nie Yuandong Tian Jay Chae Yi-An Ma A. Kejariwal Helen Zhou 150 33 0 12 Aug 2022
FEL: High Capacity Learning for Recommendation and Ranking via Federated Ensemble Learning Meisam Hejazinia Dzmitry Huba Ilias Leontiadis Kiwan Maeng Mani Malek Luca Melis Ilya Mironov Milad Nasr Kaikai Wang Carole-Jean Wu FedML 208 8 0 07 Jun 2022
Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data HeterogeneityACM Conference on Recommender Systems (RecSys), 2022 Kiwan Maeng Haiyu Lu Luca Melis John Nguyen Michael G. Rabbat Carole-Jean Wu FedML 202 37 0 30 May 2022
GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation ModelNeural Information Processing Systems (NeurIPS), 2022 Yuchi Xu Yuanxing Zhang Yufeng Cai Kaixu Ren Pengjie Wang ... Jing Chen Hongbo Deng Jian Xu Lin Qu Bo Zheng 144 5 0 23 May 2022
Heterogeneous Acceleration Pipeline for Recommendation System TrainingInternational Symposium on Computer Architecture (ISCA), 2022 Muhammad Adnan Yassaman Ebrahimzadeh Maboud Divyat Mahajan Shiyang Chen 190 23 0 11 Apr 2022
ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications Yifan Yuan Jing-yu Huang Yan Sun Tianchen Wang Jacob Nelson Dan R. K. Ports Yipeng Wang Ren Wang Charlie Tai Nam Sung Kim 171 2 0 16 Mar 2022
GPU-Initiated On-Demand High-Throughput Storage Access in the BaM System ArchitectureInternational Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2022 Zaid Qureshi Vikram Sharma Mailthody Isaac Gelado S. Min Amna Masood ... Dmitri Vainbrand I-Hsin Chung M. Garland W. Dally Wen-mei W. Hwu GNN 124 42 0 09 Mar 2022
BagPipe: Accelerating Deep Recommendation Model TrainingSymposium on Operating Systems Principles (SOSP), 2022 Saurabh Agarwal Chengpo Yan Ziyi Zhang Shivaram Venkataraman 201 25 0 24 Feb 2022
TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training JobsSymposium on Networked Systems Design and Implementation (NSDI), 2022 Weiyang Wang Moein Khazraee Zhizhen Zhong M. Ghobadi Zhihao Jia Dheevatsa Mudigere Ying Zhang A. Kewitsch 419 140 0 01 Feb 2022
RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural RecommendationInternational Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2022 Geet Sethi Bilge Acun Niket Agarwal Christos Kozyrakis Caroline Trippel Carole-Jean Wu 199 78 0 25 Jan 2022
Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion ParametersKnowledge Discovery and Data Mining (KDD), 2021 Xiangru Lian Binhang Yuan Xuefeng Zhu Yulong Wang Yongjun He ... Lei Yuan Hai-bo Yu Sen Yang Ce Zhang Ji Liu VLM 222 41 0 10 Nov 2021
Modeling Techniques for Machine Learning Fairness: A Survey Mingyang Wan Daochen Zha Ninghao Liu Na Zou SyDa FaML 259 39 0 04 Nov 2021
Sustainable AI: Environmental Implications, Challenges and OpportunitiesConference on Machine Learning and Systems (MLSys), 2021 Carole-Jean Wu Ramya Raghavendra Udit Gupta Bilge Acun Newsha Ardalani ... Maximilian Balandat Joe Spisak R. Jain Michael G. Rabbat K. Hazelwood 339 520 0 30 Oct 2021
Boost-RS: Boosted Embeddings for Recommender Systems and its Application to Enzyme-Substrate Interaction Prediction Xinmeng Li Liping Liu S. Hassoun 61 0 0 28 Sep 2021
Understanding Data Storage and Ingestion for Large-Scale Deep Recommendation Model Training Mark Zhao Niket Agarwal Aarti Basant B. Gedik Satadru Pan ... Kevin Wilfong Harsha Rastogi Carole-Jean Wu Christos Kozyrakis Parikshit Pol GNN 249 93 0 20 Aug 2021
AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated LearningMicro (MICRO), 2021 Young Geun Kim Carole-Jean Wu 215 105 0 16 Jul 2021
RecPipe: Co-designing Models and Hardware to Jointly Optimize Recommendation Quality and PerformanceMicro (MICRO), 2021 Udit Gupta Samuel Hsia J. Zhang Mark Wilkening Javin Pombra Hsien-Hsin S. Lee Gu-Yeon Wei Carole-Jean Wu David Brooks 219 33 0 18 May 2021
Alternate Model Growth and Pruning for Efficient Training of Recommendation SystemsInternational Conference on Machine Learning and Applications (ICMLA), 2021 Xiaocong Du Bhargav Bhushanam Jiecao Yu Dhruv Choudhary Tianxiang Gao Sherman Wong Louis Feng Jongsoo Park Yu Cao A. Kejariwal 136 7 0 04 May 2021
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation ModelsInternational Symposium on Computer Architecture (ISCA), 2021 Dheevatsa Mudigere Y. Hao Jianyu Huang Zhihao Jia Andrew Tulloch ... Ajit Mathews Lin Qiao M. Smelyanskiy Bill Jia Vijay Rao 479 176 0 12 Apr 2021
ECRM: Efficient Fault Tolerance for Recommendation Model Training via Erasure Coding Kaige Liu J. Kosaian K. V. Rashmi 151 4 0 05 Apr 2021
Accelerating Recommendation System Training by Leveraging Popular ChoicesProceedings of the VLDB Endowment (PVLDB), 2021 Muhammad Adnan Yassaman Ebrahimzadeh Maboud Divyat Mahajan Shiyang Chen 206 66 0 01 Mar 2021
TT-Rec: Tensor Train Compression for Deep Learning Recommendation ModelsConference on Machine Learning and Systems (MLSys), 2021 Chunxing Yin Bilge Acun Xing Liu Carole-Jean Wu 219 113 0 25 Jan 2021
CPR: Understanding and Improving Failure Tolerant Training for Deep Learning Recommendation with Partial Recovery Kiwan Maeng Shivam Bharuka Isabel Gao M. C. Jeffrey V. Saraph ... Caroline Trippel Jiyan Yang Michael G. Rabbat Brandon Lucia Carole-Jean Wu OffRL 243 39 0 05 Nov 2020
Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights Shail Dave Riyadh Baghdadi Tony Nowatzki Sasikanth Avancha Aviral Shrivastava Baoxin Li 263 98 0 02 Jul 2020