Ads recommendation is a prominent service of online advertising systems and has been actively studied. Recent studies indicate that scaling-up and advanced design of the recommendation model can bring significant performance improvement. However, with a larger model scale, such prior studies have a significantly increasing gap from industry as they often neglect two fundamental challenges in industrial-scale applications. First, training and inference budgets are restricted for the model to be served, exceeding which may incur latency and impair user experience. Second, large-volume data arrive in a streaming mode with data distributions dynamically shifting, as new users/ads join and existing users/ads leave the system. We propose the External Large Foundation Model (ExFM) framework to address the overlooked challenges. Specifically, we develop external distillation and a data augmentation system (DAS) to control the computational cost of training/inference while maintaining high performance. We design the teacher in a way like a foundation model (FM) that can serve multiple students as vertical models (VMs) to amortize its building cost. We propose Auxiliary Head and Student Adapter to mitigate the data distribution gap between FM and VMs caused by the streaming data issue. Comprehensive experiments on internal industrial-scale applications and public datasets demonstrate significant performance gain by ExFM.
View on arXiv@article{liang2025_2502.17494,
title={ External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation },
author={ Mingfu Liang and Xi Liu and Rong Jin and Boyang Liu and Qiuling Suo and Qinghai Zhou and Song Zhou and Laming Chen and Hua Zheng and Zhiyuan Li and Shali Jiang and Jiyan Yang and Xiaozhen Xia and Fan Yang and Yasmine Badr and Ellie Wen and Shuyu Xu and Hansey Chen and Zhengyu Zhang and Jade Nie and Chunzhi Yang and Zhichen Zeng and Weilin Zhang and Xingliang Huang and Qianru Li and Shiquan Wang and Evelyn Lyu and Wenjing Lu and Rui Zhang and Wenjun Wang and Jason Rudy and Mengyue Hang and Kai Wang and Yinbin Ma and Shuaiwen Wang and Sihan Zeng and Tongyi Tang and Xiaohan Wei and Longhao Jin and Jamey Zhang and Marcus Chen and Jiayi Zhang and Angie Huang and Chi Zhang and Zhengli Zhao and Jared Yang and Qiang Jin and Xian Chen and Amit Anand Amlesahwaram and Lexi Song and Liang Luo and Yuchen Hao and Nan Xiao and Yavuz Yetim and Luoshang Pan and Gaoxiang Liu and Yuxi Hu and Yuzhen Huang and Jackie Xu and Rich Zhu and Xin Zhang and Yiqun Liu and Hang Yin and Yuxin Chen and Buyun Zhang and Xiaoyi Liu and Xingyuan Wang and Wenguang Mao and Zhijing Li and Zhehui Zhou and Feifan Gu and Qin Huang and Chonglin Sun and Nancy Yu and Shuo Gu and Shupin Mao and Benjamin Au and Jingzheng Qin and Peggy Yao and Jae-Woo Choi and Bin Gao and Ernest Wang and Lei Zhang and Wen-Yen Chen and Ted Lee and Jay Zha and Yi Meng and Alex Gong and Edison Gao and Alireza Vahdatpour and Yiping Han and Yantao Yao and Toshinari Kureha and Shuo Chang and Musharaf Sultan and John Bocharov and Sagar Chordia and Xiaorui Gan and Peng Sun and Rocky Liu },
journal={arXiv preprint arXiv:2502.17494},
year={ 2025 }
}