ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2505.13300
273
7
v1v2v3 (latest)

DD-Ranking: Rethinking the Evaluation of Dataset Distillation

19 May 2025
Zekai Li
Xinhao Zhong
Samir Khaki
Zhiyuan Liang
Yuhao Zhou
M. Shi
Ziqiao Wang
Xuanlei Zhao
Wangbo Zhao
Ziheng Qin
Mengxuan Wu
Pengfei Zhou
Haonan Wang
David Junhao Zhang
Jia-Wei Liu
Shaobo Wang
Dai Liu
Linfeng Zhang
Guang Li
Kun Wang
Zheng Hua Zhu
Zhiheng Ma
Joey Tianyi Zhou
Jiancheng Lv
Yaochu Jin
Peihao Wang
Kaipeng Zhang
Lingjuan Lyu
Yiran Huang
Zeynep Akata
Zhiwei Deng
Xindi Wu
George Cazenavette
Yuzhang Shang
J. Cui
Jindong Gu
Qian Zheng
Hao Ye
Shuo Wang
Xiaobo Wang
Yan Yan
Angela Yao
Mike Zheng Shou
Tianlong Chen
Hakan Bilen
Baharan Mirzasoleiman
Manolis Kellis
Konstantinos N Plataniotis
Zhangyang Wang
Bo Zhao
Yang You
Kai Wang
    DD
ArXiv (abs)PDFHTMLGithub (68★)
Main:9 Pages
3 Figures
Bibliography:5 Pages
20 Tables
Appendix:7 Pages
Abstract

In recent years, dataset distillation has provided a reliable solution for data compression, where models trained on the resulting smaller synthetic datasets achieve performance comparable to those trained on the original datasets. To further improve the performance of synthetic datasets, various training pipelines and optimization objectives have been proposed, greatly advancing the field of dataset distillation. Recent decoupled dataset distillation methods introduce soft labels and stronger data augmentation during the post-evaluation phase and scale dataset distillation up to larger datasets (e.g., ImageNet-1K). However, this raises a question: Is accuracy still a reliable metric to fairly evaluate dataset distillation methods? Our empirical findings suggest that the performance improvements of these methods often stem from additional techniques rather than the inherent quality of the images themselves, with even randomly sampled images achieving superior results. Such misaligned evaluation settings severely hinder the development of DD. Therefore, we propose DD-Ranking, a unified evaluation framework, along with new general evaluation metrics to uncover the true performance improvements achieved by different methods. By refocusing on the actual information enhancement of distilled datasets, DD-Ranking provides a more comprehensive and fair evaluation standard for future research advancements.

View on arXiv
Comments on this paper