Salient Object Detection in the Deep Learning Era: An In-Depth Survey

As an essential problem in computer vision, salient object detection (SOD) has attracted an increasing amount of research effort over the years. Recent advances in SOD are dominantly led by deep learning-based solutions (named deep SODs). To facilitate the in-depth understanding of deep SODs, in this paper, we provide a comprehensive survey covering various aspects ranging from algorithm taxonomy to unsolved open issues. In particular, we first review deep SOD algorithms from different perspectives, including network architecture, level of supervision, learning paradigm, and object/instance level detection. Following that, we summarize and analyze existing SOD datasets and evaluation metrics. Then, we benchmark a large group of representative SOD models, and provide detailed analyses of the comparison results. Moreover, we study the performance of SOD algorithms under different attributes, which have been hardly explored previously, by constructing a novel SOD dataset with rich attribute annotations covering various salient object types, challenging factors, and scene categories. We further analyze, for the first time in the field, the robustness of SOD models w.r.t. random input perturbations and adversarial attacks. We also look into the generalization and hardness of existing SOD datasets. Finally, we discuss several open issues of SOD and outline future research directions. All the saliency prediction maps, our constructed dataset with annotations, and codes for evaluation are publicly available at https://github.com/wenguanwang/SODsurvey.