Are Sampling Heuristics Necessary in Object Detectors?

The prevalent object detectors to date, such as Faster R-CNN and RetinaNet, are always equipped with a hard or soft sampling heuristics (e.g., under-sampling or Focal Loss), which has been considered as a necessary component for mitigating the foreground-background imbalance thus far. In this report, we challenge this paradigm. Our discovery reveals that, by decoupling objectness estimation from classification to transfer the imbalance, the sampling heuristics could be abandoned in object detectors (e.g., Faster R-CNN, RetinaNet, FCOS), with equivalent performance than their vanilla models. As the sampling heuristics usually introduces laborious hyper-parameters tuning, we expect our discovery could simplify the training procedure of object detectors. Code is available at https://github.com/ChenJoya/objnessdet.