Sum-of-Parts: Self-Attributing Neural Networks with End-to-End Learning of Feature Groups

Self-attributing neural networks (SANNs) present a potential path towards interpretable models for high-dimensional problems, but often face significant trade-offs in performance. In this work, we formally prove a lower bound on errors of per-feature SANNs, whereas group-based SANNs can achieve zero error and thus high performance. Motivated by these insights, we propose Sum-of-Parts (SOP), a framework that transforms any differentiable model into a group-based SANN, where feature groups are learned end-to-end without group supervision. SOP achieves state-of-the-art performance for SANNs on vision and language tasks, and we validate that the groups are interpretable on a range of quantitative and semantic metrics. We further validate the utility of SOP explanations in model debugging and cosmological scientific discovery. Code is available atthis https URL.

@article{you2025_2310.16316,
  title={ Sum-of-Parts: Self-Attributing Neural Networks with End-to-End Learning of Feature Groups },
  author={ Weiqiu You and Helen Qu and Marco Gatti and Bhuvnesh Jain and Eric Wong },
  journal={arXiv preprint arXiv:2310.16316},
  year={ 2025 }
}