Architecture Selection via the Trade-off Between Accuracy and Robustness

We provide a general framework for characterizing the trade-off between accuracy and robustness in supervised learning. We propose a method and define quantities to characterize the trade-off between accuracy and robustness for a given architecture, and provide theoretical insight into the trade-off. Specifically we introduce a simple trade-off curve, define and study an influence function that captures the sensitivity, under adversarial attack, of the optima of a given loss function. We further show how adversarial training regularizes the parameters in an over-parameterized linear model, recovering the LASSO and ridge regression as special cases, which also allows us to theoretically analyze the behavior of the trade-off curve. In experiments, we demonstrate the corresponding trade-off curves of neural networks and how they vary with respect to factors such as number of layers, neurons, and across different network structures. Such information provides a useful guideline to architecture selection.

@article{deng2025_1906.01354,
  title={ Architecture Selection via the Trade-off Between Accuracy and Robustness },
  author={ Zhun Deng and Cynthia Dwork and Jialiang Wang and Yao Zhao },
  journal={arXiv preprint arXiv:1906.01354},
  year={ 2025 }
}