Exploring Simple, High Quality Out-of-Distribution Detection with L2 Normalization

We demonstrate that L2 normalization over feature space--an extremely simple method requiring no additional training strategies, hyperparameters, specialized loss functions or image augmentation--can produce competitive results for Out-of-Distribution (OoD) detection with a fraction of the training time (60 epochs with ResNet18, 100 epochs with ResNet50) required by more sophisticated methods. We show theoretically and empirically that our simple method decouples feature norms from the Neural Collapse (NC) constraints imposed by CE loss minimization. This decoupling preserves more feature-level information than a standard CE loss training regime, and allows greater separability between ID norms and near-OoD or far-OoD norms. Our goal is to provide insight toward fundamental, model-based approaches to OoD detection, with less reliance on external factors such as hyperparameter tuning or specialized training regimes. We suggest that L2 normalization provides a collection of benefits large enough to warrant consideration as a standard architecture choice.