We present the Subset Extended Kalman Filter (SEKF) as a method to update previously trained model weights online rather than retraining or finetuning them when the system a model represents drifts away from the conditions under which it was trained. We identify the parameters to be updated using the gradient of the loss function and use the SEKF to update only these parameters. We compare finetuning and SEKF for online model maintenance in the presence of systemic drift through four dynamic regression case studies and find that the SEKF is able to maintain model accuracy as-well if not better than finetuning while requiring significantly less time per iteration, and less hyperparameter tuning.
View on arXiv@article{hammond2025_2503.17681,
title={ Staying Alive: Online Neural Network Maintenance and Systemic Drift },
author={ Joshua E. Hammond and Tyler Soderstrom and Brian A. Korgel and Michael Baldea },
journal={arXiv preprint arXiv:2503.17681},
year={ 2025 }
}