In this paper, we study the problem of estimation and learning under temporal distribution shift. Consider an observation sequence of length , which is a noisy realization of a time-varying groundtruth sequence. Our focus is to develop methods to estimate the groundtruth at the final time-step while providing sharp point-wise estimation error rates. We show that, without prior knowledge on the level of temporal shift, a wavelet soft-thresholding estimator provides an optimal estimation error bound for the groundtruth. Our proposed estimation method generalizes existing researches Mazzetto and Upfal (2023) by establishing a connection between the sequence's non-stationarity level and the sparsity in the wavelet-transformed domain. Our theoretical findings are validated by numerical experiments. Additionally, we applied the estimator to derive sparsity-aware excess risk bounds for binary classification under distribution shift and to develop computationally efficient training objectives. As a final contribution, we draw parallels between our results and the classical signal processing problem of total-variation denoising (Mammen and van de Geer,1997; Tibshirani, 2014), uncovering novel optimal algorithms for such task.
View on arXiv@article{baby2025_2505.15803,
title={ Adaptive Estimation and Learning under Temporal Distribution Shift },
author={ Dheeraj Baby and Yifei Tang and Hieu Duy Nguyen and Yu-Xiang Wang and Rohit Pyati },
journal={arXiv preprint arXiv:2505.15803},
year={ 2025 }
}