A Lower Bound for Estimating Fréchet Means

Fr\échet means, conceptually appealing, generalize the Euclidean expectation to general metric spaces. We explore how well Fr\échet means can be estimated from independent and identically distributed samples and uncover a fundamental limitation: In the vicinity of a probability distribution $P$ with nonunique means, independent of sample size, it is not possible to uniformly estimate Fr\échet means below a precision determined by the diameter of the set of Fr\échet means of $P$. Implications were previously identified for empirical plug-in estimators as part of the phenomenon \emph{finite sample smeariness}. Our findings thus confirm inevitable statistical challenges in the estimation of Fr\échet means on metric spaces for which there exist distributions with nonunique means. Illustrating the relevance of our lower bound, examples of extrinsic, intrinsic, Procrustes, diffusion and Wasserstein means showcase either deteriorating constants or slow convergence rates of empirical Fr\échet means for samples near the regime of nonunique means.