Constrained Maximum Likelihood Estimators for Densities

We propose a framework for nonparametric maximum likelihood estimation of densities in situations where the sample is supplemented by information and assumptions about shape, support, continuity, slope, location of modes, density values, and other conditions that, individually or in combination, restrict the family of densities under consideration. We establish existence of estimators and their cluster points, strong consistency under mild assumptions, and robustness in the presence of model misspecification. The results are achieved by means of viewing densities as elements of spaces of semicontinuous functions with the hypo-distance metric. This metric emerges as natural and convenient when considering broad classes of side conditions. It also has the exceptional property that convergence of densities in this metric implies convergence of modes, near-modes, height of modes, and high-likelihood events. Thus, we automatically achieve strong consistency of a rich class of plug-in estimators for modes and related quantities. Relying on almost sure epi-convergence of criterion functions, we avoid the strong assumptions associated with uniform laws of large numbers and instead leverage a less demanding law, for which we provide a new proof. Specific examples illustrate the framework including an estimator simultaneously subject to bounds on density values and its (sub)gradients, restriction to concavity, penalization that encourages lower modes, and imprecise information about the expected value.