Concentration of posterior probabilities and normalized L0 criteria in regression

We review and extend results on frequentist properties of Bayesian and $L_0$ model selection, with a focus on (potentially non-linear) high-dimensional regression. We portray how posterior probabilities and normalized $L_0$ criteria concentrate on the (Kullback-Leibler) optimal model and other subsets of the model space. We show that, when such concentration occurs, there are theoretical bounds on frequentist probabilities of selecting the correct model, type I and type II errors. The results hold in full generality, and help establish the validity of posterior probabilities and normalized $L_0$ criteria to quantify model choice uncertainty. Regarding regression, rather than proving consistency under a given formulation, we help understand how selection depends on the formulation's sparsity and on problem characteristics such as the sample size, signal-to-noise, problem dimension and true sparsity. We also prove new results related to misspecifying the mean or correlation structures, and give tighter rates for pMOM priors than currently available. Finally, we discuss that asymptotically optimal sparse formulations may significantly reduce power, unless $n$ or the signal are large enough, justifying the adoption of less sparse choices to improve power trade-offs. This issue is compounded by the fact misspecifying the mean structure causes an exponential drop in power. Our examples confirm these findings, warning against the use of asymptotic optimality as the only rule to judge the quality of model selection procedures.