Gaussian processes and Bayesian moment estimation

Given a set of estimating equations (EE) that characterize a parameter $\theta$, we investigate a semiparametric Bayesian approach for inference on $\theta$ that does not restrict the data distribution $F$ apart from the EE. As main contribution, we construct a degenerate Gaussian process prior that, conditionally on $\theta$, restricts the $F$ generated by this prior to satisfy the EE with probability one. Our prior works even in the more involved case where the number of EE is larger than the dimension of $\theta$. We show that this prior is computationally convenient. Since the likelihood function is not specified by the model, we approximate it based on a linear functional transformation of $F$ that has an asymptotically Gaussian empirical counterpart. This likelihood is used to construct the posterior distribution. We provide a frequentist validation of our procedure by showing consistency and asymptotic normality of the posterior distribution of $\theta$.