Random Function Priors for Correlation Modeling

The likelihood model of many high dimensional data $X_n$ can be expressed as $p(X_n|Z_n,\theta)$, where $\theta\mathrel{\mathop:}=(\theta_k)_{k\in[K]}$ is a collection of hidden features shared across objects (indexed by $n$). And $Z_n$ is a non-negative factor loading vector with $K$ entries where $Z_{nk}$ indicates the strength of $\theta_k$ used to express $X_n$. In this paper, we introduce random function priors for $Z_n$ that capture rich correlations among its entries $Z_{n1}$ through $Z_{nK}$. In particular, our model can be treated as a generalized paintbox model~\cite{Broderick13} using random functions, which can be learned efficiently via amortized variational inference. We derive our model by applying a representation theorem on separately exchangeable discrete random measures.