26
153

Reconstruction of Markov Random Fields from Samples: Some Easy Observations and Algorithms

Abstract

Markov random fields are used to model high dimensional distributions in a number of applied areas. Much recent interest has been devoted to the reconstruction of the dependency structure from independent samples from the Markov random fields. We analyze a simple algorithm for reconstructing the underlying graph defining a Markov random field on nn nodes and maximum degree dd given observations. We show that under mild non-degeneracy conditions it reconstructs the generating graph with high probability using Θ(dϵ2δ4logn)\Theta(d \epsilon^{-2}\delta^{-4} \log n) samples where ϵ,δ\epsilon,\delta depend on the local interactions. For most local interaction \eps,δ\eps,\delta are of order exp(O(d))\exp(-O(d)). Our results are optimal as a function of nn up to a multiplicative constant depending on dd and the strength of the local interactions. Our results seem to be the first results for general models that guarantee that {\em the} generating model is reconstructed. Furthermore, we provide explicit O(nd+2ϵ2δ4logn)O(n^{d+2} \epsilon^{-2}\delta^{-4} \log n) running time bound. In cases where the measure on the graph has correlation decay, the running time is O(n2logn)O(n^2 \log n) for all fixed dd. We also discuss the effect of observing noisy samples and show that as long as the noise level is low, our algorithm is effective. On the other hand, we construct an example where large noise implies non-identifiability even for generic noise and interactions. Finally, we briefly show that in some simple cases, models with hidden nodes can also be recovered.

View on arXiv
Comments on this paper

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.