Data Generation as Sequential Decision Making

We connect a broad class of generative models through their shared reliance on sequential decision making. We show how changes motivated by our point of view can improve an already-strong model, and then explore this idea further in the context of data imputation -- perhaps the simplest setting in which to investigate the relation between unconditional and conditional generative modelling. We formulate data imputation as an MDP and develop models capable of representing effective policies for it. We construct our models using neural networks and train them using a form of guided policy search. Through empirical tests, we show that our approach can learn effective policies for imputation problems of varying difficulty and across multiple datasets.