Multimodal Controller for Generative Models

Class-conditional generative models are crucial tools for data generation from user-specified class labels. A number of existing approaches for class-conditional generative models require nontrivial modifications of existing architectures, in order to model conditional information fed into the model. In this paper, we introduce a plug-and-play module called 'multimodal controller' in order to generate multimodal data without introducing additional learning parameters. In the absence of the controllers, our model reduces to non-conditional generative models. We test the efficacy of multimodal controller on CIFAR10 and Omniglot datasets, and experimentally demonstrate that multimodal controlled generative models (including VAE, PixelCNN, Glow, and GAN) are capable of generating class-conditional images of better or comparable quality when compared with the state-of-the-art conditional generative models. Moreover, we show that multimodal controlled models are also capable of transiting images between classes and creating images from novel data modalities.