Variational methods for Conditional Multimodal Learning: Generating Human Faces from Attributes

Prior to this decade, the field of computer vision was primarily focused around hand-crafted feature extraction methods used in conjunction with discriminative models for specific tasks such as object recognition, detection/localization, tracking etc. A generative image understanding was neither within reach nor the prime concern of the period. In this paper, we address the following problem: Given a description of a human face, can we generate the image corresponding to it? We frame this problem as a conditional modality learning problem and use variational methods for maximizing the corresponding conditional log-likelihood. The resultant deep model, which we refer to as conditional multimodal autoencoder (CMMA), forces the latent representation obtained from the attributes alone to be 'close' to the joint representation obtained from both face and attributes. We show that the faces generated from attributes using the proposed model, are qualitatively and quantitatively more representative of the attributes from which they were generated, than those obtained by other deep generative models. We also propose a secondary task, whereby the existing faces are modified by modifying the corresponding attributes. We observe that the modifications in face introduced by the proposed model are representative of the corresponding modifications in attributes. Hence, our proposed method solves the above mentioned problem.