Improving sample diversity of a pre-trained, class-conditional GAN by changing its class embeddings

Mode collapse is a well-known issue in Generative Adversarial Networks (GANs) posing a big challenge to the research community. We propose a simple solution to mode collapse i.e. improving the sample diversity of a pre-trained class-conditional GAN by modifying only its class embeddings. We search for a class embedding that increases sample diversity over a batch of latent vectors. To keep the samples in correct classes while the embeddings change in the direction of maximizing sample diversity, we also move the embeddings in the direction of maximizing the log probability outputs of an auxiliary classifier pre-trained on the same dataset. Our method improves the sample diversity of state-of-the-art ImageNet BigGANs at both 128x128 and 256x256 resolutions. By replacing only the embeddings, we can also synthesize plausible images for Places365 using a BigGAN generator pre-trained on ImageNet, revealing the surprising expressivity of the BigGAN class embedding space.