Finding the global semantic representation in GAN through Frechet Mean

The ideally disentangled latent space in GAN involves the global representation of latent space with semantic attribute coordinates. In other words, considering that this disentangled latent space is a vector space, there exists the global semantic basis where each basis component describes one attribute of generated images. In this paper, we propose an unsupervised method for finding this global semantic basis in the intermediate latent space in GANs. This semantic basis represents sample-independent meaningful perturbations that change the same semantic attribute of an image on the entire latent space. The proposed global basis, called Fr\échet basis, is derived by introducing Fr\échet mean to the local semantic perturbations in a latent space. Fr\échet basis is discovered in two stages. First, the global semantic subspace is discovered by the Fr\échet mean in the Grassmannian manifold of the local semantic subspaces. Second, Fr\échet basis is found by optimizing a basis of the semantic subspace via the Fr\échet mean in the Special Orthogonal Group. Experimental results demonstrate that Fr\échet basis provides better semantic factorization and robustness compared to the previous methods. Moreover, we suggest the basis refinement scheme for the previous methods. The quantitative experiments show that the refined basis achieves better semantic factorization while constrained on the same semantic subspace given by the previous method.