Can deep learning help you find the perfect match?

Is he/she attractive or not? We can often answer this question in a split of a second, and this ability is one of the main reasons behind the success of recent dating apps. In this paper we explore if we can predict attractiveness from profile pictures with convolutional networks. We argue that the introduced task is difficult due to i) the large number of variations in profile pictures and ii) the noise in attractiveness labels. We find that our self-labeled dataset of $9364$ pictures is too small to apply a convolutional network directly. We resort to transfer learning and compare feature representations transferred from VGGNet and a self-trained gender prediction network. Our findings show that VGGNet features transfer better and we conclude that our best model, achieving $68.1\%$ accuracy on the test set, is moderately successful at predicting attractiveness.