Adversarial Training For Sketch Retrieval

Generative Adversarial Networks (GAN) can learn excellent representations for unlabelled data which have been applied to image generation and scene classification. The representations have not yet - to the best of our knowledge - been applied to visual search. In this paper, we show that representations learned by GANs can be applied to visual search within heritage documents that contain Merchant Marks, sketch-like symbols that are similar to hieroglyphs. We introduce a novel GAN architecture with design features that makes it suitable for sketch understanding. The performance of this sketch-GAN is compared to a modified version of the original GAN architecture with respect to simple invariance properties. Experiments suggest that sketch-GANs learn representations that are suitable for retrieval and which also have increased stability to rotation, scale and translation.