HyperQA: Hyperbolic Embeddings for Fast and Efficient Ranking of Question Answer Pairs

Deep learning has become a fundamental component in question answering (QA) research. It is a well established fact that the top QA systems are mostly comprised of neural ranking architectures. Model after model, we see architectural innovations ranging from word interaction layers to creative new methods for learning attentions. It also seems like these complex interaction mechanisms are mandatory for good performance. Unfortunately, many of these mechanisms incur a prohibitive computation and memory cost making them unfavorable for practical applications. In lieu of that, this paper tackles the question of whether it is possible to achieve competitive performance with very simple neural networks. As such, we propose a simple but novel deep learning architecture for fast and efficient QA. Specifically, we present \textsc{HyperQA}, a parameter efficient neural network model that outperforms several parameter heavy models such as Attentive Pooling BiLSTMs and Multi Perspective CNN on multiple standard benchmark datasets such as TrecQA, WikiQA and YahooQA. The key innovation to our model is a pairwise ranking objective that performs QA matching in Hyperbolic space instead of Euclidean space. This empowers our model with a self-organizing ability and enables automatic discovery of latent hierarchies while learning embeddings of questions and answers. Our model requires no feature engineering, no similarity matrix matching, no complicated attention mechanisms nor over-parameterized layers and yet outperforms many models that have these functionalities on multiple benchmarks.