Tunable GMM Kernels

The recently proposed "generalized min-max" (GMM) kernel can be efficiently linearized via hashing or the Nystrom method, with direct applications in large-scale machine learning and fast near neighbor search. As a tuning-free kernel, the linearized GMM kernel has been extensively compared with the normalized random Fourier features (NRFF). On classification tasks, the tuning-free GMM kernel performs (surprisingly) well compared to the best-tuned RBF kernel. Nevertheless, one would naturally expect that the GMM kernel ought to be further improved if we introduce tuning parameters. In this paper, we study two simple constructions of tunable GMM kernels: (i) the exponentiated-GMM (or eGMM) kernel, and (ii) the powered-GMM (or pGMM) kernel. One can of course introduce additional (and more complex) kernels by (e.g.,) combining eGMM and pGMM kernels. The pGMM kernel can still be efficiently linearized by modifying the original hashing procedure for the GMM kernel. On 47 publicly available classification datasets from the UCI repository, we verify that the eGMM and pGMM kernels (typically) improve over the original GMM kernel. On a fraction of datasets, the improvements can be astonishingly significant. We hope our introduction of tunable kernels could offer practitioners the flexibility of choosing appropriate kernels and methods for large-scale search \& learning for their specific applications.