41
0

For Kernel Range Spaces a Constant Number of Queries Are Sufficient

Main:19 Pages
1 Figures
Bibliography:3 Pages
1 Tables
Appendix:7 Pages
Abstract

We introduce the notion of an ε\varepsilon-cover for a kernel range space. A kernel range space concerns a set of points XRdX \subset \mathbb{R}^d and the space of all queries by a fixed kernel (e.g., a Gaussian kernel K(p,)=exp(p2)K(p,\cdot) = \exp(-\|p-\cdot\|^2)). For a point set XX of size nn, a query returns a vector of values RpRnR_p \in \mathbb{R}^n, where the iith coordinate (Rp)i=K(p,xi)(R_p)_i = K(p,x_i) for xiXx_i \in X. An ε\varepsilon-cover is a subset of points QRdQ \subset \mathbb{R}^d so for any pRdp \in \mathbb{R}^d that 1nRpRq1ε\frac{1}{n} \|R_p - R_q\|_1\leq \varepsilon for some qQq \in Q. This is a smooth analog of Haussler's notion of ε\varepsilon-covers for combinatorial range spaces (e.g., defined by subsets of points within a ball query) where the resulting vectors RpR_p are in {0,1}n\{0,1\}^n instead of [0,1]n[0,1]^n. The kernel versions of these range spaces show up in data analysis tasks where the coordinates may be uncertain or imprecise, and hence one wishes to add some flexibility in the notion of inside and outside of a query range. Our main result is that, unlike combinatorial range spaces, the size of kernel ε\varepsilon-covers is independent of the input size nn and dimension dd. We obtain a bound of (1/ε)O~(1/ε2)(1/\varepsilon)^{\tilde O(1/\varepsilon^2)}, where O~(f(1/ε))\tilde{O}(f(1/\varepsilon)) hides log factors in (1/ε)(1/\varepsilon) that can depend on the kernel. This implies that by relaxing the notion of boundaries in range queries, eventually the curse of dimensionality disappears, and may help explain the success of machine learning in very high-dimensions. We also complement this result with a lower bound of almost (1/ε)Ω(1/ε)(1/\varepsilon)^{\Omega(1/\varepsilon)}, showing the exponential dependence on 1/ε1/\varepsilon is necessary.

View on arXiv
@article{phillips2025_2306.16516,
  title={ Dimension-Independent Kernel ε-Covers },
  author={ Jeff M. Phillips and Hasan Pourmahmood-Aghababa },
  journal={arXiv preprint arXiv:2306.16516},
  year={ 2025 }
}
Comments on this paper