Differentially Private Support Vector Machines

We provide differentially private support vector machines: SVM algorithms that are private under the {\it $\epsilon$-differential privacy} definition due to Dwork et al. (2006). We analyze their learning performance, and provide generalization bounds for linear and nonlinear kernels. With the goal of designing practical privacy-preserving machine learning algorithms, we generalize the work of Chaudhuri and Monteleoni (2008) for learning linear separators via regularized logistic regression. Our results give privacy and utility guarantees for a larger class of convex optimization problems, and a special instance of our results is the case of linear SVMs. We provide new algorithms to address the important challenge of how to release a kernel classifier without releasing any of the training data. We develop a method to tune algorithm parameters in a privacy-preserving manner, thereby providing end-to-end privacy guarantees for the training process. In addition to providing general, efficient algorithms for linear and nonlinear kernel SVMs, coupled with $\epsilon$-differential privacy guarantees and generalization bounds, we demonstrate promising empirical performance on real and simulated data sets.