Differential Privacy with Random Projections and Sign Random Projections

In this paper, we develop a series of differential privacy (DP) algorithms from a family of random projections (RP), for general applications in machine learning, data mining, and information retrieval. Among the presented algorithms, \textbf{iDP-SignRP} is remarkably effective under the setting of ``individual differential privacy'' (iDP), based on sign random projections (SignRP). Also, \textbf{DP-SignOPORP} considerably improves existing algorithms in the literature under the standard DP setting, using ``one permutation + one random projection'' (OPORP), where OPORP is a variant of the celebrated count-sketch method with fixed-length binning and normalization. Without taking signs, among the DP-RP family, \textbf{DP-OPORP} achieves the best performance. The concept of iDP (individual differential privacy) is defined only on a particular dataset of interest. While iDP is not strictly DP, iDP might be useful in certain applications, such as releasing a dataset (including sharing embeddings across companies or countries). In our study, we find that \textbf{iDP-SignRP} is remarkably effective for search and machine learning applications, in that the utilities are exceptionally good even at a very small privacy parameter $\epsilon$ (e.g., $\epsilon<0.5$).