Nimble Algorithms for Cloud Computing

Cloud computing is a new paradigm where data is stored across multiple servers and the goal is to compute a function of all the data. We focus on a simple setting, where each server uses polynomial time and space, but communication among servers is more expensive and is ideally bounded by a polylogarithmic function of the input size. We dub algorithms that satisfy these types of resource bounds as {\it nimble}. The main contribution of the paper is to develop nimble algorithms for several areas which involve massive data and for that reason have been extensively studied in the context of streaming algorithms. The areas are approximation of frequency moments, counting bipartite homomorphisms, rank-$k$ approximation to a matrix, and $k$-means clustering. For frequency moments, we use a new importance sampling technique based on high powers of the frequencies. Estimating the $k$-th frequency moment for $k>2$ in the cloud setting has a communication lower bound of $\Omega(s^{k-1})$. We achieve a nearly matching upper bound of $O(s^k)$ improving upon previous results of Woodruff and Zhang. In the streaming model, this problem has a space lower bound of $n^{1-2/k}$. We reduce the problem of counting homomorphisms to estimating implicitly defined frequency moments. For low-rank approximation of an $n \times d$ matrix $A = A_1 + ... + A_s$, $n >> d$, in which $A_i$ is held by the $i$-th server, we avoid communication dependent on $n$ in streaming lower bounds by allowing each party to locally output the projection of his/her input matrix $A_i$ onto the same $k$-dimensional subspace found by the algorithm, so that the sum of the projections is a rank-$k$ approximation to $A$. For clustering, we use our rank-$k$ approximation and the small coreset of Chen of size at most polynomial in the dimension. We show that the computation of the coreset can be done by a nimble algorithm.