ASURA: Scalable and Uniform Data Distribution Algorithm for Storage Clusters

Large-scale storage cluster systems need to manage a vast amount of information denoting combinations of data identifiers (IDs) and corresponding nodes that store the data. Management using data distribution algorithms, rather than management using tables, has been proposed in recent research. In algorithm management, data are distributed in accordance with a data distribution algorithm that is capable of determining, on the basis of the datum ID, the node in which the required data is being stored. Among the requirements for a data distribution algorithm are short calculation times, low memory consumption, uniform data distribution in accordance with the capacity of each node and the ability to handle the addition or removal of nodes. This paper presents a data distribution algorithm called ASURA (Advanced Scalable and Uniform storage by Random number Algorithm), which satisfies these requirements. It offers roughly 0.6-{\mu}s calculation time, kilobyte-order memory consumption, less than 1% maximum variability between nodes in data distribution, data distribution in accordance with the capacity of each node and optimal data movement to maintain data distribution in accordance with node capacity when nodes are added or removed. ASURA is contrasted in this paper qualitatively and quantitatively with representative data distribution algorithms: Consistent Hashing and Straw Buckets in CRUSH. The comparison results show that ASURA can achieve the same storage cluster capacity as Consistent Hashing with dozens fewer nodes by virtue of the uniformity of its distribution with the same level calculation time. They also show that the execution time of ASURA is shorter than that of Straw Buckets in CRUSH. The results reveal that ASURA is the best algorithm for large-scale storage cluster systems.