Dispel: Byzantine SMR with Distributed Pipelining

Byzantine State Machine Replication (SMR) is a long studied topic that received increasing attention recently with the advent of blockchains as companies are trying to scale them to hundreds of nodes. Byzantine SMRs try to increase throughput by either reducing the latency of consensus instances that they run sequentially or by reducing the number of replicas that send messages to others in order to reduce the network usage. Unfortunately, the former approach makes use of resources in burst whereas the latter requires CPU-intensive authentication mechanisms. In this paper, we propose a new Byzantine SMR called Dispel that allows any node to distributively start new consensus instances whenever they detect sufficient resources locally. We evaluate the performance of Dispel within a single datacenter and across up to 380 machines over 3 continents by comparing it against other SMRs. On 128 nodes, Dispel speeds up its closest competitor by 8 times. Compared to leader-based SMRs, the inherent robustness of the Dispel distributed design prevents correlated failures from disrupting it. Finally, we evaluate Dispel in a cryptocurrency application with Bitcoin transactions and show that this SMR is not the bottleneck.