ReLU Neural Networks of Polynomial Size for Exact Maximum Flow Computation

Understanding the great empirical success of artificial neural networks (NNs) from a theoretical point of view is currently one of the hottest research topics in computer science. In order to study the expressive power of NNs with rectified linear units, we propose to view them as a model of computation and investigate the complexity of combinatorial optimization problems in that model. Using a result from arithmetic circuit complexity, we show as a first immediate result that the value of a minimum spanning tree in a graph with $n$ nodes can be computed by an NN of size $\mathcal{O}(n^3)$. Our primary result, however, is that, given a directed graph with $n$ nodes and $m$ arcs, there exists an NN of size $\mathcal{O}(m^2n^2)$ that computes a maximum flow from any possible real-valued arc capacities as input. This settles the former open questions whether such NNs with polynomial size exist. To prove our results, we develop the pseudo-code language Max-Affine Arithmetic Programs (MAAPs) and show equivalence between MAAPs and NNs concerning natural complexity measures. We then design MAAPs that exactly solve the corresponding optimization problems and translate to NNs of the claimed size.