Which scaling rule applies to Artificial Neural Networks

Although an Artificial Neural Network (ANN) is a biology-mimicking system, it is built from components designed/fabricated for use in conventional computing, created by experts trained in conventional computing. Making computing systems, actively cooperating and communicating, using segregated single processors, however, has severe performance limitations. The achievable payload computing performance sensitively depends on workload type, and this effect is only poorly known. The workload type, the Artificial Intelligence systems generate, has an exceptionally low payload computational performance for ANN applications. Unfortunately, initial successes of demo systems that comprise only a few "neurons" and solve simple tasks are misleading: scaling of computing-based ANN systems is strongly nonlinear. The paper discusses some major limiting factors that affect performance. It points out that for building biology-mimicking large systems, it is inevitable to perform drastic changes in the present computing paradigm (mainly to comprehend temporal behavior of components) and architectures.