14
62

The universal approximation theorem for complex-valued neural networks

Abstract

We generalize the classical universal approximation theorem for neural networks to the case of complex-valued neural networks. Precisely, we consider feedforward networks with a complex activation function σ:CC\sigma : \mathbb{C} \to \mathbb{C} in which each neuron performs the operation CNC,zσ(b+wTz)\mathbb{C}^N \to \mathbb{C}, z \mapsto \sigma(b + w^T z) with weights wCNw \in \mathbb{C}^N and a bias bCb \in \mathbb{C}, and with σ\sigma applied componentwise. We completely characterize those activation functions σ\sigma for which the associated complex networks have the universal approximation property, meaning that they can uniformly approximate any continuous function on any compact subset of Cd\mathbb{C}^d arbitrarily well. Unlike the classical case of real networks, the set of "good activation functions" which give rise to networks with the universal approximation property differs significantly depending on whether one considers deep networks or shallow networks: For deep networks with at least two hidden layers, the universal approximation property holds as long as σ\sigma is neither a polynomial, a holomorphic function, or an antiholomorphic function. Shallow networks, on the other hand, are universal if and only if the real part or the imaginary part of σ\sigma is not a polyharmonic function.

View on arXiv
Comments on this paper