Positive Neural Networks in Discrete Time Implement Monotone-Regular Behaviors

Many works have investigated the expressive power of various kinds of neural networks. We continue this study with inspiration from biologically plausible models. In particular, we study positive neural networks with multiple input neurons, and where neurons only excite each other and do not inhibit each other. Different behaviors can be expressed by varying the connection strengths between the neurons. We show that in discrete time, and in absence of noise, the class of positive neural networks captures the so-called monotone-regular behaviors, that are based on regular languages. A finer picture emerges if one takes into account the delay by which a monotone-regular behavior is implemented. Each monotone-regular behavior can be implemented by a positive neural network with a delay of one time unit. Some monotone-regular behaviors can be implemented with zero delay. And, interestingly, some simple monotone-regular behaviors can not be implemented with zero delay.