Mean-field theory of echo state networks

Dynamical systems driven by strong external signals are ubiquitous in nature and engineering. Here we study echo state networks, networks of a large number of randomly connected nodes, which represent a simple model of a neural network, and have important applications in machine learning. We develop a mean-field theory of echo state networks. The dynamics of the network is captured by the evolution law, similar to a logistic map, for a single collective variable. When the network is driven by many independent external signals, this collective variable reaches a steady state. But when the network is driven by a single external signal, the collective variable is non stationary but can be characterized by its time averaged distribution. The predictions of the mean-field theory, including the value of the largest Lyapunov exponent, are compared with the numerical integration of the equations of motion. DOI: 10.1103/PhysRevE.87.042809