The Sampling-and-Learning Framework: A Statistical View of Evolutionary Algorithms

Evolutionary algorithms (EAs), a large class of general purpose optimization algorithms inspired from natural phenomena, are widely used in various industrial optimizations and often show excellent performance. This paper presents an attempt towards revealing their general power from a statistical view of EAs. We summarize a large range of EAs into the sampling-and-learning framework. The framework directly admits a general analysis on the probable-absolute-approximate (PAA) query complexity. We study the framework with the learning subroutine being restricted as binary classifications, which results in the sampling-and-classification (SAC) algorithms. With the learning theory results, we give a general upper bound on the PAA query complexity of SAC algorithms. We further compare SAC algorithms with the uniform search in different situations. Under the error-target independence condition, we show that firstly, when a problem costs the uniform search a super-polynomial PAA query complexity, it can also cost a SAC algorithm a super-polynomial complexity; but secondly, SAC algorithms can achieve a polynomial reduction of the complexity of the uniform search. Under the error-successive-level independence condition, we show that a super-polynomial reduction of the complexity can be achieved. This work only touches the surface of the sample-and-learning framework, of which the power under other conditions is still open.