Projective simulation with generalization

The ability to generalize is an important feature of any intelligent agent. Not only because it may allow the agent to cope with large amounts of data, but also because in some environments, an agent with no generalization ability is simply doomed to fail. In this work we outline several criteria for generalization, and present a dynamic and autonomous machinery that enables projective simulation agents to meaningfully generalize. Projective simulation, a novel, physical, approach to artificial intelligence, was recently shown to perform well, in comparison with standard models, on both simple reinforcement learning problems, as well as on more complicated canonical tasks, such as the "grid world" and the "mountain car problem". Both the basic projective simulation model and the presented generalization machinery are based on very simple principles. This simplicity allows us to provide a full analytical analysis of the agent's performance and to illustrate the benefit the agent gains by generalizing. Specifically, we show how such an ability allows the agent to learn in rather extreme environments, in which learning is otherwise impossible.