CNS Seminar

A currently popular view of reinforcement learning is that the brain makes use of both model-based (goal-directed) and model-free (habitual) learning systems. I will discuss an alternative learning system, whose core is the successor representation (SR), which exhibits some of the operating characteristics of both model-based and model-free learning. The SR compactly encodes the manifold structure of the state transition function, and is closely related to spectral techniques for manifold learning. When applied to spatial environments, the SR learns a map that captures the underlying geometry of the state space. Such a map is consistent with the hippocampal representation of space: a variety of place field phenomena, such as changes in place fields induced by manipulations of environmental geometry and reward, arise naturally from the SR. Moreover, an eigendecomposition of the SR leads to a spatial representation resembling entorhinal grid cells, which may be useful for discovering hierarchical decompositions of space. In the second part of my talk, I describe recent human experiments that distinguish between model-based, model-free and SR models. These experiments provide evidence for an SR model that interacts with model-based learning.