Behavioral Social Neuroscience Seminar

In computational reinforcement learning (RL), reward---more specifically, a reward function---determines the problem an RL agent is trying to solve: its goal. Properties of the reward function influence how easy or hard the problem is, and how well an agent may do in trying to solve it, but RL theory and algorithms are insensitive to the source and nature of reward signals. Specifying a reward function is a critical part of designing an RL system for application to a real problem.  It is usual to make the RL agent's goal the same as the designer's goal. This assumes that the RL agent can achieve that goal, or if it can't, that what progress it can make is the best that can be done in achieving the designer's goal. 

In nature the designer is the evolutionary process, whose goal, let us say, is reproductive success. But organisms have elaborate reward systems, some of whose features bear no clear relationship to reproductive success.  In this talk I describe some computational experiments that elucidate aspects of the relationship between ultimate goals and the primary rewards that influence an agent's motivation and learning. Using the RL framework and adopting an evolutionary perspective, we search in spaces of primary reward functions for functions that lead RL agents to analogs of high evolutionary success.  Results shed light on how agent bounds influence the nature of evolutionarily-optimal reward functions. Of particular interest is the light this sheds on what psychologist D. E. Berlyne called "ludic behavior", meaning behavior that "does not have a biological function which we can clearly recognize."  This talk is based on work done with Satinder Singh, Richard Lewis, and Jonathan Sorg.