Model-Based Decision Making in the Human Brain

Author: Hampton, Alan Nicolás

Year: 2007

Degree: Dissertation (Ph.D.)

Advisor: O'Doherty, John P.

Committee Members: Bossaerts, Peter L.; O'Doherty, John P.; Adolphs, Ralph; Shimojo, Shinsuke; Quartz, Steven R.

Option: Computation and Neural Systems

DOI: 10.7907/K9Z8-E248

Abstract

Many real-life decision making problems incorporate higher-order structure, involving interdependencies between different stimuli, actions, and subsequent rewards. It is not known whether brain regions implicated in decision making, such as ventromedial prefrontal cortex, employ a stored model of the task structure to guide choice (model-based decision making) or merely learn action or state values without assuming higher-order structure, as in standard reinforcement learning. To discriminate between these possibilities we scanned human subjects with fMRI while they performed two different decision making tasks with higher-order structure: probabilistic reversal learning, in which subjects had to infer which of two choices was the more rewarding and then flexibly switch their choice when contingencies changed; and the inspection game, in which subjects had to successfully compete against an intelligent adversary by mentalizing the opponent’s state of mind in order to anticipate the opponent’s behavior in future. For both tasks we found that neural activity in a key decision making region: ventromedial prefrontal cortex, was more consistent with computational models that exploit higher-order structure, than with simple reinforcement learning. Moreover, in the social interaction game, subjects were found to employ a sophisticated strategy whereby they used knowledge of how their actions would influence the actions of their opponent to guide their choices. Specific computational signals required for the implementation of such a strategy were present in medial prefrontal cortex and superior temporal sulcus, providing insight into the basic computations underlying competitive strategic interactions. These results suggest that brain regions such as ventromedial prefrontal cortex employ an abstract model of task structure to guide behavioral choice, computations that may underlie the human capacity for complex social interactions and abstract strategizing.

Files