Journal Article

A framework for studying the neurobiology of value-based decision making

Camerer, Colin, Montague, P. Read, Rangel, Antonio

2008

Overview

Key Points Most behavioural and computational models of decision making assume that the following five processes are carried out at the time the decision is made: representation, action valuation, action selection, outcome valuation, and learning. On the basis of a sizeable body of animal and human behavioural evidence, several groups have proposed the existence of three different types of valuation systems: Pavlovian, habitual and goal-directed systems. Pavlovian systems assign value to only a small set of 'prepared' behaviours and thus have a limited behavioural repertoire. Nevertheless, they might be the driving force behind behaviours with important economic consequences (for example, overeating). Examples include preparatory behaviours, such as approaching a cue that predicts food, and consummatory behaviours, such as ingesting available food. Habit valuation systems learn to assign values to stimulus–response associations on the basis of previous experience through a process of trial-and-error. Examples of habits include a smoker's desire to have a cigarette at particular times of day (for example, after a meal) and a rat's tendency to forage in a cue-dependent location after sufficient training. Goal-directed systems assign values to actions by computing action–outcome associations and then evaluating the rewards that are associated with the different outcomes. An example of a goal-directed behaviour is the decision what to eat at a new restaurant. An important difference between habitual and goal-directed systems has to do with how they respond to changes in the environment. The goal-directed system updates the value of an action as soon as the value of its outcome changes, whereas the habit system only learns with repeated experience. The values computed by the three systems can be modulated by factors such as the risk that is associated with the decision, the time delay to the outcomes, and social considerations. The quality of the decisions made by an animal depend on how its brain assigns control to the different valuation systems in situations in which it has to make a choice between several potential actions that are assigned conflicting values. The learning properties of the habit system seem to be well-described by simple reinforcement algorithms, such as Q-learning. Some of the key computations that are predicted by these models are instantiated in the dopamine system. The past decade has seen an increasing interest in the mechanisms by which the human brain decides what actions to take. Here, Rangel and colleagues provide a framework for the exploration of the neurobiological and computational basis of value-based decision making. Neuroeconomics is the study of the neurobiological and computational basis of value-based decision making. Its goal is to provide a biologically based account of human behaviour that can be applied in both the natural and the social sciences. This Review proposes a framework to investigate different aspects of the neurobiology of decision making. The framework allows us to bring together recent findings in the field, highlight some of the most important outstanding problems, define a common lexicon that bridges the different disciplines that inform neuroeconomics, and point the way to future applications.

• Publisher: Nature Publishing Group UK,Nature Publishing Group
• Subject: Animal Genetics and Genomics / Animals / Behavioral Sciences / Biochemistry. Physiology. Immunology / Biological and medical sciences / Biological Techniques / Biomedical and Life Sciences / Biomedicine / Computational biology / Crustacea / Decision making / Decision Making - physiology / Decision Theory / Electrodiagnosis. Electric activity recording / Fundamental and applied biological sciences. Psychology / Habits / Humans / Hypotheses / Invertebrates / Investigative techniques, diagnostic techniques (general aspects) / Learning - physiology / Medical sciences / Models, Neurological / Models, Psychological / Nervous system / Neurobiology / Neuroeconomics / Neurosciences / Physiology. Development / Psychological aspects / review-article / Risk / Social sciences / Social Values / Valuation / Variables