Explore the vast range of titles available.

Midbrain dopamine signals are widely thought to report reward prediction errors that drive learning in the basal ganglia. However, dopamine has also been implicated in various probabilistic computations, such as encoding uncertainty and controlling exploration. Here, we show how these different facets of dopamine signalling can be brought together under a common reinforcement learning framework. The key idea is that multiple sources of uncertainty impinge on reinforcement learning computations: uncertainty about the state of the environment, the parameters of the value function and the optimal action policy. Each of these sources plays a distinct role in the prefrontal cortex–basal ganglia circuit for reinforcement learning and is ultimately reflected in dopamine activity. The view that dopamine plays a central role in the encoding and updating of beliefs brings the classical prediction error theory into alignment with more recent theories of Bayesian reinforcement learning.

Arguably, the most difficult part of learning is deciding what to learn about. Should I associate the positive outcome of safely completing a street-crossing with the situation ‘the car approaching the crosswalk was red’ or with ‘the approaching car was slowing down’? In this Perspective, we summarize our recent research into the computational and neural underpinnings of ‘representation learning’—how humans (and other animals) construct task representations that allow efficient learning and decision-making. We first discuss the problem of learning what to ignore when confronted with too much information, so that experience can properly generalize across situations. We then turn to the problem of augmenting perceptual information with inferred latent causes that embody unobservable task-relevant information, such as contextual knowledge. Finally, we discuss recent findings regarding the neural substrates of task representations that suggest the orbitofrontal cortex represents ‘task states’, deploying them for decision-making and learning elsewhere in the brain.

Memory retrieval involves the interaction between external sensory or internally generated cues and stored memory traces (or engrams) in a process termed ‘ecphory’. While ecphory has been examined in human cognitive neuroscience research, its neurobiological foundation is less understood. To the extent that ecphory involves ‘reawakening’ of engrams, leveraging recently developed technologies that can identify and manipulate engrams in rodents provides a fertile avenue for examining retrieval at the level of neuronal ensembles. Here we evaluate emerging neuroscientific research of this type, using cognitive theory as a guiding principle to organize and interpret initial findings. Our Review highlights the critical interaction between engrams and retrieval cues (environmental or artificial) for memory accessibility and retrieval success. These findings also highlight the intimate relationship between the mechanisms important in forming engrams and those important in their recovery, as captured in the cognitive notion of ‘encoding specificity’. Finally, we identify several questions that currently remain unanswered.

The past several years have brought revelations and paradigm shifts in research on the cerebellum. Historically viewed as a simple sensorimotor controller with homogeneous architecture, the cerebellum is increasingly implicated in cognitive functions. It possesses an impressive diversity of molecular, cellular and circuit mechanisms, embedded in a dynamic, recurrent circuit architecture. Recent insights about the diversity and dynamism of the cerebellum provide a roadmap for the next decade of cerebellar research, challenging some old concepts, reinvigorating others and defining major new research directions. Recent research has discovered new connections between cerebellar neurons, revealed abundant inputs related to reward, demonstrated a cellular solution for the temporal credit assignment problem and restructured theories of cerebellar learning.

The brain’s ability to associate different stimuli is vital for long-term memory, but how neural ensembles encode associative memories is unknown. Here we studied how cell ensembles in the basal and lateral amygdala encode associations between conditioned and unconditioned stimuli (CS and US, respectively). Using a miniature fluorescence microscope, we tracked the Ca 2+ dynamics of ensembles of amygdalar neurons during fear learning and extinction over 6 days in behaving mice. Fear conditioning induced both up- and down-regulation of individual cells’ CS-evoked responses. This bi-directional plasticity mainly occurred after conditioning, and reshaped the neural ensemble representation of the CS to become more similar to the US representation. During extinction training with repetitive CS presentations, the CS representation became more distinctive without reverting to its original form. Throughout the experiments, the strength of the ensemble-encoded CS–US association predicted the level of behavioural conditioning in each mouse. These findings support a supervised learning model in which activation of the US representation guides the transformation of the CS representation. Use of a head-mounted miniature microscope in awake, behaving mice reveals that neural ensembles in the basal and lateral amygdala encode associations between conditioned and unconditioned stimuli in a way that matches models of supervised learning. Of mice and memory decoding Most of the work exploring the substrates underlying associative memory formation has involved molecular and cellular aspects, but less is known about how neural ensembles encode these associations between stimuli. Here, Mark Schnitzer and colleagues utilize chronic microendoscopy imaging in behaving mice to observe cell ensembles within the amygdala that represent conditioned and unconditioned stimuli. As associations are formed and strengthened, the representation of the conditioned stimulus increasingly matches that of the unconditioned stimulus, with the opposite occurring during extinction of the association. These findings support a supervised learning model that could be tested in neural areas beyond the amygdala.

To make adaptive decisions, we build an internal model of the associative relationships in an environment and use it to make predictions and inferences about specific available outcomes. Detailed, identity-specific cue–reward memories are a core feature of such cognitive maps. Here we used fiber photometry, cell-type and pathway-specific optogenetic manipulation, Pavlovian cue–reward conditioning and decision-making tests in male and female rats, to reveal that ventral tegmental area dopamine (VTA DA ) projections to the basolateral amygdala (BLA) drive the encoding of identity-specific cue–reward memories. Dopamine is released in the BLA during cue–reward pairing; VTA DA →BLA activity is necessary and sufficient to link the identifying features of a reward to a predictive cue but does not assign general incentive properties to the cue or mediate reinforcement. These data reveal a dopaminergic pathway for the learning that supports adaptive decision-making and help explain how VTA DA neurons achieve their emerging multifaceted role in learning. Sias et al. show that dopamine projections to the basolateral amygdala drive the reward learning that supports the predictions and inferences needed for adaptive decision-making.

Neurons in the basolateral amygdala projecting to canonical fear or reward circuits undergo opposing changes in synaptic strength following fear or reward conditioning, and selectively activating these projection-target-defined neural populations causes either negative or positive reinforcement, respectively. Positive and negative associations hard-wired in the brain The amygdala is part of the brain important for emotional processing, handling stimuli that have either positive or negative associations — the good and the bad. Little is known about how amygdala neurons differentiate or compartmentalize these distinctions. Here, Kay Tye and colleagues identify the basolateral amygdala (BLA) as a site of divergence for circuits mediating positive and negative emotional or motivational responses. In studies in mice they find that neurons in the BLA projecting to fear or reward circuits undergo opposing changes in synaptic strength following fear or reward conditioning. Selective activation of neural populations causes, respectively, either negative or positive reinforcement. Transcriptome analysis reveals candidate genes that may mediate these functional differences. The ability to differentiate stimuli predicting positive or negative outcomes is critical for survival, and perturbations of emotional processing underlie many psychiatric disease states. Synaptic plasticity in the basolateral amygdala complex (BLA) mediates the acquisition of associative memories, both positive 1 , 2 and negative 3 , 4 , 5 , 6 , 7 . Different populations of BLA neurons may encode fearful or rewarding associations 8 , 9 , 10 , but the identifying features of these populations and the synaptic mechanisms of differentiating positive and negative emotional valence have remained unknown. Here we show that BLA neurons projecting to the nucleus accumbens (NAc projectors) or the centromedial amygdala (CeM projectors) undergo opposing synaptic changes following fear or reward conditioning. We find that photostimulation of NAc projectors supports positive reinforcement while photostimulation of CeM projectors mediates negative reinforcement. Photoinhibition of CeM projectors impairs fear conditioning and enhances reward conditioning. We characterize these functionally distinct neuronal populations by comparing their electrophysiological, morphological and genetic features. Overall, we provide a mechanistic explanation for the representation of positive and negative associations within the amygdala.

Recent success in training artificial agents and robots derives from a combination of direct learning of behavioural policies and indirect learning through value functions 1 – 3 . Policy learning and value learning use distinct algorithms that optimize behavioural performance and reward prediction, respectively. In animals, behavioural learning and the role of mesolimbic dopamine signalling have been extensively evaluated with respect to reward prediction 4 ; however, so far there has been little consideration of how direct policy learning might inform our understanding 5 . Here we used a comprehensive dataset of orofacial and body movements to understand how behavioural policies evolved as naive, head-restrained mice learned a trace conditioning paradigm. Individual differences in initial dopaminergic reward responses correlated with the emergence of learned behavioural policy, but not the emergence of putative value encoding for a predictive cue. Likewise, physiologically calibrated manipulations of mesolimbic dopamine produced several effects inconsistent with value learning but predicted by a neural-network-based model that used dopamine signals to set an adaptive rate, not an error signal, for behavioural policy learning. This work provides strong evidence that phasic dopamine activity can regulate direct learning of behavioural policies, expanding the explanatory power of reinforcement learning models for animal learning 6 . Analysis of data collected from mice learning a trace conditioning paradigm shows that phasic dopamine activity in the brain can regulate direct learning of behavioural policies, and dopamine sets an adaptive learning rate rather than an error-like teaching signal.

Granule cells constitute half of the cells in the brain, yet their activity during behavior is largely uncharacterized. The authors report that granule cells encode multisensory representations that evolve with learning into a predictive motor signal. This activity may help the cerebellum implement a forward model for action. Cerebellar granule cells, which constitute half the brain's neurons, supply Purkinje cells with contextual information necessary for motor learning, but how they encode this information is unknown. Here we show, using two-photon microscopy to track neural activity over multiple days of cerebellum-dependent eyeblink conditioning in mice, that granule cell populations acquire a dense representation of the anticipatory eyelid movement. Initially, granule cells responded to neutral visual and somatosensory stimuli as well as periorbital airpuffs used for training. As learning progressed, two-thirds of monitored granule cells acquired a conditional response whose timing matched or preceded the learned eyelid movements. Granule cell activity covaried trial by trial to form a redundant code. Many granule cells were also active during movements of nearby body structures. Thus, a predictive signal about the upcoming movement is widely available at the input stage of the cerebellar cortex, as required by forward models of cerebellar control.

Animals adapt their behavior in response to informative sensory cues using multiple brain circuits. The activity of midbrain dopaminergic neurons is thought to convey a critical teaching signal: reward-prediction error. Although reward-prediction error signals are thought to be essential to learning, little is known about the dynamic changes in the activity of midbrain dopaminergic neurons as animals learn about novel sensory cues and appetitive rewards. Here we describe a large dataset of cell-attached recordings of identified dopaminergic neurons as naive mice learned a novel cue–reward association. During learning midbrain dopaminergic neuron activity results from the summation of sensory cue-related and movement initiation-related response components. These components are both a function of reward expectation yet they are dissociable. Learning produces an increasingly precise coordination of action initiation following sensory cues that results in apparent reward-prediction error correlates. Our data thus provide new insights into the circuit mechanisms that underlie a critical computation in a highly conserved learning circuit.