Explore the vast range of titles available.

The authors developed experimental and computational approaches to study moment-to-moment changes in the activity of populations of cortical neurons as mice accumulated evidence during decision-making in virtual reality. They propose that evidence accumulation may not require winner-take-all competitions but instead emerges from general dynamical properties that instantiate short-term memory. We studied how the posterior parietal cortex combines new information with ongoing activity dynamics as mice accumulate evidence during a virtual navigation task. Using new methods to analyze population activity on single trials, we found that activity transitioned rapidly between different sets of active neurons. Each event in a trial, whether an evidence cue or a behavioral choice, caused seconds-long modifications to the probabilities that govern how one activity pattern transitions to the next, forming a short-term memory. A sequence of evidence cues triggered a chain of these modifications resulting in a signal for accumulated evidence. Multiple distinguishable activity patterns were possible for the same accumulated evidence because representations of ongoing events were influenced by previous within- and across-trial events. Therefore, evidence accumulation need not require the explicit competition between groups of neurons, as in winner-take-all models, but could instead emerge implicitly from general dynamical properties that instantiate short-term memory.

To train a computer to “recognize” elements of a scene supplied by its visual sensors, computer scientists typically use millions of images painstakingly labeled by humans. Eslami et al. developed an artificial vision system, dubbed the Generative Query Network (GQN), that has no need for such labeled data. Instead, the GQN first uses images taken from different viewpoints and creates an abstract description of the scene, learning its essentials. Next, on the basis of this representation, the network predicts what the scene would look like from a new, arbitrary viewpoint. Science , this issue p. 1204 A computer vision system predicts how a 3D scene looks from any viewpoint after just a few 2D views from other viewpoints. Scene representation—the process of converting visual sensory data into concise descriptions—is a requirement for intelligent behavior. Recent work has shown that neural networks excel at this task when provided with large, labeled datasets. However, removing the reliance on human labeling remains an important open problem. To this end, we introduce the Generative Query Network (GQN), a framework within which machines learn to represent scenes using only their own sensors. The GQN takes as input images of a scene taken from different viewpoints, constructs an internal representation, and uses this representation to predict the appearance of that scene from previously unobserved viewpoints. The GQN demonstrates representation learning without human labels or domain knowledge, paving the way toward machines that autonomously learn to understand the world around them.

Decision-making requires flexibility to rapidly switch one’s actions in response to sensory stimuli depending on information stored in memory. We identified cortical areas and neural activity patterns underlying this flexibility during virtual navigation, where mice switched navigation toward or away from a visual cue depending on its match to a remembered cue. Optogenetics screening identified V1, posterior parietal cortex (PPC), and retrosplenial cortex (RSC) as necessary for accurate decisions. Calcium imaging revealed neurons that can mediate rapid navigation switches by encoding a mixture of a current and remembered visual cue. These mixed selectivity neurons emerged through task learning and predicted the mouse’s choices by forming efficient population codes before correct, but not incorrect, choices. They were distributed across posterior cortex, even V1, and were densest in RSC and sparsest in PPC. We propose flexibility in navigation decisions arises from neurons that mix visual and memory information within a visual-parietal-retrosplenial network. Animals flexibly and rapidly adapt navigation routes to the environment and context. Here, the authors find that the flexibility in navigation decisions arises from cells distributed in posterior cortex, each of which mixes sensory and memory information.

Artificially intelligent agents are getting better and better at two-player games, but most real-world endeavors require teamwork. Jaderberg et al. designed a computer program that excels at playing the video game Quake III Arena in Capture the Flag mode, where two multiplayer teams compete in capturing the flags of the opposing team. The agents were trained by playing thousands of games, gradually learning successful strategies not unlike those favored by their human counterparts. Computer agents competed successfully against humans even when their reaction times were slowed to match those of humans. Science , this issue p. 859 Teams of artificial agents compete successfully against humans in the video game Quake III Arena in Capture the Flag mode. Reinforcement learning (RL) has shown great success in increasingly complex single-agent environments and two-player turn-based games. However, the real world contains multiple agents, each learning and acting independently to cooperate and compete with other agents. We used a tournament-style evaluation to demonstrate that an agent can achieve human-level performance in a three-dimensional multiplayer first-person video game, Quake III Arena in Capture the Flag mode, using only pixels and game points scored as input. We used a two-tier optimization process in which a population of independent RL agents are trained concurrently from thousands of parallel matches on randomly generated environments. Each agent learns its own internal reward signal and rich representation of the world. These results indicate the great potential of multiagent reinforcement learning for artificial intelligence research.

Noise correlations (that is, trial-to-trial covariations in neural activity for a given stimulus) limit the stimulus information encoded by neural populations, leading to the widely held prediction that they impair perceptual discrimination behaviors. However, this prediction neglects the effects of correlations on information readout. We studied how correlations affect both encoding and readout of sensory information. We analyzed calcium imaging data from mouse posterior parietal cortex during two perceptual discrimination tasks. Correlations reduced the encoded stimulus information, but, seemingly paradoxically, were higher when mice made correct rather than incorrect choices. Single-trial behavioral choices depended not only on the stimulus information encoded by the whole population, but unexpectedly also on the consistency of information across neurons and time. Because correlations increased information consistency, they enhanced the conversion of sensory information into behavioral choices, overcoming their detrimental information-limiting effects. Thus, correlations in association cortex can benefit task performance even if they decrease sensory information. Correlations in neural activity in association cortex can benefit behavioral performance in perceptual tasks, even when decreasing sensory information, by facilitating the propagation and the readout of information carried by population activity.

Reinforcement learning (RL) has shown great success in increasingly complex single-agent environments and two-player turn-based games. However, the real world contains multiple agents, each learning and acting independently to cooperate and compete with other agents. We used a tournament-style evaluation to demonstrate that an agent can achieve human-level performance in a three-dimensional multiplayer first-person video game, Quake III Arena in Capture the Flag mode, using only pixels and game points scored as input.We used a two-tier optimization process in which a population of independent RL agents are trained concurrently from thousands of parallel matches on randomly generated environments. Each agent learns its own internal reward signal and rich representation of the world. These results indicate the great potential of multiagent reinforcement learning for artificial intelligence research.

Decision-making requires flexibility to rapidly switch sensorimotor associations depending on behavioral goals stored in memory. We identified cortical areas and neural activity patterns that mediate this flexibility during virtual-navigation, where mice switched navigation toward or away from a visual cue depending on its match to a remembered cue. An optogenetics screen identified V1, posterior parietal cortex (PPC), and retrosplenial cortex (RSC) as necessary for accurate decisions. Calcium imaging revealed neurons that can mediate rapid sensorimotor switching by encoding a conjunction of a current and remembered visual cue that predicted the mouse's navigational choice from trial-to-trial. Their activity formed efficient population codes before correct, but not incorrect, choices. These neurons were distributed across posterior cortex, even V1, but were densest in RSC and sparsest in PPC. We propose the flexibility of navigation decisions arises from neurons that mix visual and memory information within a visual-parietal-retrosplenial network, centered in RSC. Competing Interest Statement The authors have declared no competing interest.

Cortical circuits combine new inputs with ongoing activity during a variety of behaviors, including evidence accumulation during decision-making. However, the neural circuit mechanisms underlying how populations of neurons perform the computations necessary for this process and the dynamics which govern how neuronal populations change from moment-to-moment during evidence accumulation remain unclear. Here, we trained mice to perform several novel virtual-navigation decision tasks, including one which requires the accumulation of multiple, discrete evidence cues. As mice accumulated evidence, the posterior parietal cortex (PPC) transitioned between distinguishable and largely uncorrelated activity patterns, often involving mostly different sets of active neurons from moment-to-moment. These activity patterns contained task-relevant information distributed across the neuronal population. Because animals make decisions on single trials, we chose to analyze these activity patterns on a trial-by-trial basis. As single trials unfolded, each event—whether a new evidence cue or a behavioral choice—modified the dynamics of the PPC for seconds, even across trials. These events did not change the tonic activity of a specific set of neurons; rather, each event altered the probabilities that govern how one activity pattern transitions to the next, constraining the possible future patterns of activity. Thus, representations of ongoing events were influenced both by the sequence of previous evidence cues within the current trial and by the outcome of the previous trial, thereby generating multiple distinguishable activity patterns for the same level of accumulated evidence. These observations suggest that evidence accumulation does not rely upon the explicit competition between groups of neurons (as would be predicted by winner-take-all models), but instead reflects dynamical properties of the PPC that may instantiate a form of short-term memory consistent with reservoir computing.

The spatiotemporal structure of activity in populations of neurons is critical for accurate perception and behavior. Experimental and theoretical studies have focused on noise correlations (trial-to-trial covariations in neural activity for a given stimulus) as a key feature of population activity structure. Much work has shown that these correlations limit the stimulus information encoded by a population of neurons, leading to the widely held prediction that correlations are detrimental for perceptual discrimination behaviors. However, this prediction relies on an untested assumption: that the neural mechanisms that read out sensory information to inform behavior depend only on the population total stimulus information independently of how correlations constrain this information across neurons or time. Here we make the critical advance of simultaneously studying how correlations affect both the encoding and the readout of sensory information. We analyzed calcium imaging data from mouse posterior parietal cortex during two perceptual discrimination tasks. Correlations limited the ability to encode stimulus information, but (seemingly paradoxically) correlations were higher when mice made correct choices than when they made errors. On a single-trial basis, the behavioral choice of the mouse depended not only on the stimulus information in the activity of the population as a whole, but unexpectedly also on the consistency of information across neurons and time. Because correlations increased information consistency, sensory information was more efficiently converted into a behavioral choice in the presence of correlations. Given this enhanced-by-consistency readout, we estimated that correlations produced a behavioral benefit that compensated or overcame their detrimental information-limiting effects. These results call for a reevaluation of the role of correlated neural activity, and suggest that correlations in association cortex can benefit task performance even if they decrease sensory information.

Representational sparsity is known to affect robustness to input perturbations in deep neural networks (DNNs), but less is known about how the semantic content of representations affects robustness. Class selectivity-the variability of a unit's responses across data classes or dimensions-is one way of quantifying the sparsity of semantic representations. Given recent evidence that class selectivity may not be necessary for, and in some cases can impair generalization, we investigate whether it also confers robustness (or vulnerability) to perturbations of input data. We found that networks regularized to have lower levels of class selectivity were more robust to average-case (naturalistic) perturbations, while networks with higher class selectivity are more vulnerable. In contrast, class selectivity increases robustness to multiple types of worst-case (i.e. white box adversarial) perturbations, suggesting that while decreasing class selectivity is helpful for average-case perturbations, it is harmful for worst-case perturbations. To explain this difference, we studied the dimensionality of the networks' representations: we found that the dimensionality of early-layer representations is inversely proportional to a network's class selectivity, and that adversarial samples cause a larger increase in early-layer dimensionality than corrupted samples. Furthermore, the input-unit gradient is more variable across samples and units in high-selectivity networks compared to low-selectivity networks. These results lead to the conclusion that units participate more consistently in low-selectivity regimes compared to high-selectivity regimes, effectively creating a larger attack surface and hence vulnerability to worst-case perturbations.