Asset Details
Do you wish to reserve the book?
Correlation detection as a stimulus computable account for audiovisual perception, causal inference, and saliency maps in mammals
Hey, we have placed the reservation for you!
By the way, why not check out events that you can attend while you pick your title.
Oops! Something went wrong.
Looks like we were not able to place the reservation. Kindly try again later.
Are you sure you want to remove the book from the shelf?
Correlation detection as a stimulus computable account for audiovisual perception, causal inference, and saliency maps in mammals
Do you wish to request the book?
Correlation detection as a stimulus computable account for audiovisual perception, causal inference, and saliency maps in mammals
Please be aware that the book you have requested cannot be checked out. If you would like to checkout this book, you can reserve another copy
We have requested the book for you!
Your request is successful and it will be processed during the Library working hours. Please check the status of your request in My Requests.
Oops! Something went wrong.
Looks like we were not able to place your request. Kindly try again later.
Journal Article
Correlation detection as a stimulus computable account for audiovisual perception, causal inference, and saliency maps in mammals
2025
Overview
Animals excel at seamlessly integrating information from different senses, a capability critical for navigating complex environments. Despite recent progress in multisensory research, the absence of stimulus-computable perceptual models fundamentally limits our understanding of how the brain extracts and combines task-relevant cues from the continuous flow of natural multisensory stimuli. Here, we introduce an image- and sound-computable population model for audiovisual perception, based on biologically plausible units that detect spatiotemporal correlations across auditory and visual streams. In a large-scale simulation spanning 69 psychophysical, eye-tracking, and pharmacological experiments, our model replicates human, monkey, and rat behaviour in response to diverse audiovisual stimuli with an average correlation exceeding 0.97. Despite relying on as few as 0–4 free parameters, our model provides an end-to-end account of audiovisual integration in mammals—from individual pixels and audio samples to behavioural responses. Remarkably, the population response to natural audiovisual scenes generates saliency maps that predict spontaneous gaze direction, Bayesian causal inference, and a variety of previously reported multisensory illusions. This study demonstrates that the integration of audiovisual stimuli, regardless of their spatiotemporal complexity, can be accounted for in terms of elementary joint analyses of luminance and sound level. Beyond advancing our understanding of the computational principles underlying multisensory integration in mammals, this model provides a bio-inspired, general-purpose solution for multimodal machine perception.
Publisher
eLife Sciences Publications Ltd,eLife Sciences Publications, Ltd
Subject
/ Animals
/ Auditory Perception - physiology
/ Brain
/ Computational and Systems Biology
/ Humans
/ Rats
/ Senses
/ Sensors
/ Sound
/ Speech
