Journal Article

Representational interactions during audiovisual speech entrainment: Redundancy in left posterior superior temporal gyrus and synergy in left motor cortex

Park, Hyojin, Thut, Gregor, Ince, Robin A. A., Gross, Joachim, Schyns, Philippe G.

2018

Overview

Integration of multimodal sensory information is fundamental to many aspects of human behavior, but the neural mechanisms underlying these processes remain mysterious. For example, during face-to-face communication, we know that the brain integrates dynamic auditory and visual inputs, but we do not yet understand where and how such integration mechanisms support speech comprehension. Here, we quantify representational interactions between dynamic audio and visual speech signals and show that different brain regions exhibit different types of representational interaction. With a novel information theoretic measure, we found that theta (3-7 Hz) oscillations in the posterior superior temporal gyrus/sulcus (pSTG/S) represent auditory and visual inputs redundantly (i.e., represent common features of the two), whereas the same oscillations in left motor and inferior temporal cortex represent the inputs synergistically (i.e., the instantaneous relationship between audio and visual inputs is also represented). Importantly, redundant coding in the left pSTG/S and synergistic coding in the left motor cortex predict behavior-i.e., speech comprehension performance. Our findings therefore demonstrate that processes classically described as integration can have different statistical properties and may reflect distinct mechanisms that occur in different brain regions to support audiovisual speech comprehension.

• Publisher: Public Library of Science,Public Library of Science (PLoS)
• Subject: Acoustic Stimulation / Adolescent / Adult / Auditory Perception / Biology and Life Sciences / Brain / Brain - physiology / Brain Mapping - methods / Coding / Comprehension - physiology / Cortex (auditory) / Cortex (motor) / Cortex (somatosensory) / Cortex (temporal) / Engineering and Technology / Entrainment / Female / Funding / Human behavior / Humans / Image Processing, Computer-Assisted / Information theory / Magnetic Resonance Imaging / Male / Medical imaging / Medicine and Health Sciences / Motor cortex / Motor Cortex - physiology / Neural coding / Neurosciences / Oscillations / Photic Stimulation / Physical Sciences / Physiological aspects / Redundancy / Research and Analysis Methods / Sensory integration / Social Sciences / Somatosensory cortex / Speech / Speech Perception - physiology / Studies / Superior temporal gyrus / Temporal gyrus / Temporal lobe / Temporal Lobe - physiology / Temporal lobes / Theta rhythms / Verbal communication / Visual cortex / Visual Perception / Visual signals