Out of sync: how the brain deceives you by joining sounds and images

You must have learned in high school physics classes that sound and light travel at different speeds. Imagine how complicated it would be to perceive the world with this “delay“. Luckily for us, the human brain has compensatory mechanisms that help us to synchronize different stimuli, such as temporal recalibration. The phenomenon was the subject of a new study carried out at McGill University in Montreal, Canada – which revealed how the mechanism it works.

The human brain is constantly bombarded by visual and sound stimuli.Source: Pixabay

The research team found that temporal recalibration depends on brain signals that constantly adapt to capture samples from our environment, ordering and associating sensory stimuli that actually compete with each other.

How the study was done

The researchers recruited volunteers to observe flashes shorts of light tied to sounds that exhibited a variety of delays. Afterwards, they asked these people to report if they thought that both occurred simultaneously. The participants performed the task inside a magnetoencephalography (MEG) machine – a technique for mapping brain activity by detecting the magnetic field produced by the organ’s electric currents.

Magnetoencephalography machine in use.Magnetoencephalography machine in use.Source: MEG / HCHN / Disclosure

MEG recorded and was able to visualize the volunteers’ brain waves with millisecond precision. The pairs of audiovisual stimuli changed each time, with sounds and images presented closer or more distant in time – and with random presentation orders.

The researchers found that the volunteers’ perception of simultaneity between the pair of audiovisual stimuli was strongly affected by the perceived simultaneity in the pair of stimuli that came before it. For example: when a sound followed by an image was presented with a millisecond interval – and it was perceived as out of sync – it is very likely that the receiver reports the next pair of audiovisual stimuli as being in sync, even if it is not.

Why do we use temporal recalibration?

Without active temporal recalibration, we would have a distorted or disconnected perception of reality. The mechanism prevents this from happening and helps to establish causal relationships between the images and sounds that we perceive, despite the different physical speeds at which they happen and their neural processing.

The MEG signals revealed that the brain feat is enabled by a unique interaction between fast and slow brain waves in auditory and visual brain regions. Slower brain rhythms mark the temporal fluctuations of excitability in the brain circuits. The greater the excitability, the more easily an external input is registered and processed by the receiving neural networks.

Intentional Delay

Based on the research, scientists now propose a new model for understanding recalibration – in which faster oscillations over slower fluctuations create slots time discrete and ordered to record the order of the sensations received.

That is: when an audio signal reaches the first slot of time available in the auditory cortex and the same happens with an image, the pair is perceived as simultaneous. For this to happen, the brain needs to position the slots time visuals a little later than the auditory ones – to account for the slower physiological reception of visual signals.

Auditory and visual times are different.Auditory and visual times are different.Source: Pixabay

The researchers found that this relative neural delay between the auditory and visual time intervals is a dynamic process that constantly adapts to each participant’s recent exposure to audiovisual perceptions. Their data confirmed the new model of dynamic integration, showing how these subtle delays of tens of milliseconds of rapid brain oscillations can be measured in each individual.

Disorders in temporal recalibration

In autism and speech disorders, the processing of the senses, especially that of hearing, is altered. In schizophrenia, patients can also be affected by distortions in perceived sensory information. The neurophysiological mechanisms of temporal recalibration described in McGill’s study may be altered in patients with these disorders – and the discovery may reveal new ways to improve these deficits and the quality of life of those who have them.

Leave a Comment