How you feel the world impacts how you see it

In the classic waterfall illusion, if you stare at the downward motion of a waterfall for some period of time, stationary objects - like rocks - appear to drift upward. MIT neuroscientists have found that this phenomenon, called motion aftereffect, occurs not only in our visual perception but also in our tactile perception, and that these senses actually influence one another. Put another way, how you feel the world can actually change how you see it - and vice versa.

In a paper published in the 9 April online issue of Current Biology, researchers found that people who were exposed to visual motion in a given direction perceived tactile motion in the opposite direction. Conversely, tactile motion in one direction gave rise to the illusion of visual motion in the opposite direction.

'Our discovery suggests that the sensory processing of visual and tactile motion use overlapping neural circuits,' explained Christopher Moore of the McGovern Institute for Brain Research at MIT and senior author of the paper. 'The way something looks or feels can be influenced by a stimulus in the other sensory modality.'

Volunteers watched visual motion on a computer screen while placing their right index fingertip on a tactile stimulator directly behind the screen. The stimulator consisted of a centimetre-square array composed of 60 pins to deliver precisely controlled vibrations to the fingertips. This stimulator, the only one of its kind in the world, was developed by Qi Wang of Georgia Institute of Technology and Vincent Hayward of Universite Pierre et Marie Curie in France.

To test the effect of visual motion on the subjects' perception of touch, the monitor displayed a pattern of horizontal stripes moving upward or downward for ten seconds. After this visual pattern had disappeared, a single row of horizontal pins simultaneously vibrated the subjects' fingertips. Although the pins delivered a static burst of vibration, all eight subjects perceived that the row of pins was sweeping either upward or downward, in the direction opposite to the movement of the preceding visual pattern.

To test the effect of tactile motion on visual perception, adjacent rows of pins vibrated in rapid succession, creating the sensation of a tactile object sweeping up or down the subjects' fingertips. After 10 seconds of this stimulus, the monitor displayed a static pattern of horizontal stripes. Contrary to the prevailing assumption that vision always trumps touch, subjects perceived the stripes as moving in the opposite direction to the moving tactile stimulus.

'Aftereffects were once thought to reflect fatigue in the brain circuits,' said Konkle, 'but we now know that pools of neurones are continuously coding motion information and recalibrating the brain to its sensory environment. Our neurones are not tired, they are constantly adapting to the world around us.'

Recent studies have found that a region of the visual cortex known as MT or V5 - long thought to play a major role in the perception of motion - may also process tactile motion. Moore's team intends to explore this brain region in future studies to determine whether it contributes to these cross-modal motion aftereffects.

'Neuroscientists study perceptual illusions because they help reveal how the brain gives rise to conscious experience,' Moore said. 'We don't experience the world through isolated senses, and our data support the emerging view that the brain is organised for cross talk among different sensory modalities.'

The research was supported by: McGovern Institute for Brain Research at MIT, Mitsui Foundation, National Defense Science and Engineering Graduate Fellowship, Eric L. Adler Fellowship, Natural Sciences and Engineering Research Council of Canada, and McGill University.