Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli

被引:158
作者
Lewald, J
Guski, R
机构
[1] Ruhr Univ Bochum, Fak Psychol, D-44780 Bochum, Germany
[2] Inst Arbeitsphysiol, D-44139 Dortmund, Germany
来源
COGNITIVE BRAIN RESEARCH | 2003年 / 16卷 / 03期
关键词
spatial perception; multisensory integration; ventriloquism effect; directional hearing; sound localization; human;
D O I
10.1016/S0926-6410(03)00074-0
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Under certain conditions, auditory and visual information are integrated into a single unified percept even when they originate in different locations in space. The present study shows how this illusion, known as the ventriloquism effect, depends on spatial, temporal and cognitive factors. A method of psychophysical scaling was employed in combination with simple auditory-visual stimuli (tone bursts and flashing light spots) that were presented with various spatiotemporal disparities. Participants either judged their impression of the likelihood of a common cause (Experiment 1) or spatial alignment (Experiment 2) or synchrony of sound and light (Experiment 3). In all three experiments the participants' judgements depended significantly on temporal disparity whereas influences of spatial disparity were significant in Experiments 1 and 2. Optimum scores were always obtained when auditory stimuli were presented with a delay of 50-100 ms after the visual stimuli. These results demonstrate that both temporal and spatial proximity of the two stimuli are critical for the experience of phenomenal causality. On the other hand, spatio-temporal ranges for optimal perception of phenomenal causality in Experiment I were significantly larger than predicted by simultaneous detection of spatial and temporal disparities. This finding suggests that auditory-visual binding was further facilitated by additional, cognitive, factors, associated with the specific instruction to judge the likelihood of a common cause. Obviously, these instructional influences may reflect similar perceptual effects, as have been shown previously by increasing the complexity or cognitive compellingness of auditory-visual stimuli. (C) 2003 Elsevier Science B.V. All rights reserved.
引用
收藏
页码:468 / 478
页数:11
相关论文
共 87 条
[41]   Spatial determinants of multisensory integration in cat superior colliculus neurons [J].
Meredith, MA ;
Stein, BE .
JOURNAL OF NEUROPHYSIOLOGY, 1996, 75 (05) :1843-1857
[42]   INTERACTIONS AMONG CONVERGING SENSORY INPUTS IN THE SUPERIOR COLLICULUS [J].
MEREDITH, MA ;
STEIN, BE .
SCIENCE, 1983, 221 (4608) :389-391
[43]  
MEREDITH MA, 1986, BRAIN RES, V365, P350
[44]   VISUAL, AUDITORY, AND SOMATOSENSORY CONVERGENCE ON CELLS IN SUPERIOR COLLICULUS RESULTS IN MULTISENSORY INTEGRATION [J].
MEREDITH, MA ;
STEIN, BE .
JOURNAL OF NEUROPHYSIOLOGY, 1986, 56 (03) :640-662
[45]  
Michelsen A, 1978, SENSORY ECOLOGY, P345
[46]  
Michotte A., 1963, PERCEPTION CAUSALITY
[47]   ON THE MINIMUM AUDIBLE ANGLE [J].
MILLS, AW .
JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA, 1958, 30 (04) :237-246
[48]   VISUAL SYSTEMS VIEW OF ACOUSTIC SPACE [J].
MORRELL, F .
NATURE, 1972, 238 (5358) :44-&
[49]   PARALLEL AND SERIAL PROCESSES IN THE HUMAN OCULOMOTOR SYSTEM - BIMODAL INTEGRATION AND EXPRESS SACCADES [J].
NOZAWA, G ;
REUTERLORENZ, PA ;
HUGHES, HC .
BIOLOGICAL CYBERNETICS, 1994, 72 (01) :19-34
[50]   AUDITORY PSYCHOMOTOR COORDINATION AND VISUAL-SEARCH PERFORMANCE [J].
PERROTT, DR ;
SABERI, K ;
BROWN, K ;
STRYBEL, TZ .
PERCEPTION & PSYCHOPHYSICS, 1990, 48 (03) :214-226