Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli

Under certain conditions, auditory and visual information are integrated into a single unified percept even when they originate in different locations in space. The present study shows how this illusion, known as the ventriloquism effect, depends on spatial, temporal and cognitive factors. A method of psychophysical scaling was employed in combination with simple auditory-visual stimuli (tone bursts and flashing light spots) that were presented with various spatiotemporal disparities. Participants either judged their impression of the likelihood of a common cause (Experiment 1) or spatial alignment (Experiment 2) or synchrony of sound and light (Experiment 3). In all three experiments the participants' judgements depended significantly on temporal disparity whereas influences of spatial disparity were significant in Experiments 1 and 2. Optimum scores were always obtained when auditory stimuli were presented with a delay of 50-100 ms after the visual stimuli. These results demonstrate that both temporal and spatial proximity of the two stimuli are critical for the experience of phenomenal causality. On the other hand, spatio-temporal ranges for optimal perception of phenomenal causality in Experiment I were significantly larger than predicted by simultaneous detection of spatial and temporal disparities. This finding suggests that auditory-visual binding was further facilitated by additional, cognitive, factors, associated with the specific instruction to judge the likelihood of a common cause. Obviously, these instructional influences may reflect similar perceptual effects, as have been shown previously by increasing the complexity or cognitive compellingness of auditory-visual stimuli. (C) 2003 Elsevier Science B.V. All rights reserved.