Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration

被引:136
作者
Navarra, J
Vatakis, A
Zampini, M
Soto-Faraco, S
Humphreys, W
Spence, C
机构
[1] Grp Rec Neurociencia Cognit, Barcelona, Spain
[2] Univ Oxford, Dept Expt Psychol, Oxford OX1 3UD, England
来源
COGNITIVE BRAIN RESEARCH | 2005年 / 25卷 / 02期
关键词
multisensory integration; asynchrony; temporal recalibration; speech; music; temporal order judgment;
D O I
10.1016/j.cogbrainres.2005.07.009
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We examined whether monitoring asynchronous audiovisual speech induces a general temporal recalibration of auditory and visual sensory processing. Participants monitored a videotape featuring a speaker pronouncing a list of words (Experiments 1 and 3) or a hand playing a musical pattern on a piano (Experiment 2). The auditory and visual channels were either presented in synchrony, or else asynchronously (with the visual signal leading the auditory signal by 300 ms; Experiments 1 and 2). While performing the monitoring task, participants were asked to judge the temporal order of pairs of auditory (white noise bursts) and visual stimuli (flashes) that were presented at varying stimulus onset asynchronies (SOAs) during the session. The results showed that, while monitoring desynchronized speech or music, participants required a longer interval between the auditory and visual stimuli in order to perceive their temporal order correctly, suggesting a widening of the temporal window for audiovisual integration. The fact that no such recalibration occurred when we used a longer asynchrony (1000 ms) that exceeded the temporal window for audiovisual integration (Experiment 3) supports this conclusion. (c) 2005 Elsevier B.V. All rights reserved.
引用
收藏
页码:499 / 507
页数:9
相关论文
共 44 条
[1]   Temporal ventriloquism: crossmodal interaction on the time dimension - 2. Evidence from sensorimotor synchronization [J].
Aschersleben, G ;
Bertelson, P .
INTERNATIONAL JOURNAL OF PSYCHOPHYSIOLOGY, 2003, 50 (1-2) :157-163
[2]   Temporal ventriloquism: crossmodal interaction on the time dimension - 1. Evidence from auditory-visual temporal order judgment [J].
Bertelson, P ;
Aschersleben, G .
INTERNATIONAL JOURNAL OF PSYCHOPHYSIOLOGY, 2003, 50 (1-2) :147-155
[3]  
BERTELSON P, 2004, CROSSMODAL SPACE CRO, P141, DOI DOI 10.1093/ACPROF:OSO/9780198524861.003.0007
[4]  
Bolinger D., 1989, INTONATION ITS USES
[5]   Neural correlates of auditory-visual stimulus onset asynchrony detection [J].
Bushara, KO ;
Grafman, J ;
Hallett, M .
JOURNAL OF NEUROSCIENCE, 2001, 21 (01) :300-304
[6]  
Calvert G.A., 2004, HDB MULTISENSORY PRO
[7]   Multisensory integration, perception and ecological validity [J].
De Gelder, B ;
Bertelson, P .
TRENDS IN COGNITIVE SCIENCES, 2003, 7 (10) :460-467
[8]   THE DETECTION OF AUDITORY VISUAL DESYNCHRONY [J].
DIXON, NF ;
SPITZ, L .
PERCEPTION, 1980, 9 (06) :719-721
[9]  
Enns JT., 2004, SENSATION PERCEPTION, DOI DOI 10.1002/0471264385.WEI0105
[10]   The temporal cross-capture of audition and vision [J].
Fendrich, R ;
Corballis, PM .
PERCEPTION & PSYCHOPHYSICS, 2001, 63 (04) :719-725