Exposure to asynchronous audiovisual speech extends the temporal window for audiovisual integration

被引:136
作者
Navarra, J
Vatakis, A
Zampini, M
Soto-Faraco, S
Humphreys, W
Spence, C
机构
[1] Grp Rec Neurociencia Cognit, Barcelona, Spain
[2] Univ Oxford, Dept Expt Psychol, Oxford OX1 3UD, England
来源
COGNITIVE BRAIN RESEARCH | 2005年 / 25卷 / 02期
关键词
multisensory integration; asynchrony; temporal recalibration; speech; music; temporal order judgment;
D O I
10.1016/j.cogbrainres.2005.07.009
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We examined whether monitoring asynchronous audiovisual speech induces a general temporal recalibration of auditory and visual sensory processing. Participants monitored a videotape featuring a speaker pronouncing a list of words (Experiments 1 and 3) or a hand playing a musical pattern on a piano (Experiment 2). The auditory and visual channels were either presented in synchrony, or else asynchronously (with the visual signal leading the auditory signal by 300 ms; Experiments 1 and 2). While performing the monitoring task, participants were asked to judge the temporal order of pairs of auditory (white noise bursts) and visual stimuli (flashes) that were presented at varying stimulus onset asynchronies (SOAs) during the session. The results showed that, while monitoring desynchronized speech or music, participants required a longer interval between the auditory and visual stimuli in order to perceive their temporal order correctly, suggesting a widening of the temporal window for audiovisual integration. The fact that no such recalibration occurred when we used a longer asynchrony (1000 ms) that exceeded the temporal window for audiovisual integration (Experiment 3) supports this conclusion. (c) 2005 Elsevier B.V. All rights reserved.
引用
收藏
页码:499 / 507
页数:9
相关论文
共 44 条
[21]   HEARING LIPS AND SEEING VOICES [J].
MCGURK, H ;
MACDONALD, J .
NATURE, 1976, 264 (5588) :746-748
[22]   Auditory capture of vision: examining temporal ventriloquism [J].
Morein-Zamir, S ;
Soto-Faraco, S ;
Kingstone, A .
COGNITIVE BRAIN RESEARCH, 2003, 17 (01) :154-163
[23]   Temporal constraints on the McGurk effect [J].
Munhall, KG ;
Gribble, P ;
Sacco, L ;
Ward, M .
PERCEPTION & PSYCHOPHYSICS, 1996, 58 (03) :351-362
[24]   Impaired processing of prosodic and musical patterns after right hemisphere damage [J].
Nicholson, KG ;
Baum, S ;
Kilgour, A ;
Koh, CK ;
Munhall, KG ;
Cuddy, LL .
BRAIN AND COGNITION, 2003, 52 (03) :382-389
[25]  
NOESSELT T, 2004, M 5 INT MULT RES FOR
[26]   Processing prosodic and musical patterns: A neuropsychological investigation [J].
Patel, AD ;
Peretz, I ;
Tramo, M ;
Labreque, R .
BRAIN AND LANGUAGE, 1998, 61 (01) :123-144
[27]   Auditory influences on visual temporal rate perception [J].
Recanzone, GH .
JOURNAL OF NEUROPHYSIOLOGY, 2003, 89 (02) :1078-1093
[28]  
Reeves Byron, 1993, Tech. Rep.
[29]  
RIHS S, 1995, P MOSAIC WORKSH ADV, P133
[30]   Sound alters visual motion perception [J].
Sekuler, R ;
Sekuler, AB ;
Lau, R .
NATURE, 1997, 385 (6614) :308-308