Does audiovisual speech perception use information about facial configuration?

被引:16
作者
Hietanen, JK [1 ]
Manninen, P
Sams, M
Surakka, V
机构
[1] Univ Tampere, Dept Psychol, Human Informat Proc Lab, FIN-33014 Tampere, Finland
[2] Helsinki Univ Technol, Lab Computat Engn, FIN-02150 Espoo, Finland
[3] Univ Tampere, Human Comp Interact Grp, FIN-33101 Tampere, Finland
[4] Tampere Univ Hosp, Dept Neurophysiol, Tampere, Finland
来源
EUROPEAN JOURNAL OF COGNITIVE PSYCHOLOGY | 2001年 / 13卷 / 03期
关键词
D O I
10.1080/09541440042000025
中图分类号
B84 [心理学];
学科分类号
04 ; 0402 ;
摘要
The McGurk illusion effectively demonstrates the audiovisual nature of speech perception. When an auditory syllable is dubbed onto an incongruous visual syllable the resulting percept is usually not either of the components, but their combination or fusion. The present experiment investigated the persistence of the McGurk effect when the facial configuration context of the audiovisual stimuli was manipulated. Two congruent and two incongruent audiovisual syllables were created from spoken /ipi/ and /iki/. These audiovisual tokens were uttered by seven facial configurations of five talkers. All facial configurations produced a clear McGurk effect (reported /iti/ for heard !ipi/ + seen /iki/), but the effect was significantly less when uttered by an asymmetrically scrambled face configuration. The results also showed significant differences in the persistence of the McGurk effect between different talkers. In sum, facial configural information can be used in the audiovisual integration of certain speech segments. This information is not necessary for the integration to occur, but the integration process can be disrupted by face stimuli violating the normal configural information.
引用
收藏
页码:395 / 407
页数:13
相关论文
共 38 条
[1]   UNDERSTANDING FACE RECOGNITION [J].
BRUCE, V ;
YOUNG, A .
BRITISH JOURNAL OF PSYCHOLOGY, 1986, 77 :305-327
[2]   PARALLEL PROCESSING OF THE SEX AND FAMILIARITY OF FACES [J].
BRUCE, V ;
ELLIS, H ;
GIBLING, F ;
YOUNG, A .
CANADIAN JOURNAL OF PSYCHOLOGY-REVUE CANADIENNE DE PSYCHOLOGIE, 1987, 41 (04) :510-520
[3]   THE NEUROPSYCHOLOGY OF LIPREADING [J].
CAMPBELL, R .
PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY B-BIOLOGICAL SCIENCES, 1992, 335 (1273) :39-45
[4]   Dissociating face processing skills: Decisions about lip-read speech, expression, and identity [J].
Campbell, R ;
Brooks, B ;
deHaan, E ;
Roberts, T .
QUARTERLY JOURNAL OF EXPERIMENTAL PSYCHOLOGY SECTION A-HUMAN EXPERIMENTAL PSYCHOLOGY, 1996, 49 (02) :295-314
[5]   Repetition priming for face speech images: Speech-reading primes face identification [J].
Campbell, R ;
De Haan, EHF .
BRITISH JOURNAL OF PSYCHOLOGY, 1998, 89 :309-323
[6]   FROM PIECEMEAL TO CONFIGURATIONAL REPRESENTATION OF FACES [J].
CAREY, S ;
DIAMOND, R .
SCIENCE, 1977, 195 (4275) :312-314
[7]   ROLE OF VISION IN PERCEPTION OF SPEECH [J].
DODD, B .
PERCEPTION, 1977, 6 (01) :31-40
[8]  
Ellis HD, 1986, NEUROPSYCHOLOGY FACE, P1
[9]   INTERACTION OF AUDITION AND VISION IN RECOGNITION OF ORAL SPEECH STIMULI [J].
ERBER, NP .
JOURNAL OF SPEECH AND HEARING RESEARCH, 1969, 12 (02) :423-&
[10]   WHAT CAUSES THE FACE INVERSION EFFECT [J].
FARAH, MJ ;
TANAKA, JW ;
DRAIN, HM .
JOURNAL OF EXPERIMENTAL PSYCHOLOGY-HUMAN PERCEPTION AND PERFORMANCE, 1995, 21 (03) :628-634