Functional anatomy of a common semantic system for words and pictures

THE relationship between the semantic processing of words and of pictures is a matter of debate among cognitive scientists(1,2). We studied the functional anatomy of such processing by using positron-emission tomography (PET). We contrasted activity during two semantic tasks (probing knowledge of associations between concepts, and knowledge of the visual attributes of these concepts) and a baseline task (discrimination of physical stimulus size), performed either with words or with pictures. Modality-specific activations unrelated to semantic processing occurred in the left inferior parietal lobule for words, and the right middle occipital gyrus for pictures. A semantic network common to both words and pictures extended from the left superior occipital gyrus through the middle and inferior temporal cortex to the inferior frontal gyrus. A picture-specific activation related to semantic tasks occurred in the left posterior inferior temporal sulcus, and word-specific activations related to semantic tasks were localized to the left superior temporal sulcus, left anterior middle temporal gyrus, and left inferior frontal sulcus. Thus semantic tasks activate a distributed semantic processing system shared by both words and pictures, with a few specific areas differentially active for either words or pictures.