Learning the parts of objects by auto-association

Recognition-by-components is one of the possible strategies proposed for object recognition by the brain, but little is known about the low-level mechanism by which the parts of objects can be learned without a priori knowledge. Recent work by Lee and Seung (Nature 401 (1999) 788) shows the importance of non-negativity constraints in the building of such models. Here we propose a simple feedforward neural network that is able to learn the parts of objects by the auto-association of sensory stimuli. The network is trained to reproduce each input with only excitatory interactions. When applied to a database of facial images, the network extracts localized features that resemble intuitive notion of the parts of faces. This kind of localized, parts-based internal representation is very different from the holistic representation created by the unconstrained network, which emulates principal component analysis. Furthermore, the simple model has some ability to minimize the number of active hidden units for certain tasks and is robust when a mixture of different stimuli is presented. (C) 2002 Elsevier Science Ltd. All rights reserved.