Encoding of binocular disparity by complex cells in the cat's visual cortex

To examine the roles that complex cells play in stereopsis, we have recorded extracellularly from isolated single neurons in the striate cortex of anesthetized paralyzed cats. We measured binocular responses of complex cells using a comprehensive stimulus set that encompasses all possible combinations of positions over the receptive fields for the two eyes. For a given position combination, stimulus contrast could be the same for the two eyes (2 bright or 2 dark bars) or opposite (1 bright and 1 dark). These measurements provide a binocular receptive field (RF) profile that completely characterizes complex cell responses in a joint domain of left and right stimulus positions. Complex cells typically exhibit a strong selectivity for binocular disparity, but are only broadly selective for stimulus position. For most cells, selectivity for disparity is more than twice as narrow as that for position. These characteristics are highly desirable if we assume that a disparity sensor should exhibit position invariance while encoding small changes in stimulus depth. Complex cells have nearly identical binocular RFs for bright and dark stimuli as long as the sign of stimulus contrast is the same for the two eyes. When stimulus contrast is opposite, the binocular RF also is inverted such that excitatory subregions become suppressive. We have developed a disparity energy model that accounts for the behavior of disparity-sensitive complex cells. This is a hierarchical model that incorporates specific constraints on the selection of simple cells from which a complex cell receives input. Experimental data are used to examine quantitatively predictions of the model. Responses of complex cells generally agree well with predictions of the disparity energy model. However, various types of deviations from the predictions also are found, including a highly elongated excitatory region beyond that supported by a single energy mechanism. Complex cells in the visual cortex appear to provide a next level of abstraction in encoding information for stereopsis based on the activity of a group of simple-type subunits. In addition to exhibiting narrow disparity tuning and position invariance, these cells seem to provide a partial solution to the stereo correspondence problem that arises in complex natural scenes. Based on their binocular response properties, these cells provide a substantial reduction in the complexity of the correspondence problem.