The development of shift tolerance and deformation tolerance in neural representations is discussed with reference to a prototypical paradigm, which summarizes the essential problem of representation of a distribution of input patterns containing features that are distributed uniformly throughout an image space and that are subject to variation in form. A form of sparse, local representation is proposed in which the position of a feature is localized with precision proportional to the extent of the representation's tolerance to deformation of the feature, which in turn reflects the extent to which the form of that feature is subject to variation over the probability distribution of input patterns. A local self-organizing mechanism is described which inevitably generates representations of this form, regardless of the initial configuration of the synaptic strength parameters. The form of the representation established by this mechanism is unaffected by the inclusion of superfluous representation units: the position tolerance and deformation tolerance of representation units are independent of the number of units participating in the self-organization process, provided that this number is adequate to form a complete representation. It is demonstrated that this self-organizing mechanism is able to discriminate between distinct features and represents these using separate representation units, even though the various forms of a single feature are represented by a single variation-tolerant unit. The attributes of local position tolerance and deformation tolerance arise purely in response to the invariance properties of the probability distribution of input patterns: the mechanism relies neither on the imposition of prior architectural constraints nor on associations in time between successive patterns in order to generate these attributes.
