Removing Time Variation with the Anti-Hebbian Differential Synapse

I describe a local synaptic learning rule that can be used to remove the effects of certain types of systematic temporal variation in the inputs to a unit. According to this rule, changes in synaptic weight result from a conjunction of short-term temporal changes in the inputs and the output. Formally, Delta(weight(i)) approximate to -Delta (input(i)) x Delta(output) This is like the differential rule proposed by Klopf (1986) and Kosko (19861, except for a change of sign, which gives it an anti-Hebbian character. By itself this rule is insufficient. A weight conservation condition is needed to prevent the weights from collapsing to zero, and some further constraint - implemented here by a biasing term - to select particular sets of weights from the subspace of those which give minimal variation. As an example, I show that this rule will generate center-surround receptive fields that remove temporally varying linear gradients from the inputs.