TEMPORAL PROCESSING IN NEURAL NETWORKS WITH ADAPTIVE SHORT-TERM-MEMORY - A COMPARTMENTAL MODEL APPROACH

A study of temporal processing in neural networks with adaptive short term memory (STM) is presented. In particular, a biologically motivated strategy for incorporating STM into networks is introduced based on a compartmental model of a neuron. It is shown how the STM formed by the model neuron effectively captures the neuron's temporal input history. An explicit expression for the STM is derived in terms of a time-varying weight kernel w(t) in convolution with the input. The weight w(t) is of the form SIGMA(p)w(p)g(p)(t), where {w(p)} is a set of constant weights and {g(p)} is a set of basis functions. The g(p) are determined by the membrane potential response function (Green's function) of the neuron's compartments. The parameters of the STM, which control features such as memory depth, are the membrane time constants of the neuron. It is also shown that in the presence of shunting effects (i) the inclusion of background synaptic activity modifies the membrane time constants thus providing a mechanism for modulating the STM, and (ii) the STM becomes a nonlinear function of inputs. Finally, a neural network implementation of the compartmental model STM is introduced, and learning algorithms for training the network constructed.