Learning State Space Trajectories in Recurrent Neural Networks

Many neural network learning procedures compute gradients of the errors on the output layer of units after they have settled to their final values. We describe a procedure for finding partial derivative E/partial derivative(wij) where E is an error functional of the temporal trajectory of the states of a continuous recurrent network and w(ij) are the weights of that network. Computing these quantities allows one to perform gradient descent in the weights to minimize E. Simulations in which networks are taught to move through limit cycles are shown. This type of recurrent network seems particularly suited for temporally continuous domains, such as signal processing, control, and speech.