RELATING REAL-TIME BACKPROPAGATION AND BACKPROPAGATION-THROUGH-TIME - AN APPLICATION OF FLOW GRAPH INTERRECIPROCITY

We show that signal flow graph theory provides a simple way to relate two popular algorithms used for adapting dynamic neural networks, real-time backpropagation and backpropagation-through-time. Starting with the now graph for real-time backpropagation, we use a simple transposition to produce a second graph. The new graph is shown to be interreciprocal with the original and to correspond to the backpropagation-through-time algorithm. Interreciprocity provides a theoretical argument to verify that both flow graphs implement the same overall weight update.