Predicting conditional probability densities of stationary stochastic time series

Feedforward neural networks applied to time series prediction are usually trained to predict the next time step x(t + 1) as a function of m previous values, x(t):=(x(t), x(t + 1),...,x(t - m + 1)), which, if a sum-of-squares error function is chosen, results in predicting the conditional mean (y\x(t)) However, further information about the distribution is lost, which is a serious drawback especially in the case of multimodality, where the conditional mean alone turns out to be a rather insufficient or even misleading quantity. The only satisfactory approach in the general case is therefore to predict the whole conditional probability density for the time series, P(x(t + 1)\x(t),x(t - 1),...,x(t - m + 1)). We deduce here a two-hidden-layer universal approximator network for modelling this function, and develop a training algorithm from maximum likelihood. The method is tested on three time series of different nature, which will demonstrate how state-space dependent variances and multimodal transitions can be learned. We will finally note comparisons with other recent neural network approaches to this problem, and will state results on a benchmark problem. (C) 1997 Elsevier Science Ltd.