PROBABILISTIC DESIGN OF LAYERED NEURAL NETWORKS BASED ON THEIR UNIFIED FRAMEWORK

We propose three ways of designing artificial neural networks based on a unified framework and use them to develop new models. First, we show that artificial neural networks can be understood as probability density functions with parameters. Second, we propose three design methods for new models: a method for estimating the occurrence probability of the inputs, a method for estimating the variance of the outputs, and a method for estimating the simultaneous probability of inputs and outputs. Third, we design three new models using the proposed methods: a neural network with occurrence probability estimation, a neural network with output variance estimation, and a probability competition neural network. Our experimental results show that the proposed neural networks have important abilities in information processing: they can tell how often a given input occurs, how widely the outputs are distributed, and from what kinds of inputs a given output is inferred.