PROBABILITY ESTIMATION AND INFORMATION PRINCIPLES

The paper presents a procedure for the estimation of probability density functions. The procedure is based on information theory concepts. It combines Jaynes' maximum entropy formalism with Akaike's approach to the selection of the optimal member of a hierarchy of models. In the present version of the procedure, prior information about the random variable under consideration is specified in terms of lower and upper bounds on the possible values of the variable. We deal with continuous random variables defined on a finite range, with continuous probability density functions and finite (but unknown) moments. The procedure automatically adjusts the level of sophistication of the resulting probability assignment according to the nature and quantity of the available data; thus preventing us from using too sophisticated models (models with too many adjustable parameters) if the data base is not extensive enough. In a sense the procedure provides an effective combination of 'subjective probability' characterizing the uncertainty of the situation in the case of a small sample, and 'objective probability' representing measured relative frequencies when the data base is large.