AXIOMATIC CHARACTERIZATION OF THE DIRECTED DIVERGENCES AND THEIR LINEAR COMBINATIONS

The directed divergences of two probability densities p and q are given by fp(x)log(p(x)/ q(x))dx and by the same expression with p and q interchanged; the divergence is the sum of the directed divergences. These quantities have applications in information theory and to the problem of assigning prior probabilities subject to constraints. It is shown that the directed divergences and their positive linear combinations, including the divegence, are characterized by axioms of positivity, additivity, and finiteness, which are fundamental in work on prior probabilities. In the course of the proof, the latter two are shown to imply yet another axiom: linear invariance. ©1979 IEEE