DIVERGENCE MEASURES BASED ON THE SHANNON ENTROPY

被引：2822

作者：

LIN, JH

机构：

[1] Department of Computer Science, Brandeis University, Waltham

来源：

IEEE TRANSACTIONS ON INFORMATION THEORY | 1991年 / 37卷 / 01期

关键词：

D O I：

10.1109/18.61115

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

A new class of information-theoretic divergence measures based on the Shannon entropy is introduced. Unlike the well-known Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds. These bounds are crucial in many applications of divergence measures. The new measures are also well characterized by the properties of nonnegativity, finiteness, semiboundedness, and boundedness. © 1991 IEEE

引用

页码：145 / 151

页数：7

共 32 条

[1] ACZEL J, 1975, MEASURES INFORMATION
[2] ALI SM, 1966, J ROY STAT SOC B, V28, P131
[3] BASSAT MB, 1978, INFORM CONTR, V39, P227
[4] Chen C. H., 1973, STAT PATTERN RECOGNI
[5] INFORMATION AND DISTANCE MEASURES, ERROR BOUNDS, AND FEATURE SELECTION
CHEN, CH
[J]. INFORMATION SCIENCES, 1976, 10 (02) : 159 - 173
[6] APPROXIMATING DISCRETE PROBABILITY DISTRIBUTIONS WITH DEPENDENCE TREES
CHOW, CK
LIU, CN
[J]. IEEE TRANSACTIONS ON INFORMATION THEORY, 1968, 14 (03) : 462 - +
[7] Csiszar I., 1967, STUDIA SCI MATH HUNG, V2, P299
[8] GALLAGER RG, 1968, INFORMATION THEORY R
[9] GOKHALE DV, 1978, INFORMATION CONTINGE
[10] GUIASU S, 1977, INFORMATION THEORY A

← 1 2 3 4 →