Best asymptotic normality of the kernel density entropy estimator for smooth densities

被引:27
作者
Eggermont, PPB [1 ]
LaRiccia, VN [1 ]
机构
[1] Univ Delaware, Dept Math Sci, Newark, DE 19716 USA
关键词
convexity; entropy estimation; kernel density estimators; Kullback-Leibler divergence; submartingales;
D O I
10.1109/18.761291
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In the random sampling setting we estimate the entropy of a probability density distribution by the entropy of a kernel density estimator using the double exponential kernel. Under mild smoothness and moment conditions we show that the entropy of the kernel density estimator equals a sum of independent and identically distributed (i.i.d.) random variables plus a perturbation which is asymptotically negligible compared to the parametric rate n-(1/2). An essential part in the proof is obtained by exhibiting almost sure bounds for the Kullback-Leibler divergence between the kernel density estimator and its expected value. The basic technical tools are Doob's submartingale inequality and convexity (Jensen's inequality).
引用
收藏
页码:1321 / 1326
页数:6
相关论文
共 21 条
[1]  
AHMAD IA, 1989, IEEE T INFORM THEORY, V36, P688
[2]  
[Anonymous], 1968, Probability
[3]   DISTRIBUTION ESTIMATION CONSISTENT IN TOTAL VARIATION AND IN 2 TYPES OF INFORMATION DIVERGENCE [J].
BARRON, AR ;
GYORFI, L ;
VANDERMEULEN, EC .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1992, 38 (05) :1437-1454
[4]  
Beirlant J, 1997, INT J MATH STAT SCI, V6, P17
[5]  
Devroye L., 1987, A course in density estimation
[6]   Nonlinearly smoothed EM density estimation with automated smoothing parameter selection for nonparametric deconvolution problems [J].
Eggermont, PPB ;
LaRiccia, VN .
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 1997, 92 (440) :1451-1458
[7]  
EGGERMONT PPB, 1995, J NONPARAMETR STAT, V4, P211
[8]  
EGGERMONT PPB, 1998, UNPUB OPTIMAL CONVER
[9]   NONPARAMETRIC ROUGHNESS PENALTIES FOR PROBABILITY DENSITIES [J].
GOOD, IJ ;
GASKINS, RA .
BIOMETRIKA, 1971, 58 (02) :255-+
[10]   DENSITY-FREE CONVERGENCE PROPERTIES OF VARIOUS ESTIMATORS OF ENTROPY [J].
GYORFI, L ;
VANDERMEULEN, EC .
COMPUTATIONAL STATISTICS & DATA ANALYSIS, 1987, 5 (04) :425-436