An asymptotic property of model selection criteria

被引:65
作者
Yang, YH [1 ]
Barron, AR
机构
[1] Iowa State Univ, Dept Stat, Ames, IA 50011 USA
[2] Yale Univ, Dept Stat, New Haven, CT 06520 USA
基金
美国国家科学基金会;
关键词
complexity penalty; convergence rate; model selection; nonparametric density estimation; resolvability;
D O I
10.1109/18.650993
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Probability models are estimated by use of penalized log-likelihood criteria related to AIC and MDL, The accuracies of the density estimators are shown to be related to the tradeoff between three terms: the accuracy of approximation, the model dimension, and the descriptive complexity of the model classes, The asymptotic risk is determined under conditions on the penalty term, and is shown to be minimax optimal for some cases. As an application, we show that the optimal rate of convergence is simultaneously achieved for log-densities in Sobolev spaces W-2(s)(U) without knowing the smoothness parameter s and norm parameter U in advance, Applications to neural network models and sparse density function estimation are also provided.
引用
收藏
页码:95 / 116
页数:22
相关论文
共 48 条
[1]  
Akaike H, 1973, 2 INT S INFORM THEOR, P199, DOI [10.1007/978-1-4612-1694-0, 10.1007/978-1-4612-1694-0_15]
[2]  
[Anonymous], 1975, CLASSIF SOC B
[3]  
Barron A., 1994, Proceedings. 1994 IEEE International Symposium on Information Theory (Cat. No.94CH3467-8), DOI 10.1109/ISIT.1994.394933
[4]  
Barron A. R., 1985, THESIS STANFORD U ST
[5]   UNIVERSAL APPROXIMATION BOUNDS FOR SUPERPOSITIONS OF A SIGMOIDAL FUNCTION [J].
BARRON, AR .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1993, 39 (03) :930-945
[6]   APPROXIMATION AND ESTIMATION BOUNDS FOR ARTIFICIAL NEURAL NETWORKS [J].
BARRON, AR .
MACHINE LEARNING, 1994, 14 (01) :115-133
[7]   MINIMUM COMPLEXITY DENSITY-ESTIMATION [J].
BARRON, AR ;
COVER, TM .
IEEE TRANSACTIONS ON INFORMATION THEORY, 1991, 37 (04) :1034-1054
[8]   APPROXIMATION OF DENSITY-FUNCTIONS BY SEQUENCES OF EXPONENTIAL-FAMILIES [J].
BARRON, AR ;
SHEU, CH .
ANNALS OF STATISTICS, 1991, 19 (03) :1347-1369
[9]  
BARRON AR, 1996, IN PRESS PROB THEORY
[10]  
BERNARDO JM, 1979, J R STAT SOC B, V41, P113