Model complexity control and statistical learning theory

被引：37

作者：

Vladimir Cherkassky

机构：

[1] University of Minnesota,Department of Electrical and Computer Engineering

来源：

Natural Computing | 2002年 / 1卷 / 1期

关键词：

complexity control; model selection; prediction risk; predictive learning; signal denoising; statistical learning theory; VC-generalization bounds; wavelet thresholding;

D O I：

10.1023/A:1015007927558

中图分类号：

学科分类号：

摘要：

We discuss the problem of modelcomplexity control also known as modelselection. This problem frequently arises inthe context of predictive learning and adaptiveestimation of dependencies from finite data.First we review the problem of predictivelearning as it relates to model complexitycontrol. Then we discuss several issuesimportant for practical implementation ofcomplexity control, using the frameworkprovided by Statistical Learning Theory (orVapnik-Chervonenkis theory). Finally, we showpractical applications of Vapnik-Chervonenkis(VC) generalization bounds for model complexitycontrol. Empirical comparisons of differentmethods for complexity control suggestpractical advantages of using VC-based modelselection in settings where VC generalizationbounds can be rigorously applied. We also arguethat VC-theory provides methodologicalframework for complexity control even when itstechnical results can not be directly applied.

引用

页码：109 / 133

页数：24

共 41 条

[1]

Akaike H(1970)Statistical predictor information Ann. Inst. of Stat. Math. 22 203-217

[2]

Barron A(1993)Universal approximation bounds for superposition of a sigmoid function IEEE Trans. Info. Theory 39 930-945

[3]

Bartlett PL(1998)The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network IEEE Trans. on IT 44 525-536

[4]

Baum E(1989)What size net gives valid generalization? Neural Comp. 1 151-160

[5]

Haussler D(1989)Learnability and the VC-dimension Journal of the ACM 36 926-965

[6]

Blumer A(1999)Model selection for regression using VC generalization bounds IEEE Trans on Neural Networks 10 1075-1089

[7]

Ehrenfeucht A(2001)Signal estimation and denoising using VC-theory Neural Networks. Pergamon 14 37-52

[8]

Haussler D(2001)Myopotential denoising of ECG signals using wavelet thresholding methods Neural Networks. Pergamon 14 1129-1137

[9]

Warmuth MK(1979)Smoothing noisy data with spline functions Numerische Math. 31 377-403

[10]

Cherkassky V(1977)A Simulation study of alternatives to ordinary least squares JASA 72 77-106

← 1 2 3 4 5 →