Boosting Methods for Regression

被引：18

作者：

Nigel Duffy

David Helmbold

机构：

[1] University of California,Computer Science Department

[2] Santa Cruz,undefined

来源：

Machine Learning | 2002年 / 47卷

关键词：

learning; boosting; arcing; ensemble methods; regression; gradient descent;

D O I：

暂无

中图分类号：

学科分类号：

摘要：

In this paper we examine ensemble methods for regression that leverage or “boost” base regressors by iteratively calling them on modified samples. The most successful leveraging algorithm for classification is AdaBoost, an algorithm that requires only modest assumptions on the base learning method for its strong theoretical guarantees. We present several gradient descent leveraging algorithms for regression and prove AdaBoost-style bounds on their sample errors using intuitive assumptions on the base learners. We bound the complexity of the regression functions produced in order to derive PAC-style bounds on their generalization errors. Experiments validate our theoretical results.

引用

页码：153 / 200

页数：47

共 25 条

[1]

Barron A. R.(1993)Universal approximation bounds for superposition of a sigmoidal function IEEE Trans. on Information Theory 39 930-945

[2]

Bauer E.(1999)An empirical comparison of voting classification algorithms: Bagging, boosting and variants Machine Learning 36 105-39

[3]

Kohavi R.(1996)Bagging predictors Machine Learning 24 123-140

[4]

Breiman L.(1998)Arcing classifiers The Annals of Statistics 26 801-849

[5]

Breiman L.(1999)Prediction games and arcing algorithms Neural Computation 11 1493-1517

[6]

Breiman L.(1995)Boosting a weak learning algorithm by majority Information and Computation 121 256-285

[7]

Freund Y.(1997)A decision-theoretic generalization of on-line learning and an application to boosting Journal of Computer and System Sciences 55 119-139

[8]

Freund Y.(2000)Additive logistic regression: A statistical view of boosting The Annals of Statistics 28 337-374

[9]

Schapire R. E.(1992)Asimple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training The Annals of Statistics 20 608-613

[10]

Friedman J.(1994)Cryptographic limitations on learning Boolean formulae and finite automata Journal of the ACM 41 67-95

← 1 2 3 →