Extremely randomized trees

被引:5123
作者
Geurts, P [1 ]
Ernst, D [1 ]
Wehenkel, L [1 ]
机构
[1] Univ Liege, Dept Elect Engn & Comp Sci, B-4000 Liege, Belgium
关键词
supervised learning; decision and regression trees; ensemble methods; cut-point randomization; bias/variance tradeoff; kernel-based models;
D O I
10.1007/s10994-006-6226-1
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
This paper proposes anew tree-based ensemble method for supervised classification and regression problems. It essentially consists of randomizing strongly both attribute and cut-point choice while splitting a tree node. In the extreme case, it builds totally randomized trees whose structures are independent of the output values of the learning sample. The strength of the randomization can be tuned to problem specifics by the appropriate choice of a parameter. We evaluate the robustness of the default choice of this parameter, and we also provide insight on how to adjust it in particular situations. Besides accuracy, the main strength of the resulting algorithm is computational efficiency. A bias/variance analysis of the Extra-Trees algorithm is also provided as well as a geometrical and a kernel characterization of the models induced.
引用
收藏
页码:3 / 42
页数:40
相关论文
共 49 条
[31]   Variance and bias for general loss functions [J].
James, GM .
MACHINE LEARNING, 2003, 51 (02) :115-135
[32]  
KAMATH C, 2002, P 2 SIAM INT C DAT M
[33]  
Kleinberg E., 1990, Ann. Math. Artif. Intell, V1, P207, DOI DOI 10.1007/BF01531079
[34]  
Lin Y, 2002, 1055 U WISC DEP STAT
[35]  
Maree R., 2004, Proceedings of the 6th Asian Conference on Computer Vision, P860
[36]  
Mingers J., 1989, Machine Learning, V3, P319, DOI 10.1007/BF00116837
[37]   Inference for the generalization error [J].
Nadeau, C ;
Bengio, Y .
MACHINE LEARNING, 2003, 52 (03) :239-281
[38]  
Quinlan J.R., 1993, C4.5 : programs for machine learning
[39]  
Torgo L., 1999, THESIS U PORTO
[40]  
Valentini G, 2004, J MACH LEARN RES, V5, P725