Bagging, subagging and Bragging for improving some prediction algorithms

Bagging (bootstrap aggregating), proposed by Breiman [1], is a method to improve the predictive power of some special estimators or algorithms such as regression or classification trees. First, we review a recently developed theory explaining why bagging decision trees, or also the subagging (subsample aggregating) variant, yields smooth decisions, reducing the variance and mean squared error. We then propose bragging (bootstrap robust aggregating) as a new version of bagging which, in contrast to bagging, is empirically demonstrated to improve also the MARS algorithm which itself already yields continuous function estimates. Finally, bagging is demonstrated as a "module" in conjunction with boosting for an example about tumor classification using microarray gone expressions.