University of California, Berkeley - Department of Statistics
Averaging over an essemble of regressors (trees, for example) can significantly reduce variance. Bagging (Breiman 1996) is an early successful example. But averaging does not have much effect on the bias. An adaptation of bagging is derived that reduces both bias and variance. Examples are given on a variety of data sets using trees or nearest neighbor as the base regressors. We also show, by example, that using the bias reduction scheme, quite small regression trees can give accurate classification on two class problems.