Bootstrap aggregating

( Aggregating of Engl. Bootstrap ) Bagging is a method to make predictions from various regression or classification models to combine and was developed by Leo Breiman .. The results of the models are then averaged in the simplest case, ie the result of each model prediction is included in the forecasts, with equal weight.

Ideally, one uses sampling of the circumference from the population and create predictive models (). For a value then result forecast values ​​. Is the predictive value of class membership, the most frequently predicted class could be taken as a prediction value. In the regression case, the predictive value than

Or generally with weights

The weights in both classification and regression in the case could, for example, depend on the quality of the model prediction, ie "good" models go with a larger weight a "bad" as models.

The Bagging leads in the case of unstable models, ie Models in which the structure varies greatly depending on the sample data ( see, eg, Classification and Regression Trees ) mostly to significantly improved predictions.