Regression Trees

Synonyms

Decision trees for regression; Piecewise constant models; Tree-based regression

Definition

Regression trees are supervised learning methods that address multiple regression problems. They provide a tree-based approximation , of an unknown regression function with Y ∈ ℜ and ε ≈ N(0, σ ²), based on a given sample of data D = {⟨x_i,1,...,x_i,p,y_i⟩}_i = 1ⁿ. The obtained models consist of a hierarchy of logical tests on the values of any of the p predictor variables. The terminal nodes of these trees, known as the leaves, contain the numerical predictions of the model for the target variable Y .

Motivation and Background

Work on regression trees goes back to the AID system by Morgan and Sonquist Morgan and Sonquist (1963). Nonetheless, the seminal work is the book Classification and Regression Trees by Breiman and colleagues (Breiman, Friedman, Olshen, & Stone, 1984). This book has established several standards in many theoretical aspects of tree-based regression,