Back

BART: Bayesian Additive Regression Trees

Hugh A. ChipmanEdward I. GeorgeRobert E. McCulloch

article2008Annals of Applied Statistics 2010, Vol. 4, No. 1, 266-2982,072 citations

Introduces Bayesian Additive Regression Trees (BART), a nonparametric sum-of-trees model that combines the high predictive accuracy of ensemble learning with full Bayesian uncertainty quantification and variable selection.

ensemble learningfrequenciesgenetic selectionBARTTreesPrior
arXiv: 0806.3286
  • Paper: Greedy function approximation: A gradient boosting machine, Jerome H. Friedman (2001). This seminal paper introduces gradient boosting and regularized sum-of-trees models, which directly inspire the additive tree architecture and regularization philosophy of BART.
  • Paper: Random Forests, Leo Breiman (2001). It provides foundational principles on ensemble decision-tree modeling and randomizing tree structures that form the conceptual background for ensemble regression methods like BART.
  • Paper: Bagging Predictors, L. Breiman (1996). It establishes the variance-reduction benefits of aggregating multiple decision trees, establishing the baseline ensemble methodology that BART formalizes probabilistically.
  • Paper: A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting, Yoav Freund et al. (1997). It introduces the theoretical foundations of boosting weak learners into strong predictors, serving as primary motivation for constraining BART trees to be weak learners.
  • Paper: Sparse Bayesian Learning and the Relevance Vector Machine, Michael E. Tipping (2001). It illustrates how Bayesian priors and marginal likelihood regularization yield sparse, adaptive nonparametric regression models with principled uncertainty quantification.
Cover for BART: Bayesian Additive Regression Trees

Abstract

We develop a Bayesian "sum-of-trees" model where each tree is constrained by a regularization prior to be a weak learner, and fitting and inference are accomplished via an iterative Bayesian backfitting MCMC algorithm that generates samples from a posterior. Effectively, BART is a nonparametric Bayesian regression approach which uses dimensionally adaptive random basis elements. Motivated by ensemble methods in general, and boosting algorithms in particular, BART is defined by a statistical model: a prior and a likelihood. This approach enables full posterior inference including point and interval estimates of the unknown regression function as well as the marginal effects of potential predictors. By keeping track of predictor inclusion frequencies, BART can also be used for model-free variable selection. BART's many features are illustrated with a bake-off against competing methods on 42 different data sets, with a simulation experiment and on a drug discovery classification problem.

Topics

ensemble learningfrequenciesgenetic selectioninferencemarginalsmcmcregularizationweak learnermodel free

Keywords

BARTTreesPriorValuesModelDataBayesianTreeUsedRegression

Table of Contents

  • 1 Introduction
  • 2 The BART model
  • 2.1 A sum-of-trees model
  • 2.2 A regularization prior
  • 2.2.1 Prior independence and symmetry
  • 2.2.2 The TjT_{j} prior
  • 2.2.3 The μi​j|Tj\mu_{ij}|T_{j} prior
  • 2.2.4 The σ\sigma prior
  • 2.2.5 The choice of mm
  • 3 Extracting information from the posterior
  • 3.1 A Bayesian backfitting MCMC algorithm
  • 3.2 Posterior inference statistics
  • 4 BART probit for classification
  • 5 Applications
  • 5.1 Predictive comparisons on 42 data sets
  • 5.2 Friedman’s five dimensional test function
  • 5.2.1 A simple application of BART
  • 5.2.2 Finding low dimensional structure in high dimensional data
  • 5.2.3 Out-of-sample comparisons with competing methods
  • 5.3 Classification: A drug discovery application
  • 6 Execution time considerations
  • 7 Extensions and related work
  • 8 Discussion
  • References

Knowls

  1. Knowl 1 — BART Sum-of-Trees Model Formulation

    model/method

    Bayesian Additive Regression Trees (BART) models an unknown continuous regression function f(x)=E(Y∣x)f(x) = \mathbb{E}(Y \mid x) for a pp-dimensional input vector x=(x1,…,xp)∈Rpx = (x_1, \dots, x_p) \in \mathbb{R}^p as a sum of mm binary regression trees:

    Y=∑j=1mg(x;Tj,Mj)+ε,ε∼N(0,σ2)Y = \sum_{j=1}^m g(x; T_j, M_j) + \varepsilon, \quad \varepsilon \sim N(0, \sigma^2)

    where each TjT_j denotes a binary tree structure consisting of interior node decision rules and bjb_j terminal nodes, and Mj={μ1j,μ2j,…,μbjj}M_j = \{\mu_{1j}, \mu_{2j}, \dots, \mu_{b_j j}\} denotes the set of continuous parameters assigned to the terminal nodes of TjT_j.

    The decision rules at interior nodes are binary splits of the predictor space, typically of the form {xk≤c}\{x_k \le c\} versus {xk>c}\{x_k > c\} for continuous predictors xkx_k. The function g(x;Tj,Mj)g(x; T_j, M_j) maps any input vector xx through the sequence of decision rules of tree TjT_j to a unique terminal node and assigns the corresponding parameter value μij∈Mj\mu_{ij} \in M_j. The conditional mean of the response is given by:

    E(Y∣x)=∑j=1mg(x;Tj,Mj)\mathbb{E}(Y \mid x) = \sum_{j=1}^m g(x; T_j, M_j)

    When m>1m > 1, each individual tree component g(x;Tj,Mj)g(x; T_j, M_j) acts as a weak learner explaining a small portion of the overall function, incorporating main effects when a tree splits on a single variable or interaction effects of varying orders when a tree splits on multiple distinct variables.

  2. Knowl 2 — BART Regularization Prior Specification

    model/method

    The BART model regularizes the sum-of-trees representation to keep individual tree contributions small through an independent prior over tree structures TjT_j, terminal node parameters MjM_j, and the residual variance σ2\sigma^2:

    p((T1,M1),…,(Tm,Mm),σ)=[∏j=1mp(Tj)∏i=1bjp(μij∣Tj)]p(σ)p((T_1, M_1), \dots, (T_m, M_m), \sigma) = \left[\prod_{j=1}^m p(T_j) \prod_{i=1}^{b_j} p(\mu_{ij} \mid T_j)\right] p(\sigma)

    The prior components and recommended default hyperparameters are specified as follows:

    1. Tree Structure Prior p(Tj)p(T_j): The probability that a node at depth d∈{0,1,2,… }d \in \{0, 1, 2, \dots\} is nonterminal is given by α(1+d)−β\alpha(1 + d)^{-\beta}, with default hyperparameters α=0.95\alpha = 0.95 and β=2.0\beta = 2.0. Given an interior node split, splitting variables are chosen uniformly at random from available predictors, and split values are chosen uniformly from the discrete set of available unique covariate values.

    2. Terminal Node Mean Prior p(μij∣Tj)p(\mu_{ij} \mid T_j): Conditional on TjT_j, terminal node parameters are independent conjugate normal draws μij∼N(0,σμ2)\mu_{ij} \sim N(0, \sigma_\mu^2). The response variable YY is centered and scaled so that its observed range [ymin⁡,ymax⁡][y_{\min}, y_{\max}] maps to [−0.5,0.5][-0.5, 0.5]. To ensure that the induced prior on the ensemble mean E(Y∣x)=∑j=1mμij∼N(0,mσμ2)\mathbb{E}(Y \mid x) = \sum_{j=1}^m \mu_{ij} \sim N(0, m\sigma_\mu^2) covers the range [−0.5,0.5][-0.5, 0.5] within kk standard deviations, the scale is set to:

    σμ=0.5km\sigma_\mu = \frac{0.5}{k\sqrt{m}}

    with default shrinkage parameter k=2k = 2.

    1. Residual Variance Prior p(σ)p(\sigma): An inverse chi-square prior σ2∼νλ/χν2\sigma^2 \sim \nu\lambda / \chi^2_\nu is used. Hyperparameters ν\nu (degrees of freedom) and λ\lambda (scale) are calibrated by specifying ν\nu (default ν=3\nu = 3) and choosing λ\lambda such that P(σ<σ^)=qP(\sigma < \hat{\sigma}) = q (default q=0.90q = 0.90), where σ^\hat{\sigma} is a data-based rough overestimate of σ\sigma obtained from the residual standard deviation of a least squares linear regression or the sample standard deviation of YY.

    2. Number of Trees mm: The recommended default ensemble size is m=200m = 200.

  3. Knowl 3 — Bayesian Backfitting MCMC Algorithm for BART

    algorithm

    Fitting and posterior inference for the BART sum-of-trees model Y=∑j=1mg(x;Tj,Mj)+εY = \sum_{j=1}^m g(x; T_j, M_j) + \varepsilon with ε∼N(0,σ2)\varepsilon \sim N(0, \sigma^2) is conducted using a Bayesian backfitting Gibbs sampler that sequentially updates each tree conditionally on the partial residuals of all other trees.

    Input: Data (xi,yi)i=1n(x_i, y_i)_{i=1}^n, number of trees mm, prior hyperparameters (α,β,σμ,ν,λ)(\alpha, \beta, \sigma_\mu, \nu, \lambda), number of iterations KK
    Output: Posterior draws of tree ensembles (T1(k),M1(k),…,Tm(k),Mm(k))(T_1^{(k)}, M_1^{(k)}, \dots, T_m^{(k)}, M_m^{(k)}) and residual variance σ(k)\sigma^{(k)} for k=1,…,Kk = 1, \dots, K
    Initialize Tj(0)T_j^{(0)} as single-node trees and set μ1j(0)=0\mu_{1j}^{(0)} = 0 for all j=1,…,mj = 1, \dots, m
    Initialize σ(0)=σ^\sigma^{(0)} = \hat{\sigma}
    for k=1k = 1 to KK do
        for j=1j = 1 to mm do
            Compute partial residuals Rj=y−∑l≠jg(x;Tl(k),Ml(k))R_j = y - \sum_{l \neq j} g(x; T_l^{(k)}, M_l^{(k)})
            Propose a new tree structure TjpropT_j^{\text{prop}} via one of four Metropolis-Hastings moves: GROW (probability 0.25), PRUNE (probability 0.25), CHANGE a split rule (probability 0.40), or SWAP parent-child rules (probability 0.10)
            Accept TjpropT_j^{\text{prop}} with Metropolis-Hastings acceptance ratio based on the integrated marginal likelihood:
            p(Tj∣Rj,σ)∝p(Tj)∫p(Rj∣Mj,Tj,σ)p(Mj∣Tj)dMjp(T_j \mid R_j, \sigma) \propto p(T_j) \int p(R_j \mid M_j, T_j, \sigma) p(M_j \mid T_j) dM_j
            Set Tj(k)T_j^{(k)} to the accepted or retained tree
            for each terminal node i=1,…,bji = 1, \dots, b_j of Tj(k)T_j^{(k)} do
                Sample μij(k)∼N(niσ2Rˉj,iniσ2+1σμ2,1niσ2+1σμ2)\mu_{ij}^{(k)} \sim N\left(\frac{\frac{n_i}{\sigma^2} \bar{R}_{j, i}}{\frac{n_i}{\sigma^2} + \frac{1}{\sigma_\mu^2}}, \frac{1}{\frac{n_i}{\sigma^2} + \frac{1}{\sigma_\mu^2}}\right), where nin_i is the number of observations in node ii and Rˉj,i\bar{R}_{j, i} is their average partial residual
            end for
        end for
        Compute full ensemble residuals E=y−∑j=1mg(x;Tj(k),Mj(k))E = y - \sum_{j=1}^m g(x; T_j^{(k)}, M_j^{(k)})
        Sample σ2(k)∼Inv-χ2(ν+n,νλ+∑i=1nEi2ν+n)\sigma^{2 (k)} \sim \text{Inv-}\chi^2\left(\nu + n, \frac{\nu\lambda + \sum_{i=1}^n E_i^2}{\nu + n}\right)
    end for

    Integrating MjM_j out analytically when proposing tree changes avoids the need for trans-dimensional reversible-jump MCMC across varying tree sizes.

  4. Knowl 4 — BART Probit Classification Model

    model/method

    For binary classification where Y∈{0,1}Y \in \{0, 1\}, BART is extended via a probit link specification:

    P(Y=1∣x)=Φ(G(x)),where G(x)=∑j=1mg(x;Tj,Mj)P(Y = 1 \mid x) = \Phi(G(x)), \quad \text{where } G(x) = \sum_{j=1}^m g(x; T_j, M_j)

    and Φ(⋅)\Phi(\cdot) is the standard normal cumulative distribution function. In this formulation, σ=1\sigma = 1 implicitly.

    The terminal node parameter prior is set to μij∼N(0,σμ2)\mu_{ij} \sim N(0, \sigma_\mu^2) with σμ=3.0/(km)\sigma_\mu = 3.0 / (k\sqrt{m}), where kk (default k=2k = 2) ensures that the sum G(x)=∑j=1mμijG(x) = \sum_{j=1}^m \mu_{ij} falls within the interval [−3.0,3.0][-3.0, 3.0] with high prior probability, thereby shrinking the baseline probability P(Y=1∣x)P(Y = 1 \mid x) toward 0.50.5.

    Posterior computation is implemented using the data augmentation framework of Albert and Chib (1993) by introducing continuous latent variables Z1,…,ZnZ_1, \dots, Z_n such that Zi∼N(G(xi),1)Z_i \sim N(G(x_i), 1), with Yi=1Y_i = 1 if Zi>0Z_i > 0 and Yi=0Y_i = 0 if Zi≤0Z_i \le 0. The Gibbs sampling scheme alternates between:

    1. Sampling each latent variable from a truncated normal distribution:

    Zi∣(Yi=1,G(xi))∼N(G(xi),1) truncated below by 0Z_i \mid (Y_i = 1, G(x_i)) \sim N(G(x_i), 1) \text{ truncated below by } 0 Zi∣(Yi=0,G(xi))∼N(G(xi),1) truncated above by 0Z_i \mid (Y_i = 0, G(x_i)) \sim N(G(x_i), 1) \text{ truncated above by } 0

    1. Updating the tree parameters (Tj,Mj)j=1m(T_j, M_j)_{j=1}^m via the standard BART Bayesian backfitting MCMC algorithm using Z1,…,ZnZ_1, \dots, Z_n as the continuous pseudo-response vector and fixing σ=1\sigma = 1.

  5. Knowl 5 — Model-Free Variable Selection via Ensemble Bottlenecking

    model/method

    BART performs model-free variable selection by measuring the relative splitting frequency of each predictor across posterior MCMC iterations. For each posterior draw fk∗f_k^* (k=1,…,Kk = 1, \dots, K), let zikz_{ik} denote the proportion of all tree decision rules across the ensemble that split on the ii-th covariate xix_i. The variable importance measure is defined as the average split proportion:

    vi=1K∑k=1Kzikv_i = \frac{1}{K} \sum_{k=1}^K z_{ik}

    When the ensemble size mm is large (e.g., m=200m = 200), the overcomplete representation allows spurious and irrelevant variables to appear frequently across redundant trees. However, when mm is constrained to a small value (e.g., m=5,10, or 20m = 5, 10, \text{ or } 20), a capacity bottleneck is created that forces predictors to compete for inclusion. Under small mm, the average usage viv_i becomes heavily concentrated on the subset of predictors that explain meaningful variation in the response, enabling model-free identification of true active features without assuming an encompassing parametric model.

  6. Knowl 6 — Posterior Function and Partial Dependence Inference

    model/method

    Given KK post-burn-in MCMC draws of the sum-of-trees ensemble fk∗(x)=∑j=1mg(x;Tj(k),Mj(k))f_k^*(x) = \sum_{j=1}^m g(x; T_j^{(k)}, M_j^{(k)}) for k=1,…,Kk = 1, \dots, K, posterior estimation and uncertainty quantification for the unknown function f(x)f(x) and its marginal functionals proceed as follows:

    1. Point and Interval Estimation of f(x)f(x): The posterior mean estimate of f(x)f(x) at an input xx is approximated by:

    f^(x)=1K∑k=1Kfk∗(x)\hat{f}(x) = \frac{1}{K} \sum_{k=1}^K f_k^*(x)

    A pointwise (1−α)(1 - \alpha) posterior credible interval is obtained directly from the empirical α/2\alpha / 2 and 1−α/21 - \alpha / 2 quantiles of the sample {f1∗(x),…,fK∗(x)}\{f_1^*(x), \dots, f_K^*(x)\}.

    1. Partial Dependence Functions: To isolate the marginal effect of a subset of predictors xsx_s where x=(xs,xc)x = (x_s, x_c) and xcx_c denotes the complementary predictors, the partial dependence function is defined as:

    f(xs)=1n∑i=1nf(xs,xic)f(x_s) = \frac{1}{n} \sum_{i=1}^n f(x_s, x_{ic})

    where xicx_{ic} is the ii-th observed sample value of xcx_c. For each MCMC draw kk, a posterior draw of the partial dependence function is calculated as fk∗(xs)=1n∑i=1nfk∗(xs,xic)f_k^*(x_s) = \frac{1}{n} \sum_{i=1}^n f_k^*(x_s, x_{ic}). The posterior mean and credible intervals for f(xs)f(x_s) are computed using the empirical mean and quantiles across the KK draws.

  7. Knowl 7 — Cross-Validation Benchmark on 42 Real Regression Datasets

    data/table

    BART was evaluated against gradient boosting, random forests, neural networks, and Lasso across 42 real regression datasets with sample sizes ranging from n=96n = 96 to 68066806 and p=3p = 3 to 2828 numeric predictors. For each dataset, 20 random train/test splits (5/6 train, 1/6 test) were generated (840 splits total). Relative RMSE (RRMSE) was computed on each split as the test RMSE divided by the minimum RMSE achieved by any method on that split (so the best method on a split receives RRMSE =1.0= 1.0).

    Method (50%, 75%) Quantiles of RRMSE
    Lasso (1.196, 1.762)
    Boosting (1.068, 1.189)
    Neural net (1.055, 1.195)
    Random forest (1.053, 1.181)
    BART-default (1.055, 1.164)
    BART-cv (1.037, 1.117)

    BART-cv tuned hyperparameter combinations (ν,q,k,m)(\nu, q, k, m) over a 24-point grid via 5-fold cross-validation within each training set. BART-default fixed hyperparameters to (ν,q,k,m)=(3,0.90,2,200)(\nu, q, k, m) = (3, 0.90, 2, 200) without tuning. BART-cv achieved the lowest median and 75th percentile RRMSE across all benchmarks. BART-default was competitive with or superior to all tuned competing black-box algorithms while requiring no cross-validation tuning.

  8. Knowl 8 — Out-of-Sample Performance on High-Dimensional Friedman Simulation

    data/table

    The predictive performance of BART was tested on the 5-dimensional nonlinear Friedman simulation function:

    y=10sin⁡(πx1x2)+20(x3−0.5)2+10x4+5x5+ε,ε∼N(0,1)y = 10\sin(\pi x_1 x_2) + 20(x_3 - 0.5)^2 + 10x_4 + 5x_5 + \varepsilon, \quad \varepsilon \sim N(0, 1)

    with inputs x1,…,xp∼Uniform(0,1)x_1, \dots, x_p \sim \text{Uniform}(0, 1) and sample size n=100n = 100. Uninformative noise predictors x6,…,xpx_6, \dots, x_p were appended to create dimensions p=10,100,1000p = 10, 100, 1000. For each dimension pp, 100 datasets were simulated, and out-of-sample RMSE was computed over 1000 independent test points.

    Method p=10p = 10 p=100p = 100 p=1000p = 1000
    Random forests (1.25, 1.31) (1.46, 1.52) (1.62, 1.68)
    Neural net (1.01, 1.32) (1.71, 2.11) unavailable
    Boosting (0.99, 1.07) (1.03, 1.14) (1.08, 1.33)
    BART-cv (0.90, 0.95) (0.93, 0.98) (0.99, 1.06)
    BART-default (0.89, 0.94) (1.02, 1.10) (1.48, 1.66)

    Entries show the (50%, 75%) quantiles of out-of-sample RMSE across 100 replications. At p=10p=10, both BART variants outperformed all competitors. As pp increased to 100 and 1000 (p≫np \gg n), BART-cv maintained high accuracy with minimal degradation, outperforming all competitors. BART-cv selected smaller ensembles (m=50m = 50) and more aggressive variance priors (q=0.99q = 0.99) in high dimensions to avoid diluting the stochastic tree search.

  9. Knowl 9 — BART Execution Time and Scaling Behavior

    empirical result

    Empirical runtime evaluations of BART on simulated data across sample sizes n∈[100,10000]n \in [100, 10000] and predictor dimensions p∈[10,100]p \in [10, 100] establish two key scaling properties:

    1. Invariance to Predictor Dimension pp: For a fixed sample size nn, BART's execution time remains nearly constant as the number of predictors pp increases from 10 to 100. This occurs because the Metropolis-Hastings tree proposals select splitting variables uniformly at random rather than evaluating exhaustive greedy split optimizations over all pp variables at every node.

    2. Scaling with Sample Size nn: While a short MCMC chain with fixed small trees scales strictly linearly with nn, the execution time of full default BART runs exhibits quadratic growth in nn. This quadratic behavior is an adaptive property: as sample size nn increases, BART's posterior automatically favors deeper, larger trees to capture finer function structure (e.g., growing from an average of 2.52 terminal nodes per tree at n=100n=100 to 3.34 at n=10000n=10000), increasing the per-tree computation required for residual evaluation and likelihood calculations.

  10. Knowl 10 — Semiparametric and Multivariate Extensions of BART

    model/method

    The modularity of the Bayesian backfitting MCMC algorithm allows BART to be embedded within broader hierarchical and multivariate model formulations:

    1. Semiparametric Additive Models: A nonparametric BART sum of trees can be combined with parametric predictor terms or random effects:

    Y=h1(x)+h2(z)+ε,ε∼N(0,σ2)Y = h_1(x) + h_2(z) + \varepsilon, \quad \varepsilon \sim N(0, \sigma^2)

    where h1(x)=∑j=1mg(x;Tj,Mj)h_1(x) = \sum_{j=1}^m g(x; T_j, M_j) is a sum-of-trees and h2(z)h_2(z) is a parametric linear or random effects specification with predictors zz. The MCMC algorithm updates h1h_1 via Bayesian backfitting on the partial residuals Y−h2(z)Y - h_2(z) and samples h2h_2's parameters via a standard conjugate Gibbs step conditional on Y−h1(x)Y - h_1(x).

    1. Multivariate Regression and Seemingly Unrelated Regressions (SUR): For a pp-dimensional multivariate response vector (Y1,…,Yp)(Y_1, \dots, Y_p):

    Yi=hi(xi)+εi,(ε1,…,εp)T∼N(0,Σ)Y_i = h_i(x_i) + \varepsilon_i, \quad (\varepsilon_1, \dots, \varepsilon_p)^T \sim N(0, \Sigma)

    where each hi(xi)h_i(x_i) is an independent sum-of-trees model and Σ\Sigma is a p×pp \times p covariance matrix. Updating is carried out by iteratively drawing each hih_i given {hl}l≠i\{h_l\}_{l \neq i} and Σ\Sigma, followed by sampling Σ\Sigma from an inverse Wishart conditional distribution given all ensemble residuals.

Coverage note — None was omitted; all primary methodological components (model, prior, MCMC algorithm, classification extension, variable selection, extensions) and empirical contributions (real data bake-off, Friedman simulation, scaling analysis) are covered.

References

  1. 1.ABREVEYA, J. and MCCULLOCH, R. (2006). Reversal of fortune: A statistical analysis of penalty calls in the national hockey league. Technical report, Purdue Univ.
  2. 2.ABU-NIMEH, S., NAPPA, D., WANG, X. and NAIR, S. (2008). Detecting phishing emails via Bayesian additive regression trees. Technical report, Southern Methodist Univ., Dallas, TX.
  3. 3.ALBERT, J. H. and CHIB, S. (1993). Bayesian analysis of binary and polychotomous response data. J. Amer. Statist. Assoc. 88 669–679. MR1224394
  4. 4.AMIT, Y. and GEMAN, D. (1997). Shape quantization and recognition with randomized trees. Neural Computation 9 1545–1588.
  5. 5.BLANCHARD, G. (2004). Un algorithme accelere d’echantillonnage Bayesien pour le modele CART. Revue d’Intelligence artificielle 18 383–410.
  6. 6.BREIMAN, L. (1996). Bagging predictors. Machine Learning 26 123–140.
  7. 7.BREIMAN, L. (2001). Random forests. Machine Learning 45 5–32.
  8. 8.CHANG, C.-C. and LIN, C.-J. (2001). LIBSVM: A library for support vector machines. Available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
  9. 9.CHIPMAN, H. A., GEORGE, E. I. and MCCULLOCH, R. E. (1998). Bayesian CART model search (with discussion and a rejoinder by the authors). J. Amer. Statist. Assoc. 93 935–960.
  10. 10.CHIPMAN, H. A., GEORGE, E. I. and MCCULLOCH, R. E. (2002). Bayesian treed models. Machine Learning 48 299–320.
  11. 11.CHIPMAN, H. A., GEORGE, E. I. and MCCULLOCH, R. E. (2007). Bayesian ensemble learning. In Neural Information Processing Systems 19 265–272.
  12. 12.DENISON, D. G. T., MALLICK, B. K. and SMITH, A. F. M. (1998). A Bayesian CART algorithm. Biometrika 85 363–377. MR1649118
  13. 13.DIMITRIADOU, E., HORNIK, K., LEISCH, F., MEYER, D. and WEINGESSEL, A. (2008). e1071: Misc functions of the Department of Statistics (e1071), TU Wien. R package version 1.5-18.
  14. 14.EFRON, B., HASTIE, T., JOHNSTONE, I. and TIBSHIRANI, R. (2004). Least angle regression (with discussion and a rejoinder by the authors). Ann. Statist. 32 407–499. MR2060166
  15. 15.FENG, J., LURATI, L., OUYANG, H., ROBINSON, T., WANG, Y., YUAN, S. and YOUNG, S. (2003). Predictive toxicology: Benchmarking molecular descriptors and statistical methods. Journal of Chemical Information and Computer Sciences 43 1463–1470.
  16. 16.FREUND, Y. and SCHAPIRE, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. System Sci. 55 119–139. MR1473055
  17. 17.FRIEDMAN, J. H. (1991). Multivariate adaptive regression splines (with discussion and a rejoinder by the author). Ann. Statist. 19 1–67. MR1091842
  18. 18.FRIEDMAN, J. H. (2001). Greedy function approximation: A gradient boosting machine. Ann. Statist. 29 1189–1232. MR1873328
  19. 19.GREEN, P. J. (1995). Reversible jump MCMC computation and Bayesian model determination. Biometrika 82 711–732. MR1380810
  20. 20.HASTIE, T. and TIBSHIRANI, R. (2000). Bayesian backfitting (with comments and a rejoinder by the authors). Statist. Sci. 15 196–223. MR1820768
  21. 21.KIM, H., LOH, W.-Y., SHIH, Y.-S. and CHAUDHURI, P. (2007). Visualizable and interpretable regression models with good prediction power. IEEE Transactions: Special Issue on Data Mining and Web Mining 39 565–579.
  22. 22.RIDGEWAY, G. (2004). The gbm package. R Foundation for Statistical Computing, Vienna, Austria.
  23. 23.SING, T., SANDER, O., BEERENWINKEL, N. and LENGAUER, T. (2007). ROCR: Visualizing the performance of scoring classifiers. R package version 1.0-2.
  24. 24.VENABLES, W. N. and RIPLEY, B. D. (2002). Modern Applied Statistics With S, 4th ed. Springer, New York.
  25. 25.WU, Y., TJELMELAND, H. and WEST, M. (2007). Bayesian CART: Prior specification and posterior simulation. J. Comput. Graph. Statist. 16 44–66. MR2345747
  26. 26.ZELLNER, A. (1962). An efficient method of estimating seemingly unrelated regressions and testing for aggregation bias. J. Amer. Statist. Assoc. 57 348–368. MR0139235
  27. 27.ZHANG, J. L. and HAERDLE, W. K. (2010). The Bayesian additive classification tree applied to credit risk modelling. Comput. Statist. Data Anal. 54 1197–1205.
  28. 28.ZHANG, S., SHIH, Y.-C. T. and MULLER, P. (2007). A spatially-adjusted Bayesian additive regression tree model to merge two datasets. Bayesian Anal. 2 611–634. MR2342177
  29. 29.ZHOU, Q. and LIU, J. S. (2008). Extracting sequence features to predict protein-DNA binding: A comparative study. Nucleic Acids Research 36 4137–4148.

Citation

MLA
Chipman, H. A., et al. “BART: Bayesian Additive Regression Trees”. The Annals of Applied Statistics, vol. 4, no. 1, 2010, https://doi.org/10.1214/09-AOAS285.
APA
Chipman, H. A., George, E. I., & McCulloch, R. E. (2010). BART: Bayesian additive regression trees. The Annals of Applied Statistics, 4(1). https://doi.org/10.1214/09-AOAS285
Chicago
Chipman, H. A., E. I. George, and R. E. McCulloch. 2010. “BART: Bayesian Additive Regression Trees”. The Annals of Applied Statistics 4 (1). https://doi.org/10.1214/09-AOAS285.
Harvard
Chipman, H.A., George, E.I. and McCulloch, R.E. (2010) “BART: Bayesian additive regression trees”, The Annals of Applied Statistics, 4(1). Available at: https://doi.org/10.1214/09-AOAS285.
Vancouver
1. Chipman HA, George EI, McCulloch RE (2010) BART: Bayesian additive regression trees. The Annals of Applied Statistics. https://doi.org/10.1214/09-AOAS285

BibTeX

@article{Chipman_2010, title={BART: Bayesian additive regression trees}, volume={4}, ISSN={1932-6157}, url={http://dx.doi.org/10.1214/09-AOAS285}, DOI={10.1214/09-aoas285}, number={1}, journal={The Annals of Applied Statistics}, publisher={Institute of Mathematical Statistics}, author={Chipman, Hugh A. and George, Edward I. and McCulloch, Robert E.}, year={2010}, month=Mar }
Metadata:Crossref

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF