Logistic Model Trees

Niels LandwehrMark HallEibe Frank

article2003Machine-mediated learning1,530 citations

Presents Logistic Model Trees (LMT), an algorithm that blends decision tree structures with incrementally refined leaf-level logistic regression models to achieve classification accuracy competitive with boosted trees while maintaining model interpretability and compactness.

Listen

Modern data-driven decision-making frequently requires classification models that deliver both high predictive accuracy and clear interpretability. Standard decision trees provide easily understood rules but suffer from instability and high variance, whereas linear logistic regression models offer stable predictions but cannot capture nonlinear patterns in complex data. The article addresses this trade-off by introducing and evaluating Logistic Model Trees (LMT), an automated machine learning algorithm that integrates standard tree structures with linear logistic regression models situated directly at the leaf nodes.

The primary objective of the article is to demonstrate that LMT produces compact, highly accurate classifiers that automatically scale model complexity to match domain characteristics without requiring manual parameter tuning. To evaluate this approach, the authors tested LMT across 36 diverse benchmark datasets spanning small to large sample sizes. They benchmarked LMT against standard decision trees (C4.5 and CART), linear logistic regression variants, other hybrid tree learners (such as Functional Trees and Naive Bayes Trees), and ensemble methods including boosted trees (AdaBoost) and multi-tree regression systems (M5'). Credibility was reinforced using ten runs of ten-fold cross-validation combined with corrected statistical significance testing.

The findings show that LMT consistently equals or outperforms standard decision trees and standalone logistic regression, never losing a statistically significant comparison against them across all 36 datasets. Specifically, LMT achieved significant accuracy wins over C4.5 in 16 datasets and CART in 17 datasets, while producing drastically smaller trees—often reducing leaf counts from hundreds or thousands down to fewer than a dozen. Furthermore, LMT significantly outperformed other enhanced tree learners and proved highly competitive with boosted decision trees (AdaBoost with 100 iterations), matching their accuracy across most datasets while delivering a single, interpretable tree rather than an opaque voting ensemble of 100 separate trees. On 18 of the 36 datasets, LMT automatically pruned the tree entirely back to the root, selecting a simple linear model when additional tree structure was unjustified.

These results demonstrate that organizations do not necessarily have to sacrifice model interpretability to achieve state-of-the-art predictive performance. By incorporating stepwise attribute selection and incrementally refining logistic models down the tree hierarchy, LMT avoids overfitting and isolates the most critical predictive factors. This reduces the risk of relying on misleading variables, lowers operational complexity, and facilitates regulatory compliance or stakeholder auditing through transparent decision paths.

For practical implementation, teams seeking robust, off-the-shelf classification should adopt LMT as an alternative to both basic decision trees and black-box ensemble methods. However, decision-makers should account for training time constraints, as LMT is several orders of magnitude slower to train than standard C4.5 due to nested cross-validation procedures. Future work recommended by the source includes developing faster fitting procedures to bypass repeated cross-validation and implementing more sophisticated imputation methods for missing data. Confidence in the empirical results is high given the rigorous cross-validation and statistical controls across varied real-world benchmarks.

University of Freiburg University of Waikato classifiers linear models logistic regression models logistic model trees stepwise model tree induction linear logistic regression models

Waikato/weka Landwehr et al (2003).pdf

Paper: Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid, Ron Kohavi (1996). It introduces the foundational concept of hybrid decision trees with probabilistic models at leaf nodes, which Logistic Model Trees directly builds upon.
Paper: Improved Use of Continuous Attributes in C4.5, J. R. Quinlan (1996). It details continuous attribute splitting and pruning mechanisms in C4.5 that serve as core architectural baselines for Logistic Model Trees.
Paper: A Brief Introduction to Boosting, R. Schapire (1999). It provides the foundational principles of boosting algorithms like AdaBoost and LogitBoost used to fit logistic models stagewise within LMT nodes.
Paper: Induction of Decision Trees, J. R. Quinlan (1986). It establishes the foundational top-down induction of decision trees framework upon which tree-structured classifiers rely.
Paper: Greedy function approximation: A gradient boosting machine, Jerome H. Friedman (2001). It develops gradient boosting and stagewise additive expansion techniques that underpin the fitting of logistic models at tree nodes.
Paper: On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes, Andrew Ng et al. (2001). It provides fundamental theoretical and empirical comparisons between discriminative logistic regression and generative classifiers.
Paper: Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms, Thomas G. Dietterich (1998). It outlines the corrected cross-validation and statistical significance testing methodology used directly to benchmark Logistic Model Trees.

Paper: An empirical comparison of supervised learning algorithms, Rich Caruana et al. (2006). It provides an extensive multi-metric empirical benchmark of supervised classification algorithms, contextualizing tree and logistic regression performance.
Paper: Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission, Rich Caruana et al. (2015). It advances the pursuit of combining high predictive accuracy with interpretability by introducing generalized additive models with pairwise interactions.
Paper: Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms, Chris J. Thornton et al. (2012). It automates the selection and hyperparameter optimization of classification algorithms across the WEKA framework, which houses LMT.
Paper: Do we need hundreds of classifiers to solve real world classification problems?, Manuel Fernández Delgado et al. (2014). It evaluates real-world classification performance across hundreds of classifiers to assess the competitive standing of tree-based and linear models.
Paper: Predicting good probabilities with supervised learning, Alexandru Niculescu-Mizil et al. (2005). It investigates probability calibration across supervised learning models, evaluating how tree-based and logistic models predict true probabilities.

Abstract

Tree induction methods and linear models are popular techniques for supervised learning tasks, both for the prediction of nominal classes and numeric values. For predicting numeric quantities, there has been work on combining these two schemes into 'model trees', i.e. trees that contain linear regression functions at the leaves. In this paper, we present an algorithm that adapts this idea for classification problems, using logistic regression instead of linear regression. We use a stagewise fitting process to construct the logistic regression models that can select relevant attributes in the data in a natural way, and show how this approach can be used to build the logistic regression models at the leaves by incrementally refining those constructed at higher levels in the tree. We compare the performance of our algorithm to several other state-of-the-art learning schemes on 36 benchmark UCI datasets, and show that it produces accurate and compact classifiers.

Topics

classifiers linear models logistic regression models logistics regression model supervised machine learning

Keywords

logistic model trees stepwise model tree induction linear logistic regression models LogitBoost algorithm naive Bayes trees functional trees tree-based classifiers decision trees

1. Introduction
2. Tree induction and logistic regression
2.1. Tree induction
2.2. Classification via regression
2.3. Logistic regression
3. Related tree-based learning schemes
3.1. Model trees
3.2. Stepwise model tree induction
3.3. Logistic regression trees with unbiased selection
3.4. Functional trees
3.5. Naive bayes trees
3.6. Boosting trees
4. Logistic model trees
4.1. The model
4.2. Building logistic model trees
4.3. Computational complexity
5. Experiments
5.1. Algorithms included in experiments
5.2. Datasets and methodology
5.3. The impact of variable selection for logistic regression
5.4. Empirical evaluation of LMT
6. Conclusions
6.1. Future work
Notes
References

Knowls

Knowl 1 — Logistic Model Tree Structure and Probabilistic Formulation
model/method
A Logistic Model Tree (LMT) is a decision tree classifier that incorporates linear logistic regression models at its leaf nodes. Let $S \subset \mathbb{R}^d$ denote the instance space spanned by all attributes, and let $J$ denote the number of target classes indexed by $j \in \{1, \dots, J\}$ . The tree structure partitions the instance space $S$ into a finite set of mutually disjoint regions $S_t$ , where each region is associated with a leaf $t \in T$ in the set of leaves $T$ :

$S = \bigcup_{t \in T} S_t, \quad S_t \cap S_{t'} = \emptyset \quad \text{for } t \neq t'$

At each non-terminal decision node, instances are routed using univariate axis-parallel splits: numeric attributes are compared to a real-valued threshold ( $x_k \le \theta$ versus $x_k > \theta$ ), while nominal attributes with $k$ distinct values produce a $k$ -way split.

Each leaf node $t \in T$ contains a local multinomial logistic regression function $f_t(x)$ defined over a subset of attributes $V_t \subseteq V$ , where nominal attributes have been converted to binary indicators. The posterior class probability for class $j$ given an input vector $x$ at leaf $t$ is modeled as:

$\Pr(G = j \mid X = x) = \frac{e^{F_j(x)}}{\sum_{k=1}^J e^{F_k(x)}}$

subject to the symmetric normalization constraint $\sum_{k=1}^J F_k(x) = 0$ . Each class scoring function $F_j(x)$ is linear:

$F_j(x) = \alpha_0^j + \sum_{v \in V_t} \alpha_v^j v$

The full model $f(x)$ across the entire instance space is the sum of leaf models weighted by region indicator functions:

$f(x) = \sum_{t \in T} f_t(x) \cdot I(x \in S_t)$

where $I(x \in S_t) = 1$ if $x \in S_t$ and $0$ otherwise. Predictions are made by choosing the class with maximum posterior probability: $j^* = \arg\max_j \Pr(G = j \mid X = x)$ .
Knowl 2 — LMT Tree Induction Algorithm with Incremental LogitBoost Refinement
algorithm
The LMT induction algorithm builds a decision tree top-down while incrementally fitting logistic regression models at the nodes via LogitBoost. Rather than fitting separate logistic regression models from scratch on the smaller data subsets at child nodes, each child node inherits the linear committee $F_j(x)$ , sample weights $w_{ij}$ , and probability estimates $p_j(x_i)$ from its parent node, refining them by running additional LogitBoost iterations on its local dataset.
Input: Training dataset D, candidate attribute set V, class labels {1, ..., J}
Output: Pruned Logistic Model Tree
function BuildLMT(D):
    root = new Node()
    numIters = CV_SelectIterations(D, initialModels=null)
    root.linearModels = FitLogitBoost(D, initialModels=null, iterations=numIters)
    GrowTree(root, D, root.linearModels, numIters)
    alpha = CrossValidateCARTCostComplexity(root, D)
    CARTPrune(root, alpha)
    return root
function GrowTree(node, D_node, parentModels, baseIters):
    if |D_node| < 15 or StopCriterionMet(D_node):
        node.isLeaf = true
        return
    split = FindBestC45Split(D_node)
    if split is null:
        node.isLeaf = true
        return
    node.splitTest = split
    subsets = split.Partition(D_node)
    for each subset D_child in subsets:
        child = new Node()
        node.AddChild(child)
        if |D_child| >= 5:
            childModels = ResumeLogitBoost(D_child, startModels=parentModels, iterations=baseIters)
        else:
            childModels = Copy(parentModels)
        child.linearModels = childModels
        GrowTree(child, D_child, childModels, baseIters)
Tree growing stops at a node if:
1. The node contains fewer than 15 instances.
2. The best split fails to produce at least two branches containing 2 or more instances, or fails to achieve a minimum information gain threshold.
3. An inherited model cannot be refined if a node has fewer than 5 instances (in which case the parent model is retained unchanged).
Knowl 3 — SimpleLogistic Stagewise Logistic Regression with Automatic Feature Selection
algorithm
SimpleLogistic is a standalone linear logistic regression algorithm that uses the LogitBoost framework with univariate simple linear regressions as weak learners to achieve forward stagewise attribute selection and regularized parameter estimation.
Input: Training dataset D with n instances (x_i, y_i) where y_i in {1, ..., J}, maxIterations M (default 500)
Output: Multinomial additive logistic model F_j(x)
function SimpleLogistic(D, M):
    M_opt = SelectOptimalIterationsCV(D, folds=5, maxIter=M)
    return TrainLogitBoost(D, M_opt)
function TrainLogitBoost(D, iterations):
    Initialize weights w_ij = 1/n and probabilities p_j(x_i) = 1/J for all i=1..n, j=1..J
    Initialize committee F_j(x) = 0 for all j=1..J
    for m = 1 to iterations do:
        for j = 1 to J do:
            Compute indicator y_ij* = (1 if y_i == j else 0)
            Compute working response z_ij = (y_ij* - p_j(x_i)) / (p_j(x_i) * (1 - p_j(x_i)))
            Compute observation weight w_ij = p_j(x_i) * (1 - p_j(x_i))
            For every attribute v in V, fit a univariate weighted least squares line f_{m,j,v}(x) to z_ij with weights w_ij
            Select the attribute v* minimizing weighted squared error: f_mj(x) = f_{m,j,v*}(x)
        for j = 1 to J do:
            f_mj(x) = (J-1)/J * (f_mj(x) - 1/J * sum_{k=1}^J f_mk(x))
            F_j(x) = F_j(x) + f_mj(x)
        Update p_j(x_i) = exp(F_j(x_i)) / sum_{k=1}^J exp(F_k(x_i))
    return F_1(x), ..., F_J(x)
Because each LogitBoost step fits a simple linear regression on only one attribute at a time and stops prior to asymptotic convergence via five-fold cross-validation, uninformative features are never introduced into the linear committee, producing sparse, interpretable logistic models.
Knowl 4 — CART-Based Minimal Cost-Complexity Pruning for Logistic Model Trees
model/method
Logistic model trees use CART minimal cost-complexity pruning to trade off tree size against empirical error. For any subtree $T \subseteq T_{\max}$ , the cost-complexity measure is defined as:

$C_\alpha(T) = R(T) + \alpha |T|$

where $R(T)$ is the training classification error (or deviance $-2 \sum \log \Pr(y_i \mid x_i)$ ) of subtree $T$ , $|T|$ is the number of terminal leaf nodes in $T$ , and $\alpha \ge 0$ is the cost-complexity complexity parameter penalizing tree size.

The parameter $\alpha$ is tuned via 5-fold cross-validation on the training set. Because leaf models in LMT contain flexible logistic regression functions constructed via incremental refinement, splitting a node strictly increases model complexity. Consequently, CART pruning reliably determines whether tree partitioning is necessary or whether the tree should be pruned back completely to a single root node ( $|T| = 1$ ), yielding a standard linear logistic model when the true decision boundary is linear.
Knowl 5 — Computational Complexity and Speed-Up Heuristics of LMT
model/method
The asymptotic time complexity of building an unpruned Logistic Model Tree is:

$O(d \cdot n \log n + n \cdot a^2 \cdot d + k^2)$

where $n$ is the number of training instances, $a$ is the number of attributes, $d$ is the depth of the initial unpruned tree, and $k$ is the number of nodes. Tree induction and sorting contribute $O(d \cdot n \log n)$ , fitting logistic regression models via LogitBoost contributes $O(n \cdot a^2 \cdot d)$ , and CART cost-complexity cross-validation adds $O(k^2)$ scaled by a constant factor of approximately 6 for five-fold CV.

To mitigate practical training time, LMT employs two speed-up heuristics:
1. Root Iteration Re-use: The number of LogitBoost iterations is determined via five-fold cross-validation only once at the root node (capped at 200 iterations). This fixed iteration budget is then reused directly across all descendant nodes during recursive splitting without performing inner cross-validation at every node.
2. Early Stopping during Cross-Validation: When monitoring the test-fold error curve during the initial cross-validation, LogitBoost stops training early if the minimum observed test error has not decreased for 50 consecutive iterations.
Knowl 6 — Preprocessing for Nominal Attributes and Missing Values in LMT
model/method
To handle missing data and nominal attributes across both tree splitting and LogitBoost regression fitting, LMT uses a hybrid preprocessing strategy:
1. Missing Value Imputation: Before tree construction begins, missing attribute values are globally replaced with the training set mean (for numeric attributes) or mode (for nominal attributes). When evaluating unseen test instances, missing values are imputed using these stored training statistics. LMT avoids fractional instance weighting splits.
2. Nominal Attribute Conversion: Tree split selection uses the original nominal attributes directly, creating standard multi-way branches to maximize information gain and maintain tree readability. For the logistic regression models fit at each node, nominal attributes with $k$ discrete categories are locally binarized into $k$ distinct indicator variables ( $1$ if the instance has the $l$ -th value, $0$ otherwise) prior to running LogitBoost.
Knowl 7 — Experimental Evaluation Protocol and Corrected Resampled t-Test
experimental setup
The empirical evaluation of LMT was conducted across 36 benchmark classification datasets from the UCI Machine Learning Repository, spanning varying instance counts (57 to 20,000), attributes (4 to 64), and class counts (2 to 26). Performance was measured using 10 runs of 10-fold stratified cross-validation, generating 100 paired error estimates per algorithm and dataset.

Statistical significance at the $\alpha = 0.05$ level was assessed using the Nadeau and Bengio corrected resampled $t$ -test:

$t = \frac{\frac{1}{N} \sum_{j=1}^N x_j}{\sqrt{\left(\frac{1}{N} + \frac{n_2}{n_1}\right) \hat{\sigma}^2}}$

where $N = 100$ is the total number of cross-validation folds across all 10 runs, $x_j$ is the difference in classification accuracy on fold $j$ , $n_1$ is the number of training instances per fold (90% of data), $n_2$ is the number of testing instances per fold (10% of data), and $\hat{\sigma}^2$ is the sample variance of the differences. The $\frac{n_2}{n_1}$ term explicitly corrects for sample overlap and dependency across resampled folds to prevent inflated Type I error rates.

Knowl 8 — Empirical Impact of Variable Selection: SimpleLogistic vs MultiLogistic

empirical result

Across 36 UCI datasets, standalone forward stagewise logistic regression with attribute selection (SimpleLogistic) was compared against standard full maximum-likelihood logistic regression (MultiLogistic):

SimpleLogistic significantly outperformed MultiLogistic on four datasets (Breast-cancer, Primary-tumor, Splice, and Optdigits) and was never significantly outperformed on any dataset. SimpleLogistic substantially pruned uninformative attributes from the final linear models (for instance, reducing Breast-cancer from 48 binarized features down to just 2 attributes while increasing mean accuracy from $67.77\% \pm 6.92\%$ to $74.94\% \pm 6.25\%$ ).

Knowl 9 — Empirical Accuracy and Tree Compactness of LMT vs Tree and Linear Baselines

empirical result

Across 10 runs of 10-fold cross-validation on 36 UCI datasets, LMT was evaluated against standard tree algorithms (C4.5 and CART) and standalone logistic regression (SimpleLogistic and MultiLogistic):

Metric	vs SimpleLogistic	vs MultiLogistic	vs C4.5	vs CART
Accuracy Win / Loss	23 / 12	28 / 8	29 / 7	30 / 6
Significant Accuracy Wins	8	13	16	17
Significant Accuracy Losses	0	0	0	0
Two-tailed Sign Test $p$ -value	0.0895	0.0012	0.0003	< 0.0001
Tree Size Win / Loss (Smaller)	–	–	36 / 0	36 / 0
Significant Tree Size Wins	–	–	36	32

Key findings include:

LMT never performed significantly worse than C4.5, CART, SimpleLogistic, or MultiLogistic on any of the 36 datasets.
LMT produced trees with significantly fewer leaves than C4.5 on all 36 datasets, and significantly fewer than CART on 32 datasets (e.g., on the Letter dataset, LMT averaged 41.73 leaves versus 1160.92 for C4.5 and 1125.86 for CART).
On exactly 18 of the 36 datasets (50%), LMT pruned completely back to the root node (average leaf count $< 1.5$ ), verifying that cost-complexity pruning automatically reverts to a pure linear logistic model when tree splits are not warranted.

Knowl 10 — Empirical Comparison of LMT Against Other Tree Hybrids and Multi-Tree Ensembles

empirical result

LMT was compared against other hybrid/functional decision trees (NBTree, LTree with linear discriminants, LTree with logistic regression, and Lotus) as well as multi-tree algorithms (M5' model trees for classification and AdaBoost.M1 with 100 C4.5 iterations):

Competitor	Win / Loss	Sig. Wins (LMT)	Sig. Losses (LMT)	Sign Test $p$ -value
NBTree	27 / 9	10	0	0.004
LTree (Linear)	30 / 6	15	1	< 0.0001
LTree (Logistic)	32 / 4	17	1	< 0.0001
Lotus (Simple, 2-class)	13 / 1	6	0	0.0018
Lotus (Multiple, 2-class)	8 / 4	5	0	0.388
M5' (Classification)	31 / 5	11	1	< 0.0001
AdaBoost.M1 (100 Trees)	17 / 19	7	9	0.868

LMT demonstrated statistically significant overall superiority over NBTree, LTree, Lotus, and M5' for classification. When compared to AdaBoost.M1 (100 boosting iterations of C4.5), LMT showed no statistically significant difference in performance across the 36 datasets (17 wins vs 19 losses, $p = 0.868$ ), achieving competitive predictive accuracy with an ensemble method while preserving the interpretability of a single decision tree.

Knowl 11 — Adaptive Bias-Variance Scaling Across Sample Sizes
empirical result
The scaling behavior of LMT across varying sample sizes was analyzed on an artificial domain defined by $f(x) = 2x_1^2 + x_2 + x_3 + x_4$ and binary class $g(x) = \text{sign}(f(x))$ , with attributes sampled uniformly in $[-1, 1]^4$ . Training sets varied from $N = 25$ to $N = 12,800$ instances (100 stratified datasets per size) evaluated on a fixed 10,000-instance test set.
1. Small Sample Regime ( $N \le 100$ ): Simple linear logistic regression (SimpleLogistic) and LMT achieve identical test accuracy (approx. $80\%$ to $84\%$ ) and outperform C4.5 (approx. $73\%$ to $81\%$ ). In this regime, LMT's pruning routinely collapses the model to a single root node (tree size $= 1.0$ ), leveraging the low variance of the linear model.
2. Large Sample Regime ( $N \ge 200$ ): SimpleLogistic saturates at $\approx 85\%$ accuracy because it cannot represent the quadratic term $x_1^2$ . In contrast, LMT begins creating recursive splits around $N = 200$ , matching C4.5 accuracy at roughly half the required training examples and achieving higher final accuracy ( $> 96\%$ ) while generating dramatically smaller trees (e.g., $\approx 80$ leaves for LMT versus $\approx 290$ leaves for C4.5 at $N = 12,800$ ).
Knowl 12 — Computational and Missing Data Limitations of LMT
limitation
The primary limitations of Logistic Model Trees identified by the authors are:
1. High Computational Training Overhead: Due to the nested cross-validation procedures (five-fold CV to select LogitBoost iterations at the root and cost-complexity cross-validation for CART pruning), LMT is several orders of magnitude slower in training execution time than standard decision tree induction algorithms such as C4.5.
2. Heuristic Iteration Re-use: The speed-up heuristic of setting the LogitBoost iteration count once at the root and reusing it for all subtrees is an ad-hoc procedure that lacks formal theoretical optimality guarantees, despite performing well empirically.
3. Simplistic Missing Value Treatment: LMT relies on global single-imputation (mean/mode replacement) prior to tree induction, without supporting fractional split distributions or probabilistic routing of missing values during tree evaluation.

Coverage note — None was omitted; all core contributions, including the LMT model formulation, incremental LogitBoost induction algorithm, SimpleLogistic feature selection, pruning strategy, computational speed-up heuristics, preprocessing routines, and empirical benchmarking across 36 UCI datasets and synthetic problems, are fully represented.

References

1.Blake C. and Merz, C. J. (1998). UCI repository of machine learning databases. [www.ics.uci.edu/ ∼mlearn/MLRepository.html].
2.Breiman, L., Friedman, H., Olshen, J. A., & Stone, C. J. (1984). Classification and Regression Trees. Wadsworth.
3.Chan, K. Y., & Loh, W. Y. (2004). LOTUS: An algorithm for building accurate and comprehensible logistic regression trees. Journal of Computational and Graphical Statistics, 13:4, 826–852.
4.Frank, E., Wang, Y., Inglis, S., Holmes, G., & Witten, I. H. (1998). Using model trees for classification. Machine Learning, 32:1, 63–76.
5.Freund, Y., & Schapire, R. E. (1996). Experiments with a new boosting algorithm. In Proc 13th International Conference on Machine Learning (pp. 148–156). Morgan Kaufmann.
6.Friedman, J., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: A statistical view of boosting. The Annals of Statistics, 38:2, 337–374.
7.Gama, J. (2004). Functional trees. Machine Learning, 55:3, 219–250.
8.Hastie, T., Tibshirani, R., & Friedman, J. (2001). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer-Verlag.
9.Ihaka, R. & Gentleman, R. (1996). R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics, 5:3, 299–314.
10.Kohavi, R. (1996). Scaling up the accuracy of naive bayes classifiers: A decision-tree hybrid. In Proc 2nd International Conference on Knowledge Discovery and Data Mining (pp. 202–207). Menlo Park, CA: AAAI Press.
11.Landwehr, N., Hall, M., & Frank, E. (2003). Logistic model trees. In Proc 14th European Conference on Machine Learning (pp. 241–252). Springer-Verlag.
12.Lim, T.-S., Loh, W. Y., & Shih, Y. S. (2000). A comparison of prediction accuracy, complexity, and training time for thirty-three old and new classification algorithms. Machine Learning, 40:3, 641–666.
13.Lubinsky, D. (1994). Tree structured interpretable regression. In D. Fisher, and H. J. Lenz (Eds.), Learning from Data (pp. 387–398). Springer-Verlag.
14.Malerba, D., Appice, A., Ceci, M., & Monopoli, M. (2002). Trading-off local versus global effects of regression nodes in model trees. In M.-S. Hacid, Z. W. Ras, D. A. Zighed, & Y. Kodratoff (Eds.), Proc 13th International Symposium Foundations of Intelligent Systems (pp. 393–402). Springer-Verlag.
15.Nadeau, C., & Bengio, Y. (2003). Inference for the generalization error. Machine Learning, 52:3, 239–281.
16.Perlich, C., Provost, F., & Simonoff, J. (2003). Tree inductions vs. logistic regression: A learning-curve analysis. Journal of Machine Learning Research, 4, 211–255.
17.Quinlan, J. R. (1992). Learning with continuous classes. In Proc 5th Australian Joint Conference on Artificial Intelligence (pp. 343–348). World Scientific Publishing Company Incorporated.
18.Quinlan,R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann.
19.Wang, Y. & Witten, I. (1997). Inducing model trees for continuous classes. In Proc of Poster Papers, European Conf. on Machine Learning. Prague, Czech Republic.
20.Witten, I. H., & Frank, E. (2000). Data Mining: Practical Machine Learning Tools and Techniques with Java Implemenations. San Francisco: Morgan Kaufmann.

Citation

MLA

Landwehr, N., et al. “Logistic Model Trees”. Machine Learning, vol. 59, nos. 1-2, 2005, pp. 161–205, https://doi.org/10.1007/s10994-005-0466-3.

APA

Landwehr, N., Hall, M., & Frank, E. (2005). Logistic Model Trees. Machine Learning, 59(1-2), 161–205. https://doi.org/10.1007/s10994-005-0466-3

Chicago

Landwehr, N., M. Hall, and E. Frank. 2005. “Logistic Model Trees”. Machine Learning 59 (1-2): 161–205. https://doi.org/10.1007/s10994-005-0466-3.

Harvard

Landwehr, N., Hall, M. and Frank, E. (2005) “Logistic Model Trees”, Machine Learning, 59(1-2), pp. 161–205. Available at: https://doi.org/10.1007/s10994-005-0466-3.

Vancouver

1. Landwehr N, Hall M, Frank E (2005) Logistic Model Trees. Machine Learning 59:161–205

BibTeX

@article{Landwehr_2005, title={Logistic Model Trees}, volume={59}, ISSN={1573-0565}, url={http://dx.doi.org/10.1007/s10994-005-0466-3}, DOI={10.1007/s10994-005-0466-3}, number={1-2}, journal={Machine Learning}, publisher={Springer Science and Business Media LLC}, author={Landwehr, Niels and Hall, Mark and Frank, Eibe}, year={2005}, month=May, pages={161–205} }

Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Comparison	SimpleLogistic Wins	MultiLogistic Wins	Sign Test $p$ -value
Overall Win/Loss	24	12	0.0652
Statistically Significant	4	0	–

Logistic Model Trees

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Logistic Model Tree Structure and Probabilistic Formulation

Knowl 2 — LMT Tree Induction Algorithm with Incremental LogitBoost Refinement

Knowl 3 — SimpleLogistic Stagewise Logistic Regression with Automatic Feature Selection

Knowl 4 — CART-Based Minimal Cost-Complexity Pruning for Logistic Model Trees

Knowl 5 — Computational Complexity and Speed-Up Heuristics of LMT

Knowl 6 — Preprocessing for Nominal Attributes and Missing Values in LMT

Knowl 7 — Experimental Evaluation Protocol and Corrected Resampled t-Test

Knowl 8 — Empirical Impact of Variable Selection: SimpleLogistic vs MultiLogistic

Knowl 9 — Empirical Accuracy and Tree Compactness of LMT vs Tree and Linear Baselines

Knowl 10 — Empirical Comparison of LMT Against Other Tree Hybrids and Multi-Tree Ensembles

Knowl 11 — Adaptive Bias-Variance Scaling Across Sample Sizes

Knowl 12 — Computational and Missing Data Limitations of LMT

References

Citation

BibTeX

Source Code

Access the Paper